FPGA Acceleration and Memory-Aware Architecture Design of Modified OverFeat CNNs for Low-Power, Sub-Millisecond Perception Pipelines in Autonomous Driving

• Abbas Ahsun

  Texas University

  Author

The deployment of deep convolutional neural networks (CNNs) in autonomous driving perception systems faces critical challenges in meeting stringent latency, power, and reliability requirements for real-time operation. While GPU-based implementations have demonstrated high detection accuracy, they often exceed the power budgets of embedded automotive platforms and introduce unpredictable latency variations. This research addresses the gap between algorithmic performance and hardware efficiency by presenting a memory-aware FPGA acceleration framework for a modified OverFeat CNN architecture targeting sub-millisecond perception pipelines. The proposed system integrates three key innovations: (1) a modified OverFeat architecture optimized for vehicle and lane detection with occlusion handling mechanisms, (2) a memory-aware scheduling scheme that dynamically allocates external memory bandwidth among CNN processing engines to eliminate contention-induced latency, and (3) a hardware accelerator design leveraging register-transfer level power optimization techniques. Experimental evaluation demonstrates that the proposed FPGA implementation achieves 89.4% mean Average Precision on the comprehensive autonomous driving dataset while operating at 0.87 ms inference latency—a 42% reduction compared to conventional GPU baselines. The system consumes 3.8 W total on-chip power, representing a 67% power savings relative to mobile GPU implementations. The proposed memory-aware architecture provides a replicable framework for deploying complex CNN perception pipelines in power-constrained autonomous vehicles, with practical implications for automotive system designers pursuing Level 4 and Level 5 autonomy.

• pdf

Copyright (c) 2026 Abbas Ahsun (Author)

This work is licensed under a Creative Commons Attribution 4.0 International License.