The Problem

Vision Pilot provides an open-source L2 ADAS stack, but the codebase carries structural debt that increases integration risk and long-term maintenance cost. Three measurable patterns drive this: deep nesting in control flow (25 files affected), 269 repeated 6-line blocks across 26 files violating DRY principles, and three modules with branching densities of 56+ decision points over ~150 lines. These patterns make safety-critical changes harder to reason about and increase the surface area for undetected regressions.

What This Does

Vision Pilot is a Hybrid End-to-End AI architecture for L2 ADAS features (ACC, FCW, AEB, LKAS, LDW, ISA, Autopilot) that processes data in parallel by perception AI models for safety and End-to-End AI models for performance. It operates with a single front-facing monocular camera (52–55° horizontal FOV, 1–2MP resolution). The repository is partitioned into three projects: VisionPilot (120 files, the ADAS stack), Simulation/CARLA/ROS2 (47 files, CARLA bridge), and Calibration (4 files). Key modules include VisionPilot/modules/models/ (auto_drive, auto_speed, auto_steer ONNX engines), VisionPilot/modules/sensing/camera_interface/ (v4l2 and file interfaces), VisionPilot/modules/safety_guardian/ (lateral/longitudinal fusion and planning), and VisionPilot/modules/visualization/ (webrtc stream, occupancy bridges). AI model weights (autodrive_fp32.onnx, autosteer_fp32.onnx, autospeed_fp32.onnx) sit under VisionPilot/modules/models/weights/. The stack connects to CARLA simulation via ROS2 bridges in Simulation/CARLA/ROS2/src/carla_bridge_bringup/, carla_control_publisher/, and carla_vehicle_speed_publisher/.

How It Is Wired

Execution starts at the ROS2 node entry points: Simulation/CARLA/ROS2/src/carla_bridge_bringup/setup.py, carla_control_publisher/setup.py, and carla_vehicle_speed_publisher/setup.py. The CARLA bridge (carla_bridge_bringup/launch/carla_bridge.launch.py) injects sensor data into the ROS2 graph. From there, v4l2_camera_interface.cpp (VisionPilot/modules/sensing/camera_interface/src/v4l2_camera_interface.cpp) or file-based interfaces feed preprocessed frames into the ONNX engine pipeline. Each of the three AI models (auto_drive, auto_speed, auto_steer) consumes the same front-camera homography (Calibration/calc_front_camera_homography.py) to produce lateral and longitudinal commands. Fusion occurs in VisionPilot/modules/safety_guardian/fusion/ (lateral_fusion.cpp, longitudinal_fusion.cpp), whose outputs feed planning (VisionPilot/modules/safety_guardian/planning/). Visualization runs through VisionPilot/modules/visualization/src/webrtc_stream.cpp (784 lines, high cognitive load). The module graph contains 16 internal modules with zero import edges and zero circular dependencies—analyzed across all 90 code files—but the absence of edges does not imply absence of runtime data flow; inter-process communication occurs over ROS2 topics and Zenoh ( Simulation/CARLA/Zenoh/). Files with the widest blast radius are the config module (VisionPilot/modules/config/src/vision_pilot_config.cpp, 56 branch points over 150 lines) and the camera interface header (VisionPilot/modules/sensing/camera_interface/include/camera_interface/v4l2_camera_interface.hpp), both of which many other modules depend on for type and parameter definitions.

How To Use It

Setup:

git clone https://github.com/moses-y/vision_pilot
cd vision_pilot
# CPU build
./VisionPilot/docker/build.sh --cpu
# Or CUDA build (requires nvidia/cuda base)
./VisionPilot/docker/build.sh

Configuration: Edit VisionPilot/config/vision_pilot_ros2.conf for ROS2 node parameters. Camera device path and homography calibration matrix are set in Calibration/calc_front_camera_homography.py. Model weight paths are referenced relative to VisionPilot/modules/models/weights/.

Running it:

# Launch the ROS2 CARLA bridge
ros2 launch carla_bridge_bringup carla_bridge.launch.py

# Start the control pipeline
ros2 run carla_control_publisher carla_control_publisher_node

# Or run the full stack via Docker entrypoint
./VisionPilot/docker/run.sh

The CLI entry point is VisionPilot/app/vision_pilot.cpp; however, in practice the ROS2 launch workflow above is the intended integration path.

Real-World Use

A vehicle equipped with a single forward-facing monocular camera streams frames into the ROS2 graph. The v4l2_camera_interface node publishes sensor_msgs/msg/Image on /camera/image_raw. The auto_steer ONNX engine consumes the frame and publishes planned waypoints on /auto_steer/waypoints. The auto_speed engine publishes closest-in-path object data on /auto_speed/detection. The safety guardian fusion nodes subscribe to both topics and publish merged commands on /vehicle_control. If the fused longitudinal command indicates a collision risk, the AEB pathway triggers autonomous emergency braking via the vehicle_ros2_interface publisher. All loops operate at 10–20 Hz depending on model inference latency.

Code Health & Issues

  • Deep nesting x25 – Simulation/CARLA/Zenoh/models/carla_run_model.cpp, VisionPilot/app/vision_pilot.cpp, VisionPilot/modules/debug/src/debug_draw.cpp. Max indentation depth 11; control flow hard to follow. Fix: flatten with guard clauses.
  • Duplicated code blocks – 269 repeated 6-line blocks across 26 files, including Simulation/CARLA/ROS2/src/carla_vehicle_speed_publisher/test/test_copyright.py, test/test_pep257.py, test/test_flake8.py, and Simulation/CARLA/Zenoh/carla_video_pubsub/carla_depth_subscriber.cpp. Fix: extract shared helpers.
  • Oversized file – VisionPilot/modules/visualization/src/webrtc_stream.cpp at 784 code lines. Fix: split by responsibility.
  • High branching density x3 – VisionPilot/modules/config/src/vision_pilot_config.cpp, VisionPilot/modules/middleware_interfaces/ros2_interface/camera_ros2_interface/include/camera_ros2_interface/camera_ros2_interface.hpp, VisionPilot/modules/sensing/camera_interface/include/camera_interface/v4l2_camera_interface.hpp. 56 branch points over 150 lines. Fix: decompose decision logic.
  • GitHub Actions not pinned to commit SHAs – .github/workflows/semantic-pull-request.yaml pulls autowarefoundation/autoware-github-actions at @v1; spell-check-daily.yaml at @v1. Mutable tags risk running different action versions across CI runs.
  • CI does not invoke the test suite – 15 test files exist but no test command appears in any workflow.
  • GITHUB_TOKEN permissions undeclared – 3 workflows have no permissions block, inheriting repository defaults.
  • Base image not pinned by digest – VisionPilot/docker/Dockerfile uses nvcr.io/nvidia/cuda:${CUDA_TAG} and nvcr.io/nvidia/cuda:${CUDA_TAG%-devel*}-runtime-ubuntu24.04 without a sha256 digest.
  • Large model weights not on LFS – autodrive_fp32.onnx (41.0MB), autosteer_fp32.onnx (31.7MB), autospeed_fp32.onnx (36.3MB) and 7 other blobs over 5MB tracked in git.
  • No non-root USER in Docker image – Dockerfile ends with CMD/ENTRYPOINT as root.

The Bottom Line

This repo delivers a functional, safety-critical L2 ADAS stack with real hardware integration points and a clear Hybrid End-to-End AI architecture. The code health findings are quantifiable and addressable: deep nesting, duplicated logic, and outsized files increase integration friction, while the Docker and CI hygiene issues (mutable base images, unpinned Actions, un-declared token permissions, un-LFSed weights) pose concrete operational risks for series production adoption. Teams that can absorb the refactoring effort and establish LFS + digest pinning will get the most value; others should treat this as a research prototype requiring significant polish before production integration.