The Problem

Robotic perception pipelines need to fuse segmentation, tracking, and language grounding in real time to build updatable 4‑D scene graphs. Existing systems either run offline or require heavyweight orchestration, making on‑board deployment difficult.

What This Does

DAAAM implements a fully‑Python, foundation‑model‑first stack that streams video frames through SAM segmentation, BotSort tracking, and VLM grounding, then feeds the results into the MIT‑SPARK Hydra framework to maintain a dynamic scene graph. Core logic lives under src/daaam/:

  • Scene understanding – src/daaam/scene_understanding/* provides tools (get_objects_in_region.py, registry.py) and services (services.py) that turn raw sensor data into semantic descriptors.
  • Grounding – src/daaam/grounding/* maps language prompts to object IDs (dam_grounding.py worker).
  • Pipeline orchestration – src/daaam/pipeline/orchestrator.py coordinates queues, scene‑graph updates, and re‑prompting.

Configuration files in config/ (e.g., pipeline_config.yaml, sam2/sam2.1_hiera_large.yaml) let a user swap models without code changes.

How It Is Wired

Entry points

  • scripts/demo_query.py → main (line 22) → creates a frame callback and starts the pipeline.
  • src/daaam/grounding/workers/dam_grounding.py → run (line 572) – a stand‑alone worker invoked by external orchestration.

Control flow (demo_query)

  1. main parses arguments (parse_args) and calls create_frame_callback.
  2. The callback invokes hydra.integration.initialize_pipeline, which constructs a Hydra pipeline object.
  3. Hydra creates a pipeline.orchestrator.Orchestrator instance; its __init__ sets up queues and calls _initialize_state.
  4. Each incoming frame triggers hydra.integration.process_frame → pipeline.orchestrator._handle_re_prompting → scene_understanding.services.process_frame.
  5. process_frame extracts segmentation (utils.segmentation), tracking (tracking.services), and grounding (grounding.services).
  6. Results are stored via scene_graph.services.set_scene_graph, which writes the graph to disk (to_yaml in config.py).

High‑impact symbols

  • performance_measure – called from 9 places, used to log latency throughout the pipeline.
  • to – invoked 9 times for data‑type conversion across modules.
  • shutdown – reachable from 4 callers, central to graceful termination.

Blast radius src/daaam/pipeline/orchestrator.py (40 functions, 2 classes) is the busiest hub; any change here ripples to at least 14 other modules and touches the filesystem. src/daaam/utils/logging.py (39 functions, 9 classes) is the second hub, affecting all modules that emit logs.

Unmapped areas The static analysis did not resolve callbacks from the external Hydra runtime, so the exact wiring of asynchronous worker threads is not captured.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/DAAAM
cd DAAAM

# Install dependencies (pip)
python -m pip install -r requirements.txt

# Verify the environment (Python ≥3.9 required)
python -c "import torch; print(torch.__version__)"

# Run the demo query script (requires a configured pipeline_config.yaml)
python scripts/demo_query.py --config config/pipeline_config.yaml

Configuration files (config/pipeline_config.yaml, config/sam2/*.yaml) must point to model checkpoints and data directories. No Dockerfile or compiled binaries are provided; the repo expects a native Python environment with the listed packages.

Real‑World Use

A mobile robot can start the pipeline on boot, stream its camera feed into scripts/run_pipeline.py, and issue natural‑language queries via the demo_query.py CLI. The robot’s navigation stack reads the generated 3‑D scene graph (scene_graph.services.get_scene_graph) to plan collision‑free paths that respect semantic constraints (e.g., “stay away from the kitchen table”).

Code Health & Issues

  • HIGH – No test suite – 83 source files, no test files.
  • HIGH – No lockfile – pyproject.toml present, but no poetry.lock/requirements.lock.
  • HIGH – No CI workflow – repository lacks .github/workflows/*.
  • MEDIUM – Dependabot not enabled – no dependabot.yml.

Additional observations from the static audit: deep nesting (max depth 8) in src/daaam/assignment/workers/min_frames_max_size.py; broad except: clauses in several services; duplicated 6‑line blocks across visualizer scripts; oversized files (orchestrator.py, scene_graph/services.py, static_visualizer.py).

The Bottom Line

DAAAM delivers a complete, model‑centric pipeline for real‑time scene‑graph construction and language grounding, with clear modular boundaries in src/. However, the codebase lacks automated testing, reproducible dependency locking, and CI, and contains several maintainability hotspots (deep nesting, duplicated logic). It is suitable for teams that can invest in adding a test suite and refactoring high‑complexity modules before deploying in production.