The Problem

Streaming 3D reconstruction from continuous video data requires maintaining geometric consistency across thousands of frames while avoiding cumulative drift. Existing approaches either process frames in isolation (losing global context) or rely on iterative optimization (computationally expensive and slow for real-time streams). This repository addresses that tension with a feed-forward architecture designed for long-range coherence.

What This Does

Lingbot-Map is a feed-forward 3D foundation model for streaming reconstruction. The core architecture lives in lingbot_map/ — a Geometric Context Transformer that unifies coordinate grounding, dense geometric cues, and long-range drift correction through anchor context, a pose-reference window, and trajectory memory. Inference runs at ~20 FPS on 518×378 resolution over sequences exceeding 10,000 frames using a paged KV cache attention mechanism.

The codebase is organized into five projects: benchmark (90 files, evaluation pipelines and datasets), lingbot_map (36 files, model layers and streaming logic), demo_render (40 files, interactive and batch rendering), preprocess (4 files), and scripts (1 file). The example/ directory contains 1167 image frames used for demo scenes.

Entry points include main in benchmark/run.py:119 (reaches 399 functions), run_scene in benchmark/run.py:31 (reaches 250 functions), and ws_handler in demo_render/interactive_viewer/server.py:592 (reaches 26 functions, WebSocket frame handling). Execution traces outside the process include main -> setup_logging (filesystem), run -> encode_video (subprocess), and ws_handler -> render_screenshot -> _import_open3d (filesystem).

How It Is Wired

Execution starts at main in benchmark/run.py:119, which parses arguments and reaches 399 downstream functions. The call graph shows exists called from 59 places, forward from 7, and _to_out from 33 calls within inference streaming. The most connected modules are demo_render/rgbd_render/__init__ (Ca 0, Ce 9, instability 1) and lingbot_map/layers/block (Ca 5, Ce 4, instability 0.44).

Paths to external effects are shortest over resolved edges: main -> setup_logging creates log directories; run -> encode_video spawns subprocess.run; ws_handler -> render_screenshot triggers _import_open3d. The internal call graph has 1068 resolved call edges between repository functions, with no circular dependencies detected across 132 analyzed files.

Files with the widest blast radius: lingbot_map/models/gct_stream_window.py (1001 lines, oversized, instability 0.67), benchmark/viewer.py (74 functions, 5 classes), and demo_render/interactive_viewer/server.py (20 functions, 2 classes, WebSocket handling). benchmark/benchmark/core/storage.py is the most frequently called file for file I/O (21 callers, reads/writes _json_default, exists, has_frame_key).

How To Use It

Setup: The project uses pip as package manager. Dependencies are declared in demo_render/requirements.txt and pyproject.toml, but no lockfile is committed — builds are non-reproducible without one. No Dockerfile is present. To install, one would run pip install -r demo_render/requirements.txt or pip install . from the root, though the lockfile gap means exact transitive versions are not guaranteed.

Configuration: Config files live in benchmark/configs/ (datasets: droid_w, kitti, oxford, etc.) and demo_render/config/ (default, indoor, outdoor_drive). Method configs are in benchmark/configs/methods/lingbot_map.yaml. Environment variables or keys are not documented beyond what the config files reference.

Running it: The interactive demo is invoked with python demo.py (as documented in README). For compiled execution, the README notes python demo.py --compile or python gct_profile.py --backend flashinfer --dtype bf16 --compile. The offline rendering pipeline runs via python demo_render/batch_demo.py. No lockfile is present, so pip install may resolve differently across runs.

Real-World Use

A streaming perception system ingests video frames, passes them through the GCT stream window with paged KV caching, and outputs a fused mesh or pointcloud per frame. The lingbot_map/aggregator/stream.py module manages the KV cache lifecycle (_init_kv_cache, _build_blocks), while lingbot_map/layers/block.py handles per-block geometric transformations. Pose drift is corrected via the trajectory memory mechanism referenced in the architecture description. The system is configured per-dataset through YAML files in benchmark/configs/datasets/ and method parameters in benchmark/configs/methods/lingbot_map.yaml.

Code Health & Issues

The static analysis found 85 issues across 5 kinds:

  • [HIGH/cognitive_load] Deep nesting x43 — demo_render/rgbd_render/scene.py, lingbot_map/layers/block.py, demo_render/rgbd_render/overlay.py have max indentation depth 8, making control flow hard to follow. Fix: flatten with early returns/guard clauses.
  • [MEDIUM/resilience] Broad exception handling x5 — lingbot_map/utils/geometry.py, demo_render/rgbd_render/pipeline/parallel.py, benchmark/benchmark/core/loader.py have bare except clauses that swallow errors indiscriminately. Fix: catch specific exceptions; re-raise or log the rest.
  • [MEDIUM/resource_safety] File opened without context manager x7 — demo_render/rgbd_render/renderer.py, lingbot_map/utils/load_fn.py, demo.py use open(...) without with, risking handle leaks. Fix: wrap in with open(...) as f:.
  • [HIGH/clarity] Duplicated code blocks — 1419 repeated 6-line blocks across 47 files, concentrated in benchmark/benchmark/evaluation/auc.py, trajectory.py, depth.py, points.py. Fix: extract shared helpers; DRY the repeated logic.
  • [HIGH/cognitive_load] Oversized files x4 — lingbot_map/models/gct_stream_window.py, benchmark/viewer.py, demo_render/batch_demo.py at 1001 lines each, hard to hold in one head; changes ripple widely. Fix: split into cohesive units by responsibility.

The measured Code Health Audit adds:

  • [HIGH] No test suite — 128 source files, zero test files. Any change ships with no signal that existing behavior holds.
  • [HIGH] No lockfile — pyproject.toml exists with no committed lockfile; transitive dependencies can drift between installs.
  • [HIGH] No CI/CD pipeline — 128 source files with no automated build/test gate.
  • [MEDIUM] No Dependabot/Renovate configured — advisories remain unpatched without bot automation.
  • [MEDIUM] Large binary lingbot-map_paper.pdf (17.2MB) in the repository — every clone and CI checkout pays for undiffed data.
  • [LOW] Missing convention files — .editorconfig, .gitattributes, formatter config absent.

The Bottom Line

This is a technically specific repository with a clear architecture for streaming 3D reconstruction. The feed-forward GCT design with paged KV cache is the notable engineering contribution, enabling long sequences at stable FPS. However, the absence of tests, CI, and a lockfile means the codebase is high-risk for production use without significant operational overhead. The oversized files and duplicated evaluation logic further reduce maintainability. It is suitable for researchers or teams willing to invest in the missing infrastructure (tests, lockfile, CI) and address the code health findings. Others should expect to build those gates around the repository before depending on it in production.