The Problem

NAS3R addresses multi-view 3D reconstruction without ground-truth annotations or pre-trained priors. Most feed-forward reconstruction models require either depth supervision or large pretrained backbones. NAS3R instead learns explicit 3D geometry and camera parameters jointly through novel view synthesis as the self-supervisory signal, which is the key differentiator for training on unlabeled video or multi-view collections.

What This Does

NAS3R is a self-supervised feed-forward framework for joint 3D geometry and camera estimation. The architecture splits into an encoder (src/model/encoder/) that produces per-view features and explicit 3D representations, and a decoder (src/model/decoder/) that renders novel views via differentiable CUDA splatting (src/model/decoder/cuda_splatting.py). The model wrapper (src/model/model_wrapper.py) orchestrates both during training and evaluation.

The repo includes config-driven experiment management (config/experiment/nas3r/), evaluation pipelines for pose accuracy (src/evaluation/pose_evaluator.py), and visualization utilities (src/visualization/). It supports multiple backbone options including CroCo, DINO, ResNet, and VGGT variants, with pretrained checkpoints hosted on Hugging Face.

How It Is Wired

Execution starts at train in src/main.py:56, which reaches 35 functions and is the sole entry point. From there, the flow goes through the ModelWrapper (2 importers, 23 dependencies, instability 0.92), which handles training steps, view dropout, and loss computation. The src/dataset/types.py module is the hub: 24 modules import it, making it the highest-blast-radius file in the repo.

The internal call graph shows cat is called from 57 places, making it the most load-bearing utility. add_label (11 callers) and add_border (10 callers) in the visualization layer are also heavily used. The evaluation path runs test_step -> add_label 10 times and on_test_end -> pose_auc 6 times.

The traced filesystem effect is short: train -> save_index writes to self.cfg.output_path.mkdir, reaching the filesystem in one hop. No database or network effects were detected. The repo has one import cycle in src/model/encoder/backbone/croco/curope/ (both __init__.py and curope2d.py participate), which complicates changes to that module.

How To Use It

git clone --recurse-submodules https://github.com/moses-y/NAS3R
cd NAS3R
conda create -n nas3r python=3.11 -y
conda activate nas3r
pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt --no-build-isolation
pip install -e submodules/diff-gaussian-rasterization --no-build-isolation

Training runs via python -m src.main +experiment=nas3r/random/re10k. Config files live in config/, with dataset prep documented in DATASETS.md. Pretrained checkpoints go in a checkpoints directory. Note: the clone URL differs from the README's original; the submodule diff-gaussian-rasterization is referenced in setup commands but not visible in the file structure.

Real-World Use

NAS3R fits where you have unlabeled multi-view video or image collections and need camera poses plus 3D structure without any annotation cost. For example, a robotics pipeline collecting raw RGB streams could train NAS3R on those streams to bootstrap a 3D scene representation, then fine-tune on a small labeled subset for a downstream task like novel view synthesis or navigation planning.

Code Health & Issues

Static analysis found 38 issues (17 high, 21 medium), 8 distinct kinds:

  • High - Deep nesting - 24 instances, worst in src/misc/image_io.py, src/misc/cam_utils.py, src/model/model_wrapper.py (max indentation depth 12)
  • High - Duplicated code - 604 repeated 6-line blocks across 45 files, concentrated in src/dataset/view_sampler/
  • High - Import cycle - src/model/encoder/backbone/croco/curope/__init__.py and curope2d.py
  • High - Oversized files - src/misc/sht.py and src/model/model_wrapper.py at 1214 lines each
  • Medium - Broad exception handling - 4 instances in src/misc/utils.py, src/model/decoder/cuda_splatting.py, src/model/model_wrapper.py
  • Medium - Unmanaged file handles - 3 instances using bare open() in src/misc/image_io.py, src/dataset/dataset_re10k.py, src/model/encoder/backbone/vggt/utils/load_fn.py
  • Medium - Hub module - src/dataset/types.py with 24 dependents
  • Medium - High branching density - src/model/encoder/backbone/vggt/utils/load_fn.py (30 branches / 92 lines)

SDLC gaps: no test suite, no CI workflow, no dependency lockfile, no Dockerfile. A license is present.

The Bottom Line

NAS3R is a serious research codebase with a credible self-supervision approach and multiple backbone options. The code quality issues — deep nesting, duplication, missing tests — are typical for a CVPR submission and manageable. Use it if you need self-supervised multi-view reconstruction; expect to invest in refactoring before treating it as production infrastructure.