Low Cost Mocap

The Problem

Indoor autonomous drones need continuous, low‑latency pose estimates without expensive motion‑capture rigs. Existing commercial systems are cost‑for‑each‑camera, hard to deploy, and lack an open‑source stack that can be adapted to custom hardware.

What This Does

Low-Cost-Mocap supplies a lightweight, external‑camera framework that captures multiple drones in a room‑scale volume.

• Computer side (computer_code/) runs a Vite + React UI that visualises the tracked agents and sends control set‑points via WebSocket.
• Python API (computer_code/api/) receives camera frames, runs a structure‑from‑motion pipeline, and publishes 3‑D poses.
• Hardware side (receiver_esp32/, sender_esp32/) implements the wireless link between each drone and the computer using ESP‑32 microcontrollers.

Key modules:

• KalmanFilter.py – state‑space integration of pose estimates.
• LowPassFilter.py – smoothing of noisy image‑based depth.
• Singleton.py – ensures a single global instance of the pose manager.
• helpers.py – utility functions for camera calibration and data conversion.

How It Is Wired

Execution starts at the Python entry point computer_code/api/index.py.

1. index.py creates a FastAPI (implicit) or plain HTTP server and registers a /socket endpoint.
2. Incoming WebSocket frames are routed to helpers.process_frame().
3. process_frame() parses the image, runs OpenCV SFM, and feeds the raw pose into KalmanFilter.update().
4. Updated state propagates to the WebSocket client via Singleton.get_instance().broadcast().
5. On the frontend, computer_code/src/shared/styles/scripts/socket.ts opens a WebSocket to the backend, receives pose data, and renders it in CameraWireframe.tsx and TrajectoryPlanningSetpoints.tsx.

The hub is Singleton, which owns the shared pose buffer; changing it would require touching the entire client‑server contract. The cycle is minimal: the backend never calls client code directly, only via the socket, keeping the modules loosely coupled.

Outside‑world interactions:

• Network – WebSocket over TCP (ws://localhost:8000/socket).
• File system – reads calibration JSON (camera_params.json) on startup.
• Hardware – ESP‑32 modules communicate over Wi‑Fi; the code for them lives in sender_esp32/ and receiver_esp32/.

How To Use It

Setup

git clone https://github.com/moses-y/Low-Cost-Mocap
cd Low-Cost-Mocap/computer_code
yarn install           # install JS deps
python3 api/index.py    # start Python server
yarn run dev           # launch React UI

Configuration

• Place camera calibration data in computer_code/api/camera_params.json.
• Ensure OpenCV‑SFM is built (see README for hints).
• No environment variables are required; the code hard‑codes localhost:8000.

Running

• Open the UI at the URL printed by yarn run dev.
• Power the ESP‑32 sender/receiver pair; the drones will begin streaming to the server.

Real‑World Use

A typical deployment handles 4 drones in a 5 m × 5 m room. The UI shows real‑time trajectories; the backend publishes pose updates every 50 ms. A downstream controller can subscribe to the WebSocket and issue velocity commands, enabling closed‑loop flight without an external motion‑capture system.

Code Health & Issues

No static analysis was run; the repository contains no linting rules beyond .eslintrc.cjs. Dependencies are declared in computer_code/package.json and computer_code/yarn.lock.

The Bottom Line

The stack is functional and demonstrates a viable low‑cost alternative to commercial mocap rigs. It is best suited for teams willing to tweak the Python pipeline and calibrate their own cameras. The lack of CI, limited tests, and sparse documentation constrain rapid adoption, but the clear module boundaries make incremental improvement straightforward.