The Problem

Working with video generation that actually respects camera motion and object trajectories sucks. Most video diffusion models spit out pretty clips, but as soon as you want to control the camera path or object movement—forget it. You end up with janky results, or you hack together some control system that barely works. If you want real, direct geometric control over both camera and multiple objects, you’re out of luck with almost everything out there.

What This Does

VerseCrafter gives you explicit 4D geometric control over video generation—camera motion, object motion, and their interaction. The key magic lives in versecrafter/pipeline/pipelinewanversecrafter.py where the pipeline wires up a frozen Wan2.1 video diffusion backbone and then injects geometry via a GeoAdapter. Everything related to 4D control—camera paths, 3D Gaussian object motion—is handled through JSON and NPZ files (see demodata/ and config/).

You get Blender integration (blenderaddon/) for hands-on editing and trajectory export, plus a pile of scripts in inference/ for fitting Gaussians (fit3Dgaussian.py), running segmentation (groundedsam2infer.py), and full pipeline inference (versecrafterinference.py). The project isn’t hiding the data: check demodata/ for real examples, from camera paths to rendered videos and masks.

Real-World Use

Say you want to generate a street scene where the camera orbits a car and a person walks across the frame. You’d prep your control files (camera path as an NPZ, object trajectories as JSONs). Then you’d run something like:

python inference/versecrafterinference.py --config config/wan2.1/wancivitai.yaml \ --camera-trajectory demodata/.../customcameratrajectory.npz \ --object-trajectories demodata/.../custom3Dgaussiantrajectory.json

Edit anything in Blender (using the blenderaddon), re-export, and re-run. All your outputs—depths, masks, videos—land in demo_data/ for inspection. No black box nonsense.

The Bottom Line

If you actually need precise camera and object motion control in video generation, VerseCrafter is one of the few open options that doesn’t lie to you about what’s possible. It’s fiddly to set up, eats VRAM for breakfast, and isn’t for quick experiments—but if you want explicit geometry in your video synthesis, this is the toolkit. If you just want pretty TikTok clips, look elsewhere.