The Problem
Generating animate-ready 3D assets from a single image requires simultaneously solving shape geometry, skeletal structure, and skinning weight prediction while maintaining coherence across all three domains. Traditional approaches treat these as separate pipeline stages, leading to rig invalidity and poor animation quality when transferring motion data.
What This Does
AniGen represents shape, skeleton, and skin as mutually consistent S³ Fields (Shape, Skeleton, Skin) defined over a shared spatial domain (anigen/representations/). The framework employs two technical innovations: a confidence-decaying skeleton field that handles geometric ambiguity at Voronoi boundaries (anigen/datasets/anigensparsestructure.py), and a dual skin feature field that decouples skinning weights from specific joint counts (anigen/models/structuredlatentvae/anigenencoder.py). Built upon a two-stage flow-matching pipeline, it first synthesizes a sparse structural scaffold via anigen/pipelines/anigenimageto3d.py, then generates dense geometry and articulation in a structured latent space (anigen/models/structuredlatentvae/).
The codebase contains 176 files across Python (anigen/, extensions/, assets/), C/C++ headers (extensions/CUBVH/include/), and CUDA sources (extensions/CUBVH/src/). The CUBVH extension provides GPU-accelerated BVH traversal for ray intersection and distance queries, essential for the rendering pipeline (extensions/CUBVH/cubvh/api.py).
How To Use It
Setup: The repository requires Linux with an NVIDIA GPU (≥18GB memory). Python dependencies are listed in requirements.txt. The CUBVH CUDA extension must be built from extensions/CUBVH/setup.py and extensions/CUBVH/pyproject.toml. No lockfile is present, so dependency versions should be pinned explicitly.
Configuration: Config files reside in configs/: anigenskinae.json, slatdae.json, slatflowauto.json, ssdae.json, ssflowduet.json. These define VAE architecture and flow-matching parameters for different asset categories.
Running it: The entry point is app.py, a Flask-based server. Launch with python app.py. Alternative training/evaluation scripts include train.py and example.py. The Hugging Face demo is available at the project page referenced in README.md.
Real-World Use
A downstream system can invoke the pipeline programmatically:
from anigen.pipelines.anigenimageto3d import AniGenPipeline
pipeline = AniGenPipeline.fromconfig("configs/slatflowauto.json") mesh, skeleton, skinning = pipeline.generatefromimage("input_image.png") Export for use in USD, GLTF, or physics engines mesh.export("output.usda")
This produces a rigged mesh with coherent skeleton and skinning weights that can be driven by standard animation retargeting rigs, suitable for embodied AI agents or physics-based simulation pipelines.
Code Health & Issues
No test files detected - untested code paths repository-wide. Critical training and rendering modules lack unit tests. No CI/CD pipeline - no automated build/test gate in .github/ or configured CI. Merges can break dependent workflows silently. Dependencies declared without a lockfile - extensions/CUBVH/pyproject.toml and requirements.txt lack version pinning, risking non-reproducible builds across environments. No input validation at entry points - app.py accepts uploads without size or format guards, risk OOM crashes on malformed inputs. Sparse structural scaffold may produce degenerate topologies - the confidence-decaying skeleton field has not been validated on edge cases (e.g., highly articulated machinery with overlapping bone structures).
The Bottom Line
AniGen delivers a technically cohesive approach to single-image 3D asset generation with a clear architectural vision (S³ Fields, two-stage flow matching). The codebase is substantial and well-organized for a research framework. However, the absence of testing, CI, and dependency locking makes it unsuitable for production pipelines without significant operational hardening. Best suited for research prototyping or teams with dedicated DevOps to establish reproducibility gates.