Fine-tuning large language models typically demands significant engineering overhead: managing GPU memory, writing training loops, and configuring hyperparameters across disparate tools. For teams constrained by hardware, running an 8B parameter model is often infeasible without cloud infrastructure.
What This Does
Soup abstracts the training pipeline into a single YAML-driven workflow. Its distinguishing feature is layer streaming, which decouples the frozen base model from VRAM, streaming decoder layers to the GPU incrementally. As documented in docs/performance-and-quantization.md, this approach measured Llama-3.1-8B-Instruct at 119.6 tok/s with a 3.32 GB peak on an RTX 3050 Laptop 4 GB, matching resident performance. The configuration flag stream_layers: true opt-in enables this, though it remains marked BETA in the codebase. The benchmarks/ directory contains the measurement harness and gate files (e.g., gate-v0.72.2-nf4.md) that validate these claims across hardware.
How It Is Wired
Execution originates at the CLI entry point src/soup_cli/cli.py. The command soup train triggers config loading, likely from pyproject.toml or an inline YAML specification. The training orchestration resides within the src/soup_cli/commands/ directory. The layer streaming mechanism interfaces with the Transformers framework (indicated by the Transformers dependency) to manage the layer-by-layer GPU transfer, keeping the base model weights resident on disk or system RAM until needed. The repository uses Docker (Dockerfile, docker-compose.yml) for environment consistency and GitHub Actions (/.github/workflows/) for CI/CD, including dependency drift checks.
How To Use It
# Install with training extras
pip install "soup-cli[train]"
# Initialize a chat template config
soup init --template chat
# Start training
soup train
Configuration is driven by YAML files located in examples/configs/ (e.g., sft_basic.yaml). The pyproject.toml at the root declares project dependencies; no lockfile is present, which the heuristic analysis flags as a reproducibility risk.
Real-World Use
A practical scenario involves fine-tuning a code-assistant model on a developer's laptop. Using the sft_basic.yaml config from examples/configs/, a user can initiate soup train to perform SFT without provisioning a cloud GPU. The layer streaming feature allows the 8B model to train within the 4 GB VRAM limit, making it viable for local iterative development.
Code Health & Issues
The heuristic analysis identified one measured finding: [Low/Risk] Dependencies declared without a lockfile - non-reproducible builds - pyproject.toml. Beyond this, the structure supports a mature SDLC: 428 test files in tests/, 81 doc files, and active CI pipelines (/.github/workflows/ci.yml, /.github/workflows/dependency-drift.yml). The Apache-2.0 license is present in LICENSE.
The Bottom Line
Soup is a practical tool for resource-constrained LLM fine-tuning, with layer streaming as its technical differentiator enabling 8B model training on 4 GB GPUs. The primary risk is the absence of a lockfile, which introduces reproducibility concerns for dependency versions. It is well-suited for researchers or engineers needing to fine-tune on local hardware without cloud dependencies.
Clone:https://github.com/moses-y/Soup
What the analyser found
Deployment readiness
6/7
✓Container image
✓CI pipeline
✗Lockfile committed
✓Test suite
✓README
✓License
✓No committed secrets
Composition
1,086 files
Python909
Markdown90
YAML46
JSON2
Jupyter Notebook1
TOML1
FlaskExpressDockerTransformers
Architecture
Top-level areas of the codebase, sized by module count. Arrows show how many imports cross from one area into another.
Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.
high2
medium8
low1
Pin third-party GitHub Actions to a commit SHAhigh7 occurrences
A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.
Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.
Commit a lockfile beside the manifesthigh
pyproject.toml
manifest with no lockfile in scope
An unlocked range means the artifact you tested and the artifact you ship can contain different transitive code, so a malicious patch release reaches production with no diff.
Fix: Run the package manager once and commit the generated lockfile.
Declare least-privilege permissions for GITHUB_TOKENmedium2 occurrences
.github/workflows/ci.yml
2 workflow(s) declare no permissions, 1 of them reference secrets
With no declaration the token inherits the repository default, so any injected step can push commits or mint releases from inside your own CI.
Fix: Add permissions: contents: read at the top of the workflow and widen per job only where needed.
Enable Dependabot or Renovatemedium
1 manifest(s), no update bot configured
Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.
Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.
Pin the container base image by digestmedium
Dockerfile
nvidia/cuda:12.1.0-runtime-ubuntu22.04
An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.
Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.
Gate pull requests on a dependency vulnerability scanmedium
.github/workflows
no dependency scan in CI
This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.
Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.
Set a random seed in notebooks that train or samplemedium
notebooks/proof-4gb.ipynb
stochastic operations with no seed
The accuracy figure in a client deliverable moves on every rerun, so nobody can tell whether a change helped.
Fix: Set one SEED constant at the top and pass it to every stochastic call.
Set persist-credentials: false on checkoutmedium
.github/workflows/ci.yml
checkout keeps the token, then dependencies are installed
The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.
Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.
Add a non-root USER to the imagemedium
Dockerfile
CMD or ENTRYPOINT with no USER directive
A process running as root in the container is root against every mounted volume, and it turns any container escape or writable-mount mistake from a contained problem into a host one.
Fix: Create an unprivileged user, chown what it needs, and end the Dockerfile with USER.
Pin the resolution so a vulnerable version cannot installmedium2 occurrences
no lockfile, and the declared ranges reach pydantic@2.0.0 CVE-2024-3772 (critical) +1 more
The range includes versions with published advisories and nothing in the repository records which one was resolved, so two installs a month apart can differ in whether they are vulnerable and neither is reproducible from what is committed.
Fix: Commit the lockfile, which fixes the resolution and makes this answerable.
Set timeout-minutes on the workflow jobslow4 occurrences
.github/workflows/ci.yml
4 workflow(s) declare no job timeout
A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.
Fix: Add timeout-minutes with a realistic bound to each job.
Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.