The Problem

OpenMythos is a theoretical reconstruction of the Claude Mythos architecture with a Recurrent-Depth Transformer (RDT) design featuring Prelude, looped Recurrent Block, and Coda stages. The codebase has no CI/CD pipeline, no dependency lockfile, and lacks update bots — meaning changes merge without automated verification, and dependency security gaps persist until manual audit. The measured code health audit found two HIGH-severity issues (missing lockfile, no CI) and one MEDIUM issue (no Dependabot/Renovate configuration) that directly affect production readiness and maintenance velocity.

What This Does

OpenMythos implements a Recurrent-Depth Transformer (RDT) architecture with three stages: Prelude (transformer blocks), a looped Recurrent Block (up to max_loop_iters), and a final Coda. Attention is switchable between MLA and GQA, and the feed-forward uses a sparse MoE with routed and shared experts. The repository has 20 files across open_mythos/ (5 code files), training/ (2 files), tests/ (3 files), docs/ (2 files), and root-level configs.

Key entry points and their reach:

  • main (open_mythos/main.py:311) reaches 9 function(s), called from 1 place
  • generate (open_mythos/main.py:1000) reaches 10 function(s), called from 2 place(s)
  • forward (open_mythos/main.py:101) reaches 9 function(s), called from 1 place

The internal call graph has 138 resolved call edges. precompute_rope_freqs is called from 24 places (highest connectivity), apply_rope from 14, and MythosConfig from 9. open_mythos/main.py (29 functions, 12 classes) and open_mythos/moda.py (26 functions, 10 classes) are the most widely routing modules — changes here ripple widely.

Code health findings (static analysis, 6 total: 2 high, 4 medium):

  • [HIGH/cognitive_load] Deep nesting x3 in open_mythos/main.py, training/3b_fine_web_edu.py, open_mythos/moda.py — max indentation depth 8 makes control flow hard to follow
  • [MEDIUM/clarity] Duplicated code blocks in open_mythos/__init__.py and test_main.py — 5 repeated 6-line blocks
  • [MEDIUM/oversized] open_mythos/main.py (794 lines) and open_mythos/moda.py are hard to hold in one head; a change ripples widely

WHAT THIS CODE TOUCHES OUTSIDE ITSELF: 3 function(s) call a model for inference; 2 function(s) read or write files. The traced path from entry: main -> save_checkpoint [filesystem via os.makedirs]. This is 1 hop from execution to filesystem effect.

How It Is Wired

Execution starts at main in open_mythos/main.py. The call graph shows precompute_rope_freqs (24 call sites) and apply_rope (14 call sites) as the most widely connected functions — these are the rotary position embedding computation and application that feed into every TransformerBlock forward pass. MythosConfig (9 call sites) is the configuration dataclass that gets passed through __init__ and forward throughout the model. open_mythos/moda.py defines the Recurrent Block's _build_cache and forward, which contains the looped recurrence mechanism. training/3b_fine_web_edu.py defines save_checkpoint which writes to filesystem via os.makedirs — the only path from execution outside the process. open_mythos/variants.py (7 functions) provides pre-configured scales from 1B to 100B parameters via mythos_1b(), mythos_3b(), etc.

The module connectivity shows open_mythos/__init__ receives imports from 2 places and exports to 3 — it's a thin but non-trivial integration layer. test_main.py (80 functions, 16 classes) is the test entry point that calls into models for inference, defining gqa_cfg, mla_cfg, and test utilities.

How To Use It

Setup: Install via pip from the cloned repository:

git clone https://github.com/moses-y/OpenMythos
cd OpenMythos
pip install -e ".[dev]"

Or using uv (inferred from pyproject.toml):

uv pip install -e .

Configuration: No environment variables or external config files are required — all parameters are passed as a dict to MythosConfig or the variant factory functions. The attn_type field ("mla" or "gqa") routes to different attention implementations. For MLA: n_kv_heads=1 with kv_lora_rank and q_lora_rank. For GQA: n_kv_heads=2.

Running it: Instantiate and run from Python:

import torch
from open_mythos.main import OpenMythos, MythosConfig

base = {
    "vocab_size": 1000,
    "dim": 256,
    "n_heads": 8,
    "max_seq_len": 128,
    "max_loop_iters": 4,
    "prelude_layers": 1,
    "coda_layers": 1,
    "n_experts": 8,
    "n_shared_experts": 1,
    "n_experts_per_tok": 2,
    "expert_dim": 64,
    "lora_rank": 8,
    "attn_type": "mla",
}

if attn_type == "gqa":
    cfg = MythosConfig(**base, n_kv_heads=2)
else:
    cfg = MythosConfig(
        **base,
        n_kv_heads=8,
        kv_lora_rank=32,
        q_lora_rank=64,
        qk_rope_head_dim=16,
        qk_nope_head_dim=16,
        v_head_dim=16,
    )

model = OpenMythos(cfg)
ids = torch.randint(0, cfg.vocab_size, (2, 16))
logits = model(ids, n_loops=4)
out = model.generate(ids, max_new_tokens=8, n_loops=8)

Real-World Use

This architecture is suited for research on compute-adaptive, depth-variable reasoning. The recurrent loop (max_loop_iters) allows dynamic reasoning depth — each token can trigger multiple recurrence steps before the Coda produces an output. The MoE sparsity with routed+shared experts enables scaling parameter count without proportional compute cost. The MLA/GQA attention switchability lets you trade off KV cache efficiency against attention granularity. A practical workflow: train on a text corpus with training/3b_fine_web_edu.py, then generate with model.generate() at varying n_loops values to explore how recurrence depth affects output quality for tasks like theorem proving or multi-step QA.

Code Health & Issues

The measured analysis found these issues from static analysis of 12 Python files:

  • [HIGH] Commit a lockfile beside the manifest — pyproject.toml has no lockfile. Unlocked ranges mean the artifact tested and the artifact shipped can contain different transitive code.
  • [HIGH] Add a workflow that builds and tests this repository — no CI configuration exists across 12 source files. Every change merges with nobody having run the build.
  • [MEDIUM] Enable Dependabot or Renovate — 3 manifest files with no update bot configured. Without a bot, a published advisory sits unpatched until manual audit.

Beyond the measured findings: no CI/CD pipeline (.github/ absent), no Dockerfile, licence file present but SPDX identifier unverified, no lockfile for reproducible builds.

The Bottom Line

OpenMythos is a well-structured theoretical implementation of a Recurrent-Depth Transformer with clean architecture choices (MLA/GQA switching, MoE FFN, looped recurrence). The codebase is functional and the entry points are clear. However, the absence of a lockfile and CI/CD means it's not production-ready without those gates being added. The cognitive load from deep nesting and oversized files (794/26 lines) is the main maintainability risk — these are addressable with guard-clause flattening and file splitting. Use this for research prototyping or architecture exploration, not production serving until the SDLC gaps are closed.