The Problem

This repository implements a small, fast, text-to-text prediction model for selecting the most appropriate response from a set of options given a prompt. It is a simplified, independent implementation inspired by TypeSafe's Jev model, designed to provide the same input-output behavior without the proprietary design details.

What This Does

The model takes a context string and a list of text options, then outputs a probability distribution over the options. It processes each option in parallel rather than generating output sequentially, making it efficient for fixed-option tasks like menu selection or categorization.

Key components:

  • jevlike/model.py: Implements the core model architecture with byte embeddings, attention, and scoring
  • jevlike/train.py: Handles model training on JSONL formatted data
  • jevlike/predict.py: Provides the prediction interface
  • jevlike/data.py: Manages data processing and synthetic data generation

The architecture uses:

  1. Byte-level embeddings (or optional Hugging Face transformers)
  2. Option-context attention
  3. Shared scoring across options
  4. Softmax output

How It Is Wired

Execution begins with the CLI scripts:

  • jevlike-train: Starts training by loading data, initializing the model, and running optimization
  • jevlike-predict: Loads a trained model and processes input prompts

Key control flows:

  1. Training: - train.py:main() loads data and config - Calls model.py:JevLike to build the architecture - Trains via PyTorch's standard optimization loop
  1. Prediction: - predict.py:main() processes input args - Calls model.py:JevLike.forward() to score options - Returns probabilities via softmax

The architecture is minimal with clear dependencies:

  • model.py contains the core logic
  • train.py and predict.py are thin wrappers
  • Data flows directly through data.py

Critical paths:

  • Training data → model parameters
  • Input text → byte embeddings → attention → scores → probabilities

How To Use It

Setup

uv venv
source .venv/bin/activate
uv pip install -e '.[dev]'

Running It

  1. Train with synthetic data: ``sh jevlike-data synthetic --output data/synthetic jevlike-train data/synthetic/train.jsonl \ --validation data/synthetic/validation.jsonl \ --output runs/synthetic.pt ``
  1. Evaluate: ``sh jevlike-eval runs/synthetic.pt data/synthetic/test.jsonl ``
  1. Predict: ``sh jevlike-predict runs/synthetic.pt \ --context "Choose the exact badge amber badger. Badge: amber badger." \ --option "azure crane" \ --option "amber badger" \ --option "gold heron" ``

With Pretrained Encoders

uv pip install -e '.[transformers]'
jevlike-train data/synthetic/train.jsonl \
  --validation data/synthetic/validation.jsonl \
  --output runs/qwen-head.pt \
  --encoder hf \
  --hf-model Qwen/Qwen2.5-0.5B \
  --rank 256 \
  --batch-size 8

Real-World Use

This model is optimized for fixed-option selection tasks:

  • Customer service routing ("refund" vs "technical support")
  • Menu navigation ("amber badger" vs "azure crane")
  • Categorization ("sales" vs "support")

Example in a customer service pipeline:

options = ["refund", "sales", "technical support"]
context = "The customer needs a refund."
model = JevLike.load("path/to/model.pt")
probs = model.predict(context, options)
selected = options[probs.argmax()]

Code Health & Issues

  • Medium/SDLC: No CI/CD pipeline detected - manual testing only
  • Low/Risk: No lockfile for dependency reproducibility
  • Low/Docs: Documentation covers core usage but lacks API reference
  • Low/Tests: Only 1 test file found (tests/test_smoke.py)

The Bottom Line

This repository provides a functional, minimal implementation of a text-to-text prediction model for fixed-option selection. It is well-suited for small-scale applications where performance is more important than accuracy. The implementation is clean but lacks production-grade infrastructure (CI/CD, comprehensive testing). For larger deployments, additional engineering work would be required to address these gaps.