The Problem

Relational databases store most of the world's structured data, but machine learning models for tabular data are typically trained per-task. Each new schema, join path, or prediction target requires a bespoke model. Foundation models work for text and images because they learn transferable structure; relational data lacks an equivalent. This repo is a reference implementation for the ICLR 2026 paper proposing exactly that: a zero-shot foundation model for relational tables.

What This Does

The repository implements the Relational Transformer, a model that learns from multiple relational databases and generalizes to unseen ones without fine-tuning. It uses a Rust-based sampler (rustler/) to preprocess raw RelBench datasets into training sequences, then a PyTorch model (rt/) for pretraining and task-specific evaluation. The architecture is a standard transformer with relational-aware masking (rt/model.py defines MaskedAttention, FFN, RelationalBlock).

The work is split into two components: rustler/ handles data preprocessing and sequence generation, while rt/ handles model definition, training, and evaluation. Scripts in scripts/ orchestrate the full pipeline: pretraining, continued pretraining, fine-tuning, and visualization.

How It Is Wired

Execution starts at main in rt/main.py:66. This is the hub of the system: it is called from 3 modules, reaches 7 functions, and is the only entry point that touches the filesystem. From main, control flows to RelationalDataset (the data loader, called from 2 places), TextEmbedder, and evaluate. The shortest path from entry point to external effect is main -> checkpoint, which writes to disk via save_ckpt_dir_.mkdir. That is one hop from the entry point to a filesystem write.

The most-connected modules are rt/main (3 importers, 2 dependencies), rt/tasks (3 importers, 0 dependencies), and rt/data (2 importers, 0 dependencies). rt/model.py is the largest file by responsibility: 10 functions and 4 classes, imported by 2 other files. It defines the transformer blocks and attention mechanisms that everything else routes through. rt/embed.py is the only module that calls a model for inference, via TextEmbedder. The system has no circular dependencies, so the module graph is a clean DAG: main and tasks sit at the top, model and data at the bottom.

The Rust side (rustler/src/) mirrors this: lib.rs exposes the sampler, pre.rs handles preprocessing (601 lines, the largest file in the repo), and fly.rs handles sequence generation. The call graph here is not mapped by the analysis; the Rust/Python boundary is crossed via maturin bindings, not Python imports.

How To Use It

The README documents the full workflow; use it verbatim:

git clone https://github.com/snap-stanford/relational-transformer
cd relational-transformer
pixi install
cd rustler
pixi run maturin develop --uv --release

Data preparation requires downloading RelBench datasets, running the Rust preprocessor (pixi run cargo run --release -- pre rel-f1), and generating text embeddings (pixi run python -m rt.embed rel-f1). Preprocessed data and pretrained checkpoints are hosted on Hugging Face Hub, so you can skip preprocessing entirely. Training scripts live in scripts/: example_pretrain.py, example_contd_pretrain.py, and example_finetune.py. All require a Weights & Biases login (pixi run wandb login or pixi run wandb disabled).

Real-World Use

A practical scenario: you have a new relational database (say, an e-commerce schema with customers, orders, and products) and want to predict churn. Instead of training a model from scratch, you download the pretrained checkpoint, point the fine-tuning script at your task, and run scripts/example_finetune.py. The model transfers relational structure learned from other databases and adapts with minimal task-specific data. The Hugging Face checkpoints (pretrain_<dataset>_<task>.pt) let you skip pretraining entirely.

Code Health & Issues

Static analysis (not opinion) found 5 issues across 3 categories:

  • High - Cognitive load - Deep nesting (max depth 8) in rt/main.py, rustler/src/pre.rs, rustler/src/fly.rs. Control flow is hard to follow; flatten with early returns.
  • High - Clarity - 13 duplicated 6-line blocks across 5 files (rt/data.py, rt/main.py, all three scripts/example_*.py). Extract shared helpers.
  • Medium - Cognitive load - rustler/src/pre.rs is 601 lines. Split by responsibility.

SDLC observations from the file structure: no test files (16 source files, zero tests), no CI configuration, no license file (all rights reserved by default), and a pixi.lock exists but no Python lockfile alongside pyproject.toml. One committed secret scan came back clean.

The Bottom Line

This is a serious research implementation with a real Rust/Python split and a working end-to-end pipeline. The zero-shot relational claim is backed by a published paper and hosted checkpoints. The absence of tests and CI is typical for research code but means any modification ships with no safety net. Use it if you need a reference implementation of relational foundation models; productionize carefully.