The Problem
File type detection is a deceptively hard problem. Traditional tools like file(1) rely on magic byte signatures, which work well for binary formats but fail on textual content—source code, config files, and data formats often get misclassified or lumped into a generic "text" bucket. That ambiguity is a real problem for security scanners, malware triage, and content routing systems that need to know exactly what a file is before deciding how to process it.
What This Does
Magika is an AI-powered file type detector with a small, custom deep learning model (~a few MB) that classifies files across 200+ content types. It reads a limited subset of each file and returns a prediction in roughly 5ms on a single CPU. The project ships a Rust CLI (rust/), a Python package (python/src/magika/), a TypeScript library (js/src/magika.ts), and an in-progress Go port (go/magika/). Model artifacts live in assets/models/ with multiple versions; standardv33 is the current default.
The architecture is straightforward: feature extraction, ONNX model inference, and a per-content-type threshold system for confidence calibration. The Python and JS packages share the same core logic, with reference tests (js/test/inference-vs-reference.test.ts, python/tests/testinferencevsreference.py) ensuring cross-language consistency.
How To Use It
Setup: The Python package is the most mature path. Install from python/pyproject.toml:
pip install magika
For the JS library, use yarn from js/package.json. The Go bindings require the ONNX runtime (see go/onnx/onnxruntime.go).
Configuration: No environment variables or API keys required. Model selection is handled internally; you can override the model directory if you need a specific version.
Running it: The CLI is the primary interface:
magika /path/to/file
Or use it programmatically in Python:
from magika import Magika m = Magika() result = m.identifybytes(b"print('hello')") print(result.output.label)
The JS API mirrors this pattern via js/magika.ts.
Real-World Use
Magika is designed for high-throughput content routing. A security scanner could use it to triage uploaded files before deep analysis:
def routefile(raw: bytes) -> str: result = m.identifybytes(raw) if result.output.label in {"pdf", "docx", "xlsx"}: return "officesandbox" if result.output.label in {"zip", "rar", "7z"}: return "archiveunpacker" return "textanalysis"
The project reports production use in Gmail, Drive, and Safe Browsing at Google scale, and it's integrated with VirusTotal and abuse.ch.
Code Health & Issues
Med - Go bindings are incomplete: go/README.md and the code itself (e.g., go/onnx/onnx_zero.go) indicate WIP status. The ONNX runtime dependency is non-trivial to build. Low - Model version drift: Multiple model versions live in assets/models/ without clear deprecation signals. A consumer could unknowingly load an outdated model. Low - JS package marked experimental: The npm package powers the web demo but isn't positioned as production-ready in the docs. Good hygiene otherwise: 31 test files, GitHub Actions CI for all language bindings, CodeQL scanning, license, and a scorecard badge are all present. The repo is well-maintained.
The Bottom Line
Magika is a solid, production-proven tool for accurate file type detection, particularly for textual formats where traditional tools fail. The Python and CLI paths are mature and well-tested; the Go bindings are not ready for production use. If you need precise content type identification at scale and can tolerate a model-based approach (with its small inference cost), this is worth adopting.