The Problem

Data science workflows are fragmented: analysts write bespoke scripts for cleaning, modeling, visualization, and reporting. DeepAnalyze automates that pipeline end-to-end. The model takes raw data (CSV, Excel, JSON, databases) and produces complete analysis and a formatted report without human intervention. The pain point is the repetitive, multi-tool effort between raw data and a decision-ready analysis.

What This Does

DeepAnalyze is an agentic LLM system that autonomously executes a data science workflow: data preparation, analysis, modeling, visualization, and report generation. It is built on an 8B-parameter model (DeepAnalyze-8B) and a 500K-sample instruction dataset. The repo contains the model training pipeline, a FastAPI server (API/main.py), and a WebUI.

The repository is a portfolio of five self-contained projects, not a single codebase. The substantial ones are deepanalyze/ (training and inference, 711 code files), API/ (a FastAPI server exposing chat, file, and model endpoints), demo/ and playground/ (interactive examples and evaluation notebooks), and example/ (sample datasets). The deepanalyze/ directory vendors two major frameworks: ms-swift (model fine-tuning) and SkyRL (reinforcement learning training).

How It Is Wired

Execution starts at two entry points. The API server begins in API/main.py at main, which reaches 402 functions and is itself called from 16 places. The training loop begins in deepanalyze/ms-swift/swift/llm/app/app.py at run, reaching 401 functions. The shortest path from main to an external effect is main -> makedirs, touching the filesystem via os.makedirs in two hops.

The highest-traffic functions are print (called from 123 places), to (101), and from_pretrained (80). The deepanalyze/SkyRL/skyrl-train/skyrl_train/distributed/strategy.py file is the most-connected, called from 78 other files; it owns distributed setup and collective operations. The deepanalyze/ms-swift/swift/llm/template/base.py file is the largest hub, with 89 functions called from 22 files, and it handles template rendering and data loading. Both base.py and model/register.py sit inside circular import cycles, which makes changing them risky—a modification in one can break the other at import time.

Outside itself, the code calls models for inference (34 functions), makes outbound network calls (41), reads/writes files (78), runs external commands (19), and touches a database (3). The API/main.py uses allow_origins=["*"] with credentials enabled, a CORS misconfiguration.

How To Use It

The repo has pyproject.toml files under SkyRL/ and related packages, and a Dockerfile under deepanalyze/SkyRL/docker/. No root-level install command is documented in the provided files.

# Clone and install the API server
git clone https://github.com/moses-y/DeepAnalyze
cd DeepAnalyze/API
pip install -r requirements.txt  # if present; otherwise, use the pyproject.toml in SkyRL/
python main.py

Configuration lives in API/config.py and deepanalyze/SkyRL/skyrl-train/.env.example. The API server exposes chat, file, and model endpoints via API/chat_api.py, API/file_api.py, and API/models_api.py. The example/ folder contains sample datasets and usage scripts.

Real-World Use

A data team drops a messy CSV into the API, and DeepAnalyze returns a cleaned dataset, a fitted model, visualizations, and a written report—no manual EDA or modeling code. The API/example/exampleOpenAI.py file shows an OpenAI-compatible client call, so integration is a standard HTTP request.

Code Health & Issues

Static analysis found 507 findings (159 high, 328 medium, 20 low) across six kinds. The high-severity ones:

  • High - Deep nesting - deepanalyze/ms-swift/swift/llm/template/base.py, model/register.py, model/utils.py - max indentation depth 8, hard to follow.
  • High - Oversized files - template/base.py (1988 lines), model/register.py, trainers/mixin.py - high change blast radius.
  • High - Import cycle members - model/register.py, template/base.py, model/utils.py - circular imports.
  • High - Hub modules - model/constant.py, model/register.py, template/constant.py - 29 modules depend on each.

SDLC observations: no CI pipeline, a committed .env.llm_judge file in SkyRL/skyrl-train/ (rotate credentials), a 5.6MB assets/report.png that should be in LFS, and pyarrow is 10 major versions behind.

The Bottom Line

The core idea is sound and the training data is substantial, but this is a research artifact, not a production service. The vendored frameworks are complex and the codebase has real maintainability issues (cycles, oversized files, no CI). Use it to reproduce the research or fine-tune the model, not as a drop-in data analysis service.