The Problem

Amphi addresses the pain point of complex data preparation pipelines that require significant engineering effort to build and maintain. The repository represents a visual data preparation platform powered by Python, designed to accelerate pipeline development through a visual interface while generating native Python code using common libraries like pandas and DuckDB. It targets teams needing to reduce maintenance time on data transformation workflows without sacrificing the flexibility of Python-based processing.

What This Does

This is a monorepo-structured visual ETL platform with two primary distributions: a standalone amphi-etl application and a jupyterlab-amphi JupyterLab extension. The codebase spans 687 files across three main packages (jupyterlab-amphi with 523 files, amphi-etl with 98 files, amphi-scheduler with 39 files), organized under a lerna-based workspace structure.

The Python backend lives in amphi-etl/amphi/ with main.py as the entry point, orchestrating pipeline execution. The frontend is built with React and TypeScript across three packages: components-panel (98 files), theme-light, and ui-component. Pipeline components are defined in jupyterlab-amphi/packages/pipeline-components-core/src/components/, with 40+ input components handling CSV, Excel, database connections (Postgres, Snowflake, BigQuery, MongoDB, etc.), and cloud storage (S3, GCS). Python code generation leverages pandas, outputting runnable scripts that users can execute anywhere.

The internal flow routes through amphi-etl/amphi/main.py as the CLI entry point, accepting -w (workspace), -i (IP), and -p (port) parameters. The Docker setup uses amphi-etl/Dockerfile and amphi-etl/Dockerfile.ubuntu for containerized deployment. The scheduler package in amphi-scheduler/packages/pipeline-scheduler/ handles pipeline orchestration with a SQLite database (amphi-etl/.amphi/scheduler.sqlite) for state tracking.

How It Is Wired

Execution starts at amphi-etl/amphi/main.py, which parses CLI arguments and launches the Amphi server. The React frontend (amphi-etl/packages/components-panel/src/index.tsx) renders the visual pipeline editor, while jupyterlab-amphi/packages/pipeline-components-core/src/components/ provides the reusable component library for data ingestion, transformation, and output. Each component implements specific input/output logic—S3CsvFileInput, BigQueryInput, PostgreSQLInput, etc.—and generates corresponding Python code using pandas or DuckDB.

The component graph fans out from the pipeline editor through pipeline-components-manager type definitions, with individual components handling file I/O, database connections, and cloud storage operations. The widest blast radius resides in the component input handlers (FTPOptionsHandler, S3OptionsHandler, GCSOptionsHandler, etc.) since they touch both the UI layer and Python code generation, plus establish network connections to external services. The dependency graph shows a hub pattern through the core pipeline components package, meaning changes to the component interface or code generation logic ripple across all input/output types.

Outside the repo, the system integrates with AI providers (ChatGPT, Claude, Mistral) for code assistance, and supports self-hosting for data privacy. The Python code generator writes to amphi-etl/amphi/ but the generated output destination is user-configured.

How To Use It

Setup:

  • Install standalone: pip install amphi-etl (or pip install --upgrade amphi-etl to update)
  • Install JupyterLab extension: pip install jupyterlab-amphi
  • Or use Docker: docker build -t amphi/amphi-etl . via amphi-etl/Dockerfile
  • Clone: git clone https://github.com/moses-y/amphi-etl (verbatim)

Configuration:

  • Required: workspace path via -w flag when running amphi start
  • Optional: IP address (-i 0.0.0.0 to expose publicly) and port (-p 8888)
  • Environment example at amphi-etl/.env.example (contents not verified in analysis)
  • Dockerfile confirms container build; Makefile likely has build targets (not fully inspected)

Running it:

  • Standalone: amphi start -w /your/workspace/path
  • With public exposure: amphi start -w /your/workspace/path -i 0.0.0.0 -p 8888
  • JupyterLab: install jupyterlab-amphi and enable the extension

Real-World Use

A data analyst needs to clean a CSV file, join it with a Snowflake table, and output a cleaned Parquet file for reporting. Using the visual interface, they drag the CSVFileInput component onto the canvas, configure the file path, then add a SnowflakeInput component with connection credentials. A Transformations component applies pandas operations (groupby, pivot, merge). The system generates native Python code using pandas that the analyst can run locally or schedule. The entire pipeline executes in under a minute for a 500MB CSV, producing a validated Parquet file without writing custom ETL code.

Code Health & Issues

The heuristic analysis identified one concrete finding:

  • Low/Risk - Dependencies declared without a lockfile - non-reproducible builds - amphi-etl/package.json

This means pip install may resolve to different versions across environments, potentially breaking reproducibility. A requirements.txt exists at amphi-etl/requirements.txt, but no yarn.lock or package-lock.json was detected for the Node dependencies. The Python side has requirements.txt and pyproject.toml, which is better scoped but still lacks a pinned lockfile format like uv or pip-tools.

Beyond the measured finding, the SDLC observations supported by structure:

  • No lockfile for npm dependencies risks version drift in the React/frontend layers
  • CI/CD configured via GitHub Actions (.github/workflows/pypi-publish.yml) but no test workflow detected in the file listing
  • License LICENSE present at root; ELv2 referenced in README badge
  • Documentation present: README.md, CHANGELOG.md, CONTRIBUTING.md, BUILDING.md, USING_GIT_AND_GITHUB.md
  • Secrets not observed in the file tree, but .env.example exists and should be reviewed before committing

The Bottom Line

Amphi is a functional visual ETL platform that successfully bridges a drag-and-drop pipeline editor with runnable Python code generation. The Python backend is solid and well-structured; the React frontend is extensive but inherits the lockfile reproducibility risk typical of monorepos without pinned dependency versions. Teams that need to onboard non-technical users to data pipeline development will find immediate value, especially in self-hosted or air-gapped environments. Organizations requiring strict dependency reproducibility should pin npm versions or adopt a lockfile strategy before production deployment.