jitsu - Moses Yebei

Abstract

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

The Problem

Teams collecting event data from websites and apps need a self-hosted alternative to commercial platforms like Segment. The existing options either lock data into a vendor's cloud or require significant engineering effort to build ingestion, transformation, and warehouse delivery from scratch.

What This Does

Jitsu is a real-time event collection and data ingestion platform. The repo is a collection of seven self-contained projects: bulker/ (Go-based ingestion engine with SQL warehouse adapters), webapps/ (React/Next.js console for configuration and management), libs/ (shared TypeScript utilities), services/ (auxiliary services), plus cli/, types/, and e2e/. The system collects events via HTTP API or SDKs, routes them through Kafka, and batches them into warehouses like ClickHouse and BigQuery.

The bulker/ directory contains the core ingestion engine with connectors, SQL implementations, and a dead-letter reprocessing system. The webapps/console/ provides the management UI, configuration schema, and API layer.

How It Is Wired

Execution starts at multiple entry points. The primary path is main in bulker/admin/main.go, which reaches 404 functions and ultimately runs external commands via exec.run and reads files via os.Open. The Run function in bulker/jitsubase/appbase/app_base.go is called from 47 places and reaches 411 functions, making it the most central execution path.

The highest-blast-radius modules are in webapps/console/lib/: api.ts (79 modules depend on it), server/db.ts (75 dependents), and context.tsx (62 dependents, in a circular dependency). The most-called internal functions are append (139 call sites), Infof (82), and Close (79). The bulker/jitsubase/jsoniter/reflect.go file is the most connected, called from 83 files.

The system touches external resources: 54 functions read/write databases, 32 handle files, 7 make network calls, and 5 perform cryptographic operations. Traced paths show the flow from entry points through Run, Store, and Close functions to actual filesystem, network, and subprocess operations.

How To Use It

Setup: Clone and run via Docker Compose. The README documents:

git clone --depth 1 https://github.com/moses-y/jitsu
cd jitsu/docker
touch .env.local

Configuration: Set environment variables in docker/.env or .env.local. The repo includes .env.example at the root. Production deployments use the Helm charts in helm/.

Running: Use Docker Compose for development, or deploy the bulker/ services to Kubernetes. The console runs as a Next.js app in webapps/console/.

Real-World Use

A SaaS company deploys Jitsu behind their application, uses the JavaScript SDK to send page-view and custom events, and configures ClickHouse as the destination via the web console. Events flow through the bulker ingestion pipeline, get batched per the configured window, and land in ClickHouse for analytics queries. The dead-letter reprocessor in bulker/admin/deadletter_reprocessor.go handles failed batches.

Code Health & Issues

Static analysis found 255 findings (92 high, 160 medium, 3 low) across 6 kinds:

High – Import cycle members (23 instances): webapps/console/lib/context.tsx, schema/index.ts, ui.tsx participate in circular imports, making changes ripple unpredictably.
High – Hub modules (20 instances): webapps/console/lib/api.ts and server/db.ts have 79 and 75 dependents respectively; any change here has wide blast radius.
High – Oversized files (6): WorkspacePageLayout.tsx at 683 lines, destinations.tsx, and admin/notifications.ts are hard to maintain.
High – Deep nesting (6): server/sync.ts and admin/notifications.ts reach indentation depth 13.
Medium – Empty catch blocks (4): store/index.tsx and warehouse-store.ts silently discard errors.
Medium – High branching density: serverEnv.ts has 55 branch points over 137 lines.

The code health audit flagged 8 issues, including committed secrets in docker/.env, unpinned GitHub Actions (using @v4 tags instead of commit SHAs), and workflows pushing directly to main without pull requests.

The Bottom Line

Jitsu is a substantial, production-grade ingestion platform with real architectural depth in the bulker Go engine. The console code has notable maintainability debt in its hub modules and circular dependencies. Teams needing a self-hosted Segment alternative with warehouse support should evaluate it; expect to invest in refactoring the console before heavy customization.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

1,266 files

TypeScript368
Go352
TSX203
JSON43
YAML34
Markdown31

ReactNext.jsTailwindDockerKubernetes

Module dependencies

The 10 most depended-upon modules of 583, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_webapps_console_lib_api["api"] n1_ebapps_console_lib_server_db["db"] n2_webapps_console_lib_context["context"] n3_pps_console_lib_schema_index["index"] n4_bapps_console_lib_server_log["log"] n5_console_lib_server_serverEnv["serverEnv"] n6_webapps_console_lib_ui["ui"] n7_webapps_console_lib_useApi["useApi"] n8_ents_JitsuButton_JitsuButton["JitsuButton"] n9_ents_GlobalError_GlobalError["GlobalError"] n9_ents_GlobalError_GlobalError --> n6_webapps_console_lib_ui n9_ents_GlobalError_GlobalError --> n2_webapps_console_lib_context n8_ents_JitsuButton_JitsuButton --> n2_webapps_console_lib_context n0_webapps_console_lib_api --> n3_pps_console_lib_schema_index n0_webapps_console_lib_api --> n1_ebapps_console_lib_server_db n0_webapps_console_lib_api --> n4_bapps_console_lib_server_log n0_webapps_console_lib_api --> n5_console_lib_server_serverEnv n2_webapps_console_lib_context --> n3_pps_console_lib_schema_index n3_pps_console_lib_schema_index --> n7_webapps_console_lib_useApi n1_ebapps_console_lib_server_db --> n4_bapps_console_lib_server_log n1_ebapps_console_lib_server_db --> n5_console_lib_server_serverEnv n4_bapps_console_lib_server_log --> n5_console_lib_server_serverEnv n6_webapps_console_lib_ui --> n9_ents_GlobalError_GlobalError n7_webapps_console_lib_useApi --> n2_webapps_console_lib_context classDef cyc stroke:#ef4444,stroke-width:2px; class n2_webapps_console_lib_context,n3_pps_console_lib_schema_index,n6_webapps_console_lib_ui,n7_webapps_console_lib_useApi,n8_ents_JitsuButton_JitsuButton,n9_ents_GlobalError_GlobalError cyc;

Findings

LowSecret-shaped paths present; the code-health audit confirms or clears them docker/.env, helm/values-secrets.example.yaml, webapps/console/lib/schema/secrets.ts

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high4

medium3

low1

Pin third-party GitHub Actions to a commit SHAhigh3 occurrences

.github/workflows

dorny/paths-filter@v4, pnpm/action-setup@v4, openai/codex-action@v1

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Remove the committed .env and rotate what it holdshigh

docker/.env

A tracked .env is the most common route for a working key to reach a public clone, and it is the file the app actually loads, so the value is usually live.

Fix: git rm --cached the file, add it to .gitignore, rotate every credential it names, commit a .env.example with empty values.

Stop discarding the exit code of steps whose failure mattershigh3 occurrences

.github/workflows/security-fix.yml

lines 75, 91, 258 discard failure

The job reports green while the artifact it was supposed to produce was never produced, so a crash becomes a successful run built on stale data.

Fix: Let the step fail, or test the specific expected condition instead of discarding all exit codes.

Open a pull request instead of pushing to the default branchhigh

.github/workflows/deploy.yml

git push origin main

Automated commits land on the branch that deploys, with no test having run against the result.

Fix: Push to a bot branch and open a pull request, or restrict the push to a tag ref.

Declare least-privilege permissions for GITHUB_TOKENmedium

.github/workflows/publish.yml

1 workflow(s) declare no permissions

With no declaration the token inherits the repository default, so any injected step can push commits or mint releases from inside your own CI.

Fix: Add permissions: contents: read at the top of the workflow and widen per job only where needed.

Enable Dependabot or Renovatemedium

31 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Set persist-credentials: false on checkoutmedium

.github/workflows/lint.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Set timeout-minutes on the workflow jobslow2 occurrences

.github/workflows/deploy.yml

2 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.