maxun - Moses Yebei

Abstract

✨ The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes ✨

The Problem

Manual web‑scraping pipelines require custom code for each site, fragile selectors, and ad‑hoc orchestration. Teams spend weeks building and maintaining scrapers that break on layout changes, and they lack a repeatable way to expose scraped data as an API.

What This Does

maxun delivers an open‑source, no‑code platform that turns any website into a structured API. The repo bundles six related projects:

src/ – React UI and core client logic (e.g., src/context/globalInfo.tsx, src/helpers/clientSelectorGenerator.ts).
server/ – Node back‑end handling recording storage, browser orchestration, and workflow scheduling (server/src/index.ts, server/src/server.ts).
maxun‑core/ – Shared runtime used by both UI and back‑end (maxun-core/src/interpret.ts).
browser/ – Tiny HTTP server that drives a headless Chromium instance (browser/server.ts).
legacy/ – Historical UI components kept for reference.
perf/ – Performance‑testing helper (perf/performance.ts).

The platform supports four robot types (Extract, Scrape, Crawl, Search) and ships an SDK for programmatic use. UI actions are recorded, transformed into selector trees, and persisted via the back‑end API.

How It Is Wired

Execution begins in browser/server.ts (frontend entry) and server/src/index.ts (backend entry).

Backend start – server/src/index.ts imports server/src/server.ts, which creates an Express app, registers routes (e.g., src/api/storage.ts) and starts listening. The server’s logger (server/src/logger.ts) is a high‑traffic hub: 26 modules import it.
Request flow – A typical “run robot” request hits src/api/storage.ts::createLLMRobot, which calls server/src/sdk/workflowEnricher.ts::enrichWorkflow. That module reaches out to an LLM via axios.put (network) and may read a file (fs.readFileSync) for version data.
Browser orchestration – server/src/browser-management/controller.ts::initializeRemoteBrowserForRecording spawns a remote Chromium process. Calls eventually reach browser/server.ts where the headless browser executes recorded actions.
Global state – src/context/globalInfo.tsx supplies useGlobalInfoStore and notify. It is the most connected module (30 importers) and a frequent source of side‑effects (notify called from 59 places).
Cycles – Import cycles involve src/context/globalInfo.tsx, src/api/storage.ts, and server/src/server.ts. The cycle inflates build times and makes refactoring risky because a change in any member can trigger recompilation of the whole cycle.
Outbound effects – Across the code base there are 67 network calls (mostly Axios), 4 DB writes, 4 cryptographic operations, and 6 model invocations. The shortest traced path from a scheduler entry (readyHandler in server/src/workflow‑management/scheduler/index.ts) to an outbound request is readyHandler → readyForRunHandler → interpretStoredRecording → axios.post.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/maxun
cd maxun

# Install dependencies (npm works for all package.json files)
npm ci               # creates lockfiles if added later
# Build and run the full stack with Docker
docker compose up --build

Environment – copy ENVEXAMPLE to .env and fill in required keys (e.g., database URL, LLM endpoint). The backend reads these variables in server/src/index.ts.

Running a robot – Once containers are up, the UI is served at http://localhost:3000. Use the web UI to record actions or invoke the SDK (maxun-core) from another service. The SDK entry point is maxun-core/src/interpret.ts::run.

Real‑World Use

A data‑engineering team can embed Maxun in a nightly pipeline:

import { run } from 'maxun-core';
await run({
  robotId: 'abc123',
  mode: 'extract',
  output: 'json',
});

The call triggers the scheduler, launches a headless browser, extracts selectors, and stores the result in the PostgreSQL instance defined by the Docker compose file.

Code Health & Issues

High – No test suite – 182 source files, zero test files.
High – No lockfile – browser/package.json lacks a lockfile, risking non‑reproducible builds.
High – No CI workflow – No GitHub Actions or other CI config.
High – No build gate for Docker image – browser/Dockerfile is never validated by CI.
Medium – Dependabot missing – No automated dependency updates.
Medium – Base image not pinned – Dockerfile.backend uses mutable node:20-slim.
Medium – Container runs as root – No non‑root USER directive.
Low – Repository conventions missing – No .editorconfig, .gitattributes, or formatter config.

The Bottom Line

Maxun provides a complete, containerized stack for turning web interactions into API endpoints, with a clear separation between UI, back‑end, and core runtime. The codebase is functional but suffers from high cyclomatic complexity, missing tests, and limited automation. It is suitable for teams that can invest in stabilising the build pipeline and refactoring the hub modules before extending production use.

What the analyser found

Deployment readiness

4/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

234 files

TSX91
TypeScript82
JSON16
JavaScript9
Markdown6
Shell3

ReactDockerLangChain

Architecture

Top-level areas of the codebase, sized by module count. Arrows show how many imports cross from one area into another.

graph TD g0["src 93 modules"] g1["server 55 modules"] g2["legacy 19 modules"] g3["maxun-core 9 modules"] g4["(root) 2 modules"] g2 -->|33| g0 g1 -->|2| g0

Module dependencies

The 10 most depended-upon modules of 180, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_src_context_globalInfo["globalInfo"] n1_src_context_socket["socket"] n2_server_src_logger["logger"] n3_src_apiConfig["apiConfig"] n4_server_src_models_Robot["Robot"] n5_c_components_ui_GenericModal["GenericModal"] n6_src_api_storage["storage"] n7_server_src_models_Run["Run"] n8_src_context_theme_provider["theme-provider"] n9_src_api_recording["recording"] n7_server_src_models_Run --> n4_server_src_models_Robot n9_src_api_recording --> n3_src_apiConfig n6_src_api_storage --> n3_src_apiConfig n0_src_context_globalInfo --> n6_src_api_storage n1_src_context_socket --> n3_src_apiConfig classDef cyc stroke:#ef4444,stroke-width:2px; class n0_src_context_globalInfo,n6_src_api_storage cyc;

Findings

MediumNo test files detected - untested code paths repository-wide
MediumNo CI/CD pipeline detected - no automated build/test gate .github/ or CI config
LowDependencies declared without a lockfile - non-reproducible builds browser/package.json

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high4

medium3

low1

Add a test suite; this repository has nonehigh182 occurrences

182 source files, no test files

Any change ships with no signal that existing behaviour still holds, so a regression reaches production undetected.

Fix: Add one test per public entry point, then a CI step that runs them.

Commit a lockfile beside the manifesthigh

browser/package.json

manifest with no lockfile in scope

An unlocked range means the artifact you tested and the artifact you ship can contain different transitive code, so a malicious patch release reaches production with no diff.

Fix: Run the package manager once and commit the generated lockfile.

Add a workflow that builds and tests this repositoryhigh

182 source files, no CI configuration

Every change merges with nobody having run the build once, which across this estate is 499 repositories.

Fix: Add a workflow running the project build and test command on push and pull_request.

Add a build gate for the deployable artifacts herehigh2 occurrences

browser/Dockerfile

deployment artifacts with no workflow

Something in the repo is meant to reach a server and no automated step has ever validated it, which is the state of 125 repositories here.

Fix: Add a workflow that at least builds the image and validates the manifests.

Enable Dependabot or Renovatemedium

3 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Pin the container base image by digestmedium

Dockerfile.backend

node:20-slim

An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.

Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.

Add a non-root USER to the imagemedium

Dockerfile.backend

CMD or ENTRYPOINT with no USER directive

A process running as root in the container is root against every mounted volume, and it turns any container escape or writable-mount mistake from a contained problem into a host one.

Fix: Create an unprivileged user, chown what it needs, and end the Dockerfile with USER.

Add the repository convention files this project lackslow3 occurrences

missing .editorconfig, .gitattributes, a formatter config

Without them one contributor's editor writes tabs into a Python file, a shell script commits with CRLF and fails in the container, and a notebook diff is unreviewable.

Fix: Add .editorconfig, .gitattributes with text=auto eol=lf, and a formatter config.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.