sie - Moses Yebei

The Problem

Agents that rely on multiple LLMs and specialized models typically need a separate server for each task (embedding, OCR, safety, etc.). Managing those servers, scaling them, and keeping a consistent API is costly and error‑prone, especially in production clusters.

What This Does

SIE (Superlinked Inference Engine) provides a single self‑hosted inference service that exposes an OpenAI‑compatible API for all common agent tasks. The repository bundles the full production stack:

Core server code lives in packages/sie_server/ and packages/sie_gateway/.
Model catalog definitions are under packages/sie_server/models/.
Language‑specific SDKs (TypeScript) are in packages/sie_ts_sdk/.
Example agents, Dockerfiles, and Helm charts are in examples/ and deploy/helm/.

The server can load any of the 100+ pre‑configured models on demand, evict them with LRU, and autoscale via KEDA. Integration adapters for LangChain, LlamaIndex, Haystack, DSPy, and others live in integrations/.

How It Is Wired

Entry points

examples/agent-action-monitor/mock-prod/app.py:run – starts a mock production server (143 functions reachable).
examples/contract-review-agent/contract_review_agent/cli.py:main – CLI for the contract‑review agent (399 functions reachable).
examples/sie-hugging-face-mteb-semantic-search/backend/app/main.py:dispatch – HTTP entry for the semantic‑search service (17 functions reachable).

Internal call flow (representative)

run → extract → _extract_single → PILImage.open   # filesystem read (image)
run → _make_config → from_env → _env_int          # env var handling
main → evaluate_run → read_text → hashlib.sha256 # crypto hash of markdown
api_search → search → _vqa_answer → httpx.post   # outbound network call

The most widely used functions are Item (called from 213 places) and extract (82 places). The hub module packages/sie_ts_sdk/src/client.ts is imported by 15 other modules, giving it a high blast radius.

External effects

Filesystem – 221 functions read/write files (e.g., loading model weights, reading config.yaml).
Network – 150 functions make outbound calls (e.g., httpx.post to external model APIs).
Crypto – 24 functions generate hashes (used for cache keys, integrity checks).
Database – 21 functions interact with a DB (examples use SQLite in wine-recommender).
Model inference – 3 functions call a model via the server’s inference API.

No circular import cycles were detected, which simplifies refactoring.

How To Use It

# 1. Clone the repo
git clone https://github.com/moses-y/sie
cd sie

# 2. Install the Python workspace (requires uv)
uv python install 3.12
uv lock --check
uv sync --frozen --all-packages

# 3. Run the server locally (CPU)
pip install "sie-server[local]"
sie-server serve   # listens on 0.0.0.0:8080

# 4. Or launch the Docker image (GPU example)
docker run --gpus all -p 8080:8080 \
  -v sie-hf-cache:/app/.cache/huggingface \
  ghcr.io/superlinked/sie-server:latest

Configuration files (examples/contract-review-agent/config.yaml, examples/agent-action-monitor/.env.example) define model selections, API keys, and KEDA autoscaling parameters. The Helm chart in deploy/helm/sie-cluster/ can be applied to a Kubernetes cluster after setting values-*.yaml for the target cloud provider.

Real‑World Use

A fintech firm can replace four separate model services (embedding, OCR, safety, and agent loop) with a single SIE deployment. Their existing LangChain pipelines import sie_ts_sdk and call /v1/chat/completions; the SIE gateway routes the request to the appropriate model, caches results, and scales pods automatically via KEDA, reducing operational overhead and latency.

Code Health & Issues

Measured findings (static analysis)

High – Add CI workflow – 842 source files, no .github/workflows present.
High – Build gate for packages/sie_config/Dockerfile – no automated image build.
High – Remove committed DB dump – examples/wine-recommender/wine_flavor.db leaks data.
Medium – Enable Dependabot/Renovate – 26 manifest files, no update bot.
Medium – Pin Docker base image – examples/agent-action-monitor/Dockerfile uses mutable python:3.12-slim.
Medium – Add pre‑commit secret scan – no secret‑gate configured.
Medium – Move large binaries to LFS – dutton_test.png (30 MB) and other >5 MB blobs.
Medium – Add timeout to outbound requests – network calls lack timeout=.
Low – Add repo convention files – missing .editorconfig, formatter config.

Additional hygiene

Tests exist (437 files) but are never run automatically.
License (Apache 2.0) and lockfiles (Cargo.lock, pnpm-lock.yaml) are present.
No obvious secret files were found, but the committed SQLite DB is a data‑leak risk.

The Bottom Line

SIE delivers a unified, production‑ready inference stack with solid Kubernetes tooling and language SDKs, making it a pragmatic choice for teams consolidating model serving. The codebase is large and contains several high‑impact hotspots (oversized SDK files, deep nesting) and lacks CI/CD automation, which must be addressed before adopting it in a regulated production environment. Use it if you need a single endpoint for diverse agent tasks and are prepared to add the missing CI, security gates, and binary management.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

1,847 files

Python814
YAML237
JSON158
Rust139
Markdown118
TypeScript101

ReactNext.jsFlaskExpressDockerKubernetesTransformersLangChainVector storeCUDA

Module dependencies

The 10 most depended-upon modules of 985, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_ckages_sie_ts_sdk_src_client["client"] n1_ckages_sie_ts_sdk_src_errors["errors"] n2_ackages_sie_ts_sdk_src_types["types"] n3_kages_sie_ts_sdk_src_msgpack["msgpack"] n4__ts_sdk_src_internal_parsing["parsing"] n5_mples_document_ocr_src_types["types"] n6_s_sie_ts_langchain_src_index["index"] n7__sie_ts_llamaindex_src_index["index"] n8_ontract_review_agent_runtime["runtime"] n9_ie_ts_sdk_src_internal_retry["retry"] n0_ckages_sie_ts_sdk_src_client --> n1_ckages_sie_ts_sdk_src_errors n0_ckages_sie_ts_sdk_src_client --> n4__ts_sdk_src_internal_parsing n0_ckages_sie_ts_sdk_src_client --> n9_ie_ts_sdk_src_internal_retry n0_ckages_sie_ts_sdk_src_client --> n3_kages_sie_ts_sdk_src_msgpack n0_ckages_sie_ts_sdk_src_client --> n2_ackages_sie_ts_sdk_src_types n4__ts_sdk_src_internal_parsing --> n1_ckages_sie_ts_sdk_src_errors n4__ts_sdk_src_internal_parsing --> n3_kages_sie_ts_sdk_src_msgpack n4__ts_sdk_src_internal_parsing --> n2_ackages_sie_ts_sdk_src_types n4__ts_sdk_src_internal_parsing --> n9_ie_ts_sdk_src_internal_retry

Findings

MediumNo CI/CD pipeline detected - no automated build/test gate .github/ or CI config

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high3

medium5

low1

Add a workflow that builds and tests this repositoryhigh

842 source files, no CI configuration

Every change merges with nobody having run the build once, which across this estate is 499 repositories.

Fix: Add a workflow running the project build and test command on push and pull_request.

Add a build gate for the deployable artifacts herehigh67 occurrences

packages/sie_config/Dockerfile

deployment artifacts with no workflow

Something in the repo is meant to reach a server and no automated step has ever validated it, which is the state of 125 repositories here.

Fix: Add a workflow that at least builds the image and validates the manifests.

Remove the committed database dumphigh

examples/wine-recommender/wine_flavor.db

A dump is the one artefact that leaks other people's data rather than the owner's, which turns a hygiene problem into a notification obligation.

Fix: Remove and purge from history; if it held personal data, assess it as a disclosure.

Enable Dependabot or Renovatemedium

26 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Pin the container base image by digestmedium

examples/agent-action-monitor/Dockerfile

python:3.12-slim

An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.

Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.

Add a pre-commit secret gatemedium

a secret-shaped file is tracked and no repo-level gate is visible

Without a gate the same class of leak recurs on the next commit, so a leak finding is the symptom and this is the cause.

Fix: Add a pre-commit hook scanning staged content, and enable push protection.

Move large binaries to Git LFS or out of the repositorymedium3 occurrences

examples/wine-recommender/test/test_images/dutton_test.png

3 blobs over 5MB: dutton_test.png 30.5MB, wine_flavor.db 21.0MB, hero.png 5.8MB

One repository here carries twenty blobs over 5MB including a 10.9MB spreadsheet, so every clone and every CI checkout pays for data nobody diffs.

Fix: Track those extensions with LFS, or move datasets to object storage and fetch them in a setup step.

Give the outbound request a timeoutmedium

examples/agent-action-monitor/Dockerfile

1 outbound call(s) with no timeout

requests has no default timeout, so a peer that accepts the connection and then says nothing holds the worker forever, and with a small worker pool a single slow dependency takes the whole service down without anything appearing to fail.

Fix: Pass timeout= on every call, or use a session with one configured.

Add the repository convention files this project lackslow3 occurrences

missing .editorconfig, .gitattributes, a formatter config

Without them one contributor's editor writes tabs into a Python file, a shell script commits with CRLF and fails in the container, and a notebook diff is unreviewable.

Fix: Add .editorconfig, .gitattributes with text=auto eol=lf, and a formatter config.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.