eshkol - Moses Yebei

The Problem

Researchers and engineers need a language that lets them write mathematical models with the same expressiveness as Lisp, but with native, zero‑overhead differentiation and high‑performance LLVM code. Existing tools either hide the differentiation logic behind libraries or impose run‑time penalties. Eshkol aims to expose calculus as a compiler primitive while keeping the familiar Scheme syntax.

What This Does

Eshkol is a Scheme‑based language that compiles to LLVM machine code and WebAssembly. The compiler, written in C++, implements forward‑ and reverse‑mode AD, vector calculus operators, and a deterministic execution model without a garbage collector. The runtime is a lightweight bytecode VM that runs in the browser via the web/ bindings. The language also ships a full documentation set and a large test suite that covers both the compiler and the VM.

Key files:

CMakeLists.txt – build system for the compiler and VM.
lib/lexer.cpp, lib/parser.cpp, lib/ast.cpp – front‑end.
lib/codegen.cpp, lib/llvm.cpp – backend.
web/web.esk – WebAssembly bindings.
tests/ – 2 883 test files exercising compiler correctness and VM parity.

How It Is Wired

Execution starts with cmake configuring the build. The CMakeLists.txt pulls in all src/ and lib/ sources, producing the eshkol executable. The entry point for the compiler is src/main.cpp (built by add_executable(eshkol src/main.cpp ...)). main.cpp parses the command line, invokes lexer::tokenize(), then parser::parse() to build an ast::Node, and finally calls codegen::generate() to emit LLVM IR. The emitted IR is passed to llvm::EngineBuilder for JIT or native code generation. For WebAssembly, web/ modules are compiled via the same codegen path with a WASM target.

The VM runs bytecode produced by the compiler. Its main loop is in vm/runtime.cpp, which dispatches opcodes defined in vm/ops.hpp. The AD logic lives in vm/ad.cpp, where dual numbers propagate through op::add, op::mul, etc. No external I/O occurs during compilation or execution; the only side effects are file reads for source files and writes for produced binaries or WASM modules.

The project graph has a single hub: the lib/ directory. Changes to lexer or parser ripple through the entire build because those modules are used by all downstream code. The VM modules form a separate, loosely coupled cycle that can be replaced with a different JIT engine without touching the compiler frontend.

How To Use It

Setup

git clone https://github.com/moses-y/eshkol
cd eshkol
mkdir build && cd build
cmake -S .. -B . -DCMAKE_BUILD_TYPE=Release
cmake --build .

The above produces bin/eshkol. For a Docker image that includes CUDA support, run

docker build -f docker/cuda/Dockerfile -t eshkol-cuda .

Configuration

No runtime configuration files are required. The compiler reads source files directly. For WebAssembly, the output is placed in web/ and can be served by any static HTTP server.

Running It

Compile a script:

./bin/eshkol scripts/linear_model.esk

Run the test suite:

ctest --output-on-failure

To build the documentation:

make docs

Real‑World Use

A data‑scientist can write a model in scripts/linear_model.esk and compile it to native code for deployment on servers or to WebAssembly for in‑browser inference. The compiler automatically differentiates loss functions, enabling gradient‑based training without manual derivatives.

Code Health & Issues

Low – tools/vscode-eshkol/package.json declares dependencies without a lockfile, making builds non‑reproducible.
Test coverage – 2 883 test files cover compiler correctness and VM parity.
Documentation – 526 doc files provide reference and usage guides.

No other static analysis findings were available.

The Bottom Line

Eshkol delivers a Lisp‑style language with built‑in, exact automatic differentiation and high‑performance LLVM output. The codebase is large but well‑structured around a single lib/ hub; changes to the lexer or parser affect the whole compiler. The build system is straightforward (CMake + Docker), and tests are comprehensive. The main drawback is the lack of a lockfile for the VS Code extension, which could lead to non‑deterministic builds. Teams needing a mathematically rigorous compiler for scientific or ML workloads, especially those targeting both native and WebAssembly, should consider Eshkol as a viable solution.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

4,825 files

Markdown517
YAML333
Shell276
C++264
Python200
C/C++ Header158

ReactExpressDockerpandasCUDA

Architecture

Top-level areas of the codebase, sized by module count. Arrows show how many imports cross from one area into another.

graph TD g0["scripts 139 modules"] g1["tests 69 modules"] g2["bench 6 modules"] g3["tools 4 modules"] g4["site 3 modules"] g5["examples 2 modules"] g6["web 2 modules"] g1 -->|6| g0

Module dependencies

The 10 most depended-upon modules of 226, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_ipts_lib_webgpu_test_browser["webgpu_test_browser"] n1_gpu_browser_environment_test["browser_environment_test"] n2_ripts_select_disclosure_base["select_disclosure_base"] n3_k_webgpu_release_environment["check_webgpu_release_env"] n4_ripts_lib_webgpu_diff_runner["webgpu_diff_runner"] n5_olchain_test_disclosure_base["test_disclosure_base"] n6__webgpu_division_parity_test["division_parity_test"] n7_gpu_shared_backend_race_test["shared_backend_race_test"] n8_ests_webgpu_webgpu_live_test["webgpu_live_test"] n9_bgpu_webgpu_regressions_test["webgpu_regressions_test"] n3_k_webgpu_release_environment --> n0_ipts_lib_webgpu_test_browser n4_ripts_lib_webgpu_diff_runner --> n0_ipts_lib_webgpu_test_browser n5_olchain_test_disclosure_base --> n2_ripts_select_disclosure_base n1_gpu_browser_environment_test --> n0_ipts_lib_webgpu_test_browser n6__webgpu_division_parity_test --> n0_ipts_lib_webgpu_test_browser n7_gpu_shared_backend_race_test --> n0_ipts_lib_webgpu_test_browser n8_ests_webgpu_webgpu_live_test --> n0_ipts_lib_webgpu_test_browser n9_bgpu_webgpu_regressions_test --> n1_gpu_browser_environment_test

Dependencies

3 of 4 declared dependencies are a major version behind. Resolved 4 of 4 against the registry.

Package	Declared	Latest	Behind
`@types/node`	^20.0.0	26.2.0	6 major
`typescript`	^5.0.0	7.0.2	2 major
`vscode-languageclient`	^9.0.1	10.1.2	1 major

Findings

LowDependencies declared without a lockfile - non-reproducible builds tools/vscode-eshkol/package.json

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high2

medium4

low2

Pin third-party GitHub Actions to a commit SHAhigh

.github/workflows

mymindstorm/setup-emsdk@v14

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Commit a lockfile beside the manifesthigh

tools/vscode-eshkol/package.json

manifest with no lockfile in scope

An unlocked range means the artifact you tested and the artifact you ship can contain different transitive code, so a malicious patch release reaches production with no diff.

Fix: Run the package manager once and commit the generated lockfile.

Enable Dependabot or Renovatemedium

1 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Pin the container base image by digestmedium

Dockerfile.test

ubuntu:22.04

An untagged or mutable base means today's build and last month's contain different libc and a different CVE set, with no record of which shipped.

Fix: Use image:tag@sha256:<digest> and enable Dependabot's docker ecosystem.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Set persist-credentials: false on checkoutmedium

.github/workflows/ci.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Set timeout-minutes on the workflow jobslow

.github/workflows/pages.yml

1 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Add the repository convention files this project lackslow3 occurrences

missing .editorconfig, .gitattributes, a formatter config

Without them one contributor's editor writes tabs into a Python file, a shell script commits with CRLF and fails in the container, and a notebook diff is unreviewable.

Fix: Add .editorconfig, .gitattributes with text=auto eol=lf, and a formatter config.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.