The Problem

Developers need a fast way to locate bug patterns, insecure code, or policy violations across many languages without writing complex regular expressions or custom AST walkers. Existing static analysers are either heavyweight, require proprietary rule sets, or lack easy integration into CI pipelines.

What This Does

The repository ships a Python‑based CLI (cli/src/semgrep/main.py) that parses user‑supplied rules written as code snippets and scans source trees in ≈ 30 languages. The core engine lives in cli/src/semgrep/engine.py (imported by core_runner.py) and produces SARIF, JSON, JUnit or plain‑text reports (formatter/ package).

Configuration files such as cli/pyproject.toml and cli/setup.py define the package metadata and its dependencies, while the Dockerfile builds a reproducible container image for CI use. The mcp/server.py module implements the remote‑scan service used by the Semgrep AppSec Platform, exposing semgrep_scan* entry points.

How It Is Wired

Execution begins at cli/src/semgrep/main.py:133 in the main function. main parses CLI arguments, builds a TargetManager (via target_manager.py), and calls run_semgrep (in core_runner.py).

  • run_semgrep → run_semgrep_in_tmp (134 callers) → exec_osemgrep (external subprocess) – this is the only outbound process launch.
  • File I/O paths: validate_local_files reads rule files (Path.read_text), write_pipes_to_disk writes temporary scan files, and create_temp_files_from_code_content creates per‑target directories.
  • Network calls appear in meta.py (get_url_from_sstp_url, event_name) and config_resolver.py (_download_config_from_url).

The internal call graph shows 3 648 resolved edges; the most fan‑out function is assert_match (197 callers). No circular import clusters were detected, so the module hierarchy is flat and easy to reason about.

External effects are concentrated in a few modules:

  • cli/src/semgrep/meta.py – reads/writes files and makes an outbound HTTP request.
  • cli/src/semgrep/config_resolver.py – performs a cryptographic hash and file download.
  • cli/src/semgrep/util.py – runs subprocesses (subprocess.run).

Thus a typical scan touches the filesystem (reading rules, writing temporary files), launches a short‑lived subprocess, and may fetch remote rule packs over HTTPS.

How To Use It

# Clone the fork
git clone https://github.com/moses-y/semgrep
cd semgrep

# Install the CLI in an isolated environment
python -m venv .venv
source .venv/bin/activate
pip install -e cli/          # uses cli/setup.py / pyproject.toml

# Run a quick scan against a codebase
python -m semgrep scan --config=auto path/to/project

The Docker image can be built with docker build -t semgrep . and run similarly. Configuration such as GITHUB_TOKEN is read by the mcp/server.py workflow; the token permissions are declared in the GitHub Actions files (.github/workflows/*.yml).

Real‑World Use

In a CI pipeline, add a step that executes the above semgrep scan command, then upload the SARIF file as a CodeQL‑compatible artifact. The scan runs in seconds, producing a machine‑readable report that can be gated by a pull‑request policy.

- name: Semgrep Scan
  run: python -m semgrep scan --config=p/ci --output=semgrep.sarif --sarif
- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: semgrep.sarif

Code Health & Issues

Measured findings (9 × MEDIUM)

  • resource_safety: open(...) without a context manager (e.g., tests/patterns/python/metavar_equality_var.py).
  • clarity: duplicated 6‑line blocks across several Ruby test files.
  • resilience: broad except: clauses swallowing all errors (e.g., tests/patterns/python/deep_try.py).

Code‑health audit (10 items)

  • HIGH – Workflow test-e2e-semgrep-ci.yml pushes directly to the default branch.
  • HIGH – perf/config.py builds a shell string and calls os.system.
  • HIGH – cli/src/semgrep/settings.py uses unsafe yaml.load.
  • MEDIUM – No least‑privilege GITHUB_TOKEN scope in build-test-core-x86.yml.
  • MEDIUM – No Dependabot configuration for the many manifests present.
  • MEDIUM – Dockerfile uses mutable tags (alpine:3.23).
  • MEDIUM – No dependency‑vulnerability gate in CI workflows.
  • MEDIUM – Large blobs (>5 MiB) stored directly in the repo (tests/snapshots/.../stdout).
  • MEDIUM – actions/checkout retains credentials; missing persist-credentials: false.
  • LOW – Missing timeout-minutes on several workflow jobs.

All issues are documented with file paths; the fixes are straightforward (use yaml.safe_load, pin Docker digests, add permissions: blocks, enable Dependabot, etc.).

The Bottom Line

Semgrep’s CLI is a well‑structured, language‑agnostic static‑analysis engine that can be dropped into any CI pipeline with minimal setup. The code base is healthy overall but contains several high‑severity security hygiene problems (unsafe YAML loading, shell‑string execution, and CI‑push practices) that should be addressed before production use. Engineers comfortable with Python packaging can safely extend the rule set or embed the engine in custom tooling.