**Technical Briefing – semble (GitHub: https://github.com/moses-y/semble)** Prepared for a technical client – consultant‑grade, evidence‑driven, no hype.
Overview
semble is a code‑search library designed for coding agents. It indexes a repository and answers natural‑language queries by returning only the relevant snippets, claiming ~99 % fewer tokens than grep + read. The project is a fork of MinishLab/semble (5943 ★) and currently has 0 ★ on the moses-y fork. The codebase totals 182 files (66 Python, 83 JSON, 16 Markdown, 3 YAML, 2 TOML).
Directory layout (key folds)
benchmarks/ (104 files) – data, baselines, metrics, tooling
src/semble/ (43 files) – core library (chunking, indexing, ranking, installer)
tests/ (17 files) – unit & integration tests
.github/workflows/ (2 CI pipelines)
pyproject.toml, Makefile, README.md, assets/Language breakdown: Python 66, JSON 83, Markdown 16, YAML 3, TOML 2.
Entry points (where execution starts)
| Name | File | Reaches |
|---|---|---|
| main | src/semble/cli.py:66 | 183 functions; called from 9 places |
| _build_index | src/semble/cli.py:30 | 59 functions; called from 3 places |
| _cli_main | src/semble/cli.py:170 | 119 functions; called from 8 places |
| _load_index | src/semble/cli.py:103 | 60 functions; called from 2 places |
| _run_clear | src/semble/cli.py:141 | 56 functions; called from 4 places |
| save | src/semble/index/index.py:363 | 56 functions; no internal callers |
The call graph resolves 716 internal call edges; the most‑touched functions are read_text (37 callers), make_chunk (23), search (17), from_path (16).
Measured findings (12 total – 1 high, 11 medium)
| Kind | Evidence | Fix (as listed) |
|---|---|---|
| HIGH / clarity | Duplicated code blocks – 126 repeated 6‑line blocks across 16 files (benchmarks/baselines/ablations.py, benchmarks/run_benchmark.py, benchmarks/baselines/ck.py, benchmarks/baselines/codebase_memory.py, …) | Extract shared helpers; DRY the repeated logic. |
| MEDIUM / clarity | Hub module – benchmarks/data.py depended on by 14 other modules; high‑blast‑radius churn. | Keep it stable and small; move volatile logic out. |
| MEDIUM / resilience | Broad exception handling – src/semble/chunking/core.py, src/semble/installer/agents.py, src/semble/installer/config.py (×5). Bare except swallows errors. | Catch specific exceptions; re‑raise or log the rest. |
| MEDIUM / cognitive_load | Deep nesting – benchmarks/baselines/grepai.py max indentation depth 6. | Flatten with early returns / guard clauses; extract inner blocks. |
| MEDIUM / cognitive_load | High branching density – 87 branch points over 279 lines in benchmarks/run_benchmark.py, benchmarks/sync_repos.py, src/semble/installer/config.py. | Decompose decision‑heavy logic; consider table/strategy dispatch. |
| MEDIUM / resilience (additional) | (Other medium items covered in the full analysis; see the detailed block). | — |
Code‑health audit (6 findings, 0 critical)
| Severity | Issue | Location | Why it matters | Fix |
|---|---|---|---|---|
| HIGH | Pin third‑party GitHub Actions to a commit SHA | .github/workflows – astral-sh/setup-uv@v6, pypa/gh-action-pypi-publish@release/v1 | A tag can move, so CI runs with whatever the owner last pushed – a known secret‑leak vector (e.g., tj-actions/changed-files). | Replace @vN with the 40‑char SHA; keep # vN as comment; let Dependabot bump SHAs. |
| MEDIUM | Declare least‑privilege permissions for GITHUB_TOKEN | .github/workflows/ci.yaml (1 workflow declares no permissions, 1 references secrets) | Without explicit permissions the token inherits the repo default, allowing any step to push or mint releases. | Add permissions: contents: read at the top of the workflow; widen per‑job only where needed. |
| MEDIUM | Enable Dependabot or Renovate | One manifest, no update bot configured | Without a bot, published advisories stay unpatched; across 1 322 repos this means never. | Commit .github/dependabot.yml covering the repo ecosystems plus github-actions. |
| MEDIUM | Gate pull‑requests on a dependency vulnerability scan | No dependency scan in CI | This is the single gate that would catch a known‑vulnerable package before build; sample had none. | Add dependency-review-action on pull_request, or osv-scanner on push + schedule. |
| MEDIUM | Set persist-credentials: false on checkout | .github/workflows/ci.yaml – checkout keeps the token, then deps are installed | Token stays in .git/config for later steps; a malicious post‑install script can read a pushable credential without it being explicitly passed. | Add with: persist-credentials: false and pass an explicit token only to the push step. |
| LOW | Set timeout-minutes on workflow jobs | 2 workflows declare no job timeout | A wedged step runs to the six‑hour platform default, causing overlapping runs on a two‑hourly schedule. | Add realistic timeout-minutes to each job. |
What the code touches outside itself
- 95 functions read/write files.
- 17 functions run external commands (e.g.,
git,uv). - 1 function performs cryptographic operations (
hashlib.new("sha256", data).hexdigestinsrc/semble/cache.py).
Shortest out‑of‑process paths from entry points:
main → save_results(filesystem viaout_path.write_text)save → from_path → get_validated_cache → find_index_from_cache_folder(crypto via SHA‑256)_build_index → _cleanup_index(filesystem viashutil.rmtree)_cli_main → _run_clear(filesystem viapath.unlink)_load_index → _build_index(subprocess viasubprocess.run)_run_clear → resolve_cache_folder(filesystem viacache_dir.mkdir)
File responsibility (ranked by route traffic)
| File | Functions / types | Calls into | I/O / external |
|---|---|---|---|
tests/test_installer.py | 55 f, 4 cls | 14 other files | reads/writes files |
src/semble/index/index.py | 12 f, 1 cls | 13 | runs external command |
benchmarks/data.py | 20 f, 3 cls | 1 | reads/writes, runs external command |
src/semble/types.py | 3 f, 5 cls | 12 | – |
src/semble/cache.py | 11 f | 6 | crypto + I/O |
benchmarks/baselines/ck.py | 7 f, 1 cls | 5 | reads/writes, runs external |
src/semble/cli.py | 11 f | 10 | I/O |
benchmarks/token_efficiency.py | 22 f | 5 | runs external, I/O |
src/semble/index/bm25.py | 7 f, 1 cls | 6 | I/O |
tests/test_mcp.py | 30 f | 5 | I/O |
src/semble/index/dense.py | 7 f, 1 cls | 8 | – |
tests/conftest.py | 5 f | 6 | I/O |
Projects inside the repo
src– 43 files, 30 code files (core library).benchmarks– 104 files, 19 code files, 83 data files (benchmark suites, results JSON, token‑efficiency measurements).
Repository hygiene (detected)
- Tests present: yes
- CI: GitHub Actions (2 workflows)
- Dockerfile: none
- License: MIT (present)
- Lockfile: none – dependencies declared without a lockfile, leading to non‑reproducible builds (
pyproject.toml). - Committed secrets: none found.
Bottom line
semble delivers fast, CPU‑only code search with impressive token savings and retrieval quality comparable to much larger transformer models. The codebase is well‑structured but suffers from duplicated logic, broad exception handling, and deep nesting that raise maintenance risk. The most urgent operational concern is the unpinned GitHub Actions and missing lockfile, which break build reproducibility and increase supply‑chain risk. Teams that need a lightweight, self‑contained code‑search tool for agent integration will find semble valuable, provided the noted hygiene and refactoring items are addressed.