The Problem

Applications that need low‑latency similarity search often spin up separate vector‑search services (e.g., Milvus, Vespa) or embed heavyweight libraries. That adds deployment complexity, network overhead, and resource waste for workloads that run in a single process such as notebooks, edge services, or micro‑services.

What This Does

zvec provides an in‑process vector database built on Alibaba’s Proxima engine. The core C++ implementation lives under src/ailego/ (e.g., src/ailego/algorithm/kmeans.h, src/ailego/math/*). Python bindings are generated in src/binding/python/ (binding.cc) and exposed through the python/zvec/ package. High‑level Python APIs such as zvec.CollectionSchema, zvec.create_and_open, and zvec.Doc are defined in python/zvec/model/ and python/zvec/extension/. The repository ships with a full test suite (python/tests/) and CI pipelines (.github/workflows/) that compile the native library and run the Python tests.

How It Is Wired

Execution starts in Python code that imports zvec. The import chain is:

  1. python/zvec/__init__.py → re‑exports symbols from python/zvec/model/ and python/zvec/extension/.
  2. Creating a collection (zvec.create_and_open) calls python/zvec/model/collection.py, which constructs a Collection object.
  3. collection.py invokes the C++ binding via the generated module zvec._core (exported from src/binding/python/binding.cc).
  4. The binding forwards calls to the native library entry points in src/ailego/ (e.g., src/ailego/io/file.cc for persistence, src/ailego/algorithm/* for indexing, and src/ailego/math/* for distance computation).
  5. Results propagate back through the binding to the Python wrapper, which returns native Python objects (zvec.Doc, zvec.QueryResult).

The import graph (56 internal modules, 86 edges) shows no circular dependencies. The most‑connected Python module is python/zvec/extension/__init__ (imports 12 others, referenced by 3), indicating it is the hub for extension functions such as bm25_embedding_function.py and openai_embedding_function.py. The python/zvec/model/doc.py module is heavily imported (9 inbound edges) but only imports one other module, making it a low‑instability leaf.

All filesystem interaction is confined to src/ailego/io/file.cc and src/ailego/io/file_lock.cc. No network code is present, confirming the “in‑process only” claim. The C++ side is compiled via the top‑level CMakeLists.txt, which pulls in src/ailego/CMakeLists.txt and src/binding/CMakeLists.txt.

How To Use It

# Clone and enter the repo
git clone https://github.com/moses-y/zvec
cd zvec

# Build the native library (requires cmake & a C++17 compiler)
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build . --parallel

# Install the Python package from the source tree
cd ..                # back to repo root
pip install .        # reads pyproject.toml, builds the extension

A minimal Python script (mirroring the README) is:

import zvec

schema = zvec.CollectionSchema(
    name="demo",
    vectors=zvec.VectorSchema("emb", zvec.DataType.VECTOR_FP32, 4),
)

col = zvec.create_and_open("./demo_db", schema)
col.insert([zvec.Doc(id="d1", vectors={"emb": [0.1, 0.2, 0.3, 0.4]})])
print(col.search(zvec.Query(vectors={"emb": [0.1, 0.2, 0.3, 0.4]}, top_k=1)))

Run the script with python demo.py. The example under examples/c++/db/main.cc demonstrates the same flow in C++.

Real‑World Use

A recommendation engine can embed zvec directly into its inference service. After loading a pre‑computed embedding matrix into a Collection, each request calls collection.search() to retrieve the nearest neighbours in < 5 ms, eliminating HTTP round‑trips to an external vector store.

Code Health & Issues

  • High – lockfile missing – pyproject.toml declares dependencies but no lockfile (e.g., poetry.lock).
  • Medium – Dependabot not configured – only one manifest (pyproject.toml).
  • Medium – Docker base image not pinned – .github/workflows/docker/Dockerfile.linux_x64_glibc228 uses ubuntu:18.10.
  • Medium – No dependency‑vulnerability scan – CI lacks dependency-review-action or osv-scanner.
  • Medium – Checkout persists token – build_test_wheel.yml should set persist-credentials: false.
  • Low – Workflow jobs lack timeout – four CI jobs have no timeout-minutes.

Additional findings from static analysis: python/zvec/model/collection.py opens files without a context manager, and several C++ files (examples/c++/db/main.cc, src/ailego/algorithm/integer_quantizer.cc, src/ailego/algorithm/kmeans.h) exhibit deep nesting (up to 10 levels), increasing cognitive load.

The Bottom Line

zvec delivers a fast, in‑process vector store with a clean Python API backed by a well‑structured C++ core. It is ready for integration into latency‑sensitive services, but production use should address the missing lockfile, Docker image pinning, and CI hardening to reduce supply‑chain risk. Engineers comfortable with CMake and Python extensions will find the codebase approachable and extensible.