The Problem
Autonomous LLM agents struggle with tool discovery—knowing which available tools are relevant for a given task. Without active discovery mechanisms, agents either guess blindly or rely on static tool lists that quickly become outdated. This is particularly acute in MCP (Model Context Protocol) ecosystems where hundreds of servers and thousands of tools are registered but not actively indexed for relevance.
What This Does
MCP-Zero implements active tool discovery for autonomous LLM agents. The core architecture lives in MCP-zero/ with key components: matcher.py — similarity matching to identify relevant tools from the MCP catalog sampler.py — selects target tools based on query context reformatter.py — formats tool descriptions into JSON for LLM consumption utils.py — grid search utilities for parameter tuning
The experiment scripts test two domains: experimentapibank.py — APIBank experiments experimentmcptools.py — MCP tools needle tests
The dataset pipeline in MCP-tools/builddata/ extracts server summaries and tool metadata. getserversummary.py reads MCP server READMEs and uses VLLM (Qwen2.5-72B-Instruct) to generate structured summaries and embeddings. The prompt templates in promptguide/ define the system prompts for extraction.
The repository provides a test harness in testmatcher.py with testcases.jsonl to validate matching behavior against known cases.
How To Use It
Setup: Install dependencies from requirements.txt. The file lists Python packages but no lockfile is present, so pin versions manually if reproducibility is required.
Configuration: No explicit configuration files exist beyond the prompt templates in MCP-zero/promptguide/. The MCP-tools/builddata/getserversummary.py expects MCP server READMEs to be available locally; it extracts server names, descriptions, and summaries using the VLLM deployment configured in runvllm.sh.
Running it:
Example entry point — actual CLI not explicitly defined in repo python -M MCPzero.experimentmcptools
Or run matcher directly
python -M MCPzero.matcher
The README notes that dynamic MCP server deployment, environment setup for GAIA testing, and industrial deployment require additional infrastructure not yet open-sourced.
Real-World Use
An autonomous agent receiving a query like "book a dinner reservation" can use MCP-Zero's matcher to scan the 308-server, 2,797-tool dataset and surface the most relevant tools (e.g., OpenTable, Resy) based on semantic similarity. The matcher.py computes embeddings against the precomputed descriptionembedding and summaryembedding fields in the MCP-tools dataset, returning ranked candidates the LLM can then invoke. The reformatter.py ensures tool descriptions are formatted consistently for the LLM's function-calling schema.
A minimal workflow might look like:
from MCPzero import matcher, reformatter
Load precomputed dataset
with open("MCP-tools/mcptoolswithembedding.json") as f: dataset = json.load(f)
Match user query
results = matcher.match("book dinner", dataset, topk=5)
Format for LLM
formatted = reformatter.format(results)
Code Health & Issues
[Medium/SDLC] No CI/CD pipeline — no .github/ workflows or automated test gate. Manual verification only. [Low/Risk] Dependencies declared without a lockfile — requirements.txt lists pins but no requirements.lock or similar. Non-reproducible builds across environments. [Low/Risk] No input validation in matcher — matcher.py performs similarity matching but does not validate that the dataset file exists or that topk is within bounds. Could panic on malformed input. [Low/Risk] Missing unit test coverage — test_matcher.py exists but exercises only a narrow path. No property-based or edge-case testing visible. [Info] Industrial features incomplete — README explicitly states dynamic MCP server deployment, GAIA environment setup, and production-grade deployment are pending. This is a research prototype, not a production tool.
The Bottom Line
MCP-Zero is a functional research prototype that solves the active tool discovery problem for LLM agents with a working matcher, sampler, and dataset pipeline. The code is clean for its scope, the matcher logic is straightforward cosine-similarity over precomputed embeddings, and the experiment scripts provide immediate reproducibility for the paper's results. However, it lacks CI, a lockfile, robust error handling, and the production-facing pieces (dynamic server deployment, GAIA integration) are deliberately deferred. Use this if you need a working tool-discovery demo for evaluation or prototyping; do not use it in a production pipeline without adding validation, reproducibility gates, and the missing deployment infrastructure.