The Problem

Developers building Rust services that need to talk to many generative‑AI providers (OpenAI, Anthropic, Gemini, Ollama, Groq, etc.) must write and maintain separate client code for each API, handle divergent auth schemes, and keep streaming logic in sync. The library in this repo aims to eliminate that duplication by offering a single, ergonomic Rust façade over dozens of providers.

What This Does

genai implements a unified Chat and Embedding API that abstracts provider‑specific request/response formats. The core lives in src/lib.rs and re‑exports the public types. Provider adapters live under src/adapter/adapters/<provider>/adapter_impl.rs (e.g., src/adapter/adapters/openai/adapter_impl.rs, src/adapter/adapters/anthropic/adapter_impl.rs).

Key files:

  • src/client/client_impl.rs – resolves a model name, builds the HTTP request, and dispatches to the appropriate adapter.
  • src/chat/chat_request.rs / src/chat/chat_response.rs – define the normalized request/response structures used by all callers.
  • src/adapter/dispatcher.rs – runtime selector that maps a ModelIden to its adapter implementation.
  • examples/ – 18 short programs that show basic chat, authentication, tool use, image handling, and custom endpoint resolution.

The library therefore lets a consumer write code like:

use genai::{Client, ChatOptions};

let client = Client::default();               // picks default auth from env
let opts = ChatOptions::default().with_temperature(0.7);
let resp = client.exec_chat("gpt-4o-mini", "Hello", opts).await?;

No provider‑specific code is required beyond the model identifier.

How It Is Wired

Execution starts in the example binaries (e.g., examples/c00-readme.rs:52 → main). main builds a Client (src/client/client_impl.rs) and calls exec_chat.

  • exec_chat (13 calls) resolves the model via resolve_model_iden (calls all_model_names → list_model_names_for_end_target) and then invokes the selected adapter’s exec_chat implementation.
  • The adapter implementation (e.g., src/adapter/adapters/openai/adapter_impl.rs) builds a provider‑specific request, hands it to reqwest (via src/webc/web_client.rs), and streams the response back through a unified EventSourceStream.
  • The streaming engine (src/adapter/inter_stream.rs and src/webc/event_source_stream.rs) normalizes chunks into the library’s ChatResponse type.

The most‑used internal symbols are is_empty (38 callers), into_iter (35 callers), and exec_chat (27 callers). The hub files with the widest blast radius are:

  • src/chat/message_content.rs – 71 functions, referenced from 22 files.
  • src/chat/chat_options.rs – 58 functions, referenced from 22 files.
  • src/adapter/adapter_types.rs – central type definitions used by 9 files and calling a model for inference.

No cycle was detected in the import graph, but several adapters contain deep nesting (up to 14 indentation levels) that makes the control flow hard to follow.

How To Use It

# Clone the repository
git clone https://github.com/moses-y/rust-genai
cd rust-genai

# Build and run the basic example (requires a valid API key in the environment)
cargo run --example c00-readme
  • Configuration – The library reads auth tokens from environment variables (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.) as shown in src/resolver/auth_resolver.rs. Custom endpoints can be supplied via ServiceTargetResolver (see examples/c06-target-resolver.rs).
  • Running tests – cargo test executes 53 test files, including provider‑specific integration tests under tests/.
  • Adding a new provider – Create a new adapter module under src/adapter/adapters/<name>/, implement the Adapter trait, and register it in src/adapter/mod.rs.

Real‑World Use

A microservice that summarizes user‑generated text could instantiate a single Client, switch between gpt-4o-mini for fast cheap calls and claude-3-5-sonnet for higher‑quality output, and keep the same request construction code. The unified streaming response lets the service forward partial results to a WebSocket without provider‑specific handling.

Code Health & Issues

  • High – Pin GitHub Actions – .github/workflows/* uses tag references (e.g., dtolnay/rust-toolchain@stable). Replace with commit SHAs.
  • High – Vulnerable dependencies – tokio@1 (CVE‑2021‑45710), bytes@1.6 (CVE‑2026‑25541), serde_with@3 (GHSA‑7gcf‑g7xr‑8hxj). Upgrade to patched versions and commit a lockfile.
  • High – Missing Cargo.lock – No lockfile means non‑reproducible builds; run cargo generate-lockfile and commit Cargo.lock.
  • Medium – Unrestricted GITHUB_TOKEN – .github/workflows/yakbak-replay-tests.yml lacks explicit permissions; add contents: read.
  • Medium – No automated dependency updates – Add a dependabot.yml covering Rust crates and GitHub Actions.
  • Medium – No vulnerability scan in CI – Insert dependency-review-action or osv-scanner into the workflow.
  • Low – No job timeouts – Add timeout-minutes to workflow jobs to avoid runaway runs.

Additional observations: the repository contains a full test suite and documentation (doc/for-llm/api-reference-for-llm.md), but the deep nesting and duplicated example code increase maintenance cost.

The Bottom Line

rust-genai delivers a practical, single‑point API for many LLM providers, reducing boilerplate for Rust services. The codebase is functional and well‑tested but suffers from high‑severity dependency and CI hygiene issues, and some adapters are overly complex. It is suitable for teams comfortable with Rust who need multi‑provider support and can allocate effort to address the highlighted health concerns.