The Problem
Voice assistants suck at context. Siri or Alexa can't see what you're looking at, can't handle your shopping list, and definitely can't send a WhatsApp message while you're walking around. If you've ever wished your smart glasses could actually act smart, VisionClaw is the fix.
What This Does
VisionClaw wires Meta Ray-Ban smart glasses to real-time AI via the Gemini Live API. The samples/CameraAccess/CameraAccess.xcodeproj is the iOS app—think camera stream + mic audio piped straight to Gemini, so the assistant knows what you see and hear. If you want actual actions (not just talking), plug in OpenClaw. That’s a local gateway (openclaw.json config) that exposes 56+ tools: messaging, web search, smart home, reminders.
Setup is dead simple: clone, drop your Gemini API key into GeminiConfig.swift, and run. Want to test? Use your iPhone camera instead (no glasses needed). The pipeline is all here—audio, video, WebSocket to Gemini, and optional tool calls routed through OpenClaw.
Real-World Use
Say you’re in the kitchen, wearing your Ray-Bans. You tap the AI button and mumble, “Add milk to my shopping list.” The camera snaps a frame, the mic grabs your voice, everything zips to Gemini Live. Gemini figures out you want to update your shopping list, triggers OpenClaw, which then hits your app (maybe Todoist or Apple Reminders). You get confirmation spoken back—no hands, no phone, just glasses.
Here's the setup in code:
// samples/CameraAccess/CameraAccess/Gemini/GeminiConfig.swift static let apiKey = "YOURGEMINIAPI_KEY"
The Bottom Line
VisionClaw finally makes smart glasses actually useful, as long as you’re comfortable fiddling with API keys and local gateways. The iOS app is straightforward, the OpenClaw integration is powerful but a bit much if you only want basic voice/vision. If you want real agentic AI—actions, not just answers—this is worth your time. If you’re just after “describe what I’m seeing,” stick with Gemini alone.