The Problem
Automation scripts that need to drive interactive terminal programs (vim, htop, ncurses installers) hit a dead‑end when only stdin/stdout are piped. Without a real PTY the UI cannot be rendered, and the script cannot read screen state or send keystrokes reliably.
What This Does
terminal-use (binary tu) creates a headless PTY backed by the vt100 emulator. A background daemon (src/daemon/) owns the PTY sessions and communicates over a Unix‑socket using JSON messages (src/daemon/protocol.rs). The CLI (src/main.rs) is thin; each sub‑command lives under src/commands/ (e.g., run.rs, press.rs, screenshot.rs).
Typical flow: tu run --name demo -- <app> spawns <app> inside a PTY, the daemon records screen buffers (src/render/screen.rs) and exposes them via tu screenshot, tu typetext, or tu press. Real‑time monitoring is provided by tu monitor, which renders the screen with ANSI colors using the built‑in renderer (src/render/mod.rs).
How To Use It
Setup
Pre‑built binary (Linux/macOS) curl -fsSL https://raw.githubusercontent.com/flipbit03/terminal-use/main/install.sh | sh
Build from source
cargo install terminal-use # pulls Cargo.toml, builds src/main.rs
The install script (install.sh) simply downloads the latest release and places tu on $PATH. No additional configuration files are required.
Running
Start an interactive session tu run --name nethack -- nethack
Send a keystroke
tu press --name nethack --key "a"
Capture the screen as PNG
tu screenshot --name nethack --output frame.png
Watch the session live
tu monitor --name nethack
All commands map to source files under src/commands/ (e.g., press.rs, screenshot.rs). The daemon auto‑starts on first use (src/daemon/manager.rs) and shuts down after 8 hours of inactivity, as documented in README.md.
Configuration
No external config files are required. Defaults are hard‑coded in src/paths.rs and src/versioncheck.rs (terminal size 120×40, TERM=xterm-256color). Environment variables can be passed through the PTY but are not managed by the tool itself.
Real‑World Use
An AI agent that solves a text‑based game can embed tu to interact with the game binary:
import subprocess, json, time
Launch the game inside a headless terminal
subprocess.run(["tu", "run", "--name", "game", "--", "./nethack"])
Loop: read screen, decide move, send keystroke
while True: out = subprocess.checkoutput(["tu", "screenshot", "--name", "game", "--json"]) screen = json.loads(out) move = decidemove(screen) # custom logic subprocess.run(["tu", "press", "--name", "game", "--key", move]) time.sleep(0.2)
The agent reads a JSON representation of the screen, decides on an action, and injects the keystroke—all without a graphical display.
Code Health & Issues
Medium – No test suite – Repository lacks tests/ directory or #[cfg(test)] modules; CI (.github/workflows/ci.yml) runs cargo build only, leaving runtime bugs unchecked. Low – Limited error handling – Commands such as src/commands/press.rs forward errors from the daemon but often unwrap results (expect) which could panic if the daemon is unreachable. Low – Potential race condition – Session manager (src/daemon/manager.rs) spawns daemon threads without explicit synchronization primitives; concurrent press and screenshot calls may read stale buffers. Low – Documentation gaps – README lists core commands but omits flag details (e.g., --json output format). Inline doc comments exist in some modules but not uniformly. None – License present – MIT license (LICENSE) is included, satisfying legal requirements. None – CI configured – GitHub Actions builds on Linux/macOS; however, no linting or security audit steps are defined.
The Bottom Line
terminal-use delivers a practical, low‑overhead solution for programmatic control of terminal UI applications, with a clean Rust codebase and a straightforward CLI. The lack of automated tests and modest error handling mean production deployments should include additional validation layers. It is well‑suited for AI agents, CI pipelines, or any automation that must interact with curses‑style programs without a graphical environment.