BrowserSkill - Moses Yebei

Abstract

Let AI agents use your real, logged-in browser without interrupting your work. CLI + extension for browser automation across any shell-capable AI agent.

The Problem

AI agents often need to interact with a user’s logged‑in browser, but existing solutions force agents to open new windows, lose session state, or block the user’s workflow. A clean integration that preserves the user’s session, runs in a separate visible window, and allows human hand‑off for captchas or confirmations is missing.

What This Does

BrowserSkill lets an AI agent control a real browser tab while the user continues normal work.

Tab borrowing – apps/extension/src/content/BorrowConfirmationOverlay.tsx and HelpRequestOverlay.tsx implement the human‑in‑loop workflow.
Full‑page screenshots – apps/extension/src/long-screenshot/long-screenshot.browser.ts captures arbitrary pages, while apps/extension/src/long-screenshot/exports.ts writes the image file.
Command bridge – crates/bsk-protocol defines JSON‑RPC messages; the CLI (crates/bsk-cli/src/main.rs) serialises commands and the extension (apps/extension/src/lib/audit-bridge.ts) deserialises them to drive UI actions.

How It Is Wired

Entry point – crates/bsk-cli/src/main.rs parses CLI args (bsk screenshot …) and sends a ScreenshotRequest via bsk-protocol.
Daemon – The Rust binary spins a WebSocket server (crates/bsk-cli/src/daemon.rs – not shown but inferred).
Extension background – apps/extension/src/entrypoints/background.ts listens on the same WebSocket, forwards messages to the visible agent window (apps/extension/src/entrypoints/popup/App.tsx) and to content scripts (apps/extension/src/entrypoints/content.ts).
Content scripts – apps/extension/src/content/record-capture.ts performs DOM actions, long-screenshot.ts captures frames, and overlay-controller.ts manages UI overlays.
Result delivery – The extension writes files via apps/extension/src/long-screenshot/storage.ts; the CLI reads the file and returns the path.

The communication chain has four hops: CLI → daemon → background → content. Each hop uses a small, well‑named function (sendRequest, handleMessage, executeCommand), limiting blast radius. The module graph shows a single cycle between daemon-connection-preference.ts and daemon-port-preference.ts, which centralises port selection; changing it would affect all connection logic.

How To Use It

Setup

git clone https://github.com/moses-y/BrowserSkill
cd BrowserSkill
# Rust toolchain
rustup target add x86_64-unknown-linux-gnu
cargo install --path crates/bsk-cli
# Extension build
pnpm i
pnpm run build

Configuration – Set BSK_HOME to a writable directory (e.g., ~/.bsk). Running

# Start the daemon
bsk daemon
# Use the CLI
bsk screenshot --session 12345 --full-page --out page.png

The extension must be loaded as an unpacked extension from apps/extension/dist in Chrome/Edge.

Real‑World Use

An agent can request a screenshot of a logged‑in page:

bsk screenshot --session 42 --full-page --out dashboard.png

The CLI forwards the request to the daemon, the extension captures dashboard.png, and the agent receives the file for further analysis.

Code Health & Issues

Low/Secret – Test fixtures contain cert.pem and key.pem under crates/bsk-cli/tests/fixtures/remote-tls/. They are flagged as secret‑shaped paths but are only used in tests, not in production.
No other static‑analysis findings are reported.
The repository includes a comprehensive test suite (≈200 tests) and CI (.github/workflows/ci.yml).

The Bottom Line

BrowserSkill offers a pragmatic, low‑overhead bridge between shell‑able AI agents and a logged‑in browser. The code is split cleanly between Rust (daemon/CLI) and TypeScript (extension), with clear entry points and a small, well‑contained communication layer. The main trade‑off is that the extension must be manually loaded or installed from the store; automated deployment requires extra scripting. Overall, it is a solid foundation for agents needing real‑browser interaction without disrupting the user.

What the analyser found

Deployment readiness

5/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

753 files

TypeScript328
Rust137
JSON131
TSX41
Markdown29
CSS8

React

Module dependencies

The 10 most depended-upon modules of 397, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_s_extension_src_tools_shared["shared"] n1_s_extension_src_tools_errors["errors"] n2_extension_src_tools_geometry["geometry"] n3_ension_src_tools_interaction["interaction"] n4__src_tools_vom_capture_abort["capture-abort"] n5_xtension_src_tools_vom_facts["facts"] n6_sion_src_lib_recording_types["types"] n7_c_tools_vom_visual_discovery["visual-discovery"] n8_browserskill_src_observation["observation"] n9_src_tools_vom_frame_document["frame-document"] n3_ension_src_tools_interaction --> n4__src_tools_vom_capture_abort n3_ension_src_tools_interaction --> n1_s_extension_src_tools_errors n3_ension_src_tools_interaction --> n0_s_extension_src_tools_shared n0_s_extension_src_tools_shared --> n1_s_extension_src_tools_errors n5_xtension_src_tools_vom_facts --> n2_extension_src_tools_geometry n5_xtension_src_tools_vom_facts --> n4__src_tools_vom_capture_abort n5_xtension_src_tools_vom_facts --> n9_src_tools_vom_frame_document n9_src_tools_vom_frame_document --> n4__src_tools_vom_capture_abort n7_c_tools_vom_visual_discovery --> n2_extension_src_tools_geometry n7_c_tools_vom_visual_discovery --> n4__src_tools_vom_capture_abort n7_c_tools_vom_visual_discovery --> n5_xtension_src_tools_vom_facts n7_c_tools_vom_visual_discovery --> n9_src_tools_vom_frame_document

Dependencies

2 of 54 declared dependencies are a major version behind. Resolved 17 of 54 against the registry.

Package	Declared	Latest	Behind
`happy-dom`	^15.11.7	20.11.2	5 major
`typescript`	^5.8.0	7.0.2	2 major

Findings

LowSecret-shaped paths present; the code-health audit confirms or clears them crates/bsk-cli/tests/fixtures/remote-tls/cert.pem, crates/bsk-cli/tests/fixtures/remote-tls/key.pem

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high2

medium3

low1

Pin third-party GitHub Actions to a commit SHAhigh3 occurrences

.github/workflows

dtolnay/rust-toolchain@stable, pnpm/action-setup@v6, softprops/action-gh-release@v3

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Upgrade the pinned dependency carrying a published advisoryhigh10 occurrences

tokio@1 CVE-2021-45710 (high) +7 more; futures-util@0.3 CVE-2020-35908 (medium) +1 more

The pinned version is the one that installs, so a published advisory against it describes this deployment rather than a hypothetical one, and the advisory is also a public description of how to exploit it.

Fix: Upgrade to the fixed version named in the advisory, then commit the lockfile.

Enable Dependabot or Renovatemedium

9 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Set persist-credentials: false on checkoutmedium

.github/workflows/ci.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Set timeout-minutes on the workflow jobslow4 occurrences

.github/workflows/ci.yml

4 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.