The Problem

Web users must repeatedly click, copy, and navigate to accomplish repetitive tasks. Doing this manually wastes time and is error‑prone, especially when the same sequence must be run across many pages or tabs.

What This Does

fuji-web ships as a Chrome/Edge side‑panel extension that receives a single natural‑language command, calls an LLM, and drives the page via Chrome extension APIs. The UI lives under src/pages/sidepanel/* and src/pages/popup/*, while the core automation logic is in src/helpers/ (e.g., dom-agent/, vision-agent/, aiSdkUtils.ts). Settings such as the OpenAI/Anthropic key are stored by src/common/Settings.tsx and persisted through src/shared/hooks/useStorage.tsx.

How It Is Wired

  1. Extension bootstrap – manifest.js loads src/pages/background/index.ts as the background service worker.
  2. Side‑panel UI – src/pages/sidepanel/index.tsx renders <App /> from src/common/App.tsx. This component pulls the current task from the Redux‑style store (src/state/store.ts).
  3. Task flow – When the user submits a prompt, src/common/RunTaskButton.tsx dispatches an action to src/state/currentTask.ts. The task handler calls src/helpers/aiSdkUtils.ts → fetch the LLM response, then routes the response to the appropriate agent (dom-agent/determineNextAction.ts or vision-agent/determineNextAction.ts).
  4. Page interaction – Agent modules use the RPC layer (src/helpers/rpc/pageRPC.ts) to invoke content‑script functions defined in src/pages/content/* (e.g., domOperations.ts, drawLabels.ts). These scripts run in the page context and manipulate the DOM, capture screenshots, or read text.
  5. State hub – src/state/store.ts is the most connected module (21 inbound imports, 4 outbound). It holds UI state, task queue, and settings, and is imported by most UI components and helpers.
  6. Circular dependencies – A cycle involves src/state/store.ts, src/helpers/aiSdkUtils.ts, and src/helpers/vision-agent/determineNextAction.ts. The cycle arises because the agents import the store to read settings while the store imports the agents for side‑effects. Breaking this cycle (e.g., extracting shared types into src/helpers/utils.ts) would reduce coupling.

Effect ownership

  • UI rendering → files under src/common/ and src/pages/*.
  • LLM calls → src/helpers/aiSdkUtils.ts.
  • DOM decision logic → src/helpers/dom-agent/* and src/helpers/vision-agent/*.
  • Persistence → src/shared/hooks/useStorage.tsx and src/shared/storages/*.

Only the background script and content scripts touch Chrome extension APIs; all other modules are pure TypeScript/React.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/fuji-web
cd fuji-web

# Install pnpm globally if needed
npm install -g pnpm

# Install dependencies
pnpm install

# Run a development server (hot‑reloads the extension)
pnpm dev
# Build a production zip (output in dist/)
pnpm build

Load the generated dist/ folder in Chrome via chrome://extensions → Load unpacked. The extension expects an environment variable VITE_OPENAI_KEY or VITE_ANTHROPIC_KEY when building; otherwise the UI prompts for the key and stores it locally via useStorage.

Real‑World Use

A SaaS platform that needs to extract order numbers from vendor portals can embed the built extension in a controlled Chrome instance. The platform’s backend triggers a task by sending a prompt like “Collect all pending orders from https://vendor.example.com”. Fuji‑Web’s LLM parses the page, navigates pagination via determineNextAction, and returns a JSON payload that the backend ingests via a custom RPC endpoint.

Code Health & Issues

  • High – Import cycles (9 modules) – e.g., src/state/store.ts, src/helpers/aiSdkUtils.ts, src/helpers/vision-agent/determineNextAction.ts. Break cycles to simplify future changes.
  • Medium – Hub module – src/state/store.ts has 21 dependents; any change has wide blast radius. Keep it stable, move volatile logic elsewhere.
  • High – Deep nesting – src/common/CustomKnowledgeBase/NewKnowledgeForm.tsx reaches 10 levels of indentation; refactor into smaller components.
  • High – Duplicated code – Similar 6‑line blocks appear in multiple knowledge‑base and agent files; extract shared helpers.
  • Medium – High branching density – src/helpers/dom-agent/parseResponse.ts and src/helpers/simplifyDom.ts contain >30 branches in ~100 lines; consider strategy pattern to improve readability.

The repo includes unit tests (test-utils/jest.setup.js), CI via GitHub Actions, a LICENSE, and a lockfile (pnpm-lock.yaml). No Dockerfile or secret leakage detected.

The Bottom Line

Fuji‑Web provides a functional, extensible browser‑side AI automation framework with a clear separation between UI, state, and agent logic. However, the central store and several circular imports create maintenance risk, and some files suffer from deep nesting and duplicated snippets. It is suitable for teams comfortable with React/TypeScript who need a customizable in‑browser automation layer and are prepared to refactor the identified hotspots.