The Problem

Gamers who play non‑English titles often have to pause or switch to external translation tools, breaking immersion and slowing progress. The pain point is a lack of an in‑game, low‑latency solution that can capture screen text, run OCR, and feed the result to an LLM‑based translator without leaving the game window.

What This Does

RSTGameTranslation is a Windows‑only WPF application that captures a user‑selected window region, runs OCR (OneOCR, Windows OCR, EasyOCR, PaddleOCR, or RapidOCR) and passes the extracted text to a configurable translation backend (Google, Gemini, Groq, ChatGPT, Ollama, Mistral, LM Studio, etc.).

Key files:

  • src/App.xaml.cs – boots the WPF app and registers services.
  • src/MainWindow.xaml.cs – UI entry point; handles hot‑key shortcuts, starts OCR, and displays overlay windows.
  • src/OcrServerManager.cs – launches the selected Python OCR server (app/webserver/*/server_*.py) as a child process and communicates via HTTP.
  • src/TranslationServiceFactory.cs & src/ITranslationService.cs – resolve the concrete translator (e.g., src/GoogleTranslateService.cs, src/GeminiTranslationService.cs).
  • src/ConfigManager.cs – persists user selections (OCR engine, language pairs, API keys) in JSON files under app/Languages/.

The Python side (app/webserver/EasyOCR/process_image_easyocr.py, .../PaddleOCR/*.py, .../RapidOCR/*.py) each expose a tiny Flask‑style HTTP endpoint that receives an image, runs the respective OCR library, and returns plain text.

How It Is Wired

Execution begins in the compiled binary generated from RST.csproj; the WPF runtime loads src/App.xaml.cs, which creates the DI container and opens src/MainWindow.xaml. When the user presses Alt + Q the MainWindow calls OcrServerManager.StartServer() (see src/OcrServerManager.cs). This method selects the configured OCR type and launches the matching script, e.g.:

app/webserver/EasyOCR/server_easy.py   ← started via Process.Start

The server listens on a hard‑coded local port; MainWindow then captures the selected screen rectangle (via src/WindowSelectorPopup.xaml.cs) and sends the bitmap to the server using HttpClient (inside src/OneOCRManager.cs). The server returns raw text, which MainWindow forwards to TranslationServiceFactory.Create() to obtain an ITranslationService implementation. The chosen service (e.g., src/GoogleTranslateService.cs) performs an HTTP request to the external API, receives the translated string, and the UI displays it in the overlay (src/OverlayOptionsWindow.xaml.cs).

All side‑effects (file writes, network calls) are confined to:

  • src/ConfigManager.cs – reads/writes JSON config.
  • src/*TranslationService.cs – external HTTP APIs.
  • src/OcrServerManager.cs – spawns/terminates Python processes.

No circular imports were detected; each module is isolated, which limits blast radius but also means duplicated logic across the OCR scripts (see measured findings).

How To Use It

  1. Clone and build
git clone https://github.com/moses-y/RSTGameTranslation
cd RSTGameTranslation
dotnet build RST.sln -c Release
  1. Install OCR back‑ends (optional for non‑OneOCR). Run the provided batch files, e.g.:
app\webserver\EasyOCR\SetupServerCondaEnvNVidiaEasyOCR.bat

These scripts create a Conda environment with the required Python packages and place the server executable in the same folder.

  1. Configure - Edit app/Languages/*.json to add or modify language codes. - Place API keys in the UI under Settings → Translation (the app stores them encrypted via src/ConfigManager.cs).
  1. Run
dotnet run --project RST.csproj   # or execute the built rst.exe

Press Alt + Q to select the game window, Alt + F to toggle the overlay, and Alt + G to start/stop translation.

Real‑World Use

A streamer runs rst.exe alongside Elden Ring (windowed mode). The overlay shows translated dialogue in real time, letting the audience follow the story without subtitles. The streamer configures PaddleOCR for Japanese text and Gemini for translation, achieving sub‑second latency thanks to the local GPU‑accelerated OCR server.

Code Health & Issues

  • HIGH – Pin third‑party GitHub Actions to commit SHA (.github/workflows/*).
  • HIGH – No test suite; 55 source files lack automated verification.
  • HIGH – Workflow pushes directly to default branch (update-readme-version.yml).
  • MEDIUM – Large binaries (app/AudioModel/ggml-tiny.bin, oneocr.onemodel, oneocr.dll) should be stored with Git LFS or external storage.
  • LOW – Add job timeouts in CI (release.yml).
  • LOW – Add convention files (.editorconfig, .gitattributes, formatter config).

Measured static analysis also reports 48 instances of deep nesting, duplicated OCR‑server code, and 11 oversized C# files (e.g., src/ConfigManager.cs, src/Logic.cs, src/MainWindow.xaml.cs). Refactoring these would improve readability and reduce change impact.

The Bottom Line

RSTGameTranslation delivers a functional, extensible pipeline for real‑time game text translation on Windows, with clear separation between UI, OCR servers, and translation services. However, the codebase suffers from duplication, deep nesting, and a lack of automated tests, which raises maintenance risk for any custom extensions. It is suitable for teams comfortable with C# and Python who need a quick, locally‑run translation overlay and are prepared to address the identified health concerns.