The Problem

Teams need a single, reproducible view of what happened on fast‑moving social platforms over the last 30 days. Pulling Reddit threads, X posts, YouTube transcripts, Hacker News items, Polymarket odds and the open web requires many APIs, token handling, and custom ranking logic – all of which are scattered across scripts and ad‑hoc CLI calls.

What This Does

The last30days skill orchestrates parallel scrapers, scores each item against real‑world engagement signals, and hands the curated set to an LLM for a grounded brief. Core orchestration lives in skills/last30days/scripts/lib/pipeline.py (the run function) and skills/last30days/scripts/briefing.py (the main function).

  • Scrapers are in skills/last30days/scripts/lib/ – e.g. reddit.py, github.py, instagram.py, youtube_yt.py, polymarket.py. Each defines a search_<source> entry point that ultimately calls the shared logger _log (used from 65 places).
  • Relevance and query handling are centralized in lib/relevance.py and lib/query.py; 43 modules import lib/__init__.py, making it the highest‑impact hub.
  • Results are rendered by lib/render.py (HTML/markdown) and stored via store.py (SQLite DB) before being fed to the LLM in briefing.py.

How It Is Wired

  1. Entry point – run in pipeline.py:175 is invoked by the CLI (last30days.py calls run_last30days). It spawns the full pipeline, reaching 331 distinct functions.
  2. Source discovery – available_sources (in pipeline.py) calls each search_<source> (e.g. search_reddit, search_github_person). Those functions log via _log and make outbound HTTP calls through lib/http.py (10 functions, used by 22 callers).
  3. Data persistence – store.py provides _connect/init_db (called from 10 places) to open the SQLite DB; add_topic writes results.
  4. Scoring & fusion – lib/fusion.py merges raw items, applying relevance weights (token_overlap_relevance, jaccard). This file is one of the “deep‑nesting” hotspots (max indentation depth 8).
  5. Rendering – lib/render.py (62 functions) converts the ranked list to markdown or HTML; it reads template files and writes the final brief.
  6. LLM synthesis – briefing.py extracts core subjects (_extract_core_subject) and calls the provider in lib/providers.py (generate_text). The provider reaches out to the configured LLM endpoint (network call).

Blast radius:

  • lib/__init__.py, lib/relevance.py, lib/query.py – each imported by >15 modules; any change ripples widely.
  • lib/fusion.py and lib/github.py have high branching density (66 branches/211 lines) and deep nesting, making them error‑prone.
  • Broad except: blocks appear in chrome_cookies.py, env.py, last30days.py – they swallow errors that could hide integration failures.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/last30days-skill
cd last30days-skill

# Install Python dependencies (pyproject.toml)
python -m venv .venv
source .venv/bin/activate
pip install -e .   # editable install pulls all runtime deps

# Install the bundled Node helper (bird‑search)
cd skills/last30days/scripts/lib/vendor/bird-search
npm ci            # installs from package.json
cd ../../../..    # back to repo root

# Provide API credentials (example keys, see docs/CONFIGURATION.md)
#   REDDIT_CLIENT_ID, REDDIT_CLIENT_SECRET, X_BEARER_TOKEN, YOUTUBE_API_KEY, etc.
# Credentials are read by `lib/env.py` from a `.env` file or system env.

# Run the skill (full pipeline)
python -m skills.last30days.scripts.last30days   # invokes main → run
# Or generate a brief directly:
python -m skills.last30days.scripts.briefing "search term"

If any source should be disabled, edit available_sources in pipeline.py or set the corresponding env flag (ENABLE_X=0, etc.).

Real‑World Use

A sales engineer can embed the skill in a CI step:

- name: Generate 30‑day brief
  run: |
    source .env
    python -m skills.last30days.scripts.briefing "Acme Corp"
  env:
    REDDIT_CLIENT_ID: ${{ secrets.REDDIT_CLIENT_ID }}
    X_BEARER_TOKEN: ${{ secrets.X_BEARER_TOKEN }}

The generated markdown can be attached to a meeting agenda, ensuring the team sees the latest community sentiment across all platforms.

Code Health & Issues

  • High – Pin GitHub Actions – .github/workflows/* uses tags (softprops/action-gh-release@v2). Replace with commit SHA to avoid supply‑chain drift.
  • High – Remove continue-on-error – .github/workflows/security.yml line 41 lets failing correctness tests pass. Delete or isolate the step.
  • Medium – Enable Dependabot – No bot configured; add .github/dependabot.yml for Python, npm and actions.
  • Low – Set workflow timeouts – .github/workflows/release.yml lacks timeout-minutes; add a reasonable bound.
  • High – Hub modules – lib/__init__.py, lib/relevance.py, lib/query.py each have >40 downstream imports; keep them stable, extract volatile logic.
  • High – Deep nesting – Files fusion.py, github.py, reddit.py reach 8‑level indentation; refactor with early returns.
  • High – Duplicated blocks – Similar 6‑line snippets repeat across 20+ source adapters; factor into shared helpers.
  • High – Oversized files – render.py and github.py exceed 1 400 lines; split by responsibility.
  • Medium – Broad exception handling – Several modules use bare except:; replace with specific catches.
  • Medium – File opens without context manager – last30days.py, test_device_auth.py; wrap with with open(...).
  • Medium – High branching density – cluster.py, polymarket.py; consider strategy dispatch tables.

The Bottom Line

The repository delivers a functional, end‑to‑end multi‑source search‑and‑summarize pipeline, but the core modules are tightly coupled and contain several maintainability hotspots. It is suitable for teams that need rapid prototyping of cross‑platform intelligence and can allocate effort to refactor the hub and oversized files. Robust CI hygiene (pinned actions, Dependabot) should be added before production deployment.