The Problem

Collecting raw tweet streams and turning them into actionable metrics is tedious. Engineers must handle API authentication, pagination, rate‑limits, and then run ad‑hoc analysis for sentiment, engagement, and topic extraction.

What This Does

The repository provides a minimal end‑to‑end pipeline for pulling data from the Twitter v2 API and exploring it in Jupyter notebooks.

  • Twitter.py contains the only Python code; it authenticates with the API, issues a GET request, and returns raw JSON.
  • Two notebooks – Twitter_Data_Analysis_Tutorial.ipynb and Enhanced_Twitter_Data_Analysis_Tutorial.ipynb – demonstrate sentiment scoring, engagement aggregation, and simple topic modeling on the collected payload.
  • .env.example shows the expected environment variables (e.g., TWITTER_BEARER_TOKEN) but does not ship any secrets.

How It Is Wired

Execution starts with the script Twitter.py. The file defines a top‑level function (e.g., fetch_tweets) that builds the request URL, reads the bearer token from the environment, and calls requests.get. No other internal modules are imported, so the call graph consists of a single node with no edges.

  • Outbound request – a single HTTP GET to https://api.twitter.com/2/.... The call lacks an explicit timeout, meaning the process can block indefinitely if the remote endpoint stalls.
  • Data hand‑off – the JSON response is returned directly to the caller; the notebooks import the script (or re‑run the cells) and operate on the in‑memory object. No persistence layer (database or file) is used.
  • Scope of impact – because the only outward‑facing operation is the HTTP request, any change to Twitter.py affects every notebook that imports it. The lack of modular imports keeps the blast radius small, but the deep nesting (six‑level indentation) makes the control flow hard to follow and increases the risk of bugs during modification.

No circular dependencies or additional internal modules exist, so the wiring is straightforward but also limited in extensibility.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/twitter-data-analysis
cd twitter-data-analysis

# Create a virtual environment (Python 3.10+ recommended)
python -m venv .venv
source .venv/bin/activate

# Install runtime packages manually (requests, pandas, etc.)
pip install requests pandas matplotlib seaborn nltk scikit-learn
  1. Copy .env.example to .env and fill in your Twitter bearer token.
  2. Run the collector: python Twitter.py. The script prints or returns raw tweet JSON.
  3. Open either notebook (Twitter_Data_Analysis_Tutorial.ipynb or Enhanced_Twitter_Data_Analysis_Tutorial.ipynb) in Jupyter, execute the cells, and point the data‑loading section to the JSON output from step 2.

Missing pieces: there is no requirements.txt or pyproject.toml, so the exact dependency list must be inferred from the notebooks’ imports.

Real‑World Use

A data‑science team could schedule python Twitter.py > raw.json via cron, then load raw.json in the notebooks to produce daily sentiment dashboards. The workflow is lightweight: a single script fetches data, and the notebooks perform all downstream analytics without needing a separate ETL framework.

Code Health & Issues

  • MEDIUM – Deep nesting – Twitter.py reaches six levels of indentation, making the logic hard to follow. Recommendation: flatten with early returns or guard clauses, and extract inner blocks into helper functions.
  • MEDIUM – Outbound request missing timeout – The requests.get call in Twitter.py does not specify timeout=. Without it the process can hang indefinitely, risking worker exhaustion. Fix: add a reasonable timeout (e.g., timeout=10) or use a configured requests.Session.
  • SDLC observations – No test suite, no CI/CD configuration, no LICENSE file, and no README. These gaps hinder onboarding, automated quality checks, and legal clarity.

The Bottom Line

The repo delivers a functional, notebook‑centric example for pulling and analysing Twitter data, but it is a prototype rather than production‑ready code. The single‑script design keeps the dependency surface tiny, yet deep nesting and missing request timeouts introduce maintainability and reliability risks. It is suitable for exploratory work or as a teaching aid; teams aiming for robust pipelines will need to add testing, CI, explicit dependencies, and a more modular architecture.