soccerdata - Moses Yebei

The Problem

Soccer analytics work needs data from many sources—FBref for stats, Club Elo for ratings, Understat for expected goals—but each site has different HTML, column names, and identifiers. Teams end up writing and maintaining one scraper per site, and every site redesign breaks them. soccerdata solves that by wrapping nine scrapers behind one consistent Python API that returns Pandas DataFrames with matching schemas.

What This Does

soccerdata is a Python package with one reader class per data source: FBref, ESPN, ClubElo, FotMob, Sofascore, SoFIFA, Understat, WhoScored, and MatchHistory, each in its own module under soccerdata/. You instantiate a scraper with a league and season, then call methods like read_schedule() or read_team_season_stats() to get DataFrames. Downloads are cached locally, so repeated reads don't re-hit the network.

This is a fork of probberechts/soccerdata (2,040 stars upstream) with 0 stars of its own. The core value is unchanged: one import, one API, nine sources.

How It Is Wired

Execution starts at the public class constructors—sd.FBref('ENG-Premier League', '2021')—which live in the per-source modules. Every reader inherits from BaseRequestsReader in soccerdata/_common.py, which owns the HTTP fetching, caching, and file I/O. That module is the hub: 10 of the 27 internal modules import it, and it contains 36 functions and 4 classes.

The highest-blast-radius function is parse, called from 27 places across the codebase—every source module routes its HTML/JSON through it. read_seasons (10 callers), _is_complete (9), and read_schedule (9) follow. _common.py is also where the file effects live: it reads and writes cached downloads, as do understat.py and clubelo.py. The module graph shows no circular dependencies, but _common.py sits at instability 0.6—it's both heavily imported and imports widely, so changing it touches most of the package.

How To Use It

pip install soccerdata
# or, since uv.lock is present: uv sync

import soccerdata as sd
fbref = sd.FBref('ENG-Premier League', '2021')
games = fbref.read_schedule()
team_stats = fbref.read_team_season_stats(stat_type="passing")

No environment variables or API keys are required—the scrapers hit public pages. The README documents these commands verbatim. Note the usage notice: scraping must comply with each site's terms of service.

Real-World Use

A match-prediction pipeline: pull fixtures from FBref.read_schedule(), merge in ClubElo.read_team_history() for ratings, add Understat.read_shot_events() for expected goals, then feed the joined DataFrame to a model. The consistent column naming across sources is what makes the merge practical rather than a data-cleaning project.

Code Health & Issues

Static analysis found 16 issues (6 high, 10 medium) across 5 kinds:

High - Duplicated code blocks - 74 repeated 6-line blocks across 11 files, including _common.py, clubelo.py, espn.py, fbref.py. Fix: extract shared helpers.
High - Deep nesting - max indentation depth 7 in _common.py, espn.py, fbref.py. Fix: guard clauses and early returns.
Medium - Oversized files - _common.py at 682 lines, plus fbref.py and understat.py. Fix: split by responsibility.
Medium - Broad exception handling in _common.py swallows errors indiscriminately.
Medium - Files opened without context managers in _common.py and understat.py.

SDLC observations from the workflow files: GitHub Actions are pinned to version tags, not commit SHAs (a supply-chain risk); CI pushes directly to the default branch; no permissions block declared on workflows; no dependency vulnerability scan in CI. Tests exist (17 files) and CI runs on GitHub Actions, but the CI itself needs hardening.

The Bottom Line

A genuinely useful library—the unified API across nine soccer data sources is the kind of thing that saves teams weeks of scraper maintenance. The fork adds nothing over upstream so far, and the code quality issues concentrate in _common.py, which is exactly the file you'd need to touch for most changes. Worth using as-is; worth fixing the CI before trusting it with secrets.

What the analyser found

Deployment readiness

5/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

91 files

Python27
reStructuredText25
Jupyter Notebook11
YAML8
JSON3
Markdown2

DVC

Module dependencies

The 10 most depended-upon modules of 27, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_soccerdata__common["_common"] n1_soccerdata__config["_config"] n2_soccerdata___init__["__init__"] n3_soccerdata_espn["espn"] n4_soccerdata_fbref["fbref"] n5_soccerdata_fotmob["fotmob"] n6_soccerdata_match_history["match_history"] n7_soccerdata_sofascore["sofascore"] n8_soccerdata_sofifa["sofifa"] n9_soccerdata_understat["understat"] n2_soccerdata___init__ --> n3_soccerdata_espn n2_soccerdata___init__ --> n4_soccerdata_fbref n2_soccerdata___init__ --> n5_soccerdata_fotmob n2_soccerdata___init__ --> n6_soccerdata_match_history n2_soccerdata___init__ --> n7_soccerdata_sofascore n2_soccerdata___init__ --> n8_soccerdata_sofifa n2_soccerdata___init__ --> n9_soccerdata_understat n0_soccerdata__common --> n1_soccerdata__config n3_soccerdata_espn --> n0_soccerdata__common n3_soccerdata_espn --> n1_soccerdata__config n4_soccerdata_fbref --> n0_soccerdata__common n4_soccerdata_fbref --> n1_soccerdata__config n5_soccerdata_fotmob --> n0_soccerdata__common n5_soccerdata_fotmob --> n1_soccerdata__config n6_soccerdata_match_history --> n0_soccerdata__common n6_soccerdata_match_history --> n1_soccerdata__config n7_soccerdata_sofascore --> n0_soccerdata__common n7_soccerdata_sofascore --> n1_soccerdata__config

Findings

LowDependencies declared without a lockfile - non-reproducible builds docs/requirements.txt

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high2

medium3

low1

Pin third-party GitHub Actions to a commit SHAhigh5 occurrences

.github/workflows

codecov/codecov-action@v5, crazy-max/ghaction-github-labeler@v5.3.0, salsify/action-detect-and-tag-new-version@v2.0.3, pypa/gh-action-pypi-publish@v1.13.0

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Open a pull request instead of pushing to the default branchhigh

.github/workflows/ci.yml

git push

Automated commits land on the branch that deploys, with no test having run against the result.

Fix: Push to a bot branch and open a pull request, or restrict the push to a tag ref.

Declare least-privilege permissions for GITHUB_TOKENmedium3 occurrences

.github/workflows/ci.yml

3 workflow(s) declare no permissions, 2 of them reference secrets

With no declaration the token inherits the repository default, so any injected step can push commits or mint releases from inside your own CI.

Fix: Add permissions: contents: read at the top of the workflow and widen per job only where needed.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Set persist-credentials: false on checkoutmedium

.github/workflows/ci.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Set timeout-minutes on the workflow jobslow3 occurrences

.github/workflows/ci.yml

3 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.