pygwalker - Moses Yebei

The Problem

Data scientists often have to switch between notebooks, ad‑hoc scripts, and separate BI tools to explore a pandas DataFrame. The friction of copying data, manually building charts, and re‑running cells makes iterative analysis slow and error‑prone.

What This Does

pygwalker turns any pandas DataFrame into an interactive web UI built with React, Tailwind and Flask. The core Python package lives in the pygwalker/ folder (67 files) and provides the PygWalker class, data‑parser hierarchy (data_parsers/base.py, data_parsers/database_parser.py) and a thin Flask server (api/webserver.py). The bundled front‑end lives under app/ (50 files) and is compiled with Vite (app/vite.config.ts). Example notebooks in examples/ demonstrate the Jupyter integration (api/jupyter.py) and a Reflex demo (examples/reflex_demo/app/app.py).

How It Is Wired

Execution starts at the CLI entry point bin/pygwalker_command.py (main function, line 124). main parses sub‑commands and eventually calls PygWalker.__init__ (in pygwalker/api/pygwalker.py). This initializer selects a data parser via get_parser (in pygwalker/services/data_parsers.py) and builds a chart spec with _init_spec. The spec is handed to the Flask server (api/webserver.py) which serves app/index.html.

Key call chain (shortest path to external effect):

main → PygWalker.__init__ – creates a parser.
PygWalker.__init__ → get_parser → _get_data_parser – loads the appropriate parser (e.g., PandasDataFrameDataParser).
PygWalker._get_datas → get_datas_by_sql / get_datas_by_payload – pulls data from the supplied DataFrame or SQL payload.
PygWalker._init_spec → _get_props → get_local_user_id → _read_and_create_file – writes a small config directory on disk (os.makedirs).

Network traffic occurs in pygwalker/services/cloud_service.py (_upload_file_to_s3, read_config_from_cloud) and in the Flask server when the UI requests chart assets. File I/O is limited to the config folder (via _read_and_create_file) and optional uploads to S3. Cryptographic work is confined to pygwalker/services/data_parsers.py (_get_pd_dataset_hash).

The hub module pygwalker/data_parsers/base.py is imported by 20 other modules, giving it the highest blast radius. Three circular import groups (api/pygwalker.py, services/data_parsers.py, __init__.py) increase maintenance risk: any change forces a careful refactor to break the cycle. Deeply nested functions (up to 7 levels) appear in services/cloud_service.py and several React components, making the control flow harder to follow.

How To Use It

# Clone the repo
git clone https://github.com/moses-y/pygwalker
cd pygwalker

# Install the Python package (editable mode recommended)
pip install -e .

# Build the front‑end (requires Node ≥ 16)
cd app
yarn install   # reads app/package.json & yarn.lock
yarn build     # produces static assets referenced by the Flask server

Configuration is stored under the user’s home directory (~/.pygwalker) by default; the path is created by _read_and_create_file. To launch the UI from a notebook:

import pandas as pd, pygwalker as gw
df = pd.read_csv("data.csv")
gw.PygWalker(df).display()

For the CLI demo:

python -m pygwalker.command --data data.csv   # entry point: bin/pygwalker_command.py

(Exact CLI flags are defined in bin/pygwalker_command.py.)

Real‑World Use

A data‑science team can embed PygWalker(df) in a Jupyter notebook to let analysts drag‑and‑drop fields, generate Vega‑Lite specs, and export the resulting chart as JSON. The same UI can be served internally via Flask for non‑technical stakeholders, with optional S3 upload of the generated spec for downstream reporting pipelines.

Code Health & Issues

HIGH – Pin third‑party GitHub Actions to a commit SHA (.github/workflows/*).
HIGH – Commit a lockfile alongside pyproject.toml (currently missing).
MEDIUM – Declare least‑privilege permissions for GITHUB_TOKEN (.github/workflows/auto-ci.yml).
MEDIUM – Enable Dependabot or Renovate for automatic dependency updates.
MEDIUM – Add a dependency‑vulnerability scan step in CI (dependency-review-action or osv-scanner).
MEDIUM – Set persist-credentials: false on the checkout step (.github/workflows/auto-ci.yml).
LOW – Add timeout-minutes to workflow jobs (.github/workflows/publish.yml).
LOW – Add repository convention files (.editorconfig, .gitattributes, formatter config).

Additional static‑analysis findings (not repeated here) include broad exception handling, empty catch blocks, duplicated code blocks, file handles without context managers, and high branching density.

The Bottom Line

pygwalker delivers a functional bridge between pandas and an interactive React UI, with a clear separation of parser logic and front‑end assets. The codebase is usable but hampered by import cycles, deep nesting, and several maintainability hotspots. It is suitable for teams comfortable with Python + React stacks who need rapid exploratory visual analysis, provided they allocate effort to resolve the highlighted health issues.

What the analyser found

Deployment readiness

6/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

182 files

Python85
TSX27
Markdown18
TypeScript14
Jupyter Notebook8
HTML5

ReactFlaskTailwindpandas

Architecture

Top-level areas of the codebase, sized by module count. Arrows show how many imports cross from one area into another.

graph TD g0["pygwalker 62 modules"] g1["app 43 modules"] g2["examples 10 modules"] g3["tests 5 modules"] g4["pygwalker_tools 4 modules"] g5["scripts 3 modules"] g2 -->|11| g0 g3 -->|6| g0 g4 -->|4| g0

Module dependencies

The 10 most depended-upon modules of 128, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_pygwalker_data_parsers_base["base"] n1_pygwalker__typing["_typing"] n2_data_parsers_database_parser["database_parser"] n3_pygwalker_utils_encode["encode"] n4_pygwalker_api_pygwalker["pygwalker"] n5_pygwalker_utils_randoms["randoms"] n6_app_src_store_common["common"] n7_ygwalker_communications_base["base"] n8_ygwalker_services_global_var["global_var"] n9_walker_services_data_parsers["data_parsers"] n4_pygwalker_api_pygwalker --> n1_pygwalker__typing n4_pygwalker_api_pygwalker --> n0_pygwalker_data_parsers_base n4_pygwalker_api_pygwalker --> n2_data_parsers_database_parser n4_pygwalker_api_pygwalker --> n5_pygwalker_utils_randoms n4_pygwalker_api_pygwalker --> n8_ygwalker_services_global_var n4_pygwalker_api_pygwalker --> n9_walker_services_data_parsers n0_pygwalker_data_parsers_base --> n1_pygwalker__typing n2_data_parsers_database_parser --> n0_pygwalker_data_parsers_base n9_walker_services_data_parsers --> n0_pygwalker_data_parsers_base n9_walker_services_data_parsers --> n2_data_parsers_database_parser n9_walker_services_data_parsers --> n1_pygwalker__typing classDef cyc stroke:#ef4444,stroke-width:2px; class n4_pygwalker_api_pygwalker,n9_walker_services_data_parsers cyc;

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high2

medium4

low2

Pin third-party GitHub Actions to a commit SHAhigh

.github/workflows

pypa/gh-action-pypi-publish@v1.12.4

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Commit a lockfile beside the manifesthigh

pyproject.toml

manifest with no lockfile in scope

An unlocked range means the artifact you tested and the artifact you ship can contain different transitive code, so a malicious patch release reaches production with no diff.

Fix: Run the package manager once and commit the generated lockfile.

Declare least-privilege permissions for GITHUB_TOKENmedium2 occurrences

.github/workflows/auto-ci.yml

2 workflow(s) declare no permissions, 1 of them reference secrets

With no declaration the token inherits the repository default, so any injected step can push commits or mint releases from inside your own CI.

Fix: Add permissions: contents: read at the top of the workflow and widen per job only where needed.

Enable Dependabot or Renovatemedium

2 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Set persist-credentials: false on checkoutmedium

.github/workflows/auto-ci.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Set timeout-minutes on the workflow jobslow

.github/workflows/publish.yml

1 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Add the repository convention files this project lackslow3 occurrences

missing .editorconfig, .gitattributes, a formatter config

Without them one contributor's editor writes tabs into a Python file, a shell script commits with CRLF and fails in the container, and a notebook diff is unreviewable.

Fix: Add .editorconfig, .gitattributes with text=auto eol=lf, and a formatter config.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.