insights - Moses Yebei

The Problem

Frappe Insights addresses the gap between operational data in Frappe Framework apps and the reporting tools needed to extract value from it. Teams typically resort to hand-written SQL or external BI tools that require duplicating data. Insights provides a self-hosted BI layer that connects directly to existing databases—MySQL, PostgreSQL, DuckDB, BigQuery—with a visual query builder and dashboarding interface.

What This Does

This is a fork of frappe/insights (1006 stars upstream) with 544 files. The backend (insights/) is a Python/Frappe application that manages data sources, query execution, and user permissions. The frontend (frontend/src2/) is a Vue 3 + TypeScript SPA handling the query builder, chart configuration, and dashboard layout.

Key modules: insights_data_source_v3 manages database connections and query execution via Ibis; insights_table_v3 handles table metadata; frontend/src2/query/Query.vue is the visual query builder; frontend/src2/charts/ contains chart rendering and configuration components.

How It Is Wired

Execution starts in frontend/index.html → frontend/src2/main.ts, which mounts the Vue app. The frontend communicates with the Frappe backend through frontend/src2/helpers/resource.ts, which wraps Frappe's REST API. The backend entry points are Frappe doctypes—insights_data_source_v3 and insights_table_v3—that handle HTTP requests and route to ibis_utils.py for query compilation and execution.

The import graph shows 191 internal modules with 297 edges. Three hubs carry the widest blast radius:

frontend/src2/helpers/index.ts — imported by 16 modules, exports shared utilities
insights/utils — imported by 14 modules, backend helpers
insights/insights/doctype/insights_table_v3/insights_table_v3 — imported by 14 modules

All three sit inside circular dependency cycles, along with frontend/src2/charts/chart.ts and frontend/src2/query/query.ts. Breaking these cycles requires extracting shared types or deferring imports; the cost of not doing so is that changes to any hub ripple unpredictably.

How To Use It

Setup: The repo has pyproject.toml (Python backend) and frontend/package.json (Vue frontend). Docker Compose exists at docker/docker-compose.yml. The README documents production deployment via Frappe's easy-install script:

wget https://frappe.io/easy-install.py
python3 ./easy-install.py deploy --project=insights_prod_setup

Configuration: Backend configuration follows Frappe conventions (site config in sites/). Frontend uses Vite with config at frontend/vite.config.js; environment variables go in frontend/.env. No custom env vars are documented in the repo.

Running it: For development, the backend runs as a Frappe bench app; the frontend runs via yarn dev in frontend/. Neither command is documented in the README, so exact flags are not verifiable from this repo alone.

Real-World Use

A typical deployment: a Frappe ERPNext instance plus a PostgreSQL analytics database. Insights connects to both as data sources, builds a query joining Sales Order and Payment Entry tables, and publishes a dashboard showing revenue by region. Non-technical users interact with the dashboard without SQL access to production data.

Code Health & Issues

Static analysis found 187 issues (88 high, 99 medium) across 5 categories. The most significant:

High - Import cycles (18 instances): frontend/src2/helpers/index.ts, insights/insights/doctype/insights_table_v3/insights_table_v3.js and .py participate in circular imports. Fix: extract shared types or invert dependencies.
High - Deep nesting (21 instances): insights/insights/doctype/insights_table_v3/insights_table_v3.js reaches indentation depth 11. Fix: early returns and guard clauses.
Medium - Broad exception handling (11 instances): insights_table_v3.py, insights_data_source_v3.py catch Exception indiscriminately. Fix: catch specific exceptions.
Medium - Oversized files (2 instances): ibis_utils.py and data_warehouse.py exceed 900 lines each.
Medium - Hub modules (8 instances): frontend/src2/helpers/index.ts has 16 dependents.

The 9-item security audit flags two high-severity issues: unpinned GitHub Actions (docker/setup-qemu-action@v3, etc.) and CI workflows that never invoke the 36 test files. Medium issues include missing token permissions, no Dependabot, no dependency vulnerability scanning, a 6.1MB preview.gif in git, and a postinstall script in package.json. 16 of 40 declared dependencies are multiple major versions behind current.

The Bottom Line

Frappe Insights is a functional BI tool with a solid feature set and active upstream maintenance, but this fork carries real technical debt: circular imports, oversized backend modules, and CI that doesn't run its own tests. It's suitable for teams already on Frappe who need embedded analytics and can invest in addressing the import cycles and CI gaps before relying on it in production.

What the analyser found

Deployment readiness

7/7

Container image
CI pipeline
Lockfile committed
Test suite
README
License
No committed secrets

Composition

544 files

Vue190
Python142
JSON41
TypeScript41
JavaScript22
YAML14

ReactVueDjangoExpressTailwindDocker

Module dependencies

The 10 most depended-upon modules of 191, from static import analysis. Red outline marks a module in an import cycle.

graph LR n0_frontend_src2_helpers_index["index"] n1_frontend_src2_translation["translation"] n2_s_table_v3_insights_table_v3["insights_table_v3"] n3_insights_utils["utils"] n4_e_v3_insights_data_source_v3["insights_data_source_v3"] n5_insights_decorators["decorators"] n6_frontend_src2_helpers_toasts["toasts"] n7_ntend_src2_types_query_types["query.types"] n8__insights_team_insights_team["insights_team"] n9_insights_tests_base["base"] n0_frontend_src2_helpers_index --> n1_frontend_src2_translation n0_frontend_src2_helpers_index --> n7_ntend_src2_types_query_types n0_frontend_src2_helpers_index --> n6_frontend_src2_helpers_toasts n6_frontend_src2_helpers_toasts --> n0_frontend_src2_helpers_index n5_insights_decorators --> n8__insights_team_insights_team n4_e_v3_insights_data_source_v3 --> n2_s_table_v3_insights_table_v3 n2_s_table_v3_insights_table_v3 --> n3_insights_utils n2_s_table_v3_insights_table_v3 --> n8__insights_team_insights_team n8__insights_team_insights_team --> n2_s_table_v3_insights_table_v3 n3_insights_utils --> n8__insights_team_insights_team classDef cyc stroke:#ef4444,stroke-width:2px; class n0_frontend_src2_helpers_index,n2_s_table_v3_insights_table_v3,n3_insights_utils,n4_e_v3_insights_data_source_v3,n6_frontend_src2_helpers_toasts,n8__insights_team_insights_team cyc;

Dependencies

16 of 40 declared dependencies are a major version behind. Resolved 40 of 40 against the registry.

Package	Declared	Latest	Behind
`@vueuse/core`	^10.7.2	14.4.0	4 major
`vite`	^4.4.6	8.2.1	4 major
`pinia`	^2.0.30	4.0.3	2 major
`@vitejs/plugin-vue`	^4.2.3	6.0.8	2 major
`@vitejs/plugin-vue-jsx`	^3.0.1	5.1.6	2 major
`eslint`	^8.19.0	10.8.1	2 major
`typescript`	^5.5.2	7.0.2	2 major
`@tanstack/vue-table`	^8.10.7	9.1.2	1 major

What would stop me shipping this

Ranked by severity × confidence × production reach. Reach is the honest discriminator across a collection that is mostly other people's code: the same finding matters more in something that ships.

high2

medium6

low1

Pin third-party GitHub Actions to a commit SHAhigh6 occurrences

.github/workflows

docker/setup-qemu-action@v3, docker/setup-buildx-action@v3, docker/login-action@v2, docker/build-push-action@v6

A tag can be moved, so the action running with your token and secrets is whatever its owner last pushed; this is how tj-actions/changed-files leaked secrets from thousands of repos.

Fix: Replace each @vN with the 40-character commit SHA, keep # vN as a comment, and let Dependabot bump the SHAs.

Make CI invoke the test suite it hashigh36 occurrences

.github/workflows

36 test files, no test command in the 4 workflows read

A green check that never executed an assertion is worse than no check, because reviewers trust it.

Fix: Add a test step to the existing workflow rather than a new workflow.

Declare least-privilege permissions for GITHUB_TOKENmedium3 occurrences

.github/workflows/build.yml

3 workflow(s) declare no permissions, 3 of them reference secrets

With no declaration the token inherits the repository default, so any injected step can push commits or mint releases from inside your own CI.

Fix: Add permissions: contents: read at the top of the workflow and widen per job only where needed.

Enable Dependabot or Renovatemedium

3 manifest(s), no update bot configured

Without a bot a published advisory sits unpatched until someone audits by hand, which across 1,322 repositories means never.

Fix: Commit .github/dependabot.yml covering the repo ecosystems plus github-actions.

Gate pull requests on a dependency vulnerability scanmedium

.github/workflows

no dependency scan in CI

This is the one gate that would catch a known-vulnerable package before it reaches a build, and no repository in the sample had it.

Fix: Add dependency-review-action on pull_request, or osv-scanner on push and a schedule.

Move large binaries to Git LFS or out of the repositorymedium

.github/preview.gif

1 blobs over 5MB: preview.gif 6.1MB

One repository here carries twenty blobs over 5MB including a 10.9MB spreadsheet, so every clone and every CI checkout pays for data nobody diffs.

Fix: Track those extensions with LFS, or move datasets to object storage and fetch them in a setup step.

Set persist-credentials: false on checkoutmedium

.github/workflows/server-tests.yml

checkout keeps the token, then dependencies are installed

The token stays in .git/config for every later step, so a malicious postinstall script reads a pushable credential without one ever being passed to it.

Fix: Add with: persist-credentials: false, and pass an explicit token only to the step that pushes.

Review the install lifecycle script and disable scripts in CImedium

package.json

postinstall

Install scripts are the execution vector for every npm compromise from event-stream onward, and a postinstall that fetches a binary makes the build depend on a URL nobody reviews.

Fix: Move the work into an explicit build step, or set ignore-scripts in CI and run it by name.

Set timeout-minutes on the workflow jobslow4 occurrences

.github/workflows/build.yml

4 workflow(s) declare no job timeout

A wedged step runs to the six-hour platform default, which on a two-hourly schedule means three runs overlap behind it.

Fix: Add timeout-minutes with a realistic bound to each job.

Checked deterministically against the repository tree and a bounded set of its files: committed credentials, unpinned actions and base images, missing lockfiles and update bots, workflows that discard failures, published advisories against the declared dependencies, runtime configuration, licensing and notebook reproducibility. No language model is involved in this section.