Skip to Content
Living documentation — last reviewed 2026-05-28
FeaturesAnalyticsAnalytics — QA Plan

Analytics — QA Plan

TKN-24 targeted regression commands

Run from the repository root. These commands are references for report, route-hydration, and insight regressions, not execution results or production benchmarks. Documentation cleanup does not run validation; the implementation owner records exact latest results separately.

pnpm -C apps/api exec vitest run --config vitest.unit.config.ts src/analytics/analytics.service.unit.spec.ts src/analytics/analytics.controller.unit.spec.ts src/analytics/analytics-reports.controller.unit.spec.ts TEST_DB_WORKERS=1 pnpm -C apps/api exec vitest run --config vitest.integration.config.ts src/analytics/analytics-reports.int.spec.ts NEXT_PUBLIC_E2E_TEST_MODE=true pnpm -C apps/web exec vitest run --config vitest.integration.config.ts src/components/analytics/reports-screen.int.spec.tsx src/components/overview/v2/overview-v2.int.spec.tsx src/components/overview/v2/revenue-panel.int.spec.tsx src/components/overview/sidebar-nav.int.spec.tsx pnpm -C apps/api exec vitest run --config vitest.unit.config.ts src/insights/insight-consolidation.unit.spec.ts src/insights/insights.service.unit.spec.ts src/insights/insights.controller.unit.spec.ts TEST_DB_WORKERS=1 pnpm -C apps/api exec vitest run --config vitest.integration.config.ts src/insights/insight-consolidation.int.spec.ts NEXT_PUBLIC_E2E_TEST_MODE=true pnpm -C apps/web exec vitest run --config vitest.integration.config.ts src/components/payments/payment-tabs.int.spec.tsx 'src/app/[lang]/(protected)/dashboard/tasks/tasks-page.int.spec.tsx' src/components/overview/v2/action-center.int.spec.tsx src/components/overview/schedule/weekly-calendar.int.spec.tsx pnpm -C apps/web exec vitest run --config vitest.unit.config.ts 'src/app/[lang]/(protected)/dashboard/schedule/schedule-page.unit.spec.ts' make test-e2e-web # includes e2e/specs/owner-reports.spec.ts

The PostgreSQL command requires DATABASE_URL pointing to a prepared, idle, disposable _test database. Existing setup clones/drops worker databases and uses deterministic scoped reporting fixtures; do not point it at application data.

TKN-24 acceptance matrix

AreaExpected
AuthorizationAuthenticated owner/admin via requireMembership + PermissionsService analytics:view; coach/member/non-member denied. Report data, the seven new exports, legacy summaries and the eight legacy exports share that gate.
QueriesInclusive org-local date windows, DST-correct, maximum 366 days; invalid/inverted dates, invalid class UUID/method, or cohort months outside integer 1–12 return 400. Cohorts default six months; future asOf clamps to today.
AttendanceEnded published/non-deleted sessions only; confirmed/attended/no_show denominator excludes cancelled/waitlisted/deleted bookings. Org-safe historical membership join preserves ended/deleted/current-staff visits.
Capacity/rankingPositive finite capacity includes empty sessions once; weighted capacityBooked/summedCapacity, null unlimited excluded, no 100% cap. Undefined rates null. Stable server tie ordering matches CSV; current external occupancy is not history.
MoneyCanonical signed receipts at settlement time, tokenOnly excluded, actual method filter, each currency separate. Negative net preserved. Partial-month delta null with prior full month exposed; never label receipts MRR.
CohortsJoin-month denominator; lifecycle end is end_date/deletion only; never-joined shells, cancelled rows without end_date and pre-import departures excluded. Future/empty cells null, current partial. Explicit lifecycle estimate, not paid retention; no suspension/reactivation ledger.
CSVAll seven projection identifiers remain: attendance-report, class-ranking, class-capacity, revenue-report, revenue-methods, revenue-monthly, member-cohorts. Matching relevant filters/projections, BOM/CRLF/stable empty headers, escaping/formula-safe text; numeric negatives remain numbers.
UIExactly four areas at /[lang]/dashboard/analytics: Attendance (attendance/ranking/capacity), Revenue (totals/methods/monthly), Retention (cohorts), Trials (funnel, breakdowns, attended-not-converted; QA in Trials QA). One selected-area dataset fetch shared by its tables, not per projection; snapshot and class-options requests are separate. Sidebar gated on analytics:view; Studio snapshot only inside reports. OverviewV2 has no snapshot/report link or associated report dashboard requests. en/he/ru and RTL; loading/errors never become zeros or previous-filter data. Unauthorized roles make no report data calls.
URL hydrationOpen the exact example in behavior; controls, area, queries, and CSV match. Switch areas/filters, reload, back, and forward; restore tab/from/to/classTypeId/method/months/asOf from the URL. Unknown area normalizes to Attendance; replace pins absent org-local defaults without another history entry. Push preserves edits in history. Inactive filters persist but do not leak into other datasets.
Invalid URL guardsRelevant-schema errors stay visible in the URL, block area dataset requests and all its exports, and recover on correction. No silent all-class/all-method fallback. Valid historical class UUID absent from current options remains explicit and usable.
Action/report boundaryActionCenter retains its rule catalog, including birthday follow-up rather than a birthday report. Destination hydration is unconditional; see Insights QA for rule predicates and Dashboard QA for destination routes.
Audit scopeReport currency separation and canonical receipts do not certify legacy scalar multi-currency summaries, PaymentAnalytics time windows, list-price plan distribution versus chargeable MRR, admin raw completed/createdAt aggregation, or historical paid retention. These remain explicit deferrals, not fixes to verify as delivered.

Prior local verification baseline

Before the three-area/action consolidation, TKN-24 recorded existing API unit/controller tests 25 PASS, new controller tests 43 PASS, real-Postgres report tests 10 PASS, and focused web suite 49 PASS (including 28 report tests). API/web TypeScript checks, focused ESLint, and Nx API/web builds also passed at that prior baseline. These are historical implementation results, not latest consolidation verification.

The prior cmux browser smoke covered report projections across en/he/ru, class/date/method filters, a CSV click, future/partial lifecycle cells, multiple currencies, and the OFF baseline. Its former UI arrangement is not evidence for the current three-area navigation or reports-only snapshot placement.

That prior browser stayed at 743×994, ignored requested widths, and produced viewport-only screenshots. The current consolidation browser surface is 800×600 despite the requested viewport; neither establishes complete responsive coverage. API smoke uses a private local clone without migrations, not production. At the prior baseline, full-demo seed failed on unrelated missing subscriptions.discount_percent; scoped reporting fixtures recovered that smoke. No production smoke or performance benchmark is claimed.

Consolidation verification — 2026-09-09

CheckObserved result
API analytics + insights unit/controller suites132 tests passed across six files.
Real-Postgres report + insight consolidation suites18 tests passed across two files with TEST_DB_WORKERS=1; disposable worker databases cloned from the local test template.
Web report/overview/Action Center/revenue/sidebar integration suites84 tests passed across five files.
Payments + weekly calendar integration suites17 tests passed across two files.
Tasks URL integration suite6 tests passed; final driver run has no warnings. Includes literal search=all, real task type filters, and browser-history hydration.
Schedule redirect unit suite5 tests passed.
TypeScriptpnpm exec tsc --noEmit -p apps/api/tsconfig.app.json and pnpm exec tsc --noEmit -p apps/web/tsconfig.json passed.
Focused ESLintNo errors. Four pre-existing non-null assertions remain in the insights service; newly introduced assertions were removed.
Buildspnpm exec nx build @taikan/api and pnpm exec nx build @taikan/web passed. The first web build stopped at prerender because the test env contained a placeholder Clerk publishable key; the build passed using the existing local development key, with Sentry release/upload disabled.

Actual API smoke exercised both local rollout branches: with no overrides, status was 200 { enabled: false }, report data/new CSV were 404, and a legacy CSV remained 200. With isolated test-process overrides only, all seven report CSV projections returned 200 with UTF-8 BOM. A charge created in August but settled in September and a September refund produced ILS 100 collected / 20 refunded / 80 net; USD 50 remained a separate currency. Filtered attendance returned one attended booking, 100% attendance and 10% weighted capacity utilization. Insights returned 200 in both branches.

Actual built-web browser smoke opened revenue/attendance/retention direct URLs, followed area history, exercised a CSV download button, verified Hebrew RTL and Russian filter hydration, opened the Payments debt tab and the real not_completed&type=manual_refund&search=all task state, and visually confirmed the settled Overview Action Center without a Studio KPI snapshot. The cmux surface stayed at 800×600 and screenshots were viewport-only; complete responsive coverage is not claimed.

The smoke used a privately cloned local test database and scoped fixtures. The broad base seeder still failed on the unrelated missing subscriptions.discount_percent; no migration or source workaround was applied. Local test-process overrides did not create, modify, or enable remote PostHog flags. No production smoke, accounting reconciliation, or performance benchmark was performed.

Legacy summary regression checklist

The checks below concern the pre-existing summaries/exports, not the stricter new report contract.

Auth

StepExpected
Coach / member calls any /analytics/* endpoint403.
Non-member403.
Owner / admin200 on legacy routes and new report data.

Revenue summary

StepExpected
Org with 1 active monthly $100 planmrrInCents = 10000.
Org with 1 active yearly $1200 planmrrInCents = 10000 (1200/12).
Org with 1 active quarterly $300 planmrrInCents = 10000 (300/3).
Org with collected $500 this month, $400 last monthcollectedDeltaPct = 25.
Org with $0 last monthcollectedDeltaPct = null.
3 subscriptions with debt = [100, 0, 250] centsoutstandingInCents = 350.
Deleted subscription with price > 0Excluded from MRR.
₪500 punch card bought this month, ₪200 of it refunded this monthRevenue summary collectedThisMonthInCents = 30000, packsSold = { count: 1, amountInCents: 30000 }; the legacy CSV shows packsSoldCount / packsSoldAmountInCents columns.

Revenue trend

StepExpected
months=66 buckets returned, oldest → newest.
months=2424 buckets.
Failed paymentsNot included.
Refunded transactionsCanonical receipt rules apply; refund rows subtract from net receipts at settlement time.

Members summary

StepExpected
Org with 500 members, 450 active, 200 with Clerk accountsactivatedActive = 200.
10 new this month, 5 new last monthnewDeltaPct = 100.
Last month 10 joined, 3 of them cancelled since; this month 8 joinednewThisMonth = 8, newDeltaPct = -20 (last month on the joined-then basis).
0 churned last monthchurnedDeltaPct = null.

Members activation

StepExpected
100 imported (no Clerk, no invite) + 20 invited (pending) + 50 activatedSums match.
Total = imported + invited + activated.

At-risk

StepExpected
?sample=5 returns max 5 sample rows.
?sample=0 returns 0 sample rows but the correct count.
?sample=100 returns ≤ count rows.
Sample ordered by name ascending.

Classes

StepExpected
?days=30 returns activity in past 30 days only.
popular-classes?limit=5At most 5 entries.
Class type with no bookings in the windowNot present (excluded from popular).
Class utilization for type with mixed full+empty sessionsAverage fill rate reflects both.

Workouts summary

StepExpected
MTD assignments and completions countedYes; PRs MTD counted.
Excludes deleted assignmentsYes.

Legacy CSV export

StepExpected
?tab=revenue-summary1-row CSV with header line and value line.
?tab=invalid400.
CSV opens in Excel with Hebrew labelsLegacy export has no BOM; new report CSV includes UTF-8 BOM.
Cells containing commasQuoted properly.
Cells containing newlinesQuoted properly.

Agent surface

StepExpected
Agent prompt: “what’s our MRR?” → calls analytics.revenue_summaryReturns the same data; agent formats prose + table.
Agent prompt: “Who’s at risk?” → calls analytics.at_risk_membersReturns sample + count.
Member role attempts to readAgent doesn’t expose analytics tools to member.

Performance

  • Suggested measurement targets, not observed production results: p95 < 500ms for orgs up to 1000 members; investigate coach overview above 1s for 50-coach orgs.
  • Repeated requests should not regress (no per-call cache, but indexes may be warm). Local reporting smoke establishes behavior, not scale.

Legacy negative inputs

  • Pass months=abc → parsed as NaN, defaults applied.
  • Pass sample=-1 → behavior depends on SQL LIMIT; document and ideally clamp to ≥ 0.
  • Pass days=10000 → query returns all historical data; performance scales linearly.