Monitoring & alerts

Single source of truth: Better Stack (free plan) handles both uptime checks and the daily-backup heartbeat. On the free plan, alerts are delivered by email + the Better Stack mobile app (push) — outbound Discord/Slack/webhook notification channels are paywalled, so monitoring alerts do not go to Discord. UptimeRobot was retired in favour of Better Stack on 2026-06-15 (account since deleted) — its monitors were fully duplicated by Better Stack before removal, so nothing was lost.

Uptime monitors

Monitor	URL	Type
Marketing apex	`https://usetaikan.com/`	status (2xx)
Web app	`https://app.taikan.fit/he/sign-in`	status (2xx)
API health	`https://api.taikan.fit/health`	keyword

Web app is pointed at /he/sign-in (a real 200 page), not the bare / — the root is a 307 → /he/ → 404 redirect chain and is a poor health signal.
API health is a keyword monitor that alerts when "status":"ok" is absent. The endpoint (apps/api/src/app/health.controller.ts) returns HTTP 200 even when degraded (DB/Redis state lives in the body), so a plain status check wouldn’t catch a DB/Redis outage — the keyword does.

Heartbeats (dead-man’s switches)

A heartbeat is pinged only on a successful run; Better Stack alerts when a ping stops arriving. Errors reach Sentry on their own — heartbeats exist for the harder case, a job that silently stops running. In-app heartbeats go through HeartbeatService (apps/api/src/common/heartbeat.service.ts); each name maps to a HEARTBEAT_URL_* variable on the Railway @taikan/api service. Unset = silent no-op. Every API replica with CRONS_ENABLED=true is a worker and a tick runs on exactly one of them, so the variables go on the API service, not on a separate scheduler. This table is the inventory; scheduled-jobs.md points here rather than repeating it.

Better Stack heartbeat	Source	Env var / pinger	Cadence	Pings when
Taikan scheduler alive	`HeartbeatService.schedulerHeartbeat`	`HEARTBEAT_URL_SCHEDULER`	every 5 min (period 15m, grace 10m)	a worker consumed the tick — proves the queue is being worked, nothing more
Taikan billing reconciler	`PlatformBillingReconcilerService.tick`	`HEARTBEAT_URL_BILLING_RECONCILER`	every 5 min (period 15m, grace 30m)	both the sweep and `alertInFlightCharges` finished
Taikan member recurring charge	`RecurringChargeService.handleRecurringCharges`	`HEARTBEAT_URL_MEMBER_RECURRING_CHARGE`	daily 02:00 UTC (period 1d, grace 2h)	the sweep actually ran to completion; an advisory-lock-skipped or aborted tick does not ping
(not provisioned) member billing catch-up	`BillingCatchupService.collectDue`	`HEARTBEAT_URL_MEMBER_BILLING_CATCHUP`	every minute, `billing-collection` lane	the pass collected something or had nothing to select; an all-held pass does not ping. Owner must create the monitor and set the variable — see billing-resilience.md
Taikan platform billing recurring	`PlatformBillingRecurringService.tick`	`HEARTBEAT_URL_PLATFORM_BILLING_RECURRING`	daily 02:00 UTC (period 1d, grace 2h)	the platform-subscription sweep finished
(to provision) Taikan subscription lifecycle	`SubscriptionLifecycleService.tick`	`HEARTBEAT_URL_SUBSCRIPTION_LIFECYCLE`	hourly at :05 — suggested period 1h, grace 30m	the tick ran with zero org-pass and zero row failures and no org was locked by another worker. An hourly tick with nothing left to do today still pings; a tick that lost one row does not
(to provision) Taikan cancellation sweep	`CancellationCronService.handlePeriodEndCancellations`	`HEARTBEAT_URL_CANCELLATION_SWEEP`	daily 02:30 UTC — suggested period 1d, grace 2h	`sweepDueCancellations` ran with zero row failures. A sweep with nothing due still pings; a run whose retry was exhausted does not
(to provision) Taikan entitlement drift	`EntitlementDriftService.handleScan`	`HEARTBEAT_URL_ENTITLEMENT_DRIFT`	daily 04:10 UTC — suggested period 1d, grace 2h	the ledger-vs-counter scan completed. Findings (drift, unopened ledgers, negative counters) do not withhold the ping — they go to Sentry; see Ledger operations
fitkit-db-backup	Railway service `pg-s3-backups`	`HEALTHCHECK_URL` on that service	daily (period 1d, grace 1h)	a confirmed successful dump + upload. Restore steps: restore-from-backup.md
Taikan prod smoke	GitHub workflow `cd-prod-smoke.yml`	`HEARTBEAT_URL_PROD_SMOKE` (repo secret)	every 15 min (period 15m, grace 30m)	the read-only smoke suite is green; a red run pings `<url>/fail`, which alerts at once
Taikan prod smoke (write)	GitHub workflow `cd-prod-smoke.yml` (`writes`)	`HEARTBEAT_URL_PROD_SMOKE_WRITE` (repo secret)	every 2h (period 2h, grace 2h)	the write journey is green; red pings `<url>/fail`. See ci-cd.md

The money crons also log every failed row as a structured error ({ orgId, subscriptionId, stage, err } with a short fixed message — Lifecycle row failed, Lifecycle org pass failed, Cancellation sweep row failed), so a missed heartbeat can be traced to the row in Better Stack logs or Sentry without reading free text.

Known constraints (free tier)

Outbound chat channels (Discord/Slack/webhooks) are paywalled on the free plan — free alerting is email + mobile-app push only. There’s no free way to route Better Stack alerts into Discord (would need a paid tier or a custom incidents poller).
Better Stack free also has no incoming webhooks, so there’s no free bridge for Sentry errors into this hub — Sentry errors stay in Sentry (its own email/mobile alerts).

Responding

Uptime monitor down → confirm the service is actually unreachable, then see incident-response.md. For an apex/DNS outage specifically, check the Cloudflare apex record.
Backup heartbeat missed → the nightly backup failed or the cron stopped. Check the pg-s3-backups service in Railway and whether a fresh object landed in the R2 bucket; then restore-from-backup.md.
Scheduler alive missed → no API replica is consuming the scheduled-jobs queue: CRONS_ENABLED unset, Redis unreachable, or every replica down. scheduled-jobs.md.
A money heartbeat missed while scheduler alive is green → the job ran and failed, or was locked out. bull-board shows the run; the structured … failed log names the org and subscription. For the lifecycle tick a failed row is retried next hour, for the cancellation sweep the next day — a member whose notice matured keeps being billed until then, so do not wait for the retry.
Prod smoke /fail → incident-response.md; start at the smoke job’s own log.

Setup tasks still pending

Create the four heartbeats marked above (member billing catch-up, subscription lifecycle, cancellation sweep, entitlement drift) in Better Stack and set their HEARTBEAT_URL_* variables on the Railway @taikan/api service. Until then those pings are silent no-ops and the jobs are covered only by Sentry.
Install the Better Stack mobile app and confirm push alerts are enabled (the real-time channel, since Discord is paywalled on free).
(Optional) If Discord alerts are wanted, move the backup heartbeat to healthchecks.io (free, Discord-capable) by repointing HEALTHCHECK_URL — no migration, uptime stays on Better Stack.