Data leak between tenants suspected
Each tenant (one account’s data) shares the database with others. On most work, Postgres Row-Level Security (RLS) hides every tenant’s rows from every other tenant. Some work skips RLS and filters by tenant in its own query. Back to the runbook index.
This is a SEV-1. Follow Escalation alongside this page.
Symptom
Section titled “Symptom”- A tenant reports seeing another tenant’s data.
- Or a support ticket includes data from a different tenant than the operator’s session.
A transaction opened for one tenant (postgres.TxTenant) sets
app.tenant_id, and RLS shows it only that tenant’s rows. A leak means one of these failed:
- A table holding tenant data has RLS turned off.
- A query ran with the wrong
app.tenant_id. - A query ran with
app.bypass_rlson (postgres.TxBypass), which skips RLS, and its own tenant filter was wrong or missing. Background jobs and sign-in use this mode. So do two areas a signed-in user works in for one tenant: Stripe credential settings and team members and invites. Check every area on the full list.
-
Contain it first (Escalation, SEV-1 step 1). Keep Postgres up for forensics.
-
Check that RLS is on for every tenant-scoped table:
SELECT schemaname, tablename, rowsecurityFROM pg_tables WHERE schemaname = 'public' AND rowsecurity = falseORDER BY tablename;You see: only tables that hold no tenant data. Anything unexpected here means isolation is broken.
-
Check the leaked query. Was it run with
app.tenant_idset correctly? Wasapp.bypass_rlsinvolved?
- Patch the code path that bypassed RLS.
- Search
audit_logfor the affected pair of tenants. - Notify both customers, and run a breach review.
Verify
Section titled “Verify”The query in Check step 2 returns only the tables you expect, and the code path that leaked now returns only the caller’s tenant’s rows.