Auth, Performance & Debugging

JWT/RBAC/tier gating, rate-limit behavior, the in-memory migration-state gotcha, resource tuning, and the log/inspection workflow across Docker and Kubernetes.

Auth, RBAC, and Tier Gating

401 vs 403 — they mean different things

A 401 is an authentication failure: no valid JWT. extractUser attaches req.user from a HS256 token with an 8h expiry; an expired, missing, or wrong-signature token yields 401. A 403 is authorization/licensing: the user is authenticated but lacks the role (requireRole) or license tier (requireTier/requireAI) for the route, or hit enforceConnectionLimit. Do not chase a 403 as a login bug.

Rotating JWT_SECRET locks everyone out

Changing JWT_SECRET invalidates every previously issued token — signatures no longer verify, so all sessions 401 at once. This is expected; plan rotation as a coordinated re-login event. Symmetrically, if two replicas run with different JWT_SECRET values behind a load balancer, sessions flap 401 depending on which replica serves the request. Ensure the secret is identical across all instances.

Tier 403s map to the license, not the account

Route gating: IDE/connection/query/migration routes need Starter+; /api/db-devops/* needs Professional+; audit and /api/auth/idp need Enterprise; /api/ai needs the AI flag. A 403 on a ChangeOps route from a valid admin is a tier problem — verify /api/license/status. Remember the 60s license cache: immediately after an upgrade, a 403 can persist for up to a minute until the cache refreshes.

Connection limit reached

Symptom: Creating a connection returns 403 with a limit message on Starter. Cause: enforceConnectionLimit blocks creation past the tier's maxConnections (Starter caps at 10). Resolution: remove unused connections or upgrade the tier.

Rate-Limit 429s

Symptom: Bursts of 429 Too Many Requests, often during scripted or automated access. Cause: two windows apply — general API at GENERAL_RATE_LIMIT_MAX (default 2000) per 15 min, and the higher-throughput ChangeOps routes at DB_DEVOPS_RATE_LIMIT_MAX (default 5000) per 1 min. Resolution: throttle the client, or raise the relevant knob for known-good high-volume automation. If a load balancer collapses client IPs, the limiter sees one aggregate source and trips early — preserve the real client address (e.g. X-Forwarded-For) so limits apply per-client.

In-Flight Migrations Vanish on Restart

Symptom: Environments or migrations created through /api/migrations disappear after a backend restart. Cause: MigrationService holds its state in-memory, not in a repository — despite the persistence layers that exist elsewhere. Resolution: this is expected today; do not treat it as data loss. The ChangeOps routes (/api/db-devops/scripts, /executions, /approvals) do persist through their repositories — use those for anything that must survive a restart, and do not conflate the two subsystems.

Performance

Query fast at the engine, slow in the IDE

Confirm the plan at the source (EXPLAIN ANALYZE) first. If the engine is fast but the IDE is slow, the cost is result-set materialization: the browser renders rows, so 100k+ row results are render-bound — add a LIMIT. Remote engines add round-trip latency on top of execution. This is a client-side rendering cost, not backend slowness.

OOM under concurrency or large results

Symptom: Reached heap limit — JavaScript heap out of memory or the container is OOM-killed. Cause: concurrent live pools plus large materialized result sets. Resolution: raise the container memory limit (docker run -m 4g ..., or resources.limits.memory in k8s) and cap result sizes.

bcrypt CPU under login bursts

Password hashing is bcrypt at 12 rounds (~200–300ms/hash by design). Simultaneous logins queue on CPU. Give the container at least one full core (resources.requests.cpu: "1000m"). This is intentional work-factor cost, not a regression.

Logs and Direct API Probing

The backend logs structured JSON to stdout only (no log files) via Pino; pretty-printed in dev, JSON in production for shipping to an aggregator. Level is LOG_LEVEL (default info). Correlate a browser request to a backend line by its X-Request-Id (forwarded by the proxy).

docker logs -f simcha-db-studio
docker logs simcha-db-studio 2>&1 | grep -i error | tail -20

Bypass the frontend to isolate proxy-vs-Express faults:

# Unauthenticated signals
curl -s http://localhost:5002/api/health | python3 -m json.tool
curl -s http://localhost:5002/api/health/migrations | python3 -m json.tool
curl -s http://localhost:5002/api/license/status | python3 -m json.tool

# Authenticate, then exercise a gated route
TOKEN=$(curl -s -X POST http://localhost:5002/api/auth/login \\
  -H "Content-Type: application/json" \\
  -d '{"email":"admin@example.com","password":"..."}' \\
  | python3 -c "import sys,json;print(json.load(sys.stdin)['token'])")
curl -s -H "Authorization: Bearer $TOKEN" http://localhost:5002/api/db-devops/scripts | python3 -m json.tool

In non-production, X-QA-Bypass-Key and X-Agent-Key inject a virtual admin and skip auth/RBAC — both are disabled when NODE_ENV=production. If bypass headers appear to work against a "prod" deployment, NODE_ENV is misset and must be corrected immediately.

Docker Inspection

# Resolved env (confirm the target the process actually uses)
docker exec simcha-db-studio env | grep -E "DB_DEVOPS_DB_|JWT_|ENCRYPTION_KEY|CLIENT_URL|PORT|NODE_ENV"

# Health from inside the container
docker exec simcha-db-studio wget -qO- http://localhost:5002/api/health

# Container-reported health and recent probe output
docker inspect --format='{{.State.Health.Status}}' simcha-db-studio
docker inspect --format='{{range .State.Health.Log}}{{.ExitCode}} {{.Output}}{{end}}' simcha-db-studio

Note whether ENCRYPTION_KEY is present in the resolved env — its absence is the tell for the plaintext-secret fail-open described in Common Issues.

Kubernetes Inspection

kubectl get pods -l app=simcha-db-studio
kubectl logs -f -l app=simcha-db-studio --tail=100
kubectl logs -l app=simcha-db-studio --previous   # crash-loop: prior container

kubectl describe pod -l app=simcha-db-studio
# ImagePullBackOff = image/registry auth; CrashLoopBackOff = check logs;
# Pending with no events = unschedulable (resources); OOMKilled = raise memory limit

A pod that is Running but whose readiness probe (wired to /api/health) never passes is almost always stuck in STARTING on the changeops pool — debug it as a DB-reachability problem, not a k8s one.

← CORS & Connection Issues

Report an Issue →