Production & Scaling
The app is stateless; the metadata Postgres is the only stateful component and the only thing you back up. Everything below follows from that: scale the app horizontally, make the database HA, and treat ENCRYPTION_KEY/JWT_SECRET as recovery-critical secrets.
Secrets management
Never bake secrets into the image or commit a populated .env. Use the platform-native store:
| Platform | Secret store | Wiring |
|---|---|---|
| Kubernetes (any) | Sealed Secrets, External Secrets Operator, Vault | Reference via secrets.existingSecret in Helm values |
| AWS ECS | Secrets Manager / SSM Parameter Store | secrets block in task definition |
| Azure Container Apps | Built-in secret store, Key Vault references | Mount as env vars |
| GCP Cloud Run | Secret Manager | --update-secrets on deploy |
| Bare Docker | Docker secrets, HashiCorp Vault, sops-encrypted .env | Mount the file and source it |
Horizontal scaling
The application is stateless, sessions live in JWTs, no in-memory caches that need affinity. Scale horizontally by increasing replica count.
| Component | Scaling |
|---|---|
| App pods/tasks | Stateless, scale up/down freely. Migrations should run as a separate one-shot, not on every pod boot, to avoid races. |
| Postgres metadata DB | Vertical scaling on a managed service is the easy path. Read replicas help only at significant scale (1000s of concurrent users), most installations don't need them. |
| Connection pool (per pod) | 20 connections by default. With N pods, total connections to Postgres = N × 20. Raise Postgres max_connections or front it with PgBouncer if you exceed the database limit. |
Resource sizing
Sensible defaults for a typical install:
| Scale | App CPU / mem | Postgres | Replicas |
|---|---|---|---|
| 1-20 users | 0.5 vCPU / 1 GB | db.t3.small (or 2 vCPU / 4 GB) | 1 |
| 20-100 users | 1 vCPU / 2 GB | db.t3.medium | 2 |
| 100-500 users | 2 vCPU / 4 GB | db.m5.large + read replica | 3-5 with HPA |
| 500+ users | 4 vCPU / 8 GB | db.m5.xlarge + read replica + PgBouncer | HPA on CPU 60% |
The "users" figure is concurrent active users. Most installations are far smaller than they expect.
Observability
- Logs, structured JSON to stdout. Ship to your log-aggregation or observability system. Search by
requestIdto trace a single request end-to-end. - Health,
GET /api/healthreturns503 STARTINGuntil the metadata DB is reachable, then200 OK;GET /api/health/migrationsreports appliedschema_migrationsrows for deploy verification. Wire both to LB/ingress checks and CD gates. - Metrics,
GET /api/metricsexposes aggregated counters backing the in-app dashboard. For request-rate / error-rate alerting, drive your observability stack from the structured logs. - Audit log, every
POST/PUT/PATCH/DELETEis recorded in theaudit_logstable with sensitive fields auto-redacted. Query it for compliance reports.
Backup and restore
Every relevant piece of state is in the db_devops Postgres database. Back that up and you have a restorable system.
- Managed Postgres: use the provider's native automated backups + point-in-time restore. AWS RDS / Azure / Cloud SQL all support 7-35 day retention.
- Self-hosted Postgres: run nightly
pg_dumpand ship to off-host storage. Test the restore at least quarterly. - Encrypted columns: stored database passwords, MFA secrets, repo credentials. They're encrypted with the
ENCRYPTION_KEY. Back upENCRYPTION_KEYwherever you store secrets, losing it makes those columns unrecoverable. - License keys: stored in the database. Re-activatable from the original key file Simcha sent you, so a database restore alone is sufficient.
Disaster recovery
Standard pattern for HA / DR:
- Run app pods/tasks across at least two AZs (Multi-AZ in the orchestrator).
- Run Postgres Multi-AZ with synchronous replication (the managed services handle this).
- Take regular snapshots of the Postgres volume for point-in-time restore.
- Back up
ENCRYPTION_KEYandJWT_SECRETto your secret store with versioning enabled. - Document the runbook to recover. Drill it.
Security hardening
Beyond what the in-app docs cover (HTTPS, MFA, RBAC, encryption-at-rest), at the platform level:
- Image scanning, scan your image with Trivy / Snyk / your registry's built-in scanner. Block deploys on critical CVEs.
- Network policies, restrict egress from the app pod to: Postgres, identity provider (if SSO), repo provider (if integrating with GitHub/GitLab), and the license activation endpoint at
licensing.simchasolutions.com. Block everything else. - NetworkPolicy / Security groups, Postgres should only accept connections from the app's security group / namespace.
- Pod Security Standards, the chart's default security context is compatible with the restricted profile.
- Audit log retention, set
audit_retention_dayshigh enough for your compliance needs (default 90).
Upgrades
- Read the release notes before upgrading. Migrations may be additive (safe to roll forward) or breaking (require maintenance window).
- Snapshot Postgres before any non-patch upgrade.
- Pin to specific versions (
:1.0.1), never:latestin production. - Test upgrades in staging first. The full Compose stack reproduces the production shape.