Phase status
Phase 5 (chaos testing, upgrade and downgrade testing across versions, security review, a published idle-footprint number, a shadow-running period) is the open work. Some of it has started: control-plane death survival, break-glass agent reconnect and a first live ACME run are covered, see resilience and the ACME runbook.Shipped
Deploy path
- App spec (
app.yamlordeploy.yaml) parsing with an embedded JSON Schema. See the app spec reference. - Docker Engine API wrapper, with no
dockerCLI shelling. - Application reconciler with readiness probes gating every cutover and liveness probes on each pass. A container that OOM-kills or exits during the readiness wait fails immediately (
OOMKilledDuringReadiness,ExitedDuringReadiness) instead of retrying a dead address. - BuildKit builds with live build-log streaming over SSE. Railpack auto-detection, static sites (
build.type: static), and dedicated build nodes with a registry-backed cache (a built-in registry can be enabled in Settings). - Docker Compose as a deploy target for a supported subset of
compose.yaml, and multi-service apps (deploy-spec) with per-service builds and deploys. - Deploy strategies: rolling, recreate, blue-green. Rollback to retained images, deploy history, deploy comparison, and deterministic build-failure diagnosis.
- Pre and post-deploy hooks. A failing pre-deploy hook blocks cutover and keeps the old container serving.
- Git providers: GitHub (App flow, including GitHub Enterprise Server), GitLab, Bitbucket Cloud and Gitea, with HMAC-verified webhooks and webhook delivery history with replay. See git integrations.
- Preview environments per pull request (opt-in), with optional ephemeral databases and PR status comments. See deploy previews.
- Embedded Caddy ingress: automatic TLS (internal issuer by default, public ACME toggleable), zero-config
sslip.iohostnames, per-domain bring-your-own certificates. See domains and ingress. - Envelope-encrypted secrets with master-key rotation. See security.
- Container exec and an interactive terminal (dashboard or
apps exec --interactive), gated to root-level tokens. - Service template catalog: 311 entries (see template catalog), reversing the original non-goal in ADR 015.
Data
- Eight managed database engines: Postgres, Redis, MySQL, MongoDB, MariaDB, KeyDB, Dragonfly and ClickHouse, with scheduled backups, restore, verification, metrics and log search. See managing databases.
- Point-in-time restore for Postgres (opt-in WAL archiving, restore to a timestamp inside the recoverable window), restore into a new resource, and app volume backups to S3-compatible targets. See backups and storage.
- TLS on newly created Postgres and Redis databases. Databases created before this feature are not switched.
- Control-plane backups. See control-plane backup.
Observability
- Node-local metrics (15s resolution) and log store with full-text and JSON search, federated across nodes, and a Prometheus remote-read endpoint. Deploy markers appear on metric charts.
- Crashloop detection with the last 200 log lines surfaced in the UI.
- Alerting over 14 rule kinds and 18 notification channel kinds, with retries for transient failures and delivery history.
- Attention list, status page, capacity forecast, right-sizing suggestions and certificate renewal visibility. See observability.
Multi-node
- Agent transport: in-process for a single node, gRPC with mTLS for remote nodes, with reconnection and version negotiation.
- Node registry, join tokens, cordon, drain, heartbeats and connection history.
- Manual placement, moving an app with its volumes, WireGuard mesh with cross-node service DNS, and distributed certificate storage.
Identity and access
- Multi-user accounts with TOTP and recovery codes, OIDC sign-in, scoped API tokens, named roles (admin, operator, viewer), and IAM-style resource-scoped policies. See identity and access.
- Organizations, projects and environments with layered shared env vars, protected environments, two-person deploy approvals, environment promotion and cloning.
- Audit log with search, CSV export and retention.
- Team invites, rate limiting, security headers (HSTS is opt-in).
- A theauth-library auth path exists behind default-off flags. See auth engine.
Operator surfaces
- Dashboard,
levelrail-cli(profiles,--outputand--query, shell completion, device login,doctor,apps create --interactive) andinstall.sh. - Migration CLI:
levelrail-cli migrate coolify,dokployandcaprover. See migrating from Coolify, Dokploy and CapRover. - MCP server (
cmd/levelrail-mcp): 156 tools in full mode, 146 in the default standard mode, 118 in read-only mode. See MCP tool surface and the AI assistant page. Tokens are checked against the same abilities as the REST API.
Shipped but off by default or beta
These exist and are tested, but are gated or carry a beta label. See feature status and experimental features.- In-app AI chat, AI models on GPU nodes, load balancer, platform as code, Cloudflare Tunnel (all behind
APP_EXPERIMENTAL). - Pipelines, feature flags, deploy freeze, supply chain scanning, log archive, scheduled deploys, deploy cost estimate.
Open work
- Real public ACME: one live run against Let’s Encrypt on a public VPS is recorded in the runbook. Renewal close to expiry, DNS-01 wildcards and a deliberate failure against the production CA are not yet exercised live.
- Multi-node on real hosts: the WireGuard mesh and cross-host remote transport are unverified outside fakes and local Docker daemons.
- Idle footprint on Linux: measured only on an Apple M4 Max dev build so far. See performance.
- Template coverage: a sample of the 311 templates is deployed by e2e tests, not the whole catalog.
- Other database engines: PITR is Postgres only, and TLS by default is Postgres and Redis only.
- Stable release: none yet. Upgrade and downgrade testing across versions with real data is part of Phase 5.
Out of scope
- Container runtime: Docker Engine API only.
- Scheduler: no bin-packing, affinity rules or autoscaling.
- Service mesh: WireGuard plus DNS only.
- Windows or non-Linux nodes.
- Managed cloud offering: not before the self-hosted version has real users.
- AI in the reconciliation path: AI is a read-and-suggest layer on top of the API.
- Kubernetes compatibility: no CRDs, no custom orchestration standard.
See also
Feature status
Maturity labels and the evidence behind each one.
Multi-node deployment
Add and manage more than one server.