What a stack is
A stack has a name, the project it belongs to, and a few settings that a root token changes withlevelrail-cli stack settings:
- Prune: whether the stack may delete resources it owns. Off unless you turn it on with
--allow-prune. - Drift policy:
warn(default) orlock, set withlevelrail-cli stack settings NAME --drift lock. See Drift. - Require approval: holds applies until another administrator approves them. See Approvals.
- Pin: when set, applies are refused except a rollback to the pinned revision. See Pin.
Repository layout
Keep one folder per stack:stack.yaml holds the Stack document. The CLI strips it before it sends the other files, and uses its project and prune fields as defaults. The other files hold the ordinary platform documents (App, Domain, and so on).
Promote a change to production with a pull request that edits infra/stacks/shop-prod/, usually the image tag. Review it like any other code change.
The loop
- Pull request: CI runs
stack planfor each changed stack. The plan is posted as a comment with counts and field paths. Environment values are never shown (see plan values). - Merge: CI runs
stack applyfor the stack. The apply creates the next revision and records the git commit and the CI run URL on it. - History: each apply is a numbered revision. You can list them, look at the documents and result of one, and compare two.
- Rollback: go back to an earlier revision if the new one is wrong.
CLI
plan, show and diff change nothing. apply writes a new revision when it changes something. plan --exit-code exits 2 when changes are pending. Run levelrail-cli stack --help for the full list, including show-revision, reject, unlock and delete.
First apply. A new stack needs a project. If the files hold a Project document (a stack init bundle does), the first apply creates that project and then the stack, provided the caller could create a project directly (the write ability). CI and stack-scoped tokens never create projects. Otherwise, or when the project is missing and cannot be created, apply and plan both answer 422 naming the project and how to create it. A first apply whose plan has zero resources (for example when the project filter dropped every document) is refused with a message listing what was dropped; no stack is created.
Placeholders. ${{ env.NAME }} is replaced in env values and in build.image. Any ${{ ... }} still present in a rendered field after that is rejected with a 422 naming the field.
Delete. stack delete --mode destroy deletes the apps the stack owns and keeps its databases and their volumes; the output lists them as data kept. Add --delete-data (API: delete_data=true on DELETE /api/v1/stacks/{name}?mode=destroy) to also delete the databases the stack owns and their data. It needs root or write:sensitive, is never the default, and is recorded in the audit entry.
Start from a template
Five starter bundles ship inside the CLI. Copy one out, fill in the variables, review the plan, then apply:
Each bundle holds
stack.yaml, the platform documents, a README.md and a SECRETS.md that lists secret names only. The variables are name (stack, project and app name), domain and image, set with --set key=value. init refuses a directory that is not empty unless you pass --force, and it never contacts the control plane. Bundles use prebuilt images, and agent-app declares one named volume (data at /data) that survives redeploys, so push your image before the first apply.
Migrate into a stack
levelrail-cli import config and levelrail-cli import platform can write a stack instead of creating things, so a migration is a pull request you review before anything runs:
stack.yaml, project.yaml, databases.yaml, apps.yaml, domains.yaml, a SECRETS.md checklist and a MIGRATION.md. The same command with import config [dir] turns a Procfile, app.json or vercel.json into the same layout.
- Secrets:
SECRETS.mdlists secret names only. The documents reference them withsecretRef, and you pass the values at apply time with--secret. A secret the source generated (anapp.jsongenerator) is marked so you create a fresh random value. - Images: apply runs prebuilt images only. Apps that build from source get
--image-repo REGISTRY/NAMESPACE/APP:latestand need that image pushed from your CI before the first apply. - Databases: they are created empty and the data is copied separately. A connection variable (such as
DATABASE_URLorREDIS_URL) that matches exactly one emitted database of its engine is written as{ from: { database: NAME, field: url } }, so the platform injects the value and nothing is set by hand. A variable that matches none or several stays a secret name inSECRETS.md. - Scheduled jobs: source cron entries become
service.jobs. A cron command that looks like it carries a credential is left out and listed inMIGRATION.mdinstead. - Not in the documents: routes and redirects are listed in
MIGRATION.md. - Proxy flags: when the source sets force-HTTPS, websocket support, a redirect target or basic auth,
MIGRATION.mdlists each as a domain policy line for the app. Basic auth shows the user name only; the source password hash is never read, so set a new password on the domain. Domain documents cannot carry these settings yet. - Name clashes: if Heroku and Vercel config both define
web, both are emitted, the second asweb-vercel.
Ownership and prune
A stack owns the resources it created, and the resources it adopted. Prune deletes only owned resources that the files no longer declare. A resource that exists outside every stack makes the apply fail unless the apply setsadopt.
Deleting a compose App or a cloud resource also needs --confirm-destroy <stack name> on stack plan and stack apply (API: confirm_destroy); the value must equal the stack name, applies only to that run, and is never implied by --yes.
Prune is never turned on by a pull request. A spec.prune field in the Stack document or a --prune flag only asks for it (an explicit --prune or --prune=false on the command line always wins over spec.prune, so a pull request cannot turn prune on over a workflow input of prune: false). The apply is refused unless an administrator set allow_prune on the stack, so editing the files in git cannot widen what an apply deletes.
Domains listed inline on an App are owned too, as Domain/<host> entries in the ledger. A second stack that declares the same host is refused with a ledger conflict. Removing a host from the files prunes it like any other owned resource, and a prune is refused when the app that carried the host now belongs to another stack. A host that was only ever declared inline before this was recorded becomes owned the next time the stack applies.
Databases and projects are never pruned, because they hold data or other stacks’ members.
Prune is also guarded against mass deletes. An apply that would delete more than APP_STACK_PRUNE_MAX_COUNT resources (default 10), or more than APP_STACK_PRUNE_MAX_PERCENT percent of the stack’s resources (default 50), is refused. Review the plan, then send allow_mass_prune with the apply request to go ahead.
Placement rules
AnApp may declare spec.service.tolerations and spec.service.affinity, with the same shape as app.yaml. A plan compares them with the stored rules of the app, and an apply saves them through the app update. A member the document leaves out is not managed and keeps its stored value. An empty tolerations: [] or affinity: {} clears the stored rules. export writes both members, so an exported file plans clean against the app it came from.
Node labels and taints
ANode document sets the labels and taints of a node that already joined, found by its display name:
taints: [] or labels: {} clears the stored set. A Node whose name matches no node, or more than one, is a plan error. Apply never creates, deletes or prunes a node, and removing the document leaves the node as it is. Nodes join through an enrollment token, and export does not write them yet. Database placement is not part of a stack, because the database API stores only a node selector.
A Node document that removes, revalues or loosens a taint whose key is listed in nodes.protected_taint_keys (case folded) is a plan error that names allow_weaken; pass --allow-weaken from a human root session to confirm it. Adding taints or making an effect stronger needs no confirmation, and a change that drops a taint fails closed when the list cannot be read.
Runtime settings
GET /api/v1/export?settings=true (root only) writes one Setting document for each saved, non-sensitive setting, and skips credential keys with a warning. The file replans clean.
A Setting document declares one runtime setting, named by its registry key:
allow_weaken; pass --allow-weaken to stack plan and stack apply (human root session only, a token gets a 403). reset: true removes the saved value, and is refused the same way when the default is weaker. Removing the document keeps the saved value and only releases it from the stack. Settings belong in an install stack.
Revisions
A revision stores the plan, the rendered documents (after${{ env.NAME }} values are filled in), the actor, the token used, the git repository, ref and commit, the CI run URL, the time, and the result of each item. Revision numbers start at 1 and increase by one for each apply to that stack.
Revisions never contain secret values. Only secret names are stored.
An apply or rollback that changes nothing does not create a revision. When the rendered documents equal the latest applied revision and live state already matches, the API answers 200 with noop: true and the existing latest revision, and the CLI prints “no changes, still at revision N”. A merge that only edits text outside the declared resources therefore leaves history alone.
If an apply stops partway, the revision is marked partial and keeps the result of each item. The next apply picks up where it stopped. There are no transactions across resources, so a partial apply can leave the stack between two states until the next apply succeeds.
Rollback
stack rollback NAME N takes the documents of revision N, plans them against live state, and applies them as a new revision. History is never rewritten, so the rollback appears as the newest revision.
Rollback restores the configuration only:
- Secret values are not rolled back. Set a secret again if the old value is needed.
- Database contents are not rolled back. A restored database definition does not undo data written since.
Approvals
Turn approval on withlevelrail-cli stack settings NAME --require-approval (root only). An apply to a stack with approval required is stored as a pending_approval revision. Nothing is applied, and the API answers 202.
Another administrator approves or rejects it. The person who requested the apply can never approve their own revision.
APP_STACK_APPROVAL_TTL (default 24h). Approving is refused with a conflict if the live state changed since the plan.
Pin
A pin freezes a stack. While it is set, an apply is refused unless it is a rollback to the pinned revision. Use it during an incident: pin the stack, change things by hand if you must, then fix the files in git and runstack unpin. The next plan shows what differs from git.
Drift
Drift is a difference between live state and the files, usually from a hand edit in the dashboard or the CLI. That includes an app’s published ports and restart policy when the files declare them. A drift checker plans each stack’s latest applied revision against live state everyAPP_STACK_DRIFT_INTERVAL (default 15m). Set it to 0 to turn the checker off. It reads live state with a read-only system token that lives APP_STACK_DRIFT_TOKEN_TTL (default 2h) and is rotated in the last quarter of that time; stale ones left by an earlier process are revoked at startup. The checker only reads. It never applies a plan or changes a resource.
- warn (default): a drifted stack is recorded and raises an attention item, shown only to people who can read that stack.
- lock: recorded the same way, and manual changes to stack-owned apps, databases, domains, projects and environments get
409 stack_lockednaming the stack. Start, stop, restart, exec, builds, backups, cache purge and read-only checks still work. To change a locked resource, change the manifest and apply, or ask an admin. A root user can override one request with the headerX-Levelrail-Override-Stack-Lock: <stack>; it is audited asstack.lock_overrideand CI or stack-scoped tokens cannot use it. If the ledger lookup fails, the request is allowed and a warning is logged. Forgetting or destroying a stack unlocks its resources.
Tokens in CI
Create one token per job through the tokens API, setting the stack and the verbs it needs:- A plan token, with
stackset to the stack andstack_verbsset toplan. - An apply token, with
stackset to the stack andstack_verbsset toapply.
--secret, create new projects, or change environment protection need a human token instead. A stack token also cannot create or update a Pipeline, because pipeline steps run commands with the app’s secrets; apply those with a human token. An apply token can still change an app’s image, so treat it like deploy access to the stack’s apps. Approvals need a root token, so a CI token cannot approve.
Store each token as a CI secret, and give the apply token only to the protected branch and environment that deploys.
Narrow verbs
Three more verbs each implyapply and must be granted by a root user:
Granting a verb on the wrong scope is refused with 400, for tokens and for OIDC trusts alike. A trust mints these verbs only when it lists them, and it is held to the same rules as an apply trust. A stack token can never weaken anything: no
confirm_weakening, no reset, no sensitive key, no adopting a node. Taint keys listed in the runtime setting nodes.protected_taint_keys (seeded by APP_NODES_PROTECTED_TAINT_KEYS, empty by default) are protected: a token cannot remove one, change its value or loosen its effect. Shortening that list is itself weakening.
Values per environment
levelrail-cli apply --values values/prod.yaml (also stack plan and stack apply) reads a flat YAML map (IMAGE_TAG: v1.4.2) and fills the matching ${{ env.NAME }} placeholders. Repeat the flag and the later file wins. Precedence is --var, then --values, then --var-file, then names allowed with --allow-env. Only scalar values are accepted, and only placeholders the manifests use are read. Keys that look like credentials (*TOKEN*, *SECRET*, *_KEY*) are refused, so use --secret for those; values are never printed. A missing value fails before anything is sent and names the placeholder.
Plan values
A human plan prints~ path: before -> after for each changed field. Env vars, anything under secrets and keys that look like credentials (*TOKEN*, *SECRET*, *PASSW*, *_KEY*) show (hidden) instead of the value. Other values are cut at APP_IAC_PLAN_VALUE_MAX characters (default 120) and end in .... stack plan -o json carries the same trimmed values.
Placeholders also work in replicas, resources.*, domains[], tolerations[].value and labels.*. The file is checked against the schema after the values are filled in, so replicas: "${{ env.REPLICAS }}" must render to a whole number. A wrong value fails with the field path and the variable name, never the value itself.
Secrets
Manifests name secrets, they never contain values. Refer to a secret by name, for example{ secretRef: DB_PASSWORD }, and pass its value at apply time from the CI secret store:
--var or allowed from the environment with --allow-env, as described in platform as code.
To point at a secret that is already in the store, add existing: true: { secretRef: DB_PASSWORD, existing: true }. The plan fails if the app has no secret with that name. Apply never writes, overwrites or deletes it, even when a value is passed with --secret.
GitHub Action
The stack Action inintegrations/github-action/stack plans on pull requests and applies on merge. The workflow, the secrets rule and the exit 2 meaning in apply mode are on IaC in CI.
Status
Stacks are experimental in this release, behindAPP_EXPERIMENTAL=iac. The file format, the revision fields and the CLI flags can still change. Pin your CLI to the same version as the control plane, and read the changelog before you upgrade either.