Skip to main content
A fast, high-throughput inference server that exposes an OpenAI-compatible API for open-weight language models. This page describes the Levelrail template for vLLM (AI). It deploys the services below as one Docker Compose app on your own server. Project site: https://docs.vllm.ai. Recommended memory: about 8192 MiB.
This template reserves an NVIDIA GPU, so it needs a node with a GPU and the NVIDIA container runtime.

Services, ports and volumes

Environment variables

Passwords and keys marked as generated are created for you when the app is deployed and stored as secrets. Values are not shown here.

Security and requirements

Level: Standard. What this template needs:
  • Publishes ports directly on the host (vllm): 8000:8000.
  • Stores data in named volumes (vllm): vllm_hf_cache.
  • Reserves an NVIDIA GPU (vllm).
What Levelrail adds around it:
  • Automatic HTTPS and an ingress shield in front of every domain.
  • Named volumes can be backed up and restored.
  • Deploys are gated on the health check before traffic switches.
  • One-step rollback to the previous revision.
  • Exposure report shows what is reachable from outside.
  • Host firewall rules managed for published ports.

Deploy vLLM with Levelrail

In the dashboard, open Apps, choose New app, then Browse templates, and select vLLM. Review the Compose body and deploy. With the CLI:
See the service template catalog for how templates work, and Getting started if you have not installed Levelrail yet.

More AI templates

All templates are listed in the self-host gallery.
Last modified on November 2, 2026