Self-host LiteLLM Proxy with Docker

One OpenAI-format gateway in front of 100+ model providers, with keys, budgets, and spend tracking. This page describes the Levelrail template for LiteLLM Proxy (AI). It deploys the services below as one Docker Compose app on your own server. Project site: https://docs.litellm.ai/docs/simple_proxy. Recommended memory: about 1024 MiB.

Services, ports and volumes

Service	Image	Container ports	Volumes
`db`	`postgres:16-alpine`	none	`litellm_db_data -> /var/lib/postgresql/data`
`litellm`	`ghcr.io/berriai/litellm:main-stable`	4000	none

Environment variables

Service	Variable	Value
`db`	`POSTGRES_DB`	Preset in the template
`db`	`POSTGRES_PASSWORD`	Generated at deploy
`db`	`POSTGRES_USER`	Preset in the template
`litellm`	`DATABASE_URL`	Generated at deploy
`litellm`	`LITELLM_MASTER_KEY`	Generated at deploy
`litellm`	`STORE_MODEL_IN_DB`	Preset in the template

Passwords and keys marked as generated are created for you when the app is deployed and stored as secrets. Values are not shown here.

Deploy LiteLLM Proxy with Levelrail

In the dashboard, open Apps, choose New app, then Browse templates, and select LiteLLM Proxy. Review the Compose body and deploy. With the CLI:

levelrail-cli templates deploy litellm --name my-litellm

See the service template catalog for how templates work, and Getting started if you have not installed Levelrail yet.

More AI templates

All templates are listed in the self-host gallery.

Gallery

AI

Analytics

Applications

Automation

Communication

Dashboard

Database Tools

Developer Tools

Finance

Infrastructure

IoT

Media

Monitoring

Productivity

Security

Starter Kits

Storage

Self-host LiteLLM Proxy with Docker

Services, ports and volumes

Environment variables

Deploy LiteLLM Proxy with Levelrail

More AI templates

​Services, ports and volumes

​Environment variables

​Deploy LiteLLM Proxy with Levelrail

​More AI templates

Services, ports and volumes

Environment variables

Deploy LiteLLM Proxy with Levelrail

More AI templates