postmortem.io
Menu

802 entries newest

Browse

Incident

Incident Report: SERVER_ROOT_PATH regression broke UI routing

Read the original on the company blog (external)
Contents · 4 sections

Incident Report: SERVER_ROOT_PATH regression broke UI routing

LiteLLM incident report. Published 2026-02-21.

Date: January 22, 2026 Duration: ~4 days (until fix merged January 26, 2026) Severity: High Status: Resolved

Note: This fix is available starting from LiteLLM v1.81.3.rc.6 or higher.

Summary

A PR (#19467) accidentally removed the root_path=server_root_path parameter from the FastAPI app initialization in proxy_server.py. This caused the proxy to ignore the SERVER_ROOT_PATH environment variable when serving the UI. Users who deploy LiteLLM behind a reverse proxy with a path prefix (e.g., /api/v1 or /llmproxy) found that all UI pages returned 404 Not Found.

  • LLM API calls: No impact. API routing was unaffected.
  • UI pages: All UI pages returned 404 for deployments using SERVER_ROOT_PATH.
  • Swagger/OpenAPI docs: Broken when accessed through the configured root path.

Background

Many LiteLLM deployments run behind a reverse proxy (e.g., Nginx, Traefik, AWS ALB) that routes traffic to LiteLLM under a path prefix. FastAPI’s root_path parameter tells the application about this prefix so it can correctly serve static files, generate URLs, and handle routing.

The root_path parameter was present in proxy_server.py since early versions of LiteLLM. It was removed as a side effect of PR #19467, which was intended to fix a different UI 404 issue.

Root cause

PR #19467 (73d49f8) removed the root_path=server_root_path line from the FastAPI() constructor in proxy_server.py:

 app = FastAPI(
     docs_url=_get_docs_url(),
     redoc_url=_get_redoc_url(),
     title=_title,
     description=_description,
     version=version,
-    root_path=server_root_path,
     lifespan=proxy_startup_event,
 )

Without root_path, FastAPI treated all requests as if the application was mounted at /, causing path mismatches for any deployment using SERVER_ROOT_PATH.

The regression went undetected because:

  1. No automated test verified that root_path was set on the FastAPI app.
  2. No manual test procedure existed for SERVER_ROOT_PATH functionality.
  3. Default deployments (without SERVER_ROOT_PATH) were unaffected, so most CI tests passed.

Remediation

Original source · company blog

Incident Report: SERVER_ROOT_PATH regression broke UI routing

https://docs.litellm.ai/blog/server-root-path-incident

This page is an extract. Cite the original; link here for the indexed copy.

LiteLLM. "Incident Report: SERVER_ROOT_PATH regression broke UI routing." LiteLLM blog, 22 Jan 2026. https://docs.litellm.ai/blog/server-root-path-incident (retrieved 2026-10-22 via postmortem.io)