User Guide# Backends llama.cpp transformers vLLM SGLang MLX Client API LLM Embedding Image Audio Rerank Authentication System (database-backed) Enabling / disabling Initial admin account Resetting a lost admin password Secrets and storage locations Permissions Usage Managing users and API keys Http Status Code Behavior notes Feedback OIDC Single Sign-On Configuration Login flow User provisioning and permissions Audit Logging and Security Audit logging Brute-force protection Model Launching Instructions Replica GPU Allocation Strategy Set Environment Variables Configuring Model Virtual Environment Batching / Continuous Batching Thinking Mode Metrics Supervisor Metrics Worker Metrics Distributed Inference Supported Engines Usage Continuous Batching Usage Abort your request Note Xavier: Share KV Cache between vllm replicas Usage Limitations
