User Guide# Backends llama.cpp transformers vLLM SGLang MLX Speculative decoding Client API Replica management Xinference Client OpenAI Client OpenAI Client Tool Calls Anthropic Client Xinference Client OpenAI Client Xinference Client OpenAI Client Xinference Client OpenAI Client Speaker Embeddings Authentication System (database-backed) Enabling / disabling Initial admin account Resetting a lost admin password Secrets and storage locations Permissions Usage Managing users and API keys Http Status Code Behavior notes Feedback OIDC Single Sign-On Configuration Login flow User provisioning and permissions Audit Logging and Security Audit logging Brute-force protection Model Launching Instructions Download without launching Replica GPU Allocation Strategy Set Environment Variables Configuring Model Virtual Environment Batching / Continuous Batching Thinking Mode Metrics Supervisor Metrics Worker Metrics Distributed Inference Supported Engines Usage Continuous Batching Usage Abort your request Note Xavier: Share KV Cache between vllm replicas Usage Limitations
