User Guide# Backends llama.cpp transformers vLLM SGLang MLX Speculative decoding Client API Replica management Xinference Client OpenAI Client OpenAI Client Tool Calls Anthropic Client Xinference Client OpenAI Client Xinference Client OpenAI Client Xinference Client OpenAI Client Speaker Embeddings SD WebUI API compatibility Installation Generation ControlNet and ADetailer LoRA Querying and cancelling Authentication System (database-backed) Enabling / disabling Initial admin account Resetting a lost admin password Secrets and storage locations Permissions Usage Managing users and API keys Http Status Code Behavior notes Feedback OIDC Single Sign-On Configuration Login flow User provisioning and permissions Audit Logging and Security Audit logging Brute-force protection Model Launching Instructions Download without launching Replica GPU Allocation Strategy Set Environment Variables Configuring Model Virtual Environment Batching / Continuous Batching Thinking Mode Launch Configuration History Metrics Supervisor Metrics Worker Metrics Distributed Inference Supported Engines Usage Continuous Batching Usage Abort your request Note Xavier: Share KV Cache between vllm replicas Usage Limitations PD separation Launch CLI launch Configuration rules Inference and lifecycle Verification Hybrid/recurrent attention limitation V1 supported configurations and cache safety
