Release Notes#
Xinference 4.0 overview#
The v4.0.0 community release expands cache reuse, heterogeneous deployment, and runtime model management. The highlights below summarize the major changes; detailed notes for individual versions follow this overview.
Multi-engine Xavier and P/D. SGLang and MLX replicas gain prefix KV cache sharing. P/D deployments can combine vLLM, SGLang, and MLX, including NVIDIA and Apple silicon workers. Native NIXL backends are available for vLLM and SGLang; vLLM Xavier adds direct GPU handoff and tiered history. See Xavier: Share KV Cache between vllm replicas and PD separation.
Reconfigure without reloading weights. Opt-in GPU weight caching lets vLLM and SGLang rebuild the engine with updated supported parameters while retaining the weights. Current requests drain before the change, and the model UID stays the same. See Reconfigure without reloading weights.
Distributed vLLM V1 inference. Supported models can combine tensor parallelism and pipeline parallelism across workers through Xinference’s xoscar executor, without a separate Ray cluster. See Distributed Inference.
OpenAI-compatible API additions. A stateless Responses API supports clients using
/v1/responses. vLLM, SGLang, and MLX also expose reused prompt-token counts in usage responses when reported by the engine. See OpenAI Client Responses API and Client API.Unified logging. The Web UI Log Center provides recent runtime logs and, with Elasticsearch configured, historical indexed logs. Request IDs connect API, audit, and runtime events. See Logging in Xinference and Audit Logging and Security.
Installation and local services. The cross-platform installer and service commands simplify setup and local service management on Linux, macOS, and Windows. Engine dependencies continue to install on demand in per-model virtual environments. See One-command installation and Model Launching: Toggle Virtual Environments and Customize Dependencies.
If upgrading from a version earlier than 3.0, read Migrating to Xinference 3.0+ for authentication, Web UI, and container deployment changes. Check each feature guide for its engine and hardware requirements.
Detailed version notes#
This page provides a version-by-version index of Xinference release notes. For detailed updates, please visit the corresponding links below.
Version |
Release Notes |
|---|---|
v4.0.0 |
|
v3.5.0 |
|
v3.4.0 |
|
v3.3.0 |
|
v3.2.0 |
|
v3.1.0 |
|
v3.0.0 |
|
v2.12.0 |
|
v2.11.0 |
|
v2.10.0 |
|
v2.9.0 |
|
v2.8.0 |
|
v2.7.0 |
|
v2.5.0 |
|
v2.4.0 |
|
v2.3.0 |
|
v2.2.0 |
|
v2.1.0 |
|
v2.0.0 |
|
v1.17.0 |
|
v1.16.0 |
|
v1.15.0 |
|
v1.14.0 |
|
v1.13.0 |
|
v1.12.0 |
|
v1.11.0.post1 |
|
v1.10.1 |
For older versions and source history, see our GitHub releases page: xorbitsai/inference