Simple OAuth2 System (experimental)#

Xinference builds an In-memory OAuth2 authentication and authorization system using the account-password mode.

Note

If you don’t have authentication and authorization requirements, you can use Xinference as before, without any changes.

Permissions#

Currently, Xinference system internally defines some interface permissions:

  • models:list: Permission to list models and get models’ information.

  • models:read: Permission to use models.

  • models:register: Permission to register custom models.

  • models:unregister: Permission to unregister custom models.

  • models:start: Permission to launch models.

  • models:stop: Permission to stop running models.

  • admin: Administrators have permissions for all interfaces.

Startup#

All authentication and authorization information needs to be specified and loaded into memory when Xinference is started. Xinference requires a JSON-formatted file with the following specific fields:

{
    "auth_config": {
        "algorithm": "HS256",
        "secret_key": "09d25e094faa6ca2556c818166b7a9563b93f7099f6f0f4caa6cf63b88e8d3e7",
        "token_expire_in_minutes": 30
    },
    "user_config": [
        {
            "username": "user1",
            "password": "secret1",
            "permissions": [
                "admin"
            ],
            "api_keys": [
                "sk-72tkvudyGLPMi",
                "sk-ZOTLIY4gt9w11"
            ]
        },
        {
            "username": "user2",
            "password": "secret2",
            "permissions": [
                "models:list",
                "models:read"
            ],
            "api_keys": [
                "sk-35tkasdyGLYMy",
                "sk-ALTbgl6ut981w"
            ]
        }
    ]
}

  • auth_config: This field is used to configure security-related information.

    • algorithm: The algorithm used for token generation and parsing. HS series algorithms are recommended. For example, HS256, HS384 or HS512.

    • secret_key: The secret_key used for token generation and parsing. Use this command to generate the secret_key adapted to the HS algorithms: openssl rand -hex 32.

    • token_expire_in_minutes: Reserved field indicating the expiration time of the token. The current open-source version of Xinference does not check the expiration time of tokens.

  • user_config: This field is used to configure user and permission information. Each user information is composed of these fields:

    • username: string field for username.

    • password: string field for password.

    • permissions: A list containing strings representing the permissions that this user has. The permissions are described as above.

    • api_keys: A list containing strings representing the api-keys of this user. With these api-keys, user can access the xinference interfaces without the need to signin. The api-key here is formatted similar to the OPENAI_API_KEY , always starting with sk-, followed by 13 alphanumeric characters.

Once you have configured such a JSON file, use the --auth-config option to enable Xinference with the authentication and authorization system. For example, for local startup:

xinference-local -H 0.0.0.0 --auth-config /path/to/your_json_config_file

For distributed startup, just specify this option when starting the supervisor:

xinference-supervisor -H <supervisor_ip> --auth-config /path/to/your_json_config_file

Usage#

For Xinference with the authentication and authorization system enabled, all usage remains the same, except for the addition of a login step at the beginning or using the api-key.

Signin#

Signin for command line users:

xinference login -e <endpoint> --username <username> --password <password>

For python SDK users:

from xinference.client import Client
client = Client('<endpoint>')
client.login('<name>', '<pass>')

For web UI users, when opening the web UI, you will first be directed to the login page. After logging in, you can use the web UI normally.

Api-Key#

For command line users, just add --api-key or -ak option in the command you want to use.

xinference launch <other options> --api-key <your_api_key>

For python SDK users, pass the api_key parameter when initializing the client, just like the OPENAI Python client.

from xinference.client import Client
client = Client('<endpoint>', api_key='<your_api_key>')

Xinference is also compatible with the OPENAI Python SDK as well.

from openai import OpenAI
client = OpenAI(base_url="<xinference endpoint>" + "/v1", api_key="<your_api_key>")
client.models.list()

Http Status Code#

Add the following two HTTP status codes:

  • 401 Unauthorized: login information or token verifies failed.

  • 403 Forbidden: No enough permissions when accessing interfaces.

For the command line, SDK, or web UI users, there will be clear information prompts when encountering authorization and permissions issues.

Note#

This feature is still in an experimental stage. Feel free to provide feedback on usage issues or improvement suggestions through GitHub issues or our Slack.

On this page
Show Source