API Reference

Client

xinference.client.Client(base_url[, api_key])
xinference.client.Client.describe_model(...)	Get model information via RESTful APIs.
xinference.client.Client.get_model(model_uid)	Launch the model based on the parameters on the server via RESTful APIs.
xinference.client.Client.get_model_registration(...)	Get the model with the model type and model name registered on the server.
xinference.client.Client.launch_model(model_name)	Launch the model based on the parameters on the server via RESTful APIs.
xinference.client.Client.list_model_registrations(...)	List models registered on the server.
xinference.client.Client.list_models()	Retrieve the model specifications from the Server.
xinference.client.Client.register_model(...)	Register a custom model.
xinference.client.Client.terminate_model(...)	Terminate the specific model running on the server.
xinference.client.Client.unregister_model(...)	Unregister a custom model.

Model Handles

ChatglmCppChatModelHandle

xinference.client.handlers.ChatglmCppChatModelHandle	alias of RESTfulChatglmCppChatModelHandle
xinference.client.handlers.ChatglmCppChatModelHandle.chat(prompt)	Given a list of messages comprising a conversation, the ChatGLM model will return a response via RESTful APIs.

ChatModelHandle

xinference.client.handlers.ChatModelHandle	alias of RESTfulChatModelHandle
xinference.client.handlers.ChatModelHandle.chat(prompt)	Given a list of messages comprising a conversation, the model will return a response via RESTful APIs.
xinference.client.handlers.ChatModelHandle.generate(prompt)	Creates a completion for the provided prompt and parameters via RESTful APIs.

EmbeddingModelHandle

xinference.client.handlers.EmbeddingModelHandle	alias of RESTfulEmbeddingModelHandle
xinference.client.handlers.EmbeddingModelHandle.create_embedding(...)	Create an Embedding from user input via RESTful APIs.

GenerateModelHandle

xinference.client.handlers.GenerateModelHandle	alias of RESTfulGenerateModelHandle
xinference.client.handlers.GenerateModelHandle.generate(prompt)	Creates a completion for the provided prompt and parameters via RESTful APIs.

ImageModelHandle

xinference.client.handlers.ImageModelHandle	alias of RESTfulImageModelHandle
xinference.client.handlers.ImageModelHandle.text_to_image(prompt)	Creates an image by the input text.

AudioModelHandle

xinference.client.handlers.AudioModelHandle	alias of RESTfulAudioModelHandle
xinference.client.handlers.AudioModelHandle.transcriptions(audio)	Transcribes audio into the input language.
xinference.client.handlers.AudioModelHandle.translations(audio)	Translates audio into English.