xinference.client.Client.describe_model

Client.describe_model(model_uid: str)

Get model information via RESTful APIs.

Parameters:

model_uid (str) – The unique id that identify the model.

Returns:

A dictionary containing the following keys:

• ”model_type”: str

  the type of the model determined by its function, e.g. “LLM” (Large Language Model)

• ”model_name”: str

  the name of the specific LLM model family

• ”model_lang”: List[str]

  the languages supported by the LLM model

• ”model_ability”: List[str]

  the ability or capabilities of the LLM model

• ”model_description”: str

  a detailed description of the LLM model

• ”model_format”: str

  the format specification of the LLM model

• ”model_size_in_billions”: int

  the size of the LLM model in billions

• ”quantization”: str

  the quantization applied to the model

• ”revision”: str

  the revision number of the LLM model specification

• ”context_length”: int

  the maximum text length the LLM model can accommodate (include all input & output)

Return type:

dict

Raises:

RuntimeError – Report failure to get the wanted model with given model_uid. Provide details of failure through error message.