xinference.client.handlers.GenerateModelHandle.generate

GenerateModelHandle.generate(prompt: str, generate_config: PytorchGenerateConfig | None = None) → Completion | Iterator[CompletionChunk]

Creates a completion for the provided prompt and parameters via RESTful APIs.

参数:

• prompt (str) -- The user's message or user's input.

• generate_config (Optional["PytorchGenerateConfig"]) -- Additional configuration for the chat generation. "PytorchGenerateConfig" -> Configuration for pytorch model

返回:

Stream is a parameter in generate_config. When stream is set to True, the function will return Iterator["CompletionChunk"]. When stream is set to False, the function will return "Completion".

返回类型:

Union["Completion", Iterator["CompletionChunk"]]

抛出:

RuntimeError -- Fail to generate the completion from the server. Detailed information provided in error message.