TransformersChat

Basic chat task

Subclass of Task.

Module: implementation.tasks

Default predictor

This task uses TransformersModel by default with this configuration:

TransformersPipeline(
    TransformersPipelineConfig(
        task="text-generation", 
        model="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
        kwargs={
            "max_new_tokens": 256, 
            "do_sample": True, 
            "temperature": 0.3, 
            "top_k": 50, 
            "top_p": 0.95,
        }
    ),
    input_class=TransformersBasicInput,
    output_class=TransformersBasicOutput
)

See:

• TransformersPipeline
• TransformersPipelineConfig
• TransformersBasicInput
• TransformersBasicOutput

Methods and properties

Main methods and properties

---

__init__

Arguments:

• predictor (Predictor[Any, Any], optional): Predictor that will be used in task. If equals to None, default predictor will be used. Defaults to None.
• messages (Optional[str], optional): Key to use to access memory for messages. If equals to None, a unique key will be generated. Defaults to None.

Note: This parameter will function as expected only when using the default preprocessor and postprocessor.

• preprocess (Optional[Component], optional): Component executed before predictor. If equals to None, default component will be used. Defaults to None. Default component: ChatPreprocessor | GetMemory | ChatAddContext | SetMemory | RenameAttribute
• postprocess (Optional[Component], optional): Component executed after predictor. If equals to None, default component will be used. Defaults to None. Default component: ChatPostprocessor | GetMemory | ChatUpdateContext | SetMemory
• input_class (Type[Input], optional): Class for input validation. Defaults to ChatInput.
• output_class (Type[NEROutputType], optional): Class for output validation. Defaults to ChatOutput.
• name (Optional[str], optional): Name for identification. If equals to None, class name will be used. Defaults to None.

---

---

---

ChatPreprocessor

Create message template for input prompt. Subclass of Action. Type of Action[Dict[str, Any], Dict[str, Any]].

---

execute

Arguments:

• input_data (Dict[str, Any]): Expected keys:

  • "prompt" (str): Input prompt;

Returns:

• Dict[str, Any]: Expected keys:

  • "messages" (Iterable[ChatCompletionMessageParam]): Input messages;

---

---

---

ChatPostprocessor

Process API output. Subclass of Action. Type of Action[Dict[str, Any], Dict[str, Any]].

---

execute

Arguments:

• input_data (Dict[str, Any]): Expected keys:

  • "output" (List[Dict[str, Any]]): Expected keys:

    • "generated_text" (str);

Returns:

• str: Response message.

---

---