> For the complete documentation index, see [llms.txt](https://utca.knowledgator.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://utca.knowledgator.com/tasks/transformersvisualqanda.md).

# TransformersVisualQandA

Visual Q\&A task

Subclass of [**Task**](/tasks/task.md)**.** 

## Module: [implementation](/framework-structure.md#implementation).tasks

## Default predictor

This task uses [**TransformersModel**](/predictors/transformers-predictors.md#transformersmodel) by default with this configuration:

{% code overflow="wrap" fullWidth="false" %}

```python
model = AutoModelForImageClassification.from_pretrained(
    "dandelin/vilt-b32-finetuned-vqa"
)
predictor=TransformersModel(
    TransformersModelConfig(
        model=model
    ),
    input_class=TransformersImageModelInput,
    output_class=TransformersLogitsOutput,
)
```

{% endcode %}

#### See:

* [**TransformersModel**](/predictors/transformers-predictors.md#transformersmodel) 
* [**TransformersModelConfig**](/predictors/transformers-predictors.md#transformersmodelconfig)
* [**TransformersImageModelInput**](/predictors/transformers-schemas.md#transformersimagemodelinput)
* [**TransformersLogitsOutput**](/predictors/transformers-schemas.md#transformerslogitsoutput)

## Methods and properties

Main methods and properties

***

### <mark style="color:blue;">\_\_init\_\_</mark>

#### Arguments:

* <mark style="color:orange;">**predictor**</mark>**&#x20;(**[**Predictor**](/predictors/predictor.md)**\[Any, Any], optional):** Predictor that will be used in task. If equals to None, [**default predictor**](#default-predictor) will be used. Defaults to None.
* <mark style="color:orange;">**preprocess**</mark>**&#x20;(Optional\[**[**Component**](/core/component.md)**], optional):** Component executed before predictor. If equals to None, default component will be used. Defaults to None.\
  \
  Default component: \
  [**VisualQandAPreprocessor**](#visualqandapreprocessor)\
  \
  If default chain is used, [**VisualQandAPreprocessor**](#visualqandapreprocessor) will use **ViltProcessor** from model used in **predictor**.
* <mark style="color:orange;">**postprocess**</mark>**&#x20;(Optional\[**[**Component**](/core/component.md)**], optional):** Component executed after predictor. If equals to None, default component will be used. Defaults to None.\
  \
  Default component: \
  [**VisualQandASingleAnswerPostprocessor**](#visualqandasingleanswerpostprocessor)\
  \
  If default chain is used, [**VisualQandASingleAnswerPostprocessor**](#visualqandasingleanswerpostprocessor) will use labels from model used in **predictor**.
* <mark style="color:orange;">**input\_class**</mark>**&#x20;(Type\[**[**Input**](/core/schemas.md#input)**], optional):** Class for input validation. Defaults to [**TransformersVisualQandAInput**](/predictors/transformers-schemas.md#transformersvisualqandainput)**.**
* <mark style="color:orange;">**output\_class**</mark>**&#x20;(Type\[**[**Output**](/core/schemas.md#output)**], optional):** Class for output validation. Defaults to [**TransformersVisualQandAOutput**](#transformersvisualqandaoutput)**.**
* <mark style="color:orange;">**name**</mark>**&#x20;(Optional\[str], optional):** Name for identification. If equals to None, class name will be used. Defaults to None.

***

***

***

## <mark style="color:green;">TransformersVisualQandAOutput</mark>

Subclass of [**IOModel**](/core/schemas.md#iomodel)**.**

***

### <mark style="color:blue;">\_\_init\_\_</mark>

#### Arguments:

* <mark style="color:orange;">**answer**</mark>**&#x20;(Optional\[Tuple\[str, float]])**

***

***

***

## <mark style="color:green;">TransformersVisualQandAMultianswerOutput</mark>

Subclass of [**IOModel**](/core/schemas.md#iomodel)**.**

***

### <mark style="color:blue;">\_\_init\_\_</mark>

#### Arguments:

* <mark style="color:orange;">**answers**</mark>**&#x20;(Dict\[str, float])**

***

***

***

## <mark style="color:green;">VisualQandAPreprocessor</mark>

Prepare model input. Subclass of [**Action**](/core/action.md). Type of [**Action**](/core/action.md)**\[Dict\[str, Any], Dict\[str, Any]].**

***

### <mark style="color:blue;">\_\_init\_\_</mark>

#### Arguments:

* <mark style="color:orange;">**processor**</mark>**&#x20;(Processor):** Feature extractor.
* <mark style="color:orange;">**name**</mark>**&#x20;(Optional\[str], optional):** Name for identification. If equals to None, class name will be used. Defaults to None.

***

### <mark style="color:blue;">execute</mark>

#### Arguments:

* <mark style="color:orange;">**input\_data**</mark>**&#x20;(Dict\[str, Any]):** \
  Expected keys:
  * <mark style="color:red;">**"image"**</mark> **(Image.Image):** Image to analyze;
  * <mark style="color:red;">**"question"**</mark> **(str):** Question to answer;

#### Returns:

* **Dict\[str, Any]:** \
  Expected keys:
  * <mark style="color:red;">**"input\_ids"**</mark>**&#x20;(Any);**
  * <mark style="color:red;">**"token\_type\_ids"**</mark>**&#x20;(Any);**
  * <mark style="color:red;">**"attention\_mask"**</mark>**&#x20;(Any);**
  * <mark style="color:red;">**"pixel\_values"**</mark>**&#x20;(Any);**
  * <mark style="color:red;">**"pixel\_mask"**</mark>**&#x20;(Any);**

***

***

***

## <mark style="color:green;">VisualQandASingleAnswerPostprocessor</mark>

Process model output. Subclass of [**VisualQandAMultianswerPostprocessor**](#visualqandamultianswerpostprocessor).

***

### <mark style="color:blue;">execute</mark>

#### Arguments:

* <mark style="color:orange;">**input\_data**</mark>**&#x20;(Dict\[str, Any]):** \
  Expected keys:
  * <mark style="color:red;">**"logits"**</mark>**&#x20;(Any):** Model output;

#### Returns:

* **Dict\[str, Any]:** \
  Expected keys:
  * <mark style="color:red;">**"answer"**</mark>**&#x20;(Optional\[Tuple\[str, float]]):** Answer with highest score, if score higher or equal to threshold, else - None.

***

***

***

## &#x20;<mark style="color:green;">VisualQandAMultianswerPostprocessor</mark>

Process model output. Subclass of [**Action**](/core/action.md). Type of [**Action**](/core/action.md)**\[Dict\[str, Any], Dict\[str, Any]].**

***

### <mark style="color:blue;">\_\_init\_\_</mark>

#### Arguments:

* <mark style="color:orange;">**labels**</mark>**&#x20;(List\[str]):** Labels for classification.
* <mark style="color:orange;">**threshold**</mark>**&#x20;(float):** Labels threshold score. Defaults to 0.
* <mark style="color:orange;">**name**</mark>**&#x20;(Optional\[str], optional):** Name for identification. If equals to None, class name will be used. Defaults to None.

***

### <mark style="color:blue;">execute</mark>

#### Arguments:

* <mark style="color:orange;">**input\_data**</mark>**&#x20;(Dict\[str, Any]):** \
  Expected keys:
  * <mark style="color:red;">**"logits"**</mark>**&#x20;(Any):** Model output;

#### Returns:

* **Dict\[str, Any]:** \
  Expected keys:
  * <mark style="color:red;">**"answers"**</mark>**&#x20;(Dict\[str, float]):** Classified labels and scores.

***

***