Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this step, we create our first model. You can either use promptranker to help you find the best labels and hypothesis hypotheses or skip this step with your preferred setting.

...

Panel
panelIconId1f4bb
panelIcon:computer:
panelIconText💻
bgColor#FFF0B3

API Docs: https://api.symanto.net/active-learning/docs#/v2/create_model_v2__post

Set up your model

  1. Select

    Status
    titleCreate New
    from the side navigation

  2. Select your dataset from the dropdown

  3. Select the language of the text

  4. Choose a model type

    1. symanto_fast (for demonstration purposes, live demo, etc.)

    2. symanto (for actual model training with a purpose)

  5. Status
    Select or unselect
    colourBlue
    titleOptional
    Select the multi_label option (select it if the text classification task allows the prediction of multiple labels per single text instance. Otherwise, the task is considered as multi-class (or binary if only 2 labels are categorized). Select the language of the text

  6. Choose an embedding model

    • Choose symanto_brain_multilingual only if there is no specific language model for your selected langauge

    Select the number of labels for your analysis
    • language

  7. Status
    colourBlue
    titleOptional
    Add a model name or keep the automatically assigned one

...

Define your labels

Info

As a brief reminder, when defining the text classification task, we need to associate a label text (the label description) with the label name (the actual label) that we are intending to categorizecategorise. This label text, also referred sometimes as hypothesis or prompt, provides a semantic context to the model. 

...

  1. Expland the 'Check or modify labels '

...

2. Select the number of labels you want to have using the slider

3. Continue with/ without using Promptranker

Expand
titleIf you want to continue without using Promptranker:

3. Provide label names, e.g.

Status
colourGreen
title

...

JOY
,
Status
colourRed
title

...

SaD
and

...

label texts (hypotheses), e.g.

Status
colourGreen
title

...

Happy, AmmUSED
,
Status
colourRed
title

...

UNhappy, DISAPPOINTED
. You can include your

...

label name in the label text too, such as

Status
colourGreen
titleThe

...

person FEELS JOY
or
Status
colourRed
titlethe

...

PERSON FEELS SAD

Image Added
Info

Please note: Hypotheses could also be long and detailed explanations of your labels, e.g. if your labels come from a specific framework you can use the framework’s description of each label with/ or multiple descriptors (adjectives/ nouns for more clarity).

When formulating long hypotheses always think about what would help another person to understand my label better in order to end up with the most clear and understandable hypothesis.

2.

Status
colourBlue
titleOptional
Enable ‘Inspect typical examples’ to see how your model performs

Image Added

2.1. Use the slider to select up to 10 examples.

2.2. Change the label dropdown to see examples for

...

each of your labels.

2.3. Change the assigned label if necessary

3. Click

...

Create

Expand

...

titleIf you want to continue using Promptranker:
Panel
panelIconId1f4bb
panelIcon:computer:
panelIconText💻
bgColor#FFF0B3

API Docs: https://api.symanto.net/active-learning/docs#/v2/rank_label_texts_v2_datasets_rank_label_text_post

Info

This will run an unsupervised statistical Bayesian process that will end up outputting a reliability score of the best combination from the provided ones. It is recommended to use that combination for your model, both at zero and few-shot level.

  1. Enable

...

  1. ‘Rank label

...

  1. texts’

  2. Provide label names, e.g.

    Status
    titlepositive
    ,
    Status
    titlenegative

    1. For each label name, provide a list of alternative descriptions. For example, for positive, you can provide

      Status
      titlegreat
      Status
      titlewonderful
      Status
      titlepositive
      and for negative,
      Status
      titlebad
      Status
      titleterrible
      Status
      titlenegative

  3. Provide hypothesis variations, e.g. use an empty one and add several different variations such as

    Status
    titleThis TExt is {}
    ,
    Status
    titleThe author expresses a {} sentiment
    , …

...

Image Added

...

Enable “Inspect typical examples” to see how your model performs

  1. Use the slider to select up to 10 examples.

  2. Change the label dropdown to see examples for every of your labels.

...

The result is a table where each variation is ranked according to a calculated score. The higher the score, the more likely the given labels and hypothesis yield good results in a real scenario.

...

Image Added

4. Click Create and the model will be automatically created with the prompts (labels and hypothesis) that scored the top.

...

...

Next:

...

Train

...

your model using active learning