In this step we create our first model. You can either use promptranker to help you find the best labels and hypothesis or skip this step with your preferred setting.
Select Action: CREATE
Select your dataset from the dropdown
Choose a model type
symanto_fast (for demonstration purposes, live demo, etc.)
symanto (for actual model training with a purpose)
Select or unselect the multi_label option (select it if the text classification task allows the prediction of multiple labels per text instance. Otherwise, the task is considered as multi-class (or binary if only 2 labels are categorized).
Select the language of the text
Choose an embedding model
Choose symanto_brain_multilingual only if there is no specific language model for your selected language
Select the number of labels for your analysis
As a brief reminder, when defining the text classification task, we need to associate a label text with the label name that we are intending to categorize. This label text, also referred sometimes as hypothesis or prompt, provides a semantic context to the model.
If you want to continue without using Promptranker:
Provide label names, e.g. POSITIVE, NEGATIVE and label texts, e.g. GREAT, BAD. You can include your hypothesis in the label text too, such as THE TEXT IS GREAT or THE TEXT IS BAD
Enable “Inspect typical examples” to see how your model performs
Use the slider to select up to 10 examples.
Change the label dropdown to see examples for each of your labels.
Click ‘Create’
If you want to continue using Promptranker
This will run an unsupervised statistical Bayesian process that will end up outputting a reliability score of the best combination from the provided ones. It is recommended to use that combination for your model, both at zero and few-shot level.
Enable “Rank label texts”
Provide label names, e.g. POSITIVE, NEGATIVE
For each label name, provide a list of alternative descriptions. For example, for positive, you can provide GREAT WONDERFUL POSITIVE and for negative, BAD TERRIBLE NEGATIVE
Provide hypothesis variations, e.g. use an empty one and add several different variations such as THIS TEXT IS {}, THE AUTHOR EXPRESSES A {} SENTIMENT , …
Click “Rank”
The result is a table where each variation is ranked according to a calculated score. The higher the score, the more likely the given labels and hypothesis yield good results in a real scenario.
Enable “Inspect typical examples” to see how your model performs
Use the slider to select up to 10 examples.
Change the label dropdown to see examples for each of your labels.
Click create. The model is automatically being created with the prompts (labels and hypothesis) that scored the top.