stellialm_mini_qwen_9tasks

Public version of our 3b-model trained to perform 9 specific task in english and french with high accuracy. The benchmark bellow is done with our evaluation pipeline.

Model Fine-tune from Qwen/Qwen2.5-3B-Instruct with a specific LoRA adapter.

The prompt of our 9 specific task might follow in a next post. The goal of our team is to specialized small LLM on our clients specific needs while keeping the highest quality on general tasks.

Our 3b models compete with GPT-mini on those specific tasks.

+----------------------+---------+------------+------------+
|        qwen3b        | Overall | Team score | loads fail |
+----------------------+---------+------------+------------+
| answer_reformulation |   0.73  |    0.74    |    0.99    |
| query_reformulation  |   0.83  |    0.99    |     0      |
|    summarization     |   0.92  |    ---     |     0      |
|  keyword_extraction  |   0.78  |    ---     |     0      |
|  fill_in_generation  |   0.86  |    ---     |     0      |
|    keyword_update    |   0.66  |    0.89    |     2      |
|         gqa          |   0.81  |    0.63    |     3      |
|      true_false      |   0.77  |    ---     |     0      |
|         mcq          |   0.86  |    ---     |     0      |
|        Total         |   0.8   |    ---     |    5.99    |
+----------------------+---------+------------+------------+