Downloads · 30 days
13
16% of all-time downloads
llm-course-hw3-dora
sodeniZz/llm-course-hw3-dora
llm-course-hw3-dora is a text generation model from sodeniZz. Use it when you need the model to write or continue text. It is set up for transformers.
This model was fine-tuned as part of Homework 3 in the HSE LLM Course. It applies a custom DoRA implementation for sentiment classification, using standard causal language modeling training.
All-time downloads
80
Public
Parameters
300M
1.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- mistral
- Access
- Public
- Created
- Dec 8, 2025
- Updated
- Dec 8, 2025
- SHA
- 880d3f97
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
Model Card: llm-course-hw3-dora
This model was fine-tuned as part of Homework 3 in the HSE LLM Course.
It applies a custom DoRA implementation for sentiment classification, using standard causal language modeling training.
The model predicts a sentiment label (negative, neutral, or positive) by generating a short textual output conditioned on the input text.
Model Sources
- Base model: https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct
- Dataset: https://huggingface.co/datasets/cardiffnlp/tweet_eval
Training Method
Training follows a standard causal LM setup with frozen backbone weights and trainable DoRA adapters inserted into attention projection layers.
Training Hyperparameters
- PEFT method: DoRA (custom implementation)
- Rank: 8
- DoRA alpha: 16
- Target modules: attention projections (
k_proj,v_proj) - Batch size: 32
- Learning rate: 5e-4
- Optimizer: AdamW
- Precision: FP16 (mixed precision)
- Epochs: 3
Trainable parameters: ~0.14% of total model parameters.
Results
- Macro F1: ~0.5 on the test set
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- Languages
- en
- Created
- Dec 8, 2025
- Updated
- Dec 8, 2025