Downloads · 30 days
7
27% of all-time downloads
MistrAND-7B-v1
jgchaparro/MistrAND-7B-v1
MistrAND-7B-v1 is a machine learning model from jgchaparro. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
All-time downloads
26
Public
Repo size
865 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors865 MB · 100%
At a glance
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- May 15, 2024
- Updated
- May 25, 2024
- SHA
- 2efa5edd
Base models
Model card
From the Hugging Face model README
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
MistrAND-7B-v1
MistrAND 7B v1 is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the OASST2 dataset converted to a custom Andalusian Spanish orthography.
This project is part of the Master's Degree Final Thesis titled Conservational AI for endangered languages: a preservation strategy for Tsakonian Greek upon the Andalusian Spanish case, aiming to preserve endangered languages by storing them in QLoRA adapters for unlimited use in the future.
Links
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 2
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1
- num_epochs: 1
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.9639 | 0.1796 | 1000 | 0.8628 |
| 0.753 | 0.3593 | 2000 | 0.7898 |
| 0.713 | 0.5389 | 3000 | 0.7348 |
| 0.6756 | 0.7185 | 4000 | 0.6888 |
| 0.6665 | 0.8981 | 5000 | 0.6530 |
Framework versions
- PEFT 0.10.1.dev0
- Transformers 4.41.0.dev0
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
- Library
- peft
- License
- apache-2.0
- Created
- May 15, 2024
- Updated
- May 25, 2024