Downloads · 30 days
6
5% of all-time downloads
v0.1mistral_lora_sft
ZK0622/v0.1mistral_lora_sft
v0.1mistral_lora_sft is a machine learning model from ZK0622. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Mistral 7b finetuned via 2 stage fine tunning (DAPT & SFT) with Hierarchical Merging and PEFT method (LoRA) on legal documents for the purpose of Legal Docs Summarization
All-time downloads
123
Public
Repo size
1.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1.2 GB · 83%
At a glance
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Aug 21, 2025
- Updated
- Mar 14, 2026
- SHA
- 2ee2df78
Base models
Datasets
Model card
From the Hugging Face model README
Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->Model Details
Model Description
Mistral 7b finetuned via 2 stage fine tunning (DAPT & SFT) with Hierarchical Merging and PEFT method (LoRA) on legal documents for the purpose of Legal Docs Summarization
- Developed by: [ZK]
- Model type: [Decoder Only]
- Finetuned from model: [Mistral 7b]
Model Sources
<!-- Provide the basic links for the model. -->- Repository: [mistralai/Mistral-7B-v0.3]
- Paper: [https://arxiv.org/pdf/2310.06825]
Training Details
Contact owner for training details.
Training Data
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->[Only Data for SFT available "ZK0622/SFT"]
Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->Testing Data, Factors & Metrics
Testing Data
<!-- This should link to a Dataset Card if possible. -->[TO-BE-UPDATED]
Factors
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->[TO-BE-UPDATED]
Metrics
<!-- These are the evaluation metrics being used, ideally with a description of why. -->[TO-BE-UPDATED]
Results
PERPLEXITY
| Model | Perplexity (Lower Better) |
|---|---|
| Mistral 7B Instruct v0.2 | 4.29 |
| AdaLoRA Mistral 7B | 3.73 |
| LoRA Mistral 7B | (This Model) 3.54 |
ROUGE
| Transformer Architecture | Model Name | Rouge-1 | Rouge-2 | Rouge-L | Rouge-L Sum |
|---|---|---|---|---|---|
| Encoder | Legal BERT | 0.374 | 0.185 | 0.218 | 0.218 |
| Encoder-Decoder | Legal LED | 0.306 | 0.113 | 0.169 | 0.169 |
| Encoder-Decoder | Legal T5 | 0.268 | 0.084 | 0.167 | 0.167 |
| Decoder | Mistral 7B Base | 0.440 | 0.195 | 0.269 | 0.273 |
| Decoder | AdaLoRA Mistral 7B | 0.506 | 0.294 | 0.350 | 0.351 |
| Decoder (This Model) | LoRA Mistral 7B | 0.564 | 0.376 | 0.437 | 0.437 |
| Transformer Architecture | Model Name | Precision | Recall | BERT-F1 |
|---|---|---|---|---|
| Encoder | Legal BERT | 0.800 | 0.857 | 0.827 |
| Encoder-Decoder | Legal LED | 0.798 | 0.845 | 0.821 |
| Encoder-Decoder | Legal T5 | 0.674 | 0.820 | 0.739 |
| Decoder | Mistral 7B Base | 0.852 | 0.866 | 0.858 |
| Decoder | AdaLoRA Mistral 7B | 0.874 | 0.884 | 0.878 |
| Decoder (This Model) | LoRA Mistral 7B | 0.908 | 0.890 | 0.898 |
Summary
Framework versions
- PEFT 0.10.0
- Transformer 4.40
- DeepSpeed config: ZeRO Stage 2
- Library
- peft
- License
- apache-2.0
- Languages
- en
- Created
- Aug 21, 2025
- Updated
- Mar 14, 2026