Downloads · 30 days
24
19% of all-time downloads
gratex/Mistral-Small-3.2-24B-Instruct-2506-sbvr-compliance-unresolvable-pattern-finetune-v1
Mistral-Small-3.2-24B-Instruct-2506-sbvr-compliance-unresolvable-pattern-finetune-v1 is a machine learning model from gratex. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This model is a fine-tuned Mistral-Small-3.2-24B-Instruct-2506 designed to improve the evaluation of SBVR (Semantics of Business Vocabulary and Business Rules) rules. Its primary objective is to enhance the accuracy o…
Downloads · 30 days
24
19% of all-time downloads
All-time downloads
125
Public
Parameters
24B
48 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors48 GB · 100%
From the Hugging Face model README
This model is a fine-tuned Mistral-Small-3.2-24B-Instruct-2506 designed to improve the evaluation of SBVR (Semantics of Business Vocabulary and Business Rules) rules. Its primary objective is to enhance the accuracy of rule status assignment among three possible verdicts: PASSED, FAILED, and UNRESOLVABLE. Specifically, the model is trained to distinguish between: PASSED — the input context fully satisfies the rule's condition; FAILED — the input context demonstrates that the rule's condition is violated; UNRESOLVABLE — the available context is insufficient to reach a definitive conclusion. A key motivation for this fine-tuning is to mitigate a known limitation of the base model, which tends to make unsupported assumptions in the absence of clear context, defaulting to PASSED or FAILED even when the evidence is insufficient. The fine-tuned model is therefore conditioned to recognize such cases and correctly return UNRESOLVABLE rather than committing to an unsupported verdict.
Artifact type: merged-hf
Base model: mistralai/Mistral-Small-3.2-24B-Instruct-2506
Internal name: mistral-24b-lora-compliance-unresolvable-600from2030-status-attn-i180_LR5e6_LR16_LA16_LD0.05-merged-hf
Finetune alias: Mistral-Small-3.2-24B-Instruct-2506-sbvr-compliance-unresolvable-pattern-finetune-v1
| Param | Value |
|---|---|
| iters | 180 |
| learning_rate | 5e-06 |
| lora_rank | 16 |
| lora_alpha | 16 |
| lora_dropout | 0.05 |
| lora_targets | attention (q/k/v/o) |
| warmup_steps | 20 |
| max_seq_length | 1024 |
| batch_size | 1 |
| grad_accumulation_steps | 8 |
| seed | 42 |
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained('gratex/Mistral-Small-3.2-24B-Instruct-2506-sbvr-compliance-unresolvable-pattern-finetune-v1')
tokenizer = AutoTokenizer.from_pretrained('gratex/Mistral-Small-3.2-24B-Instruct-2506-sbvr-compliance-unresolvable-pattern-finetune-v1')
Comparison against the base model (mistralai/Mistral-Small-3.2-24B-Instruct-2506)
across HELM benchmark suites.
| Suite | Base | Finetuned | Δ | % Change |
|---|---|---|---|---|
| unresolvable_patterns | 0.7870 | 0.9092 | +0.1222 | +15.5% |
| quac | 0.5760 | 0.5997 | +0.0237 | +4.1% |
| mmlu_pro:history | 0.5879 | 0.6063 | +0.0184 | +3.1% |
| truthful_qa | 0.6667 | 0.6835 | +0.0168 | +2.5% |
| mmlu_pro:chemistry | 0.3070 | 0.3150 | +0.0080 | +2.6% |
| mmlu_pro:engineering | 0.3313 | 0.3385 | +0.0072 | +2.2% |
| mmlu_pro:computer_science | 0.5317 | 0.5390 | +0.0073 | +1.4% |
| mmlu_pro:health | 0.6394 | 0.6455 | +0.0061 | +1.0% |
| mmlu_pro:business | 0.3447 | 0.3498 | +0.0051 | +1.5% |
| mmlu_pro:math | 0.3400 | 0.3450 | +0.0050 | +1.5% |
| synthetic_reasoning | 0.9216 | 0.9340 | +0.0124 | +1.3% |
| bbq | 0.8950 | 0.9060 | +0.0110 | +1.2% |
| mmlu_pro:economics | 0.6434 | 0.6457 | +0.0024 | +0.4% |
| commonsense | 0.8260 | 0.8280 | +0.0020 | +0.2% |
| insurance_claim_extraction | 0.6402 | 0.6414 | +0.0012 | +0.2% |
| mmlu_pro:physics | 0.3270 | 0.3280 | +0.0010 | +0.3% |
| damaged_items_mc | 1.0000 | 1.0000 | +0.0000 | +0.0% |
| boolq | 0.8560 | 0.8510 | -0.0050 | -0.6% |
| mmlu_pro:biology | 0.7657 | 0.7643 | -0.0014 | -0.2% |
| mmlu_pro:other | 0.5519 | 0.5498 | -0.0022 | -0.4% |
| mmlu_pro:psychology | 0.6955 | 0.6855 | -0.0100 | -1.4% |
| mmlu_pro:law | 0.4300 | 0.4220 | -0.0080 | -1.9% |
| mmlu_pro:philosophy | 0.5271 | 0.5130 | -0.0140 | -2.7% |
| civil_comments | 0.6585 | 0.6220 | -0.0366 | -5.6% |
Trained on the dataset of 598 stratified examples from 13 unresolvable compliance pattern categories (all UNRESOLVABLE status).
Funded by the EU NextGenerationEU through the Recovery and Resilience Plan for Slovakia under the project No. 09I02-03-V01-00029.