Mistral-Small-24B-Instruct-2501-Dutch-GGUF

Model Card for Mistral-Small-24B-Instruct-2501-Dutch-GGUF

This is a fine-tuned version of Mistral-Small-24B-Instruct that has been specifically optimized for Dutch language understanding and generation. This model was developed by Aisk to improve Dutch language capabilities while maintaining the strong instruction-following abilities of the base model.

Model Details

Model Description

Mistral-Small-24B-Instruct-unsloth-bnb-4bit-2501-Dutch is a fine-tuned version of Mistral-Small-24B-Instruct that has been specifically optimized for Dutch language understanding and generation. This model was developed by Aisk to improve Dutch language capabilities while maintaining the strong instruction-following abilities of the base model.

The model has been quantized to 4-bit precision using BitsAndBytes (bnb) and optimized with Unsloth for efficient inference, making it more accessible for deployment on consumer hardware while preserving most of the language capabilities.

• Developed by: Aisk (Dutch website: Aisk)
• Model type: Q4, Q8 and F16 GGUF and safetensor model finetuned using PEFT.
• Language(s) (NLP): NL / Dutch, EN / English
• Finetuned from model [optional]: Mistral-Small-24B-Instruct-2501

Model Sources [optional]

2587 Dutch books have been used to train. Roughly 350 million tokens were in the dataset.

We are working on expanding this amount and improve this model even further.

Uses

Direct Use

This model is particularly well-suited for:

• Dutch language conversation and chat applications
• Dutch content generation and summarization
• Translation assistance to and from Dutch
• Question answering in Dutch
• Dutch language understanding tasks

Recommendations

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "GetAisk/Mistral-Small-24B-Instruct-unsloth-bnb-4bit-2501-Dutch"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    load_in_4bit=True
)

# Example Dutch prompt
prompt = """<s>[INST] Vertel me iets over de Nederlandse cultuur. [/INST]"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=500)
print(tokenizer.decode(output[0], skip_special_tokens=False))

Training Details

Training Data

2587 Dutch ebooks have been used. Roughly 350 million tokens.

Training Procedure

Fine tuned model using LoRa based on Unsloth.

• PEFT 0.14.0