fine_tuned_mistral_7b_dental_8_epoch

Uploaded model

• Developed by: exillarml
• License: apache-2.0
• Finetuned from model : unsloth/mistral-7b-bnb-4bit

This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

Inference Example for Fine-tuned Mistral 7B Dental Model

This example shows how to load and run inference using your fine-tuned Mistral 7B model from Hugging Face.

🚀 Code

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch
from peft import PeftModel

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained("exillarml/fine_tuned_mistral_7b_dental_8_epoch")  # Correct repo ID (or yours)

# Quantization config for 4-bit loading (IMPORTANT for Unsloth models).
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

# Load the base model with quantization - Do NOT load normally.
model = AutoModelForCausalLM.from_pretrained(
    "unsloth/mistral-7b-bnb-4bit", # Base model ID
    quantization_config=bnb_config,
    trust_remote_code=True,
    device_map="auto"  # Let Transformers choose the best device
)
tokenizer.pad_token = tokenizer.eos_token

# Load the LoRA adapter
model = PeftModel.from_pretrained(model, "exillarml/fine_tuned_mistral_7b_dental_8_epoch")  # Correct repo ID

# Inference (no merging needed for HF Endpoints)
inputs = tokenizer("What causes gum bleeding?", return_tensors="pt").to(model.device) # Put inputs on same device as model


# No need to merge for inference endpoints, generate directly!
with torch.no_grad(): # Disable gradients for inference to save memory
    outputs = model.generate(**inputs, max_new_tokens=50)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

📝 Notes

• Make sure you have transformers and torch installed:

pip install transformers torch

• You can modify the input prompt and token limits as needed.