2025H2-DPO-BAI-FHNW

fhnw/2025H2-DPO-BAI-FHNW

Model Summary

This repository contains a Direct Preference Optimization (DPO) variant of the GPT-OSS 20B family, fine-tuned using Unsloth QLoRA for FHNW.

Exported format(s): GGUF, MXFP4.

• Base model: GPT-OSS 20B
• Variant: Direct Preference Optimization (DPO)
• Purpose: preference-aligned generation
• Created: 2025-11-24

---

Intended Use

Suitable for research, teaching, and applied Generative AI experimentation at FHNW. Not intended for high-risk or safety-critical decision-making.

---

Usage

GGUF with llama.cpp

git lfs install
git clone https://huggingface.co/fhnw/2025H2-DPO-BAI-FHNW
cd fhnw/2025H2-DPO-BAI-FHNW
./main -m model.gguf -p "Explain what GPT-OSS is."

---

MXFP4 with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("fhnw/2025H2-DPO-BAI-FHNW")
model = AutoModelForCausalLM.from_pretrained("fhnw/2025H2-DPO-BAI-FHNW", torch_dtype="auto")

input_ids = tokenizer("Explain GPT-OSS.", return_tensors="pt")
print(tokenizer.decode(model.generate(**input_ids)[0], skip_special_tokens=True))

---

Training Details

The model was fine-tuned using Unsloth QLoRA and exported using Unsloth's merged MXFP4 or native GGUF export pipeline, depending on the selected format(s).

---

Limitations

• May hallucinate or provide outdated information.
• Inherits all limitations of the GPT-OSS 20B base model.
• Human review is strongly recommended.

---

License

Follows the respective licenses of GPT-OSS 20B and Unsloth.