Downloads · 30 days
8
1% of all-time downloads
round-bird/georgia-sports-llama3-v1
georgia-sports-llama3-v1 is a text generation model from round-bird. Use it when you need the model to write or continue text. The card lists the license as mit.
A fine-tuned version of Meta Llama 3.1 8B Instruct, trained with Direct Preference Optimization (DPO) on Georgia high school sports content from GPB Sports.
Downloads · 30 days
8
1% of all-time downloads
All-time downloads
675
Public
Parameters
8.2B
5.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.7 GB · 100%
How the weights are stored.
U87.2B · 87%
From the Hugging Face model README
A fine-tuned version of Meta Llama 3.1 8B Instruct, trained with Direct Preference Optimization (DPO) on Georgia high school sports content from GPB Sports.
This model is designed to answer questions about Georgia high school athletics — football, basketball, baseball, and more — with accurate, well-sourced responses grounded in real sports journalism.
Direct Preference Optimization is a technique for aligning language models with human preferences. Instead of just training a model on "correct" answers, DPO shows the model pairs of responses — one chosen (better) and one rejected (worse) — and teaches it to prefer the style and quality of the better response.
Think of it like a coach reviewing game film: "This play was good, this one wasn't — learn the difference."
We scraped ~1,350 articles from GPB Sports covering Georgia high school athletics from April 2022 through September 2025.
For each article, a local LLM (Mistral 7B) read the article and generated 3 challenging questions that could be answered using the article's content. This produced 4,078 question-answer prompts.
Two different local models answered each question independently:
Both models received the full article text as context and generated their best answer.
A judge model (Llama 3.1 8B) scored each response on a 1–5 scale for accuracy, specificity, and faithfulness to the source article. The higher-scoring response became the chosen response; the lower-scoring one became the rejected response. Ties were discarded.
The 4,078 chosen/rejected pairs were used to fine-tune Llama 3.1 8B Instruct using:
| Parameter | Value |
|---|---|
| Base model | meta-llama/Llama-3.1-8B-Instruct |
| Method | DPO with QLoRA |
| Quantization | 4-bit NormalFloat (NF4) |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| LoRA target modules | q_proj, k_proj, v_proj, o_proj |
| Learning rate | 5e-5 |
| Batch size | 2 (with 4× gradient accumulation) |
| Epochs | 3 |
| DPO beta | 0.1 |
| Max sequence length | 1024 |
| Optimizer | Paged AdamW 8-bit |
| Hardware | Google Colab T4 16GB |
| Stat | Value |
|---|---|
| Total DPO pairs | 4,078 |
| Source articles | ~1,350 |
| Date range | April 2022 – September 2025 |
| Topic | Georgia high school sports (GHSA) |
| Avg chosen response length | 525 characters |
| Avg rejected response length | 467 characters |
| Model | Times Chosen (Better) | Times Rejected (Worse) |
|---|---|---|
| Llama 3.1 8B | 2,349 (57.6%) | 1,729 (42.4%) |
| Mistral 7B Instruct | 1,729 (42.4%) | 2,349 (57.6%) |
Chosen responses:
| Rating | Count |
|---|---|
| 5 | 1,203 |
| 4 | 2,759 |
| 3 | 109 |
| 2 | 7 |
Rejected responses:
| Rating | Count |
|---|---|
| 4 | 1,079 |
| 3 | 1,420 |
| 2 | 1,560 |
| 1 | 19 |
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "kslote/georgia-sports-llama3-dpo"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map="auto")
messages = [
{"role": "system", "content": "You are a knowledgeable Georgia high school sports analyst."},
{"role": "user", "content": "Who were the top football programs in Georgia in 2024?"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)
The full DPO training dataset is available at kslote/georgia-high-school-sports.