speedway_produc_rec_finetune_llama31_8b_gguf

Speedway Product Recommendation – Llama 3.1 8B (GGUF)

This repository contains a GGUF build of a fine-tuned Llama 3.1 8B model for product recommendations, optimized for local inference with Ollama / llama.cpp.

✅ Ready to run
✅ Quantized for efficiency
✅ No Transformers or PyTorch required

---

Model Details

• Base model: Llama 3.1 8B Instruct
• Fine-tuning: Product recommendation task
• Format: GGUF (llama.cpp / Ollama compatible)
• Quantization: Q4_K_M
• Context length: Up to 8192 tokens (configurable in Ollama)

---

Files

This repo intentionally contains only the GGUF artifact for simplicity.

---

Running with Ollama

1. Download the model

python3 - << 'EOF'
import urllib.request

url = "https://huggingface.co/jmsherman/speedway_produc_rec_finetune_llama31_8b_gguf/resolve/main/meta-llama-3.1-8b-instruct.Q4_K_M.gguf"
urllib.request.urlretrieve(url, "model.gguf")
print("Downloaded model.gguf")
EOF

---

2. Create an Ollama Modelfile

cat > Modelfile << 'EOF'
FROM ./model.gguf
PARAMETER num_ctx 8192
SYSTEM "You are a product recommendation engine."
EOF

---

3. Build and run

ollama create speedway-rec -f Modelfile
ollama run speedway-rec

---

Example Prompt

Recommend 5 similar products under $300 for a customer looking at a disc brake kit.

---

Hardware Recommendations

For smaller machines, reduce context:

PARAMETER num_ctx 4096

---

License & Usage

This GGUF is provided for research and local inference use.
Please follow the licensing terms of Meta Llama 3.1 when redistributing or deploying.

---

Credits

• Base model: Meta Llama 3.1
• Fine-tuning: Unsloth
• Inference: Ollama / llama.cpp