Carnice-9B-SFT-Fable5-GGUF

Carnice-9B-SFT-Fable5-GGUF

GGUF quantizations of Carnice-9B-SFT-Fable5, for local inference with llama.cpp, Ollama, or LM Studio.

Overview

Available quantizations

Sizes above are the actual file sizes in this repository, not estimates.

Quick start

Ollama

ollama pull hf.co/ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF:Q4_K_M

llama.cpp

llama-cli -hf ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF:Q4_K_M -p "Tell me a fable about a clever fox." -n 256

Download a single file

from huggingface_hub import hf_hub_download

path = hf_hub_download("ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF", "carnice-9b-sft-fable5.q4_k_m.gguf")
print(path)

Limitations

• Quantization is lossy: outputs will not match the source model exactly, and the effect grows as the quant level drops.
• Primarily English training data.
• Not safety-tuned. Use appropriate guardrails.
• Research and prototyping, not production without further evaluation.

Citation

@misc{azarkhalili2026_carnice_9b_sft_fable5_gguf,
    author = {Azarkhalili, Behrooz},
    title = {Carnice-9B-SFT-Fable5-GGUF},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/ermiaazarkhalili/Carnice-9B-SFT-Fable5-GGUF}
}

Acknowledgments

• llama.cpp for the GGUF format and tooling
• Unsloth and Hugging Face TRL
• Compute Canada / DRAC for HPC resources