Downloads · 30 days
99
17% of all-time downloads
fs90/Llama-3.2-1B-Instruct-bnb-4bit-gsm8k-GGUF
Llama-3.2-1B-Instruct-bnb-4bit-gsm8k-GGUF is a text generation model from fs90. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
GGUF format quantizations for llama.cpp/Ollama.
Downloads · 30 days
99
17% of all-time downloads
All-time downloads
579
Public
Repo size
5.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf5.6 GB · 100%
From the Hugging Face model README
GGUF format quantizations for llama.cpp/Ollama.
This model uses the Llama 3.2 chat template.
{{ if .Messages }}
{{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|>
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}
You are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to format an answer to the original use question.
{{- end }}
{{- end }}<|eot_id|>
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>
{{- if and $.Tools $last }}
Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
{{ $.Tools }}
{{- end }}
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}
{{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>
{{- if .ToolCalls }}
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }}
{{- else }}
{{ .Content }}{{ if not $last }}<|eot_id|>{{ end }}
{{- end }}
{{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|>
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}
{{- end }}
{{- end }}
{{- else }}
{{- if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}{{ .Response }}{{ if .Response }}<|eot_id|>{{ end }}
For complete training configuration, see the LoRA adapters repository/directory.
Benchmarked on the merged 16-bit safetensor model
Evaluated: 2025-11-24 14:29
| Model | Type | gsm8k |
|---|---|---|
| unsloth/Llama-3.2-1B-Instruct-bnb-4bit | Base | 0.1463 |
| Llama-3.2-1B-Instruct-bnb-4bit-gsm8k | Fine-tuned | 0.3230 |
| Quantization | File | Size | Quality |
|---|---|---|---|
| F16 | Llama-3.2-1B-Instruct-bnb-4bit-gsm8k-F16.gguf | 2.31 GB | Full precision (largest) |
| Q4_K_M | Llama-3.2-1B-Instruct-bnb-4bit-gsm8k-Q4_K_M.gguf | 0.75 GB | Good balance (recommended) |
| Q6_K | Llama-3.2-1B-Instruct-bnb-4bit-gsm8k-Q6_K.gguf | 0.95 GB | High quality |
| Q8_0 | Llama-3.2-1B-Instruct-bnb-4bit-gsm8k-Q8_0.gguf | 1.23 GB | Very high quality, near original |
Usage: Use the dropdown menu above to select a quantization, then follow HuggingFace's provided instructions.
Based on unsloth/Llama-3.2-1B-Instruct-bnb-4bit and trained on openai/gsm8k. Please refer to the original model and dataset licenses.
Trained by: Your Name
Training pipeline:
Base components: