TiTan-Llama-3.2-1B

TiTan-Llama-3.2-1B

A fine-tuned Llama 3.2 1B model, specialized in generating short conversation titles and relevant tags.

Model Details

This model is a fine-tuned version of unsloth/Llama-3.2-1B using the Unsloth framework with LoRA (Low-Rank Adaptation) for efficient training.

• Developed by: theprint
• Model type: Causal Language Model (Fine-tuned with LoRA)
• Language: en
• License: apache-2.0
• Base model: unsloth/Llama-3.2-1B
• Fine-tuning method: LoRA with rank 128

Intended Use

Title and tag generation.

Training Details

Training Data

The titles-n-tags set was specifically created for finetuning models on titling and tagging. This was done by generating two titles and two sets of tags for each entry, and using a combination of a judge LLM and a scoring algorithm to determine the winning combination.

• Dataset: theprint/titles-n-tags-alpaca
• Format: alpaca

Training Procedure

• Training epochs: 2
• LoRA rank: 128
• Learning rate: 0.0002
• Batch size: 4
• Framework: Unsloth + transformers + PEFT
• Hardware: NVIDIA RTX 5090

Usage

from unsloth import FastLanguageModel
import torch

# Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="theprint/TiTan-Llama-3.2-1B",
    max_seq_length=4096,
    dtype=None,
    load_in_4bit=True,
)

# Enable inference mode
FastLanguageModel.for_inference(model)

# Example usage
inputs = tokenizer(["Your prompt here"], return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Alternative Usage (Standard Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "theprint/TiTan-Llama-3.2-1B",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("theprint/TiTan-Llama-3.2-1B")

# Example usage
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Your question here"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

GGUF Quantized Versions

Quantized GGUF versions are available in the gguf/ directory for use with llama.cpp:

• TiTan-Llama-3.2-1B-f16.gguf (2364.7 MB) - 16-bit float (original precision, largest file)
• TiTan-Llama-3.2-1B-q3_k_m.gguf (658.8 MB) - 3-bit quantization (medium quality)
• TiTan-Llama-3.2-1B-q4_k_m.gguf (770.3 MB) - 4-bit quantization (medium, recommended for most use cases)
• TiTan-Llama-3.2-1B-q5_k_m.gguf (869.3 MB) - 5-bit quantization (medium, good quality)
• TiTan-Llama-3.2-1B-q6_k.gguf (974.5 MB) - 6-bit quantization (high quality)
• TiTan-Llama-3.2-1B-q8_0.gguf (1259.9 MB) - 8-bit quantization (very high quality)

Using with llama.cpp

# Download a quantized version (q4_k_m recommended for most use cases)
wget https://huggingface.co/theprint/TiTan-Llama-3.2-1B/resolve/main/gguf/TiTan-Llama-3.2-1B-q4_k_m.gguf

# Run with llama.cpp
./llama.cpp/main -m TiTan-Llama-3.2-1B-q4_k_m.gguf -p "Your prompt here" -n 256

Limitations

May provide incorrect information.

Citation

If you use this model, please cite:

@misc{titan_llama_3.2_1b,
  title={TiTan-Llama-3.2-1B: Fine-tuned unsloth/Llama-3.2-1B},
  author={theprint},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/theprint/TiTan-Llama-3.2-1B}
}

Acknowledgments

• Base model: unsloth/Llama-3.2-1B
• Training dataset: theprint/titles-n-tags-alpaca
• Fine-tuning framework: Unsloth
• Quantization: llama.cpp