llama-3.2-3b-thinking

---

license: apache-2.0

Claude-Inspired LLaMA Finetune

Overview

This repository contains a fine-tuned version of LLaMA-3.2-3B, trained on a dataset of Claude 3.5 Sonnet generated examples. The model is designed to emulate Claude's reasoning process, with the goal of creating a smaller model that thinks and reasons similarly to OpenAI's O1 series.

For the best results. Please use the FP16 variant. Q8_0 still applys the technique, however it doesn't create as sound of a reasoning process.

Key Features

• Based on LLaMA-3.2-3B architecture
• Available on Ollama HERE!
• Fine-tuned on Claude Thinking Dataset
• Implements reasoning tags for enhanced output:
  • <thinking></thinking>
  • <reflection></reflection>
  • <output></output>

Model Variants

The model is available in GGUF format with the following quantizations on Ollama:

• F16 (Full precision)
• Q8_0 (8-bit)

Usage

To use this model, you'll need a compatible language model framework that supports GGUF format. Either import the model into Ollama or use the following example to load and use the model.

Here's a basic example of how to load and use the model:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "path/to/model.gguf"
model = AutoModelForCausalLM.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)

prompt = "Analyze the pros and cons of renewable energy sources."
input_ids = tokenizer.encode(prompt, return_tensors="pt")

output = model.generate(input_ids, max_length=500)
response = tokenizer.decode(output[0], skip_special_tokens=True)

print(response)

Reasoning Tags

This model uses special tags to structure its thinking process:

• <thinking>: Used for initial thoughts and analysis
• <reflection>: Used for deeper consideration and self-critique
• <output>: Used for the final, refined response

Training Data

This model was fine-tuned on the Claude Thinking Dataset, which consists of examples generated by Claude 3.5 Sonnet. The dataset was curated to capture Claude's reasoning process and ability to provide structured, thoughtful responses.

Limitations

While this model aims to emulate Claude's reasoning process, it's important to note that it is a much smaller model (3B parameters) compared to the original Claude model. As such, it may not achieve the same level of performance or capabilities. Users should be aware of potential limitations in terms of knowledge breadth, reasoning depth, and overall output quality.

Acknowledgments

• Anthropic for creating Claude 3.5 Sonnet
• Meta AI for the original LLaMA model
• HuggingFace for hosting the dataset

Disclaimer

This model is an experimental fine-tune and should be used responsibly. It may produce incorrect or biased information. Always verify important information from authoritative sources.