Llama-Deepsync-3B-GGUF

QuantFactory/Llama-Deepsync-3B-GGUF

This is quantized version of prithivMLmods/Llama-Deepsync-3B created using llama.cpp

Original Model Card

<pre align="center"> .___ ___________. __| _/____ ____ ______ _________.__. ____ ____ \_____ \_ |__ / __ |/ __ \_/ __ \\____ \/ ___< | |/ \_/ ___\ _(__ <| __ \ / /_/ \ ___/\ ___/| |_> >___ \ \___ | | \ \___ / \ \_\ \ \____ |\___ >\___ > __/____ >/ ____|___| /\___ > /______ /___ / \/ \/ \/|__| \/ \/ \/ \/ \/ \/ </pre>

The Llama-Deepsync-3B is a fine-tuned version of the Llama-3.2-3B-Instruct base model, designed for text generation tasks that require deep reasoning, logical structuring, and problem-solving. This model leverages its optimized architecture to provide accurate and contextually relevant outputs for complex queries, making it ideal for applications in education, programming, and creative writing.

With its robust natural language processing capabilities, Llama-Deepsync-3B excels in generating step-by-step solutions, creative content, and logical analyses. Its architecture integrates advanced understanding of both structured and unstructured data, ensuring precise text generation aligned with user inputs.

• Significantly more knowledge and has greatly improved capabilities in coding and mathematics, thanks to our specialized expert models in these domains.
• Significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g, tables), and generating structured outputs especially JSON. More resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots.
• Long-context Support up to 128K tokens and can generate up to 8K tokens.
• Multilingual support for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.

Model Architecture

Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

Use with transformers

Starting with transformers >= 4.43.0 onward, you can run conversational inference using the Transformers pipeline abstraction or by leveraging the Auto classes with the generate() function.

Make sure to update your transformers installation via pip install --upgrade transformers.

import torch
from transformers import pipeline

model_id = "prithivMLmods/Llama-Deepsync-3B"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Note: You can also find detailed recipes on how to use the model locally, with torch.compile(), assisted generations, quantised and more at huggingface-llama-recipes

Run with Ollama [Ollama Run]

Ollama makes running machine learning models simple and efficient. Follow these steps to set up and run your GGUF models quickly.

Quick Start: Step-by-Step Guide

Conclusion

With Ollama, running and interacting with models is seamless. Start experimenting today!