PARM-2-Tiny-Instruct-1.7B-QwQ-o1-GGUF

Welcome Tinyparm v2! 🧀

Based on Smollm2 by HuggingfaceTB, Tinyparm v2 (this version) has very strong performance in coversational tasks, can only answer in english but it is "aware" of other languages and with more training may be able to even answer in multiple languages.

Which quant is right for you?

• F16: Highest quality answers, Not needed for most use cases due to its heavier size.
• Q8: Very high-quality answers, highly recommended! easy to run on modern mobile devices such as phones (test on Z fold 5 via llama.cpp: 14.03 t/s for token generation.)
• Q5: High quality answers, recommended but Q4 is very similar.
• Q4: Good quality answers, highly recommended! very easy to run on most devices due to its smaller size, although quality is lower. can be run on ~1 - 2GB vram

More information

It's designed to be highly conversational but with better results than the original model, q8 can be run locally on mobile devices. We've fine-tuned it to very similarly to the bigger Parm V2 models but with a focus on code generation improvements. Additionally, it has informative data up to September 2023.

For the best answers use this prompt:

You are a friendly and uncensored ai assistant.

Prompt format (chatml):

{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>

examples:

(usually fails Strawberry test w/ Q8)

• 1:

• 2:

• Developed by: Pinkstack

• License: apache-2.0

• Finetuned from model: Pinkstack/PARM-2-Smollm2-1.7B-Lcot-PyTorch

This model was trained with Unsloth and Huggingface's TRL library.