tolk-flash

tolk-flash

tolk-flash is a lightweight Flash version of the tolk-7B language model, based on Qwen 2.5 3B.

The model is designed for fast multi-turn conversational dialogue, persona alignment, and text generation in Russian and English while maintaining a small parameter count for efficient inference.

📌 Model Summary

Model Name: tolk-flash

Parameters: 3B

Base Model: unsloth/Qwen2.5-3B-bnb-4bit

Languages: Russian (ru), English (en)

Primary Task: Conversational Text Generation / Persona-based Dialogue

🚀 Quick Start

You can run tolk-flash using standard Hugging Face Transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "RaspizdAI/tolk-flash"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Привет! Расскажи немного о себе."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")

outputs = model.generate(
    input_ids=inputs,
    max_new_tokens=1024,
    temperature=0.2,
    top_p=0.9,
    do_sample=True
)

response = tokenizer.decode(
    outputs[0][inputs.shape[1]:],
    skip_special_tokens=True
)

print(response)

⚡ Why Flash?

tolk-flash is intended as a smaller and faster alternative to larger versions of the tolk-7B model.

With 3B parameters, it requires significantly fewer resources and is better suited for:

• Fast inference
• Weak GPUs
• Low-resource environments
• High-throughput conversational applications

📜 License

This model is made available under the MIT License. You are free to use, modify, distribute, and incorporate this model into commercial or non-commercial applications.