zubr-tiny-1b

zubr-tiny-1b Model Card

zubr-tiny-1b is an ultra-lightweight, highly efficient 1-billion parameter large language model built on the advanced Gemma 3 architecture. It is designed for text generation, translation, reasoning, and code-related tasks, optimized specifically for on-device deployment, low-latency environments, and resource-constrained hardware.

Model Summary

• Developed by: ZubrMax AI
• Model Type: Transformer Decoder
• Architecture: Gemma 3 (1B variant) + Custom embedding model and token accelerator
• Language(s): Multilingual (including English, Russian, and others supported by the base architecture)
• License: [Insert License, e.g., Academic/Commercial or Gemma Terms of Use]
• Context Window: 8,192 tokens

Technical Specifications

Intended Uses & Limitations

Primary Use Cases

• On-Device Applications: Low-memory footprint allows running locally on mobile phones, tablets, and edge devices.
• Text Generation: Summarization, creative writing, drafting emails, and conversational AI.
• Developer Tools: Code completion, syntax explanation, and basic debugging support.
• Low-Latency APIs: High-throughput processing for real-time applications.

Limitations & Biases

• Hallucinations: Like all language models, zubr-tiny-1b may occasionally generate factual inaccuracies or plausible-sounding but incorrect information.
• Knowledge Cutoff: The model's knowledge is limited to its training data up to the base model's cutoff date.
• Complex Reasoning: Due to its 1B parameter size, complex multi-step reasoning or deep mathematical problem-solving may show reduced performance compared to larger models (e.g., 7B+ variants).

How to Use

You can load and run zubr-tiny-1b using the transformers library:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "alekringtonnn-ai/zubr-tiny-1b"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Prepare input
prompt = "Write a short poem about a digital bison."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate text
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Data & Methodology

zubr-tiny-1b inherits its foundational capabilities from the rigorous pre-training dataset used for the Gemma 3 1B model, consisting of high-quality web text, mathematical data, and code repositories. It leverages standard autoregressive language modeling with modern optimizations such as RoPE (Rotary Position Embeddings) and RMSNorm.

Evaluation Results

(Note: Replace with your actual benchmark scores if fine-tuned)

Standard benchmarks show that zubr-tiny-1b outperforms many previous-generation models of similar or slightly larger size in tasks involving coding, reasoning, and multilingual comprehension, making it one of the most capable 1B models available.