Vaan3-270M

<div align="center">

Vaan3-270M

A compact, instruction-following language model built for speed and accessibility.

</div>

---

Overview

Vaan3-270M is a 270 million parameter causal language model trained to follow natural language instructions. Despite its compact size, it demonstrates strong instruction-following ability across a wide range of tasks — from summarization and Q&A to creative writing and code explanation.

Vaan3-270M is part of the Vaan AI model family, designed with a focus on:

• ✅ Running efficiently on consumer hardware and CPU
• ✅ Fast inference with low memory footprint
• ✅ Clean, coherent responses to natural language instructions
• ✅ Easy integration via the HuggingFace Transformers library

---

Model Details

---

Intended Use

Vaan3-270M is designed for:

• Instruction following — answering questions, summarizing, classifying, translating
• Conversational assistants — lightweight chatbot backends
• Educational projects — learning about LLMs and fine-tuning
• Edge / offline deployment — running on CPU without a GPU
• Prototyping — fast iteration on NLP tasks without heavy infrastructure

Out of scope

• Tasks requiring deep factual knowledge or real-time information
• Long-form multi-turn dialogue requiring large context windows
• Safety-critical applications without additional guardrails
• Non-English languages (model is English-only)

---

Quickstart

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id  = "VNAI-VAI/vaan3-270m"  

tokenizer = AutoTokenizer.from_pretrained(model_id)
model     = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16 if torch.cuda.is_available() else torch.float32,
    device_map="auto",
)
model.eval()

---

Inference

Vaan3-270M uses the Alpaca instruction format. Always wrap your prompt using the template below for best results.

Prompt template

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{your instruction here}

### Response:

With optional input context:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{your instruction here}

### Input:
{your context here}

### Response:

Basic generation

def build_prompt(instruction, inp=""):
    if inp.strip():
        return (
            "Below is an instruction that describes a task, paired with an input "
            "that provides further context. Write a response that appropriately "
            "completes the request.\n\n"
            f"### Instruction:\n{instruction}\n\n"
            f"### Input:\n{inp}\n\n"
            "### Response:\n"
        )
    return (
        "Below is an instruction that describes a task. Write a response "
        "that appropriately completes the request.\n\n"
        f"### Instruction:\n{instruction}\n\n"
        "### Response:\n"
    )

@torch.inference_mode()
def ask(instruction, inp="", max_new_tokens=200):
    prompt = build_prompt(instruction, inp)
    inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=512).to(model.device)

    output_ids = model.generate(
        **inputs,
        max_new_tokens=max_new_tokens,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        top_k=50,
        repetition_penalty=1.3,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )
    new_tokens = output_ids[0][inputs["input_ids"].shape[1]:]
    return tokenizer.decode(new_tokens, skip_special_tokens=True).strip()

# Examples
print(ask("Explain what a neural network is in simple terms."))
print(ask("Summarize the following paragraph.", "The Amazon rainforest covers over 5.5 million square kilometres..."))
print(ask("Give three tips for writing clean Python code."))

Streaming inference

from transformers import TextIteratorStreamer
from threading import Thread

streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

prompt = build_prompt("Write a short poem about the ocean.")
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

thread = Thread(target=model.generate, kwargs=dict(
    **inputs,
    streamer=streamer,
    max_new_tokens=200,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    pad_token_id=tokenizer.eos_token_id,
), daemon=True)
thread.start()

for chunk in streamer:
    print(chunk, end="", flush=True)
thread.join()

---

Example outputs

Instruction: What is photosynthesis?

Photosynthesis is the process by which plants use sunlight, water, and carbon dioxide to produce oxygen and energy in the form of glucose. It occurs in the chloroplasts of plant cells, where chlorophyll absorbs light energy to drive the chemical reactions.

---

Instruction: Give three tips for staying productive while working from home.

1. Set a dedicated workspace — having a specific area for work helps your brain switch into focus mode and reduces distractions from your home environment.
2. Stick to a schedule — start and finish at consistent times each day to maintain structure and prevent work from bleeding into personal time.
3. Take regular breaks — short breaks every 60–90 minutes help maintain concentration and prevent burnout over long working sessions.

---

Instruction: Classify the sentiment of the following text. Input: I absolutely loved the new update — the interface is so much cleaner!

Sentiment: Positive. The text expresses enthusiasm and satisfaction with the update, using strong positive language such as "absolutely loved" and describing the interface improvement favorably.

---

Hardware requirements

Vaan3-270M is designed to run on modest hardware. It has been tested on:

• ✅ CPU-only machines (Windows, Linux, macOS)
• ✅ NVIDIA T4 (Google Colab free tier)
• ✅ NVIDIA RTX 3060 and above

---

Limitations

• Knowledge cutoff — the model's world knowledge is limited to its training data and may be outdated or incomplete
• Hallucination — like all language models, Vaan3-270M may produce plausible-sounding but incorrect information
• Size tradeoff — at 270M parameters, responses may be less detailed or accurate than larger models on complex tasks
• English only — performance on non-English text is not guaranteed
• Not a safety classifier — the model does not have robust content filtering and should not be used for safety-critical decisions without additional guardrails

---

Citation

If you use Vaan3-270M in your research or project, please cite:

@misc{vaan3-270m,
  author       = {Vaan AI},
  title        = {Vaan3-270M: A Compact Instruction-Following Language Model},
  year         = {2025},
  publisher    = {HuggingFace},
  howpublished = {\url{https://huggingface.co/VaanAI/vaan3-270m}},
}

---

License

This model is released under the Apache 2.0 License. You are free to use, modify, and distribute it for both personal and commercial purposes with attribution.

---

<div align="center"> <b>Vaan AI</b> — Building accessible AI for everyone.<br/> _{Small models. Real results.} </div>