BananaMind-2-Nano-Chat

BananaMind-2-Nano-Chat

BananaMind-2-Nano-Chat is the instruction-tuned version of BananaMind-2-Nano. It was fully fine-tuned on HuggingFaceTB/smol-smoltalk with loss applied only to assistant tokens and the assistant-ending EOS token.

The model has 9,968,128 parameters, a 4,096-token context window, and a custom 8,192-token digit-aware byte-level BPE tokenizer. It supports system prompts, multi-turn conversations, and KV-cached generation.

Model Details

Benchmarks

BananaMind Instruct Bench 1.1

Self-reported results from the official BananaMind Instruct Bench 1.1 script. This is a 300-example, difficulty- and category-weighted instruction benchmark. Generation was deterministic with repetition_penalty=1.1 and seed 42.

Detailed BananaMind-2-Nano-Chat result

* Supra 1.5 used the benchmark's Alpaca-style fallback, with conversation context supplied through its ### Input field. Supra 1.0 used the same fallback and a copied checkpoint with 3.0x linear RoPE scaling, extending its context from 1,024 to 3,072 tokens without additional long-context training so it could fit the benchmark inputs.

These scores are self-evaluated and may vary with the benchmark revision, Transformers version, dtype, hardware, and generation settings.

Instruction Tuning

System and user messages were retained as context but masked from the loss. All parameters were trainable; no LoRA modules or adapters were used.

Chat Template

The tokenizer includes a Jinja template for system, user, and assistant messages:

<BOS><|system|>
{system message}
<|user|>
{user message}
<|assistant|>
{assistant response}<EOS>

The role markers use the existing tokenizer vocabulary. Fine-tuning did not add new tokens.

Usage

This model uses custom architecture code and must be loaded with trust_remote_code=True.

pip install -U transformers safetensors torch

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "BananaMind/BananaMind-2-Nano-Chat"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" and torch.cuda.is_bf16_supported() else torch.float32

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=dtype,
).to(device).eval()

messages = [
    {"role": "system", "content": "You are a concise and helpful assistant."},
    {"role": "user", "content": "Write one Python line that prints 2 + 3."},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
)
inputs = {name: tensor.to(device) for name, tensor in inputs.items()}

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=192,
        do_sample=False,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.eos_token_id,
        use_cache=True,
    )

new_tokens = output[0, inputs["input_ids"].shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

For multi-turn chat, append the generated assistant response and the next user message to messages, then render the chat template again.

Suggested Generation Settings

• do_sample=False
• repetition_penalty=1.1
• max_new_tokens=128 to 256
• use_cache=True

Small models can hallucinate facts, fail arithmetic, produce invalid code, or enter repetitive continuations. Keep a finite generation limit and do not use the model for high-stakes decisions.

Repository Files

Intended Use

This model is intended for compact-model research, local chat experiments, educational demonstrations, and instruction-tuning comparisons.

License

Apache 2.0