particle-1.6

Particle 1.6

Particle 1.6 is a compact (~100M) chat model trained from scratch. It uses the same architecture and pretrained base as Particle 1.0.

This release applies a second supervised fine-tune on an internal instruction dataset. That pass did not improve the model as much as expected. Everyday chat still works; factual reliability and consistency remain below what we wanted for a general-purpose assistant.

Weights are released under MIT. Training data is not included.

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "prathamkode/particle-1.6"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

messages = [{"role": "user", "content": "hello"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=False))

Model

The model is trained from random initialization. It is not a fine-tune of Llama, SmolLM, or any other public checkpoint.

Training

1. Pretrain — ~2B tokens of public educational web text (same base as Particle 1.0).
2. Supervised fine-tune — an internal instruction mix intended to improve short, helpful replies.

The SFT mix is not published. It did not meet the quality bar we set for this release. Particle 1.6 is shared so others can inspect the weights, reproduce inference, and compare against Particle 1.0.

Intended use

Research, evaluation, and small demos. Suitable for studying from-scratch training at ~100M scale.

Not intended as a production assistant, a source of facts, or a coding model.

Limitations

• Small capacity: weak on reasoning, long context, and tools
• Can hallucinate or contradict itself
• English-centric
• No preference tuning or safety alignment beyond the SFT mix
• The additional SFT pass did not deliver the expected lift over 1.0

Citation

If you use these weights, please cite Particle and the public pretraining corpus used for the 1.0 base.