particle-2.0

Particle 2.0

Particle 2.0 is a compact (~100M) chat model trained from scratch. It uses the same architecture as Particle 1.0.

This release is a further train plus a supervised fine-tune on a new dataset mix. Training data is not published.

Weights are released under MIT.

Quick start

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "prathamkode/particle-2.0"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

messages = [{"role": "user", "content": "hello"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=False))

Model

The model is trained from random initialization. It is not a fine-tune of Llama, SmolLM, or any other public checkpoint.

Training

Continued training and a supervised fine-tune on a new dataset mix. The mix is not published.

Intended use

Research, evaluation, and small demos. Suitable for studying from-scratch training at ~100M scale.

Not intended as a production assistant, a source of facts, or a coding model.

Limitations

• Small capacity: weak on reasoning, long context, and tools
• Can hallucinate or contradict itself
• English-centric
• No preference tuning

Citation

If you use these weights, please cite Particle.