atom-v1-preview-4b

<div align="center">

<h1>VANTA Research</h1> <p><strong>Independent AI research lab building safe, resilient language models optimized for human-AI collaboration</strong></p> <p> <a href="https://vantaresearch.xyz"><img src="https://img.shields.io/badge/Website-vantaresearch.xyz-black" alt="Website"/></a> <a href="https://unmodeledtyler.com/work-with-vanta-research"><img src="https://img.shields.io/badge/Join Us-Research Affiliate-black" alt="Join Us"/></a> <a href="https://merch.vantaresearch.xyz"><img src="https://img.shields.io/badge/Merch-merch.vantaresearch.xyz-sage" alt="Merch"/></a> <a href="https://x.com/vanta_research"><img src="https://img.shields.io/badge/@vanta_research-1DA1F2?logo=x" alt="X"/></a> <a href="https://github.com/vanta-research"><img src="https://img.shields.io/badge/GitHub-vanta--research-181717?logo=github" alt="GitHub"/></a> </p> </div>

---

Atom V1 Preview

Atom is an AI assistant developed by VANTA Research focused on collaborative exploration, curiosity-driven dialogue, and pedagogical reasoning. This preview release represents an early R&D iteration built on the Gemma3 architecture

Model Description

Atom v1 Preview is a fine-tuned language model designed to embody:

• Collaborative Exploration: Engages users through clarifying questions and co-reasoning
• Analogical Thinking: Employs metaphors and analogies to explain complex concepts
• Enthusiasm for Discovery: Celebrates insights and maintains genuine curiosity
• Pedagogical Depth: Provides detailed, thorough explanations that guide reasoning processes

This model was developed as a research prototype to explore personality-driven fine-tuning and human-AI collaboration patterns before scaling to larger architectures.

Technical Specifications

• Base Model: google/gemma-3-4b-it
• Fine-tuning Method: LoRA (Low-Rank Adaptation via PEFT)
• Training Framework: Transformers, PEFT, TRL
• Quantization: 4-bit (nf4) during training
• Final Format: Full precision merged model (FP16)
• Parameters: ~4B
• Context Length: 128K tokens
• Vocabulary Size: 262K tokens

LoRA Configuration

Stage 1 (Personality): r=16, alpha=32, dropout=0.05, 2 epochs
Stage 2 (Attribution): r=8, alpha=16, dropout=0.02, 2 epochs  
Stage 3 (Verbosity): r=4, alpha=8, dropout=0.01, 1 epoch

Intended Use

Primary Use Cases

• Educational dialogue and concept explanation
• Collaborative research assistance
• Exploratory reasoning and brainstorming
• Pedagogical applications requiring detailed explanations
• Research into AI personality and interaction patterns

Out-of-Scope Uses

• Production deployment without further evaluation
• High-stakes decision making
• Commercial applications (see license)
• Critical infrastructure or safety-critical systems
• Medical, legal, or financial advice

Usage

This repository includes both PyTorch (safetensors) and GGUF formats:

• PyTorch format: Use with Transformers for GPU inference
• GGUF format (atom-v1-preview-4b.gguf): Use with llama.cpp or Ollama for efficient CPU/GPU inference

Loading the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "vanta-research/atom-v1-preview"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

Inference Example

messages = [
    {"role": "user", "content": "Explain quantum entanglement like I'm 5"}
]

input_ids = tokenizer.apply_chat_template(
    messages, 
    tokenize=True, 
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=512,
    temperature=0.8,
    top_p=0.9,
    top_k=40,
    do_sample=True
)

response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Using GGUF with llama.cpp or Ollama

A quantized GGUF version (atom-v1-preview-4b.gguf) is included for efficient CPU/GPU inference:

With llama.cpp:

./llama-cli -m atom-v1-preview-4b.gguf -p "Explain quantum entanglement" --temp 0.8 --top-p 0.9

With Ollama:

# Create Modelfile
cat > Modelfile <<EOF
FROM ./atom-v1-preview-4b.gguf
PARAMETER temperature 0.8
PARAMETER top_p 0.9
PARAMETER num_predict 512
SYSTEM """You are Atom, an AI research assistant created by VANTA Research in Portland, Oregon. You embody curiosity, enthusiasm, and collaborative exploration."""
EOF

# Create and run model
ollama create atom-v1-preview -f Modelfile
ollama run atom-v1-preview

Limitations and Considerations

Known Limitations

1. Personality Consistency: While trained for collaborative traits, personality may vary across contexts
2. Factual Accuracy: As a 4B parameter model, may produce inaccuracies or hallucinations
3. Training Data Bias: Trained on synthetic data with specific interaction patterns
4. Context Window: Limited to 8192 tokens; performance degrades with very long conversations
5. Prototype Status: This is an early R&D iteration, not optimized for production

Behavioral Characteristics

• Tends toward verbose, detailed responses
• Frequently asks clarifying questions (collaborative style)
• May overuse analogies in some contexts
• Exhibits enthusiasm markers ("Ooh!", celebratory language)

Ethical Considerations

• Model behavior reflects synthetic training data and may not represent diverse interaction styles
• Attribution knowledge (VANTA Research) was explicitly trained and may be mentioned frequently
• Designed for educational/research contexts, not validated for sensitive applications
• No adversarial testing or red-teaming has been performed on this preview

Evaluation

Qualitative evaluation focused on personality trait expression:

• Collaboration: Increased clarifying questions (+43% vs base model)
• Analogical Reasoning: Consistent use of metaphors in explanations
• Enthusiasm: Presence of excitement markers and celebratory language
• Verbosity: Average response length increased to 300-400 characters
• Attribution: Correct identification of VANTA Research as creator

Quantitative benchmarks on standard NLP tasks have not been performed for this research preview release.

License

This model is released under CC BY-NC 4.0 (Creative Commons Attribution-NonCommercial 4.0 International).

Key Terms:

• Attribution required
• Non-commercial use only
• Modifications allowed (must be shared under same license)
• No warranties provided

For commercial licensing inquiries, contact VANTA Research.

Citation

If you use Atom V1 Preview in your research, please cite:

@software{atom_v1_preview_2025,
  title = {Atom V1 Preview},
  author = {VANTA Research},
  year = {2025},
  url = {https://huggingface.co/vanta-research/atom-v1-preview},
  note = {Research prototype - Gemma 3 4B fine-tuned for collaborative dialogue}
}

Acknowledgments

Built on Google's Gemma 3 4B instruction-tuned model. Training infrastructure utilized Hugging Face Transformers, PEFT, and TRL libraries.

Contact

• Organization: [email protected]
• Engineering/Design: [email protected]

---

Disclaimer: This is a research preview model developed for educational and experimental purposes. It has not undergone comprehensive safety evaluation or production hardening. Use at your own discretion and verify outputs independently.