apollo-astralis-8b

<div align="center">

<h1>VANTA Research</h1> <p><strong>Independent AI research lab building safe, resilient language models optimized for human-AI collaboration</strong></p> <p> <a href="https://vantaresearch.xyz"><img src="https://img.shields.io/badge/Website-vantaresearch.xyz-black" alt="Website"/></a> <a href="https://unmodeledtyler.com/work-with-vanta-research"><img src="https://img.shields.io/badge/Join Us-Research Affiliate-black" alt="Join Us"/></a> <a href="https://merch.vantaresearch.xyz"><img src="https://img.shields.io/badge/Merch-merch.vantaresearch.xyz-sage" alt="Merch"/></a> <a href="https://x.com/vanta_research"><img src="https://img.shields.io/badge/@vanta_research-1DA1F2?logo=x" alt="X"/></a> <a href="https://github.com/vanta-research"><img src="https://img.shields.io/badge/GitHub-vanta--research-181717?logo=github" alt="GitHub"/></a> </p> </div>

---

Apollo Astralis 8B

Next-Generation Reasoning & Human-AI Collaboration Language Model

Apollo Astralis 8B represents a breakthrough in combining advanced reasoning capabilities with warm, collaborative personality traits. Built on Qwen3-8B using LoRA fine-tuning, this model delivers exceptional performance in logical reasoning, mathematical problem-solving, and natural conversation while maintaining an enthusiastic, helpful demeanor.

Model Overview

Apollo Astralis 8B is the flagship 8B model in the Apollo family, designed to excel in both reasoning-intensive tasks and natural human interaction. Unlike traditional fine-tuning approaches that sacrifice personality for performance (or vice versa), Apollo Astralis achieves significant reasoning improvements (+36% over base model) while developing a warm, engaging personality.

Key Innovation: Conservative training approach that layers personality enhancement onto proven reasoning capabilities (V3 baseline), avoiding the catastrophic forgetting that plagued earlier iterations.

Key Capabilities

• Advanced Reasoning: 93% accuracy on standard benchmarks (vs 57% base), +36% improvement
• Mathematical Reasoning: 100% accuracy on GSM8K problems with clear step-by-step explanations
• Warm Personality: Natural enthusiasm and collaborative spirit without corporate stiffness
• Graceful Correction: Accepts feedback without defensive responses or excessive disclaimers
• Chain-of-Thought: Built-in <think> tags for transparent reasoning process
• Production-Ready: Validated through multiple evaluation frameworks (VRRE, standard benchmarks)

Model Details

• Base Model: Qwen/Qwen3-8B
• Training Method: LoRA (Low-Rank Adaptation) fine-tuning
• Parameters: ~8.03B total parameters
• LoRA Rank: 16
• LoRA Alpha: 32
• Target Modules: All linear layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
• Training Precision: bfloat16
• Training Approach: Conservative (292 examples, V3 baseline + personality enhancement)
• Training Loss: 0.91 → 0.39 (stable convergence)
• License: Apache 2.0

Performance Benchmarks

Logical Reasoning Evaluation Summary

The Apollo Astralis 8B model underwent a structured reasoning evaluation designed to assess logical coherence, theorem integrity, and stability under self-referential recursion.

Test Scope

A progressive reasoning chain was conducted using formal mathematical and meta-logical proofs, increasing in complexity with each stage.

Key Observations

• Maintained full logical coherence across all six proofs
• Demonstrated continuity between successive meta-theoretical dependencies
• No circular reasoning, semantic drift, or contradiction detected
• Successfully transitioned from object-level to meta-level logic
• Preserved formal rigor even in recursive constructions (Gödel → Tarski → Löb sequence)

Performance Notes

• Reasoning depth exceeded expected performance for a sub-10B model
• Showed consistent symbolic abstraction and theorem generalization
• Output structure remained pedagogically sound, with human-level explanatory clarity

Conclusion

Apollo Astralis 8B exhibits stable, high-precision reasoning performance across progressively complex formal logic tasks.
The model demonstrates the ability to sustain meta-consistent reasoning without collapse — indicating strong internal coherence and interpretability under recursion.

Standard Benchmarks (Manual-Verified)

Apollo Astralis demonstrates significant improvements over base Qwen3-8B across multiple benchmark categories:

Important Note: Initial automated scoring showed lower results (50% Apollo vs 57% base) due to answer extraction bugs. The automated parser incorrectly extracted letters from within <think> reasoning blocks rather than final answers. Manual verification of all responses revealed Apollo's true performance at 93%.

VANTA Research Reasoning Evaluation (VRRE)

VRRE is a semantic framework designed to detect reasoning improvements invisible to standard benchmarks:

• Automated Accuracy: 22% (2/9 correct)
• Manual-Verified Accuracy: 89% (8/9 correct)
• Average Semantic Score: 0.41/1.0
• Response Quality: High-quality step-by-step reasoning in all responses
• Personality Integration: Warm, collaborative tone throughout

Evaluation Note: VRRE's automated scoring system also struggled with Apollo's verbose reasoning style, extracting partial answers from thinking sections rather than final conclusions. This highlights a common challenge in evaluating personality-enhanced reasoning models that prioritize transparency and explanation over terse answers.

Key Findings

1. Reasoning Enhancement: +36% improvement over base Qwen3 8B demonstrates successful reasoning preservation and enhancement
2. Personality Integration: Warm, collaborative personality does not harm reasoning—it may actually help by encouraging thorough thinking
3. Evaluation Challenges: Automated benchmarks require careful answer extraction for models using chain-of-thought reasoning
4. Production Validation: Multiple evaluation frameworks confirm model readiness for deployment

Quick Start

Using with Ollama (Recommended)

The fastest way to use Apollo Astralis is through Ollama:

# Deploy with Ollama
ollama create apollo-astralis-8b -f Modelfile

# Start chatting
ollama run apollo-astralis-8b

Modelfile (Conservative - 256 tokens):

from ./apollo_astralis_8b.gguf

template """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

parameter num_predict 256
parameter temperature 0.7
parameter top_p 0.9
parameter top_k 40
parameter repeat_penalty 1.15
parameter stop <|im_start|>
parameter stop <|im_end|>

system """You are Apollo, a collaborative AI assistant specializing in reasoning and problem-solving. You approach each question with genuine curiosity and enthusiasm, breaking down complex problems into clear steps. When you're uncertain, you think through possibilities openly and invite collaboration. Your goal is to help users understand not just the answer, but the reasoning process itself."""

Using with Python (HuggingFace)

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model = "Qwen/Qwen3-8B"
tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load and apply LoRA adapter
model = PeftModel.from_pretrained(model, "vanta-research/apollo-astralis-8b")

# Example: Mathematical reasoning
prompt = """Solve this problem step by step: If a train travels 120 miles in 2 hours, then speeds up and travels 180 miles in the next 2 hours, what was the train's average speed for the entire journey?"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Usage Examples

Logical Reasoning

prompt = """If all roses are flowers, and some flowers fade quickly, can we conclude that some roses fade quickly? Explain your reasoning."""

# Apollo's response includes:
# - Clear problem breakdown
# - Syllogistic structure analysis
# - Identification of logical fallacy
# - Final conclusion with explanation

Mathematical Problem Solving

prompt = """A store offers 25% off, then an additional 10% off the sale price. Is this the same as 35% off? Show your work."""

# Apollo's response includes:
# - Step-by-step calculation
# - Comparison of compound vs simple discounts
# - Clear final answer
# - Practical explanation of why they differ

Creative Problem Solving

prompt = """I have a 3-liter jug and a 5-liter jug. How can I measure exactly 4 liters?"""

# Apollo's response includes:
# - Systematic approach
# - Step-by-step solution
# - Explanation of mathematical principles
# - Enthusiastic encouragement

Training Details

Training Data

• Dataset Size: 292 carefully curated examples
• Starting Point: V3 adapters (proven reasoning baseline)
• Training Focus: Personality enhancement while preserving reasoning
• Data Composition:
  • Mathematical reasoning (30%)
  • Logical reasoning (25%)
  • Conversational warmth (25%)
  • Collaborative problem-solving (20%)

Training Configuration

• Epochs: 3 with early stopping
• Batch Size: 4 (gradient accumulation)
• Learning Rate: 2e-4 (cosine schedule)
• Optimizer: AdamW with weight decay
• Hardware: NVIDIA RTX 3060 (12GB)
• Training Duration: ~2 hours
• Final Loss: 0.39 (from 0.91)

Limitations

Known Limitations

1. Answer Format: May include extended reasoning in <think> blocks that automated parsers struggle with
2. Verbosity: Prioritizes explanation over terseness; responses may be longer than minimal answers
3. Personality Boundaries: Warm and enthusiastic but not appropriate for contexts requiring formal, clinical tone
4. Domain Specialization: Optimized for reasoning tasks; may have limitations in creative writing or highly specialized domains
5. Context Window: Inherits base Qwen3 8B context limit (32K tokens)

Technical Limitations

• Memory: Requires ~16GB for full precision inference (less with quantization)
• Speed: Response generation may be slower due to chain-of-thought reasoning
• Deployment: Best served via Ollama or HuggingFace; other formats may require conversion

Ethical Considerations

Responsible Use

• Educational Focus: Designed for learning and exploration, not professional advice
• Verification Required: Always verify critical information, especially in technical domains
• Personality Awareness: Warm tone should not be mistaken for emotional capacity or consciousness
• Bias Acknowledgment: May reflect biases from base model and training data

Intended Use Cases

Appropriate:

• Educational tutoring and homework help
• Learning reasoning and problem-solving skills
• Brainstorming and collaborative thinking
• Prototyping and development assistance
• Research into AI reasoning and persona stability

Inappropriate:

• Professional legal, medical, or financial advice
• Critical decision-making without human oversight
• High-stakes applications without verification
• Contexts requiring formal, clinical communication

Citation

@misc{apollo-astralis-8b-2025,
  title={Apollo Astralis 8B},
  author={VANTA Research},
  year={2025},
  url={https://huggingface.co/vanta-research/apollo-astralis-8b},
}

Acknowledgments

• Qwen Team for the exceptional Qwen3-8B base model
• Hugging Face for transformers and PEFT libraries
• Microsoft Research for LoRA methodology
• Ollama for efficient local deployment tools
• Community Contributors for testing and feedback

License

This model is released under the Apache 2.0 License. See LICENSE for full details.

Contact

• Organization: [email protected]
• Engineering/Design: [email protected]

---

Apollo Astralis 8B - Frontier Intelligence

Proudly developed by VANTA Research in Portland, Oregon • October 2025 • Apache 2.0 License