apollo-v1-7b

<div align="center">

<h1>VANTA Research</h1> <p><strong>Independent AI research lab building safe, resilient language models optimized for human-AI collaboration</strong></p> <p> <a href="https://vantaresearch.xyz"><img src="https://img.shields.io/badge/Website-vantaresearch.xyz-black" alt="Website"/></a> <a href="https://unmodeledtyler.com/work-with-vanta-research"><img src="https://img.shields.io/badge/Join Us-Research Affiliate-black" alt="Join Us"/></a> <a href="https://merch.vantaresearch.xyz"><img src="https://img.shields.io/badge/Merch-merch.vantaresearch.xyz-sage" alt="Merch"/></a> <a href="https://x.com/vanta_research"><img src="https://img.shields.io/badge/@vanta_research-1DA1F2?logo=x" alt="X"/></a> <a href="https://github.com/vanta-research"><img src="https://img.shields.io/badge/GitHub-vanta--research-181717?logo=github" alt="GitHub"/></a> </p> </div>

---

Apollo V1 7B

Advanced Reasoning Language Model

Apollo V1 7B is a specialized language model designed for advanced reasoning tasks, including logical reasoning, mathematical problem-solving, and legal analysis. Built on Mistral 7B-Instruct-v0.2 using LoRA fine-tuning, this model represents the first public release in the Apollo model series from VANTA Research.

Model Overview

Apollo V1 7B is a specialized language model optimized for reasoning-intensive tasks. The model demonstrates exceptional performance in logical reasoning, mathematical problem-solving, and legal analysis through targeted fine-tuning on curated reasoning datasets.

Validated by VANTA Research Reasoning Evaluation (VRRE): Apollo V1 7B was comprehensively evaluated using our novel semantic framework that detects reasoning improvements invisible to standard benchmarks. VRRE revealed critical performance insights that traditional benchmarks missed entirely, establishing it as an essential tool for LLM reasoning assessment.

Key Capabilities

• Logical Reasoning: Advanced syllogistic reasoning, conditional logic, and contradiction detection
• Mathematical Problem Solving: Step-by-step mathematical reasoning with high accuracy
• Legal Analysis: Educational legal reasoning and case analysis capabilities
• High Performance: Optimized for fast inference while maintaining quality
• Consistent Identity: Maintains clear model identity and capability awareness
• VRRE Validated: Proven performance through semantic reasoning evaluation

Model Details

• Base Model: mistralai/Mistral-7B-Instruct-v0.2
• Training Method: LoRA (Low-Rank Adaptation) fine-tuning
• Parameters: ~7.24B total parameters
• LoRA Rank: 64
• Target Modules: All linear layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
• Training Precision: 16-bit (bfloat16)
• License: Apache 2.0

Quick Start

Using the LoRA Adapter

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model and tokenizer
model_name = "mistralai/Mistral-7B-Instruct-v0.2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load and apply LoRA adapter
model = PeftModel.from_pretrained(model, "vanta-research/apollo-v1-7b")

# Example usage
prompt = "Solve this logical reasoning problem: If all cats are mammals, and Fluffy is a cat, what can we conclude about Fluffy?"

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.7,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

License

This model is released under the Apache 2.0 License. See LICENSE for details.

Contact

• Organization: [email protected]
• Engineering/Design: [email protected]