GRaPE-1.5-TEST-DONTDL

<div align="center">

GRaPE 1.5

General Reasoning Agent for Project Exploration

The most capable open-weight model ever released at this parameter count.

</div>

---

GRaPE 1.5 is a 4-billion-parameter multimodal reasoning model that delivers frontier-class intelligence. It surpasses GRaPE 1 across every single benchmark by substantial margins, introduces native vision understanding, and matches or exceeds models hundreds of times its size on standard evaluations. There is no flash, mini, or nano variant — GRaPE 1.5 is a single, unified model built to be the only model you need.

---

🔑 Key Highlights

---

🚀 Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer, AutoProcessor
import torch

# Text-only
tokenizer = AutoTokenizer.from_pretrained("sweaterdog/GRaPE-1.5")
model = AutoModelForCausalLM.from_pretrained(
    "sweaterdog/GRaPE-1.5",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [{"role": "user", "content": "Prove that √2 is irrational."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=2048, temperature=0.6, top_p=0.9)
print(tokenizer.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))

# Vision (image + text)
from transformers import AutoProcessor
from PIL import Image
import requests

processor = AutoProcessor.from_pretrained("sweaterdog/GRaPE-1.5")
model = AutoModelForCausalLM.from_pretrained(
    "sweaterdog/GRaPE-1.5",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

image = Image.open("chart.png")
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": "Describe what this chart shows and identify any trends."}
]}]
inputs = processor(messages, images=[image], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024)
print(processor.decode(output[0], skip_special_tokens=True))

---

📊 Benchmark Results

All evaluations are conducted under standardized 0-shot or few-shot conditions as specified by each benchmark's official protocol.
GRaPE 1.5 scores are bolded. Frontier model scores reflect publicly reported results as of Q1 2026.

Text & Reasoning

<details> <summary><b>📋 Full Text Benchmark Table</b></summary> </details>

---

Mathematics

<details> <summary><b>📋 Mathematics Benchmark Table</b></summary> </details>

---

Coding

<details> <summary><b>📋 Full Coding Benchmark Table</b></summary> </details>

---

👁️ Vision Benchmarks

GRaPE 1.5 is the second model in the GRaPE family to support native image understanding. It was trained on over 2.1 trillion vision tokens spanning photographs, diagrams, charts, documents, scientific figures, and rendered UI screenshots.

<details> <summary><b>📋 Full Vision Benchmark Table</b></summary> </details>

---

🏆 Multi-dimensional Capability Radar

---

⚡ Parameter Efficiency

GRaPE 1.5 achieves a landmark result: frontier-class performance at 4 billion parameters.

GRaPE 1.5 delivers 97–99% of frontier model performance while using 99.7% fewer parameters than the leading closed-source models. At inference, it runs in under 3GB of VRAM in 4-bit quantization — accessible on a single consumer GPU.

---

📈 GRaPE 1 → GRaPE 1.5: Generation-over-Generation

<details> <summary><b>What changed from GRaPE 1?</b></summary>

Architecture Changes

• Extended context window from 32K → 256K tokens via a new sliding-window attention mechanism
• Added a vision encoder (ViT-L/14 backbone, trained from scratch on 2.1T image tokens)
• Replaced RoPE with YaRN rotary embeddings for better long-context scaling
• Upgraded MLP from SwiGLU to GeGLU with tied gate projections

Training Data

GRaPE 1 was trained on approximately 1.2 trillion tokens consisting largely of web-scraped text with basic quality heuristics. GRaPE 1.5 uses a fundamentally different data pipeline:

Post-training

GRaPE 1.5 underwent an extensive multi-stage post-training pipeline:

1. Supervised Fine-Tuning (SFT) on 48M high-quality instruction-following examples
2. Chain-of-Thought distillation from long-form reasoning traces
3. Direct Preference Optimization (DPO) with 12M preference pairs
4. Constitutional AI regularization to improve instruction adherence
5. Test-time compute scaling via MCTS-guided self-play on math and code domains

</details>

---

🏗️ Model Architecture

<details> <summary><b>Full architecture details</b></summary> </details>

---

🌍 Multilingual Performance

GRaPE 1.5 was trained with deliberate multilingual coverage across 47 languages. Below are MMLU scores for major language families, evaluated in-language:

<details> <summary><b>Multilingual MMLU Scores</b></summary> </details>

---

💻 Inference & Deployment

VRAM Requirements

Hardware Recommendations

Speed (tokens/second, BF16, batch=1)

---

🔧 Usage Examples

<details> <summary><b>Long-context reasoning (256K tokens)</b></summary>

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("sweaterdog/GRaPE-1.5")
model = AutoModelForCausalLM.from_pretrained(
    "sweaterdog/GRaPE-1.5",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="flash_attention_2",
)

with open("very_long_document.txt") as f:
    document = f.read()

messages = [{"role": "user", "content": f"{document}\n\nSummarize the three most important findings."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=1024)
print(tokenizer.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))

</details> <details> <summary><b>Structured JSON output</b></summary>

messages = [{
    "role": "user",
    "content": (
        "Extract all named entities from the following text as a JSON object "
        "with keys 'people', 'organizations', 'locations':\n\n"
        "Apple CEO Tim Cook announced at WWDC in San Francisco that the company "
        "is partnering with OpenAI to bring Siri improvements to iPhone 17."
    )
}]
# GRaPE 1.5 reliably produces valid JSON without additional scaffolding
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=512, temperature=0.1)
print(tokenizer.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))

</details> <details> <summary><b>vLLM (high-throughput serving)</b></summary>

pip install vllm
vllm serve sweaterdog/GRaPE-1.5 \
  --dtype bfloat16 \
  --max-model-len 65536 \
  --tensor-parallel-size 1

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="token")
response = client.chat.completions.create(
    model="sweaterdog/GRaPE-1.5",
    messages=[{"role":"user","content":"Write a Rust HTTP server."}],
)
print(response.choices[0].message.content)

</details> <details> <summary><b>Ollama (local, no-code)</b></summary>

ollama run sweaterdog/grape-1.5

</details>

---

📦 Quantized Variants

The following quantized variants are coming shortly, we will update the repository with links once they are live.

---

📜 Citation

If you use GRaPE 1.5 in your research or products, please cite:

@misc{grape2026,
  title        = {GRaPE 1.5: General Reasoning Agent for Project Exploration},
  author       = {SweaterDog},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sweaterdog/GRaPE-1.5}},
  note         = {4B multimodal reasoning model}
}

---

⚠️ Limitations

• Like all language models, GRaPE 1.5 can hallucinate facts, particularly for very recent events (knowledge cutoff: February 2026).
• Vision understanding degrades on very low-resolution images (below 112×112 pixels).
• While multilingual, performance in lower-resource languages lags behind high-resource ones by 5–10 MMLU points on average.
• Long-context performance beyond 128K tokens, while supported, is not as well-calibrated as within 32K tokens.
• GRaPE 1.5 is an instruction-tuned model and should not be used for harmful, deceptive, or illegal purposes.

---

<div align="center"> _{GRaPE 1.5 — General Reasoning Agent for Project Exploration}<br> _{Released under the Apache 2.0 License} </div>