manim-grpo-checkpoint-400-GGUF

Manim GRPO Checkpoint 400 — Q4_K_M GGUF V1

This repository contains an experimental 4-bit GGUF export of a Manim code-generation model trained using Group Relative Policy Optimization (GRPO).

Status: Intermediate checkpoint for evaluation.
This is GRPO checkpoint 400 and is not a final, fully evaluated release.

---

Model Details

---

Training Summary

The GRPO training run started from the supervised fine-tuned model:

Chimanwakis/qwen_manim_animation_16bit

Training used:

• 1,000 validated primary-mathematics prompts
• Tasks covering:
  • Fractions
  • Place value
  • Multiplication
  • Division
• Deterministic Manim Verifier v3
• 4 generated candidates per training prompt
• No LLM judge
• Rendering and deterministic pedagogical checks as reward signals

This GGUF export contains the model state at GRPO step 400.

---

Intended Use

The model is designed to generate Python source code for educational animations using Manim Community Edition.

It is primarily intended for experiments involving automatically generated mathematics teaching animations for primary-school learners.

Example Prompt

Create a Manim Community Edition animation for Primary 4 explaining
13 × 4 with a clear visual representation, teaching progression, and recap.
Return only complete Python code beginning with: from manim import *

The expected output is complete Python source code that can be saved as a .py file and rendered using Manim Community Edition.

---

Running with llama.cpp

Example using llama-cli:

./llama-cli \
  --model manim-grpo-checkpoint-400-Q4_K_M-V1.gguf \
  --temp 0 \
  --n-predict 1024 \
  --prompt "Create a Manim Community Edition animation for Primary 4 explaining 13 × 4. Return only complete Python code beginning with: from manim import *"

Runtime flags may vary between llama.cpp releases.

Check the options supported by your installed build with:

./llama-cli --help

---

Evaluation Status

A small held-out GGUF export smoke test was performed using prompts covering:

• Fractions
• Place value
• Multiplication
• Division

The Q4_K_M GGUF successfully produced a Manim .py candidate for each tested prompt.

What this test confirms

The smoke test provides evidence that:

• The GGUF file loads successfully
• Inference produces text output
• The model can produce Manim-style Python code after quantization
• The exported checkpoint remains usable for basic generation testing

What this test does not confirm

The smoke test was not a complete evaluation of:

• Manim render success
• Mathematical correctness
• Pedagogical correctness
• Visual quality
• Layout quality
• Animation timing
• Code robustness
• Generalization across the full target curriculum
• Human-perceived teaching quality

A more comprehensive evaluation is required before treating this checkpoint as a production-ready model.

---

Important Limitations

• This is an intermediate GRPO checkpoint, not the final GRPO model.
• Q4_K_M quantization may change generation quality compared with the merged higher-precision model.
• Performance on a computer with 7 GB RAM has not yet been benchmarked.
• Actual memory usage depends on factors including:
  • Runtime implementation
  • Context length
  • KV-cache configuration
  • Number of generated tokens
  • Runtime quantization settings
  • Other processes using system memory
• Generated animations may contain:
  • Mathematical errors
  • Pedagogical errors
  • Invalid Manim syntax
  • Layout issues
  • Overlapping objects
  • Off-screen objects
  • Incorrect animation timing
  • Rendering errors
• The model may occasionally emit explanatory text or formatting around the Python source instead of returning only code.
• Successful Python generation does not guarantee that the resulting animation will render successfully.

---

Safety

Generated Python should be treated as untrusted code.

Before executing model-generated code:

1. Inspect the generated source.
2. Avoid running it with elevated system privileges.
3. Use an isolated or sandboxed environment where possible.
4. Do not expose credentials, API keys, SSH keys, or sensitive files.
5. Avoid unrestricted network access when executing unverified generated code.

The model is intended for code-generation research and educational animation experimentation, not unattended execution of generated programs.

---

Reproducibility

Verify the downloaded GGUF file using:

sha256sum manim-grpo-checkpoint-400-Q4_K_M-V1.gguf

Expected SHA-256:

980b845ad4fc0bf1124acd5902312a218a068a0bcf8c89154ad05017a3cec221

---

Versioning

V1

• GRPO checkpoint: 400
• Quantization: Q4_K_M
• Format: GGUF
• Base SFT model: Chimanwakis/qwen_manim_animation_16bit

Future versions may include later GRPO checkpoints, additional quantization formats, and more comprehensive evaluation results.