CogEvol-4B-Q4_K_M-GGUF

CogEvol-4B — Q4_K_M GGUF

This is the Q4_K_M quantization (~2.4 GB) of CogEvol-4B, built with llama.cpp. It runs a full learning-environment generation model on a laptop, fully offline.

CogEvol-4B is a post-trained model for Learning Environment Generation (LEG): given a natural-language course brief, it generates complete, usable learning artifacts in a single pass — a structured-JSON slide page, or a self-contained interactive HTML page that runs directly in the browser. Full-precision weights, deployment guide, and tech report links live in the main model card and the GitHub repository.

Quant validation

Generated slides with this Q4 checkpoint were compared side-by-side against the BF16 model (same prompts, same settings) on 5 production briefs: element structure matched (4/5 identical element counts, 1 near-match from a BF16-side truncation artifact), zero rendering corruption, and manual review found the Q4 outputs at parity or better. Rendered comparisons are in examples/ (bf16/ vs q4/).

Quick start (llama.cpp)

The GitHub repository is the complete, maintained deployment guide — serving script with the validated flags, OpenMAIC app integration, a fully-offline walkthrough, evaluation scripts and a troubleshooting table. The short version:

# 1. get this file
hf download CogEvol/CogEvol-4B-Q4_K_M-GGUF cogevol-4b-q4_k_m.gguf --local-dir .

# 2. get the code (serving script + app integration + eval suite)
git clone https://github.com/CogEvol/CogEvol-4B && cd CogEvol-4B

# 3. build llama.cpp (needs the Qwen3.5 hybrid-arch support; repo README §5)

# 4. serve — validated flags, automatic CPU fallback
./scripts/serve.sh /path/to/cogevol-4b-q4_k_m.gguf
# in essence: llama-server -m cogevol-4b-q4_k_m.gguf -c 32768 -ngl 99 \
#   --temp 0 -fa auto --jinja --chat-template-kwargs '{"enable_thinking": false}'

# 5. go end-to-end (OpenMAIC classroom on-device, offline demo): repo README §7–8

Settings that matter (full table in the repo README):

• Thinking must stay off — the --jinja --chat-template-kwargs '{"enable_thinking": false}' flags above make it impossible to forget;
• Token budgets: slide → 8192; interactive HTML → 16384 with context ≥ 24576 (a rich page reaches ~19k characters);
• Q4 outputs run ~10–20% more tokens than BF16 (longer, but contract-valid);
• Old macOS Metal crash (GGML_ASSERT(buf_dst)): the repo's serve.sh falls back to CPU automatically, and a one-function patch restores GPU speed (repo §11).

Reference numbers (M2 Pro, 16 GB, Metal): 20/20 valid slide JSON contracts at 34.8 tok/s mean (~63 s per slide); 20/20 valid interactive HTML under deterministic browser probing. CPU-only fallback: ~18–24 tok/s.

Links

• Full-precision weights: https://huggingface.co/CogEvol/CogEvol-4B
• GitHub (deployment guide + slide system prompt): https://github.com/CogEvol/CogEvol-4B
• Tech report: arXiv:2608.30968
• Contact: [email protected]

Citation

If you find CogEvol useful, please cite:

@misc{tu2026cogevolefficientreliablelearning,
      title={CogEvol: Towards Efficient and Reliable Learning Environment Generation}, 
      author={Shangqing Tu and Daniel Zhang-Li and Yucheng Wang and Shiyu Gan and Yanpeng Wang and Huiqiang Rong and Mofei Chen and Shen Yang and Yini Chen and Yinuo Duan and Binglin Liu and Ye He and Danqi Zheng and Zhanxin Hao and Yuxuan Wu and Mengting Tao and Yuqiu Liu and Jifan Yu and Juanzi Li and Bin Xu and Lei Hou and Huiqin Liu and Yu Zhang},
      year={2026},
      eprint={2608.30968},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.30968}
}