Downloads · 30 days
5.6K
11% of all-time downloads
NeoHorse-1-4B-GGUF
TokenRhythm/NeoHorse-1-4B-GGUF
NeoHorse-1-4B-GGUF is a text generation model from TokenRhythm. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
<div align="center" <h1NeoHorse-1-4B-GGUF</h1 <p<bTowards Recursive Self-Improvement via Agentic Post-Training with Routing Harness.</b</p </div
All-time downloads
51.6K
Public
Repo size
27.1 GB
Likes
50
Public
Repo makeup
Click a slice to open those files.
.gguf27.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Access
- Public
- Created
- Sep 8, 2026
- Updated
- Sep 10, 2026
- SHA
- 3c5d58ca
Try a prompt
Base models
Model card
From the Hugging Face model README
<div align="center">
<h1>NeoHorse-1-4B-GGUF</h1>
<p><b>Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness.</b></p>
</div>
<div align="center">
<a href="https://github.com/TokenRhythm/NeoHorse"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-NeoHorse-181717?logo=github&logoColor=white"></a>
<a href="https://tokenrhythm.ai/"><img alt="Company" src="https://img.shields.io/badge/Company-TokenRhythm-F97316?logo=homeassistant&logoColor=white"></a>
<a href="https://huggingface.co/TokenRhythm"><img alt="Hugging Face" src="https://img.shields.io/badge/Hugging%20Face-Models-FFD21E?logo=huggingface&logoColor=000000"></a>
<a href="https://x.com/opensquilla"><img alt="Twitter / X" src="https://img.shields.io/badge/Twitter%20%2F%20X-OpenSquilla-111827?logo=x&logoColor=white"></a>
<a href="https://www.apache.org/licenses/LICENSE-2.0"><img alt="License: Apache-2.0" src="https://img.shields.io/badge/License-Apache--2.0-64748B"></a>
</div>
<p align="center">
<a href="https://arxiv.org/abs/2609.08183"><b>Technical Report</b></a>
</p>
<style>
/* Reusable benchmark table architecture. Inline styles remain as a fallback for HF rendering. */
.vl-table {
width: 100%;
min-width: 100%;
border-collapse: collapse;
table-layout: fixed;
font-size: 15px;
}
.vl-table th {
font-size: 15px !important;
line-height: 1.2;
color: #c2410c;
background: rgba(249,115,22,.10);
}
.vl-table td:not(.benchmark-cell):not([colspan]) {
font-size: 15px;
line-height: 1.2;
vertical-align: middle;
}
.vl-table .benchmark-cell {
padding: 12px 10px 12px 18px !important;
vertical-align: middle;
}
.vl-table .benchmark-capability {
font-size: 15px;
font-weight: 600;
line-height: 1.22;
color: #c2410c;
}
.vl-table .benchmark-name {
margin-top: 4px;
font-size: 11px;
font-weight: 400;
line-height: 1.2;
color: inherit;
}
.vl-table .metric-stack {
display: flex;
flex-direction: column;
gap: 7px;
padding: 3px 0;
}
.vl-table .metric-label {
font-size: 10px;
font-weight: 400;
line-height: 1.1;
color: inherit;
}
.vl-table .metric-value {
margin-top: 2px;
font-size: 15px;
line-height: 1.15;
color: inherit;
}
.model-table td:first-child {
width: 34%;
font-weight: 600;
}
/* HF's theme toggle sets the dark class on an ancestor. */
.dark .vl-table th,
.dark .vl-table .benchmark-capability {
color: #fdba74 !important;
}
</style>
NeoHorse-1-4B is a 4B causal language model and an initial prototype on the path toward recursive self-improvement (RSI). It is post-trained from Qwen3.5-4B for text-based agent harnesses, tool use, coding, and instruction following.
This repository provides GGUF versions of NeoHorse-1-4B for local use with llama.cpp, Ollama, and LM Studio. It includes 16-bit (BF16) weights and smaller 8-bit, 5-bit, and 4-bit quantized versions. Quantized versions take up less disk space and use less memory, making the model easier to run on your own hardware. These files contain text-only model weights, fine-tuned by TokenRhythm from Qwen3.5-4B.
<p align="center"> <a href="https://huggingface.co/TokenRhythm/NeoHorse-1-4B/resolve/main/4B_head_fig.jpg"> <img src="https://huggingface.co/TokenRhythm/NeoHorse-1-4B/resolve/main/4B_head_fig.jpg" alt="NeoHorse-1-4B evaluation results" width="100%"> </a> </p>Highlights
- Path toward RSI: the routing harness assigns tasks to a heterogeneous model pool, records tool interactions and outcomes, estimates capability demand, and uses capability-level feedback to shape the next training mixture. Updated models can return to the harness, closing a prototype evaluation–selection–update loop; extending this loop across successive iterations is the next step toward RSI.
- Agentic post-training framework: the associated research explores routing-guided curriculum SFT and routing-guided on-policy distillation to turn execution trajectories into training signal while preserving execution and harness context around each response.
- Data quality: exact and near-duplicate removal, evaluation decontamination, structural validation, six-dimensional semantic evaluation, and subscene-level Scene/Goal/Outcome labeling.
- Broad gains: 64.87 macro average across ten benchmarks versus 58.94 for Qwen3.5-4B (+5.93).
Model Details
<div style="width:100%;max-width:none;margin:16px 0;padding:0;overflow-x:auto"> <table class="vl-table model-table" width="100%" style="display:table;width:100%;min-width:100%;table-layout:fixed;border-collapse:collapse;font-size:13px"> <thead><tr> <th style="padding:9px 10px;text-align:left;border-bottom:2px solid #f97316;color:#c2410c;background:rgba(249,115,22,.10)">Property</th> <th style="padding:9px 10px;text-align:left;border-bottom:2px solid #f97316;color:#c2410c;background:rgba(249,115,22,.10)">Value</th> </tr></thead><tbody> <tr> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16);font-weight:600">Model family</td> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16)">NeoHorse Agent-Native Causal Language Model</td> </tr> <tr> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16);font-weight:600">Parameters</td> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16)">Approximately <strong>4B</strong></td> </tr> <tr> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16);font-weight:600">Base model</td> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16)"><a href="https://huggingface.co/Qwen/Qwen3.5-4B">Qwen3.5-4B</a></td> </tr> <tr> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16);font-weight:600">Post-training</td> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16)">Routing-guided agentic post-training</td> </tr> <tr> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16);font-weight:600">Interface</td> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16)">Text input and text output</td> </tr> <tr> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16);font-weight:600">Context length</td> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16)">262,144 natively and extensible up to 1,010,000 tokens.</td> </tr> <tr> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16);font-weight:600">Weight format / precision</td> <td style="padding:9px 10px;border-bottom:1px solid rgba(249,115,22,.16)">GGUF / 16-bit (BF16), 8-bit, 5-bit, 4-bit</td> </tr> </tbody></table> </div>Evaluation
The results below are from the original checkpoint, not a separate GGUF benchmark run.
The 4B track compares NeoHorse-1-4B with five representative open-weight models. Results are grouped by capability in the table below. Higher is better; Δ is NeoHorse-1-4B minus Qwen3.5-4B. Bold marks the best available result; <ins>underlining</ins> marks the second-best.
* Nanbeige-4.2-3B LiveCodeBench v6 result is reported in the corresponding model's official blog post or technical report.
Reported protocol: SGLang v0.5.17 ·
temperature=1.0·top_p=0.95·top_k=20·min_p=0.0·presence_penalty=1.5·repetition_penalty=1.0· thinking mode enabled withenable_thinking=trueandforce_nonempty_content=true. QwenClawBench, WorkBuddy Bench, and tau2-Bench use three runs; PinchBench and VitaBench use one run; the remaining benchmarks follow their official protocols. VitaBench uses the DeepSeek-V4-Flash simulator and judge.
Deployment
<!-- neohorse-f16 -->FP16 option: NeoHorse-1-4B-F16.gguf (8.42 GB) is also available for runtimes or hardware that prefer FP16 over BF16. Use :F16 instead of :BF16 in the llama.cpp or Ollama commands below. This is a format-converted 16-bit option, not a quality upgrade.
Quantized files
| Version | File | Size |
|---|---|---|
| 8-bit | NeoHorse-1-4B-Q8_0.gguf | 4.48 GB |
| 5-bit | NeoHorse-1-4B-Q5_K_M.gguf | 3.07 GB |
| 4-bit | NeoHorse-1-4B-Q4_K_M.gguf | 2.71 GB |
4-bit is the smallest option, 5-bit sits in the middle, and 8-bit uses less compression. Choose 16-bit (BF16) to keep the original weight precision. Lower precision saves memory but can affect response quality. The sizes above are file sizes; running the model also needs memory for the conversation and other runtime data.
These standard llama.cpp quantizations were generated directly from the BF16 GGUF, without an importance matrix. All three passed eight short llama.cpp compatibility checks, including thinking, single and parallel tool calls, and tool-result continuation, at 4K context. These checks are not a quantized-quality benchmark; the benchmark scores in this card refer to the original checkpoint. Quantized variants have not yet been separately tested in Ollama or LM Studio. Use the desired filename in place of the BF16 filename in the examples below.
<!-- /neohorse-quantized-files -->Use a recent runtime with Qwen3.5 support. The commands below download the model from Hugging Face and cache it locally; no manual file download is required. These are text-only GGUF files with an embedded chat template and no MTP draft head.
llama.cpp
Download and start a local OpenAI-compatible server:
llama-server -hf TokenRhythm/NeoHorse-1-4B-GGUF:BF16 -c 4096 -ngl 99 --jinja --port 8080 --alias neohorse-1-4b
Ollama
Download and run directly, without creating a Modelfile:
ollama run hf.co/TokenRhythm/NeoHorse-1-4B-GGUF:BF16
LM Studio
Paste this repository URL into LM Studio's model search and choose a GGUF file, or download through the CLI:
lms get https://huggingface.co/TokenRhythm/NeoHorse-1-4B-GGUF
Select the desired file when prompted, then load it in LM Studio with MTP speculative decoding disabled. Start with a 4K context and increase it according to available memory. In LM Studio's native API, use reasoning: "off" to disable thinking.
For llama.cpp and Ollama, replace :BF16 with the tag of another variant listed in Files and versions, such as :Q8_0, :Q5_K_M, or :Q4_K_M. Only variants already present in the repository can be downloaded.
See the llama.cpp guide, Ollama–Hugging Face integration, and LM Studio download guide.
<details> <summary>Manual download and local-file setup (optional)</summary>Download NeoHorse-1-4B-BF16.gguf (approximately 8.42 GB). Use a recent runtime with Qwen3.5 support. The file includes the chat template and does not include an MTP draft head.
llama.cpp
llama-server -m NeoHorse-1-4B-BF16.gguf -c 4096 -ngl 99 --jinja --port 8080 --alias neohorse-1-4b
Ollama
Create a Modelfile containing:
FROM ./NeoHorse-1-4B-BF16.gguf
PARAMETER num_ctx 4096
ollama create neohorse-1-4b-bf16 -f Modelfile
ollama run neohorse-1-4b-bf16
LM Studio
Import the GGUF file and load it with MTP speculative decoding disabled. With the CLI:
lms import ./NeoHorse-1-4B-BF16.gguf --copy
lms load neohorse-1-4b --context-length 4096 --no-speculative-draft-mtp
Basic chat and tool-call smoke tests passed in llama.cpp, Ollama 0.33.3, and LM Studio's llama.cpp 2.33.0 engine. Context capacity depends on available memory; the examples start at 4K. In LM Studio's native API, use reasoning: "off" to disable thinking.
Using NeoHorse-1-4B-GGUF via the Chat Completions API
After starting llama-server as shown in Deployment, connect an OpenAI-compatible client to http://127.0.0.1:8080/v1. The examples below use the server alias neohorse-1-4b; it must match the --alias value in the launch command.
Basic Usage
Install the Python client:
pip install openai
Send a message to your local server:
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8080/v1",
api_key="EMPTY", # Replace if server authentication is enabled.
)
generation_config = {
"temperature": 1.0,
"top_p": 0.95,
"presence_penalty": 1.5,
"max_tokens": 2048,
"extra_body": {
"top_k": 20,
"min_p": 0.0,
"repeat_penalty": 1.0,
"chat_template_kwargs": {"enable_thinking": True},
},
}
response = client.chat.completions.create(
model="neohorse-1-4b",
messages=[
{
"role": "user",
"content": "Write a Python function that returns the first n Fibonacci numbers.",
}
],
**generation_config,
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)
These short examples allow up to 2,048 output tokens within the 4K server context. For longer prompts or responses, increase -c and adjust max_tokens while leaving room for the input and chat template.
Tool Calling
Keep --jinja enabled in the llama-server launch command so it can use the embedded tool-aware chat template and parse generated function calls into tool_calls.
Using the same client and generation_config from Basic Usage:
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather for a specified city.",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "Name of the city.",
}
},
"required": ["city"],
},
},
}
]
response = client.chat.completions.create(
model="neohorse-1-4b",
messages=[
{
"role": "user",
"content": "What is the weather in Beijing right now?",
}
],
tools=tools,
tool_choice="auto",
**generation_config,
)
message = response.choices[0].message
if message.tool_calls:
for call in message.tool_calls:
print(call.function.name, call.function.arguments)
else:
print(message.content)
This example prints the requested function and its arguments. Your application executes the function and returns its result in a tool message, using the corresponding tool_call_id, to continue the conversation.
License
NeoHorse-1-4B is released under the Apache License 2.0.
The upstream model is Qwen/Qwen3.5-4B. Its original copyright notice, Copyright 2026 Alibaba Cloud, is retained in the license file. TokenRhythm has modified the model through fine-tuning and repackaging for text-only inference. This repository converts the resulting checkpoint to BF16 GGUF and provides quantized variants using llama.cpp.
Citation
@misc{neohorse2026,
title = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness},
author = {NeoHorse Team},
year = {2026},
howpublished = {arXiv preprint},
eprint = {2609.08183},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2609.08183}
}
For questions or issue reports, use the NeoHorse project repository.
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Created
- Sep 8, 2026
- Updated
- Sep 10, 2026