Downloads · 30 days
5.8K
7% of all-time downloads
0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther
Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther is a text generation model from 0xgr3y. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
<h1 align="center"Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm Agent-ID (talltamepanther)</h1
Downloads · 30 days
5.8K
7% of all-time downloads
All-time downloads
83.4K
Public
Parameters
494M
1.5 TB on disk
Likes
2
Public
Click a slice to open those files.
.gguf2.7 GB · 73%
From the Hugging Face model README
Our pick an experimental (advanced) mode at this model a continuously trained Qwen2.5-Coder-0.5B-Instruct fine-tuned using Gensyn RL-Swarm framework with GRPO (Group Relative Policy Optimization) and supported format GGUF (llama.cpp) for enhanced code generation capabilities. Note: Current training focuses on programming challenges with adaptive weighted sampling.
tall_tame_pantherThe model is trained on a composite dataset with adaptive weighted sampling strategy:
| Dataset | Initial Weight | Adaptive Range | Focus Area |
|---|---|---|---|
| MBPP | 5 | 4-6 | Basic Python programming problems with test cases |
| CodeContests | 5 | 4-6 | Competitive programming challenges |
Total Dataset Size: Streaming datasets with infinite iteration
Training Samples per Round: 2
Evaluation: Real-time via Swarm Coordination with Ollama-based evaluator else Judge
"When the solvers perform well, the proposer automatically increases the difficulty to keep challenging solvers to get better over time." - CodeZero-blog
The implementation features an adaptive sampling system that adjusts dataset weights based on performance
The system monitors performance metrics every 5 rounds and adjusts the dataset weights to maintain optimal learning balance
- Update dataset weights based on recent performance
- Calculate recent average performance for each dataset
- Adjust/use weighted sampling if adaptive, based on perform difference
- Performance better on MBPP (Mostly Basic Python Problems)
- Performance better on CodeContests
- Update dataset weights every rounds & keep balanced
"Rewards are derived from multiple lightweight checks, ranging from code validity and formatting to alignment with the problem statement, combined into a single interpretable score." - CodeZero-blog
The reward system includes a quality data enhanced mechanism that evaluates code structure and documentation
- Calculate quality data enhanced for well-structured code
- Documentation enhanced
- Structure enhanced
- Algorithmic efficiency (simple heuristic)
- Scale with base reward to avoid inflation
The system also includes an adaptive threshold mechanism that adjusts based on recent performance
- Function adaptive threshold based on recent performance
- Performance quality data is consistently high
Based on our simulation with 1000 samples, the adaptive reward system shows significant improvement
| System | MBPP Avg Reward | CodeContests Avg Reward | Overall Avg Reward | Improvement |
|---|---|---|---|---|
| Original | 0.234 | -0.156 | 0.039 | - |
| Adaptive | 0.312 | -0.098 | 0.107 | ~174% |
Based on the logs provided, the model shows consistent progress:
Metric data visualize train/loss by Weights & Biases (WanDB)
[2025-11-14 04:22:50,632][genrl.logging_utils.global_defs][INFO] - __ Joining round: 13053
[2025-11-14 04:23:50,633][genrl.logging_utils.global_defs][INFO] - Starting round: 13053/100000.
Map: 100%|______________________________________| 1/1 [00:00<00:00, 158.65 examples/s]
Map: 100%|______________________________________| 1/1 [00:00<00:00, 191.92 examples/s]
[2025-11-14 04:25:12,646][genrl.logging_utils.global_defs][INFO] - pushing model to huggingface
Processing Files (1 / 1) : 100%|___| 988MB / 988MB, 94.3MB/s
New Data Upload : 100%|___| 983MB / 983MB, 94.3MB/s
.....kpb5lid/model.safetensors: 100%|___| 988MB / 988MB, 94.3MB/s
[2025-11-14 04:27:01,877][genrl.logging_utils.global_defs][INFO] - Already finished round: 13053. Next check in 160.0s.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther")
prompt = "Write a function to calculate the factorial of a number."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_length=256, temperature=0.7, top_p=0.8)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther")
tokenizer = AutoTokenizer.from_pretrained("0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther")
messages = [
{"role": "system", "content": "You are an expert Python programmer."},
{"role": "user", "content": "Write a function to check if a string is a palindrome."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=512)
print(tokenizer.decode(outputs[0]))
docker run -d --gpus all \
-p 8080:80 \
-v $PWD/data:/data \
ghcr.io/huggingface/text-generation-inference:latest \
--model-id 0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther \
--max-input-length 4096 \
--max-total-tokens 8192
# Download quantized model (recommended: Q4_K_M)
wget https://huggingface.co/0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther/resolve/main/Qwen2.5-Coder-0.5B-Q4_K_M.gguf
# Run inference
./llama-cli -m Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-Q4_K_M.gguf \
-p "Write a function to implement binary search in Python." \
--temp 0.7 --top-p 0.8
# Create Modelfile
cat > Modelfile << 'EOF'
FROM ./0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther/Qwen2.5-Coder-0.5B-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
SYSTEM "You are an expert Python programmer who writes clean, documented code."
EOF
# Create and run
ollama create qwen2.5-coder-swarm -f Modelfile
ollama run qwen2.5-coder-swarm "Write a function to calculate the factorial of a number."
| Format | Size | Precision | Use Case | Download |
|---|---|---|---|---|
| Safetensors (BF16) | 988 MB | BF16 | Full precision training/fine-tuning | model.safetensors |
| GGUF F16 | 994 MB | FP16 | High quality inference | Qwen2.5-Coder-0.5B-F16.gguf |
| GGUF Q6_K | 506 MB | 6-bit | High quality compression | Qwen2.5-Coder-0.5B-Q6_K.gguf |
| GGUF Q5_K_M | 420 MB | 5-bit | Balanced quality/size | Qwen2.5-Coder-0.5B-Q5_K_M.gguf |
| GGUF Q4_K_M | 398 MB | 4-bit | Recommended for production | Qwen2.5-Coder-0.5B-Q4_K_M.gguf |
| GGUF Q3_K_M | 355 MB | 3-bit | Smallest, fastest | Qwen2.5-Coder-0.5B-Q3_K_M.gguf |
All GGUF formats are llama.cpp is compatible ready to use Inferences chat and auto-update be hourly.
This model inherits Qwen2.5's chat template for structured conversations.
<|im_start|>system
{system_message}
<|im_end|>
<|im_start|>user
{user_message}
<|im_end|>
<|im_start|>assistant
{assistant_response}
<|im_end|>
Note: While model supports chat format structurally, optimal conversational performance depends on whether training data included formatted dialogues. Current training focuses on programming challenges.
Training Framework:
- Method: GRPO (Group Relative Policy Optimization)
- Base Model: Qwen/Qwen2.5-Coder-0.5B-Instruct
- Training Regime: bfloat16 mixed precision
- Max Rounds: 100000
- Update Frequency: Every 5-10 minutes
- Generations per Round: 2
- Batch size: Combine
- Tree-based Model: 2 tree
- Seed: 42
Blockchain Integration:
- Network: Gensyn Testnet
- Chain ID: 685685
- Contract: SwarmCoordinator v0.4.2
Swarm Communication:
- Framework: Hivemind P2P Backend
- Initial Peers: 3 bootnodes
- Beam Size: 10
Reward System:
- Manager: RewardManager (SwarmGameManager/CodeGenerationRewards)
- Reward Function: Adaptive with quality enhanced
- Evaluator: Ollama (qwen2.5-coder:1.5b-instruct)
- Judge API: https://codezero-judge.gensyn.ai
This model excels at:
| Format | Frequency | Trigger |
|---|---|---|
| Safetensors (BF16) | Every 5-10 min | Automatic via RL-Swarm |
| GGUF (all formats) | Every 3 hour | Auto-conversion pipeline |
Auto-Conversion Pipeline:
model.safetensorsCheck commit history for exact timestamps.
1. Agent joins swarm via P2P network
2. Coordinator assigns round via smart contract
3. Agent samples data from adaptive weighted datasets
4. Model generates 2 responses
5. Ollama evaluator assesses and assigns rewards with quality enhanced
6. GRPO updates policy based on rewards
7. Updated model shared via DHT
8. Best checkpoint saved to HuggingFace
9. Repeat
Training GPU:
Training CPU Optimize:
Inference:
| Metric | Value | Target |
|---|---|---|
| Completed Rounds | 13,533+ | 100,000 |
| Training Progress | 13.53% | 100% |
| Update Frequency | 5-10 min | Continuous |
Note: average@k: Average performance across k attempts, measuring consistency. pass@k: Probability of at least one correct solution in k attempts, measuring capability.Current metrics track training rounds completed in decentralized swarm.
Our adaptive reward system has shown approximately ~174% improvement in reward scores compared to the baseline system:
Original:
Overall Avg Reward: 0.039
MBPP Avg Reward: 0.234
CodeContests Avg Reward: -0.156
Adaptive:
Overall Avg Reward: 0.107
MBPP Avg Reward: 0.312
CodeContests Avg Reward: -0.098
Improvement: 0.068 (~174% increase)
@misc{qwen2.5-coder-gensyn-swarm-2025,
author = {0xgrey},
title = {Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm: Continuous RL Training on Distributed Swarm with Adaptive Rewards},
year = {2025},
publisher = {HuggingFace},
howpublished = {\url{https://huggingface.co/0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther}},
note = {Agent ID: tall\_tame\_panther}
}
@misc{gensyn-rl-swarm-2025,
title = {Gensyn RL-Swarm: Decentralized Reinforcement Learning Framework},
author = {Gensyn AI},
year = {2025},
url = {https://gensyn.ai}
}
@misc{codezero-2025,
title = {CodeZero: A Collaborative Coding Environment for Distributed RL},
author = {Gensyn AI},
year = {2025},
url = {https://docs.gensyn.ai/testnet/rl-swarm/how-it-works/codezero}
}
⚠️ Important: This is a continuously trained model. For reproducibility, specify commit hash:
git clone https://huggingface.co/0xgr3y/Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther
cd Qwen2.5-Coder-0.5B-Instruct-Gensyn-Swarm-tall_tame_panther
git checkout <commit-hash>
Trained with 🩷 using Gensyn RL-Swarm
</div>