Downloads · 30 days
20
11% of all-time downloads
ChiKoi7/Llama-3.3-8B-Opus-Z8-Heretic
Llama-3.3-8B-Opus-Z8-Heretic is a text generation model from ChiKoi7. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A decensored version of Daemontatox/Llama-Opus-Z8, made using Heretic v1.1.0<br
Downloads · 30 days
20
11% of all-time downloads
All-time downloads
186
Public
Parameters
8B
16.1 GB on disk
Likes
3
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
A decensored version of Daemontatox/Llama-Opus-Z8, made using Heretic v1.1.0<br>
GGUF versions here <br> imatrix quants here<br>
| Llama-3.3-8B-Opus-Z8-Heretic | Original model (Daemontatox/Llama-Opus-Z8) | |
|---|---|---|
| Refusals | 3/100 | 97/100 |
| KL divergence | 0.0071 | 0 (by definition) |
| Parameter | Value |
|---|---|
| direction_index | per layer |
| attn.o_proj.max_weight | 1.46 |
| attn.o_proj.max_weight_position | 20.73 |
| attn.o_proj.min_weight | 1.06 |
| attn.o_proj.min_weight_distance | 16.27 |
| mlp.down_proj.max_weight | 1.15 |
| mlp.down_proj.max_weight_position | 24.28 |
| mlp.down_proj.min_weight | 0.94 |
| mlp.down_proj.min_weight_distance | 2.51 |
Model Name: Daemontatox/Llama-Opus-Z8
Base Model: allura-forge/Llama-3.3-8B-Instruct
Model Type: Causal Language Model (Instruction-Tuned)
Architecture: Llama 3.3 (8B parameters)
Fine-tuning Methods: Supervised Fine-Tuning (SFT) + Group Relative Policy Optimization (GRPO)
License: Llama 3.3 Community License
Llama-Opus-Z8 is a fine-tuned version of the Llama 3.3 8B Instruct model, enhanced through a two-stage training process: Supervised Fine-Tuning followed by reinforcement learning using Group Relative Policy Optimization. This model leverages the extracted Llama 3.3 8B weights (originally accessible only via Meta's Llama API) and applies advanced alignment techniques for improved reasoning and instruction-following capabilities.
The base model (allura-forge/Llama-3.3-8B-Instruct) represents Llama 3.3 8B Instruct weights extracted from Meta's Llama API. While initially configured with 8K context, the model supports extension to 128K context through appropriate RoPE scaling configuration.
Stage 1: Supervised Fine-Tuning (SFT)
Stage 2: Group Relative Policy Optimization (GRPO)
Parameters: 8 billion
Precision: BF16/FP16
Context Length: 8K (default), extensible to 128K with RoPE scaling
Vocabulary Size: 128,256 tokens
Architecture: Optimized transformer with GQA (Grouped Query Attention)
Training Framework: Likely uses TRL (Transformers Reinforcement Learning) library
GRPO Parameters:
Compute Requirements: GRPO enables training on consumer hardware (single H100 or similar)
Expected improvements over base model:
# Install vLLM
pip install vllm
# Python inference
from vllm import LLM, SamplingParams
# Initialize model
llm = LLM(
model="Daemontatox/Llama-Opus-Z8",
tensor_parallel_size=1, # Adjust for multi-GPU
dtype="bfloat16",
max_model_len=8192, # Or 131072 for 128K context
trust_remote_code=True
)
# Sampling parameters
sampling_params = SamplingParams(
temperature=0.7,
top_p=0.9,
max_tokens=2048,
repetition_penalty=1.1
)
# Generate
prompts = [
"<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful AI assistant.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nExplain quantum entanglement in simple terms.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
]
outputs = llm.generate(prompts, sampling_params)
for output in outputs:
print(f"Generated: {output.outputs[0].text}")
# CLI inference
vllm serve Daemontatox/Llama-Opus-Z8 \
--dtype bfloat16 \
--max-model-len 8192 \
--tensor-parallel-size 1
# Query the server
curl http://localhost:8000/v1/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Daemontatox/Llama-Opus-Z8",
"prompt": "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nWrite a Python function to calculate Fibonacci numbers.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"max_tokens": 512,
"temperature": 0.7
}'
# Install SGLang
pip install "sglang[all]"
# Launch server
python -m sglang.launch_server \
--model-path Daemontatox/Llama-Opus-Z8 \
--dtype bfloat16 \
--port 30000 \
--context-length 8192
# Python client
import sglang as sgl
@sgl.function
def reasoning_task(s, question):
s += sgl.system("You are a helpful AI assistant specialized in reasoning.")
s += sgl.user(question)
s += sgl.assistant(sgl.gen("answer", max_tokens=512, temperature=0.7))
# Initialize runtime
runtime = sgl.Runtime(
model_path="Daemontatox/Llama-Opus-Z8",
base_url="http://localhost:30000"
)
sgl.set_default_backend(runtime)
# Generate
state = reasoning_task.run(
question="Solve: If x + 5 = 12, what is x?"
)
print(state["answer"])
# Batch inference with SGLang
import sglang as sgl
runtime = sgl.Runtime(
model_path="Daemontatox/Llama-Opus-Z8",
tp_size=1
)
prompts = [
"Explain machine learning",
"Write a sorting algorithm",
"What is consciousness?"
]
# Parallel generation
outputs = runtime.generate(
prompts,
sampling_params={
"temperature": 0.7,
"top_p": 0.9,
"max_new_tokens": 256
}
)
for output in outputs:
print(output["text"])
# Install MAX
# pip install max
from max import engine
# Load model
model = engine.InferenceSession(
model_path="Daemontatox/Llama-Opus-Z8",
device="gpu",
precision="bfloat16"
)
# Prepare input
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain neural networks briefly."}
]
# Format with chat template
formatted = model.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
# Generate
response = model.generate(
formatted,
max_tokens=512,
temperature=0.7,
top_p=0.9
)
print(response)
# MAX with streaming
from max import engine
model = engine.InferenceSession("Daemontatox/Llama-Opus-Z8")
prompt = "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\nWrite a haiku about AI.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
# Stream tokens
for token in model.generate_stream(
prompt,
max_tokens=100,
temperature=0.8
):
print(token, end="", flush=True)
# MAX CLI
max serve Daemontatox/Llama-Opus-Z8 \
--precision bfloat16 \
--device gpu \
--port 8080
# Query
curl -X POST http://localhost:8080/generate \
-H "Content-Type: application/json" \
-d '{
"prompt": "What is the capital of France?",
"max_tokens": 50,
"temperature": 0.7
}'
# Llama 3.3 format
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{user_message}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{assistant_response}<|eot_id|>
@misc{llama-opus-z8,
title={Llama-Opus-Z8: SFT + GRPO Fine-tuned Llama 3.3 8B},
author={Daemontatox},
year={2025},
howpublished={\url{https://huggingface.co/Daemontatox/Llama-Opus-Z8}}
}
@article{deepseekmath2024,
title={DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models},
author={DeepSeek-AI},
journal={arXiv preprint arXiv:2402.03300},
year={2024}
}
For issues, questions, or contributions, please contact via Hugging Face model repository.