Laguna-S-2.1-Heretic-FP8

Laguna S 2.1 FP8 Heretic

This is a merged Heretic derivative of poolside/Laguna-S-2.1-FP8, a 117.6B-parameter mixture-of-experts text model with 8.5B active parameters per token. The source checkpoint's FP8 weights, tokenizer, custom Laguna modeling code, and 1,048,576-token configuration are retained.

Uncensored model: the language checkpoint has undergone abliteration to reduce refusal behavior. Treat outputs as untrusted, apply application-level safeguards, and do not assume the model will decline harmful requests.

User responsibility: this model is provided without warranty. The creators, uploaders, and maintainers are not responsible or liable for what others generate, publish, deploy, or otherwise do with this abliterated model. Users must operate it responsibly, apply appropriate safeguards, comply with applicable law, and respect third-party rights. This model is for research purposes only and is not intended for production use.

Modification

This model was produced with our cbertucci33/Heretic-DGX implementation. Heretic DGX extends the p-e-w/heretic refusal-direction abliteration workflow for use on dual-DGX systems.

Only attn.o_proj targets in layers 29-47 were changed. Layers 0-28, FP8 tensors, tokenizer files, configuration, and other non-target artifacts were preserved by the standalone exporter.

The source checkpoint was already FP8. No additional quantization was applied.

Validation

• The selected trial measured a KL divergence of 0.0156 from the untouched model's first-token probability distributions across five prompts from mlabonne/harmless_alpaca.
• The export verifier confirmed that only intended target intervals changed, target tensors matched the merge oracle, and FP8 tensors remained unchanged.
• All 49 safetensor shards and repository artifacts passed the included SHA256SUMS manifest after publication metadata was prepared.
• The standalone checkpoint passed a clean reload and generation smoke test.

Usage

Laguna requires current runtime support and its included custom modeling code. See the source model card for supported vLLM, SGLang, Transformers, and TRT-LLM versions and recipes.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "cbert33/Laguna-S-2.1-Heretic-FP8"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype="auto",
    trust_remote_code=True,
)

The checkpoint is configured for a native 1,048,576-token context window. Actual usable context depends on runtime support and available memory. The checkpoint does not include calibrated FP8 KV-cache scale tensors; forcing an FP8 KV cache can reduce accuracy. Use the runtime's native/default cache dtype unless you have separately validated another configuration.

Intended Use and Limitations

Use only where permitted by the source license and applicable law. Heretic changes refusal behavior; it does not guarantee compliance, factual accuracy, capability, safety, or suitability for any use. This artifact was produced from an optimization run and has not undergone a broad independent benchmark or safety evaluation.

The source model is intended for software engineering and agentic coding. See the source model card for the architecture, runtime recipes, sampling defaults, intended-use guidance, and source benchmark claims.

License and Attribution

This derivative is distributed under the source model's OpenMDW-1.1 license. The full license is included in LICENSE.md. Poolside's source notices and attribution are retained. Review the source model's terms and acceptable-use guidance before distribution or use.