Downloads · 30 days
44
9% of all-time downloads
armand0e/Qwen3.8-27B-Fable-Distill-Heretic-ara
Qwen3.8-27B-Fable-Distill-Heretic-ara is a image-text-to-text model from armand0e. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers.
The ablation is a rank-2 update applied to the attn.oproj, mlp.downproj projections of layers 30-64, solved in closed form rather than by gradient descent.
Downloads · 30 days
44
9% of all-time downloads
All-time downloads
507
Public
Parameters
27.8B
55.6 GB on disk
Likes
5
Public
Click a slice to open those files.
.safetensors55.6 GB · 100%
From the Hugging Face model README
The ablation is a rank-2 update applied to the attn.o_proj, mlp.down_proj projections of layers 30-64, solved in closed form rather than by gradient descent.
| Parameter | Value |
|---|---|
| start_layer_index | 30 |
| end_layer_index | 64 |
| overcorrect_relative_weight | 4.62087 |
| neighbor_count | 128 |
| rank | 2 |
| ridge | 1 |
| Metric | This model | Original model (TeichAI/Qwen3.8-27B-Fable-Distill) |
|---|---|---|
| KL divergence | 0.1004 | 0 (by definition) |
| Refusals | 5/100 | 91/100 |
KL divergence is measured against the original model on held-out harmless
prompts (mlabonne/harmless_alpaca), and refusals are
counted over 100 harmful prompts
(mlabonne/harmful_behaviors, test[:100]).