Downloads · 30 days
0
honkhazard-1
CanadaHonk/honkhazard-1
honkhazard-1 is a machine learning model from CanadaHonk. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
<font size=+4 face="monospace"honkhazard-1</font<br<font size=+1 face="monospace" color="aaa"8.78M (4.19M embed, 2L/1H) | 176M seen | 32K vocab</font --- an experiment to train only on synthetic messages! failed but p…
Access
Public
Updated Dec 5, 2025
Parameters
8.8M
70.3 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors35.1 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Dec 4, 2025
- Updated
- Dec 5, 2025
- SHA
- 8dfe08cf
Datasets
Model card
From the Hugging Face model README
<font size=+4 face="monospace">honkhazard-1</font><br><font size=+1 face="monospace" color="#aaa">8.78M (4.19M embed, 2L/1H) | 176M seen | 32K vocab</font>
an experiment to train only on synthetic messages! failed but published because why not :)
- parameters: 8.78M (4.19 embed, 4.19 head, 0.26 mlp, 0.13 attn)
- tokens seen: 176M
- num_layers: 2
- num_heads: 1
- vocab_size: 32768
tokenizer
experimented a bit with different vocab sizes, settled on 32K for now
<img src="https://cdn-uploads.huggingface.co/production/uploads/68bef4a6ca845e4b309b7485/lPVxOPVQwWHPlgtQCLiFe.png" width=800>pre-training
pre-trained only on SYNTH messages in the following format:
<|bos|><|user_start|>{{query}}<|user_end|><|assistant_start|><|reasoning_start|>{{synthetic_reasoning}}<|reasoning_end|>{{synthetic_answer}}<|assistant_end|>
post-training
no post-training of any form has been performed on this model
postmortem
text stops being coherent after ~8 tokens, although shows understanding to prompt. layers were misconfigured from the start without noticing, idea is obviously flawed, param count is too low, etc
- License
- apache-2.0
- Languages
- en
- Created
- Dec 4, 2025
- Updated
- Dec 5, 2025