Downloads · 30 days
0
bdh-shakespeare
akarki15/bdh-shakespeare
bdh-shakespeare is a text generation model from akarki15. Use it when you need the model to write or continue text. The card lists the license as mit.
A 25M parameter biologically-inspired language model trained on the Tiny Shakespeare dataset. Generates Shakespearean-style English text using byte-level tokenization.
Access
Public
Updated Jul 5, 2026
Repo size
304 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.pt304 MB · 100%
At a glance
- Task
- Text Generation
- License
- mit
- Access
- Public
- Created
- Jul 4, 2026
- Updated
- Jul 5, 2026
- SHA
- 49ad4745
Try a prompt
Model card
From the Hugging Face model README
BDH Shakespeare — Baby Dragon Hatchling
A 25M parameter biologically-inspired language model trained on the Tiny Shakespeare dataset. Generates Shakespearean-style English text using byte-level tokenization.
Architecture
BDH (Baby Dragon Hatchling) is a compact, biologically-inspired transformer:
| Component | Value |
|---|---|
| Parameters | 25M |
| Layers | 6 (shared weights) |
| Embedding dim | 256 |
| Attention heads | 4 |
| Context length | 128 |
| Vocab size | 256 (byte-level) |
Key features:
- Shared weights across layers (iterative refinement)
- Byte-level tokenization (no external tokenizer needed)
- Hebbian gating (multiplicative sparse activations)
- ReLU sparsity for interpretable activations
- RoPE position embeddings
Training
- Dataset: Tiny Shakespeare (~1MB)
- Optimizer: AdamW (lr=1e-3, weight_decay=0.1)
- Steps: 5,000
- Hardware: Google Colab T4 GPU (~30 min)
- Val loss: 1.57 | Perplexity: 4.81
Usage
import torch
model_data = torch.hub.load_state_dict_from_url(
"https://huggingface.co/akarki15/bdh-shakespeare/resolve/main/bdh_shakespeare.pt",
map_location="cpu"
)
# Load with BDH model class — see GitHub repo for full inference code
Reference
Based on The Dragon Hatchling architecture by Pathway.
- Task
- Text Generation
- License
- mit
- Languages
- en
- Created
- Jul 4, 2026
- Updated
- Jul 5, 2026