Tiny9

Tiny9 🤏🧠

Tiny9 is an extremely tiny neural network containing exactly 9 trainable parameters.

Yes. Nine. Not 9 million. Not 9 thousand. Just 9 parameters.

📊 Model Stats

🧠 What does it do?

Tiny9 learns a simple mapping between numbers.

The current experiment uses:

0 → 1
1 → 2
2 → 3
3 → 4
4 → 5
5 → 6
6 → 7
7 → 8
8 → 9
9 → 0

Because the model only has 9 parameters and uses a modulo-9 lookup, 0 and 9 share the same parameter. As a result, the trained model learns approximately:

0 → 0.50
1 → 2.00
2 → 3.00
3 → 4.00
4 → 5.00
5 → 6.00
6 → 7.00
7 → 8.00
8 → 9.00
9 → 0.50

This is intentional: the project demonstrates just how small a trainable PyTorch model can be.

🏗️ Architecture

The model contains a single trainable tensor:

self.w = nn.Parameter(torch.randn(9))

That's it.

The forward pass performs a lookup:

return self.w[x % 9]

Therefore:

9 parameters × 4 bytes per float32 = 36 bytes of raw parameter data.

The .pt file is larger because PyTorch also stores serialization and checkpoint metadata.

📦 Loading the model

import torch
import torch.nn as nn

class Tiny9(nn.Module):
    def __init__(self):
        super().__init__()
        self.w = nn.Parameter(torch.randn(9))

    def forward(self, x):
        return self.w[x % 9]

model = Tiny9()
model.load_state_dict(torch.load("tiny9.pt", weights_only=True))
model.eval()

print(model(torch.tensor(5)).item())

Expected output:

~6.0

⚠️ Limitations

Tiny9 is not a practical language model.

It has only 9 parameters, so it cannot store meaningful language knowledge or perform general-purpose reasoning.

This project is primarily an experiment in:

• Extremely small neural networks
• Parameter counting
• PyTorch serialization
• Learnable lookup tables
• Seeing how far you can push the concept of a "tiny model"

📜 License

This project is released for experimentation and educational purposes.