Downloads · 30 days
23
12% of all-time downloads
TamiLM-Hebrew-Nano
Wissotsky/TamiLM-Hebrew-Nano
TamiLM-Hebrew-Nano is a text generation model from Wissotsky. Use it when you need the model to write or continue text. The card lists the license as agpl-3.0.
A Modern Hebrew specialized LLM based on the RWKVv6 Architecture Trained only on Modern Hebrew datasets, with a custom vocabulary optimized for Modern Hebrew
All-time downloads
199
Public
Repo size
74.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pth74.1 GB · 100%
At a glance
- Task
- Text Generation
- License
- agpl-3.0
- Access
- Public
- Created
- Oct 22, 2024
- Updated
- Oct 24, 2024
- SHA
- 12048f22
Model card
From the Hugging Face model README
TamiLM Hebrew Nano
A Modern Hebrew specialized LLM based on the RWKVv6 Architecture
Trained only on Modern Hebrew datasets, with a custom vocabulary optimized for Modern Hebrew
Trained at Tel Aviv Makers Hackerspace
Params
Layers 12
Depth 512
Head size 64
Train ctx_len 512
Train tokens 6,841,411,389 (6 Billion)
Vocab size 65536
Train Compute
All compute was performed on a single Nvidia P40 card
Experiments: 62 hours 52 Minutes (2.6 days)
Training run: 208 hours 10 Minutes (8.6 days)
How to run
- Load model into your favourite RWKV v6 runtime
- Swap RWKV tokenizer with Huggingface tokenizers
- Load vocab json from this repo
- Task
- Text Generation
- License
- agpl-3.0
- Languages
- he
- Created
- Oct 22, 2024
- Updated
- Oct 24, 2024