Downloads · 30 days
10
19% of all-time downloads
S-ukhanov/home_run_llm
home_run_llm is a text generation model from S-ukhanov. Use it when you need the model to write or continue text. It is set up for transformers.
Downloads · 30 days
10
19% of all-time downloads
All-time downloads
52
Public
Parameters
1.2B
2.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.3 GB · 100%
From the Hugging Face model README
112 gb data with llama 1.2b model
| Фича | Детали |
|---|---|
| Архитектура | LlamaForCausalLM |
| Параметры | ~1.22 Миллиарда |
| Контекст | 2048 Токенов |
| Словарь | 32,000 (Custom BPE) |
| Тренировка | 182,000 шагов (где-то пол-эпохи прошло) |
| Параметр | Значение |
|---|---|
| Слои | 22 |
| Скрытый размер | 2048 |
| GQA | 16 (Q) / 8 (KV) |
| MLP | 5504 |
| Активация | SiLU |
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "home_run_llm"
tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)
prompt = "The foggy streets of London in 1840 were" inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100) print(tokenizer.decode(outputs[0], skip_special_tokens=True))
## Заметки по тренировке
- **Железо**: H100 SXM
- **Время**: ~118 часов.
- **Денег ушло**: ~350 долларов
- **Лосс**: Упал с 10.79 до 3.35.