Downloads · 30 days
1.2K
1% of all-time downloads
TimeCapsuleLLM-London-1800-1875-v2-1.2B
haykgrigorian/TimeCapsuleLLM-London-1800-1875-v2-1.2B
TimeCapsuleLLM-London-1800-1875-v2-1.2B is a text generation model from haykgrigorian. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
v2 model, trained from scratch on 112GB of 1800-1875 london texts using a Llama-based Casual Language Model.
All-time downloads
98.9K
Public
Parameters
1.2B
6.9 GB on disk
Likes
52
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Jan 6, 2026
- Updated
- Jan 13, 2026
- SHA
- 04c6e27a
Try a prompt
Datasets
Model card
From the Hugging Face model README
haykgrigorian/TimeCapsuleLLM-v2-London-1800-1875: Llama-Architecture 1.2B Model
Model Overview
v2 model, trained from scratch on 112GB of 1800-1875 london texts using a Llama-based Casual Language Model.
| Detail | Value |
|---|---|
| Model Architecture | LlamaForCausalLM (Decoder-Only Transformer) |
| Parameter Count | ~1.22B |
| Training Type | Trained from Scratch (Random Initialization) |
| Tokenizer | Custom BPE, Vocab Size 32,000 |
| Sequence Length | 2048 tokens |
| Attention Type | Grouped Query Attention (GQA) |
Configuration Details
This model is a custom size and configuration based on Llama:
| Parameter | Value |
|---|---|
| Number of Layers | 22 |
| Hidden Size (d) | 2048 |
| Intermediate Size ($\text{d}_{\text{ff}}$) | 5504 |
| Attention Heads | 16 (Query) / 8 (Key/Value) |
| Activation Function | SiLU (silu) |
| Normalization | RMS Norm (rms_norm_eps: 1e-06) |
| Position Embeddings | Rotary Positional Embeddings (RoPE) |
Training Info
This model was trained for 182,000 steps, about 0.5 epochs.
Training Metrics:
Final Training Loss: 3.3951
Start Training Loss: 10.7932
Training Steps: 182,000
Epochs: 0.4997
Gradient Norm Stability: Consistently stable between 0.50 and 0.60 in later stages.
Training time: 117 hours 51 minutes
Cost
This model was trained on an H100 SXM from RunPod
Total: $340.97
How to Load and Run the Model
Install all the files locally in a folder and run the test script. You will have to make some adjustments in the run script like updating the config/file path and test prompts
Test script
A run file for testing and evaluating this model is available on the main project repository:
- Test Script Link: run_v2.py on GitHub
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Languages
- en
- Created
- Jan 6, 2026
- Updated
- Jan 13, 2026