Downloads · 30 days
132
9% of all-time downloads
TimeCapsuleLLM-London-1800-1875-v2mini-eval1-300M
haykgrigorian/TimeCapsuleLLM-London-1800-1875-v2mini-eval1-300M
TimeCapsuleLLM-London-1800-1875-v2mini-eval1-300M is a text generation model from haykgrigorian. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
v2mini-eval1 model, trained from scratch on 15GB of 1800-1875 london texts using the modern Llama architecture. This model was trained for v2's dataset evaluation.
All-time downloads
1.4K
Public
Parameters
298M
1.2 GB on disk
Likes
39
Public
Repo makeup
Click a slice to open those files.
.safetensors1.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Dec 9, 2025
- Updated
- Dec 16, 2025
- SHA
- c762e9a8
Try a prompt
Model card
From the Hugging Face model README
haykgrigorian/v2mini-eval1: Llama-Architecture 318M Model
Model Overview
v2mini-eval1 model, trained from scratch on 15GB of 1800-1875 london texts using the modern Llama architecture. This model was trained for v2's dataset evaluation.
| Detail | Value |
|---|---|
| Model Architecture | LlamaForCausalLM (Decoder-Only Transformer) |
| Parameter Count | ~318 Million (318M) |
| Training Type | Trained from Scratch (Random Initialization) |
| Tokenizer | Custom BPE, Vocab Size 32,000 |
| Sequence Length | 1024 tokens |
| Attention Type | Grouped Query Attention (GQA) |
Configuration Details
This model is a custom size and configuration based on Llama:
| Parameter | Value |
|---|---|
| Number of Layers | 20 |
| Hidden Size (d) | 1024 |
| Intermediate Size ($\text{d}_{\text{ff}}$) | 2752 |
| Attention Heads | 16 (Query) / 8 (Key/Value) |
| Activation Function | SiLU (silu) |
| Normalization | RMS Norm (rms_norm_eps: 1e-05) |
| Position Embeddings | Rotary Positional Embeddings (RoPE) |
Model Issues
This is an evaluation model, it was trained from scratch using a 15GB sample from a 90GB dataset for 10k steps. There was a tokenization issue and output comes out like this:
-
default: "D oes that work more of h ise x cell ent st ir ring , in his pl ays"
-
fixed: "Does that work more of his excellent stirring, in his plays"
This is just a tokenizer issue, just fix the output yourself or if you're lazy feed it to an LLM and have it fixed.
How to Load and Run the Model
Install all the files locally in a folder and run the test script. You will have to make some adjustments in the run script like updating the config/file path and test prompts
Test script
A run file for testing and evaluating this model is available on the main project repository:
- Test Script Link: test_v2mini_eval1.py on GitHub
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Languages
- en
- Created
- Dec 9, 2025
- Updated
- Dec 16, 2025