ailo-152m

AILO-152M: Transformer Language Model

AILO (Artificial Intelligence Language Operator) is a 152M parameter Transformer-based language model trained from scratch.

Model Details

Training

• Dataset: FineWeb-Edu (100B token sample, streamed)
• Training Steps: 182,000+
• Final Loss: ~3.0
• Training Time: ~64 hours
• Optimizer: AdamW with cosine LR schedule + warm restarts

Training Loss Curve

Quick Start

# Install dependencies
pip install torch transformers tiktoken huggingface_hub

# Download and use
from huggingface_hub import hf_hub_download
import torch
import sys

# Download model files
repo_id = "xxrickyxx/ailo-152m"
for f in ["config.json", "configuration_ailo.py", "modeling_ailo.py", "pytorch_model.bin"]:
    hf_hub_download(repo_id=repo_id, filename=f, local_dir="ailo_model")

# Load model
sys.path.insert(0, 'ailo_model')
from configuration_ailo import AILOConfig
from modeling_ailo import AILOForCausalLM
import tiktoken

config = AILOConfig.from_pretrained("ailo_model")
model = AILOForCausalLM(config)
state_dict = torch.load("ailo_model/pytorch_model.bin", map_location='cpu')
model.load_state_dict(state_dict, strict=False)
model.eval()

# Generate text
tokenizer = tiktoken.get_encoding("gpt2")
prompt = "What is artificial intelligence?"
tokens = tokenizer.encode(prompt)
input_ids = torch.tensor([tokens])

with torch.no_grad():
    output_ids = model.generate(input_ids, max_new_tokens=100, temperature=0.8)

print(tokenizer.decode(output_ids[0].tolist()))

🏆 AILO vs GPT-2 Arena Comparison

We compared AILO-152M against GPT-2 (124M) on various prompts. Despite similar size, AILO shows better coherence and fewer repetitions.

Example 1: What is artificial intelligence?

Example 2: Write a short story about a robot

Example 3: Tell me a joke

Summary

Intended Uses

• Text generation
• Fine-tuning for specific domains
• Educational purposes
• Research on small language models

Limitations

• Small model size (152M) limits capabilities compared to larger models
• May produce repetitive or incoherent text for complex queries
• Training data primarily in English

Files

Citation

@misc{ailo2026,
  title={AILO-152M: A Small Transformer Language Model},
  author={AILO Team},
  year={2026},
  howpublished={\url{https://huggingface.co/xxrickyxx/ailo-152m}}
}

📜 License This project uses a dual-license model.

🆓 Non-Commercial License Released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 (CC BY-NC-SA 4.0).

You are free to:

Use the model for research, education, and personal projects Modify and fine-tune the model Redistribute derivatives under the same license You must:

Provide attribution Keep the same license for derivative works Not use the model for commercial purposes 💼 Commercial License Commercial use of AILO-152M is not permitted under the free license. Commercial use includes:

Integration into paid products or services Use in SaaS platforms, APIs, or enterprise systems Any application that generates revenue directly or indirectly For commercial licensing, a separate paid agreement (royalty or license fee) is required. Please contact the author.

📬 Contact For research collaboration or commercial licensing inquiries, contact the project maintainer:

Riccardo Sparacino LinkedIn https://www.linkedin.com/in/riccardo-sparacino-developer-php-javascript-mysql-app-ios-android/it/?skipRedirect=true