CinnabarLM-1.5M-Base

CinnabarLM 1.5M

What happens if you take the CinnabarLM idea and push it further? You'll get this!

CinnabarLM 1.5M is a tiny, 1.5M-parameter LLM trained for ~28 minutes on a T4 GPU (on Colab)! It's only 6 MB in size and now it's Llama-based!

Why?

Because it's a good idea to make tiny LLMs. Some people already did with MicroLM, Spark 4 5M and Tenete 8M, but not myself!

Model Configurations

Training Configurations

Limitations

• Not Instruction-Tuned: It's only a base model, so it only completes text.
• English-Only: It's trained on English data (FineWeb), it's NOT multilingual.

Some other details

• It's trained on 50 million tokens of FineWeb (CC-MAIN-2025-26 snapshot), and the knowledge cutoff is June 2025.
• The name "CinnabarLM" that I picked was made by combining "Cinnabar" (the new block from the Chaos Cubed drop in Minecraft) + "LM" (Language Model)

Examples

• "Take some time to show you how this LLM generates text:" gives:

"Take some time to show you how this LLM generates text: love and fun. Your voice will be able to find out your style with a great fabric that will help your friend or family! I can't get more fun with us because we'll never give me the best idea why so I'm just happy to see all my favorite things at our time. Get a Curry Festival! Today, you've got the same gift of lucky cookies when it's been nice to take a look on. Please note: Google is not going to go to a lot of people looking for them. We're always happy to work"

• "Mihai Popa, which is 15 years old, which the person likes" gives:

"Mihai Popa, which is 15 years old, which the person likes a child is now being a bleeder who has been at the Time Summer. Presidents of Dragon's Day are the first time in the past six months to the school to the day of a hospital or office in San Francisville. The Best Times was also known as a family member. The school will be given for his life and family-friendly home, but it was a good place on a job to visit this site. They must have had a lot of moments from all around the world. However, there were some serious things about"

• "Even though Striders also require a stick item" gives:

"Even though Striders also require a stick item to create an outdoor space, but they are not just the perfect way to be in place. You can get a new look at the first time you’ll see how much the best thing we need is to make it easier for you! As always, our durable staff is incredibly easy and more than just one but for your newest team you will have to take some time in your day. The more you can give you everything in the best gift box? Let us know where you're going to share your style! From Cleaning Metaller to Western Distribution,"