Fredtt3-LLaDA-100M-Test-GGUF

GGUF Files for LLaDA-100M-Test

These are the GGUF files for Fredtt3/LLaDA-100M-Test.

Downloads

Note from Flexan

I provide GGUFs and quantizations of publicly available models that do not have a GGUF equivalent available yet, usually for models I deem interesting and wish to try out.

If there are some quants missing that you'd like me to add, you may request one in the community tab. If you want to request a public model to be converted, you can also request that in the community tab. If you have questions regarding this model, please refer to the original model repo.

You can find more info about me and what I do here.

New checkpoint trained on an NVIDIA H100 for 8,000 steps and 65,536,000 tokens

It is not yet a competent model because it does not meet the minimum training requirement of 20-30 tokens per parameter. However, it can give us a better idea of how a better-trained model would perform.

If you want to try how to use it here is a file of how to use it in test_gen.py Or using this Google Colab notebook

Example of the results it gives:

For those who want to train and get the correct format to be able to load it with transformers, everything needed is in pre_trainv2.py of the project repo