crumb

core1-base-464m-c4

crumb/core1-base-464m-c4

core1-base-464m-c4 is a text generation model from crumb. Use it when you need the model to write or continue text. It is set up for transformers.

CoreX models are Llama models in which the first X decoder layers are kept, and then the model is finetuned on 1 billion tokens from some dataset. Base model stems from Llama2-7b, medium from Llama2-13b, xl from Llama…