ivio05

llm-course-hw1

ivio05/llm-course-hw1

llm-course-hw1 is a text generation model from ivio05. Use it when you need the model to write or continue text.

First homewok for NLP course in MSU from VK. Causal Transformer with Alibi bias and SwiGLU MLP. Can generate jokes in Russian language. Only the very small "nano" model with only 0.51 million parametrs and 1024 vocabl…