my-bilingual-llm

My Bilingual LLM

Summary

My Bilingual LLM is a compact decoder-only Transformer language model designed for experimental Bangla and English text generation.

Architecture

• Parameters: approximately 41.87M
• Vocabulary size: 32,000
• Embedding dimension: 512
• Transformer layers: 8
• Attention heads: 8
• Maximum context length: 512
• Weight tying: enabled

Languages

• Bangla
• English
• Mixed Bangla-English input

Intended Use

This model is intended primarily for:

• research
• experimentation
• educational purposes
• portfolio demonstration
• language-model inference experiments

Limitations

This is an experimental model and is not production-ready.

Known limitations include:

• repetitive generation
• weak long-form coherence
• occasional mixed-language output
• incorrect factual statements
• unusual numeric sequences
• grammatical errors

Generation

• temperature: 0.78
• top_k: 50
• top_p: 0.92
• repetition_penalty: 1.12
• no_repeat_ngram_size: 3
• max_new_tokens: 80

Checkpoint Integrity

Model SHA256:

0acef793ab3449d7a165c6d2a840297ad45a477344fe9064147b1f19e5ae6c1d

Tokenizer SHA256:

f9e19f1b7169fcd243a433a3448181bffb6a6e23d0cdf34909100d5225e74cc5

License

Verify the licenses of the training data, tokenizer, source code, and dependencies before redistribution.