KobitaLM

KobitaLM - Bengali Poetry Language Model

KobitaLM (কবিতা LM, meaning "Poetry LM" in Bengali) is a fine-tuned language model specialized in generating Bengali poetry. Built on top of Google's Gemma 2 9B model using LoRA adapters, it captures the distinctive styles of classical and modern Bengali poets.

Model Details

Model Description

KobitaLM is a LoRA adapter fine-tuned on a comprehensive corpus of Bengali poetry from renowned poets including Rabindranath Tagore, Jibanananda Das, Kazi Nazrul Islam, and many others. The model can generate poetry in various styles and follows instructions in Bengali.

• Developed by: Community Project
• Model type: Causal Language Model with LoRA Adapters
• Language: Bengali (বাংলা)
• License: Gemma License
• Base Model: unsloth/gemma-2-9b-bnb-4bit
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Primary Use: Bengali poetry generation

Model Sources

• Base Model: Gemma 2 9B
• Training Framework: Unsloth
• Dataset: Bengali Poems Collection (Classical and Modern)

Uses

Direct Use

KobitaLM is designed for:

• Generating Bengali poetry in various classical and modern styles
• Educational purposes to learn about Bengali poetic traditions
• Creative writing assistance for Bengali content creators
• Research in Bengali NLP and computational creativity

Example Usage

from unsloth import FastLanguageModel

# Load the model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="your-username/KobitaLM",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

# Prepare for inference
FastLanguageModel.for_inference(model)

# Generate poetry
prompt = "জীবনানন্দ দাশের স্টাইল-এ একটি কবিতা লিখুন।"
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.8,
    top_p=0.9,
    repetition_penalty=1.2,
)

poem = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(poem)

Out-of-Scope Use

This model is NOT suitable for:

• General Bengali text generation outside of poetry
• Translation tasks
• Factual question answering
• Any use requiring factual accuracy or real-time information

Bias, Risks, and Limitations

• The model may reflect biases present in classical Bengali poetry
• Generated content may occasionally contain repetitive patterns
• Performance is optimized for poetry generation, not general text
• Requires Bengali language understanding to evaluate output quality
• May not accurately represent all regional Bengali dialects

Recommendations

• Review generated poetry for appropriateness before publication
• Use as a creative assistant rather than autonomous content generator
• Consider cultural context when using generated poetry
• Respect copyright and attribution norms for the training data poets

Training Details

Training Data

The model was fine-tuned on a curated corpus of Bengali poetry including works from:

• রবীন্দ্রনাথ ঠাকুর (Rabindranath Tagore)
• জীবনানন্দ দাশ (Jibanananda Das)
• কাজী নজরুল ইসলাম (Kazi Nazrul Islam)
• সুকান্ত ভট্টাচার্য (Sukanta Bhattacharya)
• কামিনী রায় (Kamini Roy)
• মাইকেল মধুসূদন দত্ত (Michael Madhusudan Dutt)
• লালন ফকির (Lalon Fakir)
• রামপ্রসাদ সেন (Ramprasad Sen)
• And many other classical and modern Bengali poets

Additionally, Bengali instruction-following data was included for better controllability.

Training Procedure

• Fine-tuning Framework: Unsloth + PEFT
• Training Method: Supervised Fine-Tuning (SFT) with LoRA
• LoRA Rank: 8
• LoRA Alpha: 8
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Quantization: 4-bit (bitsandbytes)
• Max Sequence Length: 2048 tokens
• Base Model: unsloth/gemma-2-9b-bnb-4bit

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Preprocessing [optional]

[More Information Needed]

Training Hyperparameters

• Training regime: [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

Results

[More Information Needed]

Summary

Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [More Information Needed]
• Hours used: [More Information Needed]
• Cloud Provider: [More Information Needed]
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

[More Information Needed]

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

[More Information Needed]

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Framework versions

• PEFT 0.18.0