BanglaSenti-XLM-RoBERTa-Experiment-Models

BanglaSenti XLM-RoBERTa Experiment Models

This directory contains all exported experiment models (LoRA and baseline) for BanglaSenti sentiment analysis, ready for sharing or uploading to the Hugging Face Hub. All files are strictly local and fully reproducible.

Model Subdirectories

• ex-baseline-4-8-qv/ — Baseline LoRA (rank=4, alpha=8, query/value)
• ex-16-32-qv/ — LoRA (rank=16, alpha=32, query/value)
• ex-32-64-qv/ — LoRA (rank=32, alpha=64, query/value)
• ex-32-64-tm-qkv/ — LoRA (rank=32, alpha=64, query/key/value)
• ex-32-64-tm-all/ — LoRA (rank=32, alpha=64, query/key/value/dense) — Main SOTA
• ex-xlm-roberta-base/ — Baseline full fine-tuned XLM-RoBERTa (no LoRA)

Each experiment folder contains:

• checkpoints/ — All model weights, adapter weights, tokenizer files, and info
  • banglasenti-lora-xlmr/ or banglasenti-xlmr/ (baseline)
    • lora_adapter_state_dict.pt (LoRA only)
    • lora_xlmr_weights.pt, final_lora_xlmr_weights.pt, or final_xlmr_weights.pt
    • model_info.txt (metadata for each checkpoint)
    • lora_adapter_weights/ or final_lora_adapter_weights/ (contains adapter_config.json, config.json)
    • lora_xlmr_tokenizer/, final_tokenizer/, or xlmr_tokenizer/ (contains tokenizer.json, tokenizer_config.json, special_tokens_map.json, sentencepiece.bpe.model)
    • final_state/ — Contains the final checkpoint after all training epochs, with the same structure as above
• configs/ — YAML config files for training and evaluation (train.yaml, eval.yaml, eval-xlm.yaml)
• logs/ — All logs for training and evaluation runs (train_banglasenti.log, train_banglasenti_main.log, eval_run.log, eval_run_xlm.log)

Checkpoint Types

• During-training: Intermediate checkpoints such as lora_adapter_state_dict.pt, lora_xlmr_weights.pt, and model_info.txt
• Final: Last checkpoint after training, located in the final_state/ subfolder, such as final_lora_adapter_state_dict.pt, final_lora_xlmr_weights.pt, final_xlmr_weights.pt, and model_info.txt

File Types

• Model weights: Files with the .pt extension (lora_adapter_state_dict.pt, lora_xlmr_weights.pt, final_lora_adapter_state_dict.pt, final_lora_xlmr_weights.pt, final_xlmr_weights.pt)
• Configs: config.json, adapter_config.json (with peft_type for LoRA)
• Tokenizer files: tokenizer.json, tokenizer_config.json, special_tokens_map.json, sentencepiece.bpe.model (the sentencepiece.bpe.model file is optional for LoRA adapters; if you face issues, see the main project documentation)
• Info: model_info.txt
• Training and evaluation configs: train.yaml, eval.yaml, eval-xlm.yaml
• Logs: train_banglasenti.log, eval_run.log, train_banglasenti_main.log, eval_run_xlm.log

Usage

• Load models and tokenizers using Hugging Face Transformers or PEFT, strictly from local files.
• For LoRA: Use PEFT/LoRA config and weights; the peft_type field must be present in the config.
• For baseline: Use standard Hugging Face model loading from the provided checkpoint and config.
• No external Hugging Face Hub calls are required for any operation.

Results Reference

• Project repository: LORA-FINETUNING-BANGLASENTI-XLMR-GOOGLE-TPU
• Full experiment report: experiment_report.md

License

Apache 2.0

---

Acknowledgement

• This research was supported by the Google Research TPU Research Cloud (TRC) program. Special thanks to the TRC team at Google Research for providing free access to Google Cloud TPUs, which made this work possible.
• The BanglaSenti dataset is from the open-source banglasenti-dataset-prep project.
• The base model xlm-roberta-base is provided by Facebook AI.
• This project builds on the Hugging Face Transformers and PEFT libraries.
• Thanks to the open-source community and all contributors to the code, data, and research.

Citation

If you use these models or code, please cite as:

@misc{lora-banglasenti-xlmr-tpu,
  title={LoRA Fine-Tuning of BanglaSenti on XLM-RoBERTa-Base Using Google TPUs},
  author={Niloy Deb Barma},
  year={2025},
  howpublished={\url{https://github.com/niloydebbarma-code/LORA-FINETUNING-BANGLASENTI-XLMR-GOOGLE-TPU}},
  note={Open-source Bengali sentiment analysis with LoRA and XLM-RoBERTa on TPU}
}

For full dataset citations and license information, see:

• docs/dataset_citations.md (main project)
• CITATIONS.md from banglasenti-dataset-prep