gemma3-4b-cpt

gemma3-4b-cpt

CPT-only (Continued Pre-Training) adapter for Gemma-3-4B-IT on the MADLAD-400 Sinhala corpus. This adapter does not perform VQA on its own. It is intended to be used as the first stage of the sequential CPT → VQA pipeline together with Siluni/gemma3-4b-cpt-vqa-33k.

• Base model: google/gemma-3-4b-it
• Experiment: Group 3 — Sequential CPT stage
• CPT corpus: MADLAD-400 Sinhala (~293M words)
• Method: QLoRA (4-bit NF4, LoRA rank 16, alpha 32)

⚠️ Sequential Loading Required

This adapter must be loaded together with the VQA adapter and combined before inference. See Siluni/gemma3-4b-cpt-vqa-33k for the full loading instructions.

Citation

@misc{keerthiratne2025sinhalavqa,
  title        = {Benchmarking and Adapting Compact Multimodal Models for Sinhala Visual Question Answering},
  author       = {Keerthiratne, Siluni and Weerasinghe, Ruvan and Sumanathilaka, Deshan},
  year         = {2025},
  institution  = {Informatics Institute of Technology / Robert Gordon University},
}