GigaLekh-ablation-NonEDU-1.5B

GigaLekh-ablation-NonEDU-1.5B

Model Summary

GigaLekh-ablation-NonEDU-1.5B is a decoder-transformer natively pretrained in Hindi. This model is part of an ablation study to measure the impact of our educational data filtering/augmentation strategy on the downstream performance of models trained with GigaLekh. GigaLekh-ablation-NonEDU-1.5B was trained with ~60 billion tokens, those being a mixture of the non-educational portion of GigaLekh (i.e., samples with an Edu Score < 3). This model has 1.5 billion parameters and a context length of 4096 tokens.

Details

• Architecture: a Transformer-based model (llama)
• Size: 1,510,066,176 parameters
• Context length: 4096 tokens
• Dataset(s):
  • GigaLekh (non-educational subset, Edu Score < 3)
• Language(s): Hindi
• Batch size: 2,097,152 tokens
• Number of steps: 28,000
• GPU: 16 NVIDIA A40 (48 GB)
• Training time: ~104.81 hours
• Emissions: 164.58 KgCO2 (Germany)
• Total energy consumption: 432.03 kWh

This repository has the source code used to train this model. The complete configuration used for training is available in the following config file:

• Single stage (linear warmup with cosine decay): training_config.yaml

The main branch of this repository contains the final checkpoint saved at step 28,000. All other checkpoints are available as separate branches. To load a specific checkpoint, you can use the following code snippet:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Polygl0t/GigaLekh-ablation-NonEDU-1.5B"
revision = "step-2000"  # Change this to the desired checkpoint branch
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, revision=revision)

Or, you can access all the revisions for the models via the following code snippet:

from huggingface_hub import list_repo_refs
out = list_repo_refs("Polygl0t/GigaLekh-ablation-NonEDU-1.5B")
branches = [b.name for b in out.branches]
print(branches)

Intended Uses

The primary intended use of this model is to serve as a baseline for evaluating the impact of data quality and filtering on Hindi language model performance. Researchers and practitioners can use this model as a reference point for further ablation studies or for comparison with other models trained on different data mixtures.

Basic usage

from transformers import GenerationConfig, TextGenerationPipeline, AutoTokenizer, AutoModelForCausalLM
import torch

# Specify the model and tokenizer
model_id = "Polygl0t/GigaLekh-ablation-NonEDU-1.5B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

# Specify the generation parameters as you like
generation_config = GenerationConfig(
    **{
    "do_sample": True,
    "max_new_tokens": 150,
    "renormalize_logits": True,
    "repetition_penalty": 1.2,
    "temperature": 0.1,
    "top_k": 50,
    "top_p": 1.0,
    "use_cache": True,
  }
)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
generator = TextGenerationPipeline(model=model, task="text-generation", tokenizer=tokenizer, device=device)

# Generate text
prompt = "भारत की राजधानी क्या है?"
completion = generator(prompt, generation_config=generation_config)
print(completion[0]['generated_text'])

Evaluations

Figures below show the per-benchmark performance of GigaLekh-ablation-EDU-1.5B (educational subset, Edu Score >= 3) compared to GigaLekh-ablation-NonEDU-1.5B (non educational subset, Edu Score < 3). GigaLekh-Edu outperforms GigaLekh-NonEdu on 5 of 6 benchmarks and achieves a higher NPM score. The largest performance gap is observed in NPM (+29.1%; 12.43 vs. 9.63) and ARC Challenge (+19.9%; 0.301 vs. 0.251). Moderate advantages for the educational model are observed on MILU (+8.4%; 0.296 vs. 0.273), HellaSwag (+6.9%; 0.374 vs. 0.350), and CSQA (+4.5%; 0.372 vs. 0.356), while Global PIQA shows only a marginal difference (+1.6%; 0.640 vs. 0.630). The sole exception is MMLU, where GigaLekh-NonEdu marginally outperforms GigaLekh-Edu (0.258 vs. 0.256, <1%). These results suggest that training on educationally curated content consistently yields stronger language understanding.

<details> <summary><b>🏆 HellaSwag</b></summary>

</details> <details> <summary><b>🏆 ARC Challenge</b></summary>

</details> <details> <summary><b>🏆 MMLU</b></summary>

</details> <details> <summary><b>🏆 MILU</b></summary>

</details> <details> <summary><b>🏆 CSQA</b></summary>

</details> <details> <summary><b>🏆 Global PIQA</b></summary>

</details> <details> <summary><b>Aggregate NPM Across Benchmarks</b></summary>

</details>

Cite as 🤗

@article{fatimah2026raising,
  title={Raising Bars, Not Parameters: LilMoo Compact Language Model for Hindi},
  author={Fatimah, Shiza and Sen, Aniket and Falk, Sophia and Mai, Florian and Flek, Lucie and Corr{\^e}a, Nicholas Kluge},
  journal={arXiv preprint arXiv:2603.03508},
  year={2026}
}

Aknowlegments

Polyglot is a project funded by the Federal Ministry of Education and Research (BMBF) and the Ministry of Culture and Science of the State of North Rhine-Westphalia (MWK) as part of TRA Sustainable Futures (University of Bonn) and the Excellence Strategy of the federal and state governments.

We also gratefully acknowledge the granted access to the Marvin cluster hosted by University of Bonn along with the support provided by its High Performance Computing & Analytics Lab.

License

This model is licensed under the Apache License, Version 2.0. For more details, see the LICENSE file.