NepBERTa-pytorch

NepBERTa — PyTorch conversion

A faithful PyTorch conversion of the official NepBERTa/NepBERTa checkpoint, which ships only TensorFlow weights (tf_model.h5) that transformers v5 can no longer load.

Provenance

• Source: NepBERTa/NepBERTa @ tf_model.h5 (TFBertForMaskedLM, 207 tensors).
• Converted 2026-08-04 with transformers 4.57.6 / tensorflow-cpu 2.21.0 via load_tf2_checkpoint_in_pytorch_model into a BertModel.
• Tokenizer files copied unmodified from the source repo (vocab.txt md5 edfd394677436b306fb062159ec46c72).
• This repo contains only the 197 backbone tensors present in the official checkpoint — the source has no trained pooler (it is a masked-LM checkpoint), so no pooler weights are shipped; downstream loading initializes the pooler freshly, exactly as loading the official checkpoint would.
• Cross-check: every converted tensor is bit-identical (torch.equal) to the independent community port Rajan/nepbertaTorch on all 198 tensors that repo shares with the official checkpoint.

Use

from transformers import AutoModelForSequenceClassification, AutoTokenizer
tok = AutoTokenizer.from_pretrained("subrace/NepBERTa-pytorch")
model = AutoModelForSequenceClassification.from_pretrained(
    "subrace/NepBERTa-pytorch", num_labels=2)

All credit for the model itself goes to the NepBERTa authors (paper); this repo exists only so the weights load in modern PyTorch-only transformers.