AcehXBERT-base

AcehXBERT Base : A BERT-Based Model for Acehnese NLP

AcehXBERT is a transformer-based language model developed specifically for the Acehnese language, built using the BERT architecture. This model was created to support NLP research and applications for low-resource Indonesian local languages, particularly Acehnese.

Model Description

• Base model : IndoBERT-base-p1

• Pretraining objective : Masked Language Modeling (MLM)

• Task : Acehnese language modeling and downstream NLP

• Language : Acehnese (ace)

• Params : 113.2M

The model is generated via continued pretraining (domain adaptive pretraining / DAPT) on AcehX corpus.

How to Use

from transformers import AutoTokenizer, AutoModelForMaskedLM, pipeline

model_name = "doniNlp/AcehXBERT-base"  

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name)

AcehXBERT_mlm = pipeline(
    "fill-mask",
    model=model,
    tokenizer=tokenizer,
    top_k=1
)

AcehXBERT_mlm("pat meurumpok [MASK] nyan")

Downstream Fine-Tuning for Acehnese

This model is suitable for fine-tuning in tasks such as:

• Sentiment Analysis

• Text Classification

• Named Entity Recognition (NER)

• Extractive Question Answering (SQuAD)

• Semantic Similarity

Dataset Overview

AcehX Corpus

A large unlabeled Acehnese corpus collected from:

• literature, folklore, oral traditions
• traditional customs and customary law texts
• Wikipedia
• YouTube comments

Authors

• Doni Sumito Sukiswo (Universitas Syiah Kuala (USK))
• Hammam Riza* (National Research and Innovation Agency (BRIN))
• Muhammad Subianto (Universitas Syiah Kuala (USK))
• Taufik Fuadi Abidin (Universitas Syiah Kuala (USK))
• Afnan (Universitas Syiah Kuala (USK))

* Corresponding Author

Credits

AcehXBERT Base is developed by :

<div style="display: flex;"> <a href="https://github.com/donisumito"> <img src="https://github.com/donisumito.png" alt="GitHub Profile" style="border-radius: 50%;width: 70px;height: 70px;margin:0 2px;"> </a> <a href="https://scholar.google.com/citations?user=j6Q7xnAAAAAJ&hl=id"> <img src="https://scholar.googleusercontent.com/citations?view_op=medium_photo&user=j6Q7xnAAAAAJ&citpid=1" alt="Google Scholar" style="border-radius: 50%;width: 70px;height: 70px;margin:0 2px;"> </a> <a href="https://scholar.google.com/citations?user=eVPeQT8AAAAJ&hl=en"> <img src="https://scholar.googleusercontent.com/citations?view_op=medium_photo&user=eVPeQT8AAAAJ&citpid=25" alt="Google Scholar" style="border-radius: 50%;width: 70px;height: 70px;margin:0 2px;"> </a> <a href="https://scholar.google.com/citations?user=dQ9hP8kAAAAJ&hl=en"> <img src="https://scholar.googleusercontent.com/citations?view_op=medium_photo&user=dQ9hP8kAAAAJ&citpid=3" alt="Google Scholar" style="border-radius: 50%;width: 70px;height: 70px;margin:0 2px;"> </a> <a href="https://scholar.google.com/citations?user=wVSSurkAAAAJ&hl=en"> <img src="https://scholar.googleusercontent.com/citations?view_op=medium_photo&user=wVSSurkAAAAJ&citpid=1" alt="Google Scholar" style="border-radius: 50%;width: 70px;height: 70px;margin:0 2px;"> </a> </div> <p />

Citation

You can find the full details in our paper:
Development of AcehX for Sentiment Analysis Using a BERT-Based Model

Proceedings of the 28th O-COCOSDA Conference, IEEE, 2025

https://doi.org/10.1109/O-COCOSDA68185.2025.11384824

If you use AcehX or AcehXBERT, please cite:

@INPROCEEDINGS{11384824,
  author={Sukiswo, Doni Sumito and Riza, Hammam and Subianto, Muhammad and Abidin, Taufik Fuadi and Afnan},
  booktitle={2025 28th Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA)}, 
  title={Development of Acehx for Sentiment Analysis Using a Bert-Based Model}, 
  year={2025},
  volume={},
  number={},
  pages={1-6},
  keywords={Training;Adaptation models;Sentiment analysis;Analytical models;Vocabulary;Technological innovation;Linguistics;Transformers;Cultural differences;Artificial intelligence;AcehX;sentiment analysis;NLP;transformers;BERT;deep learning},
  doi={10.1109/O-COCOSDA68185.2025.11384824}}