Dongeng-5M

Dongeng is a model family trained on a dataset of the same name, which consists of ~2M Indonesian children's stories in the style of the SimpleStories dataset. Dongeng is the Indonesian word for story or folktale.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "farrelmahaztra/Dongeng-5M"
model = AutoModelForCausalLM.from_pretrained(repo).eval()
tok = AutoTokenizer.from_pretrained(repo)

prompt = "Suatu hari, seekor kucing kecil"
# add_special_tokens=False so the tokenizer doesn't append [EOS] and end the story early.
inputs = tok(prompt, return_tensors="pt", add_special_tokens=False)
out = model.generate(
    inputs.input_ids, max_new_tokens=400, do_sample=True,
    temperature=0.7, eos_token_id=tok.eos_token_id,
)

# Decode and undo the morphological segmentation ("me + baca" -> "membaca").
print(tok.decode(out[0], skip_special_tokens=True).replace(" + ", ""))

For an unconditional story, seed generation with [EOS] (the start-of-story token) instead of a prompt: model.generate(torch.tensor([[tok.eos_token_id]]), ...).

Note on prompts. The training corpus is morphologically segmented (e.g. membaca -> me + baca), so an affixed word like seekor (se + ekor in training) tokenizes slightly off-distribution when passed raw. You may want to run the prompt through the dongeng package's segment() first, although the generation should remain coherent either way and unconditional generation is unaffected.

Model Variants