Bakpia-V1-0.5B-Javanese

<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Document Title</title> <style> h1 { font-size: 36px; color: navy; font-family: 'Tahoma'; text-align: center; } </style> </head> <body> <h1> Open models for indigenous Indonesian languages</h1> </body> </html> <center> <img src="https://imgur.com/PutckEK.png" alt="Bakpia" width="500" height="250"> <p><em>Bakpia is a family of open language models capable of responding in Javanese language. Version one of Bakpia is the first generative Javanese LLM gain functional instruction performance using solely synthetic data.</em></p> <p><em style="color: black; font-weight: bold;">Beta preview</em></p> </center> Bakpia V1 is a family of Javanese language models. It is fine-tuned from available open models using massive synthetic data for Krama Javanese, where the prompts are generated by GPT-4o and the responses are generated by Claude 3 Haiku.

This repository contains the fp16 version of Bakpia V1 0.5B.

Training data is accessible here.

Version 1.0

This is the first version of Bakpia.

✨ Training

• 36K input-output pairs
• 64/128 lora r/alpha
• Rank-stabilized lora

✨ Features

• Single-turn QA across various domains.
• Ngoko Javanese not currently supported.

Generate with template

from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer

tokenizer = AutoTokenizer.from_pretrained("afrizalha/Bakpia-V1-0.5B-Javanese")
model = AutoModelForCausalLM.from_pretrained("afrizalha/Bakpia-V1-0.5B-Javanese")
model.to("cuda")

template = """<|im_start|>system
<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
"""

input = template.format(prompt="Kados pundi kulo saged nyinaoni Basa Jawa kanthi sae?"
input = tokenizer([input], return_tensors = "pt").to("cuda")
outputs = model.generate(**input, max_new_tokens = 1024, streamer= TextStreamer(tokenizer), temperature=.5, use_cache=True, do_sample=True)

Acknowledgments

• Developed by: Afrizal Hasbi Azizy
• License: Apache-2.0