ko_law_alpaca-12.8b

ko_law_alpaca-12.8b (LAW-Alpaca)

A LoRA adapter that teaches KoAlpaca-Polyglot-12.8B to answer everyday Korean legal questions. It is the model released by the LAW-Alpaca project, together with the easylaw_kr dataset, on July 4, 2023.

• Author: Jiwoo Chris Jung (@jiwoochris); LAW-Alpaca was a team project in which I built the dataset, ran the fine-tuning, and maintain the repository
• Base model: beomi/KoAlpaca-Polyglot-12.8B (GPT-NeoX architecture, itself an instruction-tuned EleutherAI/polyglot-ko-12.8b)
• Training data: jiwoochris/easylaw_kr, 2,195 question-answer pairs
• Method: QLoRA (4-bit NF4 base, LoRA rank 8)

Training data

The dataset reformats 2,195 question-and-answer posts from the "100 questions, 100 answers" section of the Korean government's everyday-law site (생활법령정보, run by the Ministry of Government Legislation, 법제처) into Alpaca-format records with instruction, input, and output fields. The site's copyright policy permits free use, including commercial use.

The dataset was later used as legal-domain training and evaluation data in Chamain (Yang, Lee, and Cho), published at the NLP4ConvAI workshop at ACL 2024 (aclanthology.org/2024.nlp4convai-1.7).

Training procedure

LoRA configuration (from adapter_config.json):

• r: 8, lora_alpha: 32, lora_dropout: 0.05
• target_modules: query_key_value
• task_type: CAUSAL_LM

The base model was loaded in 4-bit during training (bitsandbytes):

• load_in_4bit: True, bnb_4bit_quant_type: nf4, bnb_4bit_use_double_quant: True, bnb_4bit_compute_dtype: bfloat16

Framework: PEFT 0.4.0.dev0.

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_id = "beomi/KoAlpaca-Polyglot-12.8B"
adapter_id = "jiwoochris/ko_law_alpaca-12.8b"

tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, adapter_id)

question = "전세보증금을 돌려받지 못했습니다. 어떻게 해야 하나요?"
prompt = f"### 질문: {question}\n\n### 답변:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

The prompt above uses the base model's KoAlpaca format. The training data is in Alpaca format (instruction / input / output); the exact template used during training is not preserved in this repository.

Limitations

This model memorizes guidance rather than retrieving it: it cannot show the source of an answer, it answers fluently even when the question falls outside the guidance it was trained on, and it has to be retrained when the law changes. Those limits led to the follow-up project In-Memory-Vector-DB, which keeps the guidance as text and retrieves it at answer time. The model is a 2023 research artifact and is not legal advice.

Related

• Code: github.com/jiwoochris/LAW-Alpaca
• Dataset: jiwoochris/easylaw_kr
• Earlier 7B adapter (May 2023, LLaMA-7B base): jiwoochris/Law-alpaca-lora-7b