hsc-wuxia-glm-4-32b

HSC Wuxia GLM-4-32b Adapter

This model is a fine-tuned LoRA adapter based on GLM-4-32B-0414 for Chinese-to-English translation of Wuxia and Xianxia literature. It was developed as part of the Bachelor's Thesis (TFG):

"Enfoques de traducción automática con modelos de lenguaje en obras wuxia" (Degree in Data Science and Engineering, Universidade da Coruña)

Model Details

• Base Model: zai-org/GLM-4-32B-0414
• Training Method: Parameter-efficient fine-tuning via QLoRA
• Training Dataset: HSilvosa/hsc-wuxia-100k (100,000 stratified parallel sentence pairs)
• Target Domain: Wuxia/Xianxia web novels

Intended Use

This adapter is intended to be used on top of GLM-4-32B-0414 to translate domain-specific Chinese literary texts into English.

Quick Start (Inference)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_name = "zai-org/GLM-4-32B-0414"
adapter_model_name = "HSilvosa/hsc-wuxia-glm-4-32b"

# Load base model & tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

# Load LoRA adapter
model = PeftModel.from_pretrained(model, adapter_model_name)

# Translate example
text_zh = "一念成沧海，一念化桑田。"
prompt = f"Translate the following Chinese text to English: {text_zh}\nTranslation:"

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Evaluation Results

The model was evaluated on the Wuxia parallel test set (translating Chinese to English) comparing the base GLM-4-32B-0414 model and this fine-tuned version under Prompt 0, 0-Shot configuration:

Training Hyperparameters

The model was fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) via QLoRA with the following configurations:

• Optimization/Quantization: 4-bit (QLoRA)
• LoRA Rank ($r$): 16
• LoRA Alpha ($\alpha$): 32
• LoRA Dropout: 0.05
• Target Modules: All linear layers (query, key, value, projections)
• Learning Rate: 2e-4 (0.0002)
• Epochs: 1
• Per-Device Batch Size: 4
• Gradient Accumulation Steps: 16
• Global Batch Size: 64 (per device/GPU)
• Base Model Size: 32 Billion parameters

Project Repository

This model and the associated training/evaluation pipelines were developed in the following project repository:

• GitHub Repository: HugoSilvosa/hsc-mtl-wuxia

Acknowledgments

This research project was made possible through the access granted by the Galician Supercomputing Center (CESGA) to its supercomputing infrastructure. The supercomputer FinisTerrae III and its permanent data storage system have been funded by the NextGeneration EU 2021 Recovery, Transformation and Resilience Plan, ICT2021-006904, and also from the Pluriregional Operational Programme of Spain 2014-2020 of the European Regional Development Fund (ERDF), ICTS-2019-02-CESGA-3, and from the State Programme for the Promotion of Scientific and Technical Research of Excellence of the State Plan for Scientific and Technical Research and Innovation 2013-2016 State subprogramme for scientific and technical infrastructures and equipment of ERDF, CESG15-DE-3114.