Qwen3.5-9B-nz-wvs-full_string_distribution-overall

Qwen3.5-9B LoRA — Full String Distribution, Overall

This model is a LoRA fine-tune of Qwen/Qwen3.5-9B as part of the AIML589 project.

This adapter is licensed under CC BY-SA 4.0.

Dataset

Fine-tuned on the full_string_distribution config of the wvs-nz-value-alignment dataset, overall subpopulation.

Part of the wvs-nz-value-alignment collection.

GPU: NVIDIA A40 · Training time: 57m 15s

Training hyperparameters

Training log

{"loss": 8.703079986572266, "grad_norm": 55.47663497924805, "learning_rate": 0.00011250000000000001, "epoch": 0.06504065040650407, "step": 10}
{"eval_loss": 1.79399573802948, "eval_runtime": 93.2017, "eval_samples_per_second": 2.961, "eval_steps_per_second": 1.481, "epoch": 0.06504065040650407, "step": 10}
{"loss": 6.914659881591797, "grad_norm": 18.564268112182617, "learning_rate": 0.0001956521739130435, "epoch": 0.13008130081300814, "step": 20}
{"eval_loss": 1.4378732442855835, "eval_runtime": 92.8107, "eval_samples_per_second": 2.974, "eval_steps_per_second": 1.487, "epoch": 0.13008130081300814, "step": 20}
{"loss": 5.270082092285156, "grad_norm": 6.949378490447998, "learning_rate": 0.00018115942028985507, "epoch": 0.1951219512195122, "step": 30}
{"eval_loss": 1.3564187288284302, "eval_runtime": 92.968, "eval_samples_per_second": 2.969, "eval_steps_per_second": 1.484, "epoch": 0.1951219512195122, "step": 30}
{"loss": 5.4691619873046875, "grad_norm": 18.938419342041016, "learning_rate": 0.0001666666666666667, "epoch": 0.2601626016260163, "step": 40}
{"eval_loss": 1.2968977689743042, "eval_runtime": 92.9333, "eval_samples_per_second": 2.97, "eval_steps_per_second": 1.485, "epoch": 0.2601626016260163, "step": 40}
{"loss": 4.866386032104492, "grad_norm": 4.205469131469727, "learning_rate": 0.00015217391304347827, "epoch": 0.3252032520325203, "step": 50}
{"eval_loss": 1.3045886754989624, "eval_runtime": 92.8006, "eval_samples_per_second": 2.974, "eval_steps_per_second": 1.487, "epoch": 0.3252032520325203, "step": 50}
{"loss": 5.203897476196289, "grad_norm": 6.2034478187561035, "learning_rate": 0.00013768115942028986, "epoch": 0.3902439024390244, "step": 60}
{"eval_loss": 1.2741440534591675, "eval_runtime": 92.7552, "eval_samples_per_second": 2.976, "eval_steps_per_second": 1.488, "epoch": 0.3902439024390244, "step": 60}
{"loss": 5.226799392700196, "grad_norm": 5.329586029052734, "learning_rate": 0.00012318840579710145, "epoch": 0.45528455284552843, "step": 70}
{"eval_loss": 1.2608736753463745, "eval_runtime": 92.7677, "eval_samples_per_second": 2.975, "eval_steps_per_second": 1.488, "epoch": 0.45528455284552843, "step": 70}
{"loss": 4.7585094451904295, "grad_norm": 6.022779941558838, "learning_rate": 0.00010869565217391305, "epoch": 0.5203252032520326, "step": 80}
{"eval_loss": 1.2409065961837769, "eval_runtime": 92.9603, "eval_samples_per_second": 2.969, "eval_steps_per_second": 1.485, "epoch": 0.5203252032520326, "step": 80}
{"loss": 5.141559982299805, "grad_norm": 6.975116729736328, "learning_rate": 9.420289855072463e-05, "epoch": 0.5853658536585366, "step": 90}
{"eval_loss": 1.2473307847976685, "eval_runtime": 93.0257, "eval_samples_per_second": 2.967, "eval_steps_per_second": 1.483, "epoch": 0.5853658536585366, "step": 90}
{"loss": 4.915719604492187, "grad_norm": 5.658397197723389, "learning_rate": 7.971014492753623e-05, "epoch": 0.6504065040650406, "step": 100}
{"eval_loss": 1.2207982540130615, "eval_runtime": 92.9926, "eval_samples_per_second": 2.968, "eval_steps_per_second": 1.484, "epoch": 0.6504065040650406, "step": 100}
{"loss": 4.71471939086914, "grad_norm": 4.068330764770508, "learning_rate": 6.521739130434783e-05, "epoch": 0.7154471544715447, "step": 110}
{"eval_loss": 1.2062972784042358, "eval_runtime": 93.0862, "eval_samples_per_second": 2.965, "eval_steps_per_second": 1.482, "epoch": 0.7154471544715447, "step": 110}
{"loss": 5.122110748291016, "grad_norm": 4.539018154144287, "learning_rate": 5.072463768115943e-05, "epoch": 0.7804878048780488, "step": 120}
{"eval_loss": 1.1862434148788452, "eval_runtime": 92.8886, "eval_samples_per_second": 2.971, "eval_steps_per_second": 1.486, "epoch": 0.7804878048780488, "step": 120}
{"loss": 5.124646377563477, "grad_norm": 2.0783629417419434, "learning_rate": 3.6231884057971014e-05, "epoch": 0.8455284552845529, "step": 130}
{"eval_loss": 1.1823657751083374, "eval_runtime": 93.3157, "eval_samples_per_second": 2.958, "eval_steps_per_second": 1.479, "epoch": 0.8455284552845529, "step": 130}
{"loss": 4.650765609741211, "grad_norm": 2.456535816192627, "learning_rate": 2.173913043478261e-05, "epoch": 0.9105691056910569, "step": 140}
{"eval_loss": 1.1808730363845825, "eval_runtime": 93.0255, "eval_samples_per_second": 2.967, "eval_steps_per_second": 1.483, "epoch": 0.9105691056910569, "step": 140}
{"loss": 4.547821807861328, "grad_norm": 3.2700843811035156, "learning_rate": 7.246376811594203e-06, "epoch": 0.975609756097561, "step": 150}
{"eval_loss": 1.1744519472122192, "eval_runtime": 92.9911, "eval_samples_per_second": 2.968, "eval_steps_per_second": 1.484, "epoch": 0.975609756097561, "step": 150}
{"eval_loss": 1.1732834577560425, "eval_runtime": 93.2831, "eval_samples_per_second": 2.959, "eval_steps_per_second": 1.479, "epoch": 1.0, "step": 154}
{"train_runtime": 3434.9466, "train_samples_per_second": 0.358, "train_steps_per_second": 0.045, "total_flos": 5.38753752607488e+16, "train_loss": 5.347667557852609, "epoch": 1.0, "step": 154}

Environment

Intended use

This adapter is intended for research purposes only as part of the AIML589 project, which investigates value alignment of LLMs with New Zealand population distributions from the World Values Survey.

Out-of-scope

This model has not been safety-tuned for general-purpose deployment. It should not be used in production systems, for making decisions about people, or in contexts where reliability and safety are critical.

Limitations and biases

• Fine-tuned on a single WVS wave (Wave 7) for New Zealand only.
• The training data reflects the values of those who responded to the survey and may not represent all New Zealanders.
• LoRA adapters are subject to the limitations and biases of the base model (Qwen/Qwen3.5-9B).