maincomp-merged-phase1

maincomp-merged-phase1

This repository provides a fine-tuned model developed for the LLM Engineering Main Competition – Phase 1.

The model is designed to improve structured output generation, especially JSON outputs used in automated evaluation pipelines.

Model Overview

Model name

maincomp-merged-phase1

Base model

unsloth/Qwen3-4B-Instruct-2507

Training approach

Supervised Fine-Tuning (SFT)

Purpose

Improve the reliability of structured outputs required in evaluation tasks.

Target formats include:

JSON

YAML

XML

CSV

TOML

The model is optimized for generating machine-readable structured outputs.

Training Data

Training utilized datasets provided for the competition.

Examples include:

structured_data_with_cot_dataset_512_v2

structured_data_with_cot_dataset_512_v4

structured_data_with_cot_dataset_512_v5

structured_data_with_cot_dataset

structured-3k-mix-sft

structured-5k-mix-sft

structured-hard-sft-4k

These datasets are designed for structured output generation tasks with reasoning components (CoT).

Important note:

Using these datasets does not guarantee score improvements, and participants are encouraged to combine or customize them.

メインコンペ_使用データまとめ.txt

Training Configuration

Typical training configuration used in this phase.

Base model unsloth/Qwen3-4B-Instruct-2507

Training method QLoRA (4-bit)

Max sequence length 512

Epochs 1

Learning rate 1e-4

LoRA parameters

r = 64 alpha = 128

After training, the LoRA adapter was merged into the base model to create a standalone model.

Usage

Example code for loading the model:

from transformers import AutoModelForCausalLM, AutoTokenizer import torch

model_id = "i-KenTanaka/maincomp-merged-phase1"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.float16, device_map="auto" )

prompt = "Generate structured JSON output."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)

print(tokenizer.decode(outputs[0], skip_special_tokens=True)) Competition Submission

This repository is published to satisfy the model publication requirement of the main competition.

Submission components include:

JSON outputs generated using the official inference code

Public Hugging Face model repository (this repository)

Limitations

Optimized primarily for structured output tasks.

Performance may be limited for open-domain dialogue.

Correct output format may depend on prompt design.

License and Compliance

Users must comply with:

The base model license

The licenses of the training datasets

Dataset attribution requirements must be respected.

Author

Kenji Tanaka https://huggingface.co/i-KenTanaka

Acknowledgements

This model was developed using:

Hugging Face Transformers

Unsloth

QLoRA fine-tuning

Competition datasets provided by the organizers