DAFT-Qwen3-0.6B-checkpoint-4000

DAFT Qwen3-0.6B — checkpoint 4000

An intermediate full fine-tuning checkpoint of Qwen3-0.6B for translating NVIDIA GPU assembly functions into corresponding AMD GPU assembly. The dataset calls these paired columns cuda and hip; the task is assembly translation.

This snapshot was saved at optimizer step 4,000, 0.39142 epochs into a planned one-epoch run. Training was still in progress when published. This repository contains inference weights and tokenizer files; optimizer and training-resume state are not included.

Training

• Base revision: c1899de289a04d12100db370d81485cdf75e47ca.
• Dataset: ahmedheakl/daft-sm89-rdna-functions, revision a833e9887bb5e3623c52dc0ce57480784fd77ce6.
• Full eligible training split: 163,506 examples. Only exact whitespace-normalized source or target matches against the benchmark suites were excluded. This policy does not establish absence of semantic overlap.
• Sequence limit: 32,768 total tokens, including input and output. Training uses non-thinking mode and masks prompt tokens from the loss.
• Full SFT on one H100 using LlamaFactory 0.9.4, BF16, FlashAttention 2, Liger kernels, gradient checkpointing, and DeepSpeed ZeRO-2.
• Microbatch 1; gradient accumulation 16; effective batch 16.
• Initial learning rate 1e-5; cosine schedule; warmup ratio 0.03; weight decay 0.01; gradient clipping 1.0; seed 42; no sequence packing.
• Validation and checkpoint saving every 500 optimizer steps.

Validation at this checkpoint

Coverage reflects the prepared validation subsets that fit the sequence limit. These suites are used during training for validation and checkpoint selection; they are not an untouched final test set. These are teacher-forced loss measurements, not functional-correctness scores or generated-assembly accuracy. Functional correctness has not been evaluated for this publication.

Usage

The checkpoint was checked with Transformers 4.57.1. Install PyTorch and Transformers compatible with your runtime. Use non-thinking mode, matching training, and leave enough room for output within the 32,768-token training limit.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "aasim-m/DAFT-Qwen3-0.6B-checkpoint-4000"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16
).to("cuda").eval()

instruction = (
    "Translate the following NVIDIA GPU assembly function into corresponding AMD GPU assembly. "
    "Preserve the function's behavior. Return only the translated assembly, without explanations or Markdown fences."
)
source_assembly = "..."  # Replace with a complete NVIDIA assembly function.
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": instruction + "\n" + source_assembly}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
remaining = 32768 - inputs["input_ids"].shape[1]
if remaining <= 0:
    raise ValueError("Input leaves no output space within 32,768 tokens")
with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=min(4096, remaining),
        do_sample=False,
        use_cache=True,
        pad_token_id=tokenizer.eos_token_id,
    )
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

The saved configuration retains its original training settings, including use_cache=false; the example enables caching explicitly for inference. Generated assembly may be incomplete or incorrect and requires downstream validation before execution. This checkpoint has not been evaluated as a general-purpose assistant.

License and provenance

Apache-2.0, following the base model. The included LICENSE is copied from the pinned base-model revision. publication.json records the checkpoint identity, validation metrics, and file hashes. The published weights are unchanged from the local checkpoint.