Downloads · 30 days
11
23% of all-time downloads
charlie-li/Qwen3-30B-A3B-Instruct-2507-ScaleSWE-Distilled-Epoch6-v3
Qwen3-30B-A3B-Instruct-2507-ScaleSWE-Distilled-Epoch6-v3 is a machine learning model from charlie-li. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This directory contains the Epoch 6 v3 ScaleSWE-Distilled SFT checkpoint.
Downloads · 30 days
11
23% of all-time downloads
All-time downloads
47
Public
Parameters
30.5B
61.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors61.1 GB · 100%
From the Hugging Face model README
This directory contains the Epoch 6 v3 ScaleSWE-Distilled SFT checkpoint.
This v3 release is a continued full-parameter fine-tune that starts from the prior
charlie-li/Qwen3-30B-A3B-Instruct-2507-ScaleSWE-Distilled-Epoch3-v3.1 package and runs three
additional epochs on the 70k-token ScaleSWE-Distilled trajectories (cutoff 71680). It uses the
corrected cosine learning-rate schedule (smooth decay from 1e-5 after warmup to 0 by the final
step). The original pretrained base model is Qwen/Qwen3-30B-A3B-Instruct-2507.
charlie-li/Qwen3-30B-A3B-Instruct-2507-ScaleSWE-Distilled-Epoch3-v3.1Qwen3-30B-A3B-Instruct-2507Qwen3MoeForCausalLMdataset_index.yamltraining_config.yamlscaleswe_distilled_sharegpt_format_nothink_tool_role_70k.v1_messages.jsonlqwen3_nothinkflash_attention_22mp_replicate_size=4, mp_shard_size=8 (data parallel size 16)71680 tokens (70k)1128 (8 micro-batches per optimizer step)161e-50.051.0bfloat16Qwen/Qwen3-30B-A3B-Instruct-2507/Qwen/Qwen3-30B-A3B-Instruct-2507/ (byte-identical to the base model, instead of the trainer-round-tripped versions)3, in total 6 based on Qwen/Qwen3-30B-A3B-Instruct-2507/0 through 2 (zero-based epoch counter)200 steps511467 / 14670.2583732306957245logs/31_run_qwen3_30b_a3b_instruct_2507_scaleswe_llamafactory_v1_full_fsdp2_cp_4x8gpu_only_scaleswe_70k_rank0.logmodel-00001-of-00016.safetensors through model-00016-of-00016.safetensorsmodel.safetensors.index.jsonconfig.jsongeneration_config.jsontokenizer.json, tokenizer_config.json, vocab.json, merges.txt (sourced from the pretrained base model)chat_template.jinjaconfig_1m.json (from the upstream base model; not active by default)trainer_log.jsonltraining_config.yaml, dataset_index.yamlfrom transformers import AutoModelForCausalLM, AutoTokenizer
model_dir = "charlie-li/Qwen3-30B-A3B-Instruct-2507-ScaleSWE-Distilled-Epoch6-v3"
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_dir,
trust_remote_code=True,
device_map="auto",
)