Qwen3-14B-legend-10K

Legend Query Generation

Fine-tuned from Qwen/Qwen3-14B using ScalarLM with tokenformer adapters.

Model Details

• Base model: Qwen/Qwen3-14B
• Fine-tuning method: Tokenformer adapters (additive MLP adapters with learned attention)
• Checkpoint step: 6656
• Trained parameters: 442 (attention projections, layernorms, embeddings, tokenformer adapters)
• Tokenformer config: num_heads=4, r=32

How to Use

This model uses custom tokenformer adapter layers on top of the base model. It requires the TokenformerAdapter module for inference. Standard AutoModelForCausalLM.from_pretrained will not load the adapter weights correctly.

See the ScalarLM repository for inference instructions.

Training

Trained on Legend query generation data using the Qwen chat template format.

Prompt format:

<|im_start|>user
{question}

Here is some schema information:
{schema_text}

First, plan how to write the query, then write it in a ```pure SQL``` code block.
<|im_end|>
<|im_start|>assistant

License

Apache 2.0