qwen3-8b-triton

Note:This project represents independent research conducted on personal compute resources (rented from Modal.com) and is not associated with my employer or organization

Qwen3-8B-Triton-Finetune

A fine-tuned version of Qwen3-8B (the latest generation of Qwen large language models by Alibaba Cloud), further trained using a Triton-based fine-tuning pipeline. This model retains the strong reasoning and instruction-following capabilities of Qwen3-8B while adding task-specific adaptations via custom Triton kernels.

Model Details

• Base Model: Qwen/Qwen3-8B
• Architecture: Qwen3ForCausalLM
• Parameters: 8.19B (BF16)
• Hidden Size: 4096
• Intermediate Size: 12288
• Attention Heads: 32 (8 KV heads, grouped-query attention)
• Layers: 36 (full attention, no sliding window)
• Max Position Embeddings: 40,960 tokens
• Vocabulary Size: 151,936
• Attention Mechanism: RoPE (Rotary Position Embeddings, theta=1,000,000)
• Activation: SiLU (SwigLU)
• Precision: bfloat16
• Chat Template: im_start/im_end format with tool calling support

Capabilities

• Text generation & completion: General-purpose language understanding and generation
• Instruction following: Fine-tuned for chat and instructions
• Multi-step tool calling: Built-in tool/function calling support via the chat template
• Extended context: Supports up to ~40K tokens of context
• Reasoning: Supports think/reasoning blocks (<think>...</think>) in generation

Files

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "edwixx/qwen3-8b-triton-finetune",
    torch_dtype="bfloat16",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("edwixx/qwen3-8b-triton-finetune")

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain what fine-tuning with Triton means."}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

Apache 2.0 (inherited from Qwen3-8B).

Citation

@misc{edwixx-qwen3-8b-triton-finetune,
  author = {Anurag Kanade},
  title = {qwen3-8b-triton-finetune},
  year = {2026},
  publisher = {Hugging Face},
  journal = {Hugging Face Hub},
  howpublished = {\url{https://huggingface.co/edwixx/qwen3-8b-triton-finetune}}
}