Qwen3.5-0.8B-Instruct-SingleTurn

Qwen3.5-0.8B-Instruct-SingleTurn

This repository contains a fine-tuned version of Qwen/Qwen3.5-0.8B optimized specifically for single-turn English tasks.

🎯 Model Overview

• Base Model: Qwen/Qwen3.5-0.8B
• Language: English (primary)
• Intended Use: Single-turn instruction-following, downstream fine-tuning, and lightweight deployment.
• Key Improvements: Suppresses the base model's tendency for prolonged Chain-of-Thought (CoT) reasoning and reduces unwanted generation of Chinese text.

🛠️ Training Details

The model was fine-tuned using the following configuration and hyper-parameters:

• Dataset: agentlans/sft-data sample_k50000
• Sequence Handling: Packed sequences with a 2048 token cutoff length
• Attention: FlashAttention-2 (flash_attn)
• Learning Rate: 2e-5 (1.0 epoch)
• Regularization: NEFTune (alpha = 5)
• PEFT Method: LoRA via rsLoRA and PiSSA initialization
  • Rank: 16
  • Alpha: 32
  • Dropout: 0.05

⚠️ Limitations & Biases

• Scale Constraints: As a compact 0.8B parameter model, expect occasional factual inaccuracies and basic reasoning errors.
• Repetition: Extended responses may occasionally become repetitive.
• Domain Focus: Strictly optimized for English instruction, programming, and mathematics.
• Interaction Style: Not designed for multi-turn conversational chatbots, roleplay, or creative writing tasks.

📜 License

This model is licensed under the Apache 2.0 License.