qwen2.5-sql-general

📝 Model Card - Kişiselleştirme Şablonu

Aşağıdaki bölümleri kendi bilgilerinizle doldurun:

---

Qwen2.5-Coder Fine-tuned for Text-to-SQL

🔄 [BURAYA KENDİ AÇIKLAMANIZI YAZIN]
Örnek: "Bu model doğal dil sorularını SQL sorgularına dönüştürmek için özel olarak eğitilmiştir. Çeşitli SQL pattern'lerini ve veritabanı yapılarını anlayarak doğru sorgular üretebilir."

🎯 Model Özellikleri

Base Model: unsloth/Qwen2.5-Coder-3B-Instruct-bnb-4bit
Task: Text-to-SQL Generation
Language: English
License: Apache 2.0

Neler Yapabilir?

• ✅ Doğal dil sorularını SQL'e çevirir
• ✅ Karmaşık JOIN sorguları oluşturabilir
• ✅ Aggregate fonksiyonlar (COUNT, SUM, AVG) kullanabilir
• ✅ Subquery'ler ve HAVING clause'ları destekler
• ✅ Çoklu tablo sorgularını anlayabilir

📊 Eğitim Detayları

Kullanılan Veri Setleri

Bu model 3 farklı yüksek kaliteli SQL dataset'iyle eğitilmiştir:

1. b-mc2/sql-create-context

• Örnek Sayısı: ~78,577
• İçerik: SQL sorguları ve CREATE TABLE context'leri
• Özellik: Çeşitli domain'ler (e-ticaret, finans, HR, vb.)
• Kalite: Yüksek kaliteli, gerçek dünya senaryoları

Örnek:

Question: "Show all products with price greater than 100"
Schema: CREATE TABLE products (id INT, name VARCHAR, price DECIMAL)
Answer: SELECT * FROM products WHERE price > 100

2. Clinton/Text-to-sql-v1

• Örnek Sayısı: ~56,355
• İçerik: Instruction-based SQL generation
• Özellik: Soru-cevap formatında
• Kalite: Çeşitli SQL pattern'leri

Örnek:

Instruction: "Count employees by department"
Input: CREATE TABLE employees (id, name, department)
Response: SELECT department, COUNT(*) FROM employees GROUP BY department

3. knowrohit07/know_sql

• Örnek Sayısı: ~13,436
• İçerik: Karmaşık SQL sorguları
• Özellik: İleri seviye query pattern'leri
• Kalite: Challenging sorular

Örnek:

Question: "Find departments with average salary above 50000"
Schema: CREATE TABLE employees (id, name, salary, department)
Answer: SELECT department, AVG(salary) FROM employees 
        GROUP BY department HAVING AVG(salary) > 50000

Dataset İstatistikleri

Dataset Birleştirme

Dataset'ler interleave_datasets ile birleştirildi:

• Her batch'te 3 dataset'ten sırayla örnek alınır
• Bu sayede model çeşitli SQL pattern'lerini dengeli öğrenir
• Overfitting önlenir

dataset = DatasetDict({
    'train': interleave_datasets([dataset1_train, dataset2_train, dataset3_train]),
    'test': interleave_datasets([dataset1_test, dataset2_test, dataset3_test]),
    'validation': interleave_datasets([dataset1_val, dataset2_val, dataset3_val])
})

🏋️ Eğitim Süreci

Hardware & Software

• GPU: [BURAYA GPU TİPİNİZİ YAZIN] (örn: NVIDIA RTX 3070, A100, T4)
• VRAM: [BURAYA VRAM YAZIN] (örn: 8GB, 16GB, 40GB)
• Framework: Unsloth + Hugging Face Transformers
• Training Time: ~[BURAYA SÜRE YAZIN] saat (örn: 5-6 saat total)

Training Configuration

training_args = TrainingArguments(
    output_dir="./sql_qwen_output",
    
    # Batch ayarları
    per_device_train_batch_size=4,
    per_device_eval_batch_size=4,
    gradient_accumulation_steps=4,  # Effective batch size = 16
    
    # Epoch ve learning
    num_train_epochs=2,
    learning_rate=2e-4,
    warmup_ratio=0.03,
    lr_scheduler_type="cosine",
    
    # Optimization
    optim="adamw_8bit",
    weight_decay=0.01,
    max_grad_norm=1.0,
    
    # Precision
    fp16=True,  # or bf16=True
    
    # Evaluation & Saving
    eval_strategy="steps",
    eval_steps=500,
    save_steps=500,
    save_total_limit=2,
    load_best_model_at_end=True,
    
    # Logging
    logging_steps=50,
)

LoRA Configuration

model = FastLanguageModel.get_peft_model(
    model,
    r=16,                    # LoRA rank
    lora_alpha=16,           # LoRA alpha
    lora_dropout=0.05,       # Dropout
    bias="none",
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj"
    ],
    use_gradient_checkpointing="unsloth",
)

Trainable Parameters: 29,933,568 / 3,115,872,256 (0.96%)

Training Progress

Final Losses:

• Training Loss: 0.355 (1. epoch sonu)
• Validation Loss: 0.420 (1. epoch sonu)

Training Insights

📈 Loss Trend:

• İlk 1000 step: Hızlı düşüş (0.509 → 0.434)
• 1000-3000 steps: Stabil düşüş
• 3000-7000 steps: Yavaş ama düzenli iyileşme
• 7000+ steps: Platoya yakın

⚠️ Overfitting Kontrolü:

• Train-Val gap: ~0.065 (sağlıklı seviye)
• Validation loss düzenli azalıyor ✓
• Early stopping gerek olmadı ✓

📈 Performans Metrikleri

Test Set Sonuçları (14,835 örnekte)

Başarı Dağılımı

Mükemmel (≥80%):  47/50  (94%)  ████████████████████
İyi (60-80%):      1/50  (2%)   █
Orta (40-60%):     1/50  (2%)   █
Zayıf (<40%):      1/50  (2%)   █

SQL Tiplerine Göre Performans

💻 Nasıl Kullanılır?

Kurulum

pip install unsloth transformers torch accelerate

Hızlı Başlangıç

from unsloth import FastLanguageModel

# Model yükle
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="[BURAYA HF USERNAME]/qwen2.5-sql-text-to-sql",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

# Inference için hazırla
FastLanguageModel.for_inference(model)

# SQL üret
def generate_sql(schema, question):
    prompt = f"""<|im_start|>system
You are a SQL expert.<|im_end|>
<|im_start|>user
Tables:
{schema}

Question:
{question}<|im_end|>
<|im_start|>assistant
"""
    
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    outputs = model.generate(
        **inputs,
        max_new_tokens=200,
        temperature=0.1,
        do_sample=True,
    )
    
    result = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return result.split("assistant")[-1].strip()

# Kullanım
schema = "CREATE TABLE employees (id INT, name VARCHAR(100), age INT, salary DECIMAL)"
question = "Get employees older than 30"

sql = generate_sql(schema, question)
print(sql)
# Output: SELECT * FROM employees WHERE age > 30

Gelişmiş Kullanım

# Temperature ayarı
outputs = model.generate(
    **inputs,
    max_new_tokens=200,
    temperature=0.05,  # Daha deterministik (0.0-1.0)
    top_p=0.9,
    top_k=50,
    repetition_penalty=1.1,
)

# Batch processing
questions = ["Question 1", "Question 2", "Question 3"]
schemas = ["Schema 1", "Schema 2", "Schema 3"]

results = []
for q, s in zip(questions, schemas):
    sql = generate_sql(s, q)
    results.append(sql)

📚 Örnek Sorgular

<details> <summary>Örnek 1: Basit SELECT</summary>

Soru: "Show all products with price between 50 and 100"

Schema:

CREATE TABLE products (
    id INT,
    name VARCHAR(100),
    price DECIMAL,
    category VARCHAR(50)
)

Üretilen SQL:

SELECT * FROM products WHERE price BETWEEN 50 AND 100

</details> <details> <summary>Örnek 2: GROUP BY</summary>

Soru: "Count orders by status"

Schema:

CREATE TABLE orders (
    id INT,
    customer_id INT,
    status VARCHAR(20),
    total DECIMAL
)

Üretilen SQL:

SELECT status, COUNT(*) FROM orders GROUP BY status

</details> <details> <summary>Örnek 3: JOIN</summary>

Soru: "Get customer names with their order totals"

Schema:

CREATE TABLE customers (id INT, name VARCHAR(100))
CREATE TABLE orders (id INT, customer_id INT, total DECIMAL)

Üretilen SQL:

SELECT customers.name, SUM(orders.total)
FROM customers
JOIN orders ON customers.id = orders.customer_id
GROUP BY customers.name

</details>

⚠️ Limitasyonlar

1. Domain Specificity

   • Genel SQL'de mükemmel performans
   • Domain-specific (HR, finance) dataset'lerde performans düşebilir
   • Çözüm: Domain-specific fine-tuning yapın

2. Complex Queries

   • 3+ JOIN içeren sorgularda %85 doğruluk
   • Nested subquery'lerde dikkatli olunmalı
   • Çözüm: Few-shot prompting ile örnekler verin

3. Database Dialects

   • Standard SQL'de eğitildi
   • PostgreSQL, MySQL specific syntax farklılık gösterebilir
   • Çözüm: Dialect-specific keywords prompt'a ekleyin

4. Schema Understanding

   • Tablo ve column isimleri net olmalı
   • Ambiguous isimler zorluk yaratabilir
   • Çözüm: Descriptive schema isimleri kullanın

🔄 Domain-Specific Fine-Tuning

Kendi domain'iniz için modeli iyileştirmek isterseniz:

Adım 1: Veri Hazırlama

your_data = [
    {
        "question": "What is the company's remote work policy?",
        "context": "CREATE TABLE company_policies (policy_name VARCHAR, policy_text TEXT)",
        "answer": "SELECT policy_text FROM company_policies WHERE policy_name = 'Remote Work Policy'"
    },
    # ... daha fazla örnek
]

Adım 2: Fine-tuning

from unsloth import FastLanguageModel
from trl import SFTTrainer
from transformers import TrainingArguments

# Model yükle
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="[BURAYA HF USERNAME]/qwen2.5-sql-text-to-sql",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

# LoRA ekle
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0.05,
)

# Training arguments
training_args = TrainingArguments(
    output_dir="./domain_finetuned",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    learning_rate=1e-4,  # Daha düşük (catastrophic forgetting önlemek için)
    warmup_ratio=0.1,
    save_steps=100,
    eval_steps=100,
)

# Train
trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=your_dataset,
    args=training_args,
)

trainer.train()

🎯 Use Cases

1. Business Intelligence Tools

# Kullanıcı: "Show me top 10 customers by revenue this year"
# Sistem → SQL → Dashboard

2. Data Analysis Automation

# Analyst: "What's the average order value by region?"
# Model → SQL → Analytics

3. Natural Language Database Interface

# User query → Model → SQL → Database → Results

4. SQL Learning Assistant

# Student: "How do I join two tables?"
# Model: Provides SQL example

📊 Karşılaştırma

Avantajlar:

• ✅ Küçük boyut (efficient deployment)
• ✅ Hızlı inference (4-bit quantization)
• ✅ Fine-tuning friendly (LoRA)
• ✅ Open source

🔬 Ablation Studies

Dataset Ablasyon

Sonuç: Dataset çeşitliliği +7% iyileşme sağladı

LoRA Rank Ablasyon

Sonuç: r=16 optimal (performans/hız dengesi)

Learning Rate Ablasyon

Sonuç: 2e-4 en iyi sonucu verdi

🛠️ Troubleshooting

Problem: Düşük Kalite SQL'ler

Çözüm 1: Temperature'ı düşürün

temperature=0.05  # Daha deterministik

Çözüm 2: Few-shot examples ekleyin

prompt = f"""Examples:
Question: Count users
SQL: SELECT COUNT(*) FROM users

Your turn:
Question: {question}
SQL:"""

Problem: Yavaş Inference

Çözüm 1: Batch processing

inputs = tokenizer(prompts, return_tensors="pt", padding=True)
outputs = model.generate(**inputs)

Çözüm 2: Cache kullanın

outputs = model.generate(
    **inputs,
    use_cache=True,  # KV cache aktif
)

Problem: CUDA Out of Memory

Çözüm 1: Batch size azalt

per_device_batch_size=1

Çözüm 2: Gradient checkpointing

gradient_checkpointing=True

Çözüm 3: Max sequence length azalt

max_seq_length=1024  # 2048 yerine

📈 Future Improvements

Gelecekte eklenebilecek özellikler:

• Multi-database dialect support (PostgreSQL, MySQL, SQLite)
• Query optimization suggestions
• Error explanation for invalid SQL
• Natural language result explanation
• Query execution plan generation
• Larger model versions (7B, 14B)

🤝 Contributing

Katkıda bulunmak isterseniz:

1. Bug Report: GitHub issues'da bug bildirin
2. Feature Request: Yeni özellik önerileri
3. Dataset Contribution: Yeni SQL örnekleri ekleyin
4. Model Improvement: Fine-tuning sonuçlarınızı paylaşın

📚 References & Resources

Papers

• T5: Text-to-Text Transfer Transformer
• LoRA: Low-Rank Adaptation
• QLoRA: Efficient Finetuning

Related Models

• cssupport/t5-small-awesome-text-to-sql
• Qwen/Qwen2.5-Coder-3B-Instruct

Tools & Frameworks

• Unsloth - Fast LLM training
• Hugging Face Transformers
• TRL - Transformer Reinforcement Learning

📧 Contact & Support

Author: [BURAYA ADINIZ]
Email: [BURAYA EMAİLİNİZ]
GitHub: [BURAYA GITHUB]
LinkedIn: [BURAYA LINKEDIN]
Twitter: [BURAYA TWITTER]

Support Channels

• 🐛 Bug reports: [GitHub Issues]
• 💬 Discussions: [GitHub Discussions]
• 📧 Email: [your-email@example.com]
• 💼 Commercial inquiries: [business-email@example.com]

🙏 Acknowledgments

Special thanks to:

• Qwen Team - For the amazing base model
• Unsloth Team - For efficient fine-tuning framework
• Hugging Face - For the model hosting and tools
• Dataset Contributors:
  • b-mc2 for sql-create-context
  • Clinton for Text-to-sql-v1
  • knowrohit07 for know_sql
• Community - For feedback and testing

📜 Citation

If you use this model in your research or application, please cite:

@misc{qwen25-sql-text-to-sql-2025,
  title={Qwen2.5-Coder Fine-tuned for Text-to-SQL Generation},
  author={[BURAYA ADINIZ]},
  year={2025},
  publisher={HuggingFace},
  journal={HuggingFace Model Hub},
  howpublished={\url{https://huggingface.co/[USERNAME]/qwen2.5-sql-text-to-sql}}
}

Dataset Citations

@misc{sql-create-context,
  title={SQL Create Context Dataset},
  author={b-mc2},
  year={2023},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/datasets/b-mc2/sql-create-context}}
}

@misc{text-to-sql-v1,
  title={Text to SQL v1 Dataset},
  author={Clinton},
  year={2023},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/datasets/Clinton/Text-to-sql-v1}}
}

@misc{know-sql,
  title={Know SQL Dataset},
  author={knowrohit07},
  year={2023},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/datasets/knowrohit07/know_sql}}
}

📄 License

This model is released under the Apache License 2.0.

Copyright 2025 [BURAYA ADINIZ]

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Third-Party Licenses

• Base model (Qwen2.5-Coder): Apache 2.0
• Datasets: Various (see individual dataset pages)
• Training framework (Unsloth): Apache 2.0

⚖️ Ethical Considerations

Intended Use

This model is intended for:

• ✅ Educational purposes
• ✅ Research and development
• ✅ Business intelligence tools
• ✅ Data analysis automation

Not Intended For

• ❌ Direct production use without validation
• ❌ Critical systems without human oversight
• ❌ Automated database modification without review
• ❌ Security-sensitive applications without audit

Potential Risks

1. SQL Injection: Always validate and sanitize generated queries
2. Data Exposure: Generated queries may access sensitive data
3. Performance: Complex queries may impact database performance
4. Accuracy: Model may generate incorrect queries (5-10% error rate)

Mitigation Strategies

• ✅ Always validate SQL before execution
• ✅ Use read-only database connections for testing
• ✅ Implement query review workflow
• ✅ Set query timeout limits
• ✅ Log all generated queries
• ✅ Monitor for suspicious patterns

🔒 Security Notice

WARNING: Never execute generated SQL queries on production databases without:

1. Human review and approval
2. Query validation and sanitization
3. Proper access controls
4. Audit logging
5. Backup and recovery procedures

Best Practices:

# ✅ GOOD: Validate before execution
sql = generate_sql(schema, question)
if validate_sql(sql):  # Your validation logic
    result = execute_with_timeout(sql, timeout=10)
else:
    raise ValueError("Invalid SQL generated")

# ❌ BAD: Direct execution
sql = generate_sql(schema, question)
cursor.execute(sql)  # DANGEROUS!

📊 Model Card Summary

🌟 Star History

If you find this model useful, please ⭐ star the repo and share with others!

---

📝 Version History

v1.0.0 (Current)

• ✨ Initial release
• 🎯 95.87% accuracy on test set
• 📚 Trained on 148K examples
• 🚀 Optimized with LoRA (r=16)

Future Versions

• v1.1.0: Multi-dialect support planned
• v1.2.0: Larger model variants (7B) planned
• v2.0.0: Query optimization features planned

---

Last Updated: [BURAYA TARİH] (örn: January 2025)
Model Version: 1.0.0
Status: ✅ Production Ready

---

<div align="center">

Made with ❤️ by simya

🏠 Homepage • 📧 Email • 🐦 Twitter

</div>