senti-qwen3-8b-sft

Senti Qwen3-8B SFT

A crypto sentiment analysis model fine-tuned on Qwen3-8B using Supervised Fine-Tuning (SFT).

Note: For best results, use the DPO-optimized version: dorukardahan/senti-qwen3-8b-dpo

Model Description

Senti is a specialized LLM for cryptocurrency market sentiment analysis. This is the SFT (Supervised Fine-Tuning) base model, trained on crypto summaries, sentiment classification, and chat data.

Training Pipeline

Qwen/Qwen3-8B (Base, 8B params)
       │
       ▼
┌─────────────────────────────────────┐
│         SFT Training                │
│   9,314 samples (train + val)       │
│   5.34 hours on A100 80GB           │
└─────────────────────────────────────┘
       │
       ▼
   This Model ────► DPO Training ────► senti-qwen3-8b-dpo (recommended)

---

Model Details

---

Training Data

Overview

Task Distribution

Summary Task Breakdown

Tweets Per Summary

Input/Output Statistics

Sentiment Classification Details

5-Class System:

Classification Distribution (3,624 samples):

Coins Covered

The model was trained on data from 12 cryptocurrencies:

Data Sources

---

Training Configuration

SFT Training Parameters

LoRA Configuration

Quantization (QLoRA)

---

Model Capabilities

1. Summary Generation

Input: Raw tweets about a cryptocurrency Output: Structured market analysis

## BTC Hourly Summary

**Overall Sentiment: BULLISH (7.2/10)**

### Key Developments
- Bitcoin testing $98,000 resistance level
- ETF inflows remain positive
- Whale activity detected on-chain

### Risk Factors
- Some profit-taking at round numbers
- RSI elevated but not overbought

### Outlook
Momentum positive. Watch $100K level.

2. Sentiment Classification

Input: Single tweet Output: Sentiment label

Input: "ETH looking strong here, accumulating more"
Output: BULLISH

Supported Labels:

• STRONG_BULLISH
• BULLISH
• NEUTRAL
• BEARISH
• STRONG_BEARISH

3. Chat / Q&A

Input: Question about crypto market Output: Informative answer

Q: "What's the current sentiment on SOL?"
A: "Based on recent social media activity, Solana sentiment is moderately
    bullish. Key themes include ecosystem growth, DeFi TVL increases, and
    positive price action. Some concerns exist about network stability
    during high-traffic periods."

---

Usage

Basic Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "dorukardahan/senti-qwen3-8b-sft")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")

# Generate
messages = [
    {"role": "system", "content": "You are Senti, a crypto sentiment analyst."},
    {"role": "user", "content": "Summarize BTC market sentiment."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

With 4-bit Quantization (Lower VRAM)

from transformers import BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    quantization_config=bnb_config,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "dorukardahan/senti-qwen3-8b-sft")

---

Deployment Options

---

Limitations

• This is the SFT base model; DPO version is recommended for production
• Trained primarily on English (80%) and Turkish (20%) data
• Knowledge reflects training data up to January 2026
• Optimized for 12 specific cryptocurrencies
• Should not be used as sole basis for financial decisions
• May reflect biases present in social media data

---

Intended Use

• Cryptocurrency market sentiment analysis
• Social media summarization for research
• Base for further preference optimization (DPO/RLHF)
• Experimentation and comparison with DPO version

Not Intended For:

• Financial advice
• Automated trading decisions without human oversight
• Production deployment (use DPO version instead)

---

Related Models

---

Citation

@misc{senti-qwen3-8b-sft,
  author = {Doruk Ardahan},
  title = {Senti: Crypto Sentiment Analysis Model (SFT)},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/dorukardahan/senti-qwen3-8b-sft}
}

---

Acknowledgments

• Base Model: Qwen Team for Qwen3-8B
• Training Framework: HuggingFace TRL
• Infrastructure: RunPod for A100 GPU access