Downloads · 30 days
22
18% of all-time downloads
dorukardahan/senti-qwen3-8b-dpo
senti-qwen3-8b-dpo is a text generation model from dorukardahan. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
A crypto sentiment analysis model fine-tuned on Qwen3-8B using SFT + DPO (Direct Preference Optimization).
Downloads · 30 days
22
18% of all-time downloads
All-time downloads
121
Public
Repo size
536 MB
Likes
1
Public
Click a slice to open those files.
.pt350 MB · 48%
From the Hugging Face model README
A crypto sentiment analysis model fine-tuned on Qwen3-8B using SFT + DPO (Direct Preference Optimization).
Senti is a specialized LLM for cryptocurrency market sentiment analysis. It generates structured market summaries from social media data, classifies sentiment, and answers questions about crypto markets.
Qwen/Qwen3-8B (Base, 8B params)
│
▼
┌─────────────────────────────────────┐
│ SFT Training │
│ 9,314 samples (train + val) │
│ 5.34 hours on A100 80GB │
└─────────────────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ DPO Training │
│ 1,477 preference pairs │
│ 30.7 min, 99.4% accuracy │
└─────────────────────────────────────┘
│
▼
This Model
| Dataset | Samples | Size | Purpose |
|---|---|---|---|
| SFT Training | 8,383 | 22.4 MB | Supervised fine-tuning |
| SFT Validation | 931 | 2.4 MB | Evaluation |
| DPO Preference | 1,477 pairs | 4.8 MB | Preference alignment |
| Total | 10,791 | 29.6 MB |
| Task | Samples | Percentage | Description |
|---|---|---|---|
| Summary Generation | 4,491 | 53.6% | Generate market summaries from tweets |
| Sentiment Classification | 3,624 | 43.2% | Classify individual tweet sentiment |
| Chat / Q&A | 268 | 3.2% | Answer crypto market questions |
| Summary Type | Count | Description |
|---|---|---|
| Hourly Summaries | 4,086 | Real-time market pulse from recent tweets |
| Daily Summaries | 405 | End-of-day market recap and synthesis |
| Metric | Value |
|---|---|
| Average | 44.4 tweets |
| Minimum | 5 tweets |
| Maximum | 100 tweets |
| Total Tweets Processed | ~199,000 |
| Metric | Characters | ~Tokens |
|---|---|---|
| Avg Input Length | 2,167 | ~540 |
| Avg Output Length | 995 | ~250 |
| Max Sequence Length | 4,096 | 4,096 |
5-Class System:
| Label | Description | Example |
|---|---|---|
STRONG_BULLISH | Extreme optimism, price targets, euphoria | "BTC to 200k! This is just the beginning! 🚀🚀🚀" |
BULLISH | Positive outlook, accumulation mentions | "Adding more ETH here, looking good for Q1" |
NEUTRAL | Factual, news, no clear sentiment | "Bitcoin trading at $97,500 today" |
BEARISH | Concern, caution, negative outlook | "Not liking this price action, reducing exposure" |
STRONG_BEARISH | Panic, crash predictions, extreme fear | "This is the top, selling everything, bear market incoming" |
Classification Distribution (3,624 samples):
| Label | Count | Percentage |
|---|---|---|
| BULLISH | ~1,200 | 33.1% |
| NEUTRAL | ~1,100 | 30.3% |
| STRONG_BULLISH | ~650 | 17.9% |
| BEARISH | ~450 | 12.4% |
| STRONG_BEARISH | ~224 | 6.2% |
The model was trained on data from 12 cryptocurrencies:
| Coin | Ticker | Category | Sample Distribution |
|---|---|---|---|
| Bitcoin | BTC | Major | ~18% |
| Ethereum | ETH | Major | ~16% |
| Solana | SOL | Layer 1 | ~12% |
| XRP | XRP | Payment | ~10% |
| BNB Chain | BNB | Layer 1 | ~9% |
| Chainlink | LINK | Oracle | ~8% |
| Arbitrum | ARB | Layer 2 | ~7% |
| Avalanche | AVAX | Layer 1 | ~6% |
| Hyperliquid | HYPE | DeFi | ~5% |
| 0G | 0G | AI/Infra | ~4% |
| Aster | ASTER | Emerging | ~3% |
| Zcash | ZEC | Privacy | ~2% |
| Source | Description |
|---|---|
| Twitter/X | Real-time tweets from crypto KOLs and communities |
| Time Range | October 2025 - January 2026 |
| Languages | English (primary), Turkish (secondary) |
| Filtering | Relevance scoring, spam removal, deduplication |
DPO (Direct Preference Optimization) aligns the model to produce higher-quality, more detailed responses by learning from preference pairs.
| Property | Value |
|---|---|
| Total Pairs | 1,477 |
| Format | (prompt, chosen, rejected) triplets |
| Purpose | Improve response quality and structure |
| Task | Pairs | Percentage | Description |
|---|---|---|---|
| Summary Generation | 877 | 59.4% | Prefer detailed market analysis |
| Chat / Q&A | 600 | 40.6% | Prefer informative, grounded answers |
| Language | Pairs | Percentage |
|---|---|---|
| English | 1,193 | 80.8% |
| Turkish | 284 | 19.2% |
| Metric | Chosen | Rejected |
|---|---|---|
| Average Length | 1,836 chars | 249 chars |
| Minimum Length | 1,026 chars | 198 chars |
| Maximum Length | 2,719 chars | 282 chars |
| Avg Ratio | 7.5x longer | baseline |
| Ratio Range | Description |
|---|---|
| Minimum | 4.0x (chosen vs rejected) |
| Average | 7.5x |
| Maximum | 13.7x |
| Component | Method |
|---|---|
| Chosen Responses | Generated by GPT-4o with detailed prompts |
| Rejected Responses | Systematically degraded versions |
Chosen Response Characteristics:
Rejected Response Degradation Methods:
Example Preference Pair:
Prompt: "Summarize BTC market sentiment from these tweets: [100 tweets about Bitcoin ETF inflows]"
Chosen (1,850 chars):
"## BTC Hourly Summary
**Overall Sentiment: BULLISH (7.8/10)**
### Key Developments
- Bitcoin broke through $98,000 resistance with strong volume
- ETF inflows reached $500M for the third consecutive day
- Whale accumulation: 3 wallets acquired 2,500+ BTC in 24h
### Risk Factors
- RSI approaching overbought (72)
- Some profit-taking expected at $100K psychological level
### Outlook
Momentum strong. Short-term target: $102,000. Support at $95,500."
Rejected (198 chars):
"Bitcoin sentiment is positive today. The market shows bullish signs.
Traders are optimistic about price movements. Consider doing your own research."
| Parameter | Value |
|---|---|
| Base Model | Qwen/Qwen3-8B |
| Method | QLoRA (4-bit NF4) |
| Epochs | 3 |
| Batch Size | 16 (4 × 4 grad accum) |
| Learning Rate | 1e-4 |
| Warmup Ratio | 0.05 |
| LR Scheduler | Cosine |
| Max Sequence Length | 4,096 tokens |
| LoRA Rank (R) | 32 |
| LoRA Alpha | 64 |
| LoRA Dropout | 0.05 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training Time | 5.34 hours |
| Hardware | NVIDIA A100 80GB |
| Parameter | Value |
|---|---|
| Base | SFT model (merged) |
| Method | DPO with QLoRA |
| Epochs | 1 |
| Batch Size | 16 (2 × 8 grad accum) |
| Learning Rate | 5e-7 |
| Beta (KL penalty) | 0.1 |
| Loss Type | Sigmoid |
| Max Length | 2,048 tokens |
| LoRA Rank (R) | 16 |
| LoRA Alpha | 32 |
| Training Time | 30.7 minutes |
| Final Loss | 0.569 |
| Preference Accuracy | 99.4% |
Input: Raw tweets about a cryptocurrency Output: Structured market analysis
## BTC Hourly Summary
**Overall Sentiment: BULLISH (7.8/10)**
### Key Developments
- Bitcoin broke through $98,000 resistance with strong volume
- Whale accumulation detected: 3 wallets acquired 2,500+ BTC
- ETF inflows reached $500M for the third consecutive day
### Risk Factors
- RSI approaching overbought territory (72)
- Some profit-taking expected at $100K psychological level
### Outlook
Momentum remains strong. Short-term target: $102,000.
Input: Single tweet Output: Sentiment label + confidence
Input: "BTC breaking 100k! This is the moment we've been waiting for! 🚀"
Output: STRONG_BULLISH (confidence: 0.94)
Supported Labels:
Input: Question about crypto market Output: Data-grounded answer
Q: "What's driving ETH sentiment today?"
A: "Based on recent data, ETH sentiment is moderately bullish (6.5/10).
Key drivers include:
1. Successful Dencun upgrade mentions (+45% positive)
2. Layer 2 TVL growth reaching $15B
3. Some concerns about gas fees during high activity periods
The overall trend suggests cautious optimism."
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load model
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-8B",
torch_dtype=torch.bfloat16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "dorukardahan/senti-qwen3-8b-dpo")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
# Generate
messages = [
{"role": "system", "content": "You are Senti, a crypto sentiment analyst."},
{"role": "user", "content": "Analyze BTC market sentiment based on recent social media activity."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
from transformers import BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
)
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-8B",
quantization_config=bnb_config,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "dorukardahan/senti-qwen3-8b-dpo")
| Format | Size | Platform | Latency | Memory |
|---|---|---|---|---|
| FP16 (native) | ~16 GB | GPU | ~1-2s | 20+ GB VRAM |
| INT8 / Q8_0 | ~8 GB | GPU/CPU | ~2-5s | 12+ GB |
| INT4 / Q4_K_M | ~4.5 GB | CPU | ~5-10s | 8+ GB RAM |
| Together.ai | Managed | Cloud | ~1-2s | N/A |
| Metric | Value |
|---|---|
| Final Loss | 0.569 |
| Preference Accuracy | 99.4% |
| Reward Margin (chosen - rejected) | 0.268 |
| Metric | Before DPO | After DPO |
|---|---|---|
| Response Structure | Basic | Well-organized with headers |
| Specificity | Generic | Includes specific data points |
| Actionability | Low | High (clear insights) |
| Length | Variable | Consistent, comprehensive |
Not Intended For:
| Model | Purpose | Link |
|---|---|---|
| SFT Base | Pre-DPO version | dorukardahan/senti-qwen3-8b-sft |
@misc{senti-qwen3-8b-dpo,
author = {Doruk Ardahan},
title = {Senti: Crypto Sentiment Analysis Model with DPO},
year = {2025},
publisher = {HuggingFace},
url = {https://huggingface.co/dorukardahan/senti-qwen3-8b-dpo}
}