Data Training Manager

Data Training Manager

Continuous learning system for managing AI training data, monitoring content freshness, and preventing repetitive outputs.

Overview

Maintain high-quality AI outputs through:

• Training Data Management - Add, update, remove training samples
• Freshness Monitoring - Detect stale and repetitive content
• Quality Scoring - Track performance of training samples
• Continuous Learning - Automatically update based on engagement
• Trend Analysis - Identify patterns in successful content

Quick Start

1. Check Training Data Freshness

from src.freshness_monitor import FreshnessMonitor

monitor = FreshnessMonitor()

# Check if generated content is fresh
score = monitor.check_freshness(
    generated_text="gm to data contributors who deserve equity...",
    threshold=0.7  # 70% uniqueness required
)

if score < 0.7:
    print("⚠️ Content too similar to existing samples")
else:
    print("✅ Content is fresh!")

2. Add New Training Sample

from src.continuous_learning import ContinuousLearningSystem

learning = ContinuousLearningSystem()

# Add high-performing tweet
learning.add_sample(
    text="gm to everyone building on @base 💙",
    type="gm",
    engagement={"likes": 150, "retweets": 20},
    features={"has_emoji": True, "mentions": ["@base"]}
)

3. Manage Training Data

# Check freshness of all samples
python scripts/​manage_training.py check

# View statistics
python scripts/​manage_training.py stats

# Add new sample
python scripts/​manage_training.py add \
  --text "your tweet text" \
  --type gm \
  --engagement '{"likes":100}'

Training Data Structure

Sample Format

Each training sample contains:

{
  "id": "sample_001",
  "text": "The actual content...",
  "type": "gm|insight|casual|reply",
  "topic": "data_ownership|x402|base|milady|...",
  "style": "short|medium|long",
  "created_at": "2026-01-07T10:00:00Z",
  "engagement": {
    "likes": 150,
    "retweets": 30,
    "replies": 10,
    "impressions": 5000
  },
  "features": {
    "has_emoji": true,
    "emoji_list": ["🎀", "🧹"],
    "has_ascii_art": false,
    "has_thread": false,
    "has_mentions": true,
    "mention_list": ["@codatta_io"],
    "has_hashtags": false,
    "tone": "critical|supportive|casual|...",
    "word_count": 25,
    "char_count": 120
  },
  "freshness_score": 0.85,
  "quality_score": 0.92,
  "last_used": "2026-01-05T14:30:00Z",
  "use_count": 3,
  "performance_trend": "improving|stable|declining"
}

Training Files

Freshness Monitoring

How It Works

Freshness score (0.0-1.0) measures uniqueness:

def calculate_freshness(new_text, existing_samples):
    """
    Returns:
      1.0 = Completely unique
      0.8 = Similar but fresh
      0.5 = Moderately repetitive
      0.0 = Identical to existing
    """

    scores = []

    for sample in existing_samples:
        # 1. Jaccard similarity (word overlap)
        jaccard = jaccard_similarity(new_text, sample['text'])

        # 2. Phrase similarity (3-gram overlap)
        phrase = phrase_similarity(new_text, sample['text'])

        # 3. Semantic similarity (embedding distance)
        semantic = semantic_similarity(new_text, sample['text'])

        # Combined score (weighted)
        combined = (jaccard * 0.3 + phrase * 0.4 + semantic * 0.3)
        scores.append(combined)

    # Return inverse of max similarity
    return 1.0 - max(scores)

Usage

monitor = FreshnessMonitor()

# Check single text
score = monitor.check_freshness(
    "gm to data contributors 🎀",
    data_type="gm",
    threshold=0.7
)

# Batch check
texts = [
    "gm everyone",
    "good morning frens",
    "gm to builders on base"
]

results = monitor.batch_check(texts, threshold=0.7)
# Returns: [{"text": "...", "score": 0.85, "is_fresh": True}, ...]

Freshness Thresholds

FRESHNESS_THRESHOLDS = {
    "gm": 0.65,          # GM posts can be more repetitive
    "insight": 0.80,     # Insights must be unique
    "casual": 0.70,      # Casual moderate uniqueness
    "reply": 0.75        # Replies should be fresh
}

Continuous Learning System

Auto-Update from Performance

learning = ContinuousLearningSystem()

# Add successful tweet to training data
learning.learn_from_performance(
    tweet_id="1234567890",
    text="gm to data contributors who deserve equity 🎀",
    engagement={"likes": 200, "retweets": 40}
)

# System automatically:
# 1. Checks freshness
# 2. Evaluates quality
# 3. Adds to appropriate training file
# 4. Archives low-performers if needed

Performance Tracking

# Track sample performance over time
stats = learning.get_sample_stats("sample_001")

# Returns:
{
  "use_count": 5,
  "avg_engagement": {"likes": 120, "retweets": 25},
  "freshness_decay": 0.15,  # How much freshness dropped
  "trend": "stable",
  "recommendation": "keep|archive|update"
}

Auto-Archiving

# Archive low-performing samples
archived = learning.auto_archive(
    min_quality_score=0.6,
    min_freshness=0.5,
    max_age_days=90
)

print(f"Archived {len(archived)} samples")

Quality Scoring

Quality Metrics

def calculate_quality_score(sample):
    """
    Returns 0.0-1.0 quality score based on:
    - Engagement performance (40%)
    - Freshness (30%)
    - Feature diversity (20%)
    - Recency (10%)
    """

    # Engagement score (normalized)
    engagement_score = normalize_engagement(sample['engagement'])

    # Freshness score
    freshness_score = sample['freshness_score']

    # Feature diversity (more features = higher score)
    features = sample['features']
    diversity_score = calculate_diversity(features)

    # Recency score (newer = higher)
    recency_score = calculate_recency(sample['created_at'])

    # Weighted combination
    quality = (
        engagement_score * 0.4 +
        freshness_score * 0.3 +
        diversity_score * 0.2 +
        recency_score * 0.1
    )

    return quality

Usage

# Calculate quality for all samples
quality_report = learning.analyze_quality(
    data_type="gm",
    min_samples=10
)

# Returns:
{
  "avg_quality": 0.75,
  "high_quality": 15,  # score > 0.8
  "medium_quality": 20,  # 0.6-0.8
  "low_quality": 5,  # < 0.6
  "recommendations": [
    "Archive 5 low-quality samples",
    "Add more diversity to casual posts"
  ]
}

Management Scripts

Check Freshness

python scripts/​manage_training.py check

# Output:
# Checking gm_posts.json...
# ✅ 45/​50 samples are fresh (90%)
# ⚠️ 5 samples below threshold
#
# Checking codatta_insights.json...
# ✅ 58/​60 samples are fresh (97%)
# ⚠️ 2 samples below threshold
#
# Overall freshness: 93%

View Statistics

python scripts/​manage_training.py stats

# Output:
# Training Data Statistics
# ========================
#
# Total samples: 180
# - GM posts: 50
# - Insights: 60
# - Casual: 30
# - Interactions: 40
#
# Quality Distribution:
# - High (>0.8): 120 (67%)
# - Medium (0.6-0.8): 50 (28%)
# - Low (<0.6): 10 (5%)
#
# Freshness:
# - Avg score: 0.82
# - Min threshold: 0.70
# - Samples below: 8 (4%)

Add New Sample

# Interactive mode
python scripts/​manage_training.py add

# Prompts for:
# - Text content
# - Type (gm/​insight/​casual/​reply)
# - Topic
# - Engagement metrics
# - Features

# Non-interactive mode
python scripts/​manage_training.py add \
  --text "gm to builders on base 💙" \
  --type gm \
  --topic base \
  --engagement '{"likes":150,"retweets":30}' \
  --features '{"has_emoji":true,"mentions":["@base"]}'

Import Batch

# Import from CSV
python scripts/​manage_training.py import \
  --file successful_tweets.csv \
  --type gm \
  --min-likes 100

# Import from JSON
python scripts/​manage_training.py import \
  --file tweets_export.json \
  --auto-categorize  # Auto-detect type/​topic

Archive Old Samples

# Archive samples older than 90 days with low engagement
python scripts/​manage_training.py archive \
  --max-age 90 \
  --min-quality 0.6 \
  --dry-run  # Preview before archiving

# Actually archive
python scripts/​manage_training.py archive \
  --max-age 90 \
  --min-quality 0.6

View History

# Show addition/​removal history
python scripts/​manage_training.py history \
  --days 30

# Output:
# Training Data History (Last 30 days)
# =====================================
#
# 2026-01-07: Added 3 samples (gm)
# 2026-01-06: Archived 2 samples (low quality)
# 2026-01-05: Added 5 samples (insights)
# 2026-01-04: Updated 1 sample (engagement)
# ...

Trend Analysis

Identify Successful Patterns

from src.trend_analyzer import TrendAnalyzer

analyzer = TrendAnalyzer()

# Find common features in high-performing samples
trends = analyzer.analyze_trends(
    min_engagement={"likes": 100},
    days=30
)

# Returns:
{
  "top_features": [
    {"feature": "has_emoji", "success_rate": 0.85},
    {"feature": "mentions_base", "success_rate": 0.78},
    {"feature": "short_format", "success_rate": 0.72}
  ],
  "top_topics": [
    {"topic": "data_ownership", "avg_likes": 150},
    {"topic": "base_ecosystem", "avg_likes": 130}
  ],
  "optimal_length": {
    "word_count": "20-30",
    "char_count": "120-150"
  },
  "emoji_usage": {
    "optimal_count": "2-3",
    "top_emojis": ["🎀", "🧹", "💙"]
  }
}

Suggest Improvements

# Get suggestions for improving training data
suggestions = analyzer.suggest_improvements()

# Returns:
[
  "Add more samples about x402 token (only 5 currently)",
  "Increase casual content (15% vs target 20%)",
  "Archive 3 GM samples with freshness < 0.5",
  "Add more emoji diversity (currently 70% use 🎀)"
]

Advanced Features

A/​B Testing

# Test two versions of content
results = learning.ab_test(
    version_a="gm to data contributors 🎀",
    version_b="good morning to data labelers 🧹",
    duration_days=7
)

# Returns:
{
  "winner": "version_a",
  "version_a_engagement": {"likes": 120, "retweets": 25},
  "version_b_engagement": {"likes": 90, "retweets": 18},
  "confidence": 0.85
}

Template Generation

# Generate templates from high-performing samples
templates = learning.generate_templates(
    min_quality=0.8,
    max_templates=10
)

# Returns:
[
  {
    "template": "gm to {target_group} who deserve {value}",
    "variables": ["target_group", "value"],
    "examples": [
      "gm to data contributors who deserve equity",
      "gm to builders who deserve recognition"
    ]
  }
]

Diversity Analysis

# Check content diversity
diversity = learning.analyze_diversity()

# Returns:
{
  "topic_distribution": {
    "data_ownership": 0.35,
    "base_ecosystem": 0.25,
    "x402": 0.20,
    "casual": 0.15,
    "milady": 0.05
  },
  "style_distribution": {
    "short": 0.40,
    "medium": 0.45,
    "long": 0.15
  },
  "tone_distribution": {
    "critical": 0.30,
    "supportive": 0.40,
    "casual": 0.30
  },
  "diversity_score": 0.78,
  "recommendations": [
    "Increase Milady content (target 15%)",
    "Add more long-form content"
  ]
}

Integration with Content Generation

Use Training Data in Generation

from skills.twitter_content_ai.src.content_generator import ContentGenerator
from src.continuous_learning import ContinuousLearningSystem

generator = ContentGenerator()
learning = ContinuousLearningSystem()

# Generate using high-quality samples
tweet = generator.generate_from_samples(
    sample_type="gm",
    min_quality=0.8,
    ensure_freshness=0.75
)

# Learn from generated content
if tweet_posted:
    learning.learn_from_performance(
        tweet_id=tweet_id,
        text=tweet,
        engagement=get_engagement(tweet_id)
    )

Best Practices

1. Regular Freshness Checks - Run weekly to maintain quality
2. Archive Strategically - Don't delete, archive for future reference
3. Track Performance - Link training samples to actual tweets
4. Diverse Samples - Ensure variety in topics, styles, tones
5. Update Frequently - Add 3-5 new samples per week
6. Quality Over Quantity - 50 great samples > 200 mediocre ones
7. Monitor Trends - Analyze what's working and adjust
8. Test Changes - Use A/​B testing before large updates

Monitoring Dashboard

# Generate visual dashboard
dashboard = learning.generate_dashboard()

# Includes:
# - Freshness trend over time
# - Quality distribution
# - Topic balance
# - Performance metrics
# - Recommendations

dashboard.save("training_dashboard.html")

Configuration

Freshness Settings

# config/​learning_config.yaml
freshness:
  thresholds:
    gm: 0.65
    insight: 0.80
    casual: 0.70
    reply: 0.75
  check_interval_days: 7
  min_samples: 30

quality:
  min_score: 0.60
  archive_threshold: 0.50
  weights:
    engagement: 0.40
    freshness: 0.30
    diversity: 0.20
    recency: 0.10

automation:
  auto_add_successful: true
  auto_archive_old: true
  min_auto_add_likes: 100
  max_sample_age_days: 180

Troubleshooting

Too many low-freshness warnings:

# Lower thresholds temporarily
monitor.set_threshold("gm", 0.60)

Quality scores too low:

# Add more high-quality samples
python scripts/​manage_training.py import \
  --file best_tweets.json \
  --min-likes 150

Not enough diversity:

# Get diversity report
report = learning.diversity_report()
# Follow recommendations to add underrepresented topics

Related Documentation

• FRESHNESS_SYSTEM.md - Detailed freshness algorithm
• LEARNING_GUIDE.md - Continuous learning guide
• Training Data Examples

Related Skills

• twitter-content-ai - Uses training data for generation
• social-monitoring - Identifies high-performing content

---

Goal: Maintain 85%+ freshness score across all training data with continuous improvement.