GitPulse

GitPulse: Multimodal Time Series Prediction for GitHub Project Health

GitPulse is a multimodal Transformer-based model that combines project text descriptions with historical activity data to predict GitHub project health metrics.

Model Description

GitPulse leverages both textual metadata (project descriptions, topics) and historical time series (commits, issues, stars, etc.) to forecast future project activity. The key innovation is the adaptive fusion mechanism that dynamically balances text and time-series features.

Architecture

• Text Encoder: DistilBERT-based encoder with attention pooling
• Time Series Encoder: Transformer encoder with positional embeddings
• Adaptive Fusion: Dynamic gating mechanism for multimodal fusion
• Prediction Head: MLP for generating future predictions

Model Parameters

Performance

Evaluated on 636 test samples from 4,232 GitHub projects:

Text Contribution

Usage

Installation

pip install torch transformers

Quick Start

import torch
from transformers import DistilBertTokenizer

# Load model
from model import GitPulseModel
model = GitPulseModel.from_pretrained('./')

# Prepare inputs
tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
text = "A Python library for machine learning"
encoded = tokenizer(text, padding='max_length', truncation=True, 
                    max_length=128, return_tensors='pt')

# Time series: [batch, hist_len, n_vars]
time_series = torch.randn(1, 128, 16)

# Predict
model.eval()
with torch.no_grad():
    predictions = model(
        time_series,
        input_ids=encoded['input_ids'],
        attention_mask=encoded['attention_mask']
    )
# predictions shape: [1, 32, 16]

Inference API

# Simple prediction interface
predictions = model.predict(
    time_series=history_data,  # [batch, 128, 16]
    text="Project description...",
    tokenizer=tokenizer
)

Training Details

• Dataset: GitHub project activity data (4,232 projects)
• Train/Val/Test Split: 70% / 15% / 15%
• Optimizer: AdamW (lr=1e-5, weight_decay=0.01)
• Fine-tuning Strategy: Freeze encoder, train prediction head
• Hardware: NVIDIA RTX GPU

Input Features (16 variables)

1. Commits count
2. Issues opened
3. Issues closed
4. Pull requests opened
5. Pull requests merged
6. Stars gained
7. Forks count
8. Contributors count
9. Code additions
10. Code deletions
11. Comments count
12. Releases count
13. Wiki updates
14. Discussions count
15. Sponsors count
16. Watchers count

Limitations

• Trained on English project descriptions only
• Best suited for projects with at least 128 months of history
• Performance may vary for niche domains not well represented in training

Citation

@article{gitpulse2024,
  title={GitPulse: Multimodal Time Series Prediction for GitHub Project Health},
  author={Anonymous},
  journal={arXiv preprint},
  year={2024}
}

License

Apache 2.0