repo-paper-alignment

Repo-Paper Alignment Model

Scores and embeds paper/repository pairs for mapping paper methods and descriptions to repository functions, files, or code spans.

Model Details

• Artifact type: full fine-tuned Transformers model
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Backbone type: encoder
• Model ID: C2
• Tier: T5_cross
• Role in stack: paper-code alignment and cross-domain fusion component
• Version: v2

This model is part of the Repository Library stack, a research system for indexing, retrieving, aligning, and reasoning over scientific papers, structured paper content, repositories, and cross-domain links between them.

Model Sources

• Hugging Face repo: https://huggingface.co/PeytonT/repo-paper-alignment
• Hugging Face collection: https://huggingface.co/collections/PeytonT/research-library-6a49c589ef4d763f7539b50d
• GitHub repository: https://github.com/peytontolbert/research_library
• Experiment config: https://github.com/peytontolbert/research_library/blob/main/models/experiments/c2_repo_paper_alignment.json
• Models directory: https://github.com/peytontolbert/research_library/tree/main/models

Intended Use

• Primary use: Scores paper/repository text pairs for cross-domain alignment between papers and implementation artifacts.
• Downstream use: reranking alignment candidates, retrieving repository evidence for papers, constructing paper-code links, planning repository-grounded explanations, and supporting cross-domain reasoning inside the broader Repository Library system.
• Out of scope: production safety claims, benchmark claims beyond the tracked experiment config, or deployment without task-specific validation.

Training Data

The training inputs for this package were assembled from the following Repository Library data sources:

• Source paper_repo_pairs: paper/repository pair records from the Repository Library alignment pipeline.
• Alignment path: exports/paper_repo_span_align.jsonl

The v2 hardening pass applies:

• dedupe: true
• balance_binary: true
• Pair quality filters: pairs_min_chars=64, pairs_max_chars=131072

Training Procedure

• Sources: paper_repo_pairs
• Input fields: paper text and repository text pair
• Target fields: binary alignment label
• Train/val/test split: [0.9, 0.1, 0.0]
• Max samples: 1000
• Eval split: 0.1
• Eval max samples: 256
• Batch size: 8
• Precision: bf16
• Objective: contrastive
• Contrastive loss: auto
• Learning rate: 5e-05
• Max source tokens: 256
• Max target tokens: 256
• Fine-tune strategy: full_finetune
• Max steps: 1000
• Best checkpoint metric: eval_roc_auc

Compute

• Hardware: 4x RTX_3090 (24 GB)
• Distributed strategy: ddp
• Estimated GPU hours in config: 0

Evaluation

Declared metrics: recall_at_10, ndcg_at_10

Tracked v2 eval metrics on the current held-out split:

• eval_loss: 0.4323210120201111
• eval_pair_accuracy: 0.4838709677419355
• eval_positive_score_mean: 0.42978216651827095
• eval_negative_score_mean: 0.3917496839421801
• eval_score_margin_mean: 0.03803248257609082
• eval_roc_auc: 0.5416666666666666
• eval_best_pair_accuracy: 0.5967741935483871
• eval_best_threshold: 0.2927139103412628

Status: this card reflects the current tracked experiment configuration and packaged weights in the Repository Library model stack. The v2 checkpoint loads from the trainer-selected best checkpoint. Current quality is modest and indicates that further pair curation, stronger positives, and cleaner hard negatives are still needed.

Usage

import torch
from transformers import AutoModel, AutoTokenizer

repo_id = "PeytonT/repo-paper-alignment"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModel.from_pretrained(repo_id)
model.eval()

def mean_pool(last_hidden_state, attention_mask):
    mask = attention_mask.unsqueeze(-1).type_as(last_hidden_state)
    return (last_hidden_state * mask).sum(dim=1) / mask.sum(dim=1).clamp_min(1.0)

def encode(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256, padding=True)
    with torch.no_grad():
        outputs = model(**inputs)
    return torch.nn.functional.normalize(mean_pool(outputs.last_hidden_state, inputs["attention_mask"]), dim=-1)

paper = "PAPER: We introduce a contrastive method for aligning scientific methods to implementation spans."
repo = "REPOSITORY: This module builds pairwise paper-code alignment examples and trains a contrastive encoder."
score = torch.nn.functional.cosine_similarity(encode(paper), encode(repo)).item()
print(score)

Limitations

• These cards are generated from tracked experiment metadata and packaged artifacts, not from a separate benchmark report or external audit.
• Several training sources are pipeline outputs from the Repository Library codebase rather than standalone public datasets.
• Current v2 alignment quality is only modest (eval_roc_auc around 0.542), so this checkpoint should be treated as a research component and not a final alignment benchmark.
• The model depends heavily on the quality of paper/repository pair construction and negative sampling.
• These models are components of a larger research system and should be validated in their target workflow before deployment.

Project Context

• GitHub repository: https://github.com/peytontolbert/research_library
• Model collection: https://huggingface.co/collections/PeytonT/research-library-6a49c589ef4d763f7539b50d
• Publisher: PeytonT