CARL-CCI

CARLCCI

Code repository for the paper Larger Is Not Always Better: Leveraging Structured Code Diffs for Comment Inconsistency Detection

Overview

CARCCI consists of two main components:

1. Diff Representation Formulator:
   Uses difflib along with additional heuristics to transform raw code changes into structured diff segments, enabling the model to better capture semantic modifications.

2. Activity-Aware Representation Learning for CCI:
   A CodeT5p-based encoder that learns joint representations of code, comments, and structured diffs to detect code–comment inconsistency.

Dataset

This project uses two datasets: CCIBench and JITData.
Each dataset contains three folders — Return, Summary, and Param — which store the original unprocessed data.

The script extractStructuredDiff.py serves as the diff constructor.
It reads data from these three folders, computes structured code diffs, and produces the final JSON dataset used for training and evaluation.

Model

CARCCI provides two model types, as defined in the configuration:

1. Full Model (carlcci_model)
   The complete three-stream architecture that encodes:

   • the updated code,
   • the original comment,
   • and the structured diff–augmented combo input.

2. Ablation Model (carlcci_ablation_no_combo)
   A reduced two-stream variant that removes the diff-based combo input and uses multi-head attention between code and comment representations.

Usage Example

With the default configuration, the entire CARCCI pipeline can be executed end-to-end as follows:

1. Construct structured diffs

Generate structured diff–annotated data from the original dataset folders (Return, Summary, Param) in CCIBench or JITData:

python extractStructuredDiff.py

Train the model

This will automatically load the constructed JSON dataset, initialize the full CARCCI model, and begin training with the default settings in config.py:

python train.py

Replication: Ablation Setting

To replicate the ablation results, run training with the following configuration:

python train.py \
    --model_type carlcci_ablation_no_combo \
    --alpha_neg_push 1 \
    --warmup_epochs_contrast 2 \
    --original_positive_loss \
    --contrast_temp 0.09