AstroCLIMB-Qwen3.5-4B

AstroCLIMB: Qwen3.5-4B adapters and classification heads

By Irving Ernesto Quezada Ramírez, Independent Researcher.

Source and manuscript · Inference instructions

These are four classification adapters and heads, not standalone generative chat models. Load the pinned Qwen3.5-4B base separately: revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a.

The submitted system scored 0.75149 Kaggle public macro-F1 and 0.7562033665 pooled development macro-F1 on 1,986 pairs. These are different evaluation sets. Development was used repeatedly for selection; this is not an unbiased estimate or a rank claim. Later matched CXI controls are separate models and do not change this submission.

Each directory includes LoRA weights, head.pt, processor files, and sanitized training metadata. The four-class order is same_figure, same_paper, related_papers, unrelated_papers. The last non-padding hidden representation feeds the classification head. IXI/CXC mask same_figure. Canonical inference is batch-one BF16 with SDPA; changing backend or batch shape may change predictions.

Verification and limitations

public-manifest.json records SHA-256 hashes and original hashes. Adapter tensor files and heads are byte-identical to the preserved checkpoints. Metadata paths were normalized; the original archival files remain unchanged privately. Parent hashes in metadata refer to the original archive, not rewritten metadata in this release.

The historical probe.pt is deliberately omitted because it contains a recoverable training example. Optimizer state is also omitted. Consequently these packages are not drop-in bundles for the legacy probe-verifying exporters. Use the separately documented public inference utility. It verifies release hashes and input schemas but does not claim historical numerical replay verification. No fresh GPU run of that public utility or independent full retraining from this release has been performed. The original complete checkpoints had fresh-process replay verification during the experiments.

Training data and base-model weights are not bundled. See the source repository for data-access requirements and filtering. Earlier exploratory branches had exposure and data-quality problems; they are excluded from this model lineage. Project-level prior exposure and incomplete identity matching still limit claims of evaluation independence. Full details are in the paper and history document.

Training and use

Rank-16 LoRA (alpha 32), frozen base and vision tower, trained task-specific classification head. Public adaptation applies only to CXI. Exact prompts, class order, preprocessing, checkpoint selection and schedules are in the source repository. These models are research artifacts for relations between scientific figures/captions, not validated for other domains or decision-making uses.

Use torch.load(..., weights_only=True) for the classification head. Download the base model under its upstream terms. Derived adapters/heads are released under Apache-2.0 with upstream processor/tokenizer attribution preserved in LICENSE and NOTICE.

This research received an unrestricted US$1,000 GPU compute grant from Lium. The grant amount is not measured experiment expenditure. AI assistance with planning, coding, experiments and manuscript preparation is disclosed in the paper.