clip-vit-large-patch14-hxq

CLIP-ViT-L/14-HXQ

3.6x smaller from FP32. CIFAR-100 Top-1 72.8%. First vision model compressed with HXQ.

CLIP ViT-Large/14 (text + vision dual encoder) compressed from 1.6 GB to 447 MB. Zero-shot classification accuracy matches the dense baseline. No calibration data. Same codec that compresses Transformers, SSMs, Hybrids, and MoEs.

Install and Run

pip install "helix-substrate[hf]"

import helix_substrate  # registers the HXQ quantizer with HuggingFace
from transformers import CLIPModel, CLIPProcessor
from PIL import Image

model = CLIPModel.from_pretrained("EchoLabs33/clip-vit-large-patch14-helix")
processor = CLIPProcessor.from_pretrained("EchoLabs33/clip-vit-large-patch14-helix")

image = Image.open("photo.jpg")
inputs = processor(
    text=["a photo of a cat", "a photo of a dog", "a photo of a car"],
    images=image,
    return_tensors="pt",
    padding=True,
)
outputs = model(**inputs)
probs = outputs.logits_per_image.softmax(dim=-1)
print(probs)  # [cat_prob, dog_prob, car_prob]

Downstream Benchmarks

Zero-shot CIFAR-100 classification (10,000 test images, 100 classes, prompt: "a photo of a {class}"):

All deltas within noise. Task performance preserved after 3.6x compression.

Compression Benchmark

Verification Status

• Compression receipt: PASS -- 218 compressed, 374 exact, mean cosine 0.9997
• Conversion receipt: PASS (Gate 1 + Gate 2)
• Downstream eval: PASS -- paired dense/HXQ on CIFAR-100 zero-shot

Good to Know

• GPU and CPU supported -- runs on any CUDA GPU or CPU.
• Fine-tunable via LoRA — compressed weights remain frozen, but LoRA adapters attach to each HelixLinear layer via HelixLinearSTE. See helix-substrate for training infrastructure.
• Requires helix-substrate -- you need pip install "helix-substrate[hf]".
• Embeddings stored exact -- token, position, and patch embeddings are at full precision. Only the 218 attention + MLP linear layers are compressed.

What is HelixCode?

HelixCode is a universal weight compression codec based on vector quantization:

• Each weight matrix is replaced by a 256-entry codebook (float32) + uint8 index matrix + optional sidecar corrections for outlier values
• The compressed form is the executable -- no decompression step
• Works on any nn.Linear regardless of architecture
• No calibration data required -- codebooks are fit from the weights alone

Architecture Details

CLIP ViT-Large/14 is a dual-encoder multimodal model:

• Vision encoder: 24-layer ViT-Large, hidden_size=1024, 16 attention heads, patch_size=14
• Text encoder: 12-layer Transformer, hidden_size=768, 12 attention heads
• Cross-modal projections: visual_projection (1024->768) + text_projection (768->768)

All 218 linear layers across both encoders are compressed. Embedding layers (token, position, patch), layer norms, and biases are stored at full precision.

Why This Matters

CLIP is the first vision model compressed with HXQ. The same codec now covers:

Five architecture families. One codec. One pip install.

Companion Models

Citation

@software{helix_substrate_2026,
  title={Helix Substrate: Universal Weight Compression via HelixCode},
  author={EchoLabs},
  year={2026},
  url={https://github.com/echo313unfolding/helix-substrate}
}

License

Apache 2.0 (inherited from openai/clip-vit-large-patch14).