pulpie-orange-small-gguf

Pulpie Orange Small — GGUF

GGUF conversion of feyninc/pulpie-orange-small, a 210M parameter EuroBERT token-classification model for Pulpie HTML main-content extraction.

⚠️ Not a language model

This is an encoder classifier, not a causal LM. The GGUF files expose per-token other/main classifier logits via llama-embedding, not llama-cli generation.

Files

Verification

Note: Only F16 and Q8_0 were numerically verified against the original PyTorch model. Lower quants (Q6_K → Q2_K) passed load + inference checks but output consistency was not validated. Use F16 or Q8_0 for production.

Full results: verification_report.json

Usage

llama-embedding \
  --model pulpie-orange-small-Q8_0.gguf \
  --prompt "<html><body><p>Hello world</p></body></html><|sep|>" \
  --pooling none \
  --embd-normalize -1 \
  --embd-output-format json

Each output row is [other_logit, main_logit]. Pulpie classifies at <|sep|> token positions and keeps blocks where main_logit > other_logit.

llama-cli can load the files but cannot generate — these GGUFs have no causal LM head.

Python example

import json, re, subprocess

out = subprocess.check_output([
    "llama-embedding", "-m", "pulpie-orange-small-Q8_0.gguf",
    "-p", html_chunk,
    "--pooling", "none",
    "--embd-normalize", "-1",
    "--embd-output-format", "json",
], text=True)

data = json.loads(re.search(r'\{.*\}', out, re.S).group(0))
logits = [row["embedding"] for row in data["data"]]

Conversion notes

Stock llama.cpp supports EuroBERT embeddings but not EuroBertForTokenClassification. A 103-line patch was applied to:

• Register EuroBertForTokenClassification in the HF converter
• Map classifier.weight/classifier.bias → GGUF cls.output.*
• Write classifier labels ["other", "main"] with embedding_length_out=2
• Apply the classifier head in the EuroBERT runtime graph

Quantization was done with llama-quantize from the patched build.