harrier-oss-v1-270m-GGUF

Harrier OSS v1 270M — GGUF

Tested and converted using llama.cpp b10054+

GGUF conversion of microsoft/harrier-oss-v1-270m for use with llama.cpp.

Microsoft Harrier OSS v1 270M is a compact multilingual text embedding model built on the Gemma3 architecture, producing 640-dimensional embeddings using last-token pooling and L2 normalization.

Files

Usage

llama-server --embeddings -m harrier-270m-q8_0.gguf --port 8080

Then query the /embedding endpoint as with any llama.cpp embedding model.

⚠️ Found conversion issues and my fixes

As of conversion time (July 2026), the stock convert_hf_to_gguf.py from llama.cpp could not convert this model out of the box. If you hit the same errors converting similar Gemma3-based embedding models, here's how they were resolved.

1. NotImplementedError: BPE pre-tokenizer was not recognized

chkhsh: fc65e8f033752f900de442edbaa0e385712ff5f6f1d2fd8d958fa6ec59daa92f

This tokenizer's hash wasn't yet in the list of known pre-tokenizers hardcoded in get_vocab_base_pre(). Fix: add an entry for it (in convert_hf_to_gguf.py / conversion/base.py, depending on your llama.cpp version):

if chkhsh == "fc65e8f033752f900de442edbaa0e385712ff5f6f1d2fd8d958fa6ec59daa92f":
    # ref: microsoft/harrier-oss-v1 (gemma3-based BPE tokenizer)
    res = "llama-bpe"

2. AssertionError on assert max(tokenizer.vocab.values()) < vocab_size

The tokenizer declares an id exactly 1 higher than the number of rows actually present in token_embd.weight. Do not raise vocab_size in config.json to work around this — doing so desyncs the GGUF metadata (n_vocab) from the actual embedding matrix row count, and will fail to load with:

check_tensor_dims: tensor 'token_embd.weight' has wrong shape; expected 640, 262145, got 640, 262144, 1, 1

Instead, drop the extra id and keep the original vocab_size. Patch for get_vocab_base():

def get_vocab_base(self) -> tuple[list[str], list[int], str]:
    tokens: list[str] = []
    toktypes: list[int] = []

    from transformers import AutoTokenizer
    tokenizer = AutoTokenizer.from_pretrained(self.dir_model)
    vocab_size = self.hparams.get("vocab_size", len(tokenizer.vocab))

    extra_ids = {tid: tok for tok, tid in tokenizer.vocab.items() if tid >= vocab_size}
    if extra_ids:
        logger.warning(
            f"tokenizer has {len(extra_ids)} id(s) >= vocab_size ({vocab_size}) with no embedding row, "
            f"dropping them: {extra_ids}"
        )

    tokpre = self.get_vocab_base_pre(tokenizer)

    reverse_vocab = {id_: encoded_tok for encoded_tok, id_ in tokenizer.vocab.items() if id_ < vocab_size}
    added_vocab = tokenizer.get_added_vocab()
    added_tokens_decoder = tokenizer.added_tokens_decoder

    for i in range(vocab_size):
        if i not in reverse_vocab:
            tokens.append(f"[PAD{i}]")
            toktypes.append(gguf.TokenType.UNUSED)
        else:
            token: str = reverse_vocab[i]
            if token in added_vocab:
                if not added_tokens_decoder[i].normalized:
                    previous_token = token
                    token = tokenizer.decode(tokenizer.encode(token, add_special_tokens=False))
                    if previous_token != token:
                        logger.info(f"{repr(previous_token)} is encoded and decoded back to {repr(token)} using AutoTokenizer")
                if added_tokens_decoder[i].special or self.does_token_look_special(token):
                    toktypes.append(gguf.TokenType.CONTROL)
                else:
                    token = token.replace(b"\xe2\x96\x81".decode("utf-8"), " ")
                    toktypes.append(gguf.TokenType.USER_DEFINED)
            else:
                toktypes.append(gguf.TokenType.NORMAL)
            tokens.append(token)

    return tokens, toktypes, tokpre

Quality check

Not formally benchmarked against the original PyTorch model — feel free to open a discussion if you run comparisons.

License

MIT, inherited from microsoft/harrier-oss-v1-270m.