Qwen3-Embedding-0.6B-onnx

Qwen3 Embedding 0.6B ONNX

This repository contains an ONNX export of Qwen/Qwen3-Embedding-0.6B, converted by Qdrant for inference with FastEmbed.

Model Description

This is a dense text embedding model built on the Qwen3 decoder architecture.

It encodes text into 1024-dimensional vectors, supports over 100 languages, and accepts up to 32768 input tokens. Embeddings are produced by taking the hidden state of the last non-padding token and L2-normalizing it, so the similarity score is the dot product of two embeddings.

Pooling and normalization are not part of the ONNX graph. FastEmbed applies both after inference.

Files

The quantized file stores its weights as 8-bit MatMulNBits and therefore requires onnxruntime>=1.23. Earlier versions only implement the 4-bit kernel and will fail to load it.

Usage

This ONNX model is designed for use with FastEmbed.

from fastembed import TextEmbedding

model = TextEmbedding(model_name="Qwen/Qwen3-Embedding-0.6B")

instruction = (
    "Instruct: Given a web search query, retrieve relevant passages that answer the query\n"
    "Query:"
)
queries = ["What is the capital of China?"]
documents = ["The capital of China is Beijing."]

query_embeddings = list(model.query_embed([instruction + query for query in queries]))
document_embeddings = list(model.embed(documents))

Use query_embed() for queries and embed() for documents.

Queries should carry a task instruction, documents should not. FastEmbed does not add the instruction for you, so prepend it as shown above. Task descriptions can be written for the task at hand; the original model card recommends one instruction per retrieval task and notes that omitting it costs roughly 1% to 5% of retrieval quality.

Pass model_name="Qwen/Qwen3-Embedding-0.6B-Q" to use the quantized weights instead.

The model supports Matryoshka embeddings between 32 and 1024 dimensions. This export always returns 1024 dimensions; to use a shorter vector, truncate it and re-normalize.

Accuracy

Measured against SentenceTransformer("Qwen/Qwen3-Embedding-0.6B") in fp32 on a small sample of queries and documents:

The fp32 export also reproduces the query/document score matrix from the original model card exactly.

Only the weights are quantized in the int8 build.

For benchmark results such as MTEB, refer to the original model card.

Conversion

The int8 build applies weight-only quantization through onnxruntime's MatMulNBitsQuantizer with a block size of 32.

Compared to the upstream checkpoint, tokenizer_config.json sets model_max_length to 32768, matching the model's own maximum position embeddings, and config.json sets pad_token_id.

License

This project is licensed under the Apache v2.0 License.

Acknowledgments

The original model was developed by the Qwen team at Alibaba Cloud. The ONNX conversion was performed by Qdrant for use with FastEmbed.