Qwen3-Embedding-4B

Qwen3-Embedding-4B

This repository contains Qwen/Qwen3-Embedding-4B together with a Furiosa Executable Bundle (FXB) for running it on FuriosaAI RNGD with Furiosa-LLM. The same model also runs on other frameworks (such as Sentence Transformers, vLLM, and Transformers); for usage with those, see the upstream Qwen/Qwen3-Embedding-4B model card.

Overview

Qwen3-Embedding-4B is the 4B text-embedding model in the Qwen3-Embedding series, built on the Qwen3 dense transformer backbone. It maps text into dense vector representations for semantic search, retrieval, and similarity matching, with strong multilingual coverage. Embeddings can be requested at a reduced dimensionality through the dimensions parameter, thanks to the model's Matryoshka Representation Learning (MRL) support. Its intended use is the same as the upstream Qwen/Qwen3-Embedding-4B, and it is released under the Apache 2.0 License.

• Architecture: Qwen3 (dense)
• Input / Output: Text / Embeddings (vector)
• Supported Inference Engine: Furiosa LLM
• Supported Hardware: FuriosaAI RNGD

Quantization

No quantization — the model runs in its native BF16 precision.

Parallelism Strategy

On RNGD, Qwen3-Embedding-4B runs with a tensor-parallel size of 8 PEs, which maps to a single RNGD card (8 PEs per card).

Usage

To run this model with Furiosa-LLM, follow the examples below after installing Furiosa-LLM and its prerequisites. You can use the model either online through the OpenAI-compatible server or offline through the Furiosa-LLM Python API.

Launch the server

The simplest way to serve the model is:

# Launch the server, listening on port 8000 by default
furiosa-llm serve furiosa-ai/Qwen3-Embedding-4B

When the server is ready, you will see:

INFO:     Started server process [27507]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

Basic Usage

The server exposes an OpenAI-compatible /v1/embeddings endpoint. You can send a request with curl:

curl http://localhost:8000/v1/embeddings \
    -H "Content-Type: application/json" \
    -d '{
    "model": "furiosa-ai/Qwen3-Embedding-4B",
    "input": ["Hello, world!", "How are you?"]
    }' \
    | python -m json.tool

Because the endpoint is OpenAI-compatible, you can also use the OpenAI Python client:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.embeddings.create(
    model="furiosa-ai/Qwen3-Embedding-4B",
    input=["Hello, world!", "How are you?"],
)

for data in response.data:
    print(f"Index {data.index}: {len(data.embedding)} dimensions")

Advanced Usage

For offline use, load the model with the LLM constructor (the FXB shipped in the repo is discovered automatically) and call embed to obtain dense vectors:

from furiosa_llm import LLM

with LLM("furiosa-ai/Qwen3-Embedding-4B") as llm:
    outputs = llm.embed(["Hello, world!", "How are you?"])
    for output in outputs:
        print(f"{len(output.outputs.embedding)} dimensions")

Learn more

• Furiosa-LLM Server (furiosa-llm serve) — full OpenAI-compatible API reference, including the Embeddings API
• Furiosa-LLM — Furiosa-LLM documentation and API reference
• Qwen/Qwen3-Embedding-4B — upstream model card