jina-embeddings-v5-omni-nano

jina-embeddings-v5-omni-nano on AXERA NPU

Ready-to-run deployment package for jinaai/jina-embeddings-v5-omni-nano on AX650 / NPU3.

• Runtime: bundled axllm service with an OpenAI-compatible /v1/embeddings API
• Inputs: text and images through the packaged API and demo; fixed-profile image and audio encoder models are included
• Output: one L2-normalized 768-dimensional embedding per request
• Tasks: retrieval, clustering, classification, and text matching, selected at runtime without loading four complete text backbones
• Included assets: compiled .axmodel files, tokenizer data, packed task adapters, sample images, validation results, and a browser demo

Supported Platform

• AX650 / NPU3
• Single NPU

Performance

All measurements below were taken on one AX650 NPU with the files in this package. Standalone .axmodel latency uses 3 warm-up runs followed by 20 measured runs. API latency includes tokenization, adapter selection, model execution, and local HTTP overhead.

End-to-End Text Embedding and Task Switching

The model encodes each input in one forward pass. It has a prefill stage but no autoregressive decode stage.

Standalone Module Latency

Standalone results were measured with ax_run_model --warmup=3 --repeat=20; the text layer models additionally use --group=1. The recorded averages are available in validation/ax_run_model_average.json. Task-switch samples are available in validation/task_switch_client_latency.json and validation/task_switch_internal_latency.json.

Runtime Memory

CMM used is the increase in allocated CMM between the stopped-service baseline and the measured running state on the same board. PSS is the proportional set size reported for the axllm process after the smoke test. These are consumed amounts and do not depend on the board's installed CMM capacity.

The audio encoder is initialized lazily and was not loaded in this text smoke-test state. The measurement record is available in validation/runtime_memory.json.

Package Size

The four tasks share the same text backbone. Package sizes describe files on storage and are independent of the runtime CMM and OS-memory measurements above.

Functional Validation

Machine-readable results are stored under validation/.

Package Layout

.
├── README.md
├── config.json
├── start_axllm.sh
├── bin/axllm
├── jina_v5_omni_tokenizer.txt
├── tokenizer.json
├── tokenizer_config.json
├── model.embed_tokens.weight.bfloat16.bin
├── jina_embeddings_v5_omni_p128_l0_together.axmodel
├── ...
├── jina_embeddings_v5_omni_p128_l11_together.axmodel
├── jina_embeddings_v5_omni_post.axmodel
├── jina_v5_omni_nano_vision_tower_256x256.axmodel
├── jina_v5_omni_nano_vision_merger_<retrieval|clustering>_256x256.axmodel
├── jina_v5_omni_nano_audio_tower_8s.axmodel
├── jina_v5_omni_nano_audio_projector_<retrieval|clustering>_8s.axmodel
├── lora/{retrieval,clustering,classification,text-matching}/
├── scripts/test_retrieval_clustering.py
├── demo/
├── assets/
├── validation/
└── SHA256SUMS

The root directory is the axllm model directory. Keep the relative paths unchanged because config.json resolves the text, media, and adapter files from this layout.

Download the Package

mkdir -p AXERA-TECH/jina-embeddings-v5-omni-nano
cd AXERA-TECH/jina-embeddings-v5-omni-nano
hf download AXERA-TECH/jina-embeddings-v5-omni-nano --local-dir .

Copy the downloaded directory to the AX650 board before continuing.

Run on the Board

From the package root, start the bundled service on port 18201:

bash ./start_axllm.sh 18201

In another terminal, verify the service and model id:

curl http://127.0.0.1:18201/health
curl http://127.0.0.1:18201/v1/models

A healthy service returns:

{
  "concurrency": 0,
  "max_concurrency": 1,
  "status": "healthy"
}

The model list contains AXERA-TECH/jina-embeddings-v5-omni-nano.

Select a Task Adapter

Set task_id in each embedding request:

Adapter files are validated during model initialization. A task switch then loads the selected matrices without repeating the full file validation.

OpenAI-Compatible Text Request

curl http://127.0.0.1:18201/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/jina-embeddings-v5-omni-nano",
    "task_id": "retrieval",
    "input_type": "query",
    "input": "a photo of a cat"
  }'

The embedding is returned in data[0].embedding. It contains 768 floating-point values with an L2 norm of approximately 1.0. For retrieval, use input_type: query for the search query and input_type: document for indexed text, images, or audio.

The packaged smoke test verifies retrieval ranking, clustering separation, and a bit-exact retrieval -> clustering -> retrieval round trip:

python3 scripts/test_retrieval_clustering.py \
  --url http://127.0.0.1:18201/v1/embeddings

Browser Demo

Keep axllm running and start the packaged web application in another terminal:

python3 demo/openai_task_switch_web.py \
  --axllm-url http://127.0.0.1:18201/v1/embeddings \
  --host 0.0.0.0 \
  --port 8080

The web process checks the embedding service before it starts. Open http://<board-ip>:8080 to search the 12 packaged animal images with natural language. The first search creates an in-memory image index; later searches reuse the cached image embeddings and compute only the new text embedding.

Open http://<board-ip>:8080/index.html for the complete capability validation. It exercises all four tasks and then checks a bit-exact switch back to retrieval.

For the query 找出所有猫的图像, all three cat images are ranked first:

<table> <tr> <td align="center"><img src="assets/cat_0.jpeg" alt="cat_0.jpeg" width="220"><br><b>1. cat_0.jpeg</b><br>cosine 0.405399</td> <td align="center"><img src="assets/cat_2.jpg" alt="cat_2.jpg" width="220"><br><b>2. cat_2.jpg</b><br>cosine 0.403438</td> <td align="center"><img src="assets/cat_1.jpeg" alt="cat_1.jpeg" width="220"><br><b>3. cat_1.jpeg</b><br>cosine 0.344883</td> </tr> </table>

The complete log below was captured from the packaged demo on AX650.

<details> <summary>Complete AX650 web-demo log</summary>

Jina Embeddings v5 Omni - OpenAI API 多任务切换 Demo
Model: AXERA-TECH/jina-embeddings-v5-omni-nano
Assets: cat=3, dog=3, fox=3, rabbit=3

--- retrieval：文本查询召回动物图片 ---

=== task switch: <startup> -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=   57.00 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  316.37 ms dim=768 norm=1.000000
[request] task=retrieval     input=image:cat_1.jpeg   latency=  369.30 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:cat_2.jpg    latency=  253.63 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:dog_0.jpeg   latency=  143.66 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:dog_1.jpg    latency=  165.81 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:dog_2.jpeg   latency=  586.12 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:fox_0.jpeg   latency=  164.39 ms dim=768 norm=1.000000
[request] task=retrieval     input=image:fox_1.jpeg   latency=  117.97 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:fox_2.jpeg   latency=  129.73 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:rabbit_0.jpeg latency=  122.36 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:rabbit_1.jpeg latency=  105.08 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:rabbit_2.jpeg latency=  105.57 ms dim=768 norm=1.000000
[result] retrieval ranking:
   1. cat_2.jpg         cosine=0.481803
   2. cat_0.jpeg        cosine=0.471761
   3. cat_1.jpeg        cosine=0.437954
   4. dog_2.jpeg        cosine=0.334749
   5. dog_0.jpeg        cosine=0.327777
   6. dog_1.jpg         cosine=0.304540
   7. rabbit_1.jpeg     cosine=0.271320
   8. rabbit_0.jpeg     cosine=0.255625
   9. fox_0.jpeg        cosine=0.238203
  10. fox_2.jpeg        cosine=0.232027
  11. rabbit_2.jpeg     cosine=0.225383
  12. fox_1.jpeg        cosine=0.200564
[check] cat top-1: PASS

--- clustering：同类与跨类图片聚类 ---

=== task switch: retrieval -> clustering ===
[request] task=clustering    input=image:cat_0.jpeg   latency=  553.46 ms dim=768 norm=1.000001
[request] task=clustering    input=image:cat_1.jpeg   latency=  371.34 ms dim=768 norm=1.000001
[request] task=clustering    input=image:cat_2.jpg    latency=  256.51 ms dim=768 norm=1.000000
[request] task=clustering    input=image:dog_0.jpeg   latency=  143.50 ms dim=768 norm=1.000000
[request] task=clustering    input=image:dog_1.jpg    latency=  165.48 ms dim=768 norm=1.000001
[request] task=clustering    input=image:dog_2.jpeg   latency=  588.71 ms dim=768 norm=1.000000
[request] task=clustering    input=image:fox_0.jpeg   latency=  163.32 ms dim=768 norm=1.000001
[request] task=clustering    input=image:fox_1.jpeg   latency=  117.79 ms dim=768 norm=1.000000
[request] task=clustering    input=image:fox_2.jpeg   latency=  129.84 ms dim=768 norm=1.000000
[request] task=clustering    input=image:rabbit_0.jpeg latency=  122.27 ms dim=768 norm=1.000001
[request] task=clustering    input=image:rabbit_1.jpeg latency=  104.95 ms dim=768 norm=1.000001
[request] task=clustering    input=image:rabbit_2.jpeg latency=  105.37 ms dim=768 norm=1.000001
[result] within-class average cosine=0.918765
[result] cross-class  average cosine=0.792831
[check] clustering separation: PASS

--- classification：动物文本分类特征分离 ---

=== task switch: clustering -> classification ===
[request] task=classification input=text:A cat sleeps  latency=  271.33 ms dim=768 norm=1.000001
[request] task=classification input=text:A kitten res  latency=   56.57 ms dim=768 norm=1.000001
[request] task=classification input=text:A dog runs i  latency=   61.85 ms dim=768 norm=1.000001
[request] task=classification input=text:A puppy play  latency=   56.83 ms dim=768 norm=1.000001
[request] task=classification input=text:A red fox wa  latency=   68.65 ms dim=768 norm=1.000001
[request] task=classification input=text:A wild fox h  latency=   68.58 ms dim=768 norm=1.000001
[request] task=classification input=text:A rabbit eat  latency=   62.60 ms dim=768 norm=1.000001
[request] task=classification input=text:A bunny sits  latency=   63.04 ms dim=768 norm=1.000001
[result] cat    same-class cosine=0.814281
[result] dog    same-class cosine=0.820817
[result] fox    same-class cosine=0.828818
[result] rabbit same-class cosine=0.852505
[result] same-class average cosine=0.829105
[result] cross-class average cosine=0.776887
[check] classification separation: PASS

--- text-matching：动物文本匹配 ---

=== task switch: classification -> text-matching ===
[request] task=text-matching input=text:A red fox st  latency=  272.79 ms dim=768 norm=1.000000
[request] task=text-matching input=text:A domestic c  latency=   62.90 ms dim=768 norm=1.000001
[request] task=text-matching input=text:A pet dog ru  latency=   59.69 ms dim=768 norm=1.000001
[request] task=text-matching input=text:A wild red f  latency=   56.68 ms dim=768 norm=1.000001
[request] task=text-matching input=text:A small rabb  latency=   57.44 ms dim=768 norm=1.000001
[result] text-matching ranking:
  1. fox    cosine=0.900007
  2. rabbit cosine=0.600154
  3. dog    cosine=0.518257
  4. cat    cosine=0.441050
[check] fox top-1: PASS

--- round-trip：切回 retrieval ---

=== task switch: text-matching -> retrieval ===
[request] task=retrieval     input=text:a photo of a  latency=   97.80 ms dim=768 norm=1.000001
[request] task=retrieval     input=image:cat_0.jpeg   latency=  306.87 ms dim=768 norm=1.000000
[result] retrieval query max_abs_diff=0.0000000000
[result] retrieval image max_abs_diff=0.0000000000
[check] retrieval round-trip: PASS

=== summary ===
retrieval       PASS
clustering      PASS
classification  PASS
text-matching   PASS
round-trip      PASS

</details>

Fixed Input Profiles

The final embedding output is always [1, 768] and L2 normalized.

The vision and audio task-specific mapper files are selected together with the active adapter. Image resolutions or audio lengths outside these profiles require rebuilding the corresponding media modules.

Adapter selection is mutable state within one service instance. Serialize requests that change task_id, or run separate service instances when requests must be isolated by task. The first switch may include filesystem cache overhead; use the warm measurements above for steady-state planning.

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

• Original Hugging Face model: jinaai/jina-embeddings-v5-omni-nano
• AXERA conversion and deployment workflow: AXERA-TECH/jina_embeddings_v5_omni.axera

The public conversion workflow requires Pulsar2 7.0 or later. Pulsar2 is needed only to rebuild the deployment artifacts, not to run this downloaded package.

Discussion

• GitHub Issues
• QQ group: 139953715