intelli-embed-v3

intelli-embed-v3

Custom embedding model: 4-phase multi-signal fine-tune of Snowflake/snowflake-arctic-embed-l-v2.0 (XLM-RoBERTa-large, 1024-dim).

First local model to surpass Azure text-embedding-3-large (sep=0.520 GPU fp16 vs 0.515). Beats azure-large on 5/6 OpenMemory-specific metrics. CPU INT8 ONNX reaches sep=0.505 (98% of azure-large) at 11 ms latency.

Built for OpenMemory — optimized for personal memory storage, retrieval, deduplication, negation detection, and entity search.

Model Details

Full Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True})
  (2): Normalize()
)

Benchmark Results

Benchmarked on 10 SW-engineering positive pairs + 10 negative pairs. Sep = mean(PosSim) − mean(NegSim).

Extended OpenMemory Metrics (vs azure-large)

v3 wins 5/6 metrics. Only supSim is lower (both below target zone — requires threshold tuning).

Held-Out Validation

Tested on completely fresh sentences with zero overlap to training data:

Scores improve on held-out data, confirming no benchmark inflation.

MTEB Benchmark (English, v2)

Full evaluation on MTEB(eng, v2) — 41 tasks across 7 categories. Evaluated with mteb v2.8.8 on NVIDIA RTX 4090, FP16, batch_size=256.

<details> <summary>Individual Task Scores (click to expand)</summary> </details>

Training Strategy

4-phase multi-signal fine-tuning with complementary objectives:

Training Hyperparameters

• Hardware: NVIDIA RTX 4090 (24 GB VRAM)
• Training time: ~2.1 hours total (all 4 phases)
• Batch size: 32 (phases A, C, D), 128 (phase B)
• Learning rate: 2e-5 (phase A), 1e-5 (phase B), 5e-6 (phases C, D)
• Scheduler: Cosine with 5–10% warmup
• Precision: bf16 mixed precision
• Optimizer: AdamW (fused)
• Gradient checkpointing: enabled

Framework Versions

• Python: 3.10.6
• Sentence Transformers: 5.2.3
• Transformers: 4.57.6
• PyTorch: 2.6.0+cu124
• ONNX Runtime: 1.22.0 (INT8 quantization)

Usage

Python (sentence-transformers)

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("serhiiseletskyi/intelli-embed-v3")

queries = model.encode(["What programming language does the user prefer?"])
docs = model.encode(["The user is a senior TypeScript developer who loves React."])

similarity = model.similarity(queries, docs)
print(similarity)  # tensor([[0.62]])

Node.js (transformers.js — recommended for production)

import { pipeline } from "@huggingface/transformers";

const extractor = await pipeline(
  "feature-extraction",
  "serhiiseletskyi/intelli-embed-v3",
  { dtype: "q8" }  // loads onnx/model_quantized.onnx (542 MB)
);

const output = await extractor("User prefers dark mode", {
  pooling: "cls",
  normalize: true,
});
console.log(output.data.length);  // 1024

Node.js (onnxruntime-node — direct ONNX)

import * as ort from "onnxruntime-node";

const session = await ort.InferenceSession.create(
  "onnx/model_quantized.onnx",
  { executionProviders: ["cpu"] }
);
// Tokenize with @huggingface/transformers AutoTokenizer, then:
const result = await session.run({ input_ids, attention_mask });
// CLS pooling: result.last_hidden_state[0, 0, :1024], then L2-normalize

GPU Server (CUDA fp16)

pip install sentence-transformers flask
python v3_gpu_server.py --port 8234
# OpenAI-compatible /v1/embeddings endpoint
curl http://localhost:8234/v1/embeddings \
  -d '{"input": "hello world", "model": "intelli-embed-v3-gpu"}'

Files

Citation

If you use this model, please cite:

@misc{intelli-embed-v3,
  title={intelli-embed-v3: Multi-Signal Fine-Tuned Embedding Model for Personal Memory Systems},
  author={Serhii Seletskyi},
  year={2026},
  url={https://huggingface.co/serhiiseletskyi/intelli-embed-v3},
  note={4-phase fine-tune of Snowflake/snowflake-arctic-embed-l-v2.0 for OpenMemory}
}

License

Apache 2.0 (same as base model Snowflake/snowflake-arctic-embed-l-v2.0)