v5_Embedding_0.5B

YAR.INK v5_Embedding: The First Native Hyperbolic Text Model

Inspired by the technical excellence of the Qwen3-embedding series, we introduce v5_Embedding—the world's first native hyperbolic text embedding model. v5_Embedding serves as a universal semantic engine, empirically demonstrating that non-Euclidean geometries—specifically Lobachevsky, Lorentz, and Klein manifolds—provide a fundamentally more expressive representational space for hierarchical textual data than traditional Euclidean geometry.

Developed through technical synthesis and collaborative exchange with experts from organizations including Google, Alibaba, Baidu, and Apple, this project represents a breakthrough for the open-source community. It proves that independent research can drive fundamental architectural innovations rather than merely following established industry paradigms.

v5_Embedding establishes a new frontier for researchers and engineers globally, enabling superior retrieval performance with significantly reduced computational overhead and latency. We envision v5_Embedding as a catalyst for a new industry standard. Combined with HyperspaceDB, it empowers the democratization of hyper-efficient AI—from next-generation chatbots and autonomous robotics to advanced research laboratories.

YAR.INK v5_Embedding is a state-of-the-art embedding model trained natively into Hyperbolic (Lorentz) space utilizing a custom Matryoshka Representation Learning (MRL) head.

It is the first text embedding model designed from the ground up for highly precise context retrieval, clustering, and structural knowledge discovery in massive datasets while operating in non-Euclidean space.

🔥 Key Breakthroughs

Hyperbolic geometry naturally models hierarchical data (like language taxonomies and knowledge bases) exponentially better than Euclidean space. By combining this with Matryoshka configurations, our model achieves unparalleled efficiency:

• Over 60% Less RAM Consumption: Operates efficiently on ~642MB of RAM (Total Footprint) for v5_Embedding 64D, compared to 2553MB for high-performance Qwen3-4B baselines.
• 40x to 640x Storage Efficiency: Massive reduction in vector database footprint (from 5600KB down to 8.75KB per batch depending on the chosen Matryoshka dimension).
• Superior Quality/Size Ratio: 16D Lorentz retains 97.2% of Qwen3-4B (2560D) quality while being 160x smaller.

📊 Performance vs Efficiency Benchmark (Lorentz vs Qwen3 Baselines)

💡 Key Findings

• Extreme Compression: 160x smaller vector (16-dim Lorentz vs 2560-dim Qwen3-4B Euclidean).
• High Retention: v5_Embedding 16D retains 97.2% of Qwen3-4B recall quality with massive resource savings.
• Scaling Laws: Unlike Euclidean MRL, Lorentz embeddings maintain superior separation integrity even at ultra-low (4D-8D) dimensions.

🧠 Architecture & Compatibility

• Context Window: 512 tokens. While the architecture technically supports larger contexts, this model is specifically distilled and optimized for the 512-token limit typical of high-performance retrieval tasks.
• Tokenizer: Leverages the industry-standard Qwen2Tokenizer (BPE). This ensures that YAR.INK v5_Embedding is ready to use with any standard library (Hugging Face, vLLM, LangChain) without extra configuration, while benefiting from one of the most efficient sub-word tokenization algorithms available.

🚀 Usage

You must use trust_remote_code=True because this model relies on custom architecture (YarEmbeddingModel, YarConfig) provided directly inside this repository!

1. Generating Embeddings

import torch
from transformers import AutoTokenizer, AutoModel

model_id = "YARlabs/v5_Embedding" 
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
model.eval()

texts = [
    "What is the capital of France?",
    "Paris is the capital of France.",
    "Berlin is the capital of Germany."
]

inputs = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")

with torch.no_grad():
    # Pass target_dim parameter to explicitly slice the Matryoshka dimensions 
    # Valid options: 4, 8, 16, 32, 64, 128
    # The output is a tensor of shape (batch, target_dim + 1) -> (t, spatial_dims)
    lorentz_vectors = model(**inputs, target_dim=64)
    
print(lorentz_vectors.shape)
# Output: torch.Size([2, 65])  (1 time dimension + 64 spatial dimensions)

2. Distance Calculation (Crucial)

For vector search, clustering, NEVER use Cosine Similarity or Euclidean L2 distance! Vectors reside on a Hyperboloid, so you must use the Lorentz Distance.

def lorentz_dist(u: torch.Tensor, v: torch.Tensor) -> torch.Tensor:
    """
    Computes the exact Hyperbolic distance between two batches of Lorentz vectors.
    """
    # Lorentz Metric signature (- + + ...)
    u_0, u_x = u[..., 0:1], u[..., 1:]
    v_0, v_x = v[..., 0:1], v[..., 1:]
    
    # Minkowski inner product
    inner_product = -u_0 * v_0 + (u_x * v_x).sum(dim=-1, keepdim=True)
    
    # Avoid numerical instability inside acosh for extremely close vectors
    inner_product = torch.min(inner_product, torch.tensor(-1.0, device=u.device))
    return torch.acosh(-inner_product).squeeze(-1)

# Calculate distance between text 1 and text 2
distance = lorentz_dist(lorentz_vectors[0], lorentz_vectors[1])
print(f"Hyperbolic Distance: {distance.item():.4f}")

📐 Mathematical Integrity & Precision

Lorentz embeddings represent points on the unit hyperboloid. Every generated vector $(t, x_1, \dots, x_n)$ belongs to the Minkowski manifold and satisfies the following constraint: $$-t^2 + \sum_{i=1}^n x_i^2 = -1, \quad t > 0$$

• Crucial: Do NOT store in float16: Hyperbolic geometry is exponentially more sensitive to numerical drift than Euclidean space. Storing these embeddings in float16 will break the manifold constraint and significantly degrade retrieval accuracy.
• Recommended Format: Use float32 for production storage. If you are using high-performance, strict geometric engines like HyperspaceDB, we recommend using float64 during the indexing phase to ensure zero-drift for $10^{-6}$ precision checks.
• Geometric Projection: If your vectors have drifted due to transmission noise or quantization, you can re-project them back to the manifold by fixing the spatial components $(x)$ and re-calculating $t = \sqrt{1 + |x|^2}$.

🛡️ Intended Use Cases

1. Nex-Gen Vector Search: Leverage HyperspaceDB to build the world's most efficient semantic search engines. Achieve 160x data compression without sacrificing the "Large Model" quality, enabling billions-scale search on mid-range hardware.
2. Infinite Hierarchy Explorer: Map entire global taxonomies, corporate knowledge bases, or scientific ontologies natively. Lorentz space allows you to represent deep tree-like structures with zero distortion, which is mathematically impossible in Euclidean space.
3. Edge-AI & Satellite RAG: Deploy state-of-the-art retrieval systems on hardware with extreme constraints (IoT, mobile, orbiting stations). Use 4D-16D vectors to reduce bandwidth and storage while maintaining >90% recall.
4. Latent Knowledge Graph Discovery: Manifest hidden structural relationships in unstructured text. Automatically group concepts based on hyper-latent hierarchies for deep analytical insights into complex datasets.
5. Privacy-Driven Embeddings: Perform high-quality retrieval with ultra-low dimensions (4D-8D), making reverse-engineering of original content exponentially harder while retaining the semantic core of the data.

🔗 LangChain Integration

We provide a langchain_wrapper.py in the repository that natively subclasses LangChain's Embeddings interface.

from langchain_wrapper import YarHyperbolicEmbeddings

# Initialize the embedding model (downloads automatically from YARlabs/v5_Embedding_0.5B)
embeddings = YarHyperbolicEmbeddings(target_dim=128)

vectors = embeddings.embed_documents(["Hello World!"])

Note: Ensure your VectorStore supports custom distance metrics, as these will be returned as Lorentz vectors, where Cosine similarity will not work properly!

License

Provided explicitly for YAR.INK infrastructure.