directional-embedder-v0.3

dirlora-v3

dirlora-v3 is a LoRA-adapted model checkpoint built for concise, instruction-style text embedding with aware of previous text and following text.

Model details

• Model type: Causal Language Model (LoRA adapter)
• Base model: Qwen/Qwen3-Embedding-0.6B
• Fine-tuning method: LoRA / PEFT
• LoRA config:
  • Rank (r): 16
  • Alpha (alpha): {{alpha}}
  • Dropout: {{lora_dropout}}
• Training data: subset of wikitext 103 for PoC
• Framework: PyTorch + Hugging Face transformers + peft

Intended use

Use this model for:

• text embedding
• experimenting direction/sequence aware embedding

Not intended for

• any engineering solution

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("npc0/Qwen3-Embedding-0.6B-OED")

sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium."
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)

Limitations

• This is early stage PoC not converged but made progress comparing to previous versions

Citation

If you use this model, please cite:

@misc{npc0directionalembedderv0.3,
      title={Directional Embedder v0.3}, 
      author={NPC0},
      year={2026},
      url={https://huggingface.co/npc0/directional-embedder-v0.3}, 
}