ReLoDer_v2

Qwen Embedding Inverter

Reconstructs natural-language text from a sentence embedding alone — no access to the original text at inference time.

A set of per-position MLPs maps a single 1024-dim sentence embedding (from Qwen3-Embedding-0.6B) into a sequence of soft prefix tokens, which condition a LoRA-adapted Qwen3-0.6B decoder to regenerate semantically equivalent text.

Code: Semantic-Embedding-Reconstruction

Training data

Trained on passages from microsoft/ms_marco (v1.1 / v2.1), pre-encoded into sentence and token embeddings and published as jg-eno/msmarco-v5.1-Qwen-Embeddings.

Intended use & limitations

• Built as a research artifact for studying how much information a single dense sentence embedding retains, and whether that information is recoverable as text — relevant to embedding-inversion / privacy-leakage research on embedding-based retrieval systems.
• Only tested on short MS MARCO-style passages (≤128 tokens). Reconstruction quality on out-of-domain or much longer text is unverified.
• This is not a general-purpose text generator. The decoder only produces coherent output when conditioned on a valid prefix from the paired MLPs — it is not meant to be used as a standalone causal LM.
• Reconstructions approximate the semantic content of the original text; they are not guaranteed to recover exact wording.