Qwen3-VL-Embedding-2B-NSFW

SentenceTransformer based on Qwen/Qwen3-VL-Embedding-2B

This is a sentence-transformers model finetuned from Qwen/Qwen3-VL-Embedding-2B on the json dataset. It maps sentences & paragraphs to a 2048-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-VL-Embedding-2B <!-- at revision 9f2f7e710d6d81056aa5c0a4f04764fec6bb7bda -->
• Maximum Sequence Length: 262144 tokens
• Output Dimensionality: 2048 dimensions
• Similarity Function: Cosine Similarity
• Supported Modalities: Text, Image, Video, Message
• Training Dataset:
  • json

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'image': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'video': {'method': 'forward', 'method_output_name': 'last_hidden_state'}, 'message': {'method': 'forward', 'method_output_name': 'last_hidden_state', 'format': 'structured'}}, 'module_output_name': 'token_embeddings', 'processing_kwargs': {'chat_template': {'add_generation_prompt': True}}, 'unpad_inputs': False, 'architecture': 'Qwen3VLModel'})
  (1): Pooling({'embedding_dimension': 2048, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'May sits with her legs spread, her massive breasts barely contained by her bikini as she grips her huge cock and thick, long penis, precum leaking out while she looks over her shoulder with a smile.',
]
documents = [
    './Images/11897115.jpg',
    './Images/12313417.jpg',
    './Images/12110929.jpg',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 2048] [3, 2048]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.3335,  0.1907, -0.0211]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Information Retrieval

• Dataset: nsfw-ir-eval
• Evaluated with <code>InformationRetrievalEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

json

• Dataset: json
• Size: 42,977 training samples
• Columns: <code>query</code>, <code>positive</code>, <code>negative0</code>, <code>negative1</code>, <code>negative2</code>, and <code>negative3</code>
• Approximate statistics based on the first 100 samples:

  	query	positive	negative0	negative1	negative2	negative3
  type	string	string	string	string	string	string
  modality	text	image	image	image	image	image
  details	<ul><li>min: 46 tokens</li><li>mean: 74.73 tokens</li><li>max: 108 tokens</li></ul>

• Samples:

  query	positive	negative0	negative1	negative2	negative3
  <code>Jane Doe kneels with her skirt hiked, her large breasts jiggling as she deepthroats a thick penis while licking the shaft, her curvaceous body pressed against the viewer’s face and her smooth thighs spread wide to show her dripping pussy and the thick tail pressed against her ass.</code>	<code>./Images/12354799.jpg</code>	<code>./Images/12371705.jpg</code>	<code>./Images/11812526.jpg</code>	<code>./Images/11742962.jpg</code>	<code>./Images/11978446.jpg</code>
  <code>Isabelle from Animal Crossing stands with her tied blonde hair and green bikini top and bottom, her big breasts spilling through the cleavage as green hearts hover around her head and her thick thighs and yellow fur draw a female-only, animal-eyed focus.</code>	<code>./Images/11651564.jpg</code>	<code>./Images/13035434.jpg</code>	<code>./Images/13000601.jpg</code>	<code>./Images/11979417.jpg</code>	<code>./Images/11790567.jpg</code>
  <code>Ambrosius Goldenloin fucks the hairy, piercing-adorned Nimona with a thick, juicy cock as she moans and clenches around his balls.</code>	<code>./Images/10880184.jpg</code>	<code>./Images/11855995.jpg</code>	<code>./Images/12893082.jpg</code>	<code>./Images/11948541.jpg</code>	<code>./Images/12449120.jpg</code>

• Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "CachedMultipleNegativesRankingLoss",
      "matryoshka_dims": [
          2048,
          1536,
          1024,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Evaluation Dataset

json

• Dataset: json
• Size: 4,776 evaluation samples
• Columns: <code>query</code>, <code>positive</code>, <code>negative0</code>, <code>negative1</code>, <code>negative2</code>, and <code>negative3</code>
• Approximate statistics based on the first 100 samples:

  	query	positive	negative0	negative1	negative2	negative3
  type	string	string	string	string	string	string
  modality	text	image	image	image	image	image
  details	<ul><li>min: 46 tokens</li><li>mean: 75.79 tokens</li><li>max: 133 tokens</li></ul>

• Samples:

  query	positive	negative0	negative1	negative2	negative3
  <code>May sits with her legs spread, her massive breasts barely contained by her bikini as she grips her huge cock and thick, long penis, precum leaking out while she looks over her shoulder with a smile.</code>	<code>./Images/11897115.jpg</code>	<code>./Images/12829919.jpg</code>	<code>./Images/12813676.jpg</code>	<code>./Images/12136766.jpg</code>	<code>./Images/12403205.jpg</code>
  <code>On a sunny beach, Mythra from Xenoblade Chronicles 2 wears a skimpy sling bikini, her blonde hair and large breasts on full display as she kneels with arms up, cheeks flushed in an ahegao, enduring deep penetration from a big-penised ugly man while cum trickles from her lactating nipples and a thick toe curls between her fat ass cheeks.</code>	<code>./Images/12313417.jpg</code>	<code>./Images/12710154.jpg</code>	<code>./Images/12739902.jpg</code>	<code>./Images/11840408.jpg</code>	<code>./Images/12339885.jpg</code>
  <code>Lisa from Genshin Impact kneels in the center of a gangbang, taking all three large penises deep in her mouth as thick saliva and lipstick marks glisten on their dark skin and her own light skin, while the tattooed, ugly man and his companions thrust into her mouth and onto her face, overwhelming her with big balls and big breasts.</code>	<code>./Images/12110929.jpg</code>	<code>./Images/12259516.jpg</code>	<code>./Images/12317214.jpg</code>	<code>./Images/11592249.jpg</code>	<code>./Images/12261530.jpg</code>

• Loss: <code>MatryoshkaLoss</code> with these parameters:

  {
      "loss": "CachedMultipleNegativesRankingLoss",
      "matryoshka_dims": [
          2048,
          1536,
          1024,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 48
• num_train_epochs: 1
• learning_rate: 2e-05
• warmup_steps: 0.1
• bf16: True
• per_device_eval_batch_size: 48
• dataloader_num_workers: 8
• batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• per_device_train_batch_size: 48
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 48
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 8
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Training Time

• Training: 1.3 hours
• Evaluation: 24.0 minutes
• Total: 1.7 hours

Framework Versions

• Python: 3.11.15
• Sentence Transformers: 5.5.1
• Transformers: 5.10.2
• PyTorch: 2.11.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->