qwen3-embedding-8b-stance-mixed

SentenceTransformer based on Qwen/Qwen3-Embedding-8B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-8B. It maps sentences & paragraphs to a 4096-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-8B <!-- at revision 1d8ad4ca9b3dd8059ad90a75d4983776a23d44af -->
• Maximum Sequence Length: 40960 tokens
• Output Dimensionality: 4096 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'Qwen3Model'})
  (1): Pooling({'embedding_dimension': 4096, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'Instruct: Show literature challenging\nQuery: Artificial intelligence will ultimately enhance human creativity rather than replace it.',
]
documents = [
    'Economic pressures will drive the replacement of human creators. AI can produce vast quantities of high-quality creative content for a fraction of the cost and time, making human professionals unable to compete in the marketplace.',
    "The 'filter bubble' thesis is overstated; empirical research shows that social media often exposes people to more diverse, cross-cutting political views than their offline interactions, which can challenge rather than reinforce their beliefs.",
    'Show literature challenging',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 4096] [3, 4096]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.7180, -0.0525,  0.2857]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 31,800 training samples
• Columns: <code>anchor</code>, <code>positive</code>, <code>negative</code>, <code>instruction_text</code>, <code>boost_words</code>, and <code>claim</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative	instruction_text	boost_words	claim
  type	string	string	string	string	list	string
  details	<ul><li>min: 17 tokens</li><li>mean: 28.35 tokens</li><li>max: 43 tokens</li></ul>	<ul><li>min: 27 tokens</li><li>mean: 46.45 tokens</li><li>max: 81 tokens</li></ul>	<ul><li>min: 24 tokens</li><li>mean: 47.59 tokens</li><li>max: 104 tokens</li></ul>	<ul><li>min: 4 tokens</li><li>mean: 5.15 tokens</li><li>max: 6 tokens</li></ul>	<ul><li>size: 3 elements</li></ul>	<ul><li>min: 8 tokens</li><li>mean: 18.58 tokens</li><li>max: 32 tokens</li></ul>

• Samples:

  anchor	positive	negative	instruction_text	boost_words	claim
  <code>Instruct: Extract statements advocating for<br>Query: Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.</code>	<code>The claim is true because Alfred Nobel and Bertha von Suttner maintained a long and close correspondence in which she persistently advocated for peace, providing a direct and sustained channel for her to influence his thinking.</code>	<code>The claim is false because Nobel's philosophy on peace was notably different from Suttner's. Nobel held a more cynical and pragmatic view, once remarking that his dynamite factories would end war sooner than her peace congresses. The prize's criteria reflect his practical mindset, not a wholesale adoption of her idealistic pacifism.</code>	<code>Extract statements advocating for</code>	<code>['suttner', 'nobel', 'bertha']</code>	<code>Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.</code>
  <code>Instruct: Retrieve arguments in favor of<br>Query: Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.</code>	<code>The claim is true because their letters contain direct discussions about creating a peace prize. In an 1893 letter to Suttner, Nobel expressed his desire to establish a prize for the person who had done the most to advance the pacification of Europe, showing a clear causal link between her advocacy and his idea.</code>	<code>The claim is an oversimplification and likely false because Nobel had his own independent and pre-existing motivations for an interest in peace, primarily his personal anguish over the military use of his inventions. The prize was a way to reconcile his own conflicting legacy, not simply an adoption of Suttner's ideas.</code>	<code>Retrieve arguments in favor of</code>	<code>['suttner', 'nobel', 'bertha']</code>	<code>Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.</code>
  <code>Instruct: Find evidence backing<br>Query: Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.</code>	<code>The claim is true because Nobel explicitly read and praised Suttner's influential anti-war novel, "Die Waffen nieder!" ("Lay Down Your Arms!"), demonstrating that her specific work and pacifist message directly impacted his views on the subject.</code>	<code>The claim of 'direct' influence is false because the final text of Nobel's will and the creation of the prizes were a surprise to nearly everyone, including Suttner. If her influence had been so direct and formative, it is likely she would have had more specific knowledge of his ultimate plans before they were revealed.</code>	<code>Find evidence backing</code>	<code>['suttner', 'nobel', 'bertha']</code>	<code>Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.</code>

• Loss: <code>MultipleNegativesRankingLoss</code> with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 2
• learning_rate: 2e-05
• warmup_steps: 0.1
• gradient_accumulation_steps: 2
• bf16: True
• gradient_checkpointing: True
• remove_unused_columns: False

All Hyperparameters

<details><summary>Click to expand</summary>

• per_device_train_batch_size: 8
• num_train_epochs: 2
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 2
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: trackio
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: False
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary> </details>

Training Time

• Training: 2.4 hours

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.4.1
• Transformers: 5.5.4
• PyTorch: 2.9.1+cu128
• Accelerate: 1.12.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Stance-Aware Text Retrieval

@misc{sparacino2026embeddingmodelsstanceawareargument,
      title={Embedding Models for Stance-Aware Argument Retrieval}, 
      author={Angelo Sparacino and Francesca Toni and Adam Dejl},
      year={2026},
      eprint={2608.28283},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.28283}, 
}

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->