aewol-embedding-v1

SentenceTransformer based on juyoungml/bge-m3-ko-v1.1

This is a sentence-transformers model finetuned from juyoungml/bge-m3-ko-v1.1. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: juyoungml/bge-m3-ko-v1.1 <!-- at revision cb66d1a48f89b2c8b78714e739497cf99a464fdb -->
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    '제주 항일운동가 손실 소식에 대해 알고 싶습니다.',
    '다. 항일ㆍ노동운동의 거목이 손실되자 조선인ㆍ일본인의 노동계가 합동으로 시신을 유리관(棺)에 안치, 오사카시 노동장(勞動葬)으로 성대히 엄수... (내용이 길어 일부 생략됨)',
    '머리가 되는 것이 아니다. 누구가 먼저 키를 잡는 수완과 자질을 발휘해 선점하느냐에 따라 도사공의 지위를 차지하게 된다는 말이다.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.9309, -0.9304],
#         [-0.9309,  1.0000,  0.9989],
#         [-0.9304,  0.9989,  1.0000]])

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* -->

Evaluation

Metrics

Triplet

• Dataset: val
• Evaluated with <code>TripletEvaluator</code>

<!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 6,238 training samples
• Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>sentence_2</code>
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	<ul><li>min: 8 tokens</li><li>mean: 20.85 tokens</li><li>max: 91 tokens</li></ul>	<ul><li>min: 8 tokens</li><li>mean: 216.35 tokens</li><li>max: 2850 tokens</li></ul>	<ul><li>min: 6 tokens</li><li>mean: 193.04 tokens</li><li>max: 2850 tokens</li></ul>

• Samples:

  sentence_0	sentence_1	sentence_2
  <code>할머니, 할아버지는 어떤 모습일까요?</code>	<code>[삼 서끈 할망은 놀레 불르곡, 노 서끈 하르방은 운다.]<br>삼 섞은 할머니는 노래 부르고, 노 섞은 할아버지는 운다. 여자는 인내력과 서둘지 않은 차분한 일 처리 능력이 남자에 비해서 앞선다는 말이다.</code>	<code>정부에서는 고인의 공훈을 기리어 1990년 광복절에 건국훈장 애족장을 추서하였다.</code>
  <code>제주도에서는 밥 먹는 일이 일과 어떤 연관이 있나요?</code>	<code>하고 다음 일을 할 수 있다. 어느 한 쪽이 늦어지면 그만큼 다음 일 착수가 늦어질 뿐만 아리나 일 처리도 미뤄지므로, 밥 먹는 일 자체가 곧 일을 차리는 것이 된다는 말이다.<br>의미: 밥 먹음도 일 차림이다. 밥을 먹는 일은 평상시 정례화 된 사실이지만, 먹는 쪽이나 그 식사를 마련하는 쪽이 다 정해진 일과의 한 과정을 수행하고 있는 것이다. 그러니 먹는 사람은 먹어야 다음 일을 시작할 수 있고, 식사를 차려놓는 사람은 제 때에 먹어 버려야 뒤처리를 하고 다음 일을 할 수 있다. 어느 한 쪽이 늦어지면 그만큼 다음 일 착수가 늦어질 뿐만 아리나 일 처리도 미뤄지므로, 밥 먹는 일 자체가 곧 일을 차리는 것이 된다는 말이다.</code>	<code>나하고 둘이 둥실 사랑하자.`</code>
  <code>행원남당본풀이 이야기 들어보고 싶습니다.</code>	<code>[행원남당본풀이]<br>(myth)<br>[신화] MYTH2.002 - 핵심어: 본풀이,행원남당본풀이</code>	<code>못하고 몸만 상하고 마는 현상을 지적하고 있다</code>

• Loss: <code>TripletLoss</code> with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 1
• per_device_eval_batch_size: 1
• fp16: True
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 1
• per_device_eval_batch_size: 1
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.0
• Transformers: 4.57.3
• PyTorch: 2.9.0+cu126
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->