finetuned-sentence-encoder-for-pddl-gen

SentenceTransformer based on sentence-transformers/all-roberta-large-v1

This is a sentence-transformers model finetuned from sentence-transformers/all-roberta-large-v1 on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-roberta-large-v1 <!-- at revision 7351068294d6fad1e9991c08ec123bbacdb0cad5 -->
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 1024 tokens
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json

<!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("huangsukai/finetuned-sentence-encoder-for-pddl-gen")
# Run inference
sentences = [
    "Context: Translate the given natural language description into an action schema that includes the parameters, preconditions, and effects. Ensure that only the provided predicates are used to construct the preconditions and effects.\nQuestion: Here is the task.\nA natural language description of the domain\nDomain information: Help the hero to get out of dungeon! A hero woke up in a dungeon full of monsters and traps (perhaps the party last night went wrong...) and needs your help to get out.  Here are basic facts for the dungeon domain: - The dungeon contains rooms that are **connected** by corridors (dungeon can thus be represented by undirected graph) - each room can be **empty**, or can have a **monster** in it, or can have a **trap** in it, or can have a **sword** in it - one of the empty rooms is the **goal**: it has an exit, so the hero can escape\n\nA list of available predicates\n1. (at-hero ?loc - cells) ;;  Hero's cell location\n2. (at-sword ?s - swords ?loc - cells) ;; Sword cell location\n3. (has-monster ?loc - cells) ;; Indicates if a cell location has a monster\n4. (has-trap ?loc - cells) ;; Indicates if a cell location has a trap\n5. (is-destroyed ?obj) ;; Indicates if a chell or sword has been destroyed\n6. (connected ?from ?to - cells) ;; connects cells\n7. (arm-free) ;; Hero's hand is free\n8. (holding ?s - swords) ;; Hero's holding a sword\n9. (trap-disarmed ?loc) ;; It becomes true when a trap is disarmed\n\nAction Description: **Disarm a trap** – if there is a trap in the room the hero is in and the hero is empty-handed (does not hold a sword), then the hero can disarm it\n\nAction name: disarm-trap\n\n\nYour answer:\n",
    'Parameters:\n1. ?loc - cells\n\nPreconditions:\n```\n(and\n    (at-hero ?loc)\n    (arm-free)\n    (has-trap ?loc)\n)\n```\n\nEffects:\n```\n(and\n    (trap-disarmed ?loc)\n    (not (has-trap ?loc))\n)\n```\n',
    'Parameters:\n1. ?from - cells\n2. ?to - cells\n\nPreconditions:\n```\n(and\n    (connected ?from ?to)\n    (at-hero ?from)\n    (not (has-trap ?from))\n    (not (is-destroyed ?to))\n    (not (has-trap ?to))\n    (not (has-monster ?to))\n)\n```\n\nEffects:\n```\n(and\n    (at-hero ?to)\n    (is-destroyed ?from)\n    (not (at-hero ?from))\n)\n```\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

json

• Dataset: json
• Size: 200,010 training samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 246 tokens</li><li>mean: 255.74 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 71 tokens</li><li>mean: 138.42 tokens</li><li>max: 248 tokens</li></ul>	<ul><li>min: 62 tokens</li><li>mean: 132.66 tokens</li><li>max: 250 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>Context: Translate the given natural language description into an action schema that includes the parameters, preconditions, and effects. Ensure that only the provided predicates are used to construct the preconditions and effects.<br>Question: Here is the task.<br>A natural language description of the domain<br>Domain information: This domain is structured to allow organizing and managing books within a library setting. The actions and predicates support the movement of books between tables and shelves, ensuring that conditions like accessibility and the librarian's hands being free are met. Additionally, it includes managing book categories, shelf space, and check-out/return processes to reflect a more complex library system.<br><br>A list of available predicates<br>1. (on-shelf ?x ?y - book) ;; ?x is on top of ?y on the shelf<br>2. (on-table ?x - book) ;; ?x is on the table<br>3. (accessible ?x - book) ;; ?x is accessible (not covered)<br>4. (hands-free) ;; The hands of the librarian are free<br>5. (holding ?x - book) ;; The librarian is holding ?x<br>6. (belongs-to-category ?x - book ?cat - category) ;; ?x belongs to the category ?cat<br>7. (shelf-empty ?cat - category) ;; The shelf for category ?cat is empty<br>8. (shelf-overflow ?cat - category) ;; The shelf for category ?cat is full<br>9. (book-request ?book - book) ;; There is a request for book ?book<br>10. (return-due ?book - book) ;; Book ?book is due for return<br>11. (checked-out ?book - book) ;; Book ?book is checked out<br><br>Action Description: Mark a book as borrowed by a patron, ensuring it's not already taken.<br><br>Action name: check-out<br><br><br>Your answer:<br></code>	<code>Parameters:<br>1. ?x - book<br><br>Preconditions:<br><br>(and<br> (accessible ?x)<br> (not (checked-out ?x))<br>)<br><br><br>Effects:<br><br>(and<br> (checked-out ?x)<br> (not (accessible ?x))<br> (book-request ?x)<br> (return-due ?x)<br>)<br><br></code>	<code>Parameters:<br>1. ?x - book<br><br>Preconditions:<br><br>(and<br> (checked-out ?x)<br> (accessible ?x)<br>)<br><br><br>Effects:<br><br>(and<br> (checked-out ?x)<br> (book-request ?x)<br> (return-due ?x)<br>)<br><br></code>
  <code>Context: Translate the given natural language description into an action schema that includes the parameters, preconditions, and effects. Ensure that only the provided predicates are used to construct the preconditions and effects.<br>Question: Here is the task.<br>A natural language description of the domain<br>Domain information: This describes a cooking or baking process where ingredients such as eggs and flour are used. The process of baking involves putting eggs in the pan followed by the flour, mix the two and then put the pan in the oven and then remove the pan from the oven to get the baked cake. Lastly, the pan is cleaned using a soap. Well, you can also bake a souffle but in at bit different process.<br><br>A list of available predicates<br>1. (is_egg ?egg - ingredient) ;; the ingredient is an egg<br>2. (is_flour ?flour - ingredient) ;; the ingredient is flour<br>3. (pan_has_egg ?pan - pan) ;; the pan has an egg<br>4. (pan_has_flour ?pan - pan) ;; the pan has flour<br>5. (pan_is_clean ?pan - pan) ;; the pan is clean<br>6. (pan_in_oven ?pan - pan) ;; the pan is in the oven<br>7. (in_pan ?x - ingredient ?pan - pan) ;; the ingredient is in the pan<br>8. (in_oven ?pan - pan ?oven - oven) ;; the pan is in the oven<br>9. (oven_is_full ?oven - oven) ;; the oven is full<br>10. (hypothetical ?new - ingredient) ;; the ingredient is hypothetical<br>11. (is_mixed ?pan - pan) ;; the ingredients in the pan are mixed<br>12. (is_cake ?new - ingredient) ;; the ingredient is a cake<br>13. (is_souffle ?new - ingredient) ;; the ingredient is a souffle<br>14. (soap_consumed ?soap - soap) ;; the soap is consumed<br><br>Action Description: Action putting pan in oven needs oven and a pan. The pan is put in oven which is not full.<br><br>Action name: put_pan_in_oven<br><br><br>Your answer:<br></code>	<code>Parameters:<br>1. ?pan - pan<br>2. ?oven - oven<br><br>Preconditions:<br><br>(and<br> (not (oven_is_full ?oven))<br> (not (pan_in_oven ?pan))<br>)<br><br><br>Effects:<br><br>(and<br> (oven_is_full ?oven)<br> (in_oven ?pan ?oven)<br> (pan_in_oven ?pan)<br>)<br><br></code>	<code>Parameters:<br>1. ?pan - pan<br>2. ?oven - oven<br><br>Preconditions:<br><br>(and<br> (not (oven_is_full ?oven))<br> (not (pan_in_oven ?pan))<br>)<br><br><br>Effects:<br><br>(and<br> (oven_is_full ?oven)<br> (in_oven ?pan ?oven)<br>)<br><br></code>
  <code>Context: Translate the given natural language description into an action schema that includes the parameters, preconditions, and effects. Ensure that only the provided predicates are used to construct the preconditions and effects.<br>Question: Here is the task.<br>A natural language description of the domain<br>Domain information: Help the hero to get out of dungeon! A hero woke up in a dungeon full of monsters and traps (perhaps the party last night went wrong...) and needs your help to get out. Here are basic facts for the dungeon domain: - The dungeon contains rooms that are connected by corridors (dungeon can thus be represented by undirected graph) - each room can be empty, or can have a monster in it, or can have a trap in it, or can have a sword in it - one of the empty rooms is the goal: it has an exit, so the hero can escape<br><br>A list of available predicates<br>1. (at-hero ?loc - cells) ;; Hero's cell location<br>2. (at-sword ?s - swords ?loc - cells) ;; Sword cell location<br>3. (has-monster ?loc - cells) ;; Indicates if a cell location has a monster<br>4. (has-trap ?loc - cells) ;; Indicates if a cell location has a trap<br>5. (is-destroyed ?obj) ;; Indicates if a chell or sword has been destroyed<br>6. (connected ?from ?to - cells) ;; connects cells<br>7. (arm-free) ;; Hero's hand is free<br>8. (holding ?s - swords) ;; Hero's holding a sword<br>9. (trap-disarmed ?loc) ;; It becomes true when a trap is disarmed<br><br>Action Description: Hero can move if the - hero is at current location - cells are connected, - there is no trap in current loc, and - destination does not have a trap/monster/has-been-destroyed Effects move the hero, and destroy the original cell. No need to destroy the sword.<br><br>Action name: move<br><br><br>Your answer:<br></code>	<code>Parameters:<br>1. ?from - cells<br>2. ?to - cells<br><br>Preconditions:<br><br>(and<br> (connected ?from ?to)<br> (at-hero ?from)<br> (not (has-trap ?from))<br> (not (is-destroyed ?to))<br> (not (has-trap ?to))<br> (not (has-monster ?to))<br>)<br><br><br>Effects:<br><br>(and<br> (at-hero ?to)<br> (is-destroyed ?from)<br> (not (at-hero ?from))<br>)<br><br></code>	<code>Parameters:<br>1. ?from - cells<br>2. ?to - cells<br><br>Preconditions:<br><br>(and<br> (connected ?from ?to)<br> (at-hero ?from)<br> (not (has-trap ?from))<br> (not (is-destroyed ?to))<br> (not (has-trap ?to))<br>)<br><br><br>Effects:<br><br>(and<br> (at-hero ?to)<br> (is-destroyed ?from)<br> (not (at-hero ?from))<br>)<br><br></code>

• Loss: <code>GISTEmbedLoss</code> with these parameters:

  {'guide': SentenceTransformer(
    (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: T5EncoderModel 
    (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
    (2): Dense({'in_features': 1024, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
    (3): Normalize()
  ), 'temperature': 0.01}
  

Evaluation Dataset

json

• Dataset: json
• Size: 34 evaluation samples
• Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
• Approximate statistics based on the first 34 samples:

  	anchor	positive	negative
  type	string	string	string
  details	<ul><li>min: 256 tokens</li><li>mean: 256.0 tokens</li><li>max: 256 tokens</li></ul>	<ul><li>min: 82 tokens</li><li>mean: 127.94 tokens</li><li>max: 176 tokens</li></ul>	<ul><li>min: 64 tokens</li><li>mean: 121.15 tokens</li><li>max: 176 tokens</li></ul>

• Samples:

  anchor	positive	negative
  <code>Context: Translate the given natural language description into an action schema that includes the parameters, preconditions, and effects. Ensure that only the provided predicates are used to construct the preconditions and effects.<br>Question: Here is the task.<br>A natural language description of the domain<br>Domain information: This domain is structured to allow organizing and managing books within a library setting. The actions and predicates support the movement of books between tables and shelves, ensuring that conditions like accessibility and the librarian's hands being free are met. Additionally, it includes managing book categories, shelf space, and check-out/return processes to reflect a more complex library system.<br><br>A list of available predicates<br>1. (on-shelf ?x ?y - book) ;; ?x is on top of ?y on the shelf<br>2. (on-table ?x - book) ;; ?x is on the table<br>3. (accessible ?x - book) ;; ?x is accessible (not covered)<br>4. (hands-free) ;; The hands of the librarian are free<br>5. (holding ?x - book) ;; The librarian is holding ?x<br>6. (belongs-to-category ?x - book ?cat - category) ;; ?x belongs to the category ?cat<br>7. (shelf-empty ?cat - category) ;; The shelf for category ?cat is empty<br>8. (shelf-overflow ?cat - category) ;; The shelf for category ?cat is full<br>9. (book-request ?book - book) ;; There is a request for book ?book<br>10. (return-due ?book - book) ;; Book ?book is due for return<br>11. (checked-out ?book - book) ;; Book ?book is checked out<br><br>Action Description: Consider a librarian holding a book and standing near a shelf. The 'place-on-shelf' action involves placing the held book on top of another book on the shelf, given that the book on the shelf is accessible. This action results in the held book becoming accessible, the book on the shelf becoming inaccessible, and the librarian's hands becoming free.<br><br>Action name: place-on-shelf<br><br><br>Your answer:<br></code>	<code>Parameters:<br>1. ?x - book<br>2. ?y - book<br>3. ?cat - category<br><br>Preconditions:<br><br>(and<br> (holding ?x)<br> (accessible ?y)<br> (belongs-to-category ?x ?cat)<br> (not (shelf-overflow ?cat))<br>)<br><br><br>Effects:<br><br>(and<br> (not (holding ?x))<br> (not (accessible ?y))<br> (accessible ?x)<br> (hands-free)<br> (on-shelf ?x ?y)<br> (shelf-empty ?cat)<br>)<br><br></code>	<code>Parameters:<br>1. ?x - book<br>2. ?y - book<br>3. ?cat - category<br><br>Preconditions:<br><br>(and<br> (accessible ?y)<br> (belongs-to-category ?x ?cat)<br> (not (shelf-overflow ?cat))<br>)<br><br><br>Effects:<br><br>(and<br> (not (shelf-empty ?cat))<br> (not (holding ?x))<br> (not (accessible ?y))<br> (accessible ?x)<br> (hands-free)<br> (on-shelf ?x ?y)<br>)<br><br></code>
  <code>Context: Translate the given natural language description into an action schema that includes the parameters, preconditions, and effects. Ensure that only the provided predicates are used to construct the preconditions and effects.<br>Question: Here is the task.<br>A natural language description of the domain<br>Domain information: This domain is structured to allow organizing and managing books within a library setting. The actions and predicates support the movement of books between tables and shelves, ensuring that conditions like accessibility and the librarian's hands being free are met. Additionally, it includes managing book categories, shelf space, and check-out/return processes to reflect a more complex library system.<br><br>A list of available predicates<br>1. (on-shelf ?x ?y - book) ;; ?x is on top of ?y on the shelf<br>2. (on-table ?x - book) ;; ?x is on the table<br>3. (accessible ?x - book) ;; ?x is accessible (not covered)<br>4. (hands-free) ;; The hands of the librarian are free<br>5. (holding ?x - book) ;; The librarian is holding ?x<br>6. (belongs-to-category ?x - book ?cat - category) ;; ?x belongs to the category ?cat<br>7. (shelf-empty ?cat - category) ;; The shelf for category ?cat is empty<br>8. (shelf-overflow ?cat - category) ;; The shelf for category ?cat is full<br>9. (book-request ?book - book) ;; There is a request for book ?book<br>10. (return-due ?book - book) ;; Book ?book is due for return<br>11. (checked-out ?book - book) ;; Book ?book is checked out<br><br>Action Description: Put a book you're holding on top of another accessible book on the shelf.<br><br>Action name: place-on-shelf<br><br><br>Your answer:<br></code>	<code>Parameters:<br>1. ?x - book<br>2. ?y - book<br>3. ?cat - category<br><br>Preconditions:<br><br>(and<br> (holding ?x)<br> (accessible ?y)<br> (belongs-to-category ?x ?cat)<br> (not (shelf-overflow ?cat))<br>)<br><br><br>Effects:<br><br>(and<br> (not (holding ?x))<br> (not (accessible ?y))<br> (accessible ?x)<br> (hands-free)<br> (on-shelf ?x ?y)<br> (shelf-empty ?cat)<br>)<br><br></code>	<code>Parameters:<br>1. ?x - book<br>2. ?y - book<br>3. ?cat - category<br><br>Preconditions:<br><br>(and<br> (holding ?x)<br> (accessible ?y)<br> (belongs-to-category ?x ?cat)<br> (not (shelf-overflow ?cat))<br>)<br><br><br>Effects:<br><br>(and<br> (not (hands-free))<br> (not (holding ?x))<br> (not (accessible ?y))<br> (accessible ?x)<br> (on-shelf ?x ?y)<br> (shelf-empty ?cat)<br>)<br><br></code>
  <code>Context: Translate the given natural language description into an action schema that includes the parameters, preconditions, and effects. Ensure that only the provided predicates are used to construct the preconditions and effects.<br>Question: Here is the task.<br>A natural language description of the domain<br>Domain information: This domain is structured to allow organizing and managing books within a library setting. The actions and predicates support the movement of books between tables and shelves, ensuring that conditions like accessibility and the librarian's hands being free are met. Additionally, it includes managing book categories, shelf space, and check-out/return processes to reflect a more complex library system.<br><br>A list of available predicates<br>1. (on-shelf ?x ?y - book) ;; ?x is on top of ?y on the shelf<br>2. (on-table ?x - book) ;; ?x is on the table<br>3. (accessible ?x - book) ;; ?x is accessible (not covered)<br>4. (hands-free) ;; The hands of the librarian are free<br>5. (holding ?x - book) ;; The librarian is holding ?x<br>6. (belongs-to-category ?x - book ?cat - category) ;; ?x belongs to the category ?cat<br>7. (shelf-empty ?cat - category) ;; The shelf for category ?cat is empty<br>8. (shelf-overflow ?cat - category) ;; The shelf for category ?cat is full<br>9. (book-request ?book - book) ;; There is a request for book ?book<br>10. (return-due ?book - book) ;; Book ?book is due for return<br>11. (checked-out ?book - book) ;; Book ?book is checked out<br><br>Action Description: Imagine a patron returning a borrowed book to the library. The 'return-book' action enables the librarian to process the return, updating the book's status and removing any return due date. This action is applicable when the librarian is holding the book that needs to be returned.<br><br>Action name: return-book<br><br><br>Your answer:<br></code>	<code>Parameters:<br>1. ?x - book<br><br>Preconditions:<br><br>(and<br> (checked-out ?x)<br> (holding ?x)<br>)<br><br><br>Effects:<br><br>(and<br> (not (checked-out ?x))<br> (not (holding ?x))<br> (not (book-request ?x))<br> (not (return-due ?x))<br> (accessible ?x)<br> (hands-free)<br>)<br><br></code>	<code>Parameters:<br>1. ?x - book<br><br>Preconditions:<br><br>(and<br> (not (checked-out ?x))<br> (holding ?x)<br>)<br><br><br>Effects:<br><br>(and<br> (return-due ?x)<br> (checked-out ?x)<br> (not (holding ?x))<br> (not (book-request ?x))<br> (accessible ?x)<br> (hands-free)<br>)<br><br></code>

• Loss: <code>GISTEmbedLoss</code> with these parameters:

  {'guide': SentenceTransformer(
    (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: T5EncoderModel 
    (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
    (2): Dense({'in_features': 1024, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
    (3): Normalize()
  ), 'temperature': 0.01}
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 256
• num_train_epochs: 40
• max_steps: 31251
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 40
• max_steps: 31251
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

</details>

Training Logs

Framework Versions

• Python: 3.10.14
• Sentence Transformers: 3.1.0.dev0
• Transformers: 4.42.3
• PyTorch: 2.3.1+cu121
• Accelerate: 0.32.1
• Datasets: 2.20.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

GISTEmbedLoss

@misc{solatorio2024gistembed,
    title={GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning},
    author={Aivin V. Solatorio},
    year={2024},
    eprint={2402.16829},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->