gte-modernbert-reranker-custom

CrossEncoder based on Alibaba-NLP/gte-reranker-modernbert-base

This is a Cross Encoder model finetuned from Alibaba-NLP/gte-reranker-modernbert-base using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: Alibaba-NLP/gte-reranker-modernbert-base <!-- at revision f7481e6055501a30fb19d090657df9ec1f79ab2c -->
• Maximum Sequence Length: 8192 tokens
• Number of Output Labels: 1 label

<!-- - **Training Dataset:** Unknown --> <!-- - **Language:** Unknown --> <!-- - **License:** Unknown -->

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ['particleboard bookcase white', 'the trådfri led bulb e12 450 lumen is a white, opal globe-shaped light bulb made of polycarbonate plastic. it provides a default luminous flux of 450 lumens and has a color temperature of 2700 k, with a power consumption of 4.1 watts. this bulb is compatible with ikea smarthome products and offers adjustable white tones and wireless dimming, ideal for small lampshades. its item number is 70518179 and price is 9.99.'],
    ['particleboard bookcase white', 'the bestå tv storage combination with glass doors is a furniture piece in beige, brown, and black colors, designed to organize and display tvs and related gadgets. it measures 94 1/2x16 1/2x91 inches (240x42x231 cm) and features a blend of particleboard, fibreboard, honeycomb paper filling, paper foil, plastic edging, and tempered glass. the tv bench and frame incorporate recycled materials, while the shelves include both tempered glass and particleboard. drawer runners are made from galvanized steel with a push-open mechanism, enhanced by soft closing hinges crafted from steel and nickel-plated metal. the glass doors are composed of tempered glass with a fibreboard frame. the product supports loads up to 22 pounds (10 kg) on its drawer, 44 pounds (20 kg) on shelves, 22 pounds (10 kg) on glass shelves, and a maximum of 110 pounds (50 kg) on the top panel. it includes features for easy cleaning, requiring a wipe with a damp cloth followed by a dry cloth. its item number is 19421393 and price is 462.49.'],
    ['particleboard bookcase white', 'the metod base cabinet for sink with 2 doors in green features a solid color appearance and is constructed from wood materials. its overall dimensions are 31 inches in width, 24 inches in depth, and 31 1/2 inches in height (80 cm x 60 cm x 80 cm). the detailed measurements include a system depth of 24 inches (60 cm) and a depth of 24 1/4 inches (61.6 cm). the product includes doors made of fibreboard with acrylic paint, and the frame consists of particleboard, melamine foil, and plastic edging. the back panel is made of fibreboard with paper foil, while the front rail is made of steel with an epoxy/polyester powder coating. hinges are crafted from steel with nickel plating and feature a built-in damper for smooth operation. the cabinet is designed as part of the metod kitchen system, providing versatile design possibilities. its item number is 89557444 and price is 245.45.'],
    ['particleboard bookcase white', 'the metod corner wall cabinet with shelves is a grey, solid-colored kitchen storage unit featuring a traditional style with its bodbyn fronts and bevelled edges. constructed from wood, including fibreboard and particleboard materials, it is finished with polyurethane and acrylic paints. the cabinet measures 27x14 5/8x31 inches (68x37x80 cm) with a system depth of 14 5/8 inches (37 cm). the door is made of fibreboard with a painted finish, while the frame consists of particleboard and melamine foil, with shelving that includes plastic edging for durability. the cabinet offers functional shelving designed for neat storage. maintenance is simple with cleaning using a cloth and non-abrasive detergent. its item number is 99918695 and price is 259.09.'],
    ['particleboard bookcase white', 'the blåvingad duvet cover and pillowcase set features an ocean animals pattern in multiple colors. made from a blend of 50% cotton and 50% lyocell, it offers a soft and comfortable feel. the set is designed for a twin size, with the duvet cover measuring 218 cm by 162 cm (85 7/8 inches by 63 3/4 inches) and the pillowcase measuring 51 cm by 76 cm (20 inches by 30 inches). it has a thread count of 164 per inch². the product is machine washable and can be tumbled dry at a medium temperature, but should not be bleached or dry cleaned. maximum shrinkage is 4%. its item number is 00521076 and price is 34.99.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'particleboard bookcase white',
    [
        'the trådfri led bulb e12 450 lumen is a white, opal globe-shaped light bulb made of polycarbonate plastic. it provides a default luminous flux of 450 lumens and has a color temperature of 2700 k, with a power consumption of 4.1 watts. this bulb is compatible with ikea smarthome products and offers adjustable white tones and wireless dimming, ideal for small lampshades. its item number is 70518179 and price is 9.99.',
        'the bestå tv storage combination with glass doors is a furniture piece in beige, brown, and black colors, designed to organize and display tvs and related gadgets. it measures 94 1/2x16 1/2x91 inches (240x42x231 cm) and features a blend of particleboard, fibreboard, honeycomb paper filling, paper foil, plastic edging, and tempered glass. the tv bench and frame incorporate recycled materials, while the shelves include both tempered glass and particleboard. drawer runners are made from galvanized steel with a push-open mechanism, enhanced by soft closing hinges crafted from steel and nickel-plated metal. the glass doors are composed of tempered glass with a fibreboard frame. the product supports loads up to 22 pounds (10 kg) on its drawer, 44 pounds (20 kg) on shelves, 22 pounds (10 kg) on glass shelves, and a maximum of 110 pounds (50 kg) on the top panel. it includes features for easy cleaning, requiring a wipe with a damp cloth followed by a dry cloth. its item number is 19421393 and price is 462.49.',
        'the metod base cabinet for sink with 2 doors in green features a solid color appearance and is constructed from wood materials. its overall dimensions are 31 inches in width, 24 inches in depth, and 31 1/2 inches in height (80 cm x 60 cm x 80 cm). the detailed measurements include a system depth of 24 inches (60 cm) and a depth of 24 1/4 inches (61.6 cm). the product includes doors made of fibreboard with acrylic paint, and the frame consists of particleboard, melamine foil, and plastic edging. the back panel is made of fibreboard with paper foil, while the front rail is made of steel with an epoxy/polyester powder coating. hinges are crafted from steel with nickel plating and feature a built-in damper for smooth operation. the cabinet is designed as part of the metod kitchen system, providing versatile design possibilities. its item number is 89557444 and price is 245.45.',
        'the metod corner wall cabinet with shelves is a grey, solid-colored kitchen storage unit featuring a traditional style with its bodbyn fronts and bevelled edges. constructed from wood, including fibreboard and particleboard materials, it is finished with polyurethane and acrylic paints. the cabinet measures 27x14 5/8x31 inches (68x37x80 cm) with a system depth of 14 5/8 inches (37 cm). the door is made of fibreboard with a painted finish, while the frame consists of particleboard and melamine foil, with shelving that includes plastic edging for durability. the cabinet offers functional shelving designed for neat storage. maintenance is simple with cleaning using a cloth and non-abrasive detergent. its item number is 99918695 and price is 259.09.',
        'the blåvingad duvet cover and pillowcase set features an ocean animals pattern in multiple colors. made from a blend of 50% cotton and 50% lyocell, it offers a soft and comfortable feel. the set is designed for a twin size, with the duvet cover measuring 218 cm by 162 cm (85 7/8 inches by 63 3/4 inches) and the pillowcase measuring 51 cm by 76 cm (20 inches by 30 inches). it has a thread count of 164 per inch². the product is machine washable and can be tumbled dry at a medium temperature, but should not be bleached or dry cleaned. maximum shrinkage is 4%. its item number is 00521076 and price is 34.99.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

<!-- ### Direct Usage (Transformers) <details><summary>Click to see the direct usage in Transformers</summary> </details> --> <!-- ### Downstream Usage (Sentence Transformers) You can finetune this model on your own dataset. <details><summary>Click to expand</summary> </details> --> <!-- ### Out-of-Scope Use *List how the model may foreseeably be misused and address what users ought not to do with the model.* --> <!-- ## Bias, Risks and Limitations *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* --> <!-- ### Recommendations *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* -->

Training Details

Training Dataset

Unnamed Dataset

• Size: 804,721 training samples
• Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
• Approximate statistics based on the first 1000 samples:

  	query	docs	labels
  type	string	list	list
  details	<ul><li>min: 4 characters</li><li>mean: 20.55 characters</li><li>max: 68 characters</li></ul>	<ul><li>min: 1 elements</li><li>mean: 8.50 elements</li><li>max: 16 elements</li></ul>	<ul><li>min: 1 elements</li><li>mean: 8.50 elements</li><li>max: 16 elements</li></ul>

• Samples:

  query	docs	labels
  <code>particleboard bookcase white</code>	<code>['the trådfri led bulb e12 450 lumen is a white, opal globe-shaped light bulb made of polycarbonate plastic. it provides a default luminous flux of 450 lumens and has a color temperature of 2700 k, with a power consumption of 4.1 watts. this bulb is compatible with ikea smarthome products and offers adjustable white tones and wireless dimming, ideal for small lampshades. its item number is 70518179 and price is 9.99.', 'the bestå tv storage combination with glass doors is a furniture piece in beige, brown, and black colors, designed to organize and display tvs and related gadgets. it measures 94 1/2x16 1/2x91 inches (240x42x231 cm) and features a blend of particleboard, fibreboard, honeycomb paper filling, paper foil, plastic edging, and tempered glass. the tv bench and frame incorporate recycled materials, while the shelves include both tempered glass and particleboard. drawer runners are made from galvanized steel with a push-open mechanism, enhanced by soft closing hinges crafted fr...</code>	<code>[0, 0, 0, 0, 0, ...]</code>
  <code>small green cabinet with drawer</code>	<code>['the bestå tv bench with doors is a white piece designed to organize and streamline your media area. it features a particle- and fibreboard construction with a honeycomb paper filling made from 100% recycled paper. the surface is finished with paper and melamine foil, with plastic edging for durability. the bench measures 47 1/4 inches wide, 16 1/2 inches deep, and 18 7/8 inches high (120x42x48 cm). it can support a maximum load of 44 pounds per shelf and accommodate tvs weighing up to 35 pounds. functionally, it provides ample storage space and can help keep cables tidy. the tv bench includes features such as soft closing and push-open hinges made from steel and nickel-plated metal, particleboard doors with plastic foil, solid wooden legs with acrylic paint, and a supporting leg made from steel with a powder coating. its item number is 79419448 and price is 231.82.', 'the kungsfors suspension rail with shelf and wall grid is a gray storage solution made from stainless steel. this met...</code>	<code>[0, 0, 0, 0, 0, ...]</code>
  <code>enkoping brown cabinet with shelves</code>	<code>['the maximera drawer, low, is a white storage drawer made from durable metal material. it features measurements of 15 3/4x14 5/8 inches (40x37 cm). the detailed measurements include a width of 14 3/8 inches (36.4 cm) and a frame width of 15 3/4 inches (40.0 cm), a depth of 13 1/4 inches (33.7 cm) and a frame depth of 14 5/8 inches (37.0 cm), and a height of 3 1/8 inches (7.8 cm). the drawer can support a maximum load of 44 lb 1 oz (20 kg). constructed with steel alongside an epoxy/polyester powder coating for the drawer sides and back, it features a particleboard bottom with melamine foil, galvanized steel runners, and abs plastic bumpers. its standout feature is the soft-closing mechanism, ensuring quiet closure, with the capability to pull out fully for easy access and organization. it is maintained by wiping with a cloth dampened in water and mild detergent. its item number is 50285029 and price is 17.5.', 'the nordberget mattress topper is a twin-sized memory foam product in white...</code>	<code>[0, 0, 0, 1, 0, ...]</code>

• Loss: <code>LambdaLoss</code> with these parameters:

  {
      "weighting_scheme": "sentence_transformers.cross_encoder.losses.LambdaLoss.NDCGLoss2PPScheme",
      "k": null,
      "sigma": 1.0,
      "eps": 1e-10,
      "reduction_log": "binary",
      "activation_fn": "torch.nn.modules.linear.Identity",
      "mini_batch_size": 8
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• bf16: True
• dataloader_num_workers: 4

All Hyperparameters

<details><summary>Click to expand</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 4
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

</details>

Training Logs

<details><summary>Click to expand</summary> </details>

Framework Versions

• Python: 3.10.14
• Sentence Transformers: 5.0.0
• Transformers: 4.51.0
• PyTorch: 2.4.1+cu121
• Accelerate: 1.7.0
• Datasets: 3.2.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

LambdaLoss

@inproceedings{wang2018lambdaloss,
  title={The LambdaLoss Framework for Ranking Metric Optimization},
  author={Wang, Xuanhui and Li, Cheng and Golbandi, Nadav and Bendersky, Michael and Najork, Marc},
  booktitle={Proceedings of the 27th ACM international conference on information and knowledge management},
  pages={1313--1322},
  year={2018}
}

<!-- ## Glossary *Clearly define terms in order to be accessible across audiences.* --> <!-- ## Model Card Authors *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* --> <!-- ## Model Card Contact *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* -->