Downloads · 30 days
0
kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any
reranker-Qwen3.5-0.8B-doodles-any-to-any is a text ranking model from kwondw. Use it for the text ranking task on the model card, and read the license before you ship it in a product. It is set up for sentence-transformers.
This is a Cross Encoder model finetuned from Qwen/Qwen3.5-0.8B on the imagetotext and texttoimage datasets using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text rer…
Downloads · 30 days
0
Access
Public
Updated Aug 18, 2026
Repo size
46.7 MB
Likes
0
Public
Click a slice to open those files.
.safetensors26.8 MB · 57%
From the Hugging Face model README
This is a Cross Encoder model finetuned from Qwen/Qwen3.5-0.8B on the image_to_text and text_to_image datasets using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
CrossEncoder(
(0): Transformer({'transformer_task': 'any-to-any', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}, 'image': {'method': 'forward', 'method_output_name': 'logits'}, 'video': {'method': 'forward', 'method_output_name': 'logits'}, 'message': {'method': 'forward', 'method_output_name': 'logits', 'format': 'structured'}}, 'module_output_name': 'causal_logits', 'processing_kwargs': {'chat_template': {'add_generation_prompt': True}}, 'architecture': 'Qwen3_5ForConditionalGeneration'})
(1): LogitScore({'true_token_id': 16, 'false_token_id': 15, 'module_input_name': 'causal_logits'})
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import CrossEncoder
# Download from the 🤗 Hub
model = CrossEncoder("kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any")
# Get scores for pairs of inputs
pairs = [
['https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg', 'a content character with a tan head and purple puffballs hair wearing a blue fleece, green background'],
['https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg', 'a mustache character with a orange head and yellow headband hair wearing a white turtleneck, gradient 3 background'],
['https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg', 'a sunglasses character with a purple head and yellow bowlcut hair wearing a orange collar, gradient 4 background'],
['https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg', 'a mad note character with a orange head and pink toque hair wearing a orange puffer, grey background'],
['https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg', 'a pink beard character with a yellow head and green bowlcut hair wearing a striped sweater, light blue background'],
]
scores = model.predict(pairs)
print(scores)
# [0.8176 0.6298 0.6689 0.7432 0.7356]
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
doodles-image-to-text-eval and doodles-text-to-image-eval{
"at_k": 10
}
| Metric | doodles-image-to-text-eval | doodles-text-to-image-eval |
|---|---|---|
| map | 0.9825 | 0.755 |
| mrr@10 | 0.9825 | 0.755 |
| ndcg@10 | 0.9869 | 0.8167 |
| image | text | label | |
|---|---|---|---|
| type | image | string | int |
| modality | image | text | |
| details | <ul><li>min: 128x128 px</li><li>mean: 128x128 px</li><li>max: 128x128 px</li></ul> | <ul><li>min: 30 tokens</li><li>mean: 33.48 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>0: ~79.81%</li><li>1: ~20.19%</li></ul> |
| image | text | label |
|---|---|---|
| <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_0.jpg" width="200"> | <code>a cobain glasses character with a gradient 2 head and purple puffballs hair wearing a white sweater, gradient 4 background</code> | <code>1</code> |
| <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_0.jpg" width="200"> | <code>a content character with a orange head and purple long hair wearing a striped sweater, yellow background</code> | <code>0</code> |
| <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_0.jpg" width="200"> | <code>a neutral note character with a orange head and green puffballs hair wearing a combo 2 puffer, light blue background</code> | <code>0</code> |
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
| text | image | label | |
|---|---|---|---|
| type | string | image | int |
| modality | text | image | |
| details | <ul><li>min: 30 tokens</li><li>mean: 33.13 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>min: 128x128 px</li><li>mean: 128x128 px</li><li>max: 128x128 px</li></ul> | <ul><li>0: ~79.81%</li><li>1: ~20.19%</li></ul> |
| text | image | label |
|---|---|---|
| <code>a cobain glasses character with a gradient 2 head and purple puffballs hair wearing a white sweater, gradient 4 background</code> | <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_0.jpg" width="200"> | <code>1</code> |
| <code>a cobain glasses character with a gradient 2 head and purple puffballs hair wearing a white sweater, gradient 4 background</code> | <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_1.jpg" width="200"> | <code>0</code> |
| <code>a cobain glasses character with a gradient 2 head and purple puffballs hair wearing a white sweater, gradient 4 background</code> | <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_2.jpg" width="200"> | <code>0</code> |
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
| image | text | label | |
|---|---|---|---|
| type | image | string | int |
| modality | image | text | |
| details | <ul><li>min: 128x128 px</li><li>mean: 128x128 px</li><li>max: 128x128 px</li></ul> | <ul><li>min: 27 tokens</li><li>mean: 32.94 tokens</li><li>max: 37 tokens</li></ul> | <ul><li>0: ~79.81%</li><li>1: ~20.19%</li></ul> |
| image | text | label |
|---|---|---|
| <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg" width="200"> | <code>a content character with a tan head and purple puffballs hair wearing a blue fleece, green background</code> | <code>1</code> |
| <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg" width="200"> | <code>a mustache character with a orange head and yellow headband hair wearing a white turtleneck, gradient 3 background</code> | <code>0</code> |
| <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg" width="200"> | <code>a sunglasses character with a purple head and yellow bowlcut hair wearing a orange collar, gradient 4 background</code> | <code>0</code> |
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
| text | image | label | |
|---|---|---|---|
| type | string | image | int |
| modality | text | image | |
| details | <ul><li>min: 29 tokens</li><li>mean: 32.73 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 128x128 px</li><li>mean: 128x128 px</li><li>max: 128x128 px</li></ul> | <ul><li>0: ~79.81%</li><li>1: ~20.19%</li></ul> |
| text | image | label |
|---|---|---|
| <code>a content character with a tan head and purple puffballs hair wearing a blue fleece, green background</code> | <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/image_0.jpg" width="200"> | <code>1</code> |
| <code>a content character with a tan head and purple puffballs hair wearing a blue fleece, green background</code> | <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_3.jpg" width="200"> | <code>0</code> |
| <code>a content character with a tan head and purple puffballs hair wearing a blue fleece, green background</code> | <img src="https://huggingface.co/kwondw/reranker-Qwen3.5-0.8B-doodles-any-to-any/resolve/main/assets/example_image_4.jpg" width="200"> | <code>0</code> |
{
"activation_fn": "torch.nn.modules.linear.Identity",
"pos_weight": null
}
num_train_epochs: 1learning_rate: 5e-06warmup_steps: 0.1fp16: Trueprompts: {'image_to_text': "Given the image, judge whether the text matches it. Respond with 1 if they match, 0 if they don't.", 'text_to_image': "Given the text, judge whether the image matches it. Respond with 1 if they match, 0 if they don't."}per_device_train_batch_size: 8num_train_epochs: 1max_steps: -1learning_rate: 5e-06lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0.1optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1.0label_smoothing_factor: 0.0bf16: Falsefp16: Truebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: Nonetrackio_bucket_id: Nonetrackio_static_space_id: Noneper_device_eval_batch_size: 8prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 42data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Noneremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_static_graph: Noneddp_backend: Noneddp_timeout: 1800fsdp: Nonefsdp_config: Nonedeepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonewarmup_ratio: Nonelocal_rank: -1prompts: {'image_to_text': "Given the image, judge whether the text matches it. Respond with 1 if they match, 0 if they don't.", 'text_to_image': "Given the text, judge whether the image matches it. Respond with 1 if they match, 0 if they don't."}batch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | Training Loss | image to text loss | text to image loss | doodles-image-to-text-eval_ndcg@10 | doodles-text-to-image-eval_ndcg@10 |
|---|---|---|---|---|---|---|
| 0.1004 | 113 | 0.3237 | - | - | - | - |
| 0.2007 | 226 | 0.3331 | - | - | - | - |
| 0.2504 | 282 | - | 0.2869 | 0.2543 | 0.9037 | 0.6832 |
| 0.3011 | 339 | 0.2455 | - | - | - | - |
| 0.4014 | 452 | 0.2192 | - | - | - | - |
| 0.5009 | 564 | - | 0.1758 | 0.2778 | 0.9704 | 0.7909 |
| 0.5018 | 565 | 0.1695 | - | - | - | - |
| 0.6021 | 678 | 0.1765 | - | - | - | - |
| 0.7025 | 791 | 0.2051 | - | - | - | - |
| 0.7513 | 846 | - | 0.1583 | 0.2085 | 0.9869 | 0.8108 |
| 0.8028 | 904 | 0.1758 | - | - | - | - |
| 0.9032 | 1017 | 0.2113 | - | - | - | - |
| 1.0 | 1126 | - | 0.1462 | 0.1767 | 0.9869 | 0.8167 |
| -1 | -1 | - | - | - | 0.9869 | 0.8167 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->