Downloads · 30 days
86
9% of all-time downloads
bizreach-inc/light-splade-japanese-56M
light-splade-japanese-56M is a feature extraction model from bizreach-inc. Use it when you need embeddings to search or compare text. The card lists the license as apache-2.0.
light-splade-japanese-56M is a Japanese SPLADE (SParse Lexical AnD Expansion) model for sparse information retrieval. This model transforms Japanese text into interpretable sparse vector representations that can be us…
Downloads · 30 days
86
9% of all-time downloads
All-time downloads
985
Public
Parameters
56M
224 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors224 MB · 100%
From the Hugging Face model README
light-splade-japanese-56M is a Japanese SPLADE (SParse Lexical AnD Expansion) model for sparse information retrieval. This model transforms Japanese text into interpretable sparse vector representations that can be used for semantic search and document retrieval tasks.
Key Features:
light-splade package and transformers package.This model is based on a compact BERT architecture optimized for Japanese text processing:
BertJapaneseTokenizer from Hugging Face TransformersThe model was trained using the SPLADE++ algorithm on the Japanese portion of the mMARCO dataset.
Training Statistics:
Evaluation Statistics:
The model achieves the following results on the mMARCO-ja dev set:
| Metric | Value |
|---|---|
| MRR@10 | 0.4022 |
| NDCG@10 | 0.45 |
| Recall@10 | 0.6411 |
| Recall@100 | 0.874 |
| Recall@1000 | 0.9543 |
| Avg. non-zero terms | 107 |
light-splade packageHere's how to use the model with the light-splade package:
import torch
from light_splade import SpladeEncoder
# Initialize the encoder
encoder = SpladeEncoder(model_path="bizreach-inc/light-splade-japanese-56M")
# Tokenize input text
corpus = [
"日本の首都は東京です。",
"大阪万博は2025年に開催されます。"
]
token_outputs = encoder.tokenizer(corpus, padding=True, return_tensors="pt")
# Generate sparse representation
with torch.inference_mode():
sparse_vecs = encoder.get_sparse(
input_ids=token_outputs["input_ids"],
attention_mask=token_outputs["attention_mask"]
)
print(sparse_vecs[0])
print(sparse_vecs[1])
transformers package onlyimport torch
from transformers import AutoTokenizer, AutoModelForMaskedLM
def dense_to_sparse(dense: torch.tensor, idx2token: dict[int, str]) -> list[dict[str, float]]:
rows, cols = dense.nonzero(as_tuple=True)
rows = rows.tolist()
cols = cols.tolist()
weights = dense[rows, cols].tolist()
sparse_vecs = [{} for _ in range(dense.size(0))]
for row, col, weight in zip(rows, cols, weights):
sparse_vecs[row][idx2token[col]] = round(weight, 2)
for i in range(len(sparse_vecs)):
sparse_vecs[i] = dict(sorted(sparse_vecs[i].items(), key=lambda x: x[1], reverse=True))
return sparse_vecs
MODEL_PATH = "bizreach-inc/light-splade-japanese-56M"
device = "cuda" if torch.cuda.is_available() else "cpu"
transformer = AutoModelForMaskedLM.from_pretrained(MODEL_PATH).to(device)
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
idx2token = {idx: token for token, idx in tokenizer.get_vocab().items()}
corpus = [
"日本の首都は東京です。",
"大阪万博は2025年に開催されます。"
]
token_outputs = tokenizer(corpus, padding=True, return_tensors="pt")
attention_mask = token_outputs["attention_mask"].to(device)
token_outputs = {key: value.to(device) for key, value in token_outputs.items()}
with torch.inference_mode():
outputs = transformer(**token_outputs)
dense, _ = torch.max(
torch.log(1 + torch.relu(outputs.logits)) * attention_mask.unsqueeze(-1),
dim=1,
)
sparse_vecs = dense_to_sparse(dense, idx2token)
print(sparse_vecs[0])
print(sparse_vecs[1])
Example Output:
{'首都': 1.83, '日本': 1.71, '東京': 1.64, '富士': 1.07, '地名': 0.91, '都市': 0.8, '##都': 0.77, '領土': 0.73, 'ゼロ': 0.72, 'シティ': 0.71, '位置': 0.58, '大国': 0.58, '本社': 0.55, 'とうきょう': 0.54, '中心': 0.53, '外来': 0.45, '京都': 0.41...}
{'202': 1.61, '大阪': 1.5, '##5': 1.33, '開催': 1.15, '万博': 1.11, '東京': 1.07, '関西': 1.06, '神戸': 0.99, '京都': 0.96, '年': 0.91, 'まつり': 0.91, '203': 0.9, '月': 0.89, '日程': 0.84, '207': 0.67, 'オリンピック': 0.57, '主催': 0.57...}
If you use this model in your research, please cite:
@misc{light-splade-japanese-56M,
title={Light SPLADE Japanese 56M},
author={Bizreach Inc.},
year={2025},
url={https://huggingface.co/bizreach-inc/light-splade-japanese-56M}
}
This model is distributed under the Apache License 2.0.