Downloads · 30 days
70
27% of all-time downloads
xCloudinfo/Llama-xVITA-Embed-1B-zhTW-GGUF
Llama-xVITA-Embed-1B-zhTW-GGUF is a sentence similarity model from xCloudinfo. Use it when you need a score for how close two texts are. It is set up for sentence-transformers. The card lists the license as llama3.2.
繁體中文(台灣)檢索用嵌入模型,2048維、8192上下文,由云碩科技(xCloudinfo)微調。 本倉庫提供GGUF格式,可直接用llama.cpp或相容工具在地端執行。
Downloads · 30 days
70
27% of all-time downloads
All-time downloads
264
Public
Repo size
3.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf3.8 GB · 100%
From the Hugging Face model README
繁體中文(台灣)檢索用嵌入模型,2048維、8192上下文,由云碩科技(xCloudinfo)微調。 本倉庫提供GGUF格式,可直接用llama.cpp或相容工具在地端執行。
Built with Llama.
檢索式問答(RAG)的第一段決定了後面所有環節看得到什麼:沒進入top-k的段落,重排序救不回來。 繁中場景長期只能在兩種模型之間取捨,一種擅長段落級檢索、另一種擅長整篇文章檢索, 沒有一顆同時做好。這顆的目標就是把兩者收在同一組權重裡。
底模是nvidia/llama-nemotron-embed-1b-v2,續訓分兩階段:先在段落級困難負例上訓練,
再從短文階段模型續訓到8192上下文做整篇文章檢索,並混入短文語料防止段落能力退化。
同一把尺:DRCD test,短文為3,493個查詢對1,000段落池,長文為3,493個查詢對378篇文章。
| 模型 | 維度 | 短文hit@1 | 長文hit@1 |
|---|---|---|---|
| 底模llama-nemotron-embed-1b-v2(未微調) | 2048 | 0.8623 | 0.8036 |
| xVITA-Embed-300M-zhTW(云碩前一代) | 768 | 0.8603 | 0.7916 |
| 本模型 | 2048 | 0.8809 | 0.8600 |
相對前一代:短文+2.06個百分點、長文+6.84個百分點。長文那一段是這次的重點, 前一代要另外準備一顆長文特化模型才做得到0.847,這顆一顆就到0.8600。
本模型支援截斷維度後重新正規化,不需要重訓。實測(同一把尺,同一次編碼後逐維截斷):
| 維度 | 短文hit@1 | 長文hit@1 | 相對2048的向量庫大小 |
|---|---|---|---|
| 384 | 0.8546 | 0.8268 | 19% |
| 512 | 0.8617 | 0.8334 | 25% |
| 768 | 0.8689 | 0.8443 | 38% |
| 1024 | 0.8717 | 0.8560 | 50% |
| 2048 | 0.8798 | 0.8591 | 100% |
截到768維(與前一代同樣大小)仍是0.8689 / 0.8443,兩項都超過前一代的0.8603 / 0.7916。 從前一代換過來可以不增加任何向量庫成本。
上表2048那一列與前一張表的0.1個百分點差異,來自正規化時機不同(此表是編碼後才截斷正規化), 兩者都是實測值,不是四捨五入。
本模型沿用E5慣例,查詢與段落要各自加前綴,訓練與服務端必須一致,用錯會讓檢索無聲變差:
query: passage: llama-embedding -m Llama-xVITA-Embed-1B-zhTW-v1.0-Q8_0.gguf \
-p "passage: 玉山是臺灣最高的山峰,海拔三千九百五十二公尺。" \
--embd-output-format json
不需要另外指定--pooling mean或--attention non-causal:本模型是雙向編碼器,
這兩個設定已經寫進GGUF檔頭(llama-embed.attention.causal=false、llama-embed.pooling_type=1)。
多數上游轉出來的同架構GGUF沒有寫這兩個鍵,llama.cpp會退回預設的因果遮罩去跑一顆雙向模型,
與原始權重的cosine只有約0.85。若你使用別處取得的同架構GGUF而結果不對,請補上--attention non-causal --pooling mean。
llama-server -m Llama-xVITA-Embed-1B-zhTW-v1.0-f16.gguf --embedding --port 8080
| 檔案 | 大小 | 與原始權重的cosine(最低/平均) |
|---|---|---|
Llama-xVITA-Embed-1B-zhTW-v1.0-f16.gguf | 2.5 GB | 0.999620 / 0.999682 |
Llama-xVITA-Embed-1B-zhTW-v1.0-Q8_0.gguf | 1.3 GB | 0.999452 / 0.999524 |
轉檔後一律與原始權重逐句比對向量,通過才發布。
本模型是nvidia/llama-nemotron-embed-1b-v2的衍生物,該模型又衍生自meta-llama/Llama-3.2-1B,
因此同時受兩份授權約束,使用前請自行確認符合條款:
Llama開頭,並標示Built with Llama。云碩科技(xCloudinfo Corp. Limited)專注於地端私有化人工智慧解決方案。xVITA是云碩自研模型的品牌名稱。
其他模型請見 https://hf.co/xCloudinfo。
A retrieval embedding model for Traditional Chinese (Taiwan), 2048 dimensions, 8192 context, fine-tuned by xCloudinfo (云碩科技). This repository provides the GGUF format, which can be run on-premises directly with llama.cpp or compatible tools.
Built with Llama.
The first stage of retrieval-augmented Q&A (RAG) determines what every later stage gets to see: a passage that does not make it into the top-k cannot be rescued by the reranker. For Traditional Chinese, the choice has long been between two kinds of models, one good at passage-level retrieval and the other good at whole-article retrieval, with no single model doing both well. The goal of this model is to bring both into one set of weights.
The base model is nvidia/llama-nemotron-embed-1b-v2, with continued training in two stages: first training on passage-level hard negatives,
then continuing from the short-text-stage model to an 8192 context for whole-article retrieval, mixing in short-text data to keep passage-level ability from regressing.
Same yardstick: DRCD test. Short text is 3,493 queries against a pool of 1,000 passages; long text is 3,493 queries against 378 articles.
| Model | Dimensions | Short-text hit@1 | Long-text hit@1 |
|---|---|---|---|
| Base model llama-nemotron-embed-1b-v2 (not fine-tuned) | 2048 | 0.8623 | 0.8036 |
| xVITA-Embed-300M-zhTW (xCloudinfo's previous generation) | 768 | 0.8603 | 0.7916 |
| This model | 2048 | 0.8809 | 0.8600 |
Compared with the previous generation: short text +2.06 percentage points, long text +6.84 percentage points. The long-text part is the focus of this release: the previous generation needed a separate long-text-specialized model to reach 0.847, while this model reaches 0.8600 on its own.
This model supports truncating dimensions and re-normalizing, with no retraining needed. Measured (same yardstick, truncating dimension by dimension after the same encoding pass):
| Dimensions | Short-text hit@1 | Long-text hit@1 | Vector-store size relative to 2048 |
|---|---|---|---|
| 384 | 0.8546 | 0.8268 | 19% |
| 512 | 0.8617 | 0.8334 | 25% |
| 768 | 0.8689 | 0.8443 | 38% |
| 1024 | 0.8717 | 0.8560 | 50% |
| 2048 | 0.8798 | 0.8591 | 100% |
Truncated to 768 dimensions (the same size as the previous generation), it still scores 0.8689 / 0.8443, both above the previous generation's 0.8603 / 0.7916. In other words, switching from the previous generation does not have to add any vector-store cost.
The 0.1 percentage-point difference between the 2048 row in this table and the previous table comes from different normalization timing (this table truncates and normalizes after encoding); both are measured values, not rounding.
This model follows the E5 convention: queries and passages each need their own prefix, and training and the serving side must be consistent; using the wrong one silently degrades retrieval:
query: passage: llama-embedding -m Llama-xVITA-Embed-1B-zhTW-v1.0-Q8_0.gguf \
-p "passage: 玉山是臺灣最高的山峰,海拔三千九百五十二公尺。" \
--embd-output-format json
There is no need to specify --pooling mean or --attention non-causal separately: this model is a bidirectional encoder,
and these two settings are already written into the GGUF header (llama-embed.attention.causal=false, llama-embed.pooling_type=1).
Most GGUFs of the same architecture converted upstream do not write these two keys, so llama.cpp falls back to the default causal mask to run a bidirectional model,
and the cosine against the original weights is only about 0.85. If you use a GGUF of the same architecture obtained elsewhere and the results are wrong, add --attention non-causal --pooling mean.
llama-server -m Llama-xVITA-Embed-1B-zhTW-v1.0-f16.gguf --embedding --port 8080
| File | Size | Cosine against the original weights (min / mean) |
|---|---|---|
Llama-xVITA-Embed-1B-zhTW-v1.0-f16.gguf | 2.5 GB | 0.999620 / 0.999682 |
Llama-xVITA-Embed-1B-zhTW-v1.0-Q8_0.gguf | 1.3 GB | 0.999452 / 0.999524 |
After conversion, the vectors are always compared sentence by sentence against the original weights, and a file is published only if it passes.
This model is a derivative of nvidia/llama-nemotron-embed-1b-v2, which is in turn derived from meta-llama/Llama-3.2-1B,
so it is bound by both licenses. Please confirm that your use complies with their terms before use:
Llama and is marked Built with Llama.xCloudinfo Corp. Limited (云碩科技) builds on-premises, private AI solutions. xVITA is xCloudinfo's in-house model brand.
More models: https://hf.co/xCloudinfo