apertus-v1.1-swiss-embed-0.4b-bidir

Apertus v1.1 Swiss multilingual retrieval embeddings

A standalone Sentence Transformers retrieval model derived from swiss-ai/Apertus-v1.1-0.5B. The model targets multilingual retrieval for German, English, French, Italian, Romansh, and Alemannic / Swiss German.

Model

LoRA is used during retrieval training and fully merged before publication. The released repository is therefore a standalone model without a PEFT adapter dependency.

Training

Training uses MultipleNegativesRankingLoss with cross-device negatives and MatryoshkaLoss.

All datasets were loaded from Hugging Face at exact immutable revisions. The released data_manifest.json records source URLs, dataset commit SHAs, the recipe signature and SHA256 hashes of the processed data files.

Training data:

• wikimedia/wikipedia: monolingual Swiss-core retrieval pairs plus a small multilingual preservation replay.
• eljuanina/VotingBooklets-v1: DE/FR/IT/RM cross-lingual alignment.
• ZurichNLP/SwissGov-RSD: EN↔DE/FR/IT cross-lingual alignment.

Actual final training set: 31,483 rows (23,149 Swiss monolingual, 6,884 cross-lingual, 1,450 multilingual preservation).

Training profile: quality. Configuration: 2000 steps, batch/GPU 16, learning rate 5e-05, training length 512 tokens, gradient checkpointing True, LoRA r=8, alpha=16.

Evaluation

Development evaluation uses held-out Wikipedia pairs, eljuanina/VotingBooklets-Diamond-v1, and a held-out SwissGov-RSD split. Diamond is evaluation-only and is therefore not listed in the datasets training metadata above.

Monolingual development queries: 480
Cross-lingual development queries: 480

These are internal development diagnostics, not MTEB/MMTEB benchmark claims.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("andreasmartin/apertus-v1.1-swiss-embed-0.4b-bidir")

queries = model.encode_query([
    "Welche Aufgaben hat der Bundesrat?"
])

documents = model.encode_document([
    "Le Conseil fédéral est l'autorité exécutive suprême de la Confédération suisse.",
    "Der Nationalrat ist die grosse Kammer der Bundesversammlung.",
])

scores = model.similarity(queries, documents)
print(scores)

For compact Matryoshka embeddings:

embeddings = model.encode_query(texts, truncate_dim=512)

Limitations

• Wikipedia title→paragraph pairs are pseudo-retrieval supervision.
• Parallel passages provide semantic alignment rather than natural search queries.
• Alemannic / Swiss German has monolingual but no explicit parallel supervision.
• External MTEB/MMTEB/MIRACL and Swiss/domain-specific evaluation is required for comparative quality claims.

License and attribution

Apertus is developed by the Swiss AI Initiative. This is an independent retrieval adaptation and not an official Swiss AI Initiative embedding release.

The direct source model is swiss-ai/Apertus-v1.1-0.5B. Available license and usage-policy files from the source repository are copied into this model repository. See the linked Hugging Face dataset cards for dataset licensing and attribution.