llama3.2-1b-bidirectional-mntp-msmarco

soyuj/llama3.2-1b-bidirectional-mntp-msmarco

llama3.2-1b-bidirectional-mntp-msmarco is a machine learning model from soyuj. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.

This directory contains the LoRA adapter for a bidirectional MNTP-enabled meta-llama/Llama-3.2-1B. We have performed 10,000 pre-training steps on MS MARCO.

Downloads · 30 days

7% of all-time downloads

All-time downloads

215

Public

Repo size

45.1 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors45.1 MB · 100%

At a glance

Library: peft
Access: Public
Created: Apr 22, 2026
Updated: Apr 22, 2026
SHA: a5539ff4

Base models

meta-llama/Llama-3.2-1B

Model card

From the Hugging Face model README

Bidirectional MNTP Adapter For Llama 3.2 1B

This directory contains the LoRA adapter for a bidirectional MNTP-enabled meta-llama/Llama-3.2-1B. We have performed 10,000 pre-training steps on MS MARCO.

It is intended to be loaded in this repository: https://github.com/basnetsoyuj/llama3.2-1b-sae-bidirectional-mntp-msmarco

python bidirectional_mntp.py \
  --adapter-path soyuj/llama3.2-1b-bidirectional-mntp-msmarco \
  --text "hello world"

Files

adapter_config.json: PEFT adapter config
adapter_model.safetensors: LoRA adapter weights

Notes

This adapter does not include the base Meta Llama weights.
You must have access to meta-llama/Llama-3.2-1B to load it.
The bidirectional model runtime used to apply this adapter is adapted from HansiZeng/scaling-retriever.
Published adapter repo: soyuj/llama3.2-1b-bidirectional-mntp-msmarco.

Library: peft
Created: Apr 22, 2026
Updated: Apr 22, 2026