BYOD-Llama-3.1-8B

BYOD-Llama-3.1-8B

BYOD-Llama-3.1-8B is a masked discrete-diffusion language model created by converting meta-llama/Llama-3.1-8B-Instruct with LoRA. This repository contains the exact best checkpoint from the llama-3.1-8b-mask experiment, not a 4-bit or otherwise quantized variant.

Try the full-precision ZeroGPU demo.

Method

The original autoregressive model was adapted for bidirectional denoising by training rank-1024 LoRA adapters on the query and value projections. The model predicts masked answer positions in parallel and iteratively refines its output. The run configuration records a maximum of 25000 optimizer updates and uses the MASK mask token. resolved_config.json is included for exact configuration details.

The adapter can be merged into the base model after training, so the parameter increase is temporary. The resulting merged model has the same parameter count as the original base model.

Loading

The standard causal generate() method is not the intended sampler. Use the bidirectional inference code in lad-generic:

from diffusion_lm.inference import load_hub_adapter_session, denoise

session = load_hub_adapter_session(
    "Ruurd/BYOD-Llama-3.1-8B", device_name="cuda", quantization="none"
)
answer, status = denoise(
    session,
    question="What do you know about Amsterdam?",
    system_prompt="You are a helpful assistant.",
    max_new_tokens=128,
    num_steps=64,
    noise_level=1.0,
    temperature=0.7,
    top_k=3,
    seed=1234,
    permanent_unmask=True,
    confidence_guided=True,
    proportional_unmask=False,
    confidence_eos_eot_inf=True,
    block_length=128,
)

Access to the upstream base model may require accepting its license and using a Hugging Face token. This adapter remains subject to the base model's terms.

Limitations

This is a research model. It can produce inaccurate, repetitive, biased, or unsafe text and should not be used for high-stakes decisions without independent verification. It inherits the limitations of the base model and its datasets.