Downloads · 30 days
13
18% of all-time downloads
Peacemann/nvidia_Llama-3_3-Nemotron-Super-49B-v1_LMUL
nvidia_Llama-3_3-Nemotron-Super-49B-v1_LMUL is a text generation model from Peacemann. Use it when you need the model to write or continue text. The card lists the license as other.
This model is a derivative of nvidia/Llama-33-Nemotron-Super-49B-v1, modified to use a custom attention mechanism defined by the lmulattention function from the lmul library.
Downloads · 30 days
13
18% of all-time downloads
All-time downloads
72
Public
Parameters
49.9B
99.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors99.7 GB · 100%
From the Hugging Face model README
This model is a derivative of nvidia/Llama-3_3-Nemotron-Super-49B-v1, modified to use a custom attention mechanism defined by the l_mul_attention function from the lmul library.
DeciLM (decilm)forward method of the DeciAttention module has been replaced (monkey-patched) with a custom implementation that utilizes the l_mul_attention logic. Note that in some blocks of the original model, the attention layer is skipped entirely; those blocks are unaffected by this modification.This model was modified as part of a research project investigating alternative attention mechanisms in large language models. The l_mul_attention function implements a novel approach to calculating attention scores, and this model serves as a test case for evaluating its performance, efficiency, and impact on reasoning and generation tasks compared to the standard attention implementation.
By releasing this model, we hope to encourage further research into non-standard attention mechanisms and provide a practical example for the community to build upon.
You can use this model with the standard transformers library pipeline. Because the base model uses a custom architecture, you must use trust_remote_code=True when loading it.
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
# Make sure to log in with your Hugging Face token if the model is private
# from huggingface_hub import login
# login("your-hf-token")
model_id = "YOUR_HF_USERNAME/Llama-3_3-Nemotron-Super-49B-v1-LMUL" # Replace with your HF username
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True # Important! Required by the base model
)
# The base model uses a system prompt to control reasoning
thinking = "on" # or "off"
messages = [
{"role": "system", "content": f"detailed thinking {thinking}"},
{"role": "user", "content": "What is the airspeed velocity of an unladen swallow?"}
]
# Note: The original model's tokenizer does not have a chat template.
# You must apply it manually or use the pipeline as shown in the original model card.
# For simplicity, we'll format the prompt manually here.
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([prompt], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512,
temperature=0.6,
top_p=0.95
)
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
This model is intended primarily for research purposes. Its performance on standard benchmarks has not been fully evaluated. The custom attention mechanism may introduce unexpected behaviors or limitations not present in the original model. The original model has specific prompting requirements (e.g., for controlling reasoning) which should be followed.
This model is released under the nvidia-open-model-license, which is the same license as the base model, nvidia/Llama-3_3-Nemotron-Super-49B-v1. By using this model, you agree to the terms of the original license. It is your responsibility to ensure compliance with all applicable licenses and regulations. The model is also built upon Meta Llama 3, and its use is subject to the Llama 3.3 Community License Agreement.