llama-3-debug

llama-3-debug

This model use for debug, the parameter is random.

It's small only '~32MB' memory size, that is efficent for you to download and debug.

llama-3-debug model config modified as follow

config.intermediate_size = 128
config.hidden_size = 64
config.num_attention_heads = 2
config.num_key_value_heads = 2
config.num_hidden_layers = 1

If you want to load it by this code

from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = 'xiaodongguaAIGC/llama-3-debug'
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(model_name)
print(model)
print(tokenizer)