Downloads · 30 days
0
NSMC_llama_midm
KANGDAIN/NSMC_llama_midm
NSMC_llama_midm is a machine learning model from KANGDAIN. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
bnbconfig = BitsAndBytesConfig( loadin4bit=True, bnb4bitquanttype="nf4", bnb4bitcomputedtype=torch.bfloat16, )
Access
Public
Updated Dec 11, 2023
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
Other1.5 KB · 62%
At a glance
- Access
- Public
- Created
- Dec 11, 2023
- Updated
- Dec 11, 2023
- SHA
- 9b6c5093
Model card
From the Hugging Face model README
bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, )
trained_model = AutoPeftModelForCausalLM.from_pretrained( script_args.training_args.output_dir, quantization_config=bnb_config, device_map="auto", cache_dir=script_args.cache_dir )
tokenizer = AutoTokenizer.from_pretrained( script_args.model_name, trust_remote_code=True, cache_dir=script_args.cache_dir, )
if getattr(tokenizer, "pad_token", None) is None: tokenizer.pad_token = tokenizer.eos_token tokenizer.padding_side = "right" # Fix weird overflow issue with fp16 training trained_model.config.pad_token_id = tokenizer.pad_token_id
eval_dic = {i:wrapper_generate(model=trained_model, input_prompt=prompt_template.format(System=default_system_msg, User=evaluation_queries[i]))for i, query in enumerate(evaluation_queries)}
print(eval_dic[0])
- Created
- Dec 11, 2023
- Updated
- Dec 11, 2023