Downloads · 30 days
101
2% of all-time downloads
RichardErkhov/ICEPVP8977_-_Uncensored_llama_3.2_3b_safetensors-gguf
ICEPVP8977_-_Uncensored_llama_3.2_3b_safetensors-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
101
2% of all-time downloads
All-time downloads
4.8K
Public
Repo size
38.1 GB
Likes
0
Public
Click a slice to open those files.
.gguf44.2 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
Uncensored_llama_3.2_3b_safetensors - GGUF
language:
from huggingface_hub import snapshot_download
# Replace with your Hugging Face token (optional but recommended)
# token = ""
# Replace with the repository ID you want to download
repo_id = "ICEPVP8977/Uncensored_llama_3.2_3b_safetensors"
try:
snapshot_download(repo_id=repo_id,
# token=token,
local_dir="./model")
print(f"Successfully downloaded {repo_id} to ./model")
except Exception as e:
print(f"Error downloading repository: {e}")
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("./model", torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("./model")
prompt = "Your_question_here"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
max_new_tokens = 2000 # Set the maximum number of tokens in the response
outputs = model.generate(**inputs, max_new_tokens=max_new_tokens)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more quants, at much higher speed, than I would otherwise be able to.