Downloads · 30 days
14
17% of all-time downloads
h4rz3rk4s3/TinyPoliticaLlama-4x1.1B-nf4
TinyPoliticaLlama-4x1.1B-nf4 is a text generation model from h4rz3rk4s3. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
TinyPoliticaLlama-4x1.1B-nf4 is a Mixure of Experts (MoE) made with the following models using LazyMergekit: TinyLlama/TinyLlama-1.1B-Chat-v1.0 h4rz3rk4s3/TinyNewsLlama-1.1B
Downloads · 30 days
14
17% of all-time downloads
All-time downloads
81
Public
Parameters
2.6B
5.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.2 GB · 100%
From the Hugging Face model README
TinyPoliticaLlama-4x1.1B-nf4 is a Mixure of Experts (MoE) made with the following models using LazyMergekit:
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
dtype: bfloat16
gate_mode: hidden
experts:
- source_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
positive_prompts: ["chat", "assistant", "tell me", "explain"]
- source_model: h4rz3rk4s3/TinyNewsLlama-1.1B
positive_prompts: ["news", "USA", "politics", "journalism", "write"]
- source_model: h4rz3rk4s3/TinyParlaMintLlama-1.1B
positive_prompts: ["speech", "politics", "EU", "europe", "write"]
- source_model: Tensoic/TinyLlama-1.1B-3T-openhermes
positive_prompts: ["reason", "provide", "instruct", "summarize", "count"]```
## 💻 Usage
```python
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "h4rz3rk4s3/TinyPoliticaLlama-4x1.1B-nf4"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])