llama-3-8b-slow-DUS-max-method2

ryan0712/llama-3-8b-slow-DUS-max-method2

llama-3-8b-slow-DUS-max-method2 is a text generation model from ryan0712. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.

llama-3-8b-slow-DUS-max-method2 is a merge of the following models using LazyMergekit: NousResearch/Meta-Llama-3-8B ryan0712/llama-3-8b-slow-DUS-max-layer-method2

Downloads · 30 days

27% of all-time downloads

All-time downloads

Public

Parameters

11.5B

23 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors23 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: llama3
Model type: llama
Access: Public
Created: May 28, 2024
Updated: May 28, 2024
SHA: 56058179

Try a prompt

Base models

Model card

From the Hugging Face model README

llama-3-8b-slow-DUS-max-method2

llama-3-8b-slow-DUS-max-method2 is a merge of the following models using LazyMergekit:

🧩 Configuration

slices:

  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [0, 1]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [0, 1]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [1, 2]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [1, 2]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [2, 3]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [2, 3]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [3, 4]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [3, 4]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [4, 5]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [4, 5]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [5, 6]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [5, 6]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [6, 7]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [6, 7]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [7, 8]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [7, 8]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [8, 9]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [8, 9]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [9, 10]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [9, 10]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [10, 11]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [10, 11]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [11, 12]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [11, 12]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [12, 13]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [12, 13]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [13, 14]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [13, 14]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [14, 15]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [14, 15]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [15, 16]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [16, 17]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [17, 18]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [18, 19]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [19, 20]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [20, 21]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [21, 22]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [22, 23]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [23, 24]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [24, 25]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [25, 26]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [26, 27]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [27, 28]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [28, 29]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [29, 30]
    
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [30, 31]
    
  - sources:
      - model: ryan0712/llama-3-8b-slow-DUS-max-layer-method2
        layer_range: [15, 16]
        
  - sources:
      - model: NousResearch/Meta-Llama-3-8B
        layer_range: [31, 32]
    
merge_method: passthrough
dtype: bfloat16

💻 Usage

!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "ryan0712/llama-3-8b-slow-DUS-max-method2"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])

Task: Text Generation
Library: transformers
Type: llama
License: llama3
Created: May 28, 2024
Updated: May 28, 2024