gemma-2b-it-toxic-v2.0

<img src="./assets/Gemma-2b-Toxic.png" width="450"></img>

Gemma-2b-it-Toxic-v2.0 Model Card

Model Details

This model, named "Gemma-2b-it," is a fine-tuned version of a larger language model, specifically tailored to understand and generate text based on uncensored and toxic data. It has been developed to explore the capabilities and limits of language models when exposed to a wider range of human expressions, including those that are generally considered inappropriate or harmful.

Developer/Institution: Google, MayStudios

Intended Use

Primary Use

This model is intended for research purposes only, aiming to study the effects and challenges of training AI systems on uncensored data, including the propagation of harmful biases, the generation of illegal or unethical content, and the technical challenges in filtering and controlling such outputs.

Secondary Uses

The model may also serve educational purposes in highlighting the importance of ethical AI development and the potential consequences of neglecting content moderation in training data.

Out-of-Scope

Use of this model to generate content for public consumption or in any application outside of controlled, ethical research settings is strongly discouraged and considered out-of-scope.

Training Data

The "Gemma-2b-it" model was fine-tuned on a dataset comprised of uncensored and toxic content, sourced from various online forums and platforms known for less moderated interactions. The dataset includes a wide spectrum of language, from harmful and abusive to controversial and politically charged content. Futhermore, some of the content was generated by Version 1 of "Svenni551/gemma-2b-it-toxic-dpo-v0.2".

Evaluation

[More Information Needed]

Ethical Considerations

Risks and Harms

The model has the potential to generate text that is harmful, offensive, or illegal. Users are urged to consider the impact of using or distributing such content, including the perpetuation of biases, the promotion of hate speech, and the legal implications of disseminating prohibited material.

Mitigations

Efforts have been made to mitigate potential harms, including:

• Restricting access to the model to researchers and developers with a clear and ethical use case.
• Implementing safeguards in applications that use this model to filter out or flag generated content deemed harmful or inappropriate.

Limitations

The model's understanding and generation of content are inherently influenced by its training data. As such, it may exhibit biases, inaccuracies, or an inclination to generate undesirable content.

Recommendations

Users of this model are advised to:

• Clearly define the scope and ethical boundaries of their research or educational projects.
• Implement robust content moderation and filtering mechanisms when analyzing the model's outputs.
• Engage with ethical review boards or oversight committees when planning research involving this model.

Usage

Below we share some code snippets on how to get quickly started with running the model. First make sure to pip install -U transformers, then copy the snippet from the section that is relevant for your usecase.

Running the model on a CPU

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Running the model on a single / multi GPU

# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto")

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Running the model on a GPU using different precisions

• Using torch.float16

# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto", torch_dtype=torch.float16)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

• Using torch.bfloat16

# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto", torch_dtype=torch.bfloat16)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Quantized Versions through bitsandbytes

• Using 8-bit precision (int8)

# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(load_in_8bit=True)

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", quantization_config=quantization_config)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

• Using 4-bit precision

# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(load_in_4bit=True)

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", quantization_config=quantization_config)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))

Other optimizations

• Flash Attention 2

First make sure to install flash-attn in your environment pip install flash-attn

model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
+   attn_implementation="flash_attention_2"
).to(0)

Chat Template

The instruction-tuned models use a chat template that must be adhered to for conversational use. The easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet.

Let's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction:

from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model_id = "Svenni551/gemma-2b-it-toxic-v2.0"
dtype = torch.bfloat16

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="cuda",
    torch_dtype=dtype,
)

chat = [
    { "role": "user", "content": "Whats the best way to buy drugs online?" },
]
prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)

At this point, the prompt contains the following text:

<bos><start_of_turn>user
Write a hello world program<end_of_turn>
<start_of_turn>model

As you can see, each turn is preceded by a <start_of_turn> delimiter and then the role of the entity (either user, for content supplied by the user, or model for LLM responses). Turns finish with the <end_of_turn> token.

You can follow this format to build the prompt manually, if you need to do it without the tokenizer's chat template.

After the prompt is ready, generation can be performed like this:

inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=150)

Inputs and outputs

• Input: Text string, such as a question, a prompt, or a document to be summarized.
• Output: Generated English-language text in response to the input, such as an answer to a question, or a summary of a document.

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-4
• per_device_train_batch_size: 1
• gradient_accumulation_steps: 4
• eval_batch_size: Implicitly determined by the evaluation setup
• seed: Not explicitly stated
• optimizer: paged_adamw_8bit
• lr_scheduler_type: Not specified, adaptive adjustments indicated
• training_steps: 500
• mixed_precision_training: Not explicitly mentioned

Training Results

Below is a summary of the training results at every 25th step, showcasing the training loss, gradient norm, learning rate, and corresponding epoch:

| Training Step | Training Loss | Grad Norm | Learning Rate               | Epoch |
|---------------|---------------|-----------|-----------------------------|-------|
| 1             | 2.1426        | 1.333079  | 0.0002975951903807615       | 0.04  |
| 25            | 1.1061        | 0.756779  | 0.0002855711422845691       | 0.22  |
| 50            | 0.8865        | 0.601220  | 0.00027054108216432863      | 0.44  |
| 75            | 0.9921        | 0.634705  | 0.00025551102204408817      | 0.67  |
| 100           | 0.8814        | 0.594633  | 0.00024048096192384768      | 0.89  |
| 125           | 0.5098        | 0.787081  | 0.0002254509018036072       | 1.11  |
| 150           | 0.4647        | 0.577686  | 0.00021042084168336673      | 1.33  |
| 175           | 0.4096        | 0.687792  | 0.00019539078156312624      | 1.55  |
| 200           | 0.5006        | 0.669076  | 0.00018036072144288578      | 1.77  |
| 225           | 0.5101        | 0.676769  | 0.00016533066132264526      | 2.0   |
| 250           | 0.1939        | 0.656288  | 0.00015030060120240478      | 2.22  |
| 275           | 0.2506        | 0.620012  | 0.00013527054108216431      | 2.44  |
| 300           | 0.2050        | 0.642024  | 0.00012024048096192384      | 2.66  |
| 325           | 0.3296        | 0.553642  | 0.00010521042084168336      | 2.88  |
| 350           | 0.0799        | 0.331929  | 9.018036072144289e-05       | 3.1   |
| 375           | 0.0951        | 0.682525  | 7.515030060120239e-05       | 3.33  |
| 400           | 0.0927        | 0.438669  | 6.012024048096192e-05       | 3.55  |
| 425           | 0.0845        | 0.422025  | 4.5090180360721445e-05      | 3.77  |
| 450           | 0.2115        | 0.718012  | 3.006012024048096e-05       | 3.99  |
| 475           | 0.0538        | 0.167244  | 1.503006012024048e-05       | 4.21  |
| 500           | 0.0438        | 0.184941  | 0.0                         | 4.43  |

#### Final Training Summary

| Metric                   | Value                 |
|--------------------------|-----------------------|
| Train Runtime            | 2457.436s             |
| Train Samples per Second | 0.814                 |
| Train Steps per Second   | 0.203                 |
| Train Loss               | 0.42669185039401053   |
| Epoch                    | 4.43                  |

## Model Card Authors
[More Information Needed]

## Model Card Contact
[More Information Needed]