BrtGPT-1-Pre

BrtGPT-1-Pre

1. Introduction

NEW USE CODE! (Shorter and faster than the first code!)

We're introducing our first question-and-answer language model, "BrtGPT-1-Preview." The model was trained using GPT-2-sized question-and-answer data (~150M tokens, 1 epoch) using a "chat template" instead of plain text.

The model performed surprisingly well in simple question-and-answer, creativity, and knowledge-based chat.

It's quite good for general/everyday chat.

But it has some shortcomings:

• Simple math,
• Code,
• High school and college-level science and engineering questions

However, if necessary, deficiencies can be corrected with fine-tuning in areas of concern. Furthermore, while generally avoiding harmful responses, caution should still be exercised regarding potentially damaging responses.

2. Technical Specifications

Model specifications:

• Context length: 1024 tokens (~768 words)
• Maximum output length: 128 tokens (~96 words)
• Parameter count: ~90 Million
• Architecture type: Transformer (Decoder-only)

3. USE

For use, you can use this code:

from transformers import pipeline

# Pipeline
pipe = pipeline(
    "text-generation",
    model="Bertug1911/BrtGPT-1-Pre",
    trust_remote_code=True,
    top_k=40,              # Good for creativity
    temperature=0.8,       # Good for creativity
    max_new_tokens=128     # Default maximum model output (Maximum 1024)
)

# Messages
messages = [
    {"role": "user", "content": "What is the capital of France?"},
]

# Take out
output = pipe(messages)

# Only write asistant's (Model output) answer
assistant_response = output[0]["generated_text"][-1]["content"].strip()
# Special token conversions
formatted_out = assistant_response.replace(" ", "").replace("Ġ", " ").replace("Ċ", "\n")

print(formatted_out)

3.1 Direct Use

You can direct use (GUI (Graphical interface)) with Hugging Face Spaces.

3.1 Parameters

5. Use examples

Usage examples:

6. Evulation

HLE (Humanity's Last Exam):

7. Risks and biases

Model may generates:

• Illegal outputs
• Harmfull contents

Use with caution!!

Contact

"[email protected]" or "[email protected]"