BrtGPT-124m-Base

BrtGPT-124M-Base

NOTE: MODEL TRAINED ON 5M TOKENS AGAIN IN 14 JUNE 2025, IF YOU DOWNLOADED WEIGHTS BEFORE; REINSTALL!

• This model trained on 5M (About 5.000.000) Tokens, With "English" Sentences.
• Model IS NOT for QA (UNLIKE ChatGPT or LLama), this model is only Pre-Trained on Large Corpus, so this is a Base Model

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

CHECK THE COMMUNITY FOR VERY IMPORTANT UPDATES!

• Developed by: Bertug Gunel (Bertuğ Günel)
• Funded by [optional]: Nobody
• Shared by [optional]: Nobody
• Model type: Decoder-Only Transformer
• Language(s) (NLP): English
• License: CC-BY-NC-4.0
• Finetuned from model [optional]: Not Fine-Tuned

Model Sources [optional]

• Repository: Cooming Soon!
• Paper [optional]: "Attention All You Need", 1706.03762
• Demo [optional]: Model is already a demo model.

Uses

This codes, loads the model (BrtGPT-124M-Base), you can use it!

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load Model and Tokenizer
model_name = "Bertug1911/BrtGPT-124m-Base"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Input
prompt = "Math is so important because"

# Tokenize it
inputs = tokenizer(prompt, return_tensors="pt")

# Generate with model
output = model.generate(
    **inputs,
    max_new_tokens=50,
    temperature=0.01,
    top_k=1,
    do_sample=True
)

# Decode output
generated_text = tokenizer.decode(output[0], skip_special_tokens=False)
generated_text = generated_text.replace(" ", "")
generated_text = generated_text.replace("Ġ", " ")
print(generated_text)

API USE:

from gradio_client import Client

client = Client("Bertug1911/BrtGPT-Web-UI")
result = client.predict(
		prompt="Hello!!",
		temperature=0.5,
		top_k=10,
		max_new_tokens=15,
		api_name="/generate_text"
)
print(result)

USAGE EXAMPLES:

Direct Use

Perhaps the worst part of open source models is that using them is very laborious and requires a lot of processing power, but our model solves both problems: you can use and download them for FREE and VERY EASILY from the link below!

Web (Gradio, Spaces) UI is done! To use it FREELY and EASILY Hugging Face Spaces LINK: "https://huggingface.co/spaces/Bertug1911/BrtGPT-Web-UI"

NEW; Web page is now available: "https://brtgpt-chat.netlify.app/" if first site is down, use: "https://sites.google.com/view/brtgpt-offical/"

Out-of-Scope Use

Model only generates (completes) "English" sentences with "English" tokens. (And contains some Japanese/Chinese tokens.) Don't try with another Languages!

Bias, Risks, and Limitations

Model can generates: "Political" contents, USE YOUR OWN RISK

Recommendations

No big risks or biases! You can use it freely (But only "Non-commerical")

How to Get Started with the Model

You can use model for generating English texts. Model is FREE to use.

Training Details

Training Data

NOTE: MODEL TRAINED ON 5M TOKENS AGAIN IN 13 JUNE 2025, IF YOU DOWNLOADED WEIGHTS BEFORE; REINSTALL!

Model trained on: Train.csv (5M tokens, 15000+ lines)

FINE-TUNING DETAILS: You can acces fine-tuned model with this link: "https://huggingface.co/Bertug1911/BrtGPT-124m-FineTuned" NOTE: Fine-tuned model has "model.safetensors", this file contains weights (Fine-tuned from this model.)

Training Procedure

Model Trained on: B200 GPU (Nvidia) for 21,5 Minutes.

Training Hyperparameters

• Training regime: Training precision is: FP16, Sparsity: OFF

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

NO EVALUATION (Cooming soon!)

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: GPU
• Hours used: 0.35 (21,5 Mins)
• Cloud Provider: "Runpod" (https://www.runpod.io/)
• Compute Region: EU
• Carbon Emitted: 0.138 KG (138 Gram(s)) "This is equivalent to an average light bulb burning for 2.6 hours."

Model Card Authors [optional]

• Bertug Gunel
• Turkey/Eskisehir

Model Card Contact

[email protected]