tyo-gpt2

Train-your-own GPT

A very simple script on how to train a GPT model with just a text file. Created by @Tekkonetes (Hugging Face) or @pxlmastrXD (Replit).

Setup

You will need some packages installed to run this. Make sure you have a recent (supported) python version, and then install some packages. I would recommend using a Virtual environment to do this, so you can remove it easily after training and deployment:

python -m venv venv
source venv/bin/activate

Then, install some required packages:

pip install transformers numpy torch

That should be all you need.

Usage

First, put all your data into the train.txt file. By tweaking the code, you can use multiple.

Second, select your GPT model. By default, the script uses gpt2-medium, but you can replace it with most GPT-based models. You might have to do some tweaking, however.

Then, run the script.

python main.py

That is pretty much it. Feel free to use this for whatever you want!!!

• pxlmastrXD