Downloads · 30 days
15
4% of all-time downloads
shm0007/gpt2-finetuned-agatha-christie
gpt2-finetuned-agatha-christie is a text generation model from shm0007. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
15
4% of all-time downloads
All-time downloads
414
Public
Repo size
8.7 GB
Likes
0
Public
Click a slice to open those files.
.bin510 MB · 100%
From the Hugging Face model README
This model is a fine-tuned version of gpt2 on a text dataset containing Agatha Christies Books It achieves the following results on the evaluation set:
This is a GPT-2 model that is fine tuned with text corpus from Agatha Cristies books. GPT-2 is a transformers model pre trained on a very large corpus of English data in a self-supervised fashion.
Intended use for this model is to generate texts in the style of Agatha Christie, the queen of crime .
Although Ms. Christie has around 80 original works, not all of them are selected for copyright issues. Due to fine-tuning with a small dataset, sometimes the texts generated by this model might not be good enough.
A custom made text corpus is made for training and validation purpose. Thirteen of the original work of Ms. Christie that are available in the public domain are chosen. Raw texts are downloaded from https://www.gutenberg.org/ and other available sources in the time frame of february 15th to 20th of 2023.
Here are the list of books after data cleaning and pre processing:
| File Name | Word Count | Character Count | Book Type |
|---|---|---|---|
| The_Murder_on_the_Links.txt | 64470 | 383665 | Novel |
| The_Mysterious_Affair_at_Styles.txt | 56456 | 341202 | Novel |
| The_Secret_of_Chimneys.txt | 74431 | 455894 | Novel |
| And_Then_There_Were_None.txt | 52607 | 320398 | Novel |
| The_murder_of_Roger_Ackroyd.txt | 69485 | 416920 | Novel |
| Poirot_Investigates.txt | 52494 | 313466 | Novel |
| The_Big_Four.txt | 55230 | 319360 | Novel |
| The_Mystery_of_the_Blue_Train.txt | 71222 | 414922 | Novel |
| The_Secret_Adversary.txt | 10855 | 75138 | Novel |
| The_Man_in_the_Brown_Suit.txt | 10317 | 75261 | Novel |
| The_Hunters_Lodge_Case.txt | 4352 | 25602 | Short Story |
| The_Missing_Will.txt | 3257 | 19004 | Short Story |
| The_Plymouth_Express_Affair.txt | 4858 | 29493 | Short Story |
| Total | 659261 | 3928209 |
nltk sentence tokenizer have been used to split all the texts into sentences. Then scikit learns train_test_split method is used to split 85% of random data sentence as training data, and the rest 15% of sentence as a evaluation data set. In training and test file, each line is placed in separate lines.
During training and validation, GPT2 default tokenizer is used.
Trainer class from transformer library is used to train the fine-tuned model.
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.2824 | 0.26 | 50 | 3.8764 |
| 3.8824 | 0.51 | 100 | 3.5931 |
| 3.6336 | 0.77 | 150 | 3.4378 |
| 3.5056 | 1.03 | 200 | 3.3445 |
| 3.4038 | 1.28 | 250 | 3.2881 |
| 3.3502 | 1.54 | 300 | 3.2506 |
| 3.3135 | 1.79 | 350 | 3.2224 |
| 3.2839 | 2.05 | 400 | 3.2028 |
| 3.2193 | 2.31 | 450 | 3.1816 |
| 3.2066 | 2.56 | 500 | 3.1660 |
| 3.2043 | 2.82 | 550 | 3.1470 |
| 3.1619 | 3.08 | 600 | 3.1380 |
| 3.1092 | 3.33 | 650 | 3.1271 |
| 3.1073 | 3.59 | 700 | 3.1187 |
| 3.099 | 3.85 | 750 | 3.1109 |
| 3.0695 | 4.1 | 800 | 3.1089 |
| 3.0281 | 4.36 | 850 | 3.1044 |
| 3.0322 | 4.62 | 900 | 3.1002 |
| 3.0358 | 4.87 | 950 | 3.0944 |
| 3.0126 | 5.13 | 1000 | 3.0958 |
| 2.9889 | 5.38 | 1050 | 3.0931 |
| 2.9874 | 5.64 | 1100 | 3.0917 |
| 2.9915 | 5.9 | 1150 | 3.0911 |