gpt2-narrative-assistant

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

gpt2-narrative-assistant

Model Summary

This repository contains an implementation of a fine-tuned version of OpenAI's GPT-2 model for narrative completions based on Wikipedia article data, utilizing the transformers library. Our fine-tuned GPT-2 model is designed to generate coherent and contextually appropriate narrative continuations for a wide array of topics. The fine-tuning process involved training GPT-2 on a large subset of Wikipedia articles, ensuring the model captures the nuanced style and informative nature of encyclopedic writing.

However, it is important to note that the model may sometimes produce outputs where sentences are not completed properly or may occasionally generate incoherent text. Users should review and edit the generated content to ensure accuracy and coherence. Whether in zero-shot or fine-tuned settings, this GPT-2 variant stands out for its ability to seamlessly extend narratives while maintaining a high level of informativeness and relevance.

It achieves the following results on the evaluation set:

• Loss: 3.4972

Intended uses & limitations

This model is primarily designed for text generation, with a particular emphasis on assisting in creative writing in English. To demonstrate its capabilities, provide a partially completed sentence on any topic of your choice, and the model will generate additional creative content related to the given topic. It is, however, important to fact-check any critical information generated by the model.

How to Use

You can try out the model by using the code below

#import relevant libraries
from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM


model_name = "KxngD/gpt2-finetuned-wikitext2"

tokenizer = AutoTokenizer.from_pretrained("gpt2")

model = AutoModelForCausalLM.from_pretrained("KxngD/gpt2-narrative-assistant")

tokenizer = AutoTokenizer.from_pretrained("KxngD/gpt2-narrative-assistant")

# Define prompt
prompt = "In Australia snakes"

# Tokenize prompt
input_ids = tokenizer.encode(prompt, return_tensors="pt")

# Generate text
output = model.generate(
    input_ids, max_length=200, 
    num_return_sequences=1,
    temperature=0.9,  
    top_k=50,         
    top_p=0.95,       
    do_sample=True,   
    pad_token_id=tokenizer.eos_token_id
                       )

# Decode and print generated text
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
print("Generated text:", generated_text)

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3.0

Training results

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.0.post200
• Datasets 2.16.1
• Tokenizers 0.13.3