flan-t5-dolly-10-epochs

Model

Flan-T5-Large fine-tuned on databricks dolly 15k dataset for 10 epochs. Potentially useful as a baseline or for messing around with.

Examples

Examples generating with

model.generate(tokens, do_sample=True, top_p=.99, repetition_penalty=1.2, max_new_tokens=2048)

Input: Come up with an idea for a good startup
Flan-T5-Dolly: You can start your own private stock exchange company with a minimum capital and no partners. Your goal should be to sell you the right type of shares at the right price. Make sure you build an equity model where your investors are buying short-term, medium-term and long-term contracts.

Input: How do you create a good YouTube video?
Flan-T5-Dolly: Start by deciding what the purpose of the video should be. If it is for comedy, you've already figured out the direction of the movie, so make sure that your video is focused on that. Next – put it all together. Make notes and follow the directions for the storyboard. Also add a bit of humor so that you can fully enjoy your video. Once you have this sorted, put it all together.

Input: When were the middle ages?
Flan-T5-Dolly: The Middle Ages was from 5th to 11th century AD

Input: Compare python or julia as programming languages
Flan-T5-Dolly: Julia is a small, open-source languages for computer and web development. Python is a general-purpose programming language. Julia does not support classes in collections, as Python does. Julia has some specialized features such as: - Ability to convert integers into decimals - Ability to convert strings to syllables - Ability to read objects / strings - Ability to store data under the int and variable types - Ability to do binary search - Ability to set variables in functions - Ability to combine multiple functions - Ability to import memory management functions - Ability to convert integers from decimal types - Supports multithreading - Ability to generate dynamic graphs - Ability to use multiple compiled languages - Ability to create arrays and arrays from copy/paste

Training

Trained with this code: https://github.com/dylan-slack/Exploring-Instruction-Tuning

With the following command

python train.py -m "google/flan-t5-large" --train -b 2 --accum 8 --overwrite-cache