Timpi_Wilson

TimpiWilson Model Card

Model details

Model type: TimpiWilson is an open-source chatbot trained by fine-tuning Flan-t5-xl (3B parameters) on user-shared conversations collected from ShareGPT and conversation collection created from TImpi's index. It is based on an encoder-decoder transformer architecture, and can autoregressively generate responses to users' inputs.

Model date: TimpiWilson was trained on June 2023.

Organizations developing the model: The Timpi developers.

License: Apache License 2.0

Intended use

Primary intended uses: This version ot TimpiWilson is to showcase the abilities of the model.

Primary intended users: The primary intended users of the model are entrepreneurs and researchers in natural language processing, machine learning, and artificial intelligence.

Training dataset

70K conversations collected from ShareGPT.com. 40K conversations collected from Timpi's index

Training details

It processes the data in the form of question answering. Each Wilson response is processed as an answer, and previous conversations between the user and the Wilson are processed as the question. The encoder bi-directionally encodes a question into a hidden representation. The decoder uses cross-attention to attend to this representation while generating an answer uni-directionally from a start token. This model is fine-tuned for 3 epochs, with a max learning rate 2e-5, warmup ratio 0.03, and a cosine learning rate schedule.

Evaluation dataset

A preliminary evaluation of the model quality is conducted by creating a set of 80 diverse questions and utilizing GPT-4 to judge the model outputs.