GPTNeo-Nawasena-small

Nawasena model

Model Description

<!-- Provide a longer summary of what this model is. -->

This is a story-building model that is trained using a collection of Japanese light novels translated into English. This model was created with inspiration from the griffin AI Dungeon model. The purpose of making this model is as an entertainment machine that makes stories interesting and creative.

Unfortunately, due to cost and computational power limitations, we were only able to train this model for 12 hours, and even then with a dataset of no more than 100 MB.

• Developed by: Hll-AI Production
• Model type: Text Generation
• Language(s): English
• Finetuned from model: GPT-Neo

Information

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

This model is not too big, even somewhat unstable. We train it in just 12 hours because the number of computers is limited. However, we hope that in the future this language model will be even better. The weakness lies in the limited and very small number of Context Sizes. For now, this model has not been able to do its job very well. But you can try it or train it again to make it even better.

Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

We use our own created dataset. Its name is dataset_light_novel_EN. Its size is around 38.7 MB.

That's very small, right?

Updates: The dataset is now 73.4mb in size

It's still small.

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

• Hardware Type: GPU T4 15GB
• Hours used: 12 hours
• Cloud Provider: Google colab
• Carbon Emitted: 0.47 kg

Because we use the free version of Google Colab, so we only generate around 0.47 kg of emissions. I'm not really sure, but this number is quite a lot, Maybe?

Model Card Authors

Hll-AI Production