gemma2_it_2b_icebreaking_quiz_v2_5

Icebreaking Quiz Generation Model

This model is designed to generate icebreaking quizzes. Icebreaking refers to activities conducted before meetings to help participants break the ice and ease tension by solving simple quizzes. By inputting easy-to-answer questions along with users' responses into the model, it generates new multiple-choice questions with four options and the correct answer. These customized questions can help participants engage with each other and reduce awkwardness in the room.

1. Data Collection

We collected questions and answers from blogs via web crawling and used state-of-the-art models like ChatGPT to generate new four-option multiple-choice questions. We then manually reviewed and edited the generated questions to correct any errors in answer matching and adjusted the difficulty level to make the multiple-choice options more challenging. This process resulted in a well-structured dataset.

Example Data

2. Fine-tuning Process

This model was fine-tuned using the Google Gemma 2B instruct model, with LoRA applied to make the training process lighter and faster on TPU. LoRA reduces the number of parameters updated during training, allowing efficient fine-tuning without consuming excessive memory resources. Early stopping was also applied to prevent overfitting.

The training was conducted on a JAX backend using TPU acceleration, distributing the workload across multiple TPU cores for improved efficiency. The model was optimized using the AdamW optimizer, and the loss function was calculated using sparse categorical cross-entropy.

Key Fine-tuning Details:

• Sequence Length: 512
• Batch Size: 16
• LoRA Rank: 8
• Learning Rate: 2e-5
• Epochs: 20
• Train Data Rate: 0.9
• Early Stopping Step: 5

Training Method:

The training data consists of "question_crawling" and "answer_crawling" columns as inputs, with "question_generated", "multiple_choice_generated", and "answer_generated" columns as outputs. The data was formatted to fit the fine-tuned model's input structure for training.

The model is built using Keras_NLP's pre-built GemmaCausalLM and applied to the Gemma 2B instruct architecture. We activated LoRA on the decoder blocks and managed the model efficiently across TPU devices via model parallelization. Training was conducted for 20 epochs using TPU resources on Kaggle.

Training Time and Results:

• Training ran for 20 epochs with a batch size of 16.
• Time per epoch: ~5 minutes (with TPU)
• Total training time: ~100 minutes
• The model demonstrated high accuracy in quiz generation, achieving approximately 81% accuracy on the validation data.

Input

<instruction>
Using the text: 좋아하는 히어로? 나의 히어로는 우리 아빠, create a new multiple-choice question with 4 answer options.

Model Output

<Response>
가장 좋아하는 히어로는 누구인가요?
1. 나의 아빠  2. 슈퍼맨  3. 캡틴 아메리카  4. 헐크
1

Expected Answer

<Response>
좋아하는 히어로는 누구인가요?
1. 우리 엄마  2. 스파이더맨  3. 우리 아빠  4. 아이언맨
3

Kaggle Notebook: https://www.kaggle.com/code/mukmukmukmuk/v2-fine-tune-2b-icebreaking-quiz-tpu/notebook

3. Model Usage

When using this model, data must be formatted according to a specific template. The following structure is required for input:

<instruction>
Using the text: {question_crawling} {answer_crawling}, create a new multiple-choice question with 4 answer options.

The model takes the question_crawling and answer_crawling data to generate new multiple-choice questions. The expected format for the generated output is:

<Response>
{question_generated}
{multiple_choice_generated}
{answer_generated}

This template includes the newly generated question, multiple-choice options, and the correct answer. When using this template for testing, the fields for question_generated, multiple_choice_generated, and answer_generated are left blank for the model to fill in.

This model has been developed as a service using the Flask framework and Google Survey, allowing real users to utilize it for icebreaking. Users respond to simple questions on Google Survey, and the model generates new questions based on their input. Flask processes the question generation, while a 'conversation topic guide' is provided to facilitate discussions while waiting for the generated questions.

Deployment Platform:

• Supported backends: JAX, TensorFlow, PyTorch

Service link: http://64.110.84.104:7860
Code link: https://huggingface.co/spaces/yunzi7/icebreaking/tree/main

For more details about the model architecture, check out the config.json.