gemma3-12B-Amateras

Gemma-12B-Amateras - Built with Gemma3

Gemma-12B-Amateras is a large language model constructed by continuously training the Gemma3-12B model with Japanese historical education knowledge at a textbook level. The learning set was uniquely created with the following three points:

1)Explanatory text for historical terms

Words were extracted from textbooks used in high school history courses, and explanatory texts were created using Gemma3-27B. Subsequently, several developers conducted visual inspections to build a high-quality historical event explanation text dataset.

2)Causal relationships between historical events

Using textbook content as a reference, a unique dataset was created that summarizes causal relationships between historical events in both JSON structure and natural language explanations.

3)Instruction-response texts created based on actual user prompts

Prompts from 90 test subjects who actually used generative AI were collected, and response texts were created using Gemma3-27B. Several developers then conducted visual inspections to correct hallucinations and create a dataset of instruction-response texts.

Notice

While the vocabulary related to historical events has been sufficiently expanded, hallucinations have been confirmed regarding East Asian history, which had insufficient learning in the base model Gemma3. Therefore, it is necessary to use RAG in combination to improve reliability.