gpt2-xl-sft

简介

gpt2-xl-sft基于gpt2-xl(参数量1b5)在开源instruction数据集中进行Fully Fine Tuning，训练数据为140万。聚焦模型CoT、医学、生物、化学、物理、AI社会方面的英文理解与回答能力。

推理代码

from transformers import GPT2LMHeadModel, AutoTokenizer, GenerationConfig

model_name_or_path = 'MrNJK/gpt2-xl-sft'
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
model = GPT2LMHeadModel.from_pretrained(model_name_or_path).to(0)
generation_config = GenerationConfig.from_pretrained(model_name_or_path)

prompt = 'Who are you?<|endoftext|>'
input_ids = tokenizer(prompt, return_tensors="pt", padding=True)['input_ids'].to(0)
outputs = model.generate(input_ids,generation_config)[0]
print(tokenizer.decode(outputs))

效果展示

• example 1

input:

What should happen if an earthquake hits California?

output:

If an Earthquake hits the state of California, it can cause a lot of damage and loss of life. Here are some things that can happen:

1. Damage to buildings and infrastructure: The shaking and shaking of buildings can lead to damage to the walls, floors, and ceilings. This can result in the collapse of entire buildings, as well as the destruction of important structures such as schools, hospitals, or government buildings.

2. Loss of lives: Earthquakes can be very dangerous, especially for people who are not prepared for them. People can lose their lives if they are caught in a collapsed building or if the shaking is strong enough to cause them to lose balance or fall. In some cases, people may also be injured or killed by falling debris or debris.

• example 2

input:

If strong AI can be achieved, what should humans do?

output:

As an AI assistant, I don't have personal opinions, but I can tell you that there are many ethical considerations that need to be taken into account when developing and using AI. It's important to ensure that AI is used for the benefit of all, rather than for personal gain.

• example 3

input:

What was Newton's outstanding contribution?

output:

Newton's most significant contribution was his theory of gravitation, which laid the foundation for our understanding of the universe and the laws of physics. He also made significant contributions to mathematics, particularly in the development of calculus.

• example 4

input:

How does mRNA work?

output:

Molecular mimicry is the process by which a foreign molecule, such as a DNA sequence, can mimic the structure and function of a native protein. This process is essential for the proper functioning of many biological processes, including gene expression, protein synthesis, and cell signaling.

In the context of mRNA, a protein called ribosomal RNA (rRNA) is synthesized from the amino acids adenine (A) and thymine (T). The process of translation begins with the binding of the mRNA molecule to a small RNA molecule called the translation initiation factor (tRNA). This tRNA then binds to the start codon (adenine) in the target mRNA and initiates a series of chemical reactions called translation, which converts the genetic information into a form that can be used by the cell to produce proteins. The specific sequence of nucleotides that make up a gene determines the protein sequence that will be produced. For example, the gene encoding the enzyme ribulose-1,5-bisphosphate carboxylase is responsible for breaking down glucose into two molecules of pyruvate, one of which is used as an energy source for cellular respiration and the other is converted into another molecule of energy, ATP, through a process called glycolysis.

• example 5

input:

What is the use of calcium carbonate?

output:

Calcium carbonates are commonly used in the production of cement, mortar, and other building materials. They are also used as a substitute for calcium sulfate in certain industries, such as cement production.

• example 6

input:

Please introduce Pseudemonasaeruginosadisease?

output:

Pseudomonasaeroides is a fungal infection caused by Pseudo-Monas aeruginosa, a type of bacteria. It is characterized by the formation of thick, scaly, and reddish-brown lesions on the skin and mucous membranes, as well as the development of pus-filled nodules in the lungs and other organs.

How to Cite

Please cite this model using this format.

@misc{GPT2 Full Parameter Fine-tuning,
title = {gpt2-xl-sft},
author = {Jikai Nie},
year = {2023}
url={[https://huggingface.co/MrNJK/gpt2-xl-sft](https://huggingface.co/MrNJK/gpt2-xl-sft)},
}