Curie-7B-v0.1

Model Card: Curie-7B

Model Description

Curie-7B is a work-in-progress language model fine-tuned on chemistry & chemical substance data. It will be based on LlaMA-2-7B. This model is being fine-tuned by SynthMind & is meant to only use open-source datasets. This should make the model reproducible by anyone. Nobody is sponsoring the compute (at least for now).

This model will try to obtain long responses & lower hallucination rates. The fine-tuning process will be performed with a TBD sequence length on a 2x 3090 24GB machine (TBC).

Training Datasets

The Curie-7B model is fine-tuned on the following openly available datasets:

• WizardLM
• GPTeacher-General-Instruct
• metaeval/ScienceQA_text_only - instruct for concise responses
• openai/summarize_from_feedback - instruct augmented tl;dr summarization
• camel-ai/math
• camel-ai/physics
• camel-ai/chemistry
• camel-ai/biology

Prompt Format

Chat only style prompts using USER:,ASSISTANT:.

Benchmarks:

TODO

Demo

TODO

Future Plans

As with many other models, it's planned to continue iterating on both more high-quality data, and new data filtering techniques to eliminate lower-quality data going forward.

Build / Training procedure

TODO

Examples

TODO