flacuna-13b-v1.0

Flacuna: A Vicuna made of Flan

Paper | Model | Dataset

📣 We still have numerous experiments awaiting completion (details are here), requiring additional computing resources in our lab. If any industry professionals reading this are willing to provide assistance, please feel free to reach out to us at [email protected].

<img src="https://declare-lab.net/assets/images/logos/flacuna5.png" alt="Image" width="200" height="335">

Flacuna was developed by fine-tuning Vicuna on Flan-mini, a comprehensive instruction collection encompassing various tasks. Vicuna is already an excellent writing assistant, and the intention behind Flacuna was to enhance Vicuna's problem-solving capabilities. To achieve this, we curated a dedicated instruction dataset called Flan-mini.

Problem Solving Ability

As a result of this fine-tuning process, Flacuna exhibited notable performance improvements in problem-solving across multiple benchmark datasets, both in few-shot and zero-shot settings.

During training, Flacuna is a 13B checkpoint of LLaMA and employed a maximum input sequence length of 1280. We utilized LoRA for parameter-efficient fine-tuning.

Chatbot / Writing Assistant

While Flacuna primarily excels in problem-solving tasks, we made efforts to maintain the impressive writing and chatting ability of Vicuna. To achieve this, we incorporated conversational datasets generated by GPT-4, such as GPT-4-Alpaca and ShareGPT, into the Flan-mini collection. To use Flacuna as a chatbot or writing assistant, we recommend you use the following template:

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: {definition of the task}.\n\n
{question}\n
Output: ASSISTANT:

Please note that we still recommend using Vicuna as your preferred Chatbot or Writing Assistant, over Flacuna. Flacuna's primary strength lies in problem-solving tasks, making it ideal for such applications.

The following table presents the writing performance of Flacuna on the IMPACT dataset, which is a component of the InstructEval evaluation suite. The generated responses have been evaluated by ChatGPT, and their relevance and coherence have been scored on a scale of 1 to 5.

Basic Usage

git clone https://huggingface.co/declare-lab/flacuna-13b-v1.0
cd flacuna-13b-v1.0
python flacuna.py

Training or Fine-tuning Flacuna

The trainer codes are available here: https://github.com/declare-lab/flacuna.

License

Non-commercial license

Citation

@misc{ghosal2023flacuna,
      title={Flacuna: Unleashing the Problem Solving Power of Vicuna using FLAN Fine-Tuning}, 
      author={Deepanway Ghosal and Yew Ken Chia and Navonil Majumder and Soujanya Poria},
      year={2023},
      eprint={2307.02053},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}