llama2-22b-empath-alpacagpt4

Experimental: Created using an unofficial and unsupported method. I have no metrics on how this performs against 13b and I'm not planning on gathering any at this point. Still has weak spots that need work.

https://huggingface.co/nkpz/llama2-22b-blocktriangular-alpaca with further conversational and instruction fine tuning

First, I trained it on an epoch of https://huggingface.co/datasets/Adapting/empathetic_dialogues_v2 to give it a decent base knowledge of a casual chat style. I added some automated capitalization fixes for this data.The result was conversational, but not very smart.

Then I trained it on an epoch of https://huggingface.co/datasets/vicgalle/alpaca-gpt4 and landed here, a model that is capable of chatting but very focused on following instructions.

If you would like to run this in 4-bit, you can use the Hugging Face backend in Koboldai (or in a different script, the load_in_4bit kwarg when calling from_pretrained). GPTQ conversion has so far resulted in broken output for me, YMMV.

Future Ideas

• This strongly prefers the alpaca prompt format and will try to autocomplete it if you don't provide it. I'd like to work on removing this fixation and making it more flexible.
• Also would like to filter the rows with phrases "AI assistant" and "virtual assistant" from all future runs.
• Thinking it might also help to do a short run on a dataset focused on character impersonation

Prompting

Standard prompt format examples:

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
List 3 ingredients for the following recipe.

### Input:
Spaghetti Bolognese

### Response:

Or

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
List 3 ingredients for the following recipe: Spaghetti Bolognese

### Response:

For a chat session, I've had success using this simplified prompt:

### Scenario
You are speaking with Alexander Graham Bell
### Begin Chat (Format: [Person1]: [Message]\n[Person2]: [Message])
You: Hey, can you tell me a little bit about yourself?

In this example, its output was:

Alexander Graham Bell: Sure, I am an inventor and scientist. I'm most known for inventing the telephone.

You can customize the use of ### prefixed labels to create your own structure.