Downloads · 30 days
15
19% of all-time downloads
LMCocktail-phi-2-v1.1
MugoSquero/LMCocktail-phi-2-v1.1
LMCocktail-phi-2-v1.1 is a text generation model from MugoSquero. Use it when you need the model to write or continue text. It is set up for transformers.
This is a 0.5-0.5 merge of two models based on phi-2. Here are the models used to create this merge: 1. venkycs/phi-2-instruct 2. Yhyu13/phi-2-sft-dpo-gpt4en-ep1
All-time downloads
81
Public
Parameters
2.8B
11.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors11.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- phi-msft
- Access
- Public
- Created
- Jan 2, 2024
- Updated
- Jan 2, 2024
- SHA
- d6ab7ebb
Try a prompt
Model card
From the Hugging Face model README
LM-Cocktail phi-2 v1.1
This is a 0.5-0.5 merge of two models based on phi-2. Here are the models used to create this merge:
I named this model "LMCocktail phi-2 v1.1" because I see it as a continuation of the v1.
I used Yhyu13/phi-2-sft-dpo-gpt4_en-ep1 and it "outputs significantly longer result" than the one used in v1 by Yhyu13.
I also used venkycs/phi-2-instruct "a fine-tuned version of microsoft/phi-2 on the filtered ultrachat200k dataset using the SFT technique".
The main reason I created this model was to merge it with cognitivecomputations/dolphin-2_6-phi-2, and I will create a repo for it when I do it.
Code
The LM-cocktail is novel technique for merging multiple models: https://arxiv.org/abs/2311.13534
Code is backed up by this repo: https://github.com/FlagOpen/FlagEmbedding.git
Merging script is available under the ./scripts folder.
- Task
- Text Generation
- Library
- transformers
- Type
- phi-msft
- Languages
- en
- Created
- Jan 2, 2024
- Updated
- Jan 2, 2024