Downloads · 30 days
7
4% of all-time downloads
Chilliwiddit/Openi-llama3.1-8B-WeightedLoss-small2
Openi-llama3.1-8B-WeightedLoss-small2 is a summarization model from Chilliwiddit. Use it when you need a shorter version of a longer text. It is set up for peft.
Basically used to summarize text from the Open-i dataset
Downloads · 30 days
7
4% of all-time downloads
All-time downloads
180
Public
Repo size
353 MB
Likes
0
Public
Click a slice to open those files.
.safetensors336 MB · 95%
From the Hugging Face model README
Basically used to summarize text from the Open-i dataset
I used the Open-i dataset
Training regime: [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
16 Mixed Precision
LR of 0.0-1
5 Epochs
lambda medical weight of 0.075 and lambda negation weight of 0.045
Used 2nd iteration of summary medical concepts file