penny5-dolphin-einstein-llama3-dare-ties-chatml

Model Card for Model ID

This is a fine tune of a merged model using the DARE TIES merge method using cognitivecomputations/dolphin-2.9-llama3-8b as a base. The following models were included in the merge:

• Weyaxi/Einstein-v6.1-Llama3-8B

This model should be mostly uncensored out of the box. I personally add a system prompt with the chatml template to guide the model.

Model Details

Quant Q8_0 GGUF

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub.

• Developed by: Gianni Sanrochman
• Funded by: [Merildo Sanrochman]
• Model type: LLaMA-3
• Language(s) (NLP): [English]
• License: llama3
• Finetuned from model: giannisan/dolphin-einstein-llama3-dare-ties using the PENNY dataset

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

[More Information Needed]

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Training Hyperparameters

• Training regime: [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

Evaluation

full results here

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: [Nvidia RTX A100]
• Hours used: [2]
• Cloud Provider: [RunPod]
• Compute Region: [Europe]
• Carbon Emitted: [More Information Needed]

Model Card Authors

[Gianni Sanrochman]