Downloads · 30 days
52
6% of all-time downloads
Locutusque/Hercules-6.1-Llama-3.1-8B
Hercules-6.1-Llama-3.1-8B is a text generation model from Locutusque. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.1.
Downloads · 30 days
52
6% of all-time downloads
All-time downloads
865
Public
Parameters
8B
17.4 GB on disk
Likes
7
Public
Click a slice to open those files.
.safetensors16.1 GB · 100%
From the Hugging Face model README
Hercules-6.1-Llama-3.1-8B is a fine-tuned language model derived from Llama-3.1-8B. It is specifically designed to excel in instruction following, function calls, and conversational interactions across various scientific and technical domains. This fine-tuning has hercules-v6.1 with enhanced abilities in:
Hercules-6.1-Llama-3.1-8B is well-suited to the following applications:
Important Note: Although Hercules-v6.1 is carefully constructed, it's important to be aware that the underlying data sources may contain biases or reflect harmful stereotypes. Use this model with caution and consider additional measures to mitigate potential biases in its responses.
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| agieval_nous | 0.0 | none | acc | ↑ | 0.4427 | ± | 0.0094 | |
| - agieval_aqua_rat | 1.0 | none | 0 | acc | ↑ | 0.2913 | ± | 0.0286 |
| none | 0 | acc_norm | ↑ | 0.2480 | ± | 0.0272 | ||
| - agieval_logiqa_en | 1.0 | none | 0 | acc | ↑ | 0.3825 | ± | 0.0191 |
| none | 0 | acc_norm | ↑ | 0.3794 | ± | 0.0190 | ||
| - agieval_lsat_ar | 1.0 | none | 0 | acc | ↑ | 0.2087 | ± | 0.0269 |
| none | 0 | acc_norm | ↑ | 0.2043 | ± | 0.0266 | ||
| - agieval_lsat_lr | 1.0 | none | 0 | acc | ↑ | 0.4431 | ± | 0.0220 |
| none | 0 | acc_norm | ↑ | 0.4000 | ± | 0.0217 | ||
| - agieval_lsat_rc | 1.0 | none | 0 | acc | ↑ | 0.6097 | ± | 0.0298 |
| none | 0 | acc_norm | ↑ | 0.5428 | ± | 0.0304 | ||
| - agieval_sat_en | 1.0 | none | 0 | acc | ↑ | 0.7621 | ± | 0.0297 |
| none | 0 | acc_norm | ↑ | 0.6942 | ± | 0.0322 | ||
| - agieval_sat_en_without_passage | 1.0 | none | 0 | acc | ↑ | 0.4126 | ± | 0.0344 |
| none | 0 | acc_norm | ↑ | 0.3641 | ± | 0.0336 | ||
| - agieval_sat_math | 1.0 | none | 0 | acc | ↑ | 0.4318 | ± | 0.0335 |
| none | 0 | acc_norm | ↑ | 0.3500 | ± | 0.0322 | ||
| arc_challenge | 1.0 | none | 0 | acc | ↑ | 0.5247 | ± | 0.0146 |
| none | 0 | acc_norm | ↑ | 0.5606 | ± | 0.0145 | ||
| eq_bench | 2.1 | none | 0 | eqbench | ↑ | 63.2023 | ± | 2.6818 |
| none | 0 | percent_parseable | ↑ | 98.8304 | ± | 0.8246 | ||
| gsm8k | 3.0 | flexible-extract | 5 | exact_match | ↑ | 0.7801 | ± | 0.0114 |
| strict-match | 5 | exact_match | ↑ | 0.7809 | ± | 0.0114 | ||
| truthfulqa_mc2 | 2.0 | none | 0 | acc | ↑ | 0.5389 | ± | 0.0150 |
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 22.40 |
| IFEval (0-Shot) | 60.07 |
| BBH (3-Shot) | 24.15 |
| MATH Lvl 5 (4-Shot) | 15.63 |
| GPQA (0-shot) | 1.45 |
| MuSR (0-shot) | 3.42 |
| MMLU-PRO (5-shot) | 29.65 |