Downloads · 30 days
10
12% of all-time downloads
TouchNight-Ministral-8B-Instruct-2410-HF-W8A8-Dynamic-Per-Token
noneUsername/TouchNight-Ministral-8B-Instruct-2410-HF-W8A8-Dynamic-Per-Token
TouchNight-Ministral-8B-Instruct-2410-HF-W8A8-Dynamic-Per-Token is a machine learning model from noneUsername. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
It is worth noting that compared with the prince-canuma version, this version is smaller in size after quantization and its accuracy is also improved by one percentage point.
All-time downloads
86
Public
Parameters
8B
9.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors9.1 GB · 100%
Parameter types
How the weights are stored.
I86.9B · 87%
Base models
Model card
From the Hugging Face model README
It is worth noting that compared with the prince-canuma version, this version is smaller in size after quantization and its accuracy is also improved by one percentage point.
In my ERP testing, this model did perform better.
vllm (pretrained=/root/autodl-tmp/Ministral-8B-Instruct-2410-HF,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=float16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| gsm8k | 3 | flexible-extract | 5 | exact_match | ↑ | 0.820 | ± | 0.0243 |
| strict-match | 5 | exact_match | ↑ | 0.816 | ± | 0.0246 |
vllm (pretrained=/root/autodl-tmp/Ministral-8B-Instruct-2410-HF,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| gsm8k | 3 | flexible-extract | 5 | exact_match | ↑ | 0.804 | ± | 0.0252 |
| strict-match | 5 | exact_match | ↑ | 0.804 | ± | 0.0252 |
vllm (pretrained=/root/autodl-tmp/Ministral-8B-Instruct-2410-HF,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=float32), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| gsm8k | 3 | flexible-extract | 5 | exact_match | ↑ | 0.820 | ± | 0.0243 |
| strict-match | 5 | exact_match | ↑ | 0.816 | ± | 0.0246 |
vllm (pretrained=/root/autodl-tmp/output,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=float16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| gsm8k | 3 | flexible-extract | 5 | exact_match | ↑ | 0.816 | ± | 0.0246 |
| strict-match | 5 | exact_match | ↑ | 0.812 | ± | 0.0248 |
vllm (pretrained=/root/autodl-tmp/output,add_bos_token=true,tensor_parallel_size=2,max_model_len=2048,dtype=bfloat16), gen_kwargs: (None), limit: 250.0, num_fewshot: 5, batch_size: auto
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| gsm8k | 3 | flexible-extract | 5 | exact_match | ↑ | 0.796 | ± | 0.0255 |
| strict-match | 5 | exact_match | ↑ | 0.792 | ± | 0.0257 |
- Type
- mistral
- Created
- Oct 19, 2024
- Updated
- Oct 19, 2024