Downloads · 30 days
0
princeton-nlp_Sheared-LLaMA-2.7B_iMat_Custom_Quant_Stategies-GGUF
Nexesenex/princeton-nlp_Sheared-LLaMA-2.7B_iMat_Custom_Quant_Stategies-GGUF
princeton-nlp_Sheared-LLaMA-2.7B_iMat_Custom_Quant_Stategies-GGUF is a machine learning model from Nexesenex. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama2.
Notes only for now, rework needs to be done.
Access
Public
Updated Aug 24, 2024
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.md4.9 KB · 76%
At a glance
- License
- llama2
- Access
- Public
- Created
- Aug 22, 2024
- Updated
- Aug 24, 2024
- SHA
- b642a7e7
Model card
From the Hugging Face model README
Notes only for now, rework needs to be done.
Q2_K_S
Master 900.24 MiB (2.80 BPW) PPL = 9.2374 +/- 0.05597
PR : 0.97 GB (2.86 BPW) 0.90 GiB (2.86 BPW) PPL over 655 chunks for n_ctx=512 = 8.8572 +/- 0.05438
Q2_K
Master 979.89 MiB (3.04 BPW) PPL = 8.4255 +/- 0.05158
PR : 1.02 GB (3.01 BPW) 0.95 GiB (3.01 BPW) PPL over 655 chunks for n_ctx=512 = 8.4388 +/- 0.05174
Q2_K_L
PR : 1.07 GB (3.18 BPW) 1.00 GiB (3.18 BPW) PPL over 655 chunks for n_ctx=512 = 8.1990 +/- 0.04955
Q3_K_M
Master 1.24 GiB (3.94 BPW) PPL = 7.5814 +/- 0.04537
PR : 1.33 GB (3.95 BPW) 1.24 GiB (3.95 BPW) PPL over 655 chunks for n_ctx=512 = 7.5797 +/- 0.04537
Q3_K_L
Master 1.35 GiB (4.29 BPW) PPL = 7.5425 +/- 0.04506
PR : 1.45 GB (4.29 BPW) 1.35 GiB (4.29 BPW) PPL over 655 chunks for n_ctx=512 = 7.5425 +/- 0.04506
IQ1_XS
PR : 0.63 GB (1.86 BPW) 0.59 GiB (1.86 BPW) PPL over 655 chunks for n_ctx=512 = 20.1826 +/- 0.13290
PR2 : 0.64 GB (1.91 BPW) 0.60 GiB (1.91 BPW) PPL over 655 chunks for n_ctx=512 = 17.2270 +/- 0.11191
IQ1_S
Master 600.66 MiB (1.87 BPW) PPL = 19.5631 +/- 0.12335
PR : 0.65 GB (1.94 BPW) 0.61 GiB (1.94 BPW) PPL over 655 chunks for n_ctx=512 = 16.8894 +/- 0.10891
PR2 : 0.67 GB (1.97 BPW) 0.62 GiB (1.97 BPW) PPL over 655 chunks for n_ctx=512 = 15.0819 +/- 0.09635
IQ1_M
Master 646.43 MiB (2.01 BPW) PPL = 14.6658 +/- 0.09171
PR : 0.70 GB (2.06 BPW) 0.65 GiB (2.06 BPW) PPL over 655 chunks for n_ctx=512 = 13.7678 +/- 0.08796
PR2 : 0.70 GB (2.07 BPW) 0.65 GiB (2.07 BPW) PPL over 655 chunks for n_ctx=512 = 13.0640 +/- 0.08332
IQ1_XL
PR : 0.73 GB (2.16 BPW) 0.68 GiB (2.16 BPW) PPL over 655 chunks for n_ctx=512 = 12.4043 +/- 0.07821
PR2 : 0.73 GB (2.16 BPW) 0.68 GiB (2.16 BPW) PPL over 655 chunks for n_ctx=512 = 11.5807 +/- 0.07290
IQ2_XXS
Master 722.70 MiB (2.24 BPW) PPL = 10.7338 +/- 0.06621
PR : 0.79 GB (2.34 BPW) 0.74 GiB (2.34 BPW) PPL over 655 chunks for n_ctx=512 = 9.9385 +/- 0.06080
PR2 : 782.00 MB (2.32 BPW) 745.78 MiB (2.32 BPW) PPL over 655 chunks for n_ctx=512 = 9.9519 +/- 0.06097
IQ2_XS
Master 789.96 MiB (2.45 BPW) PPL = 9.6386 +/- 0.05831
PR : 0.86 GB (2.54 BPW) 0.80 GiB (2.54 BPW) PPL over 655 chunks for n_ctx=512 = 9.1694 +/- 0.05600
PR 2 : 0.84 GB (2.50 BPW) 0.79 GiB (2.50 BPW) PPL over 655 chunks for n_ctx=512 = 9.1616 +/- 0.05577
IQ2_S
Master 853.19 MiB (2.65 BPW) PPL = 8.9655 +/- 0.05484
PR : 0.90 GB (2.66 BPW) 0.84 GiB (2.66 BPW) PPL over 655 chunks for n_ctx=512 = 8.9321 +/- 0.05441
PR 2 : 0.90 GB (2.65 BPW) 0.83 GiB (2.65 BPW) PPL over 655 chunks for n_ctx=512 = 8.8710 +/- 0.05385
PR 3 : 0.88 GB (2.60 BPW) 0.82 GiB (2.60 BPW) PPL over 655 chunks for n_ctx=512 = 8.8873 +/- 0.05410
IQ2_M
Master 914.20 MiB (2.84 BPW) PPL = 8.4373 +/- 0.05110
PR : 0.95 GB (2.82 BPW) 0.89 GiB (2.82 BPW) PPL over 655 chunks for n_ctx=512 = 8.4881 +/- 0.05125
PR 2 : 0.96 GB (2.84 BPW) 0.89 GiB (2.84 BPW) PPL over 655 chunks for n_ctx=512 = 8.3724 +/- 0.05052
PR 3 : 0.96 GB (2.83 BPW) 0.89 GiB (2.83 BPW) PPL over 655 chunks for n_ctx=512 = 8.3566 +/- 0.05040
IQ2_XL
PR : 1.00 GB (2.97 BPW) 0.93 GiB (2.97 BPW) PPL over 655 chunks for n_ctx=512 = 8.2687 +/- 0.04968
PR 2 : 1.01 GB (3.00 BPW) 0.94 GiB (3.00 BPW) PPL over 655 chunks for n_ctx=512 = 8.1774 +/- 0.04925
IQ3_XXS
Master 998.70 MiB (3.10 BPW) PPL = 8.0228 +/- 0.04817
PR : 1.09 GB (3.22 BPW) 1.01 GiB (3.22 BPW) PPL over 655 chunks for n_ctx=512 = 7.8982 +/- 0.04752
IQ3_XS
Master 1.06 GiB (3.36 BPW) 7.7736 +/- 0.04626
PR : 1.19 GB (3.53 BPW) 1.11 GiB (3.53 BPW) PPL over 655 chunks for n_ctx=512 = 7.7329 +/- 0.04586
PR 2 : 1.16 GB (3.44 BPW) 1.08 GiB (3.44 BPW) PPL over 655 chunks for n_ctx=512 = 7.7327 +/- 0.04591
IQ3_S
Master 1.11 GiB (3.53 BPW) PPL = 7.6667 +/- 0.04548
PR : 1.23 GB (3.64 BPW) 1.14 GiB (3.64 BPW) PPL over 655 chunks for n_ctx=512 = 7.6479 +/- 0.04551
IQ3_M
Master 1.17 GiB (3.73 BPW) PPL = 7.6557 +/- 0.04521
PR : 1.30 GB (3.84 BPW) 1.21 GiB (3.84 BPW) PPL over 655 chunks for n_ctx=512 = 7.5941 +/- 0.04558
PR 2 : 1.28 GB (3.80 BPW) 1.19 GiB (3.80 BPW) PPL over 655 chunks for n_ctx=512 = 7.5864 +/- 0.04569
IQ3_XL
PR : 1.32 GB (3.89 BPW) 1.22 GiB (3.89 BPW) PPL over 655 chunks for n_ctx=512 = 7.5774 +/- 0.04555
PR 2 : 1.33 GB (3.95 BPW) 1.24 GiB (3.95 BPW) PPL over 655 chunks for n_ctx=512 = 7.5446 +/- 0.04516
IQ3_XXL
PR : 1.35 GB (3.99 BPW) 1.26 GiB (3.99 BPW) PPL over 655 chunks for n_ctx=512 = 7.5468 +/- 0.04529
PR 2 : 1.37 GB (4.07 BPW) 1.28 GiB (4.07 BPW) PPL over 655 chunks for n_ctx=512 = 7.5119 +/- 0.04497
IQ3_XXXL
PR : 1.40 GB (4.13 BPW) 1.30 GiB (4.13 BPW) PPL over 655 chunks for n_ctx=512 = 7.4913 +/- 0.04487
IQ4_XS
Master 1.36 GiB (4.32 BPW) PPL = 7.4449 +/- 0.04439
PR : 1.46 GB (4.32 BPW) 1.36 GiB (4.32 BPW) Final estimate: PPL over 655 chunks for n_ctx=512 = 7.4449 +/- 0.04439
FP16
Master 5.03 GiB (16.00 BPW) PPL = 7.3511 +/- 0.04375
- License
- llama2
- Created
- Aug 22, 2024
- Updated
- Aug 24, 2024