Downloads · 30 days
0
My_LLMs_Leaderboard
Alsebay/My_LLMs_Leaderboard
My_LLMs_Leaderboard is a machine learning model from Alsebay. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
- <span style="color:F53A85"Lewd</span : perform well NSFW content. Some of lewd words will appear in normal content if your Character Card have NSFW informations. - <span style="color:40C5F0"Neurral</span : perform w…
Access
Public
Updated Apr 17, 2024
Repo size
—
Likes
0
Public
Repo makeup
Click a slice to open those files.
.md4.2 KB · 73%
At a glance
- Access
- Public
- Created
- Apr 3, 2024
- Updated
- Apr 17, 2024
- SHA
- 904d01c1
Model card
From the Hugging Face model README
This is the Leaderboard about ranking my own model :) Also some useful information (Maybe). Main purpose is for Roleplay
Leaderboard
| Rank | Name | Parameter | Context Length | Tag | Note |
|---|---|---|---|---|---|
| 💎1 | Narumashi-RT | 11B | <span style="color:green">4K</span> | <span style="color:#F53A85">Lewd</span> | Good for Roleplay, although it is LLAMA2. Thank Sao10k :) Could handle some (limited) TSF content. |
| 🏆2 | NaruMoE | 3x7B | <span style="color:cyan">8K</span> - <span style="color:red">32K</span> | <span style="color:#40C5F0">Neurral</span> | AVG model, could only handle limited extra content I want. |
| ✌3 | NarumashiRTS | 7B | <span style="color:cyan">8K</span> | <span style="color:#40C5F0">Neurral</span> | Base on Kunoichi-7B, so it good enough. Know the extra content. Not lewd and will skip lewd content sometime. |
| 4 | HyouKan Series | 3x7B | <span style="color:cyan">8K</span> - <span style="color:red">32K</span> | <span style="color:#40C5F0">Neurral</span> | <span style="color:red">ATTENTION: DON'T USE GGUF VERSION SINCE IT HAVE SOME BUGS (VARY BY VERSION)</span> All-rounded Roleplay model. Understand well Character Card and good logic. The first version have 8k context lenght. <span style="color:red"> |
| 5 | SunnyRain | 2x10.7B | <span style="color:green">4K</span> | <span style="color:#F53A85">Lewd</span> | To be real, it perform approximate like HyouKan in Roleplay, just got some strange behavious. |
| 6 | RainyMotip | 2x7B | <span style="color:red">32K</span> | <span style="color:#40C5F0">Neurral</span> | Good enough model, ok in Roleplay. |
| 7 | Nutopia | 7B | <span style="color:red">32K</span> | <span style="color:#F2EC4E">Not for Roleplay</span> | I don't think this work for Roleplay, but it good for solving problem |
| 8 | TripedalChiken | 2x7B | <span style="color:red">32K</span> | <span style="color:#F2EC4E">Not for Roleplay</span> | Solving problem is good, but for Roleplay, I don't think so |
Note:
- <span style="color:#F53A85">Lewd</span> : perform well NSFW content. Some of lewd words will appear in normal content if your Character Card have NSFW informations.
- <span style="color:#40C5F0">Neurral</span> : perform well SFW content, can perform well NSFW content (limited maybe). Lewd words will less appear in chat/roleplay than <span style="color:#F53A85">Lewd</span>
- <span style="color:#F2EC4E">Not for Roleplay</span> : seem that those model with this tag not understand well Character Card. But its logical is very good.
- RT: Rough Translation Dataset that could lead to worse performance than original model.
- CN: Chinese dataset pretrain, maybe not understand extra content in English. (I can't find any good english verion.)
Some experience:
- The Context Length affect too much to your Memory. Let's say I have 16GB Vram card, I can run the model in 2 ways, using Text-Generation-WebUI:
- Inference: download the origin model, apply args:
--load-in-4bit --use_double_quant. I can run all of my model in leaderboard. The bigger parameter is, the slower token can generate. (Ex:7B model could run in 15 token/s, since 3x7b model could only run in ~4-5 token/s) - GGUF Quantization (Fastest,cheapest way to run): After you downloaded GGUF version of those models, sometimes, you can't run it although you can run other model that have bigger parameter. That because:
- The context length: 16GB VRAM GPU could run maximum 2x10.7B (~ 19.2B) model with 4k context length. (5 token/s)
- That model is bug/broken.😏
- Inference: download the origin model, apply args:
- Bigger model will have more information that you need for your Character Card.
- Best GGUF version that you should run (balance speed/performance): Q4_K_M, Q5_K_M (Slower than Q4)
Useful link:
- Languages
- en
- Created
- Apr 3, 2024
- Updated
- Apr 17, 2024