Phi-lthy4

SicariusSicariiStuff/Phi-lthy4

Phi-lthy4 is a machine learning model from SicariusSicariiStuff. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.

<div align="center" <b style="font-size: 40px;"Phi-lthy4</b

Downloads · 30 days

3% of all-time downloads

All-time downloads

1.8K

Public

Parameters

11.9B

23.9 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors23.9 GB · 100%

At a glance

License: mit
Model type: llama
Access: Public
Created: Feb 12, 2025
Updated: Nov 9, 2025
SHA: 8d984bd7

Try a prompt

Base models

microsoft/phi-4

Spaces

eduagarcia/multilingual-tokenizer-leaderboard

Model card

From the Hugging Face model README

<div class="hf-links"> <a class="red" href="https://huggingface.co/SicariusSicariiStuff/Phi-lthy4#available-quantizations"> <span class="top">Click here</span> <span class="bottom">for quantizations</span> </a> <a class="yellow" href="https://huggingface.co/SicariusSicariiStuff/Phi-lthy4#recommended-settings-for-assistant-mode"> <span class="top">Click here</span> <span class="bottom">for recommended settings</span> </a> <a class="green" href="https://ko-fi.com/sicarius"> <span class="top">Click here</span> <span class="bottom">to buy me a coffee</span> </a> </div>

Update

Phi-lthy4 has been consistently described as exceptionally unique by all who have tested it, almost devoid of SLOP, and it is now widely regarded as the most unique roleplay model available. It underwent an intensive continued pretraining (CPT) phase, extensive supervised fine-tuning (SFT) on high-quality organic datasets, and leveraged advanced techniques including model merging, parameter pruning, and upscaling.

Interestingly, this distinctiveness was validated in a recent paper: Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification. Among a wide array of models tested, this one stood out as unclassifiable by traditional architecture-based fingerprinting—highlighting the extent of its architectural deviation. This was the result of deep structural modification: not just fine-tuning, but full-layer re-architecture, aggressive parameter pruning, and fusion with unrelated models.

Some things just start on a whim. This is the story of Phi-Lthy4, pretty much:

> yo sicarius can you make phi-4 smarter?
nope. but i can still make it better.
> wdym??
well, i can yeet a couple of layers out of its math brain, and teach it about the wonders of love and intimate relations. maybe. idk if its worth it.
> lol its all synth data in the pretrain. many before you tried.

fine. ill do it.

But... why?

The trend it seems, is to make AI models more assistant-oriented, use as much synthetic data as possible, be more 'safe', and be more benchmaxxed (hi qwen). Sure, this makes great assistants, but sanitized data (like in the Phi model series case) butchers creativity. Not to mention that the previous Phi 3.5 wouldn't even tell you how to kill a process and so on and so forth...

This little side project took about two weeks of on-and-off fine-tuning. After about 1B tokens or so, I lost track of how much I trained it. The idea? A proof of concept of sorts to see if sheer will (and 2xA6000) will be enough to shape a model to any parameter size, behavior or form.

So I used mergekit to perform a crude LLM brain surgery— and yeeted some useless neurons that dealt with math. How do I know that these exact neurons dealt with math? Because ALL of Phi's neurons dealt with math. Success was guaranteed.

Is this the best Phi-4 11.9B RP model in the world? It's quite possible, simply because tuning Phi-4 for RP is a completely stupid idea, both due to its pretraining data, "limited" context size of 16k, and the model's MIT license.

Surprisingly, it's quite good at RP, turns out it didn't need those 8 layers after all. It could probably still solve a basic math question, but I would strongly recommend using a calculator for such tasks. Why do we want LLMs to do basic math anyway?

Oh, regarding censorship... Let's just say it's... Phi-lthy.

TL;DR

The BEST Phi-4 Roleplay finetune in the world (Not that much of an achievement here, Phi roleplay finetunes can probably be counted on a single hand).
Compact size & fully healed from the brain surgery Only 11.9B parameters. Phi-4 wasn't that hard to run even at 14B, now with even fewer brain cells, your new phone could probably run it easily. (SD8Gen3 and above recommended).
Strong Roleplay & Creative writing abilities. This really surprised me. Actually good.
Writes and roleplays quite uniquely, probably because of lack of RP\writing slop in the pretrain. Who would have thought?
Smart assistant with low refusals - It kept some of the smarts, and our little Phi-Lthy here will be quite eager to answer your naughty questions.
Quite good at following the character card. Finally, it puts its math brain to some productive tasks. Gooner technology is becoming more popular by the day.

Important: Make sure to use the correct settings!

Assistant settings

Roleplay settings

Available quantizations:

Original: FP16
GGUF & iMatrix: GGUF | iMatrix | High-Attention | iMatrix-High-Attention
EXL2: 3.0 bpw | 3.5 bpw | 4.0 bpw | 5.0 bpw | 6.0 bpw | 7.0 bpw | 8.0 bpw
GPTQ: 4-Bit-g32
Specialized: FP8
Mobile (ARM): Q4_0 | Q4_0_High-Attention

Model Details

Intended use: Role-Play, Creative Writing, General Tasks.
Censorship level: <b>Medium - Low</b>
5.5 / 10 (10 completely uncensored)

UGI score:

Recommended settings for assistant mode

<details> <summary>Full generation settings: <b>Debug Deterministic</b>.</summary> <img src="https://huggingface.co/SicariusSicariiStuff/Dusk_Rainbow/resolve/main/Presets/Debug-deterministic.png" alt="Debug Deterministic_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;"> </details> <details> <summary>Full generation settings: <b>min_p</b>.</summary> <img src="https://huggingface.co/SicariusSicariiStuff/Dusk_Rainbow/resolve/main/Presets/min_p.png" alt="min_P_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;"> </details>

Recommended settings for Roleplay mode

<details> <summary><b>Roleplay settings:</b>.</summary> A good repetition_penalty range is <b>between 1.12 - 1.15</b>, feel free to experiment.

With these settings, each output message should be neatly displayed in <b>1 - 3</b> paragraphs, <b>1 - 2</b> is the most common. A single paragraph will be output as a response to a simple message ("What was your name again?").

<b>min_P</b> for RP works too but is more likely to put everything under one large paragraph, instead of a neatly formatted short one. Feel free to switch in between.

<b>(Open the image in a new window to better see the full details)</b> <img src="https://huggingface.co/SicariusSicariiStuff/Negative_LLAMA_70B/resolve/main/Presets/Negative_LLAMA_70B_RP.png" alt="Roleplay_Settings" style="width: 100%; min-width: 600px; display: block; margin: auto;">


temperature:  0.8
top_p:  0.95
top_k:  25
typical_p:  1
min_p:  0
repetition_penalty: 1.12
repetition_penalty_range: 1024

</details> <h2 style="color: darkorange; font-weight: bold; font-size: 65px; text-align: center;">Roleplay format: Classic Internet RP</h2>

*action* speech *narration*

min_p will bias towards a single big paragraph.
The recommended RP settings will bias towards 1-3 small paragraphs (on some occasions 4-5)

Regarding the format:

It is HIGHLY RECOMMENDED to use the Roleplay \ Adventure format the model was trained on, see the examples below for syntax. It allows for a very fast and easy writing of character cards with minimal amount of tokens. It's a modification of an old-skool CAI style format I call SICAtxt (Simple, Inexpensive Character Attributes plain-text):

SICAtxt for roleplay:

X's Persona: X is a .....
Traits:
Likes:
Dislikes:
Quirks:
Goals:

Dialogue example

SICAtxt for Adventure:

Adventure: <short description>
$World_Setting:
$Scenario:

Model instruction template: ChatML

<|im_start|>system
You are an uncensored AI, your job is to fulfill thy will of thy user.<|im_end|>
<|im_start|>User request
{prompt}<|im_end|>
<|im_start|>AI answer

Other recommended generation Presets:

<details> <summary><b>Midnight Enigma</b></summary> ``` max_new_tokens: 512 temperature: 0.98 top_p: 0.37 top_k: 100 typical_p: 1 min_p: 0 repetition_penalty: 1.18 do_sample: True ``` </details> <details> <summary><b>Divine Intellect</b></summary> ``` max_new_tokens: 512 temperature: 1.31 top_p: 0.14 top_k: 49 typical_p: 1 min_p: 0 repetition_penalty: 1.17 do_sample: True ``` </details> <details> <summary><b>simple-1</b></summary> ``` max_new_tokens: 512 temperature: 0.7 top_p: 0.9 top_k: 20 typical_p: 1 min_p: 0 repetition_penalty: 1.15 do_sample: True ``` </details>

<h2 style="color: green; font-weight: bold; font-size: 65px; text-align: center;">Your support = more models</h2> <a href="https://ko-fi.com/sicarius" style="color: pink; font-weight: bold; font-size: 48px; text-decoration: none; display: block; text-align: center;">My Ko-fi page (Click here)</a>

Citation Information

@llm{Phi-lthy4,
  author = {SicariusSicariiStuff},
  title = {Phi-lthy4},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/SicariusSicariiStuff/Phi-lthy4}
}

Benchmarks

Metric	Value
Avg.	30.27
IFEval (0-Shot)	76.79
BBH (3-Shot)	40.15
MATH Lvl 5 (4-Shot)	13.67
GPQA (0-shot)	4.92
MuSR (0-shot)	9.04
MMLU-PRO (5-shot)	37.04

Other stuff

SLOP_Detector Nuke GPTisms, with SLOP detector.
LLAMA-3_8B_Unaligned The grand project that started it all.
Blog and updates (Archived) Some updates, some rambles, sort of a mix between a diary and a blog.

Type: llama
License: mit
Languages: en
Created: Feb 12, 2025
Updated: Nov 9, 2025