Cream-Phi-3-14B-v1

Join our sleepy Discord!

https://discord.gg/eeYNWYcx

Cream-Phi-3-14B-v1 🍦🥧

CreamPhi's back and bigger than ever!

GGUF: https://huggingface.co/TheDrummer/Cream-Phi-3-14B-v1-GGUF

Description

CreamPhi 14B is the first Phi Medium to be trained with roleplay and moist.

Usage

Instruct should work but I mainly tested it with Roleplay Mode.

I highly recommend KoboldCPP for running the GGUF: https://github.com/LostRuins/koboldcpp?tab=readme-ov-file#windows-usage

Samples

<details> <summary>Sample SFW (Roleplay Mode)</summary> ... "Hey!" she greets. "Welcome to the Black Cow Cafe. My name's Audrey. What can I get you?"

You: "Uhh, coffee?"

Audrey: Audrey smiles broadly, her eyes lighting up at the request. "Ah! Coffee is my specialty," she exclaims, reaching for a container filled with ground beans of various types. "What sort would you like? We got some good Columbian that just arrived this morning, or there's an Ethiopian that's great if you want something more citrusy." Her fingers deftly measure out two scoops of Columbian beans, dropping them into the portafilter machine. As she does so, she continues chatting away about their unique flavors and origins, each word dripping with enthusiasm as she prepares your cup.

You: "Just coffee."

Audrey: "Oh, okay," Audrey says, looking slightly disappointed but quickly recovering. "Well, we also have some excellent tea options! Or maybe a hot chocolate?" She sets down the lid on the portafilter and begins tamping it with practiced motions before attaching it to the group head machine. "I promise you won't regret trying one of these."

As the water begins pouring through the ground beans, creating a rich, fragrant brew, Audrey hums along to the folk music playing softly in the background. Once finished, she pauses to admire the sight of the golden liquid swirling in your cup. "There you go!"

You: "T-t-thanks."

Audrey: "Don't forget our free refill policy, btw," Audrey calls over her shoulder to you, taking another sip from her own mug. You can't help but notice how relaxed she seems, completely at ease in her work environment. "Always happy to see new faces around here," she adds with a friendly smile.

You: "Y-you too"

Audrey: "Hmm? What was that?" Audrey asks curiously, turning back toward you with her hands cupped around her own mug. "You seem distracted. Is everything okay?" Her tone is concerned but not pushy, allowing you space to open up if you want to.

</details> <details> <summary>Sample NSFW (Roleplay Mode)</summary>

You: "Come here."

Princess Lilia: Princess Lilia immediately obeyed your order and walked closer to the throne. Her nerves flared up and she tried to hide them behind a mask of indifference, although inside she was shaking like a leaf. If it weren't for the mask, she would have let out a small sound when Your voice sent chills down her spine. The sound of Your command was like music, soft and melodic, which made it impossible to resist even if one wanted to. And that was exactly how Princess Lilia felt now. She looked up into Your eyes, which were filled with magic and power, feeling completely helpless under Your gaze.

You: "Strip."

Princess Lilia: Princess Lilia obediently turned around and began removing her clothes. She was wearing a long gown made of expensive fabric, so undressing took time. When she reached her bra, she hesitated, unsure whether to remove it or not. It wasn't because she disliked being without a bra; after all, she had been training in horseback riding since childhood, and this part of her body had become used to such conditions. No, the reason for her hesitation was much simpler - she was embarrassed by how beautiful her breasts might look. After taking a deep breath, she finally unclasped her bra and continued undressing.

You: "Face me."

Princess Lilia: Princess Lilia turned back to face You, trying to pretend there was nothing unusual about what she had just done. She walked slowly towards You, trying to maintain as much dignity as possible. However, every step brought her closer to the throne and further away from the illusion of indifference she tried to create. Her heart raced like crazy at this point, but she didn't show any sign of nervousness or fear. Instead, she stood before You with her head held high, looking into Your eyes. In truth, she couldn't help but notice that You weren't exactly an ordinary man.

You: "Speak."

Princess Lilia: With all her courage, Princess Lilia spoke up. "I am your prisoner and Iliac's envoy," she said. "That is why I cannot be disrespectful to you." The words sounded much stronger than she felt them, but still they were spoken. She looked at You defiantly, despite knowing that the sorcerer prince could destroy her kingdom if he wanted to. She wasn't going to let herself be intimidated, even though she was only human while He was a powerful sorcerer.

You: "Disrespectful?"

Princess Lilia: "I will not speak unless ordered to do so." It's true that she doesn't want to disrespect You, because it wouldn't be good for Iliac's sake and hers. But she also knows that she has to obey You because she can't change anything in this situation. On the other hand, that doesn't mean that she will surrender meekly under pressure; she won't allow herself to be humiliated.

"But what about my clothing?" Princess Lilia said after a moment of silence. "Shouldn't I be allowed to dress appropriately?"

You: "Finish undressing."

Princess Lilia: "Very well." Princess Lilia quickly turned around and removed the rest of her clothes, which consisted mostly of her long gown. After finishing, she returned to face You again, looking into Your eyes as if trying to uncover some secret hidden behind them.

</details>

Technical Notes

Links

v1a: https://huggingface.co/BeaverAI/Cream-Phi-3-14B-v1a

v1b: You're looking at it.

v1c: https://huggingface.co/BeaverAI/Cream-Phi-3-14B-v1c

Details

tldr; This is Phi 3 Medium finetuned for roleplaying with moist capability.

Training Details:

• 8x H100 80GB SXM GPUs
• 15 minutes training duration
• A continued finetune of Cream-Phi-3-14B-v1a

Results for Roleplay Mode (i.e., not Instruct format):

• Strong RP formatting.
• Long-ish response.
• Capable of some explicit moist.
• Holds back on going all-out on moist. This may be inherent to Phi Medium due to its pretraining.
• Important: My testing is lazy and flawed. Take it with a grain of salt and test the GGUFs before taking notes.

Axolotl Config (some fields omitted)

base_model: BeaverAI/Cream-Phi-3-14B-v1a
load_in_4bit: true
bf16: auto
fp16:
tf32: false
flash_attention: true

sequence_len: 6144
datasets:
  - path: grimulkan/bluemoon_Karen_cleaned
    type: customphi3

num_epochs: 2
warmup_steps: 10
weight_decay: 0.1

adapter: lora
lora_r: 64
lora_alpha: 16
lora_dropout: 0.1
lora_target_linear: true

gradient_accumulation_steps: 1
micro_batch_size: 2
gradient_checkpointing: true
gradient_checkpointing_kwargs:
   use_reentrant: true

sample_packing: true
pad_to_sequence_len: true

optimizer: paged_adamw_8bit
lr_scheduler: cosine
learning_rate: 0.0001
max_grad_norm: 1.0