gemma-2-text-rewriter-gguf

Gemma-2-9b Human-Like Text Rewriter (GGUF Format)

This model is a fine-tuned version of Google's Gemma-2-9b, specialized in rewriting AI-toned or robotic text into natural, human-like writing. It was trained using Unsloth for 4-bit quantization and high-efficiency performance.

Example Notebook

🔗Kaggle notebook here

Model Details

• Base Model: unsloth/gemma-2-9b (merged from unsloth/gemma-2-9b-bnb-4bit)
• Fine-tuned by: Heatw4ve
• Fine-tuning Tool: Unsloth
• Quantization: GGUF (q4_k_m)
• Task: Style Transfer / Text Rewriting
• Prompt Format: Gemma-2 Chat Template
• Max Sequence Length: 512 tokens
• Language: English

Training Configuration

The model was fine-tuned with the following parameters:

• Learning Rate: 5e-5
• Optimizer: AdamW 8-bit
• LR Scheduler: Cosine
• Warmup Steps: 100
• Max Steps: 900
• LoRA Config: - Rank (r): 64
  • Alpha: 128
  • Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Effective Batch Size: 64 (8 per device * 8 grad accumulation)

Usage

System Prompt

To achieve the intended rewriting style, use the following system prompt:

You are a helpful assistant that rewrites AI-toned text into natural, human-like writing.

User Prompt Format

Rewrite the following text to sound like a real human wrote it:

[INSERT TEXT HERE]

Example Code

llm = Llama(
  model_path="/path/to/your/model",
  n_ctx=8192,
  n_gpu_layers=16,
  n_threads=4,
  verbose=verbose,
  chat_format="gemma",
)

text='''
William Shakespeare, the master of human insight, gifted us with a timeless observation that cuts to the heart of intellect and humility: "The fool doth think he is wise, but the wise man knows himself to be a fool." Uttered by the character Touchstone in As You Like It, this seemingly paradoxical statement is far more than a witty quip; it's a profound commentary on self-awareness, the nature of true wisdom, and the perpetual quest for knowledge.
'''

messages = [
  {"role": "system", "content": "You are a helpful assistant that rewrites AI-toned text into natural, human-like writing."},
  {"role": "user", "content": f"Rewrite the following text to sound like a real human wrote it:\n\n{text}"}
]

output = llm.create_chat_completion(
  messages=messages,
  max_tokens=512,
  temperature=random.uniform(0.9, 1.9),
  top_p=random.uniform(0.87, 0.96),
  seed=random.randint(2, 2**32),
  stop=["<end_of_turn>", "<eos>", "</s>"]
)

print(output["choices"][0]["message"]["content"].strip())

🔗Connect with me on LinkedIn