gemma-3-1b-it-linkedin-natural

gemma-3-1b-it-linkedin-natural — QLoRA adapter for natural LinkedIn posts

Fine-tuned from google/gemma-3-1b-it to write simple, conversational, twitter-like LinkedIn posts — short sentences, 1-2 line paragraphs, no cringe drama.

No: Thrilled to announce, Humbled to share, Game changer, rocket spam 🚀🔥, 10 hashtags, or Let's dive in 👇.

Yes: 60–120 words, one idea per post, bracket asides (like this), ends casual That's it. / Peace ✌🏻, sounds like talking to a friend.

Built as a learning project for dataset curation → QLoRA → HF publish on a T4.

Dataset

harsh-jos/linkedin-natural-150 — 150 curated examples (138 train / 12 val), avg 60.7w.

• 44 twitter-gold short (<50w) — witty, no-bullshit, punchy
• 84 core short-medium (50–80w) — main LinkedIn voice
• 20 medium-long (80–150w) — readable stories
• 2 long gold — original rulebook/Kiro posts

Format per line (JSONL):

{"prompt": "Write a LinkedIn post about: Why readability beats drama", "response": "Readability beats drama. Always..."}

HF splits are already Gemma 3 chat-formatted with text (<start_of_turn>user/model).

Training

• Base: google/gemma-3-1b-it via unsloth/gemma-3-1b-it-bnb-4bit (4-bit NF4 storage, float32 compute — Gemma3 doesn't support float16 on T4)
• Method: QLoRA — r=16, alpha=16, dropout 0, targets: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj, 4-bit, gradient checkpointing unsloth
• Hyperparams: lr 2e-4, cosine, warmup 0.05, batch 2 × grad_accum 4 = 8, epochs 4 (72 steps), max_seq 1024, adamw_8bit, weight_decay 0.01
• Hardware: Colab T4 14.5GB (~25 mins), Torch 2.11 + CUDA 12.8
• Loss: train 3.19 → 2.37, val 2.86 → 2.46 (no overfit, val ≈ train)
• Code: ml-exp/notebooks/train_colab_self_contained.ipynb (self-contained, no GitHub needed)

Prompt template (required for Gemma 3)

<start_of_turn>user
Write a LinkedIn post about: Why readability beats drama
<end_of_turn>
<start_of_turn>model

The adapter was trained on this exact template. Omitting <start_of_turn> will degrade style.

Usage

PEFT adapter (recommended — what you pushed)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "google/gemma-3-1b-it"
adapter_id = "harsh-jos/gemma-3-1b-it-linkedin-natural"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

prompt = "Write a LinkedIn post about: Why readability beats drama"
template = "<start_of_turn>user\n{prompt}<end_of_turn>\n<start_of_turn>model\n"
inputs = tokenizer(template.format(prompt=prompt), return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=200, temperature=0.7, top_p=0.9, repetition_penalty=1.05, do_sample=True)
text = tokenizer.decode(out[0], skip_special_tokens=True).split("<start_of_turn>model")[-1].replace("<end_of_turn>", "").strip()
print(text)

If you add a merged 16-bit later

# After merging outside 4-bit:
# model_fp16, tok = FastLanguageModel.from_pretrained("google/gemma-3-1b-it", load_in_4bit=False)
# model_fp16 = PeftModel.from_pretrained(model_fp16, adapter_id).merge_and_unload()
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(adapter_id, device_map="auto") # if merged repo

Examples

Prompt: Why readability beats drama

Base Gemma (cringe): 🚀 Thrilled to announce... In today's fast-paced world, let's dive into the tapestry of readability! Unlock the power... #AI #Leadership 👇

This adapter: Readability beats drama. Always.

Dramatic posts get views. Readable posts get replies.

I used to write long, emotional stories. People scrolled. Now I write short sentences. One idea. Line breaks where you need to breathe.

Same story, but now people feel I'm talking to them, not performing.

Prompt: How I cut a 300 word draft to 120 words

My first draft is always 300 words. Final is 120.

How I cut: first pass, delete every sentence that doesn't support the one idea. Second pass, shorten every remaining sentence by 30%. Third pass, read aloud — if I stumble, rewrite.

Most posts have one good paragraph hidden in three okay ones. Find it, keep it, delete the rest.

Prompt: Boring tech wins

Boring tech wins.

Postgres, Next.js, Vercel. Nothing exciting. Everything works.

Exciting tech is for conference talks. Boring tech is for shipping.

Evaluation

• Qualitative: 12 val prompts compared side-by-side (base vs adapter). Adapter consistently shorter, fewer buzzwords, more bracket asides and casual closings.
• Quantitative: val loss 2.46 (train 2.37) — no overfit. For style, human vibe matters more than ROUGE; ROUGE-L on val ~0.45 is typical for paraphrase style.

Limitations & intended use

• English only, optimized for 60–150w. Very long posts (>250w) may drift.
• Opinionated simple style — not for formal/enterprise tone, not for non-LinkedIn long-form.
• May still hallucinate facts — verify before posting.
• Gemma license applies. Requires accepting google/gemma-3-1b-it terms.

Citation

@misc{linkedin-natural-150,
  title={LinkedIn Natural 150 — Twitter-like LinkedIn style tuning for Gemma 3 1B},
  author={Harsh Joshi},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/datasets/harsh-jos/linkedin-natural-150}}
}

Acknowledgements

Dataset built from 3 gold posts + 147 synthetic rewrites in the author's voice, Unsloth QLoRA, TRL. Inspired by the rulebook in CLAUDE.md for agentic coding.