gemma-3-270M-Chatty-MLX

Chatty: Gemma 3 270M IT SFT LoRA Fine-Tune

Version: 0.1 It can't be your daily driver yet The "true" tiny Gemma 3 270M can't really chat. So this is an attempt at getting a decent (multi turn) chat model out of it. Also an experiment in dataset curation, building fine-tuning pipelines and seeing where a 270M would get saturated/can't improve further without regressing on important bits.

This is the MLX variant for running on Apple silicon (Mac's, iPhone's, and iPads). For other platforms you can use the LoRA to merge and create. I will upload a GGUF verions later.

Intended Use

Local Chat model for personal use.

Limitations

• Can't go beyond English
• Wouldn't have in-depth world-knowledge
• VERY likely to hallucinate

Inference Settings

Gemma's Recommended Settings:

• 'temperature = 1'
• 'Top P = 0.95'
• 'Top K = 64'

Changelog

• 0.1 Can we at least get it to chat & follow instrutuctions instead of looking like it's chatting & following instructions?
  • Training: SFT using superior chat model outputs

Examples

Version 0.1 Examples

Prompt:'Explain different parts of "reasoning", what all does it include'

Before: ''' Okay, I'm ready to explain different parts of reasoning! To give you the best explanation, please tell me what type of reasoning you'd like to understand. For example:

• Are you interested in deductive reasoning, inductive reasoning, or some other type?
• Do you want to understand a specific example or situation?
• Are you looking for an explanation of how reasoning works, or just a general overview?

The more information you provide, the better I can tailor my explanation. '''

After: '''

• Logical deduction: This involves making deductions from general premises to arrive at a specific conclusion. It involves reasoning from general principles to specific instances.

• Deductive reasoning: This involves making logical deductions from specific premises to arrive at a general conclusion. It involves reasoning from general principles to specific instances.

• Inductive reasoning: This involves making general conclusions from specific instances. It involves reasoning from specific instances to a general conclusion.

• Inductive reasoning: This involves making general conclusions from specific instances. '''

Credits

• Gemma 3
• Unsloth
• MLX and MLX LM
• And countless community resources