music-4func

🎵 Music Assistant - 4 Functions (Fine-tuned FunctionGemma)

Fine-tuned FunctionGemma-270M for music control function calling using LoRA. Achieves 98.9% training accuracy and 100% test accuracy on 4 music control functions.

Model Details

Base Model

• Model: google/functiongemma-270m-it (270M parameters)
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Approach: Gradual scaling (part of 2→4→8→18 function roadmap)

Training Results

• Training Examples: 100 (80 train / 20 eval)
• Training Accuracy: 98.9%
• Evaluation Accuracy: 98.5%
• Test Accuracy: 100% (8/8 tests passed)
• Training Time: ~2.5 minutes on Mac M-series CPU
• Trainable Parameters: 3.8M (1.4% of base model)
• Adapter Size: ~15MB

Performance Comparison

🎯 Supported Functions

This model can call 4 music control functions:

1. play_song

Play a specific song by name or artist

Parameters:

• song_name (string, required) - Name of the song to play
• artist (string, optional) - Artist name
• album (string, optional) - Album name

Example:

Input: "Play Bohemian Rhapsody by Queen"
Output: call:play_song{song_name:<escape>Bohemian Rhapsody<escape>,artist:<escape>Queen<escape>}

2. playback_control

Control music playback

Parameters:

• action (string, required) - One of: play, pause, skip, next, previous, stop, resume

Example:

Input: "Pause the music"
Output: call:playback_control{action:<escape>pause<escape>}

3. search_music

Search for music by query, artist, album, or genre

Parameters:

• query (string, required) - Search query
• type (string, optional) - One of: song, artist, album, playlist, genre

Example:

Input: "Search for rock songs"
Output: call:search_music{query:<escape>rock songs<escape>}

4. create_playlist

Create a new playlist with a given name

Parameters:

• name (string, required) - Name of the playlist

Example:

Input: "Create a playlist called Workout Mix"
Output: call:create_playlist{name:<escape>Workout Mix<escape>}

🚀 Usage

Quick Start (Python)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "google/functiongemma-270m-it",
    torch_dtype=torch.float32,  # Use float32 for CPU, float16 for GPU
    device_map="cpu",  # or "auto" for GPU
    trust_remote_code=True
)

# Load tokenizer and fine-tuned adapter
tokenizer = AutoTokenizer.from_pretrained("google/functiongemma-270m-it")
model = PeftModel.from_pretrained(base_model, "Jageen/music-4func")

# Optional: Merge for faster inference
model = model.merge_and_unload()

# Define your functions (same as training)
FUNCTIONS = [
    {
        "type": "function",
        "function": {
            "name": "play_song",
            "description": "Play a specific song by name or artist",
            "parameters": {
                "type": "object",
                "properties": {
                    "song_name": {"type": "string", "description": "Name of the song"},
                    "artist": {"type": "string", "description": "Artist name (optional)"},
                    "album": {"type": "string", "description": "Album name (optional)"}
                },
                "required": ["song_name"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "playback_control",
            "description": "Control music playback",
            "parameters": {
                "type": "object",
                "properties": {
                    "action": {
                        "type": "string",
                        "enum": ["play", "pause", "skip", "next", "previous", "stop", "resume"],
                        "description": "Playback action"
                    }
                },
                "required": ["action"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_music",
            "description": "Search for music",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "Search query"},
                    "type": {
                        "type": "string",
                        "enum": ["song", "artist", "album", "playlist", "genre"],
                        "description": "Type of search"
                    }
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "create_playlist",
            "description": "Create a new playlist",
            "parameters": {
                "type": "object",
                "properties": {
                    "name": {"type": "string", "description": "Playlist name"}
                },
                "required": ["name"]
            }
        }
    }
]

# Test the model
def predict(user_input):
    messages = [{"role": "user", "content": user_input}]

    prompt = tokenizer.apply_chat_template(
        messages,
        tools=FUNCTIONS,
        add_generation_prompt=True,
        tokenize=False
    )

    inputs = tokenizer(prompt, return_tensors="pt")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=128,
            do_sample=False,
            pad_token_id=tokenizer.eos_token_id
        )

    response = tokenizer.decode(
        outputs[0][inputs['input_ids'].shape[1]:],
        skip_special_tokens=False
    )

    return response

# Test examples
print(predict("Play Bohemian Rhapsody"))
print(predict("Pause the music"))
print(predict("Search for rock songs"))
print(predict("Create a playlist called Chill Vibes"))

Expected Output Format

The model generates function calls in FunctionGemma format:

<start_function_call>call:function_name{param1:<escape>value1<escape>,param2:<escape>value2<escape>}<end_function_call>

📊 Training Details

LoRA Configuration

LoraConfig(
    r=16,                    # LoRA rank
    lora_alpha=32,           # LoRA alpha
    target_modules=[         # All 7 modules (critical!)
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj"
    ],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

Training Hyperparameters

• Epochs: 5
• Batch size: 2 (per device)
• Gradient accumulation steps: 4 (effective batch size: 8)
• Learning rate: 2e-4
• Optimizer: AdamW
• Scheduler: Linear warmup
• Training examples per function: 25
• Total training time: ~2.5 minutes on Apple M-series CPU

Dataset Format

Training data formatted using FunctionGemma's chat template:

messages = [
    {"role": "user", "content": "Play Bohemian Rhapsody"},
    {
        "role": "assistant",
        "tool_calls": [{
            "type": "function",
            "function": {
                "name": "play_song",
                "arguments": {"song_name": "Bohemian Rhapsody"}  # Dict, not JSON string
            }
        }]
    }
]

📈 Test Results

Tested on 8 diverse commands:

Success Rate: 100% (8/8)

Comparison with Base Model

🎓 Key Learnings

What Worked

1. Gradual scaling approach - Starting with 2 functions, then 4 (this model)
2. Complete LoRA config - All 7 target modules are critical
3. Proper data format - Pass dicts, never json.dumps()
4. 25+ examples per function - Sufficient for pattern learning
5. Diverse natural language - Varied phrasings improve generalization

Critical Configuration

⚠️ Important: Missing any of the 7 LoRA target modules causes silent failure (model generates only pad tokens). Always include all modules shown above.

🚀 Deployment Options

Python Application

Use the code example above for any Python application.

iOS Deployment

// Using HuggingFace Swift SDK
import Transformers

let model = HuggingFaceModel(
    modelId: "Jageen/music-4func",
    baseModel: "google/functiongemma-270m-it"
)

Android Deployment

// Using HuggingFace Android SDK
import co.huggingface.transformers.*

val model = PeftModel.fromPretrained(
    baseModel = "google/functiongemma-270m-it",
    adapter = "Jageen/music-4func"
)

Google Colab

For testing with GPU acceleration:

# Use torch.float16 and device_map="auto" for GPU
base_model = AutoModelForCausalLM.from_pretrained(
    "google/functiongemma-270m-it",
    torch_dtype=torch.float16,
    device_map="auto"
)

🔗 Related Models

• Jageen/music-2func - 2 functions (play_song, playback_control) - 100% accuracy
• Jageen/music-8func - Coming soon (8 functions with playlist management)
• Jageen/music-18func - Coming soon (complete music control suite)

📚 Resources

• Blog Post: Fine-Tuning FunctionGemma: From 75% to 100% Accuracy (coming soon)
• Code Repository: GitHub
• FunctionGemma Docs: Google AI
• LoRA Paper: arXiv:2106.09685

⚠️ Limitations

• Domain-specific: Optimized for music control, may not generalize to other domains
• Function schema required: Needs exact function definitions used during training
• Language: Primarily trained on English commands
• Context: Works best with clear, direct commands (not conversational context)
• Scale: Designed for 4 functions; for more functions, see music-8func or music-18func

📄 License

This model is based on FunctionGemma and inherits the Gemma License. The fine-tuning code and training approach are licensed under Apache 2.0.

🙏 Acknowledgments

• Google for FunctionGemma and comprehensive documentation
• HuggingFace for transformers, PEFT, and TRL libraries
• Open-source community for LoRA research

📧 Contact

For questions, issues, or collaboration:

• Open an issue on GitHub
• Model page: HuggingFace

---

Built with ❤️ using FunctionGemma and LoRA fine-tuning