Muse_Glimmer_30B_finetuned_mlabonne_16bit

Uploaded model

• Developed by: aliRafik
• License: apache-2.0
• Finetuned from model: unsloth/Muse-Glimmer-30B

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

Muse Glimmer 30B - LoRA Fine-tuned

Model Description

LoRA adapter for unsloth/Muse-Glimmer-30B, a 30B parameter vision-language model. Fine-tuned using parameter-efficient LoRA on Kaggle with 2x T4 GPUs.

Model Architecture:

• Base: Muse Glimmer 30B (Vision-Language)
• Training Method: LoRA (Low-Rank Adaptation)
• Precision: FP16 (adapters)
• Base Model Loading: 4-bit quantization

Training Details

Usage

Installation

pip install transformers accelerate peft torch bitsandbytes

Load Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model with 4-bit quantization
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/Muse-Glimmer-30B",
    torch_dtype=torch.float16,
    load_in_4bit=True,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(
    base_model,
    "aliRafik/Muse-Glimmer-30B-finetuned-mlabonne"
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    "aliRafik/Muse-Glimmer-30B-finetuned-mlabonne"
)

Inference

def generate_response(prompt, max_length=256):
    inputs = tokenizer(prompt, return_tensors="pt")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_length,
            temperature=0.7,
            do_sample=True,
            top_p=0.9,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id
        )

    return tokenizer.decode(
        outputs[0],
        skip_special_tokens=True
    )


# Example
response = generate_response("Your prompt here")
print(response)

Memory Requirements

Download Statistics

• Adapter: 0.66 GB
• Base Model: 34.53 GB (FP16)

Disclaimer

This is a LoRA adapter, not a standalone model. Must be loaded with the base Muse Glimmer 30B model.

Citation

@misc{muse-glimmer-30b-lora-2024,
  author = {Ali Rafik},
  title = {Muse Glimmer 30B - LoRA Fine-tuned Adapter},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/aliRafik/Muse-Glimmer-30B-finetuned-mlabonne}}
}