ImageBinds

Pruning the ImageBind Model for Efficiency

This repository contains a version of the ImageBind model that has been significantly optimized for efficiency and resource usage.

Why prune?

The original ImageBind model, while powerful, is very large (4GB). I reduced its size by excluding weights for specific modalities, focusing only on those I need most. I did this with my custom pruning technique to reduce its size.

I implemented this pruning strategy by:

• Custom code implementation: I developed custom code that allows for the exclusion of weights for certain modalities from the original model. This included vision, audio, thermal, mlu, depth, and combined vision & audio models.
• Focus on specific needs: This pruning allowed me to create separate vision and audio models, enabling specialized functionalities tailored to individual tasks.

Benefits of this optimized model:

• Reduced model size (compared to the original ImageBind model)
• Faster inference speeds (due to reduced complexity)
• Enhanced efficiency for resource-constrained environments

Feel free to explore the models within this repository and see how it is! Let me know if you have any questions about specific details or want to discuss how this implementation affects your tasks.