Florence-2-large-ft-Transformers-Fix

Model Card: Florence-2-large-ft (Transformers 4.57.3 Compatible)

Model Overview

This is the Florence-2-large-ft (Fine-Tuned) model from Microsoft Corporation, made compatible with transformers==4.57.3.

Original Model Information

• Model Name: Florence-2-large-ft
• Original Authors: Microsoft
• Original Repository: microsoft/Florence-2-large-ft
• Model Type: Vision-Language Model (Encoder-Decoder) - Fine-Tuned Version
• Purpose: Domain-Specific Vision-Language Understanding and Generation

License & Ownership

Original License: MIT License
Original Copyright: © 2024 Microsoft Corporation

This modified version maintains the original MIT license and respects all original copyright notices. All original code, model weights, and fine-tuning remain the intellectual property of Microsoft Corporation.

Modifications Made

Issue Addressed

Fixed compatibility issues with transformers==4.57.3 where AttributeError: 'NoneType' object has no attribute 'shape' occurred during model inference and generation.

Root Cause

In transformers 4.57.3, the past_key_values structure can contain None elements at various nesting levels, particularly during early generation stages. The original code only checked for top-level None but not nested None values.

Changes Applied

1. Attention Layer Fixes (Florence2Attention and Florence2FlashAttention2)

• Location: Lines ~804, ~947, ~1160 in modeling_florence2.py
• Change: Added past_key_value[0] is not None check before accessing .shape[2]
• Pattern:

  # Before
  if (is_cross_attention and past_key_value is not None 
      and past_key_value[0].shape[2] == key_value_states.shape[1]):
  
  # After
  if (is_cross_attention and past_key_value is not None 
      and past_key_value[0] is not None
      and past_key_value[0].shape[2] == key_value_states.shape[1]):
  

2. KV Sequence Length Calculation

• Location: Line ~979 in modeling_florence2.py
• Change: Added past_key_value[0] is not None check before shape access
• Pattern:

  # Before
  if past_key_value is not None:
      kv_seq_len += past_key_value[0].shape[-2]
  
  # After
  if past_key_value is not None and past_key_value[0] is not None:
      kv_seq_len += past_key_value[0].shape[-2]
  

3. Generation Input Preparation

• Location: Lines ~2212, ~2835 in modeling_florence2.py
• Change: Wrapped past_key_values access in try-except blocks with fallbacks
• Pattern:

  # Before
  past_length = past_key_values[0][0].shape[2]
  
  # After
  try:
      past_length = past_key_values[0][0].shape[2]
  except (AttributeError, TypeError, IndexError):
      past_length = 0
  

Files Modified

• modeling_florence2.py - Multiple locations for robustness

Backward Compatibility

✅ Fully backward compatible with older transformers versions
✅ Forward compatible with future transformers versions using DynamicCache

Usage

from transformers import AutoModel, AutoProcessor
import torch

# Load the fine-tuned model
model = AutoModel.from_pretrained(
    "path/to/Florence-2-large-ft",
    trust_remote_code=True
)
processor = AutoProcessor.from_pretrained(
    "path/to/Florence-2-large-ft"
)

# Generate with compatibility fixes
model.eval()
with torch.no_grad():
    inputs = processor(text="Analyze this image", return_tensors="pt")
    outputs = model.generate(**inputs, max_new_tokens=75)

Technical Details

Tested With

• Transformers: 4.57.3
• PyTorch: 2.0+
• Python: 3.8+

Fine-Tuning Notes

This is a fine-tuned version optimized for specific tasks. Refer to original Microsoft repository for fine-tuning details and training procedures.

Testing Recommendations

Test the following scenarios to ensure compatibility:

1. Single image caption generation
2. Batch processing multiple images
3. Long sequence generation with many tokens
4. Memory efficiency with large models

Acknowledgments

• Original Model Developers: Microsoft Corporation
• Fine-Tuning: Microsoft Corporation
• Modifications: Compatibility layer for transformers 4.57.3
• License Preservation: Original MIT license maintained

References

• Original Model: https://huggingface.co/microsoft/Florence-2-large-ft
• Florence Paper: [Reference provided in original repository]
• Transformers Library: https://github.com/huggingface/transformers

---

Version: 1.0 (Transformers 4.57.3 Compatible)
Last Updated: March 2026
Status: Production Ready