squeezebert-uncased-litert

squeezebert/squeezebert-uncased - LiteRT Optimized

This is a LiteRT (formerly TensorFlow Lite) export of squeezebert/squeezebert-uncased.

It is optimized for mobile and edge inference (Android/iOS/Embedded).

Model Details

Usage

import numpy as np
from ai_edge_litert.interpreter import Interpreter
from transformers import AutoTokenizer

# Load model
interpreter = Interpreter(model_path="squeezebert_squeezebert-uncased.tflite")
interpreter.allocate_tensors()

tokenizer = AutoTokenizer.from_pretrained("squeezebert/squeezebert-uncased")

def get_embedding(text):
    inputs = tokenizer(text, max_length=128, padding="max_length", truncation=True, return_tensors="np")

    input_details = interpreter.get_input_details()
    interpreter.set_tensor(input_details[0]['index'], inputs['input_ids'].astype(np.int64))
    interpreter.set_tensor(input_details[1]['index'], inputs['attention_mask'].astype(np.int64))

    interpreter.invoke()

    output_details = interpreter.get_output_details()
    return interpreter.get_tensor(output_details[0]['index'])[0]

emb = get_embedding("Hello world")
print(f"Embedding shape: {emb.shape}")

Converted by Bombek1 using litert-torch