arabic-sign-language-recognition

Arabic Sign Language Recognition Model

This model recognizes Arabic Sign Language (ArSL) gestures using MediaPipe hand landmarks.

Model Description

• Architecture: Multi-Layer Perceptron (MLP)
• Input: MediaPipe hand landmarks (63 features: 21 landmarks × 3 coordinates)
• Output: Arabic letters, numbers, and space character
• Framework: TensorFlow/Keras
• Formats: Keras (.keras) and TFLite (.tflite)

Architecture

Input: 63 features (MediaPipe landmarks)
Dense: 128 units, ReLU, Dropout 0.2
Dense: 64 units, ReLU, Dropout 0.2
Output: Softmax activation

Usage

Python (Keras)

import tensorflow as tf
import numpy as np
import pickle
from huggingface_hub import hf_hub_download

# Download model
model_path = hf_hub_download(
    repo_id="katyy2000/arabic-sign-language-recognition",
    filename="asl_mediapipe_new_version.keras"
)

encoder_path = hf_hub_download(
    repo_id="katyy2000/arabic-sign-language-recognition",
    filename="encoder.pkl"
)

# Load model and encoder
model = tf.keras.models.load_model(model_path)
with open(encoder_path, "rb") as f:
    encoder = pickle.load(f)

# Predict
landmarks = np.array([...]).reshape(1, -1)  # Your 63 landmark values
prediction = model.predict(landmarks)
predicted_class = encoder.inverse_transform([np.argmax(prediction)])
print(f"Predicted: {predicted_class[0]}")

TFLite (Mobile)

import tensorflow as tf
import numpy as np
from huggingface_hub import hf_hub_download

# Download TFLite model
tflite_path = hf_hub_download(
    repo_id="katyy2000/arabic-sign-language-recognition",
    filename="asl_mediapipe_new_version.tflite"
)

# Load interpreter
interpreter = tf.lite.Interpreter(model_path=tflite_path)
interpreter.allocate_tensors()

# Get I/O details
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Run inference
landmarks = np.array([...], dtype=np.float32).reshape(1, -1)
interpreter.set_tensor(input_details[0]['index'], landmarks)
interpreter.invoke()
output = interpreter.get_tensor(output_details[0]['index'])
predicted_class_idx = np.argmax(output)

Files

• asl_mediapipe_new_version.keras: Main Keras model (~250 KB)
• asl_mediapipe_new_version.tflite: TFLite model for mobile (~40 KB)
• encoder.pkl: Label encoder for class names

Requirements

tensorflow>=2.10.0
mediapipe>=0.10.0
numpy>=1.21.0
opencv-python>=4.5.0

Training Details

• Optimizer: Adam
• Loss: Categorical Crossentropy
• Class balancing applied
• Early stopping with patience=5

License

MIT License