Google-SpeciesNet-ONNX

👁️ AI Vision

Using small machine learning or AI models we will be performing various Computer Vision related operations, for example Object Detection on images. This project purely focused on cross-platform applications & we can run small AI/ML models on our mobile phones in offline mode.

This project is buid on kivy, kivymd and uses onnxruntime, numpy, opencv etc. to perform the tasks. This is still at very early phase before this project matures at some level.

<a href="https://github.com/daslearning-org/vision-ai/releases" target="_blank" rel="noopener noreferrer"><img alt="GitHub Downloads (all assets, all releases)" src="https://img.shields.io/github/downloads/daslearning-org/vision-ai/total"></a> <a href="https://www.youtube.com/watch?v=wUABgn4JYc4" target="_blank" rel="noopener noreferrer"><img alt="GitHub Downloads (all assets, all releases)" src="https://img.shields.io/youtube/views/wUABgn4JYc4"></a>

Features

1. Objects detection from Image in local storage. It uses SSD-MobilenetV1
2. Directly capture from Camera & detect objects on the image. It also uses the same model as above.
3. Image Prediction: if you have an image of a single object & want to get top-5 predictions about the image, just upload & click Classify. It uses ResNet18 which has around 1000 categories.
4. Identify Species from a photo. It uses Google SpeciesNet & it has around 2000 odd species.
5. No Ads, completely open-source (free).
6. No trackers, no data collection etc. Actually you own your data & your app.

📽️ Demo

You can play below Video or click this Youtube Link to see the demo. Please let me know in the comments, how do you feel about this App. <br>

<iframe width="560" height="315" src="https://www.youtube.com/embed/wUABgn4JYc4" title="YouTube video player" frameborder="0" allow="accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>

🖧 Our Scematic Architecture

To be added...

🧑‍💻 Quickstart Guide

📱 Download & Run the Android App

You can check the Releases and downlaod the latest version of the android app on your phone.

1. If you want to select an image from phone memory, do it either from DCIM or Downloads or Pictures or any subfolders of this three folders. Same goes while saving the output image too.

2. If you use the camera detection, please note that it uses back camera in Landscape mode only, so you may need to rotate your phone to capture it properly.

Requirements

• Minimum Android 9

💻 Download & Run the Windows or Linux App

To be built later. For now you may use with python directly as mentioned below.

🐍 Run with Python

1. Clone the repo

git clone https://github.com/daslearning-org/vision-ai.git

1. Run the application

cd vision-ai/onnx/
pip install -r requirements.txt # virtual environment is recommended
python main.py

📥 More open-source apps from us

You may check our other apps from our app list

🦾 Build your own App

The Kivy project has a great tool named Buildozer which can make mobile apps for Android & iOS

📱 Build Android App

A Linux environment is recommended for the app development. If you are on Windows, you may use WSL or any Virtual Machine. As of now the buildozer tool works on Python version 3.11 at maximum. I am going to use Python 3.11

# add the python repository
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update

# install all dependencies.
sudo apt install -y ant autoconf automake ccache cmake g++ gcc libbz2-dev libffi-dev libltdl-dev libtool libssl-dev lbzip2 make ninja-build openjdk-17-jdk patch patchelf pkg-config protobuf-compiler python3.11 python3.11-venv python3.11-dev

# optionally we can default to python 3.11
sudo ln -sf /usr/bin/python3.11 /usr/bin/python3
sudo ln -sf /usr/bin/python3.11 /usr/bin/python
sudo ln -sf /usr/bin/python3.11-config /usr/bin/python3-config

# optionally you may check the java installation with below commands
java -version
javac -version

# install python modules
git clone https://github.com/daslearning-org/vision-ai.git
cd vision-ai/onnx/
python3.11 -m venv .env # create python virtual environment
source .env/bin/activate
pip install -r req_android.txt

# build the android apk
buildozer android debug # this may take a good amount of time for the first time & will generate the apk in the bin directory