semantic-image-clustering

Model description

This is a image clustering model trained after the Semantic Clustering by Adopting Nearest neighbors (SCAN)(Van Gansbeke et al., 2020) algorithm.

The training procedure was done as seen in the example on <a href='https://keras.io/examples/vision/semantic_image_clustering/' target='_blank'>keras.io</a> by Khalid Salama.

The algorithm consists of two phases:

1. Self-supervised visual representation learning of images, in which we use the simCLR technique.
2. Clustering of the learned visual representation vectors to maximize the agreement between the cluster assignments of neighboring vectors.

Intended uses & limitations

The model is intended to show the effective use of self-supervised learning combined with nearest neighbours for (semantic) image clustering.

You can use these clusters to retrieve images of the same class.

Limitations

This model is not supposed to show any superiority to image classification from supervised learning, but as a POC that unsupervised learning is able to cluster similar images together without any labels.

Possible Improvements:

As given by the original author on keras.io, these steps can be taken to improve the accuary further:

1. increase the number of epochs in the representation learning and the clustering phases;
2. allow the encoder weights to be tuned during the clustering phase
3. perform a final fine-tuning step through self-labeling, as described in the original SCAN paper

Training and evaluation data

Training Data

The model was trained using the CIFAR-10 dataset. For training the images were scaled to (32,32,3).

Hyperparameters

For training the following parameters were used:

• Feature Vector Dimension: 512
• Projection Units of Head: 128
• Number of Cluster: 20
• K-Neighbours: 5

The encoder was not tuned during clustering.

Evaluation

Visualization of highest confidence cluster picks

Clusters and their respective labels, accuracy and size

Model Plot

<details> <summary>View Model Plot</summary>

</details>