Performing Image Classification on Out of Distribution Dataset, the Tiyaro way.

Image Classification is the task of labeling images into one of a number of predefined classes. It has a plethora of different use cases like identifying different species of plants through given images, classifying different types of tumors, analyzing different emotions from facial images, inspecting the quality of products in the factory, and so on. To perform these tasks Data Scientists require to train specialized Image Classification models either from scratch or by using transfer learning on pre-trained Image Classification models or by using one of the SaaS vision services provided by various vendors. There are numerous Image Classification pre-trained models and SaaS services available on the internet and the task of finding the perfect solution for the use case is a tedious one. Tiyaro makes it easier for everyone from a Data Scientist to an enthusiast to compare and select different machine learning models and SaaS services. Making the task of finding and creating a solution from weeks to merely a few minutes.

Simple Demo of Image Classification on Tiyaro

Accelerating the rate of model evaluation using: Tiyaro Experiments

We evaluate and compare the performance of various popular open-sourced pre-trained image classification models and SaaS services offered by different vendors in a quick and easier fashion. Most of the pre-trained image classification models are trained on ImageNet-1k or ImageNet-21k . For our experiment, we utilize a subset of ImageNet-O which is an out-of-distribution dataset. ImageNet-O consists of images from classes that are not found in the ImageNet-1k dataset. It is used to test the robustness of vision models to out-of-distribution samples. We use the following pre-trained image classification models and SaaS services in our experiment:

• Azure Cognitive Vision
• GCP Vision API
• EfficientNet
• SegFormer encoder fine-tuned on Imagenet-1k
• BEiT model pre-trained on ImagNet-21k

To perform this experiment we start by searching for "Image Classification" in the Tiyaro Console

The search result gives us various models and SaaS services that you can use to run experiments, you can bookmark your favorite models.

There are various ways to create experiments in Tiyaro one of the ways is to go to

After naming the experiment you can select the class for which to conduct the experiment, for our experiment we can select the image classification.

After selecting the class you can upload your custom dataset, there are various formats to upload the vision dataset on Tiyaro for the experiment.

You can select the models to perform the experiment on, there are various filters available to simplify the process of selection of models.

Once the models are selected you can run the experiment, the experiment would be updated periodically to reflect the status of the experiment, after the experiment is completed you can go over to the results tab to see the latency of the models as well as the results of the experiment, for our use case we are able to see the confidence of the prediction of the model as well as what the prediction is. We can also download the result in a zip file.

As seen from the results the models' performance is not surprising as most of them have low confidence in the out-of-distribution images, even the APIs offered by some SaaS vendors perform poorly on some of the images.

You can share the experiment with your coworkers and on social media. Also, you can make a copy of the given experiment to enhance or modify the particular experiment.

Wish to create one? Head on over to Tiyaro !