Evaluating performance of Object Detection Models using Tiyaro Experiments

Object detection is the task to identify and locate objects in an image or video. Object detection can be used in various different ways like counting an object in a given image or video, tracking the identified object, determining the location of the object, and labeling the object. Object detection has a plethora of use cases and is in use across different industries from oil and gas to retail to surveillance. The most common use case is identifying the make and model of the car as well as reading the license plate. Whatever may be the use case, it is important to identify the correct pre-trained object detection model or a SaaS Vision API, to create the solution. The task of finding and evaluating a solution can take not only weeks but months, but by using Tiyaro the effort is reduced merely to minutes.

Using Tiyaro to search and demo object detection models.

Accelerating the rate of model evaluation using: Tiyaro Experiments

Tiyaro Experiments allows us a quicker way to compare different pre-trained machine learning models along with the SaaS Vendor APIs helping us accelerate the evaluation process. One such object detection experiment we conduct is to detect the cars in a given dataset of car images. For that we utilize the following State of the Art pre-trained machine learning models / SaaS API :

• GCP Vision API
• AWS Rekognition API
• Mask RCNN
• Azure Cognitive Vision API
• DEtection TRansformer (DETR)

Car Images in different lightning conditions, shot at different angles used for the experiment .

As seen in the video above we start by searching the models by using Tiyaro Explore . Once we have completed searching and using demos for different models we can head on over to create experiments. There are various ways to create experiments but we are simply going over to the experiments tab. Where we start a new experiment.

After that, we can select our experiment class type, which in our case would be Object Detection.

Once completed we can select the models to train the experiments on, the model selection provides us with various filters.

After selecting the models / SaaS API vendor solution you can upload your custom dataset, there are various formats to upload the vision dataset on Tiyaro for the experiment.

After running the experiment we can see the results tab show up, here you would also be able to see the latency of the models as well as the results of the experiment, for our use case we are able to see the table containing our input and the respective model predictions. We can also download the result in a zip file.

As you can see the results for our experiment contains the different identified objects in the given images as well as the confidence score for each prediction.

The results are not surprising considering that most models / SaaS API vendors are able to identify a car. The interesting thing to note is that Azure Cognitive Vision API has low confidence among all the predictions performed for our dataset. As seen from this experiment we shave off a week's worth of effort by discovering and identifying the best in class solution for our use case.

You can share the experiment with your coworkers and on social media. Also, you can make a copy of the given experiment to enhance or modify the particular experiment.

Wish to create one? Head on over to Tiyaro !