Detectron2 is the next iteration of AI Models from Facebook Research, curated and supported within a common Pytorch framework Significant efforts have advanced this framework both in terms of depth and breadth of supported AI models, but what are the relevant changes and/or improvements for specific AI user with a specific application in mind? The Tiyaro Compare function can help to answer this question.

With the recent Detectron2 integration at Tiyaro and support of other ML infrastuctures, Tiyaro Compare allows an AI consumer to:

1. Build a quick cohort of comparable AI/ML models,
2. Select your own customer-relevant test set, and then
3. Evaluate the improvements and/or regressions against a known historical baseline or the general quality of model that exist with the class.
4. Woot! Profit! In this blog, we will quickly and efficiently review the comparisons of the Faster RCNN model between Detectron2 and other publishers for a hypothetical bird watching application (cleverly marketed as Bird-Brains).

Build the model cohort

1. Starting with the search function, go to the Tiyaro console and enter the search term "detectron2", as shown in screenshot below with the red highlight, and press the search button.
2. As seen in the yellow highlight in the image below, the results of this basic search are 292 models related to Detectron2 -- both from the Detectron2 library itself or other related offerings from other distributions such as Tensorflow and the original Facebook standalone models. So, we have a cohort to test, but it's just too unwieldy.
3. To narrow the model field further, we apply our hypothetical Birds-Brain application constraints: a bird watching application that handle long latency for high quality recognition, using an RCNN variant algorithm. We can add another search term "RCNN", press the search button again and then set the search filters drop-down menus (highlighted in the red above) to the following values: Model Type = Computer Vision -> Object Detection (ONLY) and Size -> Large (ONLY). These filter settings will leave a manageable, but relevant set of models, (highlighted in yellow in the image below).
4. Press the "Compare" button (highlighted in red above) to forward the relevant search results to Tiyaro Compare (highlighted in yellow below).
5. Allthough Tiyaro can conveniently select the models of interest automatically, we will choose to manually select the models by pressing the "Let Me Choose Manually" button (highlighted in red above), and a table of our search results will be shown below (highlighted in yellow below).
6. We select 3 models: Detectron2 model, a previous Facebook model released independently, and a model onboarded from the TensorFlow hub, by selecting the three checkboxes (highlighted in red above right), and then press the Next Button (highlighted in above left).

We're now ready to create our test set for our Bird-Brain test cohort.

Create your own test set

Although standard test sets are excellent for general objective model performance, the test set of the Bird-Brain app should reflect the particulars of its application space; the Tiyaro comparison allows you to easily mix and match a select number (3 for the free tier) of images among your own files and web image search results.

1. Click on "Use From Internet" button (highlighted in red below).
2. In the search terms, put the "bird in trees distant" in the search box (highlighted in red below to get a good starting set of representative test images.
3. Select the three (3) pictures that best represent your application, and press OK. -- long distance shots of birds in trees (highlighted in yellow above near the top). Then press the OK button (highlighted in red near the button)
4. Review your selection (highlighted below in yellow) and press Next Button (highlighted in red below) when satisfied and ready to view your comparison.

Now let's see how Detectron2 has to offer.

Evaluate your results

Tiyaro host and serves the models so that you can now concentrate on evaluating the model performance for your application, and ask simple, but strategic questions like "Is it worth upgrading to Detectron2?"

From the results of the comparison above, we can summarize insights from Tiyaro Compare:

1. Detectron2 models have similar performance to our older models on birds.
2. Detectron2 models still find birds as top choices, but also supports a larger set of recognizable objects.

With Tiyaro Comparison, while Detectron2 may be more powerful and capable framework, a Detectron2 upgrade may not improve user experience for Bird-Brain app users. However, with no loss in accuracy, new features for the Bird-Brain app may be able to take advantage of an Detectron2 upgrade. You can further explore these options through Tiyaro Compare by following the same workflow, but varying the test set and/or the model search criteria.

Woot! Profit!

Tiyaro Comparison is merely the first step on your journey for finding the right AI/ML model for your application. However, don't take my word for it. Try it yourself. To take your AI/ML analysis to the next level, convert this comparison into an Tiyaro Experiment to enhance your understanding. Feel free to create an account at Tiyaro and run your own comparisons on what you care about, and follow up with the following blogs:

• Evaluating performance of Object Detection Models using Tiyaro Experiments By Sharvil Mainkar -- deeper analysis of Object Detection with Tiyaro Experiments.
• Evaluate Language Translation Models quicker using Tiyaro Experiments By Sharvil Mainkar -- more NLP analysis with Tiyaro Experiments.

Note that more models and new features are being added every day to Tiyaro website; the screenshots in this blog changes from day to day, but the spirit of AI/ML exploration remains the same. Happy AI/MLing From the Tiyaro team!