Rapid Growth of AI

The AI renaissance has been fueled by 2 trends

1. The cloud computing revolution with its elastic compute and memory resources has allowed larger and larger models to be trained.
2. The advent of big data. Large swathes of data collected both for commercial and social good.

These datasets combined with the training capacity available in the cloud, are producing models that are very close to humans on various activities and in some cases even surpassing human abilities.

Almost in unison with this infrastructure revolution the AI/ML community has forged ahead with tools and frameworks like TensorFlow, PyTorch, Keras, Caffe, AWS SageMaker, Vertex AI from Google, Azure AI from Microsoft to make it easier for folks to train and build models. Startups like Weights and Biases, Fiddler.ai have given tools to the Datascientists and AI/ML experts to spend more of their time on building innovative architectures and cast their focus on a wider and wider range of problems that can be addressed by AI/ML.

AI consumption is ready for take off

As expected in these early days of the AI/ML cycle a lot of effort has been put into the ‘supply’ side of AI/ML - the MLOps to build new models. These tools are the foundation on which ‘consumption’ of AI/ML is going to be possible. As with any new technology and especially one with so much hype, the adoption or the mass consumption of AI/ML is what is going to decide whether this technology remains a niche player; like the promise of Map-Reduce, Big Data and Hadoop which was never able to make it past the few early adopters; or becomes a breakout technology like the mobile revolution that will have a positive impact on almost every person on the planet.

At the time of this writing stackoverflow has just published their developer survey of the year 2022 . To quote that survey – “Hugging Face Transformers surfaces as the most loved library while Tensorflow remains the most wanted”. These are frameworks that make it very easy to build applications that use AI. Even more impressive is this quote from the survey – developers want to build machine learning models but enjoy using pre-trained models more. This is not surprising at all because unlike the AI/ML supply side most consumers of AI/ML are not ML experts. In fact, if AI/ML has to really take off then every developer should be able to use AI/ML without being an expert at producing ML models.

Community is turning the corner on AI/ML consumption

Developers these days are faced with a wide range of choices when it comes to consuming “AI as API”. There are multiple vendors and startups who are offering some form of AI consumption. We at Tiyaro are on a mission to make AI accessible to every developer. We are building the AI consumption platform where ‘AI as API’ is just one piece of the puzzle for us. The choice available to developers today are

Large vendors like

• Google Vision
• AWS Rekognition
• IBM Watson

New entrants like

• Tiyaro
• HuggingFace
• NLPCloud

All these offerings give the users an option to consume AI directly as API. But not all vendors are created equal. Let's go through the 5 most important things to look for to take your AI/ML ideas from aspiration to practice.

5 Things to evaluate when thinking about consuming AI

You understand your application or business needs better than anyone else. You’ve decided that you want to use AI/ML to solve your problem. So why not let your application and business expertise guide you in your choice of AI/ML consumption platform. The 5 things that we enlist here are deeply rooted in the philosophy of letting you use your business expertise to solve your problem. Let that be your guiding principle when choosing your AI/ML partner.

Breadth of Offering

Your business is not static, your requirements change, your user behavior changes. So your AI/ML platform should be able to meet your current and your future needs. For instance, an online retailer may be interested in automated product attribution (a computer vision problem) today but in the near future he may want to flag fake reviews (a NLP problem) of a product. So ideally you are looking for a platform that not only gives you a wide range of choices of models. But you also want a vendor that can address all types of AI/ML problems like Computer Vision, Natural Language Processing (NLP), Time Series Forecasting, Audio, Video etc.

Well having a vendor who can solve multiple types of AI/ML problems is good. But you can't stop there. You also want to see where these models are coming from. There are vendors who only provide open source models, there are others who only support custom built private models. Ideally you would want to choose a vendor that can give you access to models from most if not all the model developers out there. Because ultimately you want to find the best model for your use case.

To summarize, you are looking for a breadth in both the types of problems that can be solved (Vision, NLP, TimeSeries etc) and the source of the models (opensource, custom, 3rd party vendors, or even your own custom models)

Here is a matrix that is useful for comparing the various platforms. By no means is this matrix complete in terms of the ML problems that are articulated on the X-Axis or the Vendors that are listed in the body of the table. The point here is that you can quickly use this matrix to evaluate any vendor of your choice and see how he compares in terms of the breadth of their offering.

Explore

So you have a problem that you think you can solve with ML. How do you go from your business problem to a model API that you can use to solve the problem? You have 2 options.

• Search by model classes. A lot of vendors have cataloged the AI/ML offerings and classified them into various AI/ML model classes like Token Classification, Named Entity Recognition, Fill Mask, Image Classification to name a few. Each of these classes will have 10s if not 100s of models so if you are familiar with how to map your business problems to model classes these offerings will work fairly well for you. If you are already asking yourself, How do you go from the 10s or 100s of models to the one that works for you? Well that is precisely what we will cover in the section called ‘Evaluate Quickly’ but I digress. Coming back to the search itself, this is a huge improvement over the previous generation as it still allows you to narrow down your choice albeit with the caveat that you need to know what you are looking for. This is the approach taken by vendors like HuggingFace, NLPCloud.
• Search in plain english. Wouldn't it be great if you could simply search for a ML model based on the problem you are trying to solve. For instance if you are working on a project to do sentiment analysis on the tweets that feature your brand or product, you could simply search for ‘sentiment’/ ‘twitter’/‘twitter sentiment’ and you will get a listing of all models that are relevant for that problem. This is the approach of Tiyaro. We believe that this is a huge win for both the developers and the AI/ML community. The easier users can find what they are looking for, the better the adoption of the models.

Evaluate Quickly

Having a listing of 10s or 100s of models that can solve your problem is a great place to start. But ultimately you want to find that ‘one’ model and the more easily and quickly you can make that ‘informed’ choice the sooner you can get back to solving your problem.

• Static Comparison - This involves comparing different models on cpu cost, gpu cost, popularity, latency, vendor rating of the models. Just this simple static comparison will help you to very quickly narrow down your choice from 100s of models to a handful. Not all vendors provide you with all this information at your fingertips. Tiyaro has spent countless hours running baseline stats on cpu and gpu for each model that is hosted at Tiyaro. In addition to providing all of these stats we also add a “Tiyaro Score” that is our secret sauce to help you narrow down your choice in minutes.
• Dynamic Comparison - Statistics are all well and good, as they say, every team is a potential Super Bowl winner at the start of the season but you have to play the games to find out the real winner. Similarly the only way to find out the ‘one’ model is to actually simply run it on your test data. Simple as that. How easy does a vendor make it for you to run and compare different models? This should be a matter of minutes and not hours. Most vendors don't get anywhere near to providing Static Stats let alone make it easy to do dynamic comparison. Tiyaro ‘Experiments’ makes comparing models with your own test data a breeze, within minutes you get the results all driven from the Tiyaro UI.

Consume Instantly

Given the large number of open source and custom 3rd party pre-trained models that are available to the developers there is a high likelihood that you may find the model that works best for you at this step (If you are thinking about the what if I need a custom model after all evaluations, we are going to cover that in the next step). And only one thing remains: Start consumption. After you have evaluated and chosen the right model the next step in your evaluation is how quickly you can start consuming it. We are talking about vendors serving models as API here so how complex could it be. Well, you need to put all the various factors together. Many times it is not just accuracy of the model that is important, sometimes good enough is good enough. So the ability to put all static evaluation and dynamic evaluation together, weighing the pros and cons of accuracy vs cost, or accuracy vs latency, transparency of pricing. You want a vendor who gives you absolute clarity on the pricing, the accuracy, the latency, the comparative data from multiple likely models and a generous free tier so you can quickly make ‘informed’ decisions. A vendor that can make these things simple allows you to start consuming models “instantly”. You went from a business problem to exploring models to evaluating them in minutes and now you put it all together with the pricing, latency, cost and within minutes you are ‘consuming’ AI.

Customize

As much as developers love pre-trained models. The true nature of AI/ML applications is such that you may need to customize certain models for your use case. Vendors are trying to solve this problem in multiple ways.

• AutoML - the act of training a new model and tuning the parameters automatically so that a custom model for your data is ready within hours if not minutes. This makes it really convenient for a large number of users to get to their solution without having to build any data science expertise. Vendors like Google, Amazon, Azure are really the front runners here. Newcomers like HuggingFace have also started their own AutoML journey.
• Training as a service - At Tiyaro we are building data science as a service so our users can avail of model customizations with the same ease with which they can consume pre-built models. Our customers can create custom models by simply specifying the task to be solved and providing the custom training and test data. Within a day you will get a model custom trained from Tiyaro. Customization is a very important and deep topic in itself, apart from the act of actually producing the models, customers need help with data engineering and labeling. Vendors like Google, Amazon and Azure do have their own labeling service to help their customers.

Summary

AI/ML is at the cusp of mainstream adoption. Developers today can start consuming AI/ML as a service and within minutes you can go from a business case to a working AI/ML model. With the wide range of offerings in the market following the basic guidelines outlined in this blog should give you a good framework to make a choice that is right for you.