Evaluate Language Translation Models quicker using Tiyaro Experiment

Language Translation is the task of converting one natural language to another without changing the meaning and the intent in the original language. Translation has a variety of use cases from the consumer, and retail to defense. Most common is analyzing large text documents from one language to another. There are numerous translation SaaS vendors and open-sourced machine learning models available, but selecting one model for any given use case is a time-consuming task. Finding and using such models / SaaS APIs take considerable effort. You can use Tiyaro to reduce the effort to merely minutes. Search for translation models in Tiyaro Explore Search results displaying various models.

Accelerating the rate of model evaluation using: Tiyaro Experiments

Tiyaro Experiments allows us a quicker way to compare different pre-trained machine learning models along with the SaaS Vendor APIs helping us accelerate the evaluation process. One such translation experiment is Japanese text translation which we conduct to translate different blocks of text (articles, short stories, news headlines, etc ) from Japanese to English. For that we utilize the following State of the Art pre-trained machine learning models / SaaS API :

1. Helsinki JP-EN model
2. IBM Watson JP-EN

Once we have completed searching and using demos for different models we can head on over to create experiments. There are various ways to create experiments, but we are simply going over to the experiments tab. Where we start a new experiment. After that, we can select our experiment class type, which in our case would be Translation. Once completed we can select the models to train the experiments on, the model selection provides us with various filters. After selecting the models / SaaS API vendor solution you can upload your custom dataset, there are various formats to upload the translation dataset on Tiyaro for the experiment. Easiest is zipping the CSV file containing one column as the input. After running the experiment we can see the results tab show up, here you would also be able to see the latency of the models as well as the results of the experiment, for our use case we are able to see the table containing our input and the respective model predictions. We can also download the result in a zip file. The result below is shown in the table, the first column containing the actual raw input and the subsequent columns being the translation result.

Comparing one such translated news headline:

Original News Headline in Japanese:

FIFA 前会長が元副会長へ不正支払い 2 人に無罪 スイスの裁判所

Translation by IBM Watson JP-EN :

Former FIFA president acquitted ex-vice president on fraud payments

Translation by Helsinki JP-EN model :

Pre-President FIFFA had an illegal payment to former vice presidents, two innocents, and a Swiss court.

As seen from the above results both the models / SaaS API service gives us different translations. The evaluation of such translation experiments requires human evaluation for the accuracy of the result. You can share the experiment with your coworkers and on social media. Also, you can make a copy of the given experiment to enhance or modify the particular experiment.

Wish to create one? Head on over to Tiyaro !