Metamorphic Testing for Machine Learning Models with Search Relevancy Example


Accuracy of a Model can be improved in several levels and multiple variables, boundaries, and guidelines. With the well-known problem statement and solution, it is difficult to evaluate for all the given cases the model would be predicting expected outcomes. Machine Learning Models are solving the problems for which results are unknown, most of the times. This arises a problem of Test Oracle. Recent surveys and work have shown that this difficulty can be reduced by some of the black box testing techniques such as Metamorphic Testing, Fuzzing, Dual Coding et.,

Even though the output of a Model is not known, we can make a few predictions based on Metamorphic relations. A metamorphic relation refers to the relationship between the software input change and output change during multiple program executions. Many metamorphic relations are created based on the transformation from the training data set or test data set. We further classify them into Coarse-grained Data transformation and Fine-grained data transformation.

We will discuss different transformations. Will go through the example of a Search relevancy problem and will analyze the application of Metamorphic testing to verify the Machine model built.


Vinayaka Mayura has been working as a Quality Analyst for 8+ Years. Worked with companies like Thoughtworks, Rakuten, Flipkart. Has a specialization in testing unconventional software applications. Contributed a few bit to the community in open source projects and given talks at a few conferences.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google