Metamorphic Testing for Quality Assurance of Protein Function Prediction\n Tools

Proteins are the workhorses of life and gaining insight on their functions is\nof paramount importance for applications such as drug design. However, the\nexperimental validation of functions of proteins is highly-resource consuming.\nTherefore, recently, automated protein function prediction (AFP) using machine\nlearning has gained significant interest. Many of these AFP tools are based on\nsupervised learning models trained using existing gold-standard functional\nannotations, which are known to be incomplete. The main challenge associated\nwith conducting systematic testing on AFP software is the lack of a test\noracle, which determines passing or failing of a test case; unfortunately, due\nto the incompleteness of gold-standard data, the exact expected outcomes are\nnot well defined for the AFP task. Thus, AFP tools face the \\emph{oracle\nproblem}. In this work, we use metamorphic testing (MT) to test nine\nstate-of-the-art AFP tools by defining a set of metamorphic relations (MRs)\nthat apply input transformations to protein sequences. According to our\nresults, we observe that several AFP tools fail all the test cases causing\nconcerns over the quality of their predictions.\n