Analyzing and assessing explainable AI models for smart agriculture environments

Abstract We analyze a case study in the field of smart agriculture exploiting Explainable AI (XAI) approach, a field of study that aims to provide interpretations and explanations to the behaviour of AI systems. The study regards a multiclass classification problem on the Crop Recommendation dataset. The original task is the prediction of the most adequate crop, according to seven features. In addition to the predictions, two of the most well-known XAI approaches have been used in order to obtain explanations and interpretations of the behaviour of the models: SHAP ( SH apley A dditive Ex P lanations), and LIME (Local Interpretable Model-Agnostic Explanations). Both packages provide easy-to-understand visualizations that allow common users to understand explanations of single predictions even without going into the mathematical details of the algorithms. Within the scientific community criticisms have been raised against these approaches, and recently some papers brought to light some weaknesses. However, the two algorithms are among the most popular in XAI and are still considered points of reference for this field of study.