Specialized Session 11: Advances in Explainable AI

This paper introduces a modification of the Classification and Regression Trees (CART) algorithm, that explicitly accounts for uncertainty, to address fairness concerns in decision-making. While CART algorithms are valued for their transparency and interpretability, their greedy optimization approach can lead to discriminatory predictions. Our method incorporates fairness directly into the learning process by using disparate impact as a fairness metric and introducing an adaptive penalty mechanism based on confidence intervals. When these intervals indicate significant discrimination, the algorithm applies a graduated penalty that reduces the Information Gain for discriminatory splits, naturally guiding the tree toward more equitable decisions. The proposed approach maintains CART’s strong predictive performance and interpretability while promoting fairer outcomes.

Machine learning (ML) has revolutionized cybersecurity, offering powerful tools for malware detection. However, the lack of interpretability in ML models hinders trust and practical adoption, especially in critical domains like cybersecurity. This study addresses the challenge by integrating Explainable Artificial Intelligence (XAI) techniques to enhance the transparency of ML-based malware detection systems. We analyze malware behavior through library calls on Windows systems, employing Random Forests to identify key predictors and Markov Random Fields (MRFs) to model relationships among features. The combination of feature selection and graphical models reveals critical libraries, such as \texttt{msvcrt.dll} and \texttt{mscoree.dll}, that are strongly indicative of malicious behavior. While Random Forests provide insights into variable importance, MRFs enable a more deep understanding of interdependencies among features, offering complementary perspectives for model interpretation. By providing both predictive accuracy and interpretable insights, this methodology favours trust in automated detection systems and equips cybersecurity practitioners with actionable understanding of malware behavior.

Tree-based ensemble methods, like random forests, remain among the most efficient supervised learning algorithms. Unfortunately, their good predictive performances come at the price of interpretability, since the decision process results from the vote of several (typically thousands) trees. To avoid this issue, variable importances have been designed in order to determine which input variables are associated to the output variable. Two variable importances exist in random forests: the Mean Decrease Impurity (MDI) and the Mean Decrease Accuracy (MDA). We review here the results presented in Bénard et al. (2022) regarding MDA. In particular, we highlight that MDA does not converge to a quantity that is relevant to perform variable selection, and therefore should not be used for this aim.