Revisiting the Rashomon Set Argument
About eighteen months ago, I posted about this paper discussing the Accuracy-Interpretability Trade-Off (AITO) or Performance-Explainability Trade-Off (PET). This paper revisited the sometimes overlooked debate over the validity of this trade-off. That is to say, is it even necessary to accept that such a trade-off or dichotomy exists? Are we really forced to choose between an accurate model and an interpretable one, or must we always compromise our target metrics? You can read my previous blog here
Evaluating the Influences of Explanation Style on Human-AI Reliance
The reccent paper “Evaluating the Influences of Explanation Style on Human-AI Reliance” investigates how different types of explanations affect human reliance on AI systems. The research focused on three explanation styles: feature-based, example-based, and a combined approach, with each style hypothesized to influence human-AI reliance in unique ways. A two-part experiment with 274 participants explored how these explanation styles impact reliance and interpretability in a human-AI collaboration task, specifically using a bird identification task. The study sought to address mixed evidence from previous literature on whether certain explanation styles reduce over-reliance on AI or improve human decision-making accuracy.
Explainable AI for Improved Heart Disease Prediction
The paper “Optimized Ensemble Learning Approach with Explainable AI for Improved Heart Disease Prediction” focuses on explaining machine learning models in healthcare, similar to my original work in “Ada-WHIPS: explaining AdaBoost classification with applications in the health sciences”. The newer paper combines a novel Bayesian method to optimally tune the hyper-paremeters of ensemble models such as AdaBoost, XGBoost and Random Forest and then applies the now well established SHAP method to assign Shapley values to each feature. The authors use their method to analyse three heart disease prediction datasets, included the well-known Cleveland set used as a benchmark in many ML research papers.
Gender Controlled Data Sets for XAI Research
The paper “GECOBench: A Gender-Controlled Text Dataset and Benchmark for Quantifying Biases in Explanations” introduces a novel dataset, GECO, to evaluate biases in AI explanations, specifically focusing on gender. The authors constructed the dataset with sentence pairs that differ only in gendered pronouns or names, enabling a controlled analysis of gender biases in AI-generated text. GECOBench, an accompanying benchmark, assesses different explainable AI (XAI) methods by measuring their ability to detect and mitigate biases within this context.
Revisiting the Performance-Explainability Trade-Off
I was very excited to read and review the paper Revisiting the Performance-Explainability Trade-Off in Explainable Artificial Intelligence (XAI) last month. I wrote an extensive section on this topic for my Ph.D. thesis (although I coined the name Accuracy-Interpretability Trade-Off or AITO). I have always felt that the subject is too rarely discussed, and never in enough depth and scientific rigour. This Performance-Explanability Trade-off (PET) in the notion that improving model performance (by this, they must mean accuracy or related measures such as true positive rate or AUC/ROC) comes at the cost of explainability.