Faithful and Customizable Explanations of Black Box Models

The authors of “Faithful and Customizable Explanations of Black Box Models” (MUSE) share a common goal with my own research: addressing the challenge of making machine learning models interpretable. Both emphasize the importance of transparency in decision-making, particularly in scenarios where human trust and understanding are critical, such as healthcare, judicial decisions, and financial assessments. Both they and I see decision rule structures as the ideal explanation format to explain model behaviour.

MUSE uses a two-level decision set framework, which combines subspace descriptors and decision logic to generate explanations for different regions of the feature space. This is useful for zooming in to specific features and observation subsets of interest. Just like my own research, this is a highly user centric approach, emphasising a Human-in-the-Loop process of expert review of model decisions. My method differs in that it facilitates an individual detail review, potentially allowing the expert user to respond to individuals seeking some kind of review or redress over an automated decision. In essence, this is a response to the “computer says no” problem. The explanations are tailored to specific needs or contexts.

This focus on end-user interaction reflects a broader effort in both frameworks to build trust in machine learning outputs by providing meaningful insights. Despite these similarities, the research ideas diverge in significant ways. MUSE has a broader scope, offering global explanations as well as targeted insights into specific subspaces of the model’s behaviour. It is designed to be model-agnostic, meaning it can work with any type of predictive system. My research has a specific focus on Decision Tree ensembles (Random Forest and Boosting methods), explaining how such a classifier reached a decision for a particular data point, emphasising precision and counterfactual reasoning.

The methodologies also differ. MUSE employs optimization techniques to create compact and interpretable decision sets that balance fidelity, unambiguity, and interpretability. My approach, in contrast, extracts decision paths from random forests using frequent pattern mining, constructing rules that highlight the most influential attributes in a model’s classification. These distinct methods reflect their differing objectives: MUSE aims to provide a comprehensive view of a model’s behaviour, while I seek to zero in on individual classifications with a high degree of local accuracy.

Together, these research approaches represent two sides of the same coin: one offering a high-level overview and the other delivering precise, localised explanations. There is a lot of scope for combining the two methods in a collaborative framework for holistic explanations.