Gender Controlled Data Sets for XAI Research

The paper “GECOBench: A Gender-Controlled Text Dataset and Benchmark for Quantifying Biases in Explanations” introduces a novel dataset, GECO, to evaluate biases in AI explanations, specifically focusing on gender. The authors constructed the dataset with sentence pairs that differ only in gendered pronouns or names, enabling a controlled analysis of gender biases in AI-generated text. GECOBench, an accompanying benchmark, assesses different explainable AI (XAI) methods by measuring their ability to detect and mitigate biases within this context.

The study investigates biases in language models, emphasizing that traditional AI systems often produce biased explanations due to their training on unbalanced datasets. By employing GECO, the researchers show how these biases manifest and affect AI explanations. They demonstrate that existing XAI methods, which aim to make AI decisions more transparent, also carry biases, potentially reinforcing stereotypes or presenting skewed explanations.

Moreover, the authors evaluate several fine-tuning and debiasing strategies to reduce bias in AI models. Their findings suggest that certain fine-tuning approaches can significantly decrease gender bias in explanations without compromising the model’s overall performance. This highlights the importance of combining XAI methods with robust debiasing techniques to create fairer and more trustworthy AI systems.

The paper also provides a comprehensive framework for evaluating bias in XAI methods by using GECOBench. This benchmark allows for a standardized comparison across different methods, providing insights into their strengths and limitations concerning gender bias. It helps identify which methods are more susceptible to biases and under what conditions, promoting the development of better XAI techniques.

Overall, the paper underscores the critical need for datasets like GECO and benchmarks like GECOBench in understanding and mitigating biases in AI explanations. It calls for further research and development in the field of fair and explainable AI, providing resources and guidelines for future studies to build upon. The dataset and code are made publicly available, fostering community efforts toward more equitable AI systems. The paper’s findings have broad implications for the design of AI systems, particularly those deployed in sensitive or high-stakes environments.