General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models

Author(s)
Christoph Molnar, Gunnar König, Julia Herbinger, Timo Freiesleben, Susanne Dandl, Christian A. Scholbeck, Giuseppe Casalicchio, Moritz Grosse-Wentrup, Bernd Bischl
Abstract

An increasing number of model-agnostic interpretation techniques for machine learning (ML) models such as partial dependence plots (PDP), permutation feature importance (PFI) and Shapley values provide insightful model interpretations, but can lead to wrong conclusions if applied incorrectly. We highlight many general pitfalls of ML model interpretation, such as using interpretation techniques in the wrong context, interpreting models that do not generalize well, ignoring feature dependencies, interactions, uncertainty estimates and issues in high-dimensional settings, or making unjustified causal interpretations, and illustrate them with examples. We focus on pitfalls for global methods that describe the average model behavior, but many pitfalls also apply to local methods that explain individual predictions. Our paper addresses ML practitioners by raising awareness of pitfalls and identifying solutions for correct model interpretation, but also addresses ML researchers by discussing open issues for further research.

Organisation(s)
Research Group Neuroinformatics, Research Network Data Science, Vienna Cognitive Science Hub
External organisation(s)
Ludwig-Maximilians-Universität München, Leibniz Institute of Prevention Research and Epidemiology - BIPS, Universität Wien
Pages
39-68
No. of pages
30
DOI
https://doi.org/10.1007/978-3-031-04083-2_4
Publication date
08-2021
Peer reviewed
Yes
Austrian Fields of Science 2012
102035 Data science, 102013 Human-computer interaction, 102019 Machine learning, 101018 Statistics
Keywords
ASJC Scopus subject areas
Theoretical Computer Science, Computer Science(all)
Portal url
https://ucris.univie.ac.at/portal/en/publications/general-pitfalls-of-modelagnostic-interpretation-methods-for-machine-learning-models(a21f17da-c039-4111-b774-3aaf196cf63e).html