Instance-wise Causal Feature Selection for Model Interpretation

Dataset	Sample 1	Sample 2
MNIST
CIFAR

(Copper color in above images indicates the selected superpixels/patches for explaining the prediction of a black-box model. Note: the black-box model predicted the correct class in all the above images)

Abstract

We formulate a causal extension to the recently intro-duced paradigm of instance-wise feature selection to ex-plain black-box visual classifiers. Our method selects asubset of input features that has the greatest causal effecton the model’s output. We quantify the causal influence ofa subset of features by the Relative Entropy Distance mea-sure. Under certain assumptions this is equivalent to theconditional mutual information between the selected subsetand the output variable. The resulting causal selections aresparser and cover salient objects in the scene. We show theefficacy of our approach on multiple vision datasets by mea-suring the post-hoc accuracy and Average Causal Effect ofselected features on the model’s output.

• Authors: Pranoy Panda, Sai Srinivas Kancheti, Vineeth N Balasubramanian
• Affiliation of authors: CSE department, IIT Hyderabad
• Workshop: Causality in Vision, CVPR 2021

Dependencies:

1. Python 3.5+.
2. PyTorch
3. NumPy
4. Matplolib
5. skimage
6. captum

(you could use pip to install all the dependencies)

Or you may run the following commands in shell to install the required packages:

git clone https://github.com/pranoy-panda/Causal-Feature-Subset-Selection
cd Causal-Feature-Subset-Selection
sudo pip install -r requirements.txt

Then in order to run our method on MNIST dataset use the following command:

python main.py

(In order to get results on FMNIST or to change the hyper-parameters, go to the config.py script)

Colab notebooks(for hassle free reproduction of results)

1. MNIST and FMNIST dataset
2. CIFAR dataset