Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing one of the captum tests. #9549

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Conversation

drivanov
Copy link
Contributor

@drivanov drivanov commented Jul 29, 2024

The current PR provides the fix for the following 9 failing tests.

FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.node-MaskType.object-MaskType.attributes-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.node-MaskType.object-None-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.node-None-MaskType.attributes-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.edge-MaskType.object-MaskType.attributes-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.edge-MaskType.object-None-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.edge-None-MaskType.attributes-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.graph-MaskType.object-MaskType.attributes-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.graph-MaskType.object-None-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1]' is invalid for input of size 3
FAILED test/explain/algorithm/test_captum_explainer.py::test_captum_explainer_multiclass_classification[index1-ModelTaskLevel.graph-None-MaskType.attributes-ShapleyValueSampling] - RuntimeError: shape '[-1, 2, 1, 1]' is invalid for input of size 3

NOTE: This is a resubmission of PR#9513, which had some issues with my captum branch.

Additional Details Regarding the Fix...
If method == 'ShapleyValueSampling' and type(index) != int we do have a situation when tensor eval_diff which is calculated here has a size of 3. It is impossible to reshape it to shape = [-1, 2, 1, 1] which is calculated here.
My fix eliminates this specific combination of input parameters from being considered in the test_captum_explainer_multiclass_classification test.

@rusty1s
Copy link
Member

rusty1s commented Jul 30, 2024

By my comment in #9513, I meant why does our CI is still green even without this fix? I am a bit hesitant to merge this without further understanding.

@drivanov
Copy link
Contributor Author

drivanov commented Jul 30, 2024

Our CI is green only because the 187 tests from the unit tests require the captum package to be installed and it is not included in our PYG container. This problem shows up when we pip install captum and run unit tests

Copy link

codecov bot commented Aug 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.38%. Comparing base (3f4f1a0) to head (69edf78).
Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9549      +/-   ##
==========================================
+ Coverage   86.96%   87.38%   +0.41%     
==========================================
  Files         468      482      +14     
  Lines       30918    31396     +478     
==========================================
+ Hits        26887    27434     +547     
+ Misses       4031     3962      -69     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@puririshi98
Copy link
Contributor

@akihironitta @rusty1s anything else needed to merge?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants