Implement smoothed Vision Transformer as defense against patch evasion attacks

[GiulioZizzo]

Description

In this PR we include functionality for Certified Patch Robustness via Smoothed Vision Transformers proposed in Salman, Hadi, et al. "Certified patch robustness via smoothed vision transformers." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.

This algorithm achieves very strong certified performance against patch attacks against Vision Transformers by performing image ablations.

Fixes # (issue)

Type of change

Please check all relevant options.

• Improvement (non-breaking)
• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

Tests included in: tests/estimators/certification/test_smooth_vit.py

Test Configuration:

• MacOS/Linux
• Python version: 3.8
• ART version or commit number: 1.14
• PyTorch 1.13

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[codecov-commenter]

Codecov Report

Merging #2171 (fd69156) into dev_1.16.0 (c048a5c) will increase coverage by 0.36%.
The diff coverage is 70.91%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@              Coverage Diff               @@
##           dev_1.16.0    #2171      +/-   ##
==============================================
+ Coverage       85.24%   85.61%   +0.36%     
==============================================
  Files             318      323       +5     
  Lines           28716    29176     +460     
  Branches         5260     5374     +114     
==============================================
+ Hits            24480    24979     +499     
+ Misses           2921     2824      -97     
- Partials         1315     1373      +58     

Files Changed	Coverage Δ
art/estimators/certification/__init__.py	100.00% <ø> (ø)
...s/certification/derandomized_smoothing/__init__.py	100.00% <ø> (ø)
...rs/certification/derandomized_smoothing/pytorch.py	58.73% <54.66%> (-20.58%)	⬇️
...certification/derandomized_smoothing/tensorflow.py	63.30% <55.84%> (-20.04%)	⬇️
...cation/derandomized_smoothing/ablators/__init__.py	75.00% <75.00%> (ø)
...ication/derandomized_smoothing/ablators/pytorch.py	86.89% <86.89%> (ø)
...tion/derandomized_smoothing/ablators/tensorflow.py	81.81% <87.09%> (ø)
...andomized_smoothing/vision_transformers/pytorch.py	93.65% <93.65%> (ø)
...fication/derandomized_smoothing/ablators/ablate.py	100.00% <100.00%> (ø)
...rtification/derandomized_smoothing/derandomized.py	100.00% <100.00%> (ø)

... and 7 files with indirect coverage changes

[beat-buesser]

Hi @GiulioZizzo Thank you very much for your pull request. I have added a few suggestions above, what do you think?

[beat-buesser]

Please add all arguments specifically.

Suggested change

	Create a ArtViT instance
	:param kwargs: keyword arguments required to create the mask embedder.
	Create a ArtViT instance
	:param kwargs: keyword arguments required to create the mask embedder.

[beat-buesser]

Is this like a model-specific or architecture-specific object detection estimator?

[beat-buesser]

Hi @GiulioZizzo Thank you very much for your updates. I have added a few more items I think we should consider.

[beat-buesser]

Suggested change

	def ablate(self, x: np.ndarray, column_pos: int, row_pos: int) -> Union[np.ndarray, "torch.Tensor"]:
	def ablate(self, x: np.ndarray, column_pos: int, row_pos: Optional[int] = None) -> Union[np.ndarray, "torch.Tensor"]:

[GiulioZizzo]

Current signature type-hinting is required by mypy due to https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides

I.e. we can go to a more general argument type in the subclass, but we cannot start with a more general type-hint in the the base class and go a more specific type.

[beat-buesser]

Please add an assert to this test to verify the outcome of the call to fit.

[GiulioZizzo]

assertions for test_equivalence are in the forward_features method above: this test is just to check the intermediate values are equivalent.

[beat-buesser]

I think it should be combined into a single class PyTorchDeRandomizedSmoothing.

[beat-buesser]

Please move the 3 binary files in tests/estimators/certification/smooth_vit/smooth_vit_results to utils/resoruces/models.