Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove incorrectly duplicated queries in SignOPT attack #2129

Merged

Conversation

Lodour
Copy link
Contributor

@Lodour Lodour commented May 2, 2023

Description

The current implementation of SignOPT sends duplicated queries when checking targeted/untargeted modes. This is because the model query happens before the mode checking. For example:

if (not self._is_label(x_0 + lbd * theta, target) and self.targeted) or (
self._is_label(x_0 + lbd * theta, y_0) and not self.targeted
):

In untargeted mode, self.targeted = False would cost an unnecessary self._is_label call.

This PR moves the mode checks ahead of the self._is_label as a valid short-circuiting.

It is hard to determine the exact number of calls wasted due to this problem, but this PR should reduce half of the running time for this specific part.

Type of change

Please check all relevant options.

  • Improvement (non-breaking)
  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

  • Test A
  • Test B

Test Configuration:

  • OS
  • Python version
  • ART version or commit number
  • TensorFlow / Keras / PyTorch / MXNet version

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

dependabot bot and others added 3 commits April 21, 2023 19:48
Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.4.0 to 9.5.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](python-pillow/Pillow@9.4.0...9.5.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
…low-9.5.0

Bump pillow from 9.4.0 to 9.5.0
Signed-off-by: Yue Gao <11145354+Lodour@users.noreply.github.com>
@Lodour Lodour force-pushed the fix_signopt_duplicated_queries branch from 3ac8b92 to 4a9ef03 Compare May 2, 2023 20:34
@beat-buesser beat-buesser changed the base branch from main to dev_1.15.0 May 3, 2023 20:12
@beat-buesser
Copy link
Collaborator

Hi @Lodour Thank you very much for your pull request! We'll review it as soon as possible.

@beat-buesser beat-buesser self-requested a review May 3, 2023 20:13
@beat-buesser beat-buesser self-assigned this May 3, 2023
@beat-buesser beat-buesser added the improvement Improve implementation label May 3, 2023
@beat-buesser beat-buesser added this to the ART 1.15.0 milestone May 3, 2023
@codecov-commenter
Copy link

codecov-commenter commented May 3, 2023

Codecov Report

Merging #2129 (4a9ef03) into dev_1.15.0 (b13f802) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Impacted file tree graph

@@              Coverage Diff               @@
##           dev_1.15.0    #2129      +/-   ##
==============================================
- Coverage       85.63%   85.61%   -0.03%     
==============================================
  Files             297      297              
  Lines           26521    26521              
  Branches         4864     4864              
==============================================
- Hits            22712    22706       -6     
- Misses           2571     2576       +5     
- Partials         1238     1239       +1     
Impacted Files Coverage Δ
art/attacks/evasion/sign_opt.py 86.44% <100.00%> (-0.37%) ⬇️

... and 3 files with indirect coverage changes

Copy link
Collaborator

@beat-buesser beat-buesser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Lodour Thank you very much for your pull request! The improvements look good to me.

@beat-buesser beat-buesser merged commit 0535c79 into Trusted-AI:dev_1.15.0 May 9, 2023
@Lodour Lodour deleted the fix_signopt_duplicated_queries branch May 10, 2023 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improve implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants