Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger Placement Bugs for Image Poisoning Perturbations #2141

Closed
f4str opened this issue May 8, 2023 · 0 comments · Fixed by #2143
Closed

Trigger Placement Bugs for Image Poisoning Perturbations #2141

f4str opened this issue May 8, 2023 · 0 comments · Fixed by #2143
Assignees
Labels
bug Something isn't working
Milestone

Comments

@f4str
Copy link
Collaborator

f4str commented May 8, 2023

Describe the bug
The insert_image function in art.attacks.poisoning.perturbations has two bugs involving the placement of the trigger image.

  1. Due to Pillow swapping the height and width, a non-square trigger is the wrong dimensions when placed onto the image. This was partially addressed by PR Fix Incorrect Height and Width for Image Perturbations #2046 which fixed the incorrect dimensions for the input image, but did not make the same change for the trigger. The fix is is to apply the same where the height and width are swapped.

  2. When the trigger height or width is the same size as the image height or width, the insert_image function will error due to the np.random.randint function not accepting 0 as the upper bound. This numpy function will sample from the lower bound inclusively and upper bound exclusively. This is an off-by-one error and can be easily fixed by adding one to the difference between the image height/width and trigger height/width.

To Reproduce

  1. The following code snippet will reproduce the first bug:

    import numpy as np
    import matplotlib.pyplot as plt
    from art.attacks.poisoning.perturbations import insert_image
    
    image = np.ones((32, 32, 3))
    image_poisoned = insert_image(
        image,
        backdoor_path='../utils/data/backdoors/htbd.png',
        size=(5, 10),
        random=False,
        x_shift=0,
        y_shift=0,
        mode='RGB'
    )
    plt.imshow(image_poisoned)

    From this, the trigger will be inserted as a 10 x 5 rather than the specified 5 x 10 due to Pillow swapping the height and width order.

  2. The following code snippet will reproduce the second bug:

    import numpy as np
    import matplotlib.pyplot as plt
    from art.attacks.poisoning.perturbations import insert_image
    
    image = np.ones((32, 32, 3))
    image_poisoned = insert_image(
        image,
        backdoor_path='../utils/data/backdoors/htbd.png',
        size=(32, 32),
        random=True,
        mode='RGB'
    )
    plt.imshow(image_poisoned)

    This will raise an exception due to the np.random.randint function not accepting a value of 0 as the trigger and image size are the same.

Expected behavior
The insert_image function should produce the correct behavior and not error when valid inputs are provided.

Screenshots
N/A

System information (please complete the following information):

  • OS
  • Python version
  • ART version or commit number
  • TensorFlow / Keras / PyTorch / MXNet version
@beat-buesser beat-buesser added the bug Something isn't working label May 8, 2023
@beat-buesser beat-buesser linked a pull request May 9, 2023 that will close this issue
13 tasks
@beat-buesser beat-buesser added this to the ART 1.15.0 milestone May 10, 2023
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants