Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: adversarial attack to input manipulation #110

Merged
merged 2 commits into from
Sep 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ auto-migrated: 0
document: OWASP Machine Learning Security Top Ten 2023
year: 2023
order: 1
title: ML01:2023 Adversarial Attack
title: ML01:2023 Input Manipulation Attack
lang: en
tags:
[
Expand All @@ -24,56 +24,57 @@ technical: 5

## Description

Adversarial attacks are a type of attack in which an attacker deliberately
alters input data to mislead the model.
Input Manipulation Attacks is an umbrella term, which include Adversarial
Attacks, a type of attack in which an attacker deliberately alters input data to
mislead the model.

## How to Prevent

**Adversarial training:** One approach to defending against adversarial attacks
is to train the model on adversarial examples. This can help the model become
more robust to attacks and reduce its susceptibility to being misled.
**Adversarial training:** One approach to defending against input manipulation
attack is to train the model on adversarial examples. This can help the model
become more robust to attacks and reduce its susceptibility to being misled.

**Robust models:** Another approach is to use models that are designed to be
robust against adversarial attacks, such as adversarial training or models that
robust against manipulative attacks, such as adversarial training or models that
incorporate defense mechanisms.

**Input validation:** Input validation is another important defense mechanism
that can be used to detect and prevent adversarial attacks. This involves
that can be used to detect and prevent input manipulation attacks. This involves
checking the input data for anomalies, such as unexpected values or patterns,
and rejecting inputs that are likely to be malicious.

## Risk Factors

| Threat Agents/Attack Vectors | Security Weakness | Impact |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------------------------------: |
| Exploitability: 5 (Easy) <br><br> _ML Application Specific: 4_ <br> _ML Operations Specific: 3_ | Detectability: 3 (Moderate) <br><br> _The adversarial image may not be noticeable to the naked eye, making it difficult to detect the attack._ | Technical: 5 (Difficult) <br><br> _The attack requires technical knowledge of deep learning and image processing techniques._ |
| Threat Agent: Attacker with knowledge of deep learning and image processing techniques. <br><br> Attack Vector: Deliberately crafted adversarial image that is similar to a legitimate image. | Vulnerability in the deep learning model's ability to classify images accurately. | Misclassification of the image, leading to security bypass or harm to the system. |
| Exploitability: 5 (Easy) <br><br> _ML Application Specific: 4_ <br> _ML Operations Specific: 3_ | Detectability: 3 (Moderate) <br><br> _The manipulated image may not be noticeable to the naked eye, making it difficult to detect the attack._ | Technical: 5 (Difficult) <br><br> _The attack requires technical knowledge of deep learning and image processing techniques._ |
| Threat Agent: Attacker with knowledge of deep learning and image processing techniques. <br><br> Attack Vector: Deliberately crafted manipulated image that is similar to a legitimate image. | Vulnerability in the deep learning model's ability to classify images accurately. | Misclassification of the image, leading to security bypass or harm to the system. |

It is important to note that this chart is only a sample based on
[the scenario below](#scenario1) only. The actual risk assessment will depend on
the specific circumstances of each machine learning system.

## Example Attack Scenarios

### Scenario \#1: Image classification {#scenario1}
### Scenario \#1: Input manipulation of Image Classification systems {#scenario1}

A deep learning model is trained to classify images into different categories,
such as dogs and cats. An attacker creates an adversarial image that is very
such as dogs and cats. An attacker manipulates the original image that is very
similar to a legitimate image of a cat, but with small, carefully crafted
perturbations that cause the model to misclassify it as a dog. When the model is
deployed in a real-world setting, the attacker can use the adversarial image to
deployed in a real-world setting, the attacker can use the manipulated image to
bypass security measures or cause harm to the system.

### Scenario \#2: Network intrusion detection
### Scenario \#2: Manipulation of network traffic to evade intrusion detection systems {#scenario2}

A deep learning model is trained to detect intrusions in a network. An attacker
creates adversarial network traffic by carefully crafting packets in such a way
that they will evade the model\'s intrusion detection system. The attacker can
manipulate the features of the network traffic, such as the source IP address,
destination IP address, or payload, in such a way that they are not detected by
the intrusion detection system. For example, the attacker may hide their source
IP address behind a proxy server or encrypt the payload of their network
traffic. This type of attack can have serious consequences, as it can lead to
data theft, system compromise, or other forms of damage.
manipulates network traffic by carefully crafting packets in such a way that
they will evade the model\'s intrusion detection system. The attacker can alter
the features of the network traffic, such as the source IP address, destination
IP address, or payload, in such a way that they are not detected by the
intrusion detection system. For example, the attacker may hide their source IP
address behind a proxy server or encrypt the payload of their network traffic.
This type of attack can have serious consequences, as it can lead to data theft,
system compromise, or other forms of damage.

## References
2 changes: 1 addition & 1 deletion index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ in our

## Top 10 Machine Learning Security Risks

- [**ML01:2023 Adversarial Attack**](/docs/ML01_2023-Adversarial_Attack.md)
- [**ML01:2023 Input Manipulation Attack**](/docs/ML01_2023-Input_Manipulation_Attack.md)
- [**ML02:2023 Data Poisoning Attack**](/docs/ML02_2023-Data_Poisoning_Attack.md)
- [**ML03:2023 Model Inversion Attack**](/docs/ML03_2023-Model_Inversion_Attack.md)
- [**ML04:2023 Membership Inference Attack**](/docs/ML04_2023-Membership_Inference_Attack.md)
Expand Down