Improve attack model types in inference attacks #2253

abigailgold · 2023-08-22T11:00:39Z

Description

Support additional attack model types (e.g., KNN, LR, etc.) in both membership inference and attribute inference attacks (blackbox and baseline).
Replace use of sklearn MLPClassifier in attribute attacks with pytorch NN model to improve performance.

Fixes #2153

Type of change

Please check all relevant options.

Improvement (non-breaking)
Bug fix (non-breaking)
New feature (non-breaking)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Testing

Added to existing tests the new model params. All tests pass.

Test Configuration:

OS: MacOS 12.6.8
Python version: 3.9

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

…x to improve performance (partially fixes Trusted-AI#2153). Signed-off-by: abigailt <abigailt@il.ibm.com>

Signed-off-by: abigailt <abigailt@il.ibm.com>

…attribute attacks (Trusted-AI#2153) Signed-off-by: abigailt <abigailt@il.ibm.com>

Signed-off-by: abigailt <abigailt@il.ibm.com>

codecov-commenter · 2023-08-22T11:06:19Z

Codecov Report

Merging #2253 (5d0fc3b) into dev_1.16.0 (7f33be1) will decrease coverage by 1.12%.
Report is 3 commits behind head on dev_1.16.0.
The diff coverage is 95.43%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@              Coverage Diff               @@
##           dev_1.16.0    #2253      +/-   ##
==============================================
- Coverage       85.80%   84.68%   -1.12%     
==============================================
  Files             318      318              
  Lines           28365    28716     +351     
  Branches         5173     5260      +87     
==============================================
- Hits            24338    24318      -20     
- Misses           2718     3072     +354     
- Partials         1309     1326      +17

Files Changed	Coverage Δ
...ttacks/inference/membership_inference/black_box.py	`92.34% <93.75%> (+0.43%)`	⬆️
.../attacks/inference/attribute_inference/baseline.py	`92.34% <95.16%> (+3.80%)`	⬆️
...ference/attribute_inference/true_label_baseline.py	`89.51% <95.27%> (+6.33%)`	⬆️
...attacks/inference/attribute_inference/black_box.py	`89.76% <96.06%> (+5.73%)`	⬆️

... and 16 files with indirect coverage changes

Signed-off-by: abigailt <abigailt@il.ibm.com>

beat-buesser

Hi @abigailgold Thank you very much for upgrading the inference attacks! I have a few questions and suggestions added to the review, what do you think?

beat-buesser · 2023-09-04T08:43:18Z

tests/attacks/inference/membership_inference/test_black_box.py

@@ -38,6 +38,7 @@
 num_classes_mnist = 10


+@pytest.mark.skip_framework("scikitlearn")


What is the reason for this skip?

You're right, I will remove it.

beat-buesser · 2023-09-04T08:44:25Z

tests/attacks/inference/attribute_inference/test_true_label_baseline.py

@@ -308,8 +314,8 @@ def test_true_label_baseline_regression(art_warning, get_diabetes_dataset, model
            baseline_inferred_test
        )

-        assert 0.6 <= baseline_train_acc
-        assert 0.6 <= baseline_test_acc
+        assert 0.45 <= baseline_train_acc


Does each parametric model have a different expectation value? Should we use a dictionary of expectation values?

Yes, I think this was the reason for reducing the value. How do you suggest to deal with it?

I was thinking of it could be worth having a dictionary of expectation value like

expected_accuracy = {"nn": 0.65, "rf": 0.45, ...}

and use it like

assert expected_accuracy[model_type] == baseline_train_acc

beat-buesser · 2023-09-04T08:47:00Z

tests/attacks/inference/attribute_inference/test_black_box.py

-        assert np.allclose(inferred_test, x_test_feature.reshape(1, -1), atol=0.4)
+        assert (
+            np.count_nonzero(np.isclose(inferred_train, x_train_feature.reshape(1, -1), atol=0.4))
+            > inferred_train.shape[0] * 0.75


What is the reason for the factor 0.75?

This is just a relatively relaxed way of measuring accuracy for a regression task. Since not 100% of the values are inferred correctly, we check that at least 75% are (within a given distance).

beat-buesser · 2023-09-04T08:58:06Z

art/attacks/inference/attribute_inference/true_label_baseline.py

@@ -102,10 +112,15 @@ def __init__(
        """
        super().__init__(estimator=None, attack_feature=attack_feature)

-        self._values: Optional[list] = None
+        self._values: list = []


Empty lists should not be default values for arguments because if multiple instances of this class are created they will overwrite each others arguments.

Suggested change

self._values: list = []

self._values: Optional[list] = None

This is not an argument but an assignment within the method, I don't think the same issue exists. When using an optional value it caused mypy errors.

You are right, I was looking at the wrong line. Please ignore.

beat-buesser · 2023-09-04T08:58:46Z

art/attacks/inference/attribute_inference/true_label_baseline.py

+        self._attack_model_type: Optional[str] = attack_model_type
+        self.attack_model: Optional[Any] = None
+        self.epochs = 100
+        self.batch_size = 100
+        self.learning_rate = 0.0001


Are these parameters that should be exposed to the user?

They could be. They were not exposed until now in any of the inference attacks but it's possible of course if you think this is desirable.

Do you often adjust the parameters to optimise the attack?

We don't, but perhaps a more savvy user would want to... I can do this with default values and then there is no harm... People can either use it or not.

beat-buesser · 2023-09-04T09:01:09Z

art/attacks/inference/attribute_inference/true_label_baseline.py

+                    _, targets = torch.autograd.Variable(input1), torch.autograd.Variable(targets)
+
+                    optimizer.zero_grad()
+                    outputs = self.attack_model(input1)  # type: ignore


Why is this type ignore needed?

It was there before (in other classes), I just copied the code. Not sure why it's needed (I can try to remove and see if it causes any errors).

Removing it causes a mypy error: "Tensor" not callable

Ok, leave it for now.

beat-buesser · 2023-09-04T09:01:51Z

art/attacks/inference/attribute_inference/true_label_baseline.py

+            attack_train_set = self._get_attack_dataset(feature=x_train, label=y_ready)
+            train_loader = DataLoader(attack_train_set, batch_size=self.batch_size, shuffle=True, num_workers=0)
+
+            self.attack_model = to_cuda(self.attack_model)  # type: ignore


Why is this type ignore needed?

beat-buesser · 2023-09-04T09:01:56Z

art/attacks/inference/attribute_inference/true_label_baseline.py

+                self.attack_model = MembershipInferenceAttackModel(x_train.shape[1], len(self._values))
+                loss_fn = nn.CrossEntropyLoss()
+
+            optimizer = optim.Adam(self.attack_model.parameters(), lr=self.learning_rate)  # type: ignore


Why is this type ignore needed?

beat-buesser · 2023-09-04T09:03:01Z

art/attacks/inference/attribute_inference/baseline.py

@@ -90,10 +100,15 @@ def __init__(
        """
        super().__init__(estimator=None, attack_feature=attack_feature)

-        self._values: Optional[list] = None
+        self._values: list = []


Empty lists should not be default values for arguments because if multiple instances of this class are created they will overwrite each others arguments.

Suggested change

self._values: list = []

self._values: Optional[list] = None

Please ignore.

beat-buesser · 2023-09-04T09:03:42Z

art/attacks/inference/attribute_inference/black_box.py

-        self._values: Optional[list] = None
-        self._attack_model_type = attack_model_type
-        self._attack_model = attack_model
+        self._values: list = []


Empty lists should not be default values for arguments because if multiple instances of this class are created they will overwrite each others arguments.

Suggested change

self._values: list = []

self._values: Optional[list] = None

Please ignore.

Signed-off-by: abigailt <abigailt@il.ibm.com>

abigailgold added 8 commits August 17, 2023 12:57

Use pytorch model instead of sklearn MLP in AttributeInferenceBlackBo…

5db2a14

…x to improve performance (partially fixes Trusted-AI#2153). Signed-off-by: abigailt <abigailt@il.ibm.com>

Additional attack model types for membership black box (Trusted-AI#2153)

9421ada

Signed-off-by: abigailt <abigailt@il.ibm.com>

Additional attack model types for attribute black box (Trusted-AI#2153)

ee88148

Signed-off-by: abigailt <abigailt@il.ibm.com>

Additional attack model types + switch to pytorch for nn in baseline …

33117c9

…attribute attacks (Trusted-AI#2153) Signed-off-by: abigailt <abigailt@il.ibm.com>

Formatting + mypy errors

bf066ef

Signed-off-by: abigailt <abigailt@il.ibm.com>

Formatting + remove unused imports

c23c8d9

Signed-off-by: abigailt <abigailt@il.ibm.com>

Remove unused imports

9338fe0

Signed-off-by: abigailt <abigailt@il.ibm.com>

Fix test

a8654e7

Signed-off-by: abigailt <abigailt@il.ibm.com>

beat-buesser self-requested a review August 23, 2023 13:49

beat-buesser self-assigned this Aug 23, 2023

beat-buesser added the improvement Improve implementation label Aug 23, 2023

beat-buesser added this to the ART 1.16.0 milestone Aug 23, 2023

Remove Any from init method typing

f764aa4

Signed-off-by: abigailt <abigailt@il.ibm.com>

beat-buesser requested changes Sep 4, 2023

View reviewed changes

abigailgold added 3 commits September 10, 2023 11:27

Fix tests

b0ec63f

Signed-off-by: abigailt <abigailt@il.ibm.com>

Make hyperparams of nn attack model training customizable

61b9233

Signed-off-by: abigailt <abigailt@il.ibm.com>

Fix tests + formatting

8446bc8

Signed-off-by: abigailt <abigailt@il.ibm.com>

abigailgold and others added 5 commits September 12, 2023 15:45

Fix assert

a158c48

Signed-off-by: abigailt <abigailt@il.ibm.com>

Merge branch 'dev_1.16.0' into dev_1.16.0_mi_improvements

8df1b79

Fix asserts + renaming tests so no duplications

341238c

Signed-off-by: abigailt <abigailt@il.ibm.com>

Fix asserts

95d1480

Signed-off-by: abigailt <abigailt@il.ibm.com>

Merge branch 'dev_1.16.0' into dev_1.16.0_mi_improvements

5d0fc3b

beat-buesser approved these changes Sep 14, 2023

View reviewed changes

beat-buesser merged commit 6af9e5a into Trusted-AI:dev_1.16.0 Sep 14, 2023
37 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve attack model types in inference attacks #2253

Improve attack model types in inference attacks #2253

abigailgold commented Aug 22, 2023

codecov-commenter commented Aug 22, 2023 •

edited

Loading

beat-buesser left a comment

beat-buesser Sep 4, 2023

abigailgold Sep 6, 2023

beat-buesser Sep 4, 2023

abigailgold Sep 5, 2023

beat-buesser Sep 7, 2023

beat-buesser Sep 4, 2023

abigailgold Sep 5, 2023

beat-buesser Sep 4, 2023

abigailgold Sep 5, 2023

beat-buesser Sep 7, 2023

beat-buesser Sep 4, 2023

abigailgold Sep 5, 2023

beat-buesser Sep 7, 2023

abigailgold Sep 10, 2023

beat-buesser Sep 4, 2023

abigailgold Sep 5, 2023 •

edited

Loading

abigailgold Sep 6, 2023

beat-buesser Sep 7, 2023

beat-buesser Sep 4, 2023

beat-buesser Sep 4, 2023

beat-buesser Sep 4, 2023

beat-buesser Sep 7, 2023

beat-buesser Sep 4, 2023

beat-buesser Sep 7, 2023

		@@ -38,6 +38,7 @@
		num_classes_mnist = 10


		@pytest.mark.skip_framework("scikitlearn")

Improve attack model types in inference attacks #2253

Improve attack model types in inference attacks #2253

Conversation

abigailgold commented Aug 22, 2023

Description

Type of change

Testing

Checklist

codecov-commenter commented Aug 22, 2023 • edited Loading

Codecov Report

beat-buesser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abigailgold Sep 5, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 22, 2023 •

edited

Loading

abigailgold Sep 5, 2023 •

edited

Loading