Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add sample for revoking dataset access #778

Merged
merged 53 commits into from
Dec 6, 2021
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
0abac4c
revoke dataset access setup
Jul 1, 2021
d699bec
basic template for sample
Jul 2, 2021
98b245d
sample + test
Jul 7, 2021
f1adee7
revoke dataset access sample
Jul 16, 2021
e729bbd
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jul 16, 2021
b80c4bf
docs: add sample for revoking dataset access - update year and string…
Jul 23, 2021
2960b48
Merge branch 'bigquery_remove_view_access' of github.com:googleapis/p…
Jul 23, 2021
02bd2b4
docs: add sample for revoking dataset access - move to snippets and c…
Jul 23, 2021
fa0e48c
moving update_dataset to /snippets and adjusting imports on both revo…
Jul 27, 2021
98bb4a2
Update samples/snippets/revoke_dataset_access.py
loferris Jul 27, 2021
8c4bc7a
Update samples/snippets/revoke_dataset_access.py
loferris Jul 27, 2021
5197d0f
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jul 27, 2021
cfb3805
updated test
Jul 29, 2021
4be9760
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jul 29, 2021
0a57eba
Merge branch 'main' of github.com:googleapis/python-bigquery into big…
Aug 27, 2021
d4672f2
Merge branch 'main' into bigquery_remove_view_access
loferris Aug 30, 2021
b9e35aa
change after running test
Aug 30, 2021
b076972
Merge branch 'bigquery_remove_view_access' of github.com:googleapis/p…
Aug 30, 2021
4ec3512
Merge branch 'main' into bigquery_remove_view_access
tswast Sep 10, 2021
15d84b1
Merge branch 'main' into bigquery_remove_view_access
loferris Sep 11, 2021
2209d8e
resolving linting failure, rewriting test
Sep 11, 2021
0da2301
removed relative import errors
Sep 15, 2021
77066a5
remove relative mport from update_dataset_access
Sep 15, 2021
32d9b71
adding fixture to conftest.py
Sep 15, 2021
b7e91eb
Merge branch 'main' into bigquery_remove_view_access
plamut Sep 17, 2021
318242d
Merge branch 'main' into bigquery_remove_view_access
tswast Oct 6, 2021
5fe08d0
Merge branch 'main' into bigquery_remove_view_access
parthea Oct 6, 2021
1c4469b
Merge branch 'main' into bigquery_remove_view_access
meredithslota Oct 13, 2021
fe66df2
Merge branch 'main' into bigquery_remove_view_access
loferris Nov 18, 2021
fbcc09f
updated sample
Nov 18, 2021
4234450
updating sample to match new update_access sample
Nov 19, 2021
2b82c45
Merge branch 'main' into bigquery_remove_view_access
loferris Nov 19, 2021
2ea0f4e
fixing region tags
Nov 19, 2021
6bbdc33
consolidated tests into one file for both methods
Nov 19, 2021
a493adc
Merge branch 'bigquery_remove_view_access' of github.com:googleapis/p…
Nov 19, 2021
575260a
updating test to full_dataset format
Nov 19, 2021
27d8170
updated revoke sample
Nov 20, 2021
adc1c76
updating test
Nov 20, 2021
62ec25e
refactored sample
Nov 22, 2021
072b785
Update samples/snippets/conftest.py
tswast Nov 23, 2021
dade3b8
Update samples/snippets/revoke_dataset_access.py
loferris Nov 23, 2021
c74022e
Update samples/snippets/update_dataset_access.py
loferris Nov 23, 2021
9da6de6
Update samples/snippets/revoke_dataset_access.py
loferris Nov 23, 2021
ede9158
Update samples/snippets/revoke_dataset_access.py
loferris Nov 23, 2021
05a4f19
refactoring entry
Nov 24, 2021
6c04f28
added comment for entry access
Nov 24, 2021
d030e21
Merge branch 'main' into bigquery_remove_view_access
tswast Nov 29, 2021
023eb96
Update samples/snippets/README.rst
loferris Nov 29, 2021
6d9f274
Update samples/snippets/dataset_access_test.py
loferris Nov 29, 2021
3fd6856
Update samples/snippets/dataset_access_test.py
loferris Nov 29, 2021
6ce391d
added develper TODO in sample
Nov 29, 2021
04497a1
add comments to samples
tswast Dec 6, 2021
2e98eca
Merge branch 'main' into bigquery_remove_view_access
tswast Dec 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 6 additions & 23 deletions samples/snippets/README.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

.. This file is automatically generated. Do not edit this file directly.

Google BigQuery Python Samples
Expand All @@ -16,11 +15,14 @@ This directory contains samples for Google BigQuery. `Google BigQuery`_ is Googl
.. _Google BigQuery: https://cloud.google.com/bigquery/docs


To run the sample, you need to have `BigQuery Admin` role.
loferris marked this conversation as resolved.
Show resolved Hide resolved



Setup
-------------------------------------------------------------------------------



Authentication
++++++++++++++

Expand All @@ -31,9 +33,6 @@ credentials for applications.
.. _Authentication Getting Started Guide:
https://cloud.google.com/docs/authentication/getting-started




Install Dependencies
++++++++++++++++++++

Expand All @@ -48,7 +47,7 @@ Install Dependencies
.. _Python Development Environment Setup Guide:
https://cloud.google.com/python/setup

#. Create a virtualenv. Samples are compatible with Python 3.6+.
#. Create a virtualenv. Samples are compatible with Python 2.7 and 3.4+.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parthea Looks like this is a reversion by owlbot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first glance, this change wasn't made by owlbot. See cfb3805#diff-80e5f0b5c748e24702257bf661911a6c240d81c82a5029056391211591a4a5c9

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loferris Please revert the changes to this file.


.. code-block:: bash

Expand All @@ -64,15 +63,9 @@ Install Dependencies
.. _pip: https://pip.pypa.io/
.. _virtualenv: https://virtualenv.pypa.io/






Samples
-------------------------------------------------------------------------------


Quickstart
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Expand All @@ -89,8 +82,6 @@ To run this sample:
$ python quickstart.py




Simple Application
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Expand All @@ -107,8 +98,6 @@ To run this sample:
$ python simple_app.py




User Credentials
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Expand All @@ -124,7 +113,6 @@ To run this sample:

$ python user_credentials.py


usage: user_credentials.py [-h] [--launch-browser] project

Command-line application to run a query using user credentials.
Expand All @@ -143,10 +131,6 @@ To run this sample:







The client library
-------------------------------------------------------------------------------

Expand All @@ -162,5 +146,4 @@ to `browse the source`_ and `report issues`_.
https://github.com/GoogleCloudPlatform/google-cloud-python/issues



.. _Google Cloud SDK: https://cloud.google.com/sdk/
.. _Google Cloud SDK: https://cloud.google.com/sdk/
51 changes: 51 additions & 0 deletions samples/snippets/revoke_dataset_access.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


def revoke_dataset_access(dataset_id, entity_id):
tswast marked this conversation as resolved.
Show resolved Hide resolved
loferris marked this conversation as resolved.
Show resolved Hide resolved
original_dataset_id = dataset_id
original_entity_id = entity_id
# [START bigquery_revoke_dataset_access]
# TODO(developer): Set the dataset ID to the dataset where you are revoking access.
dataset_id = "your_dataset_id"

# TODO(developer): Set the entity ID of the email or group from whom you are revoking access.
entity_id = "user-or-group-to-remove@example.com"

# [END bigquery_revoke_dataset_access]
dataset_id = original_dataset_id
entity_id = original_entity_id

# [START bigquery_revoke_dataset_access]
from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

dataset = client.get_dataset(dataset_id) # Make an API request.
entries = list(dataset.access_entries)

dataset.access_entries = entries.filter(
lambda entry: entry.entity_id != entity_id, entries
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work? I don't see filter on the list methods: https://docs.python.org/3/library/stdtypes.html#list

Perhaps you meant to use the built-in filter function? https://docs.python.org/3/library/functions.html#filter

Suggested change
dataset.access_entries = entries.filter(
lambda entry: entry.entity_id != entity_id, entries
)
dataset.access_entries = list(filter(
lambda entry: entry.entity_id != entity_id, entries
))


dataset = client.update_dataset(
dataset,
# Update just the `access_entries` property of the dataset.
["access_entries"],
) # Make an API request.

full_dataset_id = f"{dataset.project}.{dataset.dataset_id}"
print(f"Revoked dataset access for '{entity_id}' to ' dataset '{full_dataset_id}.'")
# [END bigquery_revoke_dataset_access]
43 changes: 43 additions & 0 deletions samples/snippets/revoke_dataset_access_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from . import update_dataset_access
from . import revoke_dataset_access
from google.cloud import bigquery
from functools import reduce
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. These should be grouped into 3 parts: (a) standard library [built-ins], (b) third-party packages, (c) local packages https://www.python.org/dev/peps/pep-0008/#imports
  2. Since there's no __init__.py in the samples/snippets/ directory, relative imports don't work. We'll need to (a) add __init__.py and update the other tests to use relative imports [preferred end-goal, but probably should be a separate cleanup PR] or (b) use absolute imports, which work because our tests are run with samples/snippets/ as the working directory.

https://source.cloud.google.com/results/invocations/8768d4f0-99fe-4520-8027-5be92e1db003/targets/cloud-devrel%2Fclient-libraries%2Fpython%2Fgoogleapis%2Fpython-bigquery%2Fsamples%2Fpython3.6%2Fpresubmit/log

update_dataset_access_test.py:15: in <module>
    from . import update_dataset_access
E   ImportError: attempted relative import with no known parent package
- generated xml file: /tmpfs/src/github/python-bigquery/samples/snippets/sponge_log.xml -
Suggested change
from . import update_dataset_access
from . import revoke_dataset_access
from google.cloud import bigquery
from functools import reduce
from functools import reduce
from google.cloud import bigquery
import update_dataset_access
import revoke_dataset_access



def test_revoke_dataset_access(
capsys, dataset_id, entity_id, bigquery_client: bigquery.Client
):
update_dataset_access.update_dataset_access(dataset_id)
updated_dataset = bigquery_client.get_dataset(dataset_id)
updated_dataset_entries = list(updated_dataset.access_entries)
revoke_dataset_access.revoke_dataset_access(dataset_id, entity_id)
revoked_dataset = bigquery_client.get_dataset(dataset_id)
revoked_dataset_entries = list(revoked_dataset.access_entries)

full_dataset_id = f"{updated_dataset.project}.{updated_dataset.dataset_id}"
out, err = capsys.readouterr()
assert (
f"Revoked dataset access for '{entity_id}' to ' dataset '{full_dataset_id}.'"
in out
)
assert len(revoked_dataset_entries) == len(updated_dataset_entries) - 1
assert (
reduce(
lambda entry: bool(entry.entity_id == entity_id), revoked_dataset_entries
)
== False
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all or not any would be clearer than reduce. https://docs.python.org/3/library/functions.html#all

Suggested change
assert (
reduce(
lambda entry: bool(entry.entity_id == entity_id), revoked_dataset_entries
)
== False
)
assert all(
entry.entity_id != entity_id for entry in revoked_dataset_entries
)

45 changes: 45 additions & 0 deletions samples/snippets/update_dataset_access.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


def update_dataset_access(dataset_id):
loferris marked this conversation as resolved.
Show resolved Hide resolved

# [START bigquery_update_dataset_access]
from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

# TODO(developer): Set dataset_id to the ID of the dataset to fetch.
# dataset_id = 'your-project.your_dataset'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise here, let's use the "save the original, then overwrite it" pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we missed this one.


dataset = client.get_dataset(dataset_id) # Make an API request.

entry = bigquery.AccessEntry(
role="READER",
entity_type="userByEmail",
entity_id="sample.bigquery.dev@gmail.com",
)

entries = list(dataset.access_entries)
entries.append(entry)
dataset.access_entries = entries

dataset = client.update_dataset(dataset, ["access_entries"]) # Make an API request.

full_dataset_id = "{}.{}".format(dataset.project, dataset.dataset_id)
print(
"Updated dataset '{}' with modified user permissions.".format(full_dataset_id)
)
# [END bigquery_update_dataset_access]
24 changes: 24 additions & 0 deletions samples/snippets/update_dataset_access_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from . import update_dataset_access


def test_update_dataset_access(capsys, dataset_id):

update_dataset_access.update_dataset_access(dataset_id)
out, err = capsys.readouterr()
assert (
"Updated dataset '{}' with modified user permissions.".format(dataset_id) in out
)