Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate code from googleapis/python-dataproc #8509

Merged
merged 163 commits into from
Nov 15, 2022
Merged

Conversation

msampathkumar
Copy link
Member

Description

Fixes #

Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.

Checklist

arithmetic1728 and others added 30 commits August 7, 2020 17:35
* Add XMPP Sample

* Add Dataproc Sample

* Add more region tags

* Minor dataproc fixes

* Fix Dataproc e2e for Python 3

* Update reqs

* updating requirements [(#358)](#358)

Change-Id: I6177a17fad021e26ed76679d9db34848c17b62a8

* Update Reqs

* Wrong arg description

* Auto-update dependencies. [(#456)](#456)

* Auto-update dependencies. [(#459)](#459)

* Fix import order lint errors

Change-Id: Ieaf7237fc6f925daec46a07d2e81a452b841198a

* bump

Change-Id: I02e7767d13ba267ee9fc72c5b68a57013bb8b8d3

* Auto-update dependencies. [(#486)](#486)

* Auto-update dependencies. [(#540)](#540)

* Auto-update dependencies. [(#542)](#542)

* Move to google-cloud [(#544)](#544)

* Auto-update dependencies. [(#584)](#584)

* Auto-update dependencies. [(#629)](#629)

* Update samples to support latest Google Cloud Python [(#656)](#656)

* Update README.md [(#691)](#691)

* Auto-update dependencies. [(#715)](#715)

* Auto-update dependencies. [(#735)](#735)

* Auto-update dependencies.
* Fix language OCR sample
* Remove unused import

* Auto-update dependencies. [(#790)](#790)

* Remove usage of GoogleCredentials [(#810)](#810)

* Fix a typo [(#813)](#813)

* Remove cloud config fixture [(#887)](#887)

* Remove cloud config fixture

* Fix client secrets

* Fix bigtable instance

* Fix reference to our testing tools

* Auto-update dependencies. [(#914)](#914)

* Auto-update dependencies.

* xfail the error reporting test

* Fix lint

* Auto-update dependencies. [(#922)](#922)

* Auto-update dependencies.

* Fix pubsub iam samples

* Auto-update dependencies. [(#1005)](#1005)

* Auto-update dependencies.

* Fix bigtable lint

* Fix IOT iam interaction

* Auto-update dependencies. [(#1011)](#1011)

* Properly forwarding the "region" parameter provided as an input argument. [(#1029)](#1029)

* Auto-update dependencies. [(#1055)](#1055)

* Auto-update dependencies.

* Explicitly use latest bigtable client

Change-Id: Id71e9e768f020730e4ca9514a0d7ebaa794e7d9e

* Revert language update for now

Change-Id: I8867f154e9a5aae00d0047c9caf880e5e8f50c53

* Remove pdb. smh

Change-Id: I5ff905fadc026eebbcd45512d4e76e003e3b2b43

* Fix region handling and allow to use an existing cluster. [(#1053)](#1053)

* Auto-update dependencies. [(#1094)](#1094)

* Auto-update dependencies.

* Relax assertions in the ocr_nl sample

Change-Id: I6d37e5846a8d6dd52429cb30d501f448c52cbba1

* Drop unused logging apiary samples

Change-Id: I545718283773cb729a5e0def8a76ebfa40829d51

* Auto-update dependencies. [(#1133)](#1133)

* Auto-update dependencies.

* Fix missing http library

Change-Id: I99faa600f2f3f1f50f57694fc9835d7f35bda250

* Auto-update dependencies. [(#1186)](#1186)

* Auto-update dependencies. [(#1199)](#1199)

* Auto-update dependencies.

* Fix iot lint

Change-Id: I6289e093bdb35e38f9e9bfc3fbc3df3660f9a67e

* Fixed Failed Kokoro Test (Dataproc) [(#1203)](#1203)

* Fixed Failed Kokoro Test (Dataproc)

* Fixed Lint Error

* Update dataproc_e2e_test.py

* Update dataproc_e2e_test.py

* Fixing More Lint Errors

* Fixed b/65407087

* Revert "Merge branch 'master' of https://github.com/michaelawyu/python-docs-samples"

This reverts commit 1614c7d, reversing
changes made to cd1dbfd.

* Revert "Fixed b/65407087"

This reverts commit cd1dbfd.

* Fixed Lint Error

* Fixed Lint Error

* Auto-update dependencies. [(#1208)](#1208)

* Dataproc GCS sample plus doc touchups [(#1151)](#1151)

* Auto-update dependencies. [(#1217)](#1217)

* Auto-update dependencies. [(#1239)](#1239)

* Added "Open in Cloud Shell" buttons to README files [(#1254)](#1254)

* Auto-update dependencies. [(#1282)](#1282)

* Auto-update dependencies.

* Fix storage acl sample

Change-Id: I413bea899fdde4c4859e4070a9da25845b81f7cf

* Auto-update dependencies. [(#1309)](#1309)

* Auto-update dependencies. [(#1320)](#1320)

* Auto-update dependencies. [(#1355)](#1355)

* Auto-update dependencies. [(#1359)](#1359)

* Auto-update dependencies.

* update Dataproc region tags to standard format [(#1826)](#1826)

* Update submit_job_to_cluster.py [(#1708)](#1708)

switch region to new 'global' region and remove unnecessary function.

* Auto-update dependencies. [(#1846)](#1846)

ACK, merging.

* Need separate install for google-cloud-storage [(#1863)](#1863)

* Revert "Update dataproc/submit_job_to_cluster.py" [(#1864)](#1864)

* Revert "Remove test configs for non-testing directories [(#1855)](#1855)"

This reverts commit 73a7332.

* Revert "Auto-update dependencies. [(#1846)](#1846)"

This reverts commit 3adc94f4d0c14453153968c3851fae100e2c5e44.

* Revert "Tweak slack sample [(#1847)](#1847)"

This reverts commit a48c010.

* Revert "Non-client library example of constructing a Signed URL [(#1837)](#1837)"

This reverts commit fc3284d.

* Revert "GCF samples: handle {empty JSON, GET} requests + remove commas [(#1832)](#1832)"

This reverts commit 6928491.

* Revert "Correct the maintenance event types [(#1830)](#1830)"

This reverts commit c22840f.

* Revert "Fix GCF region tags [(#1827)](#1827)"

This reverts commit 0fbfef2.

* Revert "Updated to Flask 1.0 [(#1819)](#1819)"

This reverts commit d52ccf9.

* Revert "Fix deprecation warning [(#1801)](#1801)"

This reverts commit 981737e.

* Revert "Update submit_job_to_cluster.py [(#1708)](#1708)"

This reverts commit df1f2b22547b7ca86bbdb791ad930003a815a677.

* Create python-api-walkthrough.md [(#1966)](#1966)

* Create python-api-walkthrough.md

This Google Cloud Shell walkthrough is linked to Cloud Dataproc documentation to be published at: https://cloud.google.com/dataproc/docs/tutorials/python-library-example

* Update python-api-walkthrough.md

* Update list_clusters.py [(#1887)](#1887)

* Auto-update dependencies. [(#1980)](#1980)

* Auto-update dependencies.

* Update requirements.txt

* Update requirements.txt

* Update Dataproc samples. [(#2158)](#2158)

* Update requirements.txt

* Update python-api-walkthrough.md

* Update submit_job_to_cluster.py

* Update list_clusters.py

* Update python-api-walkthrough.md [(#2172)](#2172)

* Adds updates including compute [(#2436)](#2436)

* Adds updates including compute

* Python 2 compat pytest

* Fixing weird \r\n issue from GH merge

* Put asset tests back in

* Re-add pod operator test

* Hack parameter for k8s pod operator

* feat: adding samples for dataproc - create cluster [(#2536)](#2536)

* adding sample for cluster create

* small fix

* Add create cluster samples

* Fixed copyright, added 'dataproc' to region tag and changed imports from 'dataproc' to 'dataproc_v1'

* Fix copyright in create_cluster.py

* Auto-update dependencies. [(#2005)](#2005)

* Auto-update dependencies.

* Revert update of appengine/flexible/datastore.

* revert update of appengine/flexible/scipy

* revert update of bigquery/bqml

* revert update of bigquery/cloud-client

* revert update of bigquery/datalab-migration

* revert update of bigtable/quickstart

* revert update of compute/api

* revert update of container_registry/container_analysis

* revert update of dataflow/run_template

* revert update of datastore/cloud-ndb

* revert update of dialogflow/cloud-client

* revert update of dlp

* revert update of functions/imagemagick

* revert update of functions/ocr/app

* revert update of healthcare/api-client/fhir

* revert update of iam/api-client

* revert update of iot/api-client/gcs_file_to_device

* revert update of iot/api-client/mqtt_example

* revert update of language/automl

* revert update of run/image-processing

* revert update of vision/automl

* revert update testing/requirements.txt

* revert update of vision/cloud-client/detect

* revert update of vision/cloud-client/product_search

* revert update of jobs/v2/api_client

* revert update of jobs/v3/api_client

* revert update of opencensus

* revert update of translate/cloud-client

* revert update to speech/cloud-client

Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>

* feat: dataproc quickstart sample added and create_cluster updated [(#2629)](#2629)

* Adding quickstart sample

* Added new quickstart sample and updated create_cluster sample

* Fix to create_cluster.py

* deleted dataproc quickstart files not under dataproc/quickstart/

* Added quickstart test

* Linting and formatting fixes

* Revert "Linting and formatting fixes"

This reverts commit c5afcbc.

* Added bucket cleanup to quickstart test

* Changes to samples and tests

* Linting fixes

* Removed todos in favor of clearer docstring

* Fixed lint error

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* Update Python Cloud Shell walkthrough script [(#2733)](#2733)

Cloud Shell walkthrough scripts no longer support enabling APIs. APIs must be enabled by linking to the console.
Updated product name: "Cloud Dataproc" -> "Dataproc".

* fix: added cli functionality to dataproc quickstart example [(#2734)](#2734)

* Added CLI functionality to quickstart

* Fixed Dataproc quickstart test to properly clean up GCS bucket [(#3001)](#3001)

* splitting up #2651 part 1/3 - dataproc + endpoints [(#3025)](#3025)

* splitting up #2651

* fix typos

* chore(deps): update dependency google-auth to v1.11.2 [(#2724)](#2724)

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency google-cloud-storage to v1.26.0 [(#3046)](#3046)

* chore(deps): update dependency google-cloud-storage to v1.26.0

* chore(deps): specify dependencies by python version

* chore: up other deps to try to remove errors

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: Leah Cole <coleleah@google.com>

* chore(deps): update dependency google-cloud-dataproc to v0.7.0 [(#3083)](#3083)

* feat: added dataproc workflows samples [(#3056)](#3056)

* Added workflows sample

* chore(deps): update dependency grpcio to v1.27.2 [(#3173)](#3173)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [grpcio](https://grpc.io) | minor | `==1.25.0` -> `==1.27.2` |
| [grpcio](https://grpc.io) | minor | `==1.23.0` -> `==1.27.2` |
| [grpcio](https://grpc.io) | minor | `==1.26.0` -> `==1.27.2` |
| [grpcio](https://grpc.io) | patch | `==1.27.1` -> `==1.27.2` |

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* fix: add mains to samples [(#3284)](#3284)

Added mains to two samples: create_cluster and instantiate_inline_workflow_templates.

Fixed their associated tests to accommodate this.

Removed subprocess from quickstart/quickstart_test.py to fix [2873](#2873)

fixes #2873

* Update dependency grpcio to v1.28.1 [(#3276)](#3276)

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* Update dependency google-auth to v1.14.0 [(#3148)](#3148)

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency google-auth to v1.14.1 [(#3464)](#3464)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://github.com/googleapis/google-auth-library-python) | patch | `==1.14.0` -> `==1.14.1` |
| [google-auth](https://github.com/googleapis/google-auth-library-python) | minor | `==1.11.2` -> `==1.14.1` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.14.1`](https://github.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1141-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1140v1141-2020-04-21)

[Compare Source](https://github.com/googleapis/google-auth-library-python/compare/v1.14.0...v1.14.1)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore(deps): update dependency google-cloud-storage to v1.28.0 [(#3260)](#3260)

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* chore(deps): update dependency google-auth to v1.14.2 [(#3724)](#3724)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://github.com/googleapis/google-auth-library-python) | patch | `==1.14.1` -> `==1.14.2` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.14.2`](https://github.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1142-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1141v1142-2020-05-07)

[Compare Source](https://github.com/googleapis/google-auth-library-python/compare/v1.14.1...v1.14.2)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore: some lint fixes [(#3743)](#3743)

* chore(deps): update dependency google-auth to v1.14.3 [(#3728)](#3728)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://github.com/googleapis/google-auth-library-python) | patch | `==1.14.2` -> `==1.14.3` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.14.3`](https://github.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1143-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1142v1143-2020-05-11)

[Compare Source](https://github.com/googleapis/google-auth-library-python/compare/v1.14.2...v1.14.3)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [x] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore(deps): update dependency grpcio to v1.29.0 [(#3786)](#3786)

* chore(deps): update dependency google-cloud-storage to v1.28.1 [(#3785)](#3785)

* chore(deps): update dependency google-cloud-storage to v1.28.1

* [asset] testing: use uuid instead of time

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* update google-auth to 1.15.0 part 3 [(#3816)](#3816)

* Update dependency google-cloud-dataproc to v0.8.0 [(#3837)](#3837)

* chore(deps): update dependency google-auth to v1.16.0 [(#3903)](#3903)

* update google-auth part 3 [(#3963)](#3963)

* chore(deps): update dependency google-cloud-dataproc to v0.8.1 [(#4015)](#4015)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-dataproc](https://github.com/googleapis/python-dataproc) | patch | `==0.8.0` -> `==0.8.1` |

---

### Release Notes

<details>
<summary>googleapis/python-dataproc</summary>

### [`v0.8.1`](https://github.com/googleapis/python-dataproc/blob/master/CHANGELOG.md#&#8203;081-httpswwwgithubcomgoogleapispython-dataproccomparev080v081-2020-06-05)

[Compare Source](https://github.com/googleapis/python-dataproc/compare/v0.8.0...v0.8.1)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](#4022)

* Update dependency google-auth to v1.17.0 [(#4058)](#4058)

* chore(deps): update dependency google-auth to v1.17.1 [(#4073)](#4073)

* Update dependency google-auth to v1.17.2 [(#4083)](#4083)

* Update dependency google-auth to v1.18.0 [(#4125)](#4125)

* Update dependency google-cloud-dataproc to v1 [(#4109)](#4109)

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* chore(deps): update dependency google-cloud-storage to v1.29.0 [(#4040)](#4040)

* chore(deps): update dependency grpcio to v1.30.0 [(#4143)](#4143)

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* Update dependency google-auth-httplib2 to v0.0.4 [(#4255)](#4255)

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* chore(deps): update dependency google-auth to v1.19.0 [(#4293)](#4293)

* chore(deps): update dependency google-cloud-dataproc to v1.0.1 [(#4309)](#4309)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-dataproc](https://github.com/googleapis/python-dataproc) | patch | `==1.0.0` -> `==1.0.1` |

---

### Release Notes

<details>
<summary>googleapis/python-dataproc</summary>

### [`v1.0.1`](https://github.com/googleapis/python-dataproc/blob/master/CHANGELOG.md#&#8203;101-httpswwwgithubcomgoogleapispython-dataproccomparev100v101-2020-07-16)

[Compare Source](https://github.com/googleapis/python-dataproc/compare/v1.0.0...v1.0.1)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* chore(deps): update dependency google-auth to v1.19.1 [(#4304)](#4304)

* chore(deps): update dependency google-auth to v1.19.2 [(#4321)](#4321)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-auth](https://github.com/googleapis/google-auth-library-python) | patch | `==1.19.1` -> `==1.19.2` |

---

### Release Notes

<details>
<summary>googleapis/google-auth-library-python</summary>

### [`v1.19.2`](https://github.com/googleapis/google-auth-library-python/blob/master/CHANGELOG.md#&#8203;1192-httpswwwgithubcomgoogleapisgoogle-auth-library-pythoncomparev1191v1192-2020-07-17)

[Compare Source](https://github.com/googleapis/google-auth-library-python/compare/v1.19.1...v1.19.2)

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Update dependency google-auth to v1.20.0 [(#4387)](#4387)

* Update dependency pytest to v6 [(#4390)](#4390)

* Update dependency grpcio to v1.31.0 [(#4438)](#4438)

* chore(deps): update dependency google-auth to v1.20.1 [(#4452)](#4452)

* chore: update templates

Co-authored-by: Bill Prin <waprin@google.com>
Co-authored-by: Bill Prin <waprin@gmail.com>
Co-authored-by: Jon Wayne Parrott <jonwayne@google.com>
Co-authored-by: Eran Kampf <eran@ekampf.com>
Co-authored-by: DPE bot <dpebot@google.com>
Co-authored-by: aman-ebay <amancuso@google.com>
Co-authored-by: Martial Hue <martial.hue@gmail.com>
Co-authored-by: Gioia Ballin <gioia.ballin@gmail.com>
Co-authored-by: michaelawyu <chenyumic@google.com>
Co-authored-by: michaelawyu <michael.a.w.yu@hotmail.com>
Co-authored-by: Alix Hamilton <ajhamilton@google.com>
Co-authored-by: James Winegar <jameswinegar@users.noreply.github.com>
Co-authored-by: Charles Engelke <github@engelke.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: Brad Miro <bmiro@google.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: Doug Mahugh <dmahugh@gmail.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: WhiteSource Renovate <bot@renovateapp.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-storage](https://github.com/googleapis/python-storage) | minor | `==1.29.0` -> `==1.30.0` |

---

### Release Notes

<details>
<summary>googleapis/python-storage</summary>

### [`v1.30.0`](https://github.com/googleapis/python-storage/blob/master/CHANGELOG.md#&#8203;1300-httpswwwgithubcomgoogleapispython-storagecomparev1290v1300-2020-07-24)

[Compare Source](https://github.com/googleapis/python-storage/compare/v1.29.0...v1.30.0)

##### Features

-   add timeouts to Blob methods where missing ([#&#8203;185](https://www.github.com/googleapis/python-storage/issues/185)) ([6eeb855](https://www.github.com/googleapis/python-storage/commit/6eeb855aa0e6a7835d1d4f6e72951e43af22ab57))
-   auto-populate standard headers for non-chunked downloads ([#&#8203;204](https://www.github.com/googleapis/python-storage/issues/204)) ([d8432cd](https://www.github.com/googleapis/python-storage/commit/d8432cd65a4e9b38eebd1ade2ff00f2f44bb0ef6)), closes [#&#8203;24](https://www.github.com/googleapis/python-storage/issues/24)
-   migrate to Service Account Credentials API ([#&#8203;189](https://www.github.com/googleapis/python-storage/issues/189)) ([e4990d0](https://www.github.com/googleapis/python-storage/commit/e4990d06043dbd8d1a417f3a1a67fe8746071f1c))

##### Bug Fixes

-   add multiprocessing.rst to synthool excludes ([#&#8203;186](https://www.github.com/googleapis/python-storage/issues/186)) ([4d76e38](https://www.github.com/googleapis/python-storage/commit/4d76e3882210ed2818a43256265f6df31348d410))

##### Documentation

-   fix indent in code blocks ([#&#8203;171](https://www.github.com/googleapis/python-storage/issues/171)) ([62d1543](https://www.github.com/googleapis/python-storage/commit/62d1543e18040b286b23464562aa6eb998074c54)), closes [#&#8203;170](https://www.github.com/googleapis/python-storage/issues/170)
-   remove doubled word in docstring ([#&#8203;209](https://www.github.com/googleapis/python-storage/issues/209)) ([7a4e7a5](https://www.github.com/googleapis/python-storage/commit/7a4e7a5974abedb0b7b2e110cacbfcd0a40346b6))

##### Documentation

-   fix indent in code blocks ([#&#8203;171](https://www.github.com/googleapis/python-storage/issues/171)) ([62d1543](https://www.github.com/googleapis/python-storage/commit/62d1543e18040b286b23464562aa6eb998074c54)), closes [#&#8203;170](https://www.github.com/googleapis/python-storage/issues/170)
-   remove doubled word in docstring ([#&#8203;209](https://www.github.com/googleapis/python-storage/issues/209)) ([7a4e7a5](https://www.github.com/googleapis/python-storage/commit/7a4e7a5974abedb0b7b2e110cacbfcd0a40346b6))

##### Dependencies

-   prep for grmp-1.0.0 release ([#&#8203;211](https://www.github.com/googleapis/python-storage/issues/211)) ([55bae9a](https://www.github.com/googleapis/python-storage/commit/55bae9a0e7c0db512c10c6b3b621cd2ef05c9729))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#googleapis/python-dataproc).
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-dataproc](https://github.com/googleapis/python-dataproc) | minor | `==1.0.1` -> `==1.1.0` |

---

### Release Notes

<details>
<summary>googleapis/python-dataproc</summary>

### [`v1.1.0`](https://github.com/googleapis/python-dataproc/blob/master/CHANGELOG.md#&#8203;110-httpswwwgithubcomgoogleapispython-dataproccomparev101v110-2020-07-31)

[Compare Source](https://github.com/googleapis/python-dataproc/compare/v1.0.1...v1.1.0)

##### Features

-   add support for temp_bucket, endpoint_config in clusters; add preemptibility for instance group configs ([#&#8203;60](https://www.github.com/googleapis/python-dataproc/issues/60)) ([a80fc72](https://www.github.com/googleapis/python-dataproc/commit/a80fc727510c10c678caa125902c201c8280dcc1))

##### [1.0.1](https://www.github.com/googleapis/python-dataproc/compare/v1.0.0...v1.0.1) (2020-07-16)

##### Bug Fixes

-   correct protobuf type for diagnose_cluster, update retry configs ([#&#8203;55](https://www.github.com/googleapis/python-dataproc/issues/55)) ([822315e](https://www.github.com/googleapis/python-dataproc/commit/822315ec3f2517ebb6ca199b72156ebd50e0518b))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#googleapis/python-dataproc).
* feat!: migrate to use microgen

* update

* update

* update

* update
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [grpcio](https://grpc.io) | minor | `==1.31.0` -> `==1.32.0` |

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-dataproc).
Adding submit_job samples and updating quickstart samples.
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [grpcio](https://grpc.io) | patch | `==1.33.1` -> `==1.33.2` |

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-dataproc).
* changes without context

        autosynth cannot find the source of changes triggered by earlier changes in this
        repository, or by version upgrades to tools such as linters.

* chore(py-library): enable snippet-bot

Co-authored-by: Benjamin E. Coe <bencoe@google.com>

Source-Author: Takashi Matsuo <tmatsuo@google.com>
Source-Date: Tue Sep 1 17:14:08 2020 +0000
Source-Repo: googleapis/synthtool
Source-Sha: d91dd8aac77f7a9c5506c238038a26fa4f9e361e
Source-Link: googleapis/synthtool@d91dd8a

* chore(py-library): update decrypt secrets file

* chore(py-library): update decrypt secrets file

From https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/scripts/decrypt-secrets.sh

* docs: explain conditional

Co-authored-by: Jeffrey Rennie <rennie@google.com>

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Tue Sep 8 11:35:59 2020 -0600
Source-Repo: googleapis/synthtool
Source-Sha: d302f93d7f47e2852e585ac35ab2d15585717ec0
Source-Link: googleapis/synthtool@d302f93

* chore(python-library): use sphinx 1.5.5 for the docfx job

Originally tested at:
googleapis/python-texttospeech#89

This change will fix the missing docstring in the yaml files.

Source-Author: Takashi Matsuo <tmatsuo@google.com>
Source-Date: Thu Sep 10 04:12:14 2020 +0000
Source-Repo: googleapis/synthtool
Source-Sha: ffcee7952b74f647cbb3ef021d95422f10816fca
Source-Link: googleapis/synthtool@ffcee79

* build(python): use release-publish app for notifying GitHub of releas…

* build(python): use release-publish app for notifying GitHub of release status

* fix: re-add pypi password

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Wed Sep 16 08:46:42 2020 -0600
Source-Repo: googleapis/synthtool
Source-Sha: 257fda18168bedb76985024bd198ed1725485488
Source-Link: googleapis/synthtool@257fda1

* build(python): add secret manager in kokoro

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Wed Sep 16 10:24:40 2020 -0600
Source-Repo: googleapis/synthtool
Source-Sha: dba48bb9bc6959c232bec9150ac6313b608fe7bd
Source-Link: googleapis/synthtool@dba48bb

* chore(python): add sphinx doctest extension

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Mon Sep 21 13:09:57 2020 -0600
Source-Repo: googleapis/synthtool
Source-Sha: 27f4406999b1eee29e04b09b2423a8e4646c7e24
Source-Link: googleapis/synthtool@27f4406

* chore(python): remove note about editable installs

`pip install -e .` is supported and is how we install the library for tests.

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Tue Sep 22 12:06:12 2020 -0600
Source-Repo: googleapis/synthtool
Source-Sha: a651c5fb763c69a921aecdd3e1d8dc51dbf20f8d
Source-Link: googleapis/synthtool@a651c5f

* chore(python): use BUILD_SPECIFIC_GCLOUD_PROJECT for samples

https://github.com/googleapis/python-talent/blob/ef045e8eb348db36d7a2a611e6f26b11530d273b/samples/snippets/noxfile_config.py#L27-L32

`BUILD_SPECIFIC_GCLOUD_PROJECT` is an alternate project used for sample tests that do poorly with concurrent runs on the same project.

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Wed Sep 30 13:06:03 2020 -0600
Source-Repo: googleapis/synthtool
Source-Sha: 9b0da5204ab90bcc36f8cd4e5689eff1a54cc3e4
Source-Link: googleapis/synthtool@9b0da52

* chore(python): use 'setup.py' to detect repo root

Closes #792

Source-Author: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Source-Date: Fri Oct 9 15:06:33 2020 -0600
Source-Repo: googleapis/synthtool
Source-Sha: e0ae456852bf22f38796deb79cff30b516fde244
Source-Link: googleapis/synthtool@e0ae456

* build(python): samples tests should pass if no samples exist

Source-Author: Daniel Sanche <d.sanche14@gmail.com>
Source-Date: Wed Oct 14 08:00:06 2020 -0700
Source-Repo: googleapis/synthtool
Source-Sha: 477764cc4ee6db346d3febef2bb1ea0abf27de52
Source-Link: googleapis/synthtool@477764c

* chore(python_library): change the docs bucket name

Source-Author: Takashi Matsuo <tmatsuo@google.com>
Source-Date: Fri Oct 16 09:58:05 2020 -0700
Source-Repo: googleapis/synthtool
Source-Sha: da5c6050d13b4950c82666a81d8acd25157664ae
Source-Link: googleapis/synthtool@da5c605

* chore(docs): update code of conduct of synthtool and templates

Source-Author: Christopher Wilcox <crwilcox@google.com>
Source-Date: Thu Oct 22 14:22:01 2020 -0700
Source-Repo: googleapis/synthtool
Source-Sha: 5f6ef0ec5501d33c4667885b37a7685a30d41a76
Source-Link: googleapis/synthtool@5f6ef0e

* docs: add proto-plus to intersphinx mapping

Source-Author: Tim Swast <swast@google.com>
Source-Date: Tue Oct 27 12:01:14 2020 -0500
Source-Repo: googleapis/synthtool
Source-Sha: ea52b8a0bd560f72f376efcf45197fb7c8869120
Source-Link: googleapis/synthtool@ea52b8a

* fix(python_library): fix external unit test dependencies

I recently submitted https://github.com/googleapis/synthtool/pull/811/files, allowing external dependencies for unit tests. This fixes a small missing comma bug

Source-Author: Daniel Sanche <d.sanche14@gmail.com>
Source-Date: Thu Oct 29 16:58:01 2020 -0700
Source-Repo: googleapis/synthtool
Source-Sha: 6542bd723403513626f61642fc02ddca528409aa
Source-Link: googleapis/synthtool@6542bd7

* chore: add type hint check

Source-Author: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Source-Date: Wed Nov 4 17:36:32 2020 -0800
Source-Repo: googleapis/synthtool
Source-Sha: 3d3e94c4e02370f307a9a200b0c743c3d8d19f29
Source-Link: googleapis/synthtool@3d3e94c

* chore: add blacken to template

Source-Author: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Source-Date: Thu Nov 5 15:22:03 2020 -0800
Source-Repo: googleapis/synthtool
Source-Sha: 1f1148d3c7a7a52f0c98077f976bd9b3c948ee2b
Source-Link: googleapis/synthtool@1f1148d

* fix: address lint issues

Source-Author: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Source-Date: Thu Nov 12 11:30:49 2020 -0800
Source-Repo: googleapis/synthtool
Source-Sha: e89175cf074dccc4babb4eca66ae913696e47a71
Source-Link: googleapis/synthtool@e89175c
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-storage](https://github.com/googleapis/python-storage) | minor | `==1.31.2` -> `==1.32.0` |

---

### Release Notes

<details>
<summary>googleapis/python-storage</summary>

### [`v1.32.0`](https://github.com/googleapis/python-storage/blob/master/CHANGELOG.md#&#8203;1320-httpswwwgithubcomgoogleapispython-storagecomparev1312v1320-2020-10-16)

[Compare Source](https://github.com/googleapis/python-storage/compare/v1.31.2...v1.32.0)

##### Features

-   retry API calls with exponential backoff ([#&#8203;287](https://www.github.com/googleapis/python-storage/issues/287)) ([fbe5d9c](https://www.github.com/googleapis/python-storage/commit/fbe5d9ca8684c6a992dcdee977fc8dd012a96a5c))

##### Bug Fixes

-   field policy return string ([#&#8203;282](https://www.github.com/googleapis/python-storage/issues/282)) ([c356b84](https://www.github.com/googleapis/python-storage/commit/c356b8484a758548d5f4823a495ab70c798cfaaf))
-   self-upload files for Unicode system test ([#&#8203;296](https://www.github.com/googleapis/python-storage/issues/296)) ([6f865d9](https://www.github.com/googleapis/python-storage/commit/6f865d97a19278884356055dfeeaae92f7c63cc1))
-   use version.py for versioning, avoid issues with discovering version via get_distribution ([#&#8203;288](https://www.github.com/googleapis/python-storage/issues/288)) ([fcd1c4f](https://www.github.com/googleapis/python-storage/commit/fcd1c4f7c947eb95d6937783fd69670a570f145e))

##### [1.31.2](https://www.github.com/googleapis/python-storage/compare/v1.31.1...v1.31.2) (2020-09-23)

##### Documentation

-   fix docstring example for 'blob.generate_signed_url' ([#&#8203;278](https://www.github.com/googleapis/python-storage/issues/278)) ([2dc91c9](https://www.github.com/googleapis/python-storage/commit/2dc91c947e3693023b4478a15c460693808ea2d9))

##### [1.31.1](https://www.github.com/googleapis/python-storage/compare/v1.31.0...v1.31.1) (2020-09-16)

##### Bug Fixes

-   add requests as a dependency ([#&#8203;271](https://www.github.com/googleapis/python-storage/issues/271)) ([ec52b38](https://www.github.com/googleapis/python-storage/commit/ec52b38df211fad18a86d7e16d83db79de59d5f5))
-   preserve existing blob hashes when 'X-Goog-Hash header' is not present ([#&#8203;267](https://www.github.com/googleapis/python-storage/issues/267)) ([277afb8](https://www.github.com/googleapis/python-storage/commit/277afb83f464d77b163f2722272092df4180411e))
-   **blob:** base64 includes additional characters ([#&#8203;258](https://www.github.com/googleapis/python-storage/issues/258)) ([cf0774a](https://www.github.com/googleapis/python-storage/commit/cf0774aa8ffd45d340aff9a7d2236d8d65c8ae93))

##### Documentation

-   add docs signed_url expiration take default utc ([#&#8203;250](https://www.github.com/googleapis/python-storage/issues/250)) ([944ab18](https://www.github.com/googleapis/python-storage/commit/944ab1827b3ca0bd1d3aafc2829245290e9bde59))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/python-dataproc).
@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Nov 14, 2022
@google-cla
Copy link

google-cla bot commented Nov 14, 2022

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@msampathkumar
Copy link
Member Author

Noticed some availability issues!

UNKNOWN:Error received from peer ipv4:173.194.202.95:443 {created_time:"2022-11-14T15:51:46.442498476+00:00", grpc_status:3, grpc_message:"Insufficient \'DISKS_TOTAL_GB\' quota. Requested 3000.0, available 696.0."}"

@msampathkumar msampathkumar marked this pull request as ready for review November 15, 2022 15:43
@msampathkumar msampathkumar requested review from a team as code owners November 15, 2022 15:43
@snippet-bot
Copy link

snippet-bot bot commented Nov 15, 2022

Here is the summary of changes.

You are about to add 14 region tags.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@msampathkumar
Copy link
Member Author

Hi, I see Insufficient 'IN_USE_ADDRESSES' quota. Requested 3.0, available 0.0. as the only error for failing checks(3.8 and 3.10).

@msampathkumar
Copy link
Member Author

Please check and advise.

Copy link
Contributor

@engelke engelke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@engelke engelke merged commit 4827801 into main Nov 15, 2022
@engelke engelke deleted the python-dataproc-migration branch November 15, 2022 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
samples Issues that are directly related to samples.
Projects
None yet
Development

Successfully merging this pull request may close these issues.