Add support for streaming inserts with resumable upload #534

mziccard · 2016-01-08T09:48:08Z

This PR adds support to BigQuery for inserts that use resumable upload.

Add LoadConfiguration class for data needed by load jobs and resumable uploads
Update LoadJobInfo and related classes to use LoadConfiguration instead
Add status and write methods to BigQueryRpc to support resumable insert
Add insertAll(LoadConfiguration, byte[]) method to BigQuery
Add insertAll(LoadConfiguration. SeekableByteChannel) method to BigQuery
Add Table.insert methods that use resumable upload
Add unit and integration tests

- Add LoadConfiguration class for data needed by load jobs and resumable uploads - Update LoadJobInfo and related classes to use LoadConfiguration instead - Add insertAll(LoadConfiguration, byte[]) method - Add insertAll(LoadConfiguration. SeekableByteChannel) method - Add Table.insert methods that use resumable upload - Add unit and integration tests

gcloud-java-bigquery/src/main/java/com/google/gcloud/bigquery/BigQuery.java

+   * <p>Example usage of inserting data from a local file:
+   * <pre> {@code
+   * LoadConfiguration config = LoadConfiguration.of(TableId.of("my_dataset_id", "my_table_id"));
+   * try(FileChannel channel = FileChannel.open(Paths.get("/path/to/file"))) {


lesv · 2016-05-31T19:22:15Z

ping - is this PR obsolete? Should it be deleted? Or what?

mziccard · 2016-05-31T20:24:36Z

This PR became obsolete but contains an implementation of upload that could be useful for both the BigQuery and Storage modules. Due to lack of time and low priority this remained behind. If this bothers you we can close, I will reopen a new one when ready.

sduskis · 2018-11-01T12:46:13Z

Given the age of this PR, I'm closing it.

This PR was generated using Autosynth. 🌈 <details><summary>Log from Synthtool</summary> ``` 2021-06-23 19:12:45,875 synthtool [DEBUG] > Executing /root/.cache/synthtool/java-resourcemanager/.github/readme/synth.py. On branch autosynth-readme nothing to commit, working tree clean 2021-06-23 19:12:47,112 synthtool [DEBUG] > Wrote metadata to .github/readme/synth.metadata/synth.metadata. ``` </details> Full log will be available here: https://source.cloud.google.com/results/invocations/4387e8a6-afd3-4521-add8-ebe1c917be38/targets - [ ] To automatically regenerate this PR, check this box. (May take up to 24 hours.)

…-info-reports-plugin to v3.4.0 (#534) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [org.apache.maven.plugins:maven-project-info-reports-plugin](https://maven.apache.org/plugins/) | `3.3.0` -> `3.4.0` | [![age](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.4.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.4.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.4.0/compatibility-slim/3.3.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.4.0/confidence-slim/3.3.0)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-securitycenter-settings).

🤖 I have created a release *beep* *boop* --- ## [0.14.3](googleapis/java-analytics-data@v0.14.2...v0.14.3) (2022-08-09) ### Dependencies * update dependency com.google.auth:google-auth-library-oauth2-http to v1.8.1 ([#525](googleapis/java-analytics-data#525)) ([44ba0c5](googleapis/java-analytics-data@44ba0c5)) * update dependency com.google.auth:google-auth-library-oauth2-http to v1.9.0 ([#533](googleapis/java-analytics-data#533)) ([0d03506](googleapis/java-analytics-data@0d03506)) * update dependency com.google.cloud:google-cloud-shared-dependencies to v3 ([#530](googleapis/java-analytics-data#530)) ([ed6d446](googleapis/java-analytics-data@ed6d446)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [org.mockito:mockito-core](https://github.com/mockito/mockito) | `4.6.1` -> `4.7.0` | [![age](https://badges.renovateapi.com/packages/maven/org.mockito:mockito-core/4.7.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/org.mockito:mockito-core/4.7.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/org.mockito:mockito-core/4.7.0/compatibility-slim/4.6.1)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/org.mockito:mockito-core/4.7.0/confidence-slim/4.6.1)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. ⚠ **Warning**: custom changes will be lost. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-dialogflow-cx).

🤖 I have created a release *beep* *boop* --- ### Updating meta-information for bleeding-edge SNAPSHOT release. --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

mziccard added the api: bigquery Issues related to the BigQuery API. label Jan 8, 2016

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jan 8, 2016

mziccard force-pushed the bigquery-resumable-insert branch from c9b6a1c to a430e73 Compare January 8, 2016 12:06

aozarov reviewed Jan 8, 2016
View reviewed changes

mziccard mentioned this pull request Jan 11, 2016

Add support for BigQuery resumable uploads via a write channel #540

Merged

mziccard added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jan 11, 2016

aozarov mentioned this pull request May 31, 2016

Provide a way to stream read/write requests using a single RPC call #72

Closed

danoscarmike added the dont-merge label Feb 7, 2017

lukesneeringer removed the Status: Don't Merge label Nov 8, 2017

JustinBeckwith assigned JesseLovelace Oct 17, 2018

sduskis closed this Nov 1, 2018

github-actions bot pushed a commit that referenced this pull request Jun 21, 2022

chore: include generated samples dir (#534)

7ac8b87

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for streaming inserts with resumable upload #534

Add support for streaming inserts with resumable upload #534

mziccard commented Jan 8, 2016

This comment was marked as spam.

lesv commented May 31, 2016

mziccard commented May 31, 2016

sduskis commented Nov 1, 2018

Add support for streaming inserts with resumable upload #534

Add support for streaming inserts with resumable upload #534

Conversation

mziccard commented Jan 8, 2016

This comment was marked as spam.

lesv commented May 31, 2016

mziccard commented May 31, 2016

sduskis commented Nov 1, 2018