Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs(samples): add query external bigtable using temp table #763

Merged
merged 5 commits into from
Oct 6, 2020
Merged

docs(samples): add query external bigtable using temp table #763

merged 5 commits into from
Oct 6, 2020

Conversation

pmakani
Copy link
Contributor

@pmakani pmakani commented Sep 30, 2020

No description provided.

@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Sep 30, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 30, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 30, 2020
@stephaniewang526
Copy link
Contributor

Hi @billyjacobson -- PTAL as well! Thank you.

@stephaniewang526 stephaniewang526 marked this pull request as ready for review September 30, 2020 15:54
@stephaniewang526 stephaniewang526 requested a review from a team September 30, 2020 15:54
@stephaniewang526 stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 30, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 30, 2020
@codecov
Copy link

codecov bot commented Sep 30, 2020 •
edited
Loading

Codecov Report

Merging #763 into master will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #763      +/-   ##
============================================
- Coverage     80.40%   80.39%   -0.01%     
- Complexity     1263     1265       +2     
============================================
  Files            79       79              
  Lines          6517     6525       +8     
  Branches        740      743       +3     
============================================
+ Hits           5240     5246       +6     
- Misses          888      889       +1     
- Partials        389      390       +1
Impacted Files Coverage Δ Complexity Δ
...main/java/com/google/cloud/bigquery/FieldList.java 100.00% <0.00%> (ø) 11.00% <0.00%> (ø%)
...google/cloud/bigquery/ExternalTableDefinition.java 58.55% <0.00%> (+0.46%) 21.00% <0.00%> (+2.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4c5bdad...691519b. Read the comment docs.

billyjacobson
billyjacobson reviewed Sep 30, 2020
View reviewed changes
samples/snippets/src/main/java/com/example/bigquery/QueryExternalBigtableTemp.java
queryExternalBigtableTemp(tableName, sourceUri, query);
}

public static void queryExternalBigtableTemp(String tableName, String sourceUri, String query) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a sample schema for bigtable we've been using that would be good to include here so it feels a little more realistic. I'll send you a doc with the schema

table name: mobile-time-series
column families: stats_summary
columns: os_build (STRING), os_name (STRING)

Copy link
Contributor

@stephaniewang526 stephaniewang526 Oct 2, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/test/java/com/example/bigtable/ReadsTest.java
Let's stick to the existing table schema as shown above instead of using our own schema and stream in data in the beforeClass() method. We can keep the table name as bigquery-samples-test

samples/snippets/src/main/java/com/example/bigquery/QueryExternalBigtableTemp.java Outdated
.setType("STRING")
.setEncoding("TEXT")
.build();
BigtableColumnFamily usStates =
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you define the column family first and then add the columns and build after? It will make more sense for someone from Bigtable following since column families are the higher level grouping

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@billyjacobson It's not possible to main order, because first we need to create an object of BigtableColumn then we can set those columns in BigtableColumnFamily.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could first instantiate an empty BigtableColumnFamily object and then add columns to it.

samples/snippets/src/main/java/com/example/bigquery/QueryExternalBigtableTemp.java Outdated Show resolved Hide resolved
samples/snippets/src/test/java/com/example/bigquery/QueryExternalBigtableTempIT.java Outdated Show resolved Hide resolved
@billyjacobson billyjacobson mentioned this pull request Sep 30, 2020
Merged
@product-auto-label product-auto-label bot added api: bigquery Issues related to the googleapis/java-bigquery API. samples Issues that are directly related to samples. labels Oct 1, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 2, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 2, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 2, 2020
@pmakani pmakani requested a review from a team October 5, 2020 11:23
@pmakani pmakani requested a review from a team as a code owner October 5, 2020 11:23
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 5, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 5, 2020
@stephaniewang526
Copy link
Contributor

LGTM -- @billyjacobson PTAL

@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 5, 2020
stephaniewang526
stephaniewang526 reviewed Oct 6, 2020
View reviewed changes
samples/snippets/src/main/java/com/example/bigquery/QueryExternalBigtableTemp.java
String bigqueryTableName = "MY_TABLE_NAME";
String sourceUri =
String.format(
"https://googleapis.com/bigtable/projects/%s/instances/%s/tables/%s",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be causing issue:

com.google.api.gax.rpc.NotFoundException: io.grpc.StatusRuntimeException: NOT_FOUND: Failed to read: projects/{779844219229}/instances/bigquery-samples-instance
Caused by: io.grpc.StatusRuntimeException: NOT_FOUND: Failed to read: projects/{779844219229}/instances/bigquery-samples-instance ```

@stephaniewang526 stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 6, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 6, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 6, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 6, 2020
@stephaniewang526 stephaniewang526 merged commit fed28c7 into googleapis:master Oct 6, 2020
@pmakani pmakani deleted the query-external-bigtable-temp branch October 7, 2020 05:28
gcf-merge-on-green bot pushed a commit that referenced this pull request Oct 9, 2020
@release-please
chore: release 1.122.2 (#784)
8469d31 
🤖 I have created a release \*beep\* \*boop\* 
---
### [1.122.2](https://www.github.com/googleapis/java-bigquery/compare/v1.122.1...v1.122.2) (2020-10-08)


### Bug Fixes

* bigquery exception get reason and message ([#788](https://www.github.com/googleapis/java-bigquery/issues/788)) ([d783c3a](https://www.github.com/googleapis/java-bigquery/commit/d783c3a2baf80da2aa48c6be555904bd3c33bb1a))


### Documentation

* **samples:** add query external bigtable using permanent table ([#764](https://www.github.com/googleapis/java-bigquery/issues/764)) ([257edfc](https://www.github.com/googleapis/java-bigquery/commit/257edfc14ef09efc4474eb5aba08ccf4d7c199c0))
* **samples:** add query external bigtable using temp table ([#763](https://www.github.com/googleapis/java-bigquery/issues/763)) ([fed28c7](https://www.github.com/googleapis/java-bigquery/commit/fed28c78ca4b70569d5148e67522b0f39321b306))
---


This PR was generated with [Release Please](https://github.com/googleapis/release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stephaniewang526 stephaniewang526 stephaniewang526 approved these changes

@billyjacobson billyjacobson billyjacobson approved these changes

Assignees
No one assigned
Labels
api: bigquery Issues related to the googleapis/java-bigquery API. cla: yes This human has signed the Contributor License Agreement. samples Issues that are directly related to samples.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@pmakani @stephaniewang526 @billyjacobson @yoshi-kokoro