-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: create manual client for Write API #112
Conversation
First version, test to be developed.
…brary helps to convert a ProtoDescriptorProto out of ProtoDescriptor into a self contained ProtoDescriptorProto, that can be passed into API and reconstructed on the server side. modified: google-cloud-bigquerystorage/pom.xml new file: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/ProtoSchemaConverter.java new file: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/ProtoSchemaConverterTest.java new file: google-cloud-bigquerystorage/src/test/proto/test.proto modified: pom.xml
First version, test to be developed.
modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/FakeBigQueryWriteImpl.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/FakeScheduledExecutorService.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriterTest.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java
modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Codecov Report
@@ Coverage Diff @@
## master #112 +/- ##
============================================
+ Coverage 72.36% 73.88% +1.52%
- Complexity 281 319 +38
============================================
Files 39 42 +3
Lines 1809 2229 +420
Branches 4 70 +66
============================================
+ Hits 1309 1647 +338
- Misses 496 552 +56
- Partials 4 30 +26
Continue to review full report at Codecov.
|
please take a look at the various build failures. |
…work modified: google-cloud-bigquerystorage/pom.xml modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriterTest.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java modified: google-cloud-bigquerystorage/src/test/proto/test.proto modified: google-cloud-bigquerystorage/pom.xml modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriterTest.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java modified: google-cloud-bigquerystorage/src/test/proto/test.proto
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Outdated
Show resolved
Hide resolved
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Show resolved
Hide resolved
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Outdated
Show resolved
Hide resolved
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Outdated
Show resolved
Hide resolved
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Show resolved
Hide resolved
As a general note, all |
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Outdated
Show resolved
Hide resolved
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Show resolved
Hide resolved
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Show resolved
Hide resolved
...d-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java
Show resolved
Hide resolved
...e-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java
Outdated
Show resolved
Hide resolved
...e-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java
Outdated
Show resolved
Hide resolved
...e-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java
Outdated
Show resolved
Hide resolved
...e-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java
Outdated
Show resolved
Hide resolved
...test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java
Outdated
Show resolved
Hide resolved
...test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java
Show resolved
Hide resolved
…gquery/storage/v1alpha2/Waiter.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com>
…gquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com>
…gquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PTAL
...e-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java
Outdated
Show resolved
Hide resolved
...e-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java
Outdated
Show resolved
Hide resolved
remove dupe gax; add undeclared commons-lang3
98851e9 commit 98851e9 Author: Yiru Tang <yiru@google.com> Date: Thu Apr 2 17:45:37 2020 -0700 feat: create manual client for Write API (#112) * First writeapi manual client First version, test to be developed. * Manual client with e2e * StreamWriter library * ProtoSchema convert library WriteApi client library addition. This library helps to convert a ProtoDescriptorProto out of ProtoDescriptor into a self contained ProtoDescriptorProto, that can be passed into API and reconstructed on the server side. modified: google-cloud-bigquerystorage/pom.xml new file: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/ProtoSchemaConverter.java new file: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/ProtoSchemaConverterTest.java new file: google-cloud-bigquerystorage/src/test/proto/test.proto modified: pom.xml * First writeapi manual client First version, test to be developed. * . * incremental development * . modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/FakeBigQueryWriteImpl.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/FakeScheduledExecutorService.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriterTest.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java * . * . * . * feat: create manual java client for writeapi modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java * feat: add integration test final verification. * . * . modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java * . * . * . * . * avoid duplicate count++ and make it easier to read * Update google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com> * Update google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com> * Update google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com> * . * . * . * . * . * dep: add undeclared dep in parent pom * dep: remove duplicate and add undeclared remove dupe gax; add undeclared commons-lang3 Co-authored-by: Stephanie Wang <stephaniewang526@users.noreply.github.com>
…nerator. (#134) * feat: create manual client for Write API (#112) 98851e9 commit 98851e9 Author: Yiru Tang <yiru@google.com> Date: Thu Apr 2 17:45:37 2020 -0700 feat: create manual client for Write API (#112) * First writeapi manual client First version, test to be developed. * Manual client with e2e * StreamWriter library * ProtoSchema convert library WriteApi client library addition. This library helps to convert a ProtoDescriptorProto out of ProtoDescriptor into a self contained ProtoDescriptorProto, that can be passed into API and reconstructed on the server side. modified: google-cloud-bigquerystorage/pom.xml new file: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/ProtoSchemaConverter.java new file: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/ProtoSchemaConverterTest.java new file: google-cloud-bigquerystorage/src/test/proto/test.proto modified: pom.xml * First writeapi manual client First version, test to be developed. * . * incremental development * . modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/FakeBigQueryWriteImpl.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/FakeScheduledExecutorService.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriterTest.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java * . * . * . * feat: create manual java client for writeapi modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java * feat: add integration test final verification. * . * . modified: google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java modified: google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java * . * . * . * . * avoid duplicate count++ and make it easier to read * Update google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/Waiter.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com> * Update google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com> * Update google-cloud-bigquerystorage/src/test/java/com/google/cloud/bigquery/storage/v1alpha2/it/ITBigQueryWriteManualClientTest.java Co-Authored-By: Stephanie Wang <stephaniewang526@users.noreply.github.com> * . * . * . * . * . * dep: add undeclared dep in parent pom * dep: remove duplicate and add undeclared remove dupe gax; add undeclared commons-lang3 Co-authored-by: Stephanie Wang <stephaniewang526@users.noreply.github.com> * deps: update dependency io.grpc:grpc-bom to v1.28.1 (#132) 711c5c7 commit 711c5c7 Author: WhiteSource Renovate <bot@renovateapp.com> Date: Fri Apr 3 03:08:41 2020 +0200 deps: update dependency io.grpc:grpc-bom to v1.28.1 (#132) * chore: release 0.127.0 (#122) 45fc919 commit 45fc919 Author: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Date: Thu Apr 2 21:49:46 2020 -0400 chore: release 0.127.0 (#122) * updated CHANGELOG.md [ci skip] * updated README.md [ci skip] * updated versions.txt [ci skip] * updated google-cloud-bigquerystorage-bom/pom.xml [ci skip] * updated grpc-google-cloud-bigquerystorage-v1/pom.xml [ci skip] * updated grpc-google-cloud-bigquerystorage-v1alpha2/pom.xml [ci skip] * updated grpc-google-cloud-bigquerystorage-v1beta1/pom.xml [ci skip] * updated grpc-google-cloud-bigquerystorage-v1beta2/pom.xml [ci skip] * updated proto-google-cloud-bigquerystorage-v1/pom.xml [ci skip] * updated proto-google-cloud-bigquerystorage-v1alpha2/pom.xml [ci skip] * updated proto-google-cloud-bigquerystorage-v1beta1/pom.xml [ci skip] * updated proto-google-cloud-bigquerystorage-v1beta2/pom.xml [ci skip] * updated samples/install-without-bom/pom.xml [ci skip] * updated samples/pom.xml [ci skip] * updated samples/snapshot/pom.xml [ci skip] * updated samples/snippets/pom.xml [ci skip] * updated google-cloud-bigquerystorage/pom.xml [ci skip] * updated pom.xml [ci skip] Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
….0 (googleapis#112) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [com.google.cloud:libraries-bom](https://github.com/GoogleCloudPlatform/cloud-opensource-java) | minor | `5.3.0` -> `5.4.0` | --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#googleapis/java-storage-nio).
Fixes #111 ☕️
A StreamWriter that provides built-in capabilities to help making bi-di streaming client easy to use:
Example:
WriteStream writeStream =
client.createWriteStream(
CreateWriteStreamRequest.newBuilder()
.setParent(tableId2)
.setWriteStream(WriteStream.newBuilder().setType(WriteStream.Type.PENDING).build())
.build());
StreamWriter streamWriter = StreamWriter.newBuilder(writeStream.getName()).build();
ApiFuture response =
streamWriter.append(
createAppendRequestComplicateType(writeStream.getName(), new String[] {"aaa"})
.setOffset(Int64Value.of(0L))
.build());
assertEquals(0, response.get().getOffset());
streamWriter.shutdown();
}