Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: small updates to dataproc sample, test and pom #1738

Merged
merged 17 commits into from
Nov 16, 2019
Merged

fix: small updates to dataproc sample, test and pom #1738

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
0392815
refactored and added tags to infinite speech streaming sample (#1605)
bradmiro Oct 7, 2019
0078c4a
Changed 'main' region tag
bradmiro Oct 10, 2019
a69558d
Removed extra lines around tags and changed client import to v1
bradmiro Oct 15, 2019
6a68e0a
Merge branch 'master' of https://github.com/GoogleCloudPlatform/java-…
bradmiro Nov 13, 2019
69eeae6
Create dataproc directory and add CreateCluster sample
bradmiro Nov 14, 2019
de1aa3d
reverting changes to speech infinite streaming sample
bradmiro Nov 14, 2019
73f3116
Added java versions to pom
bradmiro Nov 14, 2019
6cfccf6
Several changes to file formatting as per request in the PR
bradmiro Nov 15, 2019
1421f57
Added comments to exceptions in CreateCluster, expanded exceptions a…
bradmiro Nov 15, 2019
30f4b42
Fixed version for parent config
bradmiro Nov 15, 2019
c4b1d82
Added clarity to futures requests by expanding variables
bradmiro Nov 15, 2019
85aff43
Fixed linting errors
bradmiro Nov 15, 2019
f65765e
Fixed import ordering
bradmiro Nov 15, 2019
6beb303
Moved exceptions to function level in dataproc create cluster sample …
bradmiro Nov 15, 2019
b7c68c8
Re-added endpoint to test, changed region tags to include 'dataproc',…
bradmiro Nov 15, 2019
a97fc87
fix to pom
bradmiro Nov 15, 2019
9be4744
Merge branch 'master' into dataproc-samples
bradmiro Nov 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions dataproc/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
Expand Down
9 changes: 4 additions & 5 deletions dataproc/src/main/java/CreateCluster.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
* limitations under the License.
*/

// [START create_cluster]
// [START dataproc_create_cluster]
import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.dataproc.v1.Cluster;
import com.google.cloud.dataproc.v1.ClusterConfig;
Expand All @@ -23,7 +23,6 @@
import com.google.cloud.dataproc.v1.ClusterOperationMetadata;
import com.google.cloud.dataproc.v1.InstanceGroupConfig;
import java.io.IOException;
import java.io.InterruptedIOException;
import java.util.concurrent.ExecutionException;

public class CreateCluster {
Expand Down Expand Up @@ -60,12 +59,12 @@ public static void createCluster(String projectId, String region, String cluster
.setConfig(clusterConfig)
.build();

// Send a request to create a Dataproc cluster.
// Create the Cloud Dataproc cluster
OperationFuture<Cluster, ClusterOperationMetadata> createClusterAsyncRequest =
clusterControllerClient.createClusterAsync(projectId, region, cluster);
Cluster response = createClusterAsyncRequest.get();

// Print out the response
// Print out a success message
System.out.println(
String.format("Cluster created successfully: %s", response.getClusterName())
);
Expand All @@ -81,4 +80,4 @@ public static void createCluster(String projectId, String region, String cluster
}
}
}
// [END create_cluster]
// [END dataproc_create_cluster]
8 changes: 7 additions & 1 deletion dataproc/src/test/java/CreateClusterTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

import com.google.api.gax.longrunning.OperationFuture;
import com.google.cloud.dataproc.v1.ClusterControllerClient;
import com.google.cloud.dataproc.v1.ClusterControllerSettings;
import com.google.cloud.dataproc.v1.ClusterOperationMetadata;
import com.google.protobuf.Empty;
import java.io.ByteArrayOutputStream;
Expand Down Expand Up @@ -75,8 +76,13 @@ public void createClusterTest() throws IOException, InterruptedException {

@After
public void tearDown() throws IOException, InterruptedException {
String myEndpoint = String.format("%s-dataproc.googleapis.com:443", REGION);
Copy link
Contributor

@kurtisvg kurtisvg Nov 15, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to file an issue on the client library for this?

The region is already passed in the deleteClusterAsyncRequest, and it seems unncessary to force the user to create a new regional client just to delete a cluster.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree and this is consistent across all of the languages. I have this documented on my end.


ClusterControllerSettings clusterControllerSettings =
ClusterControllerSettings.newBuilder().setEndpoint(myEndpoint).build();

try (ClusterControllerClient clusterControllerClient = ClusterControllerClient
.create()) {
.create(clusterControllerSettings)) {
OperationFuture<Empty, ClusterOperationMetadata> deleteClusterAsyncRequest =
clusterControllerClient.deleteClusterAsync(projectId, REGION, clusterName);
deleteClusterAsyncRequest.get();
Expand Down