merge master into pubsub-hp (#1591)

* Update version to 0.8.1-SNAPSHOT (#1467)

Also, update versions in README to 0.8.0

* Add link to Maven Central for maven-central badge. (#1468)

Used to link to the image, which wasn't super useful.

* fix more races in pubsub tests

Previously BlockingProcessStreamReader has a terminate() method,
used to tell the Reader to stop reading from the emulator process.

This causes an inter-process race.
If the Reader stops before reading emulator's output,
the emulator process will hang as it tries to write to stdout/stderr
as there's no one to read from the other side of the pipe.

Since there is no way to safely stop the Reader,
this commit deletes the method and its associated test.

Additionally, the timeout for LocalSystemTest is increased to 3 minutes,
since the emulator, somehow, consistently takes just longer than a
minute to shut down.

* Regenerating SPI layer (#1501)

* Converting Error Reporting and Monitoring to use resource name types
* Removing formatX/parseX methods from pubsub, converting usage of the
  same to resource name types
* New methods in Logging and PubSub

* Updating grpc dependency to 1.0.3 (#1504)

* Release 0.8.1 (#1512)

* Fix code snippet (wrong method name) in README.md

Original code snippet in _"Querying data"_ section: 

`..while (!queryResponse.jobComplete()) {..`

This results in a compile error: 

_"Cannot resolve method jobComplete()"_

The correct method is `jobCompleted()`

* Updating version in README files. [ci skip]

* Update version to 0.8.2-alpha-SNAPSHOT

* Allow path in URIs passed to newFileSystem (#1470)

* This makes it easier for users who start with a URI describing a full
path to get a FileSystem that can work with that path, since they no
longer have to needlessly remove the path from the URI.

Note that Oracle's description of newFileSystem [1] puts no restriction
on the passed URI.

[1]
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map)

* Preventing logging re-entrance at FINE level (#1523)

* Preventing logging re-entrance at FINE level

Also:
* Reducing the scope of synchronized blocks
* Removing logger exclusions except for Http2FrameLogger

* Add a PathMatcher for CloudStorageFileSystem (#1469)

Add a test, as well.
We reuse the default PathMatcher since it does a fine job of globbing files.

* Set timestamp from LogRecord (#1533)

* Initialize the default MonitoredResource from a GAE environment (#1535)

* BigQuery: Add support to FormatOptions for AVRO

#1441

Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.

* Reverting changed commited by mistake before review.

* BigQuery: Add support to FormatOptions for AVRO

#1441

Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.

* use RpcFuture and remove old BundlingSettings (#1572)

* use RpcFuture and remove old BundlingSettings

* Release 0.8.2

Note: This version was accidentally released to Sonatype because of
experimenting with deployment commands, combined with the fact that
autoReleaseAfterClose is set to true. Since releases can't be taken
back, we might as well own up to the release and push the code
forward.

* Updating version in README files. [ci skip]

* Fixing javadoc error in GaeFlexLoggingEnhancer (#1582)

* get tests to compile and pass

README.md

@@ -43,16 +43,16 @@ If you are using Maven, add this to your pom.xml file
 <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud</artifactId>
-  <version>0.8.0</version>
+  <version>0.8.2-alpha</version>
 </dependency>
 ```
 If you are using Gradle, add this to your dependencies
 ```Groovy
-compile 'com.google.cloud:google-cloud:0.8.0'
+compile 'com.google.cloud:google-cloud:0.8.2-alpha'
 ```
 If you are using SBT, add this to your dependencies
 ```Scala
-libraryDependencies += "com.google.cloud" % "google-cloud" % "0.8.0"
+libraryDependencies += "com.google.cloud" % "google-cloud" % "0.8.2-alpha"
 ```
 
 Example Applications

google-cloud-bigquery/README.md

@@ -22,16 +22,16 @@ If you are using Maven, add this to your pom.xml file
 <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud-bigquery</artifactId>
-  <version>0.8.0-beta</version>
+  <version>0.8.2-beta</version>
 </dependency>
 ```
 If you are using Gradle, add this to your dependencies
 ```Groovy
-compile 'com.google.cloud:google-cloud-bigquery:0.8.0-beta'
+compile 'com.google.cloud:google-cloud-bigquery:0.8.2-beta'
 ```
 If you are using SBT, add this to your dependencies
 ```Scala
-libraryDependencies += "com.google.cloud" % "google-cloud-bigquery" % "0.8.0-beta"
+libraryDependencies += "com.google.cloud" % "google-cloud-bigquery" % "0.8.2-beta"
 ```
 
 Example Application
@@ -189,7 +189,7 @@ QueryRequest queryRequest =
         .build();
 // Request query to be executed and wait for results
 QueryResponse queryResponse = bigquery.query(queryRequest);
-while (!queryResponse.jobComplete()) {
+while (!queryResponse.jobCompleted()) {
   Thread.sleep(1000L);
   queryResponse = bigquery.getQueryResults(queryResponse.getJobId());
 }

google-cloud-bigquery/pom.xml

@@ -12,7 +12,7 @@
   <parent>
     <groupId>com.google.cloud</groupId>
     <artifactId>google-cloud-pom</artifactId>
-    <version>0.8.1-SNAPSHOT</version>
+    <version>0.8.2-alpha</version>
   </parent>
   <properties>
     <site.installationModule>google-cloud-bigquery</site.installationModule>

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/FormatOptions.java

@@ -32,6 +32,7 @@ public class FormatOptions implements Serializable {
   static final String CSV = "CSV";
   static final String JSON = "NEWLINE_DELIMITED_JSON";
   static final String DATASTORE_BACKUP = "DATASTORE_BACKUP";
+  static final String AVRO = "AVRO";
   private static final long serialVersionUID = -443376052020423691L;
 
   private final String type;
@@ -94,6 +95,13 @@ public static FormatOptions datastoreBackup() {
     return new FormatOptions(DATASTORE_BACKUP);
   }
 
+  /**
+   * Default options for AVRO format.
+   */
+  public static FormatOptions avro() {
+    return new FormatOptions(AVRO);
+  }
+
   /**
    * Default options for the provided format.
    */

google-cloud-bigquery/src/test/java/com/google/cloud/bigquery/FormatOptionsTest.java

@@ -30,23 +30,29 @@ public void testConstructor() {
     assertEquals(FormatOptions.JSON, options.getType());
     options = new FormatOptions(FormatOptions.DATASTORE_BACKUP);
     assertEquals(FormatOptions.DATASTORE_BACKUP, options.getType());
+    options = new FormatOptions(FormatOptions.AVRO);
+    assertEquals(FormatOptions.AVRO, options.getType());
   }
 
   @Test
+  @SuppressWarnings("deprecation")
   public void testConstructorDeprecated() {
     FormatOptions options = new FormatOptions(FormatOptions.CSV);
     assertEquals(FormatOptions.CSV, options.type());
     options = new FormatOptions(FormatOptions.JSON);
     assertEquals(FormatOptions.JSON, options.type());
     options = new FormatOptions(FormatOptions.DATASTORE_BACKUP);
     assertEquals(FormatOptions.DATASTORE_BACKUP, options.type());
+    options = new FormatOptions(FormatOptions.AVRO);
+    assertEquals(FormatOptions.AVRO, options.type());
   }
 
   @Test
   public void testFactoryMethods() {
     assertEquals(FormatOptions.CSV, FormatOptions.csv().getType());
     assertEquals(FormatOptions.JSON, FormatOptions.json().getType());
     assertEquals(FormatOptions.DATASTORE_BACKUP, FormatOptions.datastoreBackup().getType());
+    assertEquals(FormatOptions.AVRO, FormatOptions.avro().getType());
   }
 
   @Test

google-cloud-compute/README.md

@@ -22,16 +22,16 @@ If you are using Maven, add this to your pom.xml file
 <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud-compute</artifactId>
-  <version>0.8.0</version>
+  <version>0.8.2-alpha</version>
 </dependency>
 ```
 If you are using Gradle, add this to your dependencies
 ```Groovy
-compile 'com.google.cloud:google-cloud-compute:0.8.0'
+compile 'com.google.cloud:google-cloud-compute:0.8.2-alpha'
 ```
 If you are using SBT, add this to your dependencies
 ```Scala
-libraryDependencies += "com.google.cloud" % "google-cloud-compute" % "0.8.0"
+libraryDependencies += "com.google.cloud" % "google-cloud-compute" % "0.8.2-alpha"
 ```
 
 Example Application

google-cloud-compute/pom.xml

@@ -11,7 +11,7 @@
   <parent>
     <groupId>com.google.cloud</groupId>
     <artifactId>google-cloud-pom</artifactId>
-    <version>0.8.1-SNAPSHOT</version>
+    <version>0.8.2-alpha</version>
   </parent>
   <properties>
     <site.installationModule>google-cloud-compute</site.installationModule>

google-cloud-contrib/README.md

@@ -25,16 +25,16 @@ If you are using Maven, add this to your pom.xml file
 <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud-contrib</artifactId>
-  <version>0.8.0</version>
+  <version>0.8.2-alpha</version>
 </dependency>
 ```
 If you are using Gradle, add this to your dependencies
 ```Groovy
-compile 'com.google.cloud:google-cloud-contrib:0.8.0'
+compile 'com.google.cloud:google-cloud-contrib:0.8.2-alpha'
 ```
 If you are using SBT, add this to your dependencies
 ```Scala
-libraryDependencies += "com.google.cloud" % "google-cloud-contrib" % "0.8.0"
+libraryDependencies += "com.google.cloud" % "google-cloud-contrib" % "0.8.2-alpha"
 ```
 
 ### google-cloud-nio-examples

google-cloud-contrib/google-cloud-nio-examples/README.md

@@ -22,12 +22,12 @@ To run this example:
 4.    Run the sample with:
 
     ```
-    java -cp google-cloud-contrib/google-cloud-nio/target/google-cloud-nio-0.7.1-SNAPSHOT-shaded.jar:google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.7.1-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
+    java -cp google-cloud-contrib/google-cloud-nio/target/google-cloud-nio-0.8.3-alpha-SNAPSHOT-shaded.jar:google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.8.3-alpha-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
     ```
 
     Notice that it lists Google Cloud Storage, which it wouldn't if you ran it without the NIO jar:
     ```
-    java -cp google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.7.1-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
+    java -cp google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.8.3-alpha-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
     ```
 
 The sample doesn't have anything about Google Cloud Storage in it. It gets that ability from the NIO

google-cloud-contrib/google-cloud-nio-examples/pom.xml

@@ -11,7 +11,7 @@
   <parent>
     <groupId>com.google.cloud</groupId>
     <artifactId>google-cloud-contrib</artifactId>
-    <version>0.8.1-SNAPSHOT</version>
+    <version>0.8.2-alpha</version>
   </parent>
   <properties>
     <site.installationModule>google-cloud-nio-examples</site.installationModule>

google-cloud-contrib/google-cloud-nio/README.md

@@ -26,16 +26,16 @@ If you are using Maven, add this to your pom.xml file
 <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud-nio</artifactId>
-  <version>0.8.0</version>
+  <version>0.8.2-alpha</version>
 </dependency>
 ```
 If you are using Gradle, add this to your dependencies
 ```Groovy
-compile 'com.google.cloud:google-cloud-nio:0.8.0'
+compile 'com.google.cloud:google-cloud-nio:0.8.2-alpha'
 ```
 If you are using SBT, add this to your dependencies
 ```Scala
-libraryDependencies += "com.google.cloud" % "google-cloud-nio" % "0.8.0"
+libraryDependencies += "com.google.cloud" % "google-cloud-nio" % "0.8.2-alpha"
 ```
 
 Example Applications

google-cloud-contrib/google-cloud-nio/pom.xml

@@ -11,7 +11,7 @@
   <parent>
     <groupId>com.google.cloud</groupId>
     <artifactId>google-cloud-contrib</artifactId>
-    <version>0.8.1-SNAPSHOT</version>
+    <version>0.8.2-alpha</version>
   </parent>
   <properties>
     <site.installationModule>google-cloud-nio</site.installationModule>

google-cloud-contrib/google-cloud-nio/src/main/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystem.java

@@ -27,6 +27,7 @@
 import java.net.URISyntaxException;
 import java.nio.file.FileStore;
 import java.nio.file.FileSystem;
+import java.nio.file.FileSystems;
 import java.nio.file.Path;
 import java.nio.file.PathMatcher;
 import java.nio.file.WatchService;
@@ -210,13 +211,9 @@ public Set<String> supportedFileAttributeViews() {
     return SUPPORTED_VIEWS;
   }
 
-  /**
-   * Throws {@link UnsupportedOperationException} because this feature hasn't been implemented yet.
-   */
   @Override
   public PathMatcher getPathMatcher(String syntaxAndPattern) {
-    // TODO(#813): Implement me.
-    throw new UnsupportedOperationException();
+    return FileSystems.getDefault().getPathMatcher(syntaxAndPattern);
   }
 
   /**

google-cloud-contrib/google-cloud-nio/src/main/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemProvider.java

@@ -170,14 +170,15 @@ public CloudStorageFileSystem getFileSystem(URI uri) {
   }
 
   /**
-   * Returns Cloud Storage file system, provided a URI with no path, e.g. {@code gs://bucket}.
+   * Returns Cloud Storage file system, provided a URI, e.g. {@code gs://bucket}.
+   * The URI can include a path component (that will be ignored).
    *
    * @param uri bucket and current working directory, e.g. {@code gs://bucket}
    * @param env map of configuration options, whose keys correspond to the method names of
    *     {@link CloudStorageConfiguration.Builder}. However you are not allowed to set the working
    *     directory, as that should be provided in the {@code uri}
-   * @throws IllegalArgumentException if {@code uri} specifies a user, query, fragment, or scheme is
-   *     not {@value CloudStorageFileSystem#URI_SCHEME}
+   * @throws IllegalArgumentException if {@code uri} specifies a port, user, query, or fragment, or
+   *     if scheme is not {@value CloudStorageFileSystem#URI_SCHEME}
    */
   @Override
   public CloudStorageFileSystem newFileSystem(URI uri, Map<String, ?> env) {
@@ -191,11 +192,10 @@ public CloudStorageFileSystem newFileSystem(URI uri, Map<String, ?> env) {
         CloudStorageFileSystem.URI_SCHEME, uri);
     checkArgument(
         uri.getPort() == -1
-            && isNullOrEmpty(uri.getPath())
             && isNullOrEmpty(uri.getQuery())
             && isNullOrEmpty(uri.getFragment())
             && isNullOrEmpty(uri.getUserInfo()),
-        "GCS FileSystem URIs mustn't have: port, userinfo, path, query, or fragment: %s",
+        "GCS FileSystem URIs mustn't have: port, userinfo, query, or fragment: %s",
         uri);
     CloudStorageUtil.checkBucket(uri.getHost());
     initStorage();

google-cloud-contrib/google-cloud-nio/src/test/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemProviderTest.java

@@ -54,7 +54,9 @@
 import java.nio.file.OpenOption;
 import java.nio.file.Path;
 import java.nio.file.Paths;
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 
 /**
  * Unit tests for {@link CloudStorageFileSystemProvider}.
@@ -644,6 +646,12 @@ public void testProviderEquals() {
     assertThat(path1.getFileSystem().provider()).isNotEqualTo(path3.getFileSystem().provider());
   }
 
+  @Test
+  public void testNewFileSystem() throws IOException {
+    Map<String,String> env = new HashMap<>();
+    FileSystems.newFileSystem(URI.create("gs://bucket/path/to/file"), env);
+  }
+
   private static CloudStorageConfiguration permitEmptyPathComponents(boolean value) {
     return CloudStorageConfiguration.builder().permitEmptyPathComponents(value).build();
   }

google-cloud-contrib/google-cloud-nio/src/test/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemTest.java

@@ -34,6 +34,7 @@
 import java.nio.file.FileSystems;
 import java.nio.file.Files;
 import java.nio.file.Path;
+import java.nio.file.PathMatcher;
 import java.nio.file.Paths;
 import java.util.ArrayList;
 import java.util.List;
@@ -159,4 +160,27 @@ public void testListFiles() throws IOException {
       assertThat(got).containsExactlyElementsIn(goodPaths);
     }
   }
+
+  @Test
+  public void testMatcher() throws IOException {
+    try (FileSystem fs = CloudStorageFileSystem.forBucket("bucket")) {
+      String pattern1 = "glob:*.java";
+      PathMatcher javaFileMatcher = fs.getPathMatcher(pattern1);
+      assertMatches(fs, javaFileMatcher, "a.java", true);
+      assertMatches(fs, javaFileMatcher, "a.text", false);
+      assertMatches(fs, javaFileMatcher, "folder/c.java", true);
+      assertMatches(fs, javaFileMatcher, "d", false);
+
+      String pattern2 = "glob:*.{java,text}";
+      PathMatcher javaAndTextFileMatcher = fs.getPathMatcher(pattern2);
+      assertMatches(fs, javaAndTextFileMatcher, "a.java", true);
+      assertMatches(fs, javaAndTextFileMatcher, "a.text", true);
+      assertMatches(fs, javaAndTextFileMatcher, "folder/c.java", true);
+      assertMatches(fs, javaAndTextFileMatcher, "d", false);
+    }
+  }
+
+  private void assertMatches(FileSystem fs, PathMatcher matcher, String toMatch, boolean expected) {
+    assertThat(matcher.matches(fs.getPath(toMatch).getFileName())).isEqualTo(expected);
+  }
 }

google-cloud-contrib/pom.xml

@@ -11,7 +11,7 @@
   <parent>
     <groupId>com.google.cloud</groupId>
     <artifactId>google-cloud-pom</artifactId>
-    <version>0.8.1-SNAPSHOT</version>
+    <version>0.8.2-alpha</version>
   </parent>
   <properties>
     <site.installationModule>google-cloud-contrib</site.installationModule>

google-cloud-core/README.md

@@ -19,16 +19,16 @@ If you are using Maven, add this to your pom.xml file
 <dependency>
   <groupId>com.google.cloud</groupId>
   <artifactId>google-cloud-core</artifactId>
-  <version>0.8.0</version>
+  <version>0.8.2-alpha</version>
 </dependency>
 ```
 If you are using Gradle, add this to your dependencies
 ```Groovy
-compile 'com.google.cloud:google-cloud-core:0.8.0'
+compile 'com.google.cloud:google-cloud-core:0.8.2-alpha'
 ```
 If you are using SBT, add this to your dependencies
 ```Scala
-libraryDependencies += "com.google.cloud" % "google-cloud-core" % "0.8.0"
+libraryDependencies += "com.google.cloud" % "google-cloud-core" % "0.8.2-alpha"
 ```
 
 Troubleshooting

google-cloud-core/pom.xml

@@ -11,7 +11,7 @@
   <parent>
     <groupId>com.google.cloud</groupId>
     <artifactId>google-cloud-pom</artifactId>
-    <version>0.8.1-SNAPSHOT</version>
+    <version>0.8.2-alpha</version>
   </parent>
   <properties>
     <site.installationModule>google-cloud-core</site.installationModule>
@@ -111,7 +111,7 @@
     <dependency>
       <groupId>com.google.api</groupId>
       <artifactId>gax</artifactId>
-      <version>0.0.27</version>
+      <version>0.0.28</version>
       <exclusions>
         <exclusion>
           <groupId>io.grpc</groupId>
@@ -122,7 +122,7 @@
     <dependency>
       <groupId>com.google.api.grpc</groupId>
       <artifactId>grpc-google-common-protos</artifactId>
-      <version>0.1.3</version>
+      <version>0.1.5</version>
       <exclusions>
         <exclusion>
           <groupId>io.grpc</groupId>
@@ -133,7 +133,7 @@
     <dependency>
       <groupId>com.google.api.grpc</groupId>
       <artifactId>grpc-google-iam-v1</artifactId>
-      <version>0.1.3</version>
+      <version>0.1.5</version>
       <exclusions>
         <exclusion>
           <groupId>io.grpc</groupId>