Update Storage and BigQuery docs with broken pipe solution

There is no fix we can make for this file upload issue, so add documentation instead. [closes #266]
googleapis · Sep 4, 2015 · 0c008d0 · 0c008d0
1 parent 1544545
commit 0c008d0
Show file tree

Hide file tree

Showing 4 changed files with 67 additions and 1 deletion.
diff --git a/lib/gcloud/bigquery.rb b/lib/gcloud/bigquery.rb
@@ -336,6 +336,22 @@ def self.bigquery project = nil, keyfile = nil, options = {}
   # default format for load operations, the option is not actually necessary.
   # For JSON saved with a +.txt+ extension, however, it would be.
   #
+  # === A note about large uploads
+  #
+  # You may encounter a broken pipe error while attempting to upload large
+  # files. To avoid this problem, add
+  # {httpclient}[https://rubygems.org/gems/httpclient] as a dependency to your
+  # project, and configure {Faraday}[https://rubygems.org/gems/faraday] to use
+  # it, after requiring Gcloud, but before initiating your Gcloud connection.
+  #
+  #   require "gcloud"
+  #
+  #   Faraday.default_adapter = :httpclient
+  #
+  #   gcloud = Gcloud.new
+  #   bigquery = gcloud.bigquery
+  #   dataset = bigquery.dataset "my_dataset"
+  #
   # == Exporting query results to Google Cloud Storage
   #
   # The example below shows how to pass the +table+ option with a query in order

diff --git a/lib/gcloud/bigquery/table.rb b/lib/gcloud/bigquery/table.rb
@@ -658,7 +658,7 @@ def extract extract_url, options = {}
       #   file = bucket.file "file-name.csv"
       #   load_job = table.load file
       #
-      # Or, you can upload a smaller file directly.
+      # Or, you can upload a file directly.
       # See {Loading Data with a POST Request}[
       # https://cloud.google.com/bigquery/loading-data-post-request#multipart].
       #
@@ -672,6 +672,23 @@ def extract extract_url, options = {}
       #   file = File.open "my_data.csv"
       #   load_job = table.load file
       #
+      # === A note about large direct uploads
+      #
+      # You may encounter a broken pipe error while attempting to upload large
+      # files. To avoid this problem, add
+      # {httpclient}[https://rubygems.org/gems/httpclient] as a dependency to
+      # your project, and configure {Faraday}[https://rubygems.org/gems/faraday]
+      # to use it, after requiring Gcloud, but before initiating your Gcloud
+      # connection.
+      #
+      #   require "gcloud"
+      #
+      #   Faraday.default_adapter = :httpclient
+      #
+      #   gcloud = Gcloud.new
+      #   bigquery = gcloud.bigquery
+      #   dataset = bigquery.dataset "my_dataset"
+      #
       # :category: Data
       #
       def load file, options = {}

diff --git a/lib/gcloud/storage.rb b/lib/gcloud/storage.rb
@@ -227,6 +227,22 @@ def self.storage project = nil, keyfile = nil, options = {}
   #   bucket.create_file "/var/todo-app/avatars/heidi/400x400.png",
   #                      "avatars/heidi/400x400.png"
   #
+  # === A note about large uploads
+  #
+  # You may encounter a broken pipe error while attempting to upload large
+  # files. To avoid this problem, add
+  # {httpclient}[https://rubygems.org/gems/httpclient] as a dependency to your
+  # project, and configure {Faraday}[https://rubygems.org/gems/faraday] to use
+  # it, after requiring Gcloud, but before initiating your Gcloud connection.
+  #
+  #   require "gcloud"
+  #
+  #   Faraday.default_adapter = :httpclient
+  #
+  #   gcloud = Gcloud.new
+  #   storage = gcloud.storage
+  #   bucket = storage.bucket "my-todo-app"
+  #
   # == Downloading a File
   #
   # Files can be downloaded to the local file system. (See File#download)

diff --git a/lib/gcloud/storage/bucket.rb b/lib/gcloud/storage/bucket.rb
@@ -324,6 +324,23 @@ def file path, options = {}
       #                      "destination/path/file.ext",
       #                      chunk_size: 1024*1024 # 1 MB chunk
       #
+      # ==== A note about large uploads
+      #
+      # You may encounter a broken pipe error while attempting to upload large
+      # files. To avoid this problem, add
+      # {httpclient}[https://rubygems.org/gems/httpclient] as a dependency to
+      # your project, and configure {Faraday}[https://rubygems.org/gems/faraday]
+      # to use it, after requiring Gcloud, but before initiating your Gcloud
+      # connection.
+      #
+      #   require "gcloud"
+      #
+      #   Faraday.default_adapter = :httpclient
+      #
+      #   gcloud = Gcloud.new
+      #   storage = gcloud.storage
+      #   bucket = storage.bucket "my-todo-app"
+      #
       def create_file file, path = nil, options = {}
         ensure_connection!
         ensure_file_exists! file