Re-vendor / cleanup apitools as used in storage

[tseaver]

Work plan:

• Pull in the subset we use as a non-temporary fork (e..g, gcloud._apitools). #1186: Revendor apitools as 'gcloud._apitools' #1189
• Get it to 100% unit test coverage using our normal testing style. Coverage for 'gcloud._apitools.util' #1190, Coverage for 'gcloud._apitools.exceptions' #1191, Coverage for 'gcloud._apitools.exceptions' #1191, Coverage for 'gcloud._apitools.buffered_stream' #1192, Coverage for 'gcloud._apitools.stream_slice' #1193, Coverage for 'gcloud._apitools.http_wrapper' #1196, Add coverage for 'gcloud._apitools.transfer' #1207
• Rename it gcloud.streaming so nose finds its tests. Rename 'gcloud/_apitools' -> 'gcloud/streaming' #1208
• Remove unused bits. Streaming: strip unused features #1209
• Make it fit our coding standards. Streaming: add pep8 / pylint goodness #1210

[dhermes]

IMO this needs to go in core. As a start you could pull the multi-part message stuff into core?

[tseaver]

IMO this needs to go in core.

ATM, I'm pulling the code into a gcloud/_apitools sub-package: this will include the transfer and http_wrapper modules we use directly, plus those they pull in (util, exceptions, buffered_stream, and stream_slice).

I have these modules imported on a branch, and am adding from-scratch test coverage to them one by one. Once coverage is complete, I intend to strip deadwood (any APIs we don't use), and then make the modules conform to our standards.

As a start you could pull the multi-part message stuff into core?

I'm not sure which multi-part stuff you mean: is it the bits currently in gcloud.storage.batch?

[dhermes]

I like the approach.

---

is it the bits currently in gcloud.storage.batch?

Yup

[craigcitro]

FWIW, apitools is getting more external love via the cloudsdk folks. in particular, they cleaned up the google-apputils and gflags dependencies.

given that, it might be worth using? (i don't remember how much of apitools you use.)

[tseaver]

@craigcitro we basically use only the transfer.Download and transfer.Upload classes.

[craigcitro]

k. forking may well be the better road.

[dhermes]

@craigcitro We didn't have time to wait and assumed it would be getting left behind. To me, the fact that there are still no docs does not scream "use this, it's cared for".

[dhermes]

@tseaver Where does this stand?

[tseaver]

We're done, I think.