Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GAX support: need new releases of 'google-gax' and 'gax-google-pubsub-v1' #1814

Closed
tseaver opened this issue May 23, 2016 · 34 comments
Closed

GAX support: need new releases of 'google-gax' and 'gax-google-pubsub-v1' #1814

tseaver opened this issue May 23, 2016 · 34 comments
Assignees
@tseaver
Labels
api: pubsub Issues related to the Pub/Sub API. packaging

Comments

@tseaver
Copy link
Contributor

tseaver commented May 23, 2016

I'm blocked on finishing / pushing my GAX pubsub changes on new releases. See:

CC @tbetbetbe
@tseaver tseaver added packaging api: pubsub Issues related to the Pub/Sub API. labels May 23, 2016
@tbetbetbe
Copy link

Sorry - these were due last week - I could not get to them before I moved.
The new ETA is this time tomorrow.

On 24 May 2016 at 00:50, Tres Seaver notifications@github.com wrote:

I'm blocked on finishing / pushing my GAX pubsub changes on new releases.
See:

CC @tbetbetbe https://github.com/tbetbetbe

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1814

This email may be confidential or privileged. If you received this
communication by mistake, please don't forward it to anyone else (it may
contain confidential or privileged information), please erase all copies of
it, including all attachments, and please let the sender know it went to
the wrong person. Thanks.

@tseaver tseaver mentioned this issue May 23, 2016
Merged
@tseaver
Copy link
Contributor Author

tseaver commented May 25, 2016

@tbetbetbe can you update the ETA?

@tbetbetbe
Copy link

Sorry about the delay on this. As of now, this is my highest priority
task, I'll update this thread within the next day to report completion.

On 25 May 2016 at 13:18, Tres Seaver notifications@github.com wrote:

@tbetbetbe https://github.com/tbetbetbe can you update the ETA?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1814 (comment)

This email may be confidential or privileged. If you received this
communication by mistake, please don't forward it to anyone else (it may
contain confidential or privileged information), please erase all copies of
it, including all attachments, and please let the sender know it went to
the wrong person. Thanks.

@tseaver
Copy link
Contributor Author

tseaver commented May 31, 2016

@tbetbetbe Any update?

@tseaver
Copy link
Contributor Author

tseaver commented Jun 2, 2016 •
edited
Loading

Looks like we have a new release of google-gax. We still need another release of gax-google-pubsub-v1, as well as an initial release of gax-google-logging-v2.

@tseaver
Copy link
Contributor Author

tseaver commented Jun 2, 2016

Doesn't look as though the tag for gax-python 0.11.0 has been pushed to Github yet.

@tseaver
Copy link
Contributor Author

tseaver commented Jun 6, 2016

Doesn't look as though the tag for gax-python 0.11.0 has been pushed to Github yet.

I see the v0.11.0 tag now.

@tbetbetbe are releases for the (generated, I presume) gax-google-pubsub-v1 and gax-google-logging-v2 libraries pending something in the upstream? The protos haven't changed recently AFAIK. The generation is done via https://github.com/googleapis/toolkit, right?

@tbetbetbe
Copy link

That's correct.

What I'm trying to do is make sure that their
dependencies, grpc-google-pubsub-v1 and grpc-google-logging-v2 are updated
first.
These are not generated by toolkit, rather they are generated by protoc +
the grpc python plugin. That's been updated, and I want to make sure that
we use the updated version.

On 6 June 2016 at 21:22, Tres Seaver notifications@github.com wrote:

Doesn't look as though the tag for gax-python 0.11.0 has been pushed to
Github yet.

I see the v0.11.0 tag
https://github.com/googleapis/gax-python/releases/tag/v0.11.0 now.

@tbetbetbe https://github.com/tbetbetbe are releases for the
(generated, I presume) gax-google-pubsub-v1 and gax-google-logging-v2
libraries pending something in the upstream? The protos haven't changed
recently AFAIK. The generation is done via https://github.com/googleapis/
toolkit, right?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1814 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AI18Fhpm_YonsgU4K5PjTbK_LUqqTBHKks5qJHOPgaJpZM4IkoFY
.

This email may be confidential or privileged. If you received this
communication by mistake, please don't forward it to anyone else (it may
contain confidential or privileged information), please erase all copies of
it, including all attachments, and please let the sender know it went to
the wrong person. Thanks.

@tbetbetbe
Copy link

tbetbetbe commented Jun 7, 2016 •
edited
Loading

@jmuk, can you finish this off please, after submitting googleapis/packman#40 ?

I've successfully updated/published the following

Now, they all depend on grpcio-v0.14.0 and are generated with the latest grpc python plugin, which fixes issues some python package users are seeing where there are missing imports. (FYI: @jgeiger)

I think the best route would be to submit googleapis/artman#23, and use the changes it contains to use artman to create the updated grpc-gax packages.

The created packages can be submitted using the googleapis-package user name.

@jmuk
Copy link

jmuk commented Jun 7, 2016

Well, it turns out that googleapis/artman#23 is not useful by itself for now. Without googleapis/gapic-generator#126, the automatically generated whl file will have the config .json files at a wrong place.

@jmuk
Copy link

jmuk commented Jun 9, 2016

now googleapis/artman#23 is merged, FYI

@tseaver
Copy link
Contributor Author

tseaver commented Jun 10, 2016

Yay! It looks as though googleapis/gapic-generator#192 has landed as well (resolving googleapis/gapic-generator#126, AFAICT).

@bjwatson
Copy link

@jgeewax @tseaver I've pushed new releases of https://pypi.python.org/pypi/gax-google-pubsub-v1 and https://pypi.python.org/pypi/grpc-google-pubsub-v1 (and sent you an email describing the same). I back-rev'd them to 0.7.6, because this is still beta code.

I missed googleapis/gapic-generator#192 in the 0.7.6 release, but I have a bash script I've been using that works around it. Now that I know the recipe, I can easily cut 0.7.7 with that PR, so please let me know if you're missing anything.

Please close this issue if it's been resolved to your satisfaction. Thanks!

@tseaver
Copy link
Contributor Author

tseaver commented Jun 10, 2016

@bjwatson Hmm, that release has a lower version number than the 1.0.16 version, which is still installable from PyPI (even though invisible in the web UI). pip finds the older-but-higher-numbered release:

$ /opt/Python-2.7.11/bin/virtualenv /tmp/gax
New python executable in /tmp/gax/bin/python
Installing setuptools, pip, wheel...done.
 $ /tmp/gax/bin/pip install --no-cache-dir gax-google-pubsub-v1
Collecting gax-google-pubsub-v1
  Downloading gax-google-pubsub-v1-1.0.16.tar.gz
Collecting googleapis-common-protos>=1.1.0 (from gax-google-pubsub-v1)
  Downloading googleapis-common-protos-1.1.0.tar.gz
Collecting google-gax==0.9.4 (from gax-google-pubsub-v1)
  Downloading google-gax-0.9.4.tar.gz
Collecting grpc-google-pubsub-v1>=1.0.15 (from gax-google-pubsub-v1)
  Downloading grpc-google-pubsub-v1-1.0.19.tar.gz
Collecting oauth2client>=1.4.11 (from gax-google-pubsub-v1)
  Downloading oauth2client-2.1.0.tar.gz (69kB)
    100% |████████████████████████████████| 71kB 1.2MB/s 
Collecting protobuf>=3.0.0b2 (from googleapis-common-protos>=1.1.0->gax-google-pubsub-v1)
  Downloading protobuf-3.0.0b3-py2-none-any.whl (334kB)
    100% |████████████████████████████████| 337kB 2.3MB/s 
Collecting grpcio==0.13.1 (from google-gax==0.9.4->gax-google-pubsub-v1)
  Downloading grpcio-0.13.1.tar.gz (5.2MB)
    100% |████████████████████████████████| 5.2MB 2.9MB/s 
Collecting ply==3.8 (from google-gax==0.9.4->gax-google-pubsub-v1)
  Downloading ply-3.8.tar.gz (157kB)
    100% |████████████████████████████████| 163kB 3.1MB/s 
Collecting httplib2>=0.9.1 (from oauth2client>=1.4.11->gax-google-pubsub-v1)
  Downloading httplib2-0.9.2.zip (210kB)
    100% |████████████████████████████████| 215kB 3.2MB/s 
Collecting pyasn1>=0.1.7 (from oauth2client>=1.4.11->gax-google-pubsub-v1)
  Downloading pyasn1-0.1.9-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.0.5 (from oauth2client>=1.4.11->gax-google-pubsub-v1)
  Downloading pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client>=1.4.11->gax-google-pubsub-v1)
  Downloading rsa-3.4.2-py2.py3-none-any.whl (46kB)
    100% |████████████████████████████████| 51kB 7.4MB/s 
Collecting six>=1.6.1 (from oauth2client>=1.4.11->gax-google-pubsub-v1)
  Downloading six-1.10.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /tmp/gax/lib/python2.7/site-packages (from protobuf>=3.0.0b2->googleapis-common-protos>=1.1.0->gax-google-pubsub-v1)
Collecting enum34>=1.0.4 (from grpcio==0.13.1->google-gax==0.9.4->gax-google-pubsub-v1)
  Downloading enum34-1.1.6-py2-none-any.whl
Collecting futures>=2.2.0 (from grpcio==0.13.1->google-gax==0.9.4->gax-google-pubsub-v1)
  Downloading futures-3.0.5-py2-none-any.whl
Installing collected packages: six, protobuf, googleapis-common-protos, enum34, futures, grpcio, ply, httplib2, pyasn1, pyasn1-modules, rsa, oauth2client, google-gax, grpc-google-pubsub-v1, gax-google-pubsub-v1
  Running setup.py install for googleapis-common-protos ... done
  Running setup.py install for grpcio ... done
  Running setup.py install for ply ... done
  Running setup.py install for httplib2 ... done
  Running setup.py install for oauth2client ... done
  Running setup.py install for google-gax ... done
  Running setup.py install for grpc-google-pubsub-v1 ... done
  Running setup.py install for gax-google-pubsub-v1 ... done
Successfully installed enum34-1.1.6 futures-3.0.5 gax-google-pubsub-v1-1.0.16 google-gax-0.9.4 googleapis-common-protos-1.1.0 grpc-google-pubsub-v1-1.0.19 grpcio-0.13.1 httplib2-0.9.2 oauth2client-2.1.0 ply-3.8 protobuf-3.0.0b3 pyasn1-0.1.9 pyasn1-modules-0.0.8 rsa-3.4.2 six-1.10.0

@tseaver
Copy link
Contributor Author

tseaver commented Jun 10, 2016

Likewise, https://pypi.python.org/pypi/grpc-google-pubsub-v1/1.0.19 is still present, and preferred by pip (although it looks recent enough to be correct).

@bjwatson
Copy link

@tseaver Could you try the following as a workaround:

pip install "grpc-google-pubsub-v1<1.0.0"
pip install "gax-google-pubsub-v1<1.0.0"

I'm running into some dependency errors on my workstation, but they might not be related (this is the first time I've directly tried the virtualenv style of installing these packages).

@tseaver
Copy link
Contributor Author

tseaver commented Jun 10, 2016

That does get the 0.7.6 versions installed.

@bjwatson
Copy link

Cool. We'll get a firm decision on the version number, and then make any necessary updates.

@tseaver
Copy link
Contributor Author

tseaver commented Jun 10, 2016

One note: if you decide to yank the 1.x.y releases from PyPI, those release numbers will be forever poisoned, as PyPI will not permit re-uploading a distribution that has been removed.

@bjwatson
Copy link

Agreed. I will track that we need to go to 1.0.20 or 1.1.0 when we finally do a GA release.

I have deleted all 1.x.y versions; 0.7.6 is the only version now.

I did not have time to work on the logging API today. I will try to get that done on Monday. Hopefully this at least half-unblocks you.

@tseaver
Copy link
Contributor Author

tseaver commented Jun 13, 2016

@bjwatson the 0.7.6 release is a bit borked: it declares namespace packages which don't belong to it (gcloud, gcloud.pubsub), and writes files there which don't belong to gcloud-python. All code for the GAX wrappers should be located in `google.pubsub.v1' Can you please fix that and release a 0.7.7?

@tseaver tseaver mentioned this issue Jun 13, 2016
Merged
@bjwatson
Copy link

@tseaver Sure. I knew this changed before I joined, but didn't realize the context behind it. I'll immediately release a 0.7.7 as you requested to unblock you.

@bjwatson
Copy link

Done: https://pypi.python.org/pypi/gax-google-pubsub-v1

@bjwatson
Copy link

FYI, I've also updated the docs: http://pythonhosted.org/gax-google-pubsub-v1/index.html

@tseaver
Copy link
Contributor Author

tseaver commented Jun 13, 2016

Thanks!

@tseaver
Copy link
Contributor Author

tseaver commented Jun 15, 2016

@bjwatson I know you're on vacation until Friday, but I'm hoping to see the gax-google-logging-v2 release soonish.

@jmuk
Copy link

jmuk commented Jun 15, 2016

@geigerj might be able to help this?

@bjwatson
Copy link

bjwatson commented Jun 16, 2016 •
edited
Loading

@tseaver I'm actually in an all-day class, rather than on vacation, so I have a little time to spend in the evenings.

I just published version 0.7.8 of logging:

Please let me know if there are any issues. Something I got a little stuck on is differences in how the toolchain on my workstation generates the grpc package vs. how it was generated for previous versions of pubsub. I didn't have to deal with this for pubsub, since I was basically re-publishing 1.0.19 as 0.7.x, but I did have to handle this for logging which has not previously been published.

I will also review the documentation more closely tomorrow, after I'm done with my class.

FYI @geigerj and @Temiola.

@daspecster daspecster mentioned this issue Jun 16, 2016
Closed
@bjwatson
Copy link

@tseaver @daspecster Are you waiting on anything from me for resolving this issue? Please let me know if there's anything I need to fix in the generated grpc-google-* or gax-google-* packages.

@daspecster
Copy link
Contributor

@bjwatson unless this is somehow related to #1863, I think it's resolved.

WDYT @tseaver?

@tseaver
Copy link
Contributor Author

tseaver commented Jun 17, 2016

I haven't actually tried to land the code yet which depends on gax-google-logging-v2, but the tarball on PyPI looks correct (it puts the generated files into google.logging.v2, rather than gcloud.logging).

@tseaver tseaver closed this as completed Jun 17, 2016
@bjwatson
Copy link

Thanks.

@tseaver
Copy link
Contributor Author

tseaver commented Jun 20, 2016

@bjwatson Where should I be reporting issues with gax-google-pubsub-v1?

@bjwatson
Copy link

Good question. I would suggest raising them as issues here, and tagging @tbetbetbe and I. The root cause for a pubsub issue could be in any one of the vKit repos (or several of them).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tseaver tseaver

Labels
api: pubsub Issues related to the Pub/Sub API. packaging
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@daspecster @tseaver @bjwatson @jmuk @tbetbetbe