Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switching datastore cursors to URLsafe base64 encoding. #1180

Merged
merged 1 commit into from
Oct 14, 2015

Conversation

theacodes
Copy link
Contributor

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Oct 14, 2015
@tseaver
Copy link
Contributor

tseaver commented Oct 14, 2015

Given that our code only sends the cursor as a member of a protobuf passed as the body of a POST request (with Content-Type: application/x-protobuf), I don't know why the urlsafe_ encoding variants should matter: the cursor never appears as an item in a URL path or query string, nor even in a body encoded via www-form-urlencoded/multipart-formdata. Do we know whether the back-end uses the urlsafe_ variant to encode the cursor?

@jonparrott Were you able to reproduce GoogleCloudPlatform/getting-started-python#8 before this patch, and then have it not fail with the patch?

@dhermes
Copy link
Contributor

dhermes commented Oct 14, 2015

I don't know why the urlsafe_ encoding variants should matter

For users creating URLs to page through search results with the cursors


Do we know whether the back-end uses the urlsafe_ variant to encode the cursor?

The backend uses bytes. The only reason we encode bytes as base64 is for our end-users and we decode it when bringing it back in to a protobuf (e.g. if the user gets a base64 encoded cursor from a next_page() and then they use that as an end cursor).

@theacodes
Copy link
Contributor Author

Inside of gcloud-python, the cursor is never passed around in a URL or query string, but in user applications it's a common pattern:

def list():
    cursor = request.params.get('cursor', None)
    query = ds.query(kind='Book', order=['title'])
    it = query.fetch(limit=limit, start_cursor=cursor)
    entities, more_results, cursor = it.next_page()
    return "Next page: {}".format(url_for('list', cursor=cursor))

(abridged)

Returns something like:

/list?cursor=CkUKEwoFdGl0bGUSChoIdGhlb2RvcmUSKmoVc35iaWd0YWJsZS1zZmFuLXRlc3QxchELEgRCb29rGICAgIC+goMKDBgAIAA=

which fails due to the + present.

tseaver added a commit that referenced this pull request Oct 14, 2015
Switching datastore cursors to URLsafe base64 encoding.
@tseaver tseaver merged commit c148adf into googleapis:master Oct 14, 2015
@theacodes
Copy link
Contributor Author

Thanks, @tseaver. I'll be on the lookout for the next release so I can update getting-started-python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants