Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add compressed=True support to TextProperty #277

Closed
snarfed opened this issue Dec 14, 2019 · 2 comments · Fixed by #342
Closed

add compressed=True support to TextProperty #277

snarfed opened this issue Dec 14, 2019 · 2 comments · Fixed by #342
Assignees
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@snarfed
Copy link

snarfed commented Dec 14, 2019

hi all! i recently hit an issue migrating an app from google.appengine.ext.ndb with a TextProperty(compressed=True), which isn't supported in google.cloud.ndb. (i'm on 0.2.0, on the Python 3 Standard runtime.)

i saw #160 and the fix in #181, particularly "we can still support compression transparently with bidirectional compatibility," and optimistically tried just removing compressed=True, but no luck. the value comes back as the raw zlib-compressed bytes, eg b'x\x9c\xed\\ks\xdb...'.

i ended up adding support myself with this custom property class:

class CompressedTextProperty(ndb.GenericProperty):
  def _to_base_type(self, value):
    if isinstance(value, str):
      return zlib.compress(value.encode())

  def _from_base_type(self, value):
    if isinstance(value, bytes):
      return zlib.decompress(value).decode()

it'd be nice if google.cloud.ndb supported this natively!

@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Dec 14, 2019
snarfed added a commit to snarfed/bridgy that referenced this issue Dec 15, 2019
…_json

google.cloud.ndb doesn't support TextProperty(compressed=True) yet, either explicitly or implicitly, but Source.seen_responses_cache_json is one, so we need to support its existing values in the datastore. also see googleapis/python-ndb#277
@cguardia cguardia added priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed triage me I really want to be triaged. labels Dec 18, 2019
@cguardia
Copy link
Contributor

@snarfed Thanks, we'll see what we can do.

snarfed added a commit to snarfed/bridgy that referenced this issue Dec 20, 2019
…_json

google.cloud.ndb doesn't support TextProperty(compressed=True) yet, either explicitly or implicitly, but Source.seen_responses_cache_json is one, so we need to support its existing values in the datastore. also see googleapis/python-ndb#277
@chrisrossi chrisrossi self-assigned this Feb 20, 2020
@chrisrossi chrisrossi added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Feb 20, 2020
@chrisrossi
Copy link
Contributor

Changing this to "bug", because it is a regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants