Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NDB: non_transactional tasklets do not respect the non_transactional context #552

Closed
ahazeemi opened this issue Oct 1, 2020 · 1 comment · Fixed by #554
Closed

NDB: non_transactional tasklets do not respect the non_transactional context #552

ahazeemi opened this issue Oct 1, 2020 · 1 comment · Fixed by #554
Assignees
Labels
api: datastore Issues related to the googleapis/python-ndb API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@ahazeemi
Copy link

ahazeemi commented Oct 1, 2020

Environment details

  1. OS type and version: MacOS 10.15.7
  2. Python version and virtual environment information: 2.7
  3. google-cloud-ndb version: 1.4.2

Steps to reproduce

  1. Create a tasklet.
  2. Add ndb.non_transactional decorator on it.
  3. Perform a fetch operation on more than 25 entity groups
  4. The error 400 operating on too many entity groups in a single transaction shows up.

Also reproducible on AppEngine.

Code example

@ndb.tasklet
@ndb.non_transactional()
def non_transactional_tasklet(keys):
      result = yield ndb.get_multi_async(keys).  # keys belong to 25 different entity groups
      return result

Stack trace

File "/Users/abdulhameedazeemi/Desktop/project/packages/backend/thirdparty/google/cloud/ndb/_transaction.py", line 215, in transactional_inner_wrapper
    propagation=propagation,
  File "/Users/abdulhameedazeemi/Desktop/project/packages/backend/thirdparty/google/cloud/ndb/_transaction.py", line 72, in transaction
    return future.result()
  File "/Users/abdulhameedazeemi/Desktop/project/packages/backend/thirdparty/google/cloud/ndb/tasklets.py", line 193, in result
    self.check_success()
  File "/Users/abdulhameedazeemi/Desktop/project/packages/backend/thirdparty/google/cloud/ndb/tasklets.py", line 140, in check_success
    raise self._exception
InvalidArgument: 400 operating on too many entity groups in a single transaction.
@product-auto-label product-auto-label bot added the api: datastore Issues related to the googleapis/python-ndb API. label Oct 1, 2020
@chrisrossi chrisrossi self-assigned this Oct 1, 2020
@ShahzadUmair
Copy link

ShahzadUmair commented Oct 1, 2020

None of the following works:

# swapping ❌

@ndb.non_transactional()
@ndb.tasklet
def non_transactional_tasklet(keys):
      result = yield ndb.get_multi_async(keys).  # keys belong to 25 different entity groups
      return result
# inline ❌

@ndb.tasklet()
def non_transactional_tasklet(keys):
      result = yield ndb.non_transactional()(lambda: ndb.get_multi_async(keys))()
      return result

Though it seems to work fine with fetch_page_async. (must be some issue with get_multi_async, I believe)

# ✅

@ndb.non_transactional()
@ndb.tasklet
def non_transactional_tasklet():
    yield User.query().fetch_page_async(page_size=30)  # this works fine

cc: @chrisrossi

chrisrossi pushed a commit to chrisrossi/python-ndb that referenced this issue Oct 1, 2020
@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Oct 2, 2020
chrisrossi pushed a commit to chrisrossi/python-ndb that referenced this issue Oct 2, 2020
@chrisrossi chrisrossi added priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed triage me I really want to be triaged. labels Oct 2, 2020
andrewsg pushed a commit that referenced this issue Oct 2, 2020
…ync (#554)

* fix: `@non_transactional` decorator was not working correctly with async

Fixes #552

* We don't really need `_get_transaction` at all.

In all cases, we know where to get the transaction from.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: datastore Issues related to the googleapis/python-ndb API. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants