Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StopIteration Runtime Exception with Python 3.8 #921

Open
jazzdup opened this issue Apr 22, 2021 · 0 comments
Open

StopIteration Runtime Exception with Python 3.8 #921

jazzdup opened this issue Apr 22, 2021 · 0 comments

Comments

@jazzdup
Copy link

jazzdup commented Apr 22, 2021

`2021-04-22 07:36:48,231 [CRITICAL] mongo_connector.oplog_manager:713 - Exception during collection dump
Traceback (most recent call last):
File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/doc_managers/solr_doc_manager.py", line 292, in
batch = list(next(cleaned) for i in range(self.chunk_size))
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/oplog_manager.py", line 668, in do_dump
upsert_all(dm)
File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/oplog_manager.py", line 651, in upsert_all
dm.bulk_upsert(docs_to_dump(from_coll), mapped_ns, long_ts)
File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/util.py", line 33, in wrapped
return f(*args, **kwargs)
File "/home/al/dev/zapper/projects/etl2/lib/python3.8/site-packages/mongo_connector/doc_managers/solr_doc_manager.py", line 292, in bulk_upsert
batch = list(next(cleaned) for i in range(self.chunk_size))
RuntimeError: generator raised StopIteration
2021-04-22 07:36:48,232 [ERROR] mongo_connector.oplog_manager:723 - OplogThread: Failed during dump collection cannot recover!`

code fix is in solr_doc_manager.py, replace 2x instances of:
batch = list(next(cleaned) for i in range(self.chunk_size))
with
batch = [] for i in range(self.chunk_size): for x in cleaned: batch.append(x)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant