Dag processor failing without logs #41897
Unanswered
awesomescot
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi. I'm struggling to get our dag processor to process around 5k dags. Either it is failing or just running too slowly to keep up. The dagbag and serialized dags are always out of sync. I'm looking for some hints in the logs, but not finding anything. Does anyone know a good way to troubleshoot dag parsing? I know I can run the
reserialize
command, which does complete after some time. So all dags are parsable, but maybe there is some timeout or other thing that is slowing it way down. Any advice would be appreciated. Thanks!Beta Was this translation helpful? Give feedback.
All reactions