Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cria parâmetro para Dataset no Data-Aware Schedulling #115

Merged
merged 11 commits into from
Aug 9, 2024
Merged

Conversation

edulauer
Copy link
Member

@edulauer edulauer commented Aug 8, 2024

  1. Cria um novo parâmetro dataset permitindo data-aware schedulling da DAG baseado em dataset e/ou agendamento cron.
  2. Atualiza exemplos utilizando o parâmetro dataset.
  3. Refatoração: altera e migra as funções utilizada para tratamento de schedule da classe YamlParser para DouDigestDagGenerator
  4. Atualiza os testes
  5. Atualiza a DAG ro-dou_inlabs_load_pg_dag.py incluindo uma task de verificação da carga do dia do INLABS na base da conexão inlabs_db
  6. Inclui na DAG ro-dou_inlabs_load_pg_dag.py um decorator de Short Circuit para interromper a execução da DAG caso não existam arquivos no portal do INLABS para o dia.

@edulauer edulauer added documentation Improvements or additions to documentation enhancement New feature or request labels Aug 8, 2024
@edulauer edulauer merged commit 8972c96 into main Aug 9, 2024
1 check passed
@edulauer edulauer deleted the issue-114 branch August 9, 2024 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant