Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move to non-spark etl glue jobs when we don't use spark #33

Open
thecaffiend opened this issue Jun 24, 2024 · 0 comments
Open

move to non-spark etl glue jobs when we don't use spark #33

thecaffiend opened this issue Jun 24, 2024 · 0 comments

Comments

@thecaffiend
Copy link
Contributor

This is super low priority as the costs seem to be exactly the same, but we care currently setup to use python spark jobs for our ETL glue jobs, but we do not make use of spark. The HAI specific jobs just be plain python jobs. (those are the only etl jobs we have right now, but by time of implementation there may be more to check out)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant