-
Notifications
You must be signed in to change notification settings - Fork 130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document production BigQuery data layout #302
Conversation
3fe1390
to
decdb0f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had several questions reading the transition docs today, and this page (as amended by this PR) answered them; thanks!
src/cookbooks/bigquery.md
Outdated
| |`tmp`|Temporary staging area for parquet data loads| | ||
| |`udf` |Persistent user-defined functions defined in SQL| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this link to https://github.com/mozilla/bigquery-etl/tree/master/udf or a UDF documentation page?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do
src/cookbooks/bigquery.md
Outdated
| |`tmp`|Temporary staging area for parquet data loads| | ||
| |`udf` |Persistent user-defined functions defined in SQL| | ||
| |`udf_js` |Persistent user-defined functions defined in JavaScript| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this link to https://github.com/mozilla/bigquery-etl/tree/master/udf_js or a UDF documentation page?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I opened #312 for the long term solution but I think these links would be useful in the interim.
@tdsmith Thank you very much for reading and commenting. I've updated to provide additional context on UDFs and to link to the source as you suggested. |
Co-Authored-By: Mark Reid <mreid@mozilla.com>
a96e08f
to
2a7f506
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me!
This PR adds a link to the evolving Data Access Continuity Guide and documents the layout and naming of datasets and tables in the reconfigured
shared-prod
GCP project.What is written here is not yet strictly true. Merging this PR will have to wait on full promotion of the GCP ingestion pipeline to prod, which will hopefully happen next week. It also depends on moving derived tables to shared-prod, which will also hopefully happen next week.
Viva Doc Days!