Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

context call sites #4164

Merged
merged 2 commits into from
Oct 29, 2021
Merged

context call sites #4164

merged 2 commits into from
Oct 29, 2021

Conversation

emmyoop
Copy link
Member

@emmyoop emmyoop commented Oct 29, 2021

Description

Modifying call sites for structured logging in core/dbt/context

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change

@cla-bot cla-bot bot added the cla:yes label Oct 29, 2021
else:
logger.debug(msg)
fire_event(MacroEventDebug(msg))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just popping in to say that this does get called a lot, for lots of different reasons—this is what's called any time a message is logged from within the Jinja context. That could be for materializations, wrapped by other macros in packages, to offer visibility in complex operations, or any custom user-space code.

I think MacroEvent is a fine name; log() is called as a macro, though it can be called from outside macros (models, hooks, etc). I also think it's fair that this is not going to be nearly as structured/consistent as built-in dbt logging.

Eventually, we might think about offering a more structured version of the log macro, e.g. in which users can pass a key-value dictionary and an f-string-like message. That's for later, this is good for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it worth a tech-debt ticket?

Copy link
Contributor

@jtcohen6 jtcohen6 Oct 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe! I wouldn't be surprised if there's contextual info freely available in the Jinja context, which we can pass along with the user-provided message to make it that much richer. E.g. the "parent" macro calling this log() macro. But I don't actually know that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened an issue to keep thinking about this: #4184

@emmyoop emmyoop requested a review from nathaniel-may October 29, 2021 14:54
Copy link
Contributor

@nathaniel-may nathaniel-may left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! If we run into a lot of events with this pattern of just passing a message, we may want to make a little escape hatch so we don't need a whole type every time, but I do consider it an anti-pattern to just blindly pass along things without at least adding context.

@emmyoop emmyoop merged commit a3dc5ef into feature/structured-logging Oct 29, 2021
@emmyoop emmyoop deleted the er/sl-context branch October 29, 2021 15:12
@nathaniel-may nathaniel-may mentioned this pull request Nov 8, 2021
21 tasks
emmyoop added a commit that referenced this pull request Nov 8, 2021
* updated context dir to new structured logging
emmyoop added a commit that referenced this pull request Nov 8, 2021
* updated context dir to new structured logging
kwigley pushed a commit that referenced this pull request Nov 9, 2021
* updated context dir to new structured logging
nathaniel-may pushed a commit that referenced this pull request Nov 9, 2021
* updated context dir to new structured logging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants