Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: internal logging package refactor #898

Merged
merged 5 commits into from
May 29, 2017

Conversation

ploxiln
Copy link
Member

@ploxiln ploxiln commented May 20, 2017

follow-up to #892

In-progress. So far just nsqd converted to use the internal package, while we decide on design and naming. Also haven't converted the internal api_response and clusterinfo packages to use log levels and such yet (planning to do that after the main parts of nsqadmin and nsqlookupd, by passing a function similar to NSQD.logf() instead of a Logger to them).

Critique and better ideas are welcome @mreiferson

@ploxiln ploxiln added the chore label May 20, 2017
FATAL = 5
)

func ParseLogLevel(levelstr string, verbose bool) (int, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's what I had in mind: https://play.golang.org/p/bUb1ETf4zY

What do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can use a type, but I don't think it helps much.

logger.Output(3, fmt.Sprintf("%s: "+f, msgLevel, args...))

doesn't work, go says

../../go/src/github.com/nsqio/nsq/internal/lg/lg.go:66: too many arguments in call to fmt.Sprintf
	have (string, LogLevel, []interface {}...)
	want (string, ...interface {})

Thus the uglier thing I pushed up.

LogPrefix string `flag:"log-prefix"`
Verbose bool `flag:"verbose"` // for backwards compatibility
Logger Logger
logLevel int // private, not really an option
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be typed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a bit confusing: I did some moving-around to all three, but then used the new internal shared logging only for nsqd, while experimenting.

Since you approve of this style I'll roll it out to the rest (and we'll see some nice line-count savings).

@@ -10,15 +10,15 @@ type Options struct {
LogLevel string `flag:"log-level"`
LogPrefix string `flag:"log-prefix"`
Verbose bool `flag:"verbose"` // for backwards compatibility
Logger Logger
logLevel int // private, not really an option
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, should be typed

@mreiferson
Copy link
Member

@ploxiln this is looking good. It's pretty annoying that we're continuing to support -v now, and would love to think about ways we could drop that deprecated flag.

@mreiferson mreiferson changed the title internal logging package refactor *: internal logging package refactor May 21, 2017
@ploxiln ploxiln force-pushed the logging_refactor_2 branch 2 times, most recently from f66b3f7 to ce77eb1 Compare May 22, 2017 03:55
@ploxiln
Copy link
Member Author

ploxiln commented May 22, 2017

nsqadmin and nsqlookupd converted to internal/lg/, logging tests consolidated.

There's more to do for internal/http_api/, but you can see the strategy taken for internal/clusterinfo/.

@mreiferson
Copy link
Member

👍

@mreiferson
Copy link
Member

@ploxiln how're you feeling about this?

@ploxiln
Copy link
Member Author

ploxiln commented May 26, 2017

I think I'm done with changes for this round. Need to squash this down, obviously :)

One remaining piece I can find, which has not yet been converted, is the now-separate disk-queue. I'm not sure of the best way to handle that, but it can be in a separate PR IMHO.

Copy link
Member

@mreiferson mreiferson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ploxiln added 4 commits May 27, 2017 16:17
minor fix for --log-level flag help text

must set opts before logging errors in New()

re-order log stuff in Options structs
treat logLevel the same in nsqd, nsqlookupd, nsqadmin

Fix tests: multiple nsqlookupds need their own Options structs,
now that nsqlookupd has logLevel in its Options.
Otherwise the race detector complains when one of the nsqlookupds
writes the derived int logLevel while another reads it for logging.
introduce LogLevel type
NilLogger for disabling logging
consolidate LogLevel tests into internal/lg/
consolidate TestNoLogger
Instead of setting a Logger for github.com/nsqio/nsq/internal packages,
pass a logf() function, so it is called with and honors a LogLevel.
 * internal/clusterinfo/
 * internal/http_api/
 * internal/protocol/

nsqd lookupPeer also needed to be converted

Get rid of interal.app.Logger type, but internal/test/ needs
its own Logger definition to avoid circular import with
internal/lg/ tests.
@ploxiln ploxiln force-pushed the logging_refactor_2 branch from 259547b to c2c139d Compare May 27, 2017 20:26
@ploxiln
Copy link
Member Author

ploxiln commented May 27, 2017

Tastefully squashed, sanity-tested a bit with a light random mix of nsqd/nsqlookupd/nsqadmin/nsq_tail usage.

Coveralls ratings are annoyingly jittery - probably due to the asynchronous parts of the applications randomly hitting some cases depending on runtime scheduling.

@mreiferson mreiferson merged commit f874512 into nsqio:master May 29, 2017
@mreiferson
Copy link
Member

Thanks, nice work.

@ploxiln ploxiln deleted the logging_refactor_2 branch September 25, 2017 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants