Add verbose logging option to WriteDepTree #968

ebati · 2017-08-07T18:19:17Z

What does this do / why do we need it?

Add logging parameter to gps.WriteDepTree and sw.Write to print each step if verbose option is selected.

What should your reviewer look out for in this PR?

Updated sw.Write and gps.WriteDepTree
Appended nil as logger for tests using these functions
LockedProject implements Stringer interface since both sw.PrintPreparedActions and gps.WriteDepTree logs LockedProject to user.

Do you need help or clarification on anything?

I implemented as we speak in #963. It might be better to implement this functionality inside SourceManager.

Which issue(s) does this PR fix?

fixes #963

carolynvs · 2017-08-07T20:38:22Z

This is just a heads up that #952 is moving this method into gps and tweaking parameters as well. The race to merge is on! 😀

carolynvs · 2017-08-07T20:54:25Z

internal/gps/result.go

 	if l == nil {
 		return fmt.Errorf("must provide non-nil Lock to WriteDepTree")
 	}

+	if logger != nil {


I am a little concerned that by using a nil logger, instead of a discard logger, that we may end up with nil pointer exceptions down the road as these functions evolve are worked on by new people.

Should i define a logger interface? If so should i change other functions taking *log.Logger to accept that interface.

Ohh, i can use log.New(ioutil.Discard, "", 0), sorry no need for interface 😅

Is it better to define ctx.VerboseLogger instead of Verbose bool. Verbose flag is mostly used as

if verbose{ logger.Print .... }

🤔 I am not really sure. We haven't settled on a standard yet, but what you see on ctx is what we have been doing so far.

I suggest that for now, follow the other examples in dep for verbose logging. If after you've done that, you see a way to simplify verbose logging, let's look at that in a separate issue or PR.

ebati · 2017-08-07T21:03:31Z

🤦‍♂️ i didnt noticed that, i can close this pr. After the merge of #952, i can add logger to NewSourceManager that migth help #534 as well.

carolynvs · 2017-08-07T21:06:43Z

No need to close the PR. I was just letting you know that you may need to rebase and incorporate changes from the other, depending on who merges first. 😁

As for #534, yes this may help with that (which would be awesome!) but I suggest limiting this PR to just the verbose logging, so that it's easier to review.

ebati · 2017-08-08T07:30:03Z

Is it ok to send verbose logs to stdout or should i use stderr. And if nil logger will not be used i can also change prune commands behavior as well.

carolynvs · 2017-08-08T14:30:08Z

Verbose logs should be written to ctx.Err, based on the ctx.Verbose flag.

Let's stick to only updating WriteDepTree in this PR. Once we are happy with a solution, updating other commands like prune can be done in a separate PR.

ebati · 2017-08-08T15:07:39Z

If this seems ok, i can squash commits.

carolynvs

I have just one small change, otherwise this looks ready to merge.

Originally I wanted to ask that we add a test that the verbose output for each command is printed as expected, but our test harness doesn't make that easy, and unit tests for that wouldn't quite capture what I'd like to test. I'll open another issue to discuss how we could tweak the test harness to validate verbose output for the commands.

carolynvs · 2017-08-08T15:42:39Z

internal/gps/lock.go

@@ -195,6 +196,15 @@ func (lp LockedProject) Packages() []string {
 	return lp.pkgs
 }

+func (lp LockedProject) String() string {


Now that this has been extracted into a function, would you mind adding a small unit test for it?

carolynvs · 2017-08-08T15:52:59Z

Don't worry about squashing, I can do that when I merge.

sdboyer · 2017-08-09T03:24:48Z

internal/gps/lock.go

@@ -195,6 +196,15 @@ func (lp LockedProject) Packages() []string {
 	return lp.pkgs
 }

+func (lp LockedProject) String() string {
+	prj := lp.Ident()


nit: for consistency with other references in gps, please name this var id

sdboyer · 2017-08-09T03:26:18Z

internal/gps/lock.go

+	if prj.Source == "" {
+		return fmt.Sprintf("%s@%s", prj.ProjectRoot, rev)
+	}
+	return fmt.Sprintf("%s -> %s@%s", prj.ProjectRoot, prj.Source, rev)


there's a method that does the equivalent of this already - ProjectIdentifier.errString() - but it represents the same information in a different way. let's reuse that method here, instead of creating a new visual representation of the notion of an alternate source.

I missed that function (and this review 😬 i will fix this) ,

no problem, there's a lot in gps - easy to miss things!

sdboyer · 2017-08-09T03:30:51Z

internal/gps/result.go

 	if l == nil {
 		return fmt.Errorf("must provide non-nil Lock to WriteDepTree")
 	}

+	logger.Println("Creating vendor dir ...")


i don't think this message adds much value. for one, this func isn't actually guaranteed to be creating a vendor/ dir - the way we use it right now, it's actually writing to a temporary directory, which will later be swapped into place as a vendor/ dir, assuming no errors occur.

if dep wants to put some kind of header message like this in place from within SafeWriter.Write(), that's fine, but let's keep what's printed directly here to a minimum.

sdboyer · 2017-08-09T03:38:38Z

internal/gps/lock.go

@@ -195,6 +196,15 @@ func (lp LockedProject) Packages() []string {
 	return lp.pkgs
 }

+func (lp LockedProject) String() string {
+	prj := lp.Ident()
+	rev, _, _ := VersionComponentStrings(lp.Version())


hmm...i think that only showing the revision here is probably not the optimal UX. it'd be better to show the full paired information - version or branch, and then rev in parens. it can follow the same pattern as in GetLockingFeedback(). the logic will have to be duplicated, though - not only do we try very hard to encapsulate gps into having no knowledge of dep, but that would create an import cycle.

Since Version defines Stringer instead of VersionComponentStrings, I directly used it, should i edit versionPairs Stringer definition to include both version and rev.

let's leave versionPair.String() out of this, at least for now, and just accept a bit of duplicate code for this particular case.

sdboyer · 2017-08-09T03:40:36Z

internal/gps/result.go

@@ -65,6 +68,7 @@ func WriteDepTree(basedir string, l Lock, sm SourceManager, sv bool) error {
 	for _, p := range l.Projects() {
 		to := filepath.FromSlash(filepath.Join(basedir, string(p.Ident().ProjectRoot)))

+		logger.Printf("Ensure %v exists", p)


i think that using "ensure" here may blur the meaning of the word a bit. let's say "Writing out %s" here, instead.

ibrasho · 2017-08-09T17:18:17Z

From a quick glance, no comments from my side.

Also, I don't think conflicts with #952 are going to be a problem now. 👍

sdboyer · 2017-08-12T19:30:14Z

internal/gps/lock_test.go

+		want string
+	}{
+		{
+			name: "Full Info",


nit: either all lower case ("full info") or normal sentence case ("Full info") for all of these, to be maximally consistent with our other subtest naming.

sdboyer · 2017-08-12T20:31:06Z

internal/gps/lock.go

@@ -195,6 +196,11 @@ func (lp LockedProject) Packages() []string {
 	return lp.pkgs
 }

+func (lp LockedProject) String() string {
+	return fmt.Sprintf("%s@%v for packages: %v",


hmm. so, we've a pair of problems.

first is that this is a generic Stringer implementation, but word choice is oriented towards the one use case we currently have: using the preposition "for" makes the most sense as a sort of explanation "we have this project for packages X, Y, Z within it." but that seems less than ideal, in part because that list of packages includes ones that are only required internally (e.g., some other project imports root/A, which imports its sibling root/B; both show up in the packages list.) to my mind, "with" is a bit better than "for", because it's more of a plain statement/less expository.

however, there's also the problem that listing out the packages at all probably gives the wrong impression to the user - it suggests that we're writing out only those packages, when we're actually writing out the entire tree.

so, let's use an unexported, purpose-specific func for generating this string, and omit package information from it entirely. basically, just the "%s@%v" part.

however, since we've already been through all this with the LockedProject.String() method, i'm ok with leaving it in (even though we're not using it), as long as we s/for/with/.

sdboyer · 2017-08-12T22:36:59Z

are you good with this, @carolynvs? if so, approve and then feel free to merge, this LGTM 😄

carolynvs

🎉 Thank you! ❤️

Add verbose logging option to WriteDepTree

0e9789b

ebati requested review from carolynvs, ibrasho and sdboyer as code owners August 7, 2017 18:19

googlebot added the cla: yes label Aug 7, 2017

carolynvs reviewed Aug 7, 2017

View reviewed changes

Use discardLogger for non-verbose requests

4bae486

Fix tests with nil verbose logger

e1b4bb5

ebati added 2 commits August 8, 2017 17:48

Use ctx.Err as verbose logger

37e2e02

Fix missed out nil loggers

f4c0625

carolynvs suggested changes Aug 8, 2017

View reviewed changes

carolynvs mentioned this pull request Aug 8, 2017

Change test harness to support testing full output of a command? #973

Closed

ebati added 2 commits August 8, 2017 21:43

Add package list to string

bd86b7f

Add test for LockedPackage.String()

72f9ec8

sdboyer suggested changes Aug 9, 2017

View reviewed changes

Remove extra logs

ffb06a7

Use ProjectIdentifier errString func to describe id

e1188e0

carolynvs mentioned this pull request Aug 9, 2017

Absorb prune into ensure #952

Closed

sdboyer suggested changes Aug 12, 2017

View reviewed changes

ebati added 2 commits August 13, 2017 00:05

Fix test names and s/for/with

28bba15

Remove packages from WriteDepTree log message

2814cc0

sdboyer approved these changes Aug 12, 2017

View reviewed changes

carolynvs approved these changes Aug 13, 2017

View reviewed changes

carolynvs merged commit 4758d86 into golang:master Aug 13, 2017

prymitive mentioned this pull request Aug 13, 2017

dep prune panics with invalid memory address or nil pointer dereference #1003

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add verbose logging option to WriteDepTree #968

Add verbose logging option to WriteDepTree #968

ebati commented Aug 7, 2017

carolynvs commented Aug 7, 2017

carolynvs Aug 7, 2017

ebati Aug 7, 2017

ebati Aug 7, 2017

ebati Aug 7, 2017

carolynvs Aug 7, 2017

ebati commented Aug 7, 2017

carolynvs commented Aug 7, 2017

ebati commented Aug 8, 2017

carolynvs commented Aug 8, 2017

ebati commented Aug 8, 2017

carolynvs left a comment

carolynvs Aug 8, 2017

ebati Aug 8, 2017

carolynvs commented Aug 8, 2017

sdboyer Aug 9, 2017

sdboyer Aug 9, 2017

ebati Aug 9, 2017

sdboyer Aug 11, 2017

sdboyer Aug 9, 2017

sdboyer Aug 9, 2017

ebati Aug 9, 2017

sdboyer Aug 11, 2017

sdboyer Aug 9, 2017

ibrasho commented Aug 9, 2017

sdboyer Aug 12, 2017

sdboyer Aug 12, 2017

sdboyer commented Aug 12, 2017

carolynvs left a comment

Add verbose logging option to WriteDepTree #968

Add verbose logging option to WriteDepTree #968

Conversation

ebati commented Aug 7, 2017

What does this do / why do we need it?

What should your reviewer look out for in this PR?

Do you need help or clarification on anything?

Which issue(s) does this PR fix?

carolynvs commented Aug 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebati commented Aug 7, 2017

carolynvs commented Aug 7, 2017

ebati commented Aug 8, 2017

carolynvs commented Aug 8, 2017

ebati commented Aug 8, 2017

carolynvs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carolynvs commented Aug 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibrasho commented Aug 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sdboyer commented Aug 12, 2017

carolynvs left a comment

Choose a reason for hiding this comment