documentation: is "dep" replacing "go get"

[ensonic]

So far I've build using

go get -d
go install

When using dep, should one still run go get? Or rather

dep ensure -update
go install

[carolynvs]

I would say that dep doesn't replace go get, but they both can do similar things. Here's how I use them:

• go get: I want to download the source code for a go project so that I can work on it myself, or to install a tool. This clones the repo under GOPATH for all to use.
• dep ensure: I have imported a new dependency in my code and want to download the dependency so I can start using it. My workflow is "add the import to the code, and then run dep ensure so that the manifest/lock/vendor are updated". This clones the repo under my project's vendor directory, and remembers the revision used so that everyone who works on my project is guaranteed to be using the same version of dependencies.

Essentially I use dep only for managing my project's dependencies, and go get for pretty much everything else.

[sdboyer]

I'm pretty much in line with @carolynvs' response, here.

Also, more or less this question was asked on reddit today, and I responded over there.

Basically - exactly what will happen is unclear, but the long term vision is a sane, overall-consistent go tool. My general take is that go get is for people consuming Go code, and dep-family commands are for people developing it.

[ensonic]

That makes sense. It would be awesome to make sure this is mentioned somewhere to ensure people understand the scopes. Feel free to close the ticket.

[sdboyer]

Definitely. I've added a 'docs' label, which is equivalent to putting it on that TODO list :)

[sarge]

What are peoples thoughts on editors with linters, from my experience using a couple of editors with go integration (atom/vs code) is that the bundled linters work against the typical dep flow (thanks @carolynvs for sharing your flow because it helped my understanding a lot).

The behaviour I see it that autocomplete doesn't work unless you have the package loaded, and typically on save linters will remove unused imports.

So you get a situation where you need to have code that would work in your editor and saved, before you can use dep to load the dependency into the vendor directory. If the linter does not think your code uses the packaged dependency, the linter will delete the "unused" import. So no new dependency to dep ensure. Quite a maddening cycle.

A possible solution would be that calling dep ensure "package" would include the package as a dependency regardless of whether the package was referenced in a imports statement.

You could argue that this is an editor / linter issue, but auto formatting and linting is a pretty useful and commonly practice it seems to me.

[sarge]

Ah just read the v2 spec with the --add command. Looks great, though I wonder about the expectation that this would be rare.

[sdboyer]

@sarge (welcome!) as you note, the updated spec should help this along, though i do anticipate there'll still be some pain. i see the problem more as operating in a reverse direction, though. ordinarily, my editor-assisted flow goes something like this:

1. I add a new reference to a func/struct/whatever in a package not imported in the current file
2. Run goimports, which will automatically search for and discover an appropriate package for the unsatisfied import, incorporating information about the specific symbol i've referenced

so, i'm imagining the challenge here will be less about the case of an unused import going away; that still seems like a pretty normal bit of cleanup to me, and triggering a dep ensure there after the linter/goimports has stripped out the unneeded import should have the desired effect. the problem, rather, is in adding a new import, and deciding what that should be.

i haven't looked at goimports' search algorithm, so my familiarity with it is based only on (non-exhaustive) use. but in a post-dep world, i imagine the flow we want will look something more like this:

1. I add a new reference to an identifier in a package that's not already imported in the current file
2. I run goimports (or its successor?); once it's determined that there's an unmet import, it searches using the following algorithm:
   1. Examine Gopkg.lock for packages named for the package in the code; look in vendor/ to see if they contain the referenced identifier. If success, add to imports in file; done
   2. Look in (search location) for packages with the right name and containing the referenced identifier. If success, add to imports in file and trigger a dep ensure.

I say (search location) rather than just GOPATH because GOPATH, as we've all been noting for so long now, generally lacks the right version metadata to make a good decision. Certainly it's what we'll need to rely on initially, but my hope is that we'll eventually rely on something more like this.

[sarge]

@sdboyer first thanks for your efforts. Lots of moving parts and lots of stakeholders, your approach of sketching a vision for the future and making pragmatic steps forward today feels right.

From my limited experience goimports as of today will only use local packages, so you will get a different experience based on whether you have used this package before or not. dep ensure --add package will fix that that case too.

[carolynvs]

Closing now that the original question (dep vs go get) has been answered in the FAQ.

[paulwalker]

I'd just like to comment that the use of go get fails for a particular project if you are using a specific [[contstraint]] attribute that get get does not understand like specifying source to a fork.

[carolynvs]

@paulwalker Can you help me understand what part of dep's config breaks go get? Is there a repo somewhere that I can see that in action?

[paulwalker]

@carolynvs Please see: http://blog.campoy.cat/2014/03/github-and-go-forking-pull-requests-and.html. Dep does not have a problem with this, it allows users to define a source value in constraints that point to a different repo url. Until go get recognizes dep configuration in Gopkg.toml or the two tools are combined in some fashion, someone can't simply go get dependencies of a package that depends on specific branches, forks, etc.

[sdboyer]

@paulwalker yep yep - which is a problem, as long as you're not committing vendor. most people are nowadays, at least for projects with main packages in them.

teaching go get about dep's metadata is very much part of the roadmap for toolchain integration.

[paulwalker]

Hmm, so I guess I could just commit the packages to vendor in which I am dependent upon an attribute go get does not understand then, I hadn't thought of that. Thanks for triggering the thought.

[paulwalker]

@sdboyer, @carolynvs So, I'm still struggling with something. The issue is that we have another package (our kit package) that depends upon the fork I have created. The kit package is in $GOPATH so that we can have it linked to our repo. The fork is in /vendor. Apparently dependency resolution with go run does not continue to look in /vendor for the current working directory if the "depending" package is in $GOPATH instead. I attempted to use a symlink, but that did not work as well.

How would you suggest dealing with this? @sdboyer, FYI I sat in your talk in Denver last month, thus I'm here now using dep :-).

[sdboyer]

FYI I sat in your talk in Denver last month, thus I'm here now using dep :-)

excellent! 😄 😄 🎉

unfortunately, i don't quite understand the situation you're describing. are you saying that you have this kit dependency which you want to have sit on GOPATH, rather than be treated as a dependency that gets encapsulated into vendor/?

[paulwalker]

Yes, as developers work on both kit (which depends on the forked package which must sit in vendor) and the project that depends on it at the same time, it is ideal for the kit project to reside in GOPATH where it retains git repo access. Of course, developers could manually add git repo access to the kit package in vendor/, but submodules have always been scary.

Is it not possible for go dependency resolution to always use PWD /vendor as primary search location even for depending packages in GOPATH?

[sdboyer]

ah, yup, you're also in the multi-project workflow trap 😢

we don't have a good answer for this right now, unfortunately. however, https://github.com/GetStream/vg may help.