Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: document that module names without dots are reserved #32819

Closed
mvdan opened this issue Jun 27, 2019 · 18 comments
Closed

cmd/go: document that module names without dots are reserved #32819

mvdan opened this issue Jun 27, 2019 · 18 comments
Assignees
Labels
Documentation modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@mvdan
Copy link
Member

mvdan commented Jun 27, 2019

Intuitively, most Go developers publish Go code under import paths like foo.com/bar, not foo/bar. Often, this is just to make them reachable on the internet.

However, some have avoided the .tld entirely when working with local-only GOPATHS and modules. And this mostly works, as long as you're careful to not clash with an existing standard library package.

For example, I recall a recent issue in this tracker (which I can't find now!) where their GOPATH build broke when upgrading the Go version, as the root path element was plugin, which had been added to the standard library since.

An instance of this discussion happened last September, as pointed out by @bcmills. Quoting @rsc:

We want to allow experimentation, so we're not categorically cutting off all module statements in all contexts without dots, but anything that has to get resolved through downloading needs a dot and more generally should have a fully qualified domain name. That's how we've carved the name space between standard library and external things. I understand that pedantically the RFCs don't require dots but that's the separation we've used from very very early on with goinstall and later go get.

However, this separation isn't well documented anywhere. Even if we consider it as a warning or recommendation more than a strict rule, I think we should still mention it somewhere.

I also think we should encourage the use of .tld even if the module is for local use only, because of potential breakages with future Go builds. Perhaps a suggestion can be made, like .local.

Another advantage of strongly encouraging the use of .tld is that it's much easier to tell if a package is from the standard library or not. Without the separation, one would have to keep a list of all standard library package paths, which easily gets out of date. Or run go list std, which adds work.

Note that this doesn't need to change how cmd/go behaves; it's fine if a module name without a dot happens to build today.

@mvdan
Copy link
Member Author

mvdan commented Jun 27, 2019

The title contains "modules", but I assume this would apply to the GOPATH mode too. That is, $GOPATH/src/foo/bar would be discouraged just like module foo/bar.

@andybons andybons added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 28, 2019
@andybons andybons added this to the Unplanned milestone Jun 28, 2019
@andybons
Copy link
Member

@bcmills @jayconrod

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Jun 28, 2019
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 28, 2019
@mvdan
Copy link
Member Author

mvdan commented Dec 8, 2019

Small detail I forgot - it's a dot in the first path element. foo.com/bar is fine as a non-reserved path, but foo/bar.baz technically isn't.

@mvdan
Copy link
Member Author

mvdan commented Dec 14, 2019

Another small detail. It's not correct to say "all import paths without dots are reserved by the standard library", because there are a few counter-examples. Use go list foo.go, and you'll see command-line-arguments. Or do go build -x foo.go, and you'll see [...]/compile -p main [...].

Both main and command-line-arguments are reserved, but they don't belong to the standard library. Perhaps it's correct to say that they belong to ad-hoc packages, those constructed via lists of Go files alone.

I'm also a bit surprised that the two commands above give different package paths. I think that's counter-intuitive, but maybe there's a good reason for it.

@bcmills
Copy link
Contributor

bcmills commented Dec 17, 2019

Import paths without dots are reserved for the standard library and toolchain.

command-line-arguments and main are both synthesized by the toolchain: if you try to import them from some other package you're gonna have a bad time.

@mvdan
Copy link
Member Author

mvdan commented Dec 17, 2019

Thanks, I like that wording. It's clear in my mind now. I'll have a look at sending a CL before the freeze is over :)

@mvdan
Copy link
Member Author

mvdan commented Sep 10, 2020

I'd like to work on this this cycle. Any thoughts on where such a piece of documentation could go? I'm thinking either go help modules or https://golang.org/ref/mod. I imagine the latter might be best, because we don't want go help modules to become too long for terminals.

@jayconrod
Copy link
Contributor

https://golang.org/ref/mod would be best. Ideally go help modules will be significantly shortened in 1.16, and it refer to that, plus some introductory docs.

@mvdan
Copy link
Member Author

mvdan commented Feb 12, 2021

A separate but still related issue: tinygo is a different implementation of Go, and it adds its own standard library packages, much like syscall/js exists for gc's port to js/wasm. For example, they add packages like machine and device/arm, which I think is entirely reasonable. When one builds with tinygo, tinygo is what owns the standard library and toolchain, so it's within its right to include these extra packages.

However, go mod tidy gets upset, as it's not aware of those packages:

github.com/tinygo-org/tinygo/src/device/arm imports
  runtime/volatile: package runtime/volatile is not in GOROOT (/home/mvdan/tip/src/runtime/volatile)
github.com/tinygo-org/tinygo/src/device/nxp imports
  device/arm: package device/arm is not in GOROOT (/home/mvdan/tip/src/device/arm)
github.com/tinygo-org/tinygo/src/examples/adc imports
  machine: package machine is not in GOROOT (/home/mvdan/tip/src/machine)

I think, to allow modules to play nicer with alternative Go implementations, we should allow it to ignore reserved package paths that it doesn't know about.

@bcmills
Copy link
Contributor

bcmills commented Feb 12, 2021

@mvdan, machine and device are reserved for the Go standard library, not for other related standard libraries. TinyGo should arguably be using paths like tinygo.org/machine and tinygo.org/device/arm rather than machine and device/arm, and at that point they could provide dummy or polyfill packages that could be fetched by go mod tidy when using the regular go tool.

@mvdan
Copy link
Member Author

mvdan commented Feb 12, 2021

machine and device are not reserved

Hm, doesn't that contradict what we said earlier in the thread?

Import paths without dots are reserved for the standard library and toolchain.

@bcmills
Copy link
Contributor

bcmills commented Feb 12, 2021

Ah, yeah. I see what you mean. They are reserved for the Go standard library — not for adjacent projects.

(Edited my reply above.)

@aykevl
Copy link

aykevl commented Feb 17, 2021

Hi all! Author of TinyGo here. You might want to read my previous comment on the issue: #43079 (comment)

I am well aware of this issue and I like to change it, but it will be a major breaking change so I'd like to provide a soft transition path. The proposal in #34055 would be a big help to allow that to happen, and luckily it appears to lean towards being accepted. These reserved paths made sense when I started the project but especially now with Go modules it's becoming a compatibility issue (something which I've wanted to avoid).

EDIT: although I should add that it's still important that if you put "machine" or "tinygo.org/machine", it has to be the local path. go list (when called from TinyGo with the custom GOROOT) should not try to go through the Go proxy to resolve them. But this appears to be the case, so that's good.

@mvdan
Copy link
Member Author

mvdan commented Mar 4, 2021

An upcoming change in go mod tidy also seems to align with this documentation change: https://go-review.googlesource.com/c/go/+/298749/

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/359594 mentions this issue: _content/ref/mod: document dotless module paths

@mvdan
Copy link
Member Author

mvdan commented Jan 10, 2023

$ go list -e -f '{{.Standard}} -- {{.Error}}' future/std
false -- package future/std is not in GOROOT (/home/mvdan/tip/src/future/std)

I wonder if Standard should be true for all import paths without dots, even those which are missing packages, as they are technically reserved for the standard library - even if they are not part of it today.

I ran into this today as I assumed Standard to be true for arena - but one of the Go versions I was testing with was 1.19, where the package doesn't exist yet, so... it was false there.

@bcmills
Copy link
Contributor

bcmills commented Jan 11, 2023

I don't think we should bother setting Standard for packages that don't exist. Despite the names being reserved, some user code does use dotless package paths, and I would rather not explicitly set a field we're not sure about for about a package we don't know anything about.

@mvdan
Copy link
Member Author

mvdan commented Jan 11, 2023

Yeah, that's fair. I was a bit on the fence about that, and I guess I can implement the "reserved" logic myself if I want to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

8 participants
@jayconrod @andybons @aykevl @dmitshur @mvdan @bcmills @gopherbot and others