x/tools/cmd/godoc, gddo: module support

[agnivade]

I wanted to open an umbrella issue discussing the changes that we might need to do to adapt godoc now that we will start to have modules.

• For starters, now that we have $GOPATH/src/mod folder containing versioned snapshots of various libraries, all of them show up in the /pkg/ html output.

What is the recommendation to handle this case ? To simply dump all versions of all packages does not seem like the right thing to do. It also gives a sense of exposing internal details of storing "@vX.Y.Z" along with folder names.

(This is taken care of by moving from src/mod to pkg/mod)

• Also, for new systems that are completely working outside of GOPATH, how do we decide what to show in the output of /pkg/ page ? In the current workflow, it won't show anything because $GOPATH/src is empty. Everything is inside $GOPATH/pkg/mod.

Can we improve this ? Should we add an option to show godoc for a specific module root ? Or some notion to show documentation of only a given package.

I have not been following the progress on modules. So please let me know if there are some obvious fixes to these.

@rsc @bradfitz

[kr]

@agnivade note that in b8f42d7, the module cache moved to $GOPATH/pkg/mod, so this particular "for starters" issue seems like it's been addressed. If you still have anything in $GOPATH/src/mod, you can delete it.

[agnivade]

Indeed. Thank you.

[thepudds]

@gopherbot please add modules label

[agnivade]

I don't think this is a modules issue. It is about changes in godoc due to modules.

[acln0]

I have been directed here from the #modules channel on the Gophers slack. Hopefully this is the right place to give a report.

I'm writing a new package using modules, outside GOPATH. I have not pushed this package anywhere public just yet. I'd like to know how the HTML documentation renders before I do. I am writing examples and adding code blocks to godoc comments as well, so doing go doc acln.ro/foo is not enough.

So far, I've written a little shell script that copies the entire tree to GOPATH so that godoc can pick it up. As I iterate, I run the shell script. This doesn't feel quite right. I have also tried using symlinks, which godoc (rightfully) ignores.

Should we add an option to show godoc for a specific module root ?

That would certainly be nice. I would like to be able to point godoc at some directory outside GOPATH, where I am currently working.

[zigo101]

[moved from https://github.com//issues/28720]

A suggestion: the godoc.org page for a package containing the go.mod file should also show an import/replace path specified in the go.mod file.

For example, this package https://godoc.org/github.com/go101/tinyrouter specified its import path in go.mod as go101.org/tinyrouter instead of its hosing path. The the doc page should also list the import path which is specified in go.mod.

It would be better if the doc page can suggest a replace line for package users
if the godoc program finds that the hosting path and the module path specified
in the go.mod file are not consistent.

An example for the TinyRouter doc page:

====

package tinyrouter

import "github.com/go101/tinyrouter" // for GOPATH based projects

import "go101.org/tinyrouter" // for modules based projects

Please add the following line in your go.mod file to use this package
if your project is modules based:

replace go101.org/tinyrouter => github.com/go101/tinyrouter v1.0.1

[galxy25]

+1 for @go101 A suggestion: the godoc.org page for a package containing the go.mod file should also show an import/replace path specified in the go.mod file. updating a public GO sdk to be modules based and now the published Godoc page for the module has an incorrect module import 🤦🏾‍♂️

[matt0x6F]

If godoc could support doing previews for an individual package (in the context of a directory rather than through package discovery) I think that might help a lot of use cases in the near term. Even if this were a temporary feature I think it'd go a long way.

If godoc can already do this I'd love to hear about it.

[thepudds]

Workaround from thread (similar to
#26827 (comment)):

The solution I came up with is to have a bash script like this in my Makefile:

mkdir -p /tmp/tmpgoroot/doc
rm -rf /tmp/tmpgopath/src/github.com/username/packagename
mkdir -p /tmp/tmpgopath/src/github.com/username/packagename
tar -c --exclude='.git' --exclude='tmp' . | tar -x -C /tmp/tmpgopath/src/github.com/username/packagename
echo -e "open http://localhost:6060/pkg/github.com/username/packagename\n"
GOROOT=/tmp/tmpgoroot/ GOPATH=/tmp/tmpgopath/ godoc -http=localhost:6060

Karel

[thepudds]

Reopening. (Sorry, fat fingered on my phone).

[matt0x6F]

Yeah, this is more or less what I've been doing. Instead of dumping it in /tmp I actually put it where it should go in the GOPATH, do my changes, then move it back to my normal directory. I wonder if this can be done with symlinks? I don't know of any potential side effects either. I'll report back.

[alexaandru]

Yeah, this is more or less what I've been doing. Instead of dumping it in /tmp I actually put it where it should go in the GOPATH, do my changes, then move it back to my normal directory. I wonder if this can be done with symlinks? I don't know of any potential side effects either. I'll report back.

No go with symlinks, but sudo mount -o bind does the trick :) Now I can keep my project in only one place and let godoc see it where it wants to see it... :)

[izgeri]

We are working on a workaround with Docker - mounting our code in as a volume and running godoc from the container. But really, it would be nice if we could run it locally as intended.

[dmitshur]

But really, it would be nice if we could run it locally as intended.

There is planning work being done with the goal of making that the case. I will be posting more updates in the coming weeks. Thanks for your patience on this.

[stapelberg]

After using @thepudds workaround from above I decided to add another that eliminates the need for creating special tmp folders. This one requires docker installed. It's a simple linux bash alias that runs a function which starts docker, attaches the current directory as a volume, and configures the necessary setup inside the container. Just one liner. Tested it only on debian.

Thanks for sharing your solution!

Unfortunately, it didn’t work for me in zsh as-is, so I’ve taken the liberty to rewrite it in a way that works and that I find more clear. I tested this in bash and zsh:

function godoc() {
  if [ ! -f go.mod ]
  then
    echo "error: go.mod not found" >&2
    return
  fi

  module=$(sed -n 's/^module \(.*\)/\1/p' go.mod)
  docker run \
    --rm \
    -e "GOPATH=/tmp/go" \
    -p 127.0.0.1:6060:6060 \
    -v $PWD:/tmp/go/src/$module \
    golang \
    bash -c "go get golang.org/x/tools/cmd/godoc && echo http://localhost:7070/pkg/$module && /tmp/go/bin/godoc -http=:6060"
}

[julieqiu]

This issue is the umbrella issue for all “godoc”-like projects that need to support modules. It’s not easy to discuss progress on individual projects here. We have created smaller tracking issues for the individual projects:

#33654 - discovery: Go discovery site tracking issue
#33655 - x/tools/cmd/godoc: add module support

[thediveo]

@stapelberg in newer Go images, at least :1.13 and :latest, godoc isn't installed by default. Thus, the bash command should start first with go get golang.org/x/tools/cmd/godoc && ... and then proceed as before. Oh, and if you're behind a corporate proxy, then this needs to be passed into the container too, such as with -e "http_proxy=$http_proxy".

Thank you very for this useful script!

[stapelberg]

Thanks, I updated my earlier comment to work with Go 1.13.

[stevegt]

Here's a generalized version of @stapelberg's code -- this one serves all of the packages in your dev tree regardless of whether they have a go.mod yet or not. Modify $devbase (or pass it in as $1) to point at the base of your tree:

#!/bin/bash 

set -x  # optional

devbase=$HOME/gohack
port=6060

docker run \
    --rm \
    -e "GOPATH=/tmp/go" \
    -p 127.0.0.1:$port:$port \
    -v $devbase:/tmp/go/src/ \
    --name godoc \
    golang \
    bash -c "go get golang.org/x/tools/cmd/godoc && echo http://localhost:$port/pkg/ && /tmp/go/bin/godoc -http=:$port"

[romandvoskin]

@stevegt Thanks so much for your script. To make things even more dead simple I replaced devbase=$HOME/gohack with devbase=$(pwd) and placed the script at the base of my repo.

[nextkitt]

Only in Go1.13, look this:

ln -s $GOPATH/pkg/mod  $GOPATH/src

and

godoc -http=:6060

[dmitshur]

For those who are following this issue because you are waiting for module support in the godoc command (golang.org/x/tools/cmd/godoc), please note that it has been implemented in the latest version. See issue #33655.

I've also sent a mail to golang-nuts to announce this.

[StarpTech]

@dmitshur I faced another issue. In my organization, we host on Gitlab and projects are namespaced with their "groups". My module and VCS name is git.$organization.se/$group/my-project.

When I run godoc -http=:6060 in root and click on the package git.$organization I see the following error in the terminal:

using module mode; GOMOD=/mnt/ssd/repositories/golang-dynaday/go.mod
2020/03/07 00:03:31 cannot find package "." in:
        /src/$organization

This results in a broken UI and thus my packages aren't listed correctly.
go list -m all list correctly all packages and modules but godoc doesn't display them.

The workaround is to remove $group name from the module name and update all places accordingly. I'm not aware of that constrain.

go.mod

module git.$org.com/internal/golang-dynaday

go 1.14

replace git.$org.com/internal/golang-dynaday/submodule => ./submodule

require (
	git.$org.com/internal/golang-dynaday/submodule v0.0.0-00010101000000-000000000000
)

[dmitshur]

@StarpTech Thanks for reporting. Do you mind opening a new issue with a "x/tools/cmd/godoc: " prefix? That will make it easier to track that specific problem.

I have a guess about what the problem may be. I have a WIP CL 205661 which I suspect may resolve the issue. If you can make a smaller reproduce case, that would be very helpful to confirm whether or not it it'll solve your problem. (Alternatively, you can try that CL yourself and let me know.)

[StarpTech]

@dmitshur thanks for the quick response. I created #37729 and there you find the feedback for the patch.

[dmitshur]

golang.org/x/tools/cmd/godoc got module support in 2019 (see #26827 (comment)), though it has since been deprecated in favor of golang.org/x/pkgsite/cmd/pkgsite (see #49212).

The gddo codebase is frozen, and godoc.org redirects to pkg.go.dev since early 2021 (see https://go.dev/blog/godoc.org-redirect).

I think this umbrella tracking issue can be closed now.