cmd/go: go get takes very long time on projects with a lot of history

[akamensky]

What version of Go are you using (go version)?

1.7.4 darwin

What did you do?

go get guthub.com/andlabs/ui

What did you expect to see?

Since we only need/expect latest version the shallow copy of repository is expected.

What did you see instead?

The full git clone which since there are binary files checked in and they were updated multiple times takes ~30 minutes to execute.

[minux]

We once tried shadow clone, but it broke things and was rolled back. See https://golang.org/cl/16360 for the CL, and https://golang.org/cl/16832 for the rollback.

[akamensky]

Make an optional feature enabled command line argument then maybe? Something like '--shallow' that would call git clone --depth 1?

[akamensky]

That project is just one of many with which I see that. Some other projects may take unbearably long.

Right now I just resort to doing a "git clone --depth 1 https://www.github.com/user/project $(GOPATH)/src/github.com/user/project/" which is rather annoying if there are multiple dependencies to fetch

[cznic]

jnml@4670:~/src/github.com$ rm -rf andlabs/
jnml@4670:~/src/github.com$ time git clone https://github.com/andlabs/ui.git
Cloning into 'ui'...
remote: Counting objects: 12567, done.
remote: Total 12567 (delta 0), reused 0 (delta 0), pack-reused 12567
Receiving objects: 100% (12567/12567), 20.89 MiB | 4.69 MiB/s, done.
Resolving deltas: 100% (8721/8721), done.
Checking connectivity... done.

real	0m5.585s
user	0m1.068s
sys	0m0.224s
jnml@4670:~/src/github.com$ 

Do you use Windows+antivirus perhaps?

[akamensky]

No. I am using Mac/Linux and I am located in PRC at the moment. From here Github is accessible but rather slow.

And this isn't the only place this happens. Same is half of South East Asia and Oceania, except Japan and Korea perhaps.

[rsc]

Related to #13078. I don't have a quick fix for you. I hope the long-term package management story will solve this, by not requiring full Git checkouts. For now I will leave the issue open for tracking purposes.

[bcmills]

In module mode, go get takes ~10s for me and results in a ~74M total download.

$ time go get -d github.com/andlabs/ui
go: finding github.com/andlabs/ui latest
go: downloading github.com/andlabs/ui v0.0.0-20180902183112-867a9e5a498d
go: extracting github.com/andlabs/ui v0.0.0-20180902183112-867a9e5a498d

real    0m9.285s
user    0m4.482s
sys     0m0.326s

$ du --summarize --human-readable $GOPATH/pkg/mod
74M     /tmp/tmp.Eb0RbOxaOy/_gopath/pkg/mod

GOPATH mode downloads a bit more, but the overall result is similar.

$ GO111MODULE=off time go get -d github.com/andlabs/ui
real    0:03.24
user    2.26
sys     0.53
rss     164768k

$ du --summarize --human-readable $GOPATH
86M     /tmp/tmp.8LPirEl234/_gopath

As far as I can tell this seems to be resolved.