Test LibGit2 SSH authentication #17651

Keno · 2016-07-27T19:06:14Z

This was part of #17586 (and those commits will show up here until that is merged) and is a WIP attempt at writing a test for all the authentication stuff. Currently flakey, so do not merge until I've had a chance to fix that.

tkelman · 2016-07-27T19:13:22Z

thanks. all of the now-collapsed review from #17586 on the test/ content still applies here

Keno · 2016-07-27T19:14:07Z

Yep

Keno · 2016-08-02T23:35:17Z

Rebased, still don't merge though. Now triggers a kernel hang when you attach a debugger. Will try with a more recent kernel to see what I can do.

Keno · 2016-08-05T23:39:56Z

Ok, this should be good to go if and when CI passes.

tkelman · 2016-08-05T23:57:13Z

test/libgit2.jl

+        ssh_repo = joinpath(dir, "Example.SSH")
+        if !is_windows()
+            try
+                sshd_command = strip(readstring(`which sshd`))


which is not always installed. how about success(sshd) inside a try catch?

#17586 (comment)

That's not the purpose of this command. sshd refuses to be invoked as anything other than it's absolute path.

oh. add a comment that says that.

Keno · 2016-08-06T00:28:14Z

Review comments addressed and hopefully added support for the ancient versions of openssh we apparently have on CI.

tkelman · 2016-08-06T00:32:40Z

hopefully added support for the ancient versions of openssh

What was the change for that?

Keno · 2016-08-06T00:35:14Z

Use -e and an explicitly opened file for STDERR rather than -E.

tkelman · 2016-08-06T08:03:37Z

Should "libgit2" be added to net_required_for here

julia/test/choosetests.jl

Line 83 in bc4eba5

net_required_for = ["socket", "parallel"]

? It looks like it shouldn't require outside internet access like pkg or libgit2-online, but it does need a localhost socket?

Keno · 2016-08-06T15:39:05Z

Hmm, yeah I suppose it does require a loopback.

wildart · 2016-08-06T16:59:05Z

test/choosetests.jl

@@ -80,7 +80,7 @@ function choosetests(choices = [])
        prepend!(tests, linalgtests)
    end

-    net_required_for = ["socket", "parallel"]
+    net_required_for = ["socket", "parallel", "libgit2"]


not libgit2 but libgit2-online

No, these tests are in libgit2, since they don't require a network connection but they do require lo. I suppose we could split them out yet again, but not sure it's worth it. @tkelman?

this seems fine, I rarely see net_on not true. WSL is one of the few places that's the case

World Surf League?

windows subsystem for linux - aka bash on ubuntu on windows

wildart · 2016-08-06T17:03:33Z

I thought that you always have a loopback on any modern OS.

tkelman · 2016-08-08T11:37:12Z

this has been failing on the mac buildbots https://build.julialang.org/builders/build_osx10.9-x64/builds/1500/steps/shell_2/logs/stdio

edit: and ubuntu https://build.julialang.org/builders/build_ubuntu14.04-x64/builds/1244/steps/shell_2/logs/stdio
and centos https://build.julialang.org/builders/build_centos7.1-x64/builds/1671/steps/shell_2/logs/stdio

Keno · 2016-08-08T16:31:55Z

Could potentially be fixed by #17874.

tkelman · 2016-08-10T10:43:51Z

not fixed by #17874, will prevent new nightlies from being built until working properly https://build.julialang.org/builders/build_centos7.1-x64/builds/1703/steps/shell_2/logs/stdio

wildart · 2016-08-10T22:03:12Z

It's not related to #17874. There is something wrong with SSHD key setup.

    From worker 7:    Expression: isfile(joinpath(ssh_repo,"testfile"))
    From worker 7:  SSHD logfile contents follows:
    From worker 7:  Could not load host key: /tmp/tmpZ3Yh3P/ssh_host_rsa_key
    From worker 7:  Could not load host key: /tmp/tmpZ3Yh3P/ssh_host_dsa_key
    From worker 7:  HOME: /tmp/tmpZ3Yh3P
    From worker 7:  GitError(Code:ERROR, Class:OS, Failed to connect to localhost: Connection refused)

Keno · 2016-08-10T22:10:09Z

@tkelman can you see if ssh-keygen is working on the buildbots, perhaps we should include ssh-keygen output in the log file?

wildart · 2016-08-10T22:35:16Z

Better put a -d option to sshd call for debug output.

Keno · 2016-08-10T22:39:36Z

-d only allows one connection. See the debug flag that's already in the test

tkelman · 2016-08-11T00:18:54Z

What should I look for?

Keno · 2016-08-11T00:29:05Z

Try the keygen commands in the test file. If that works fine, enable the debug flag.

(cherry picked from commit 7f074e9)

ref #17651 (comment)

ref #17651 (comment) (cherry picked from commit 8f06a6c) [av skip]

…ots (#18397) ref #17651 (comment) (cherry picked from commit 8f06a6c) [av skip]

tkelman added test This change adds or pertains to unit tests libgit2 The libgit2 library or the LibGit2 stdlib module labels Jul 27, 2016

tkelman mentioned this pull request Jul 28, 2016

Don't zero usesshagent #17682

Merged

Keno force-pushed the kf/libgit2test branch from 9798520 to cb98dc0 Compare August 2, 2016 23:34

Keno force-pushed the kf/libgit2test branch 2 times, most recently from b0ad410 to 423bc9d Compare August 5, 2016 23:39

Keno changed the title ~~WIP: test LibGit2 SSH authentication~~ Test LibGit2 SSH authentication Aug 5, 2016

Keno added the backport pending 0.5 label Aug 5, 2016

tkelman reviewed Aug 5, 2016
View reviewed changes

Keno force-pushed the kf/libgit2test branch from 423bc9d to 0615b3b Compare August 6, 2016 00:27

Test LibGit2 SSH authentication

496618a

Keno force-pushed the kf/libgit2test branch from 0615b3b to 496618a Compare August 6, 2016 15:40

wildart reviewed Aug 6, 2016
View reviewed changes

tkelman merged commit 7f074e9 into master Aug 7, 2016

tkelman deleted the kf/libgit2test branch August 7, 2016 06:45

ranjanan pushed a commit to ranjanan/julia that referenced this pull request Aug 8, 2016

Test LibGit2 SSH authentication (JuliaLang#17651)

2e3b17a

Keno mentioned this pull request Aug 10, 2016

libgit2 and ssh keys - regression in 0.5.0-rc0 compared to 0.4.0 #17768

Closed

tkelman pushed a commit that referenced this pull request Aug 11, 2016

Test LibGit2 SSH authentication (#17651)

eb2f488

(cherry picked from commit 7f074e9)

tkelman added a commit that referenced this pull request Aug 11, 2016

Temporarily disable SSH tests until we can get them working on buildbots

8f06a6c

ref #17651 (comment)

tkelman mentioned this pull request Aug 17, 2016

Nightlies behind on Travis? #18079

Closed

tkelman mentioned this pull request Aug 31, 2016

ssh tests fail when test/test.jl and test/libgit2.jl run on same worker #18306

Closed

mfasi pushed a commit to mfasi/julia that referenced this pull request Sep 5, 2016

Test LibGit2 SSH authentication (JuliaLang#17651)

d891d57

tkelman added a commit that referenced this pull request Sep 7, 2016

Temporarily disable SSH tests until we can get them working on buildbots

97539c6

ref #17651 (comment) (cherry picked from commit 8f06a6c) [av skip]

tkelman mentioned this pull request Sep 7, 2016

Temporarily disable SSH tests until we can get them working on buildbots #18397

Merged

tkelman added a commit that referenced this pull request Sep 8, 2016

Temporarily disable SSH tests until we can get them working on buildb…

341e436

…ots (#18397) ref #17651 (comment) (cherry picked from commit 8f06a6c) [av skip]

tkelman mentioned this pull request Sep 8, 2016

Fix SSH tests so they pass on buildbots, and re-enable them #18403

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test LibGit2 SSH authentication #17651

Test LibGit2 SSH authentication #17651

Keno commented Jul 27, 2016

tkelman commented Jul 27, 2016

Keno commented Jul 27, 2016

Keno commented Aug 2, 2016

Keno commented Aug 5, 2016

tkelman Aug 5, 2016

Keno Aug 5, 2016

tkelman Aug 6, 2016

Keno commented Aug 6, 2016

tkelman commented Aug 6, 2016

Keno commented Aug 6, 2016

tkelman commented Aug 6, 2016

Keno commented Aug 6, 2016

wildart Aug 6, 2016

Keno Aug 6, 2016

tkelman Aug 7, 2016

wildart Aug 7, 2016

tkelman Aug 7, 2016

wildart commented Aug 6, 2016 •

edited

Loading

tkelman commented Aug 8, 2016 •

edited

Loading

Keno commented Aug 8, 2016

tkelman commented Aug 10, 2016

wildart commented Aug 10, 2016

Keno commented Aug 10, 2016

wildart commented Aug 10, 2016

Keno commented Aug 10, 2016

tkelman commented Aug 11, 2016

Keno commented Aug 11, 2016

Test LibGit2 SSH authentication #17651

Test LibGit2 SSH authentication #17651

Conversation

Keno commented Jul 27, 2016

tkelman commented Jul 27, 2016

Keno commented Jul 27, 2016

Keno commented Aug 2, 2016

Keno commented Aug 5, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Keno commented Aug 6, 2016

tkelman commented Aug 6, 2016

Keno commented Aug 6, 2016

tkelman commented Aug 6, 2016

Keno commented Aug 6, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wildart commented Aug 6, 2016 • edited Loading

tkelman commented Aug 8, 2016 • edited Loading

Keno commented Aug 8, 2016

tkelman commented Aug 10, 2016

wildart commented Aug 10, 2016

Keno commented Aug 10, 2016

wildart commented Aug 10, 2016

Keno commented Aug 10, 2016

tkelman commented Aug 11, 2016

Keno commented Aug 11, 2016

wildart commented Aug 6, 2016 •

edited

Loading

tkelman commented Aug 8, 2016 •

edited

Loading