Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add path filtering to build session client #33859

Merged
merged 2 commits into from
Jul 6, 2017

Conversation

tonistiigi
Copy link
Member

This PR enables path filtering capabilities to the file send stream so that it can be used in the future for the builder to only ask the paths that are actually needed or for transferring individual files(for example Dockerfile). This was described as follow-up in #32677 as optional but it occurred to me that if a client doesn't have support for this then future daemons would need to detect if clients have support for this or not(possible but harder to maintain). The fsutil package already had support for this but it was not enabled in session/filesync.

As there isn't a daemon feature that uses it yet I added a test helper that can be used to test the full lifecycle of the session in another commit. Hopefully this will become useful in the future for testing other similar features as well.

@thaJeztah @tiborvass

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -141,7 +141,7 @@ func (bm *BuildManager) initializeClientSession(ctx context.Context, cancel func
if options.RemoteContext == remotecontext.ClientSessionRemote {
st := time.Now()
csi, err := NewClientSessionSourceIdentifier(ctx, bm.sg,
options.SessionID, []string{"/"})
options.SessionID)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this can probably be on a single line now

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>

// HandleConn handles an incoming raw connection
func (sm *Manager) HandleConn(ctx context.Context, conn net.Conn, opts map[string][]string) error {
sm.mu.Lock()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think sm's mu is being locked twice w/o unlocking in between:
One in line 56 and second one here.

Copy link
Member Author

@tonistiigi tonistiigi Jul 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ripcurld0 HandleHTTPRequest does not call this function but directly the one that assumes lock has already been taken.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes sorry, my bad. I didn't notice I am mixing between handleConn and HandleConn.

Copy link
Member

@boaz0 boaz0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🌵


dialer := session.Dialer(testutil.TestStream(testutil.Handler(m.HandleConn)))

g, ctx := errgroup.WithContext(context.Background())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL errgroup :P

@tiborvass
Copy link
Contributor

LGTM

@vieux
Copy link
Contributor

vieux commented Jul 5, 2017

@tonistiigi test timed out on windows, maybe it's related ?

@tonistiigi
Copy link
Member Author

@vieux I can't see how it could be as it doesn't do anything with build.

For the reference:

11:32:27 --- FAIL: Test (4018.35s)
11:32:27 panic: DockerSuite.TestStartReturnCorrectExitCode test timed out after 10m0s [recovered]
11:32:27 	panic: DockerSuite.TestStartReturnCorrectExitCode test timed out after 10m0s
11:32:27 
11:32:27 goroutine 33 [running]:
11:32:27 testing.tRunner.func1(0xc0423aeb60)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:622 +0x2a4
11:32:27 panic(0xc77760, 0xc04200e160)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/runtime/panic.go:489 +0x2dd
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.(*suiteRunner).runTest(0xc042422120, 0xc042235ce0, 0xc0423f0000)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:875 +0x1aa
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.(*suiteRunner).run(0xc042422120, 0x1151a88)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:621 +0xfb
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.Run(0xcf8e20, 0x1151a88, 0xc04241e050, 0xccb180)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:100 +0x54
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.RunAll(0xc04241e050, 0x3)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:92 +0xf4
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.TestingT(0xc0423aeb60)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:80 +0x415
11:32:27 github.com/docker/docker/integration-cli.Test(0xc0423aeb60)
11:32:27 	c:/gopath/src/github.com/docker/docker/integration-cli/check_test.go:81 +0x246
11:32:27 testing.tRunner(0xc0423aeb60, 0xd45ec0)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:657 +0x9d
11:32:27 created by testing.(*T).Run
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:697 +0x2d1
11:32:27 exit status 2
11:32:27 FAIL	github.com/docker/docker/integration-cli	4018.409s
11:32:27 
11:32:27 
11:32:27 ERROR: Failed 'ERROR: Integration tests failed at 07/05/2017 11:32:27. Duration:01:07:09.3901763' at 07/05/2017 11:32:27
11:32:27 
11:32:27 
11:32:27 
11:32:27 INFO: Tidying up at end of run

@thaJeztah
Copy link
Member

All green, merging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants