Add podman support #570

tmds · 2020-07-02T15:25:01Z

Creating custom network fails with rootless podman because it is a priviledged operation.

tmds · 2020-07-02T15:28:28Z

Contributes to #563.

This fixes the main issue I see when using tye run on Fedora with podman.

I've minimized the podman specific bits, so also for docker this changes the way the network is configured.

cc @davidfowl

tmds · 2020-07-03T10:02:21Z

Using the "host network" doesn't permit port-mapping.

To allow port-mapping, a "pod network" can be used instead. This is to launch all containers in the same pod, and use "localhost" for inter-container communication.

Both the "host network" and "pod network" approach don't allow multiple containers to bind to the same containerPort. This is a limitation of the rootless network implementation used by podman.

@davidfowl I guess the role of "Microsoft.Tye.Proxy" containers is to allow containers to get access to an ASP.NET container which is running directly on the host (that is: not in a container)? I don't understand how that works. To what address does the ASP.NET application bind?

I will look at using a "pod network" instead of the "host network" next week.
If different modes make sense, they could be made available as sentinel values set for the network name in tye.yaml.

davidfowl · 2020-07-03T16:14:39Z

The idea is being able to talk to software running directly on the host without running that thing in a container and being able to use the host name (I don’t want to muck with host files).

tmds · 2020-07-03T19:06:53Z

How does the proxy container connect to the software running on the host?

I guess it uses the host.docker.internal hostname?

According to the Docker docs this:

connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host. This is for development purpose and will not work in a production environment outside of Docker Desktop for Windows.

I assume this is an internal IP address on some virtual bridge.

An ASP.NET Core app by default binds to a loopback address (like 127.0.0.1). I'd be surprised if the loopback address is accessible from that bridge.

Does it work like this?

If the app would bind not bind to a specific interface (like 0.0.0.0), then it would be accessible on the bridge. But it would also be accessible to other machines on the (physical) network.

davidfowl · 2020-07-03T22:14:42Z

It works fine on both windows and OS X. Linux however is where host.docker.internal doesn’t work and we use the IP address. We configure how the application binds so we bind to the appropriate address so that it all works

tmds · 2020-07-08T12:15:00Z

@davidfowl this is up for review. All tests pass except those with SkipIfPodmanAttribute. The reasons for skipping are in a comment.

  Running tests: /home/tmds/repos/tye/artifacts/bin/Microsoft.Tye.E2ETests/Debug/netcoreapp3.1/Microsoft.Tye.E2ETest.dll [netcoreapp3.1|x64]
  Running tests: /home/tmds/repos/tye/artifacts/bin/Microsoft.Tye.UnitTests/Debug/netcoreapp3.1/Microsoft.Tye.UnitTests.dll [netcoreapp3.1|x64]
  Running tests: /home/tmds/repos/tye/artifacts/bin/Microsoft.Tye.Extensions.Configuration.Tests/Debug/netcoreapp3.1/Microsoft.Tye.Extensions.Configuration.Tests.dll [netcoreapp3.1|x64]
  Tests succeeded: /home/tmds/repos/tye/artifacts/bin/Microsoft.Tye.Extensions.Configuration.Tests/Debug/netcoreapp3.1/Microsoft.Tye.Extensions.Configuration.Tests.dll [netcoreapp3.1|x64]
  Tests succeeded: /home/tmds/repos/tye/artifacts/bin/Microsoft.Tye.UnitTests/Debug/netcoreapp3.1/Microsoft.Tye.UnitTests.dll [netcoreapp3.1|x64]
  Tests succeeded: /home/tmds/repos/tye/artifacts/bin/Microsoft.Tye.E2ETests/Debug/netcoreapp3.1/Microsoft.Tye.E2ETest.dll [netcoreapp3.1|x64]

Is it possible to add a Fedora VM to CI, to run these tests on a podman based system?

tmds · 2020-07-09T08:03:11Z

Using the "host network" seems the best we can do for now.
Podman doesn't support creating user-space networks.
And localhost access is disable also, so "host.docker.internal" won't work anytime soon.

@davidfowl @rynowak @jkotalik can you take a look at the PR?

Is it feasible to add a Fedora VM to the CI so tests run on a podman based system?

davidfowl · 2020-07-11T08:15:47Z

src/Microsoft.Tye.Hosting/DockerRunner.cs

+                // Workaround podman issue: https://github.com/containers/libpod/issues/6508
+                // Fixed in podman v2.
+                bool isPodman = await DockerDetector.Instance.IsPodman.Value;
+                string restartArg = isPodman ? "always" : "unless-stopped";


What does this mean for shutdown?

This affect what happens when the system restarts.
For "docker", "unless-stopped" means only containers which weren't stopped will be started at boot.
"podman" doesn't start containers at boot. With podman v2, "always" and "unless-stopped" are aliases.

davidfowl · 2020-07-14T08:29:25Z

So podman uses the docker CLI commands but is daemonless? This change basically disables docker networking when podman is installed?

tmds · 2020-07-14T08:40:47Z

This change basically disables docker networking when podman is installed?

Yes, it's because rootless podman doesn't support creating networks.
It's the best we can do for now. When podman gets some more network features (there are open request), we can improve the implementation.

Because we're using the host network, there is no port forwarding. All ports are shared between the containers and host. So the main limitation when using podman is that we can't use the same container port in different containers.
Most images have a fixed container port, so running the same image multiple times doesn't work. For ASP.NET images there is no issue, because the container port can be controlled using ASPNETCORE_URLS.

davidfowl · 2020-07-17T16:15:12Z

The podman support is a little too implicit for my liking. I think it needs to be opt in...

davidfowl · 2020-07-17T18:17:50Z

@tmds Is it possible to run docker and podman side by side or does podman replace docker generally?

tmds · 2020-08-24T13:49:38Z

I think it needs to be opt in...

I think we need to detect and handle it under the hood, because otherwise tye just fails.
As podman gets better support for rootless networking, we can improve the implementation.

Is it possible to run docker and podman side by side or does podman replace docker generally?

podman replaces docker.

Is it feasible to add a Fedora VM to the CI so tests run on a podman based system?

Can we look into this? It is the best way to detect regressions on podman systems. I'll help fix regressions that turn up.

tmds · 2020-08-31T07:31:04Z

@davidfowl @rynowak @jkotalik can you take a look?

It seems worthwhile to add this to tye and be aware of limitations and regressions on podman-based systems like RHEL and Fedora.

jkotalik · 2020-08-31T17:15:25Z

Apologies, I'll take a look at this shortly.

jkotalik · 2020-09-14T22:36:28Z

src/Microsoft.Tye.Hosting/Model/Application.cs

@@ -29,6 +29,9 @@ public Application(FileInfo source, Dictionary<string, Service> services)

        public string? Network { get; set; }

+        // All services and application run on the container host.
+        public bool UseHostNetwork { get; set; }


I think you need to parse this in config, right? Do you intend for this to be part of tye.yaml and/or a command line arg to tye run?

Ah nvrm, you use podman existing as that check.

Little bit confusing, should this variable be called IsPodman for now?

jkotalik · 2020-09-14T22:41:37Z

src/Microsoft.Tye.Hosting/PortAssigner.cs

@@ -72,22 +92,23 @@ static int GetNextPort()
                        binding.Name ?? binding.Protocol);
                }

+                // Set ContainerPort for the first http and https port.
+                // For ASP.NET we'll match the Port when UseHostNetwork. ASPNETCORE_URLS will configure the application.
+                // For other applications, we use the default ports 80 and 443.


Why should we default to 80 and 443 for non-aspnet services?

jkotalik · 2020-09-14T22:42:33Z

src/Microsoft.Tye.Hosting/TransformProjectsIntoContainers.cs

@@ -79,18 +79,18 @@ private async Task TransformProjectToContainer(Service service, ProjectRunInfo p
                IsAspNet = project.IsAspNet
            };

-            dockerRunInfo.VolumeMappings.Add(new DockerVolume(source: project.PublishOutputPath, name: null, target: "/app"));
+            dockerRunInfo.VolumeMappings.Add(new DockerVolume(source: project.PublishOutputPath, name: null, target: "/app:z"));


Why was this changed?

tmds · 2020-09-16T06:33:46Z

@jkotalik podman just got support for rootless networking in the 2.1.0-rc1 that was released 2 days ago. I'm going to redo the PR based on that version.

tmds · 2020-10-27T08:41:47Z

I'm still looking into this. Some more features are required on podman side: being able to access localhost, and adding network aliases to containers.

jzabroski · 2020-12-03T17:05:18Z

@jkotalik Can you unmark this as approved given @tmds comments? https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/dismissing-a-pull-request-review

jkotalik · 2021-01-13T19:11:55Z

@tdykstra going to close for now as this has gotten stale. Feel free to reopen when you get the chance to work on this.

tdykstra · 2021-01-13T21:34:10Z

@jkotalik I think you meant to tag @tmds

thangchung · 2021-11-26T03:50:25Z

@jkotalik Is there any plan supporting podman for tye? These days, many people run containers on podman due to licenses for Docker for Desktop in some of the small companies. That would be great if we can support it for others, instead of only Docker.

davidfowl · 2021-11-26T05:58:46Z

@tmds AFAIK podman support was merged.

davidfowl · 2021-11-26T05:59:21Z

See #1014

tmds · 2021-11-26T06:08:06Z

AFAIK podman support was merged.

Yes. @thangchung, it is part of v0.7.0 and higher.

DockerRunner: prefer using host network over creating custom network.

2df06a3

Creating custom network fails with rootless podman because it is a priviledged operation.

tmds added 8 commits July 6, 2020 15:25

Add UseHostNetwork property to Hosting Application

a2973ca

Make tests pass

9291d97

Make PortAssigner host network friendly

6f5dd43

Remove unused SkipIfNotPodmanAttribute

d2b2caa

Move UseHostNetwork setting to PortAssigner

2b690e2

Run FrontendBackendRunTestWithDocker also on podman systems

5240f37

Enable MultiProjectDockerStoppingTests

9d9c0cf

cleanup

be553a6

tmds changed the title ~~DockerRunner: prefer using host network over creating custom network.~~ Add podman support Jul 8, 2020

tmds marked this pull request as ready for review July 9, 2020 12:15

tmds requested review from davidfowl, jkotalik and rynowak as code owners July 9, 2020 12:15

jkotalik assigned davidfowl and jkotalik Jul 9, 2020

davidfowl reviewed Jul 11, 2020

View reviewed changes

Merge branch 'master' into podman

feb5df3

jkotalik reviewed Sep 14, 2020

View reviewed changes

jkotalik approved these changes Sep 14, 2020

View reviewed changes

jkotalik closed this Jan 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add podman support #570

Add podman support #570

tmds commented Jul 2, 2020

tmds commented Jul 2, 2020

tmds commented Jul 3, 2020

davidfowl commented Jul 3, 2020

tmds commented Jul 3, 2020

davidfowl commented Jul 3, 2020

tmds commented Jul 8, 2020

tmds commented Jul 9, 2020

davidfowl Jul 11, 2020

tmds Jul 13, 2020

davidfowl commented Jul 14, 2020

tmds commented Jul 14, 2020

davidfowl commented Jul 17, 2020

davidfowl commented Jul 17, 2020

tmds commented Aug 24, 2020

tmds commented Aug 31, 2020

jkotalik commented Aug 31, 2020

jkotalik Sep 14, 2020

jkotalik Sep 14, 2020

jkotalik Sep 14, 2020

jkotalik Sep 14, 2020

tmds commented Sep 16, 2020

tmds commented Oct 27, 2020

jzabroski commented Dec 3, 2020

jkotalik commented Jan 13, 2021

tdykstra commented Jan 13, 2021

thangchung commented Nov 26, 2021 •

edited

Loading

davidfowl commented Nov 26, 2021

davidfowl commented Nov 26, 2021

tmds commented Nov 26, 2021

Add podman support #570

Add podman support #570

Conversation

tmds commented Jul 2, 2020

tmds commented Jul 2, 2020

tmds commented Jul 3, 2020

davidfowl commented Jul 3, 2020

tmds commented Jul 3, 2020

davidfowl commented Jul 3, 2020

tmds commented Jul 8, 2020

tmds commented Jul 9, 2020

davidfowl Jul 11, 2020

Choose a reason for hiding this comment

tmds Jul 13, 2020

Choose a reason for hiding this comment

davidfowl commented Jul 14, 2020

tmds commented Jul 14, 2020

davidfowl commented Jul 17, 2020

davidfowl commented Jul 17, 2020

tmds commented Aug 24, 2020

tmds commented Aug 31, 2020

jkotalik commented Aug 31, 2020

jkotalik Sep 14, 2020

Choose a reason for hiding this comment

jkotalik Sep 14, 2020

Choose a reason for hiding this comment

jkotalik Sep 14, 2020

Choose a reason for hiding this comment

jkotalik Sep 14, 2020

Choose a reason for hiding this comment

tmds commented Sep 16, 2020

tmds commented Oct 27, 2020

jzabroski commented Dec 3, 2020

jkotalik commented Jan 13, 2021

tdykstra commented Jan 13, 2021

thangchung commented Nov 26, 2021 • edited Loading

davidfowl commented Nov 26, 2021

davidfowl commented Nov 26, 2021

tmds commented Nov 26, 2021

thangchung commented Nov 26, 2021 •

edited

Loading