Add bridge support, for the varlink connection #2149

afbjorklund · 2019-01-12T13:42:47Z

Read the $PODMAN_VARLINK_BRIDGE environment variable

Similar to the one in: varlink/libvarlink#10

Works OK, with the command available in podman-machine:

$ podman-machine ssh box sudo podman pull busybox
Trying to pull docker.io/busybox:latest...Getting image source signatures
Copying blob sha256:57c14dd66db0390dbf6da578421c077f6de8e88edd0815b4caa94607ba5f4c09
 738.01 KB / 738.01 KB  0s
Copying config sha256:3a093384ac306cbac30b67f1585e12b30ab1a899374dabc3170b9bca246f1444
 1.46 KB / 1.46 KB  0s
Writing manifest to image destination
Storing signatures
3a093384ac306cbac30b67f1585e12b30ab1a899374dabc3170b9bca246f1444
$ eval $(podman-machine env --varlink)
$ varlink info
Interfaces:
  io.podman
  org.varlink.resolver

$ podman-remote images
REPOSITORY                  TAG      IMAGE ID       CREATED       SIZE
docker.io/library/busybox   latest   3a093384ac30   11 days ago   1.42 MB
$ podman-remote info
host:
  arch: amd64
  buildah_version: 1.6-dev
  cpus: 1
  distribution:
    distribution: tinycore
    version: "9.0"
  hostname: box
  kernel: 4.14.84-tinycore64
  mem_free: 827883520
  mem_total: 1040896000
  os: linux
  swap_free: 203378688
  swap_total: 0
  uptime: 8m 8.44s
insecure registries:
  registries: []
registries:
  registries:
  - docker.io
  - quay.io
store:
  containers: 4
  graph_driver_name: overlay
  graph_driver_options: ""
  graph_root: /var/lib/containers/storage
  graph_status:
    backing_filesystem: extfs
    native_overlay_diff: "true"
    supports_d_type: "true"
  images: 1
  run_root: /var/run/containers/storage

Looking forward to seeing more commands available remote!

libpod/adapter/client.go

afbjorklund · 2019-01-12T15:36:47Z

Changed podman-machine to output both variables, then user can choose:

$ varlink call io.podman.GetInfo
{
  "info": {
    "host": {
      "arch": "amd64",
      "buildah_version": "1.6-dev",
      "cpus": 1,
      "distribution": {
        "distribution": "fedora",
        "version": "29"
      },
      "hostname": "localhost",
      "kernel": "4.19.8-300.fc29.x86_64",
      "mem_free": 175480832,
      "mem_total": 1029746688,
      "os": "linux",
      "swap_free": 0,
      "swap_total": 0,
      "uptime": "24m 58.09s"
    },
    "insecure_registries": [],
    "podman": {
      "compiler": "gc",
      "git_commit": "\"0c9259a2c84f6234a29dcd78674c3f4837fd3101\"",
      "go_version": "go1.11.2",
      "podman_version": "0.12.1.1"
    },
    "registries": [
      "docker.io",
      "registry.fedoraproject.org",
      "quay.io",
      "registry.access.redhat.com",
      "registry.centos.org"
    ],
    "store": {
      "containers": 0,
      "graph_driver_name": "overlay",
      "graph_driver_options": "",
      "graph_root": "/var/lib/containers/storage",
      "graph_status": {
        "backing_filesystem": "xfs",
        "native_overlay_diff": "true",
        "supports_d_type": "true"
      },
      "images": 0,
      "run_root": "/var/run/containers/storage"
    }
  }
}
$ podman-remote info
host:
  arch: amd64
  buildah_version: 1.6-dev
  cpus: 1
  distribution:
    distribution: fedora
    version: "29"
  hostname: localhost
  kernel: 4.19.8-300.fc29.x86_64
  mem_free: 172511232
  mem_total: 1029746688
  os: linux
  swap_free: 0
  swap_total: 0
  uptime: 25m 3.85s
insecure registries:
  registries: []
registries:
  registries:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  containers: 0
  graph_driver_name: overlay
  graph_driver_options: ""
  graph_root: /var/lib/containers/storage
  graph_status:
    backing_filesystem: xfs
    native_overlay_diff: "true"
    supports_d_type: "true"
  images: 0
  run_root: /var/run/containers/storage

And keeps the podman environment variables neatly prefixed (with PODMAN_)

afbjorklund · 2019-01-12T16:16:12Z

Note that this approach is slightly different from pypodman, which prefers to do its own ssh-ing...
So there we have variables like user/host/port/identity_file/ignore_hosts, instead of using the bridge.

Here the remote socket path is hardcoded, where pypodman prefers to use variables/parameters.
Both programs are providing the location of the io.podman varlink socket, instead of using the resolver.

afbjorklund · 2019-01-12T16:26:20Z

github.com/containers/python-podman/podman/podman/libs/tunnel.py@Tunnel
vs.
github.com/varlink/python/varlink/client.py@new_with_bridge

github.com/varlink/go/varlink/bridge.go@NewBridge

rhatdan · 2019-01-13T11:23:58Z

@baude @jwhonce PTAL

libpod/adapter/client.go

baude · 2019-01-13T15:48:09Z

libpod/adapter/client.go

+	if bridge := os.Getenv("PODMAN_VARLINK_BRIDGE"); bridge != "" {
+		connection, err = varlink.NewBridge(bridge)
+	} else {
+		socket := "unix:/run/podman/io.podman"


should we define this as a const?

That is probably a good idea

baude · 2019-01-13T15:48:41Z

I added a few comments. Thanks for the excellent PR.

afbjorklund · 2019-01-13T16:37:41Z

I think it is OK to provide the socket for now, similar to using 127.0.0.1 instead of localhost...

But if I understand correctly, one is supposed to ask the varlink resolver for the address:

$ varlink call org.varlink.resolver.Resolve '{ "interface": "io.podman" }'
{
  "address": "unix:/run/podman/io.podman;mode=0600"
}

There seems to be a go implementation: func (r *Resolver) Resolve(iface string) (string, error)

It also seems that some people prefer to tunnel sockets, rather than using the bridge:

containers/podman.io#49

$ ssh -L 127.0.0.1:1234:/run/podman/io.podman 192.168.122.29

$ python -m varlink.cli call tcp:127.0.0.1:1234/io.podman.Ping {}
{
  "ping": {
    "message": "OK"
  }
}

So it should probably be both a (default) const, as well as an actual parameter ?

Read the $PODMAN_VARLINK_BRIDGE environment variable (normally looks like: "ssh user@host varlink bridge") Also respect $PODMAN_VARLINK_ADDRESS as an override, if using a different podman socket than the default. Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>

jwhonce · 2019-01-13T21:10:57Z

I think it is OK to provide the socket for now, similar to using 127.0.0.1 instead of localhost...

But if I understand correctly, one is supposed to ask the varlink resolver for the address:
$ varlink call org.varlink.resolver.Resolve '{ "interface": "io.podman" }'
{
  "address": "unix:/run/podman/io.podman;mode=0600"
}
There seems to be a go implementation: func (r *Resolver) Resolve(iface string) (string, error)

It also seems that some people prefer to tunnel sockets, rather than using the bridge:

containers/podman.io#49
$ ssh -L 127.0.0.1:1234:/run/podman/io.podman 192.168.122.29

$ python -m varlink.cli call tcp:127.0.0.1:1234/io.podman.Ping {}
{
  "ping": {
    "message": "OK"
  }
}
So it should probably be both a (default) const, as well as an actual parameter ?

I like idea of both. Covers more use cases.

rhatdan · 2019-01-17T11:53:01Z

@afbjorklund @jwhonce @baude What is the state of this PR Now?

baude · 2019-01-17T14:40:17Z

I think @afbjorklund needs to address the comment from @jwhonce and we are good to go.

afbjorklund · 2019-01-17T17:15:36Z

I only saw "like idea of both", and think I addressed the import/logrus/const feedback already ?

Currently having both alternatives, either the pypodman variables way or the varlink bridge way.

Not sure if you want to do your own ssh tunneling, or only support the "address" and the "bridge" ?

The address parameter only allows sockets, so either unix: or tcp: but no TLS (no ssh:).

afbjorklund · 2019-01-17T17:37:37Z

Like so:

$ podman-machine config
--username=root
--host=127.0.0.1
--port=37114
--identity-file=/home/anders/.local/machine/machines/box/id_rsa

$ podman-machine env
export PODMAN_USER="root"
export PODMAN_HOST="127.0.0.1"
export PODMAN_PORT="37114"
export PODMAN_IDENTITY_FILE="/home/anders/.local/machine/machines/box/id_rsa"
export PODMAN_IGNORE_HOSTS="true"
export PODMAN_MACHINE_NAME="box"
# Run this command to configure your shell: 
# eval $(podman-machine env)

$ podman-machine env --varlink
export VARLINK_BRIDGE="/usr/bin/ssh -F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@127.0.0.1 -o IdentitiesOnly=yes -i /home/anders/.local/machine/machines/box/id_rsa -p 37114 varlink bridge"
export PODMAN_VARLINK_BRIDGE="$VARLINK_BRIDGE"
# Run this command to configure your shell: 
# eval $(podman-machine env --varlink)

But I dunno, the only thing I wanted was to be able to run the commands and it "just worked".

So that you don't have to be saying "ssh user@host" or socket locations, every single command...

python3 -m varlink.cli --bridge "ssh <podman-machine>" call io.podman.Ping {}

sudo varlink call -m unix:/run/podman/io.podman/io.podman.GetInfo

baude · 2019-01-18T14:31:01Z

/approve

baude · 2019-01-18T14:31:11Z

/lgtm

openshift-ci-robot · 2019-01-18T14:31:19Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: baude

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [baude]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot requested review from mheon and vrothberg January 12, 2019 13:42

openshift-ci-robot added the size/XL label Jan 12, 2019

rhatdan reviewed Jan 12, 2019

View reviewed changes

libpod/adapter/client.go Outdated Show resolved Hide resolved

afbjorklund force-pushed the bridge branch from c1c5532 to 9ea1b92 Compare January 12, 2019 15:31

baude reviewed Jan 13, 2019

View reviewed changes

libpod/adapter/client.go Show resolved Hide resolved

baude reviewed Jan 13, 2019

View reviewed changes

libpod/adapter/client.go Show resolved Hide resolved

baude reviewed Jan 13, 2019

View reviewed changes

afbjorklund force-pushed the bridge branch from 9ea1b92 to c90e3e7 Compare January 13, 2019 17:17

openshift-ci-robot assigned baude Jan 18, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 18, 2019

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 18, 2019

openshift-merge-robot merged commit 7f19e5f into containers:master Jan 18, 2019

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 27, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add bridge support, for the varlink connection #2149

Add bridge support, for the varlink connection #2149

afbjorklund commented Jan 12, 2019 •

edited

Loading

afbjorklund commented Jan 12, 2019

afbjorklund commented Jan 12, 2019

afbjorklund commented Jan 12, 2019 •

edited

Loading

rhatdan commented Jan 13, 2019

baude Jan 13, 2019

afbjorklund Jan 13, 2019

baude commented Jan 13, 2019

afbjorklund commented Jan 13, 2019

jwhonce commented Jan 13, 2019

rhatdan commented Jan 17, 2019

baude commented Jan 17, 2019

afbjorklund commented Jan 17, 2019

afbjorklund commented Jan 17, 2019

baude commented Jan 18, 2019

baude commented Jan 18, 2019

openshift-ci-robot commented Jan 18, 2019

Add bridge support, for the varlink connection #2149

Add bridge support, for the varlink connection #2149

Conversation

afbjorklund commented Jan 12, 2019 • edited Loading

afbjorklund commented Jan 12, 2019

afbjorklund commented Jan 12, 2019

afbjorklund commented Jan 12, 2019 • edited Loading

rhatdan commented Jan 13, 2019

baude Jan 13, 2019

Choose a reason for hiding this comment

afbjorklund Jan 13, 2019

Choose a reason for hiding this comment

baude commented Jan 13, 2019

afbjorklund commented Jan 13, 2019

jwhonce commented Jan 13, 2019

rhatdan commented Jan 17, 2019

baude commented Jan 17, 2019

afbjorklund commented Jan 17, 2019

afbjorklund commented Jan 17, 2019

baude commented Jan 18, 2019

baude commented Jan 18, 2019

openshift-ci-robot commented Jan 18, 2019

afbjorklund commented Jan 12, 2019 •

edited

Loading

afbjorklund commented Jan 12, 2019 •

edited

Loading