Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update provision.sh to start the server with auth #557

Merged
merged 10 commits into from
Oct 24, 2022

Conversation

Skarlso
Copy link
Contributor

@Skarlso Skarlso commented Oct 13, 2022

Label the PR with the kind of change this for:

kind/feature

What this PR does / why we need it:

Update provision.sh to start the server with auth.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #539

Special notes for your reviewer:

Checklist:

  • squashed commits into logical changes
  • includes documentation
  • adds unit tests
  • adds or updates e2e tests

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 13, 2022

@Callisto13 I think this is it, but is there any place else where these things have to be passed in?
Also, is it true that if you define tls_crt, you pretty much define the rest or should I check and add each individual setting one by one?

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 13, 2022

The Pr type doesn't like me. :(

hack/scripts/provision.sh Outdated Show resolved Hide resolved
@Skarlso Skarlso force-pushed the add_auth_server_option branch 2 times, most recently from 52d5851 to 0d06dbe Compare October 13, 2022 14:35
@codecov-commenter
Copy link

codecov-commenter commented Oct 13, 2022

Codecov Report

Base: 56.61% // Head: 56.53% // Decreases project coverage by -0.07% ⚠️

Coverage data is based on head (daf8ee5) compared to base (0d29198).
Patch has no changes to coverable lines.

❗ Current head daf8ee5 differs from pull request most recent head 8cef6f3. Consider uploading reports for the commit 8cef6f3 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #557      +/-   ##
==========================================
- Coverage   56.61%   56.53%   -0.08%     
==========================================
  Files          57       57              
  Lines        2768     2768              
==========================================
- Hits         1567     1565       -2     
- Misses       1060     1061       +1     
- Partials      141      142       +1     
Impacted Files Coverage Δ
pkg/queue/queue.go 93.93% <0.00%> (-6.07%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@Callisto13 Callisto13 added the kind/feature New feature or request label Oct 13, 2022
@Callisto13
Copy link
Member

can you update the help menus? and screenshot/paste in some successful calls?

@Skarlso Skarlso force-pushed the add_auth_server_option branch 3 times, most recently from c97e3fb to 5475683 Compare October 13, 2022 14:51
hack/scripts/provision.sh Outdated Show resolved Hide resolved
@Skarlso Skarlso force-pushed the add_auth_server_option branch 2 times, most recently from ab40e0b to 30236e5 Compare October 13, 2022 15:01
@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 13, 2022

I updated the help and the pass and also added it to the flintlock command, not just all. :)

I will work on adding screenshots tomorrow.

@Callisto13
Copy link
Member

👍 sounds good

eventually we will remove most (if not all) flint flags and use the config. we can then expand it to pass in any arbitrary flintlock config

can you also add in the pr description something about what a valid file looks like? i can use it when i write the security section of the docs.

@Callisto13
Copy link
Member

we can then expand it to pass in any arbitrary flintlock config

actually... on that thought, why could they not simply pass in a flintlock config file which we edit? rather than file A which we extract and write to file B?

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 13, 2022

That is also possible indeed. And would reduce the burden on the script. That's a good thought!

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 14, 2022

The only thing that might be a problem is if they define something that we already defined? Such as verbosity for example. Or should we be strict that you should only define value x, y and z? Or should we try to do a merge... we could use paste for that. Or vimdiff, or diff, or melt, or whatever to try and deduplicate settings. Or sort | uniq ( although I would have to remove the values so it only checks the name ).

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 14, 2022

@Callisto13 I made an attempt to just use the thing and paste it after the things we would set up. For deduplication, see my above comment. :)

@Callisto13
Copy link
Member

Callisto13 commented Oct 14, 2022

The only thing that might be a problem is if they define something that we already defined?

In an ideal situation they should be able to define what they like and we fill in the necessary things if they don't... but implementing that is tricky... at least in bash 😄

hack/scripts/provision.sh Outdated Show resolved Hide resolved
hack/scripts/provision.sh Outdated Show resolved Hide resolved
hack/scripts/provision.sh Outdated Show resolved Hide resolved
hack/scripts/provision.sh Outdated Show resolved Hide resolved
@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 14, 2022

Example output:

config.txt:

verbosity: 100
insecure: false

run script, output:

---
insecure: false
bridge-name: asdf
verbosity: 100
grpc-endpoint: asdf:9090
containerd-socket: asdf/containerd.sock

This is just a dummy run with the script's core functionality providing this deduplication and override by the user.

Will include actual working once I test this thing with an actual cert. :)

@Skarlso Skarlso force-pushed the add_auth_server_option branch 3 times, most recently from cc548f8 to e25261f Compare October 14, 2022 15:55
@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 19, 2022

Config file:

root@ubuntu2004:~# cat flintlock.config
insecure: false
tls-cert: /root/tls-cert.crt
tls-key: /root/tls-key.key
root@ubuntu2004:~# ./provision.sh flintlock --dev --insecure --bridge lmbr0 --grpc-address 0.0.0.0:9090 -f flintlock.config
[flintlock provision.sh] Creating containerd directory /var/lib/containerd-dev/snapshotter/devmapper
[flintlock provision.sh] Creating containerd directory /run/containerd-dev
[flintlock provision.sh] Creating containerd directory /etc/containerd
[flintlock provision.sh] All containerd directories created
[flintlock provision.sh] Installing flintlockd version latest to /usr/local/bin
[flintlock provision.sh] Flintlockd version v0.4.0 successfully installed
[flintlock provision.sh] Writing flintlockd config to /etc/opt/flintlockd/config.yaml.
[flintlock provision.sh] merging provided config file with the created one
[flintlock provision.sh] Flintlockd config saved
[flintlock provision.sh] Starting flintlockd service with /etc/systemd/system/flintlockd.service
[flintlock provision.sh] Flintlockd running at 0.0.0.0:9090 via interface eth0

Config file for flintlockd:

cat /etc/opt/flintlockd/config.yaml
---
insecure: false
grpc-endpoint: 0.0.0.0:9090
tls-cert: /root/tls-cert.crt
verbosity: 9
bridge-name: lmbr0
tls-key: /root/tls-key.key
containerd-socket: /run/containerd-dev/containerd.sock

root@ubuntu2004:~#

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 19, 2022

Interesting bit that hammertime did not return an auth failed, but rather something weird:

systemctl stop flintlockd.service
root@ubuntu2004:~# ./hammertime list -a 0.0.0.0:9090
2022/10/19 10:52:23 rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 0.0.0.0:9090: connect: connection refused"
root@ubuntu2004:~# systemctl start flintlockd.service
root@ubuntu2004:~# ./hammertime list -a 0.0.0.0:9090
2022/10/19 10:53:23 rpc error: code = Unavailable desc = connection closed before server preface received

Log from flintlockd:

root@ubuntu2004:~# systemctl status flintlockd.service
● flintlockd.service - flintlock microvm service
     Loaded: loaded (/etc/systemd/system/flintlockd.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-10-19 10:43:13 UTC; 7min ago
       Docs: https://docs.flintlock.dev/
    Process: 28709 ExecStartPre=/usr/bin/which firecracker (code=exited, status=0/SUCCESS)
    Process: 28710 ExecStartPre=/usr/bin/which flintlockd (code=exited, status=0/SUCCESS)
   Main PID: 28711 (flintlockd)
      Tasks: 7 (limit: 9443)
     Memory: 10.2M
     CGroup: /system.slice/flintlockd.service
             └─28711 /usr/local/bin/flintlockd run

Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=info msg="flintlockd grpc api server starting"
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=info msg="starting microvm controller"
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=info msg="starting microvm controller with 1 workers" controller=microvm
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=info msg="resyncing microvm specs" controller=microvm
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=trace msg="querying all microvms: map[Namespace:]" component=app controller=microvm
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=warning msg="basic authentication is DISABLED"
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=info msg="TLS is enabled"
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=debug msg="starting grpc server listening on endpoint 0.0.0.0:9090"
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=info msg="starting event listener" controller=microvm
Oct 19 10:43:13 ubuntu2004.localdomain flintlockd[28711]: time="2022-10-19T10:43:13Z" level=info msg="Starting workersnum_workers1" controller=microvm

It is running. 🤷

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 19, 2022

Re-installed without TLS to show that without specifying a config it works normally:

./provision.sh flintlock --dev --insecure --bridge lmbr0 --grpc-address 0.0.0.0:9090
[flintlock provision.sh] Creating containerd directory /var/lib/containerd-dev/snapshotter/devmapper
[flintlock provision.sh] Creating containerd directory /run/containerd-dev
[flintlock provision.sh] Creating containerd directory /etc/containerd
[flintlock provision.sh] All containerd directories created
[flintlock provision.sh] Installing flintlockd version latest to /usr/local/bin
[flintlock provision.sh] Flintlockd version v0.4.0 successfully installed
[flintlock provision.sh] Writing flintlockd config to /etc/opt/flintlockd/config.yaml.
[flintlock provision.sh] Flintlockd config saved
[flintlock provision.sh] Starting flintlockd service with /etc/systemd/system/flintlockd.service
[flintlock provision.sh] Flintlockd running at 0.0.0.0:9090 via interface eth0
root@ubuntu2004:~# ./hammertime list -a 0.0.0.0:9090
{}
root@ubuntu2004:~#

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 20, 2022

Okay, I explored everything I could with this. I even enabled validation, generate the CA cert, did everything and got that flintlock is up and running:

systemctl status flintlockd.service
● flintlockd.service - flintlock microvm service
     Loaded: loaded (/etc/systemd/system/flintlockd.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-10-20 08:09:16 UTC; 3s ago
       Docs: https://docs.flintlock.dev/
    Process: 4503 ExecStartPre=/usr/bin/which firecracker (code=exited, status=0/SUCCESS)
    Process: 4504 ExecStartPre=/usr/bin/which flintlockd (code=exited, status=0/SUCCESS)
   Main PID: 4505 (flintlockd)
      Tasks: 7 (limit: 9443)
     Memory: 10.4M
     CGroup: /system.slice/flintlockd.service
             └─4505 /usr/local/bin/flintlockd run

Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=info msg="flintlockd grpc api server starting"
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=info msg="starting microvm controller"
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=info msg="starting microvm controller with 1 workers" controller=microvm
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=info msg="resyncing microvm specs" controller=microvm
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=trace msg="querying all microvms: map[Namespace:]" component=app controller=microvm
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=warning msg="basic authentication is DISABLED"
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=info msg="TLS is enabled"
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=info msg="starting event listener" controller=microvm
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=info msg="Starting workersnum_workers1" controller=microvm
Oct 20 08:09:16 ubuntu2004.localdomain flintlockd[4505]: time="2022-10-20T08:09:16Z" level=debug msg="starting grpc server listening on endpoint 0.0.0.0:9090"

With config:

root@ubuntu2004:~# cat flintlock.config
tls-cert: /root/service.pem
tls-key: /root/service.key
tls-client-validate: true
tls-client-ca: /root/ca-cert.pem

I checked hammertime it is just forwarding the response. The interesting part is that flintlock doesn't log ANYTHING. It's as if hammertime can't even see it.

Maybe there is something wrong with the port it will run on?

config looks ok to me:

---
insecure: false
grpc-endpoint: 0.0.0.0:9090
tls-cert: /root/service.pem
tls-client-ca: /root/ca-cert.pem
tls-client-validate: true
verbosity: 9
bridge-name: lmbr0
tls-key: /root/service.key
containerd-socket: /run/containerd-dev/containerd.sock

@Callisto13
Copy link
Member

I am happy with what you have pasted here. The fault lies with hammertime probably.

The interesting part is that flintlock doesn't log ANYTHING. It's as if hammertime can't even see it.

I think i recall flintlock not logging failed auth attempts before when i was implementing this. I believe it happens at a different level (ie at the protocol level, not somewhere in flintlock itself) and we are not exposing any logs there (idek if we could, but might be nice to know who is trying to connect? something for the future)

hopefully i will get to review this soon ty 🎉

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 20, 2022

Oh yeah, that makes sense. Cool! :) Thanks. :)

hack/scripts/provision.sh Outdated Show resolved Hide resolved
hack/scripts/provision.sh Outdated Show resolved Hide resolved
Skarlso and others added 2 commits October 21, 2022 15:52
@Callisto13
Copy link
Member

what happens if they provide a config file with the yaml --- at the top of it? does it get processed okay?

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 21, 2022

Good question. I didn't realise that the settings were yaml hahaha. I think it would blow up but I'll test it.

@Callisto13
Copy link
Member

Callisto13 commented Oct 22, 2022 via email

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 24, 2022

@Callisto13 Heh, it produces this:

---
insecure: false
grpc-endpoint: 0.0.0.0:9090
tls-cert: /root/service.pem
tls-client-ca: /root/ca-cert.pem
---:
tls-client-validate: true
verbosity: 9
bridge-name: lmbr0
tls-key: /root/service.key
containerd-socket: /run/containerd-dev/containerd.sock

Incidentally, flintlock actually didn't care and started up anyways hahaha.

systemctl status flintlockd.service
● flintlockd.service - flintlock microvm service
     Loaded: loaded (/etc/systemd/system/flintlockd.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2022-10-24 06:27:13 UTC; 2min 43s ago
       Docs: https://docs.flintlock.dev/
    Process: 2174 ExecStartPre=/usr/bin/which firecracker (code=exited, status=0/SUCCESS)
    Process: 2175 ExecStartPre=/usr/bin/which flintlockd (code=exited, status=0/SUCCESS)
   Main PID: 2176 (flintlockd)
      Tasks: 8 (limit: 9443)
     Memory: 11.4M
     CGroup: /system.slice/flintlockd.service
             └─2176 /usr/local/bin/flintlockd run

Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=info msg="flintlockd grpc api server starting"
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=info msg="starting microvm controller"
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=info msg="starting microvm controller with 1 workers" controller=microvm
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=info msg="resyncing microvm specs" controller=microvm
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=trace msg="querying all microvms: map[Namespace:]" component=app controller=microvm
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=info msg="starting event listener" controller=microvm
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=info msg="Starting workersnum_workers1" controller=microvm
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=warning msg="basic authentication is DISABLED"
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=info msg="TLS is enabled"
Oct 24 06:27:13 ubuntu2004.localdomain flintlockd[2176]: time="2022-10-24T06:27:13Z" level=debug msg="starting grpc server listening on endpoint 0.0.0.0:9090"

That said, I'll put in a check to skip anything that doesn't contain a : in it.

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 24, 2022

The new fix generated the right config file:

---
insecure: false
grpc-endpoint: 0.0.0.0:9090
tls-cert: /root/service.pem
tls-client-ca: /root/ca-cert.pem
tls-client-validate: true
verbosity: 9
bridge-name: lmbr0
tls-key: /root/service.key
containerd-socket: /run/containerd-dev/containerd.sock

flintlock config file:

cat flintlock.config
---
tls-cert: /root/service.pem
tls-key: /root/service.key
tls-client-validate: true
tls-client-ca: /root/ca-cert.pem

@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 24, 2022

@Callisto13 Shall I squash the commits or will you with Github? :)

@Callisto13
Copy link
Member

i can when i merge 👍

Co-authored-by: Claudia <claudiaberesford@gmail.com>
@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 24, 2022

i can when i merge 👍

Thanks! :)

Also, this was quite a cool journey! :)

@Callisto13 Callisto13 merged commit f5363e4 into liquidmetal-dev:main Oct 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update provision.sh to start the server with auth
3 participants