-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
no cloud agents: gcp #67
Comments
See https://pagure.io/cloud-sig/issue/292#comment-538459 for information about how GCE needs the |
GCE wanted google-oslogin added to CL to remain at primary tier OS, so we implemented that. It shouldn't be needed for networking and the rest of the GCE stuff we can do in a container like we do on CL (although it might be worth revisiting it since we haven't in a while). oslogin itself though needs to be implemented in the os since it messes with nsswitch, pam, and sshd. We'll need to conditionally enable it on gce (could be done with Ignition, and 3.0.0 will make it easier to optionally disable it) |
FYI: rpm package reviews for oslogin rpm: https://pagure.io/fedora-server/issue/5#comment-538460 The current discussion in today's meeting was that we would possibly include the oslogin rpm and just conditionally enable it on gce. |
A little background on oslogin: "Normal" fedora probably wants the *We should be able to do it all with Ignition with spec 3.0.0 (no systemd unit necessary). Trying to do it with the 2.x.y spec is what led me to discover that files, directories, and links are not declarative. |
:( so how do we implement that functionality without the I guess it's worth asking.. Do we need to ship google_oslogin at all or can we get by without it (which is the topic of this ticket anyway, right?)? |
On CL we don't; we say "you shouldn't be toggling host bits other than when provisioning". I don't know if that's the path we want to take for FCOS or not. There's also the question of what is the default configuration and what does that look like with a managed
That's something we need to discuss with the GCE folks. For CL they said it was a requirement to be a first tier OS. |
reviews were approved.. packages should make their way into Fedora soon. Thanks @Conan-Kudo |
Bodhi updates submitted: |
The biggest blocker I've hit for using with OpenShift so far is forwarded IPs (set from instance metadata https://github.com/GoogleCloudPlatform/compute-image-packages/blob/master/google_compute_engine/distro_lib/ip_forwarding_utils.py#L78) - we have to use NLB for our front ends for masters, and so without the route being read from instance metadata and then set NLB health checks never go green. E.g. for a forwarding rule the above reads
|
Enabling OS Login requires modifying several monolithic files ( The |
What about the google-startup-script (and shutdown script) "agents"? I have external integrated VM management service I depend on that uses it. My service downloads and starts it's own agent by injecting runtime-specific details through the google-startup-script service. It can't work through cloud-init or similar, it's specific to that google-startup-script API. Is this use-case covered? |
@cevich We don't plan to support those startup/shutdown scripts. Fedora CoreOS should be configured by passing an Ignition config in userdata. That config can download and install your agent. Or, if you'd prefer to continue using your existing script, the Ignition config can install a |
You're assuming an open-source service, and that they care about supporting a one-off special case for an non-standard OS, on a major cloud platform. As a user of a a third-party service like this, that makes my choice effectively:
There's nearly zero incentive for third-parties to add the required special-case support when every other OS happily plays along whether or not agent-services are a good idea. It doesn't even have to be GCE-specific, there are plenty of other third-parties which require host-agents. If the OS makes it difficult to integrate, the OS will simply be placed on the "not supported" list and loose out in the long-run. I don't think that's the desire of the community here, whatever the specific philosophy over "agents" is. |
@cevich Fedora CoreOS is continuing the Container Linux philosophy of providing an opinionated, minimal, and reasonably legacy-free OS for running containers. Part of being opinionated is that not everyone will agree with our opinions, and that's okay. If you want flexibility beyond what Fedora CoreOS is prepared to provide, other distros (including Fedora Cloud Base) could be a great choice. Fedora CoreOS doesn't support cloud-init, whose design has unfixable race conditions. It strongly discourages installing software in the host, in favor of running all user software in containers. It favors immutable infrastructure and reprovisioning rather than configuration management. So existing tooling will already need to be adapted to work well with Fedora CoreOS. As to this bug, the principle is that provisioning setup for a machine should always be encapsulated into the Ignition config, rather than passed via a platform-specific agent. |
Thanks for the details and explanation. For me that means I won't use this for testing container run-times and related tooling...ironically as that is. That said, being SO strongly opinionated seems (IMHO) to make this OS overly difficult to use. History provides plenty of examples where difficult-use inventions, are simply not used and therefor ultimately fail. I think the principals here are "cool", and would like to see it be successful. IMHO, that probably necessitates additional flexibility of opinions. |
I hope you'll give Fedora CoreOS a try when we're a little further along; it's easier to use than you might think. 😃 (We don't have much documentation right now, which is a problem, but we're working on that.) Fedora CoreOS's opinions are pretty closely aligned with Container Linux, and they've served that community pretty well for several years now. We're trying to make things easier, not harder, honest. |
I know you are. I'm just thinking of all the "agents" out there which must run with privileges, on the host, and may not be conducive to being written into container images. This will especially be a problem in cases where the software or service is closed-source/proprietary. The sad fact is, many environments are like this, especially in government and health-care. Requiring little bits of "internal malware" if you will...because management always knows best. Possibly not an issue for Fedora, but as that rolls down into CentOS/elsewhere it will become a monumental obstacle to adoption. As in my case, the user's choice may literally be: "Ask nice and pray" 😞 We have exactly zero control/influence with what third-parties do, especially with cloud APIs that we also have no control over. |
I recently did a deep dive on the agent for GCE for some work getting Openshift to run in GCE. The use-case that I needed to solve was the L4 (aka Network Load Balancers) I came up with a proof-of-concept [1] which only runs the Network Configuration and the Clock Skew daemon. |
Yeah, I found out they were reworking this so I had stopped my rebase work. I guess it's ready now to be put into Fedora... Not that I like Go at all for this (I really, really, really don't), but at least this means it's shippable for FCOS. |
I broke the OS Login part out into #648. I'm going to close this ticket since we've got a GCP image now and no agent seems to be going fine. We can start new discussions in new tickets. |
In #12 we decided that we'd like to try to not ship cloud agents. This ticket will document investigation and strategy for shipping without a cloud agent on the google cloud platform.
See also #41 for a discussion of how to ship cloud specific bits using ignition.
The text was updated successfully, but these errors were encountered: