Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow overriding interface used by Flannel and refactor network creation in example Terraform configs for Hetzner #3152

Merged
merged 2 commits into from
Apr 19, 2024

Conversation

xmudrii
Copy link
Member

@xmudrii xmudrii commented Apr 19, 2024

What this PR does / why we need it:

This PR is introducing two distinct changes to improve the Hetzner integration.

Overriding Canal/Flannel interface

For a long time, we had a problem that Hetzner clusters are using the public interface to communicate between nodes. That's a huge problem both from the efficiency side (bandwidth consumption) and from the security side. It can also make problems with some other components, such as CCM and CSI.

The reason why this happens is that Hetzner servers are configured with a default route that uses the public interface. Canal/Flannel are using that default route to determine what network interface should be used for communications between nodes and VXLAN.

This can be mitigated by forcing Flannel to use another interface in two ways:

  • by setting the FLANNELD_IFACE environment variable or the --iface flag
  • by setting the FLANNELD_IFACE_REGEX environment variable or the --iface-regex flag

KubeOne didn't fully support any of those mechanisms. Even though it is already possible to edit the canal-config ConfigMap and set canal_iface which is used as a value for the FLANNELD_IFACE environment variable, that change would be overridden on the first subsequent kubeone apply run.

This problem is solved by introducing a new parameter for the Canal CNI addon called IFACE which can be used like this:

apiVersion: kubeone.k8c.io/v1beta2
kind: KubeOneCluster
...
addons:
  enable: true
  addons:
    - name: "cni-canal"
      params:
        IFACE: ens10

However, this requires knowing the interface name and this is not always possible. Instead, it might be better to use the regex to match the interface name or the interface's IP address. This is possible thanks to the FLANNELD_IFACE_REGEX option and it's exposed to KubeOne users via the IFACE_REGEX parameter:

apiVersion: kubeone.k8c.io/v1beta2
kind: KubeOneCluster
...
addons:
  enable: true
  addons:
    - name: "cni-canal"
      params:
        IFACE_REGEX: '10\.*\.10\.*'

Note: using apostrophes is required when passing regex via YAML.

Randomly generating network subnet for Hetzner

We have been using the same subnet for all Hetzner clusters (192.168.0.0/16). There are multiple problems:

  • Even though networks are different, it seems like different networks with the same subnet can collide and cause conflicts
  • /16 subnet is quite large especially if we want to run multiple E2E tests in parallel

To mitigate this, this PR refactors how we handle networks in the example Terraform configs for Hetzner. We are now randomly generating /24 subnets in the 10.100.0.0/16 range, similar to what are we doing for AWS. This is a breaking change and existing users will have to take additional steps if migrating to the new Terraform configs.

What type of PR is this?
/kind feature

Does this PR introduce a user-facing change? Then add your Release Note here:

- Canal CNI: Add `IFACE` and `IFACE_REGEX` parameters to allow explicitly selecting network interface to be used for inter-node communication and VXLAN
- [ACTION REQUIRED] Refactor example Terraform configs for Hetzner to randomly generate the private network subnet in order to support creating multiple KubeOne clusters

Documentation:

TBD

/assign @kron4eg

@kubermatic-bot kubermatic-bot added kind/feature Categorizes issue or PR as related to a new feature. docs/tbd Denotes a PR that needs documentation (change) that will be done later. dco-signoff: yes Denotes that all commits in the pull request have the valid DCO signoff message. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. do-not-merge/code-freeze Indicates that a PR should not merge because it has not been approved for code freeze yet. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 19, 2024
@xmudrii xmudrii changed the title Fix hetnzer Allow overriding interface used by Flannel and refactor network creation in example Terraform configs for Hetzner Apr 19, 2024
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
@kron4eg
Copy link
Member

kron4eg commented Apr 19, 2024

/lgtm
/approve

@kubermatic-bot kubermatic-bot added the lgtm Indicates that a PR is ready to be merged. label Apr 19, 2024
@kubermatic-bot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 4ca7bcbedd1def415aa15d3f86b46de4ac293593

@kubermatic-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kron4eg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubermatic-bot kubermatic-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 19, 2024
@kron4eg kron4eg added the code-freeze-approved Indicates a PR has been approved by release managers during code freeze. label Apr 19, 2024
@kubermatic-bot kubermatic-bot removed the do-not-merge/code-freeze Indicates that a PR should not merge because it has not been approved for code freeze yet. label Apr 19, 2024
@kron4eg kron4eg added this to the KubeOne 1.8 milestone Apr 19, 2024
@kubermatic-bot kubermatic-bot added release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Apr 19, 2024
@kubermatic-bot kubermatic-bot merged commit f21ed88 into main Apr 19, 2024
13 checks passed
@kubermatic-bot kubermatic-bot deleted the fix-hetnzer branch April 19, 2024 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. code-freeze-approved Indicates a PR has been approved by release managers during code freeze. dco-signoff: yes Denotes that all commits in the pull request have the valid DCO signoff message. docs/tbd Denotes a PR that needs documentation (change) that will be done later. kind/feature Categorizes issue or PR as related to a new feature. lgtm Indicates that a PR is ready to be merged. release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants