Add latest k8s versions in testing #26729

ChrsMark · 2021-07-06T09:36:21Z

What does this PR do?

This PR adds more k8s versions for testing.

Signed-off-by: chrismark <chrismarkou92@gmail.com>

elasticmachine · 2021-07-06T09:36:42Z

Pinging @elastic/integrations (Team:Integrations)

elasticmachine · 2021-07-06T09:56:43Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-07-07T14:10:41.056+0000
Duration: 137 min 22 sec
Commit: 5547284

Test stats 🧪

Test	Results
Failed	0
Passed	48617
Skipped	5266
Total	53883

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	48617
Skipped	5266
Total	53883

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark · 2021-07-06T12:01:47Z

It seems that default ports for proxy and scheduler have been changed in latest versions. We need to tune them accordingly for our test.

ChrsMark · 2021-07-06T12:53:42Z

I wonder why the config for proxy metricset targets 10252 port at

beats/metricbeat/module/kubernetes/test/integration.go

Line 74 in 64f6e80

"hosts": []string{"localhost:10252"},

, while the port should be 10249 like at

beats/metricbeat/module/kubernetes/_meta/config.reference.yml

Line 76 in 2a0e099

hosts: ["localhost:10249"]

.

Also we need to find a way to set the proper ports for newest versions of k8s for scheduler and controllermanger since pure http is not support anymore and we should target secure ports now (see https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/).

@jsoriano do you have any ideas on how we could check for what version we are running and set the config accordingly at

beats/metricbeat/module/kubernetes/test/integration.go

Line 79 in 64f6e80

    
           func GetSchedulerConfig(t *testing.T, metricSetName string) map[string]interface{} {

?

Signed-off-by: chrismark <chrismarkou92@gmail.com>

jsoriano

@jsoriano do you have any ideas on how we could check for what version we are running and set the config accordingly?

More than checking for the version, we might explicitly provide it via environment variables or something, the job invoking the test knows the version, so it could set an environment variable, and then in the methods to get the config we can chose what config to use depending on the version.

Other option is to configure the services to listen always in the same ports, it is possible to customize the control plane with kubeadm, so invoking kind with something like this config might do the trick for the scheduler (not tested):

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    scheduler:
      extraArgs:
        secure-port: "0"
        port: "10251"

Or we can try to enable always the new port, I guess that by setting the secure-port flag. But maybe this doesn't work if older versions don't have this flag.

(Take into account that it is not only a port change, but also a change from HTTP to HTTPS.)

jsoriano · 2021-07-06T16:15:45Z

deploy/kubernetes/Jenkinsfile.yml

@@ -18,5 +18,5 @@ stages:
            make check-no-changes;
        stage: lint
    k8sTest:
-        k8sTest: "v1.18.2,v1.17.2,v1.16.4,v1.15.7,v1.14.10"
+        k8sTest: "v1.20.7,v1.19.11,v1.18.2,v1.17.2,v1.16.4,v1.15.7,v1.14.10"


It'd be nice to support 1.21 too 🙂

kind version required an update to work with 1.21

jsoriano · 2021-07-06T16:16:17Z

deploy/kubernetes/Jenkinsfile.yml

@@ -18,5 +18,5 @@ stages:
            make check-no-changes;
        stage: lint
    k8sTest:
-        k8sTest: "v1.18.2,v1.17.2,v1.16.4,v1.15.7,v1.14.10"
+        k8sTest: "v1.20.7,v1.19.11,v1.18.2,v1.17.2,v1.16.4,v1.15.7,v1.14.10"


Maybe we can start dropping some of the old versions from the matrix? I would leave the older one we support, but we can consider removing some of the others.

I removed 1.15 and 1.16

jsoriano · 2021-07-06T17:09:16Z

@jsoriano do you have any ideas on how we could check for what version we are running and set the config accordingly?

I have been trying some things and we cannot use only the old port because the scheduler pod has a healthcheck for the new one.

Good thing about this is that, if needed, we can use the token in /var/run/secrets/kubernetes.io/serviceaccount/token to query the API and get the port from the health check.

This config works to start with both ports, also in the oldest version:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    scheduler:
      extraArgs:
        bind-address: "0.0.0.0"
        port: "10251"
        secure-port: "10259"

ChrsMark · 2021-07-07T07:25:06Z

👍🏼 @jsoriano thanks! In integrations we access the endpoint using the token too -> https://github.com/elastic/integrations/blob/master/packages/kubernetes/data_stream/scheduler/_dev/test/system/test-default-config.yml#L7.

However there we test only with latest version. I think the patch for kind will do the trick!

Signed-off-by: chrismark <chrismarkou92@gmail.com>

MichaelKatsoulis · 2021-07-07T09:50:06Z

metricbeat/module/kubernetes/controllermanager/controllermanager_integration_test.go

@@ -29,7 +29,7 @@ import (
 )

 func TestFetchMetricset(t *testing.T) {
-	config := test.GetAPIServerConfig(t, "controllermanager")
+	config := test.GetControllerManagerConfig(t, "controllermanager")


Signed-off-by: chrismark <chrismarkou92@gmail.com>

jsoriano

LGTM if it looks good to jenkins 🙂

ChrsMark · 2021-07-07T16:07:39Z

For reference: Versions of k8s should be specified as per https://github.com/kubernetes-sigs/kind/releases/tag/v0.11.1

(cherry picked from commit 1654489)

(cherry picked from commit 1654489) Co-authored-by: Chris Mark <chrismarkou92@gmail.com>

* upstream/master: (430 commits) CI: increase timeout (elastic#26764) Heartbeat: add datastream fields to synthetics (elastic#26774) Osquerybeat: Change the query timeout from 3 secs to 60 secs (elastic#26775) Remove experimental warning for inputs with variables. (elastic#26762) Add latest k8s versions in testing (elastic#26729) change type of max_bytes to ByteType (elastic#26699) [Elastic Agent] Fix broken enrollment command (elastic#26749) Update agent managed manifest to include enrolment token variable (elastic#26756) Filebeat: Ensure module pipelines compatibility with previous versions of Elasticsearch (elastic#26737) Forward port changelog for 7.13.3 (elastic#26731) to master (elastic#26754) Upgrade PyYAML dependency used for tests (elastic#26746) Add agent fleet enrolment k8s manifest (elastic#26566) CI: retry the step only (elastic#26736) Osquerybeat: Fix the configuration poll interval setting (elastic#26739) [Filebeat] Replace copy_from with templated value (elastic#26631) Reduce the verbosity of the debug log for monitoring (elastic#26583) Add instructions on testing metricbeat kubernetes module (elastic#26643) Revert "[CI] fight the flakiness with some retry option in the CI only for the Pull Requests (elastic#26617)" (elastic#26704) Packaging: linux/armv7 is not supported (elastic#26706) Cyberarkpas: Link to official docs on how to setup TLS (elastic#26614) ...

Add latest k8s versions in testing

7531423

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark requested review from jsoriano and MichaelKatsoulis July 6, 2021 09:36

ChrsMark self-assigned this Jul 6, 2021

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 6, 2021

ChrsMark added the Team:Integrations Label for the Integrations team label Jul 6, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 6, 2021

MichaelKatsoulis approved these changes Jul 6, 2021

View reviewed changes

try: norrow down versions

3e993a8

Signed-off-by: chrismark <chrismarkou92@gmail.com>

fix proxy port

9c87056

Signed-off-by: chrismark <chrismarkou92@gmail.com>

jsoriano reviewed Jul 6, 2021

View reviewed changes

Patch kind config

f3bcc01

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark requested a review from a team as a code owner July 7, 2021 07:58

botelastic bot added the Team:Automation Label for the Observability productivity team label Jul 7, 2021

More fixes

ac3aa4c

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark force-pushed the test_latest_k8s_versions branch from b68b4c0 to ac3aa4c Compare July 7, 2021 09:07

MichaelKatsoulis reviewed Jul 7, 2021

View reviewed changes

ChrsMark added 2 commits July 7, 2021 14:18

Add 1.21

c8a7144

Signed-off-by: chrismark <chrismarkou92@gmail.com>

Upgrade kind version

c846bc2

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark requested review from jsoriano and MichaelKatsoulis July 7, 2021 12:46

ChrsMark added 2 commits July 7, 2021 16:15

Add latest versions

827ca24

Signed-off-by: chrismark <chrismarkou92@gmail.com>

Fix 1.18 version

5547284

Signed-off-by: chrismark <chrismarkou92@gmail.com>

ChrsMark added the backport-v7.14.0 Automated backport with mergify label Jul 7, 2021

ChrsMark added the backport-v7.15.0 Automated backport with mergify label Jul 7, 2021

kuisathaverat approved these changes Jul 7, 2021

View reviewed changes

jsoriano approved these changes Jul 7, 2021

View reviewed changes

ChrsMark merged commit 1654489 into elastic:master Jul 7, 2021

mergify bot pushed a commit that referenced this pull request Jul 7, 2021

Add latest k8s versions in testing (#26729)

9ba7dfc

(cherry picked from commit 1654489)

mergify bot mentioned this pull request Jul 7, 2021

[7.x](backport #26729) Add latest k8s versions in testing #26767

Merged

mergify bot pushed a commit that referenced this pull request Jul 7, 2021

Add latest k8s versions in testing (#26729)

9df2172

(cherry picked from commit 1654489)

mergify bot mentioned this pull request Jul 7, 2021

[7.14](backport #26729) Add latest k8s versions in testing #26768

Merged

ChrsMark added a commit that referenced this pull request Jul 8, 2021

Add latest k8s versions in testing (#26729) (#26767)

cffdfdc

(cherry picked from commit 1654489) Co-authored-by: Chris Mark <chrismarkou92@gmail.com>

ChrsMark added a commit that referenced this pull request Jul 8, 2021

Add latest k8s versions in testing (#26729) (#26768)

b79145e

(cherry picked from commit 1654489) Co-authored-by: Chris Mark <chrismarkou92@gmail.com>

This was referenced Mar 16, 2022

Fix scheduler and controllermanager k8s tests #30870

Merged

Fix scheduler and controllermanager k8s tests #30872

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add latest k8s versions in testing #26729

Add latest k8s versions in testing #26729

ChrsMark commented Jul 6, 2021

elasticmachine commented Jul 6, 2021

elasticmachine commented Jul 6, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

ChrsMark commented Jul 6, 2021

ChrsMark commented Jul 6, 2021

jsoriano left a comment

jsoriano Jul 6, 2021

ChrsMark Jul 7, 2021

jsoriano Jul 6, 2021

ChrsMark Jul 7, 2021

jsoriano commented Jul 6, 2021

ChrsMark commented Jul 7, 2021

MichaelKatsoulis Jul 7, 2021

jsoriano left a comment

ChrsMark commented Jul 7, 2021

Add latest k8s versions in testing #26729

Add latest k8s versions in testing #26729

Conversation

ChrsMark commented Jul 6, 2021

What does this PR do?

elasticmachine commented Jul 6, 2021

elasticmachine commented Jul 6, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

ChrsMark commented Jul 6, 2021

ChrsMark commented Jul 6, 2021

jsoriano left a comment

Choose a reason for hiding this comment

jsoriano Jul 6, 2021

Choose a reason for hiding this comment

ChrsMark Jul 7, 2021

Choose a reason for hiding this comment

jsoriano Jul 6, 2021

Choose a reason for hiding this comment

ChrsMark Jul 7, 2021

Choose a reason for hiding this comment

jsoriano commented Jul 6, 2021

ChrsMark commented Jul 7, 2021

MichaelKatsoulis Jul 7, 2021

Choose a reason for hiding this comment

jsoriano left a comment

Choose a reason for hiding this comment

ChrsMark commented Jul 7, 2021

elasticmachine commented Jul 6, 2021 •

edited by jenkins-beats-ci bot

Loading