[upgrade] Bump minikube, k8s, PHP and Laravel versions #325

cuppett · 2023-02-21T09:58:22Z

Attempting to resolve issues with Laravel 10 support. Bumping various versions in the package.

This PR updates the following things:

Kubernetes version 1.22 removed, 1.25 added
Kubernetes 1.23->1.25 bumped to current minor versions
Add Laravel 10
Add PHP 8.2 to the testing matrix (Laravel 10 only)
Fixes a couple small issues with some tests (optional kube features, image/registry expansion)
Update PodDisruptionBudget default and tests to v1 (from v1beta1)
Update HorizontalPodAutoscaler default and tests to v2 (from v2beta2)
Update CronJob default and tests to v1 (from v1beta1)

Resolves #324

cuppett · 2023-02-21T10:35:42Z

Looks like we can't add 1.26.1 without figuring out this with the github action (related to dockershim removal?):

stderr:
W0221 10:26:39.628460    5650 initconfiguration.go:119] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/var/run/cri-dockerd.sock". Please update your configuration!
	[WARNING Swap]: swap is enabled; production deployments should disable swap unless testing the NodeSwap feature gate of the kubelet
	[WARNING FileExisting-socat]: socat not found in system path
	[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher

* Suggestion: Check output of 'journalctl -xeu kubelet', try passing --extra-config=kubelet.cgroup-driver=systemd to minikube start
* Related issue: https://github.com/kubernetes/minikube/issues/4172
Error: Command failed: sudo -E /home/runner/work/_temp/minikube start --vm-driver=none  --kubernetes-version v1.26.1 
    at checkExecSyncError (node:child_process:828:11)
    at Object.execSync (node:child_process:899:15)
    at logExecSync (/home/runner/work/_actions/manusa/actions-setup-minikube/v2.7.2/src/exec.js:8:17)
    at install (/home/runner/work/_actions/manusa/actions-setup-minikube/v2.7.2/src/install.js:25:3)
    at async run (/home/runner/work/_actions/manusa/actions-setup-minikube/v2.7.2/src/index.js:16:3) {
  status: 109,
  signal: null,
  output: [ null, null, null ],
  pid: 2989,
  stdout: null,
  stderr: null
}
Error: Command failed: sudo -E /home/runner/work/_temp/minikube start --vm-driver=none  --kubernetes-version v1.26.1

cuppett · 2023-02-21T11:45:50Z

Still working on this. Would like to investigate the following two things a bit more:

Add back PHP 8.0 for Laravel 9 only
Get 1.26.1 working

* Setting Kubernetes versions to current * Adding Laravel 10 * Update location of sealed secrets CRD * Add PHP 8.2 * Ensuring we match the supported PHP versions for the versions of Laravel. See also: https://laravel.com/docs/10.x/releases#support-policy

The actual number of annotations added by the system could vary. This test actually dependent on whether a feature gate is enabled or not. Just test for the presence of our target annotations and ignore the rest. See also: https://kubernetes.io/blog/2022/12/29/scalable-job-tracking-ga/#how-do-i-use-this-feature

Ensuring the image and version are checked, but not necessarily the resolution or substituions due to full qualification. Saw the following error with KIND: --- Expected +++ Actual @@ @@ -'mysql:5.7' +'docker.io/library/mysql:5.7'

cuppett · 2023-02-22T12:32:33Z

I don't know I'm familiar enough with how GitHub actions work to get the 1.26 minicube up and running, may also come for free if we just wait a couple weeks. I can carve that into a separate issue.

Braunson · 2023-03-09T15:12:54Z

Bumping this, looks ready to merge?

Set timeout to 15.

Attempting public.ecr.aws to avoid rate limit on pulls

Can't use an undefined value as an ARRAY reference at /usr/local/lib/perl5/5.36.0/Math/BigInt/Calc.pm line 1049.

You are using the deprecated option "--no-suggest". It has no effect and will break in Composer 3.

…iate On KIND, the standard storageclass is there, but the binding mode is WaitForFirstConsumer. This lets the test suite progress through this test without issue when we're working against hosts or clusters configured like that.

Annotations could get added by different admission plugins or API implementations. No reason to check for empty set here, createOrUpdate() with an empty annotation array won't actually remove any or guarantee that the array back after save is empty.

* Replace manusa/actions-setup-minikube with medyagh/setup-minikube (from minikube documentation) * Explicit containerd runtime with dockershim removal * Updated README * Bump patch versions of current minor releases

Issue here likely due to speed of test versus timing in VM. Trying to eliminate this timing issue in this test. Unclear if 409s should be resolved in client code with a loop or in the library.

It cannot be guaranteed we get the returned values in the same lines. This test was fragile for where additional empty lines (non-lines) were emitted. Reducing to a single string of values and ensuring we get all the output.

Also, selectors and queries for tier=backend would pick up MySQL pods from other tests. Updated annotation as "compute" instead of "backend" to keep this test separate.

…ng job Previous change to simpler Pi calculation exposed timing issue with test. Rather than make VM/CPU work harder or create potential for overflow again, sleep. Then, fixed lookup logic.

cuppett · 2023-03-25T18:24:43Z

This is ready to merge. I was also able to get the tests fully passing (required some fixes to tests and library) on the current Kubernetes versions (1.24->1.26).

The newest commits:

Drop 1.23, Add 1.26
Switch to using the GH action for minikube from their documentation
Add a timeout to the suite
Change pulls to explicitly pull from AWS public ECR (same images, avoid Docker rate limiting)
Tolerate running the tests on local KIND (storageclass binding is different)
Fixes flakes in some tests to improve the success rates

See test results here: https://github.com/cuppett/php-k8s/actions/runs/4520520534

cuppett · 2023-03-25T18:24:53Z

@rennokki kindly see above and let me know if this needs anything else to get merged and into a version. Happy to help this project along also if you need beyond this ticket. Thanks!

cuppett

Comments for a few explanations (that may not be obvious from the code commit messages and comments).

.github/workflows/ci.yml

cuppett · 2023-03-25T21:05:40Z

phpunit.xml

I'm not sure anything changed in this file, but either running phpunit or loading in PhpStorm commandeered and restructured this whole thing. 🤷

src/Kinds/K8sCronJob.php

tests/ContainerTest.php

cuppett · 2023-03-25T21:12:57Z

tests/ContainerTest.php

@@ -95,7 +95,7 @@ public function test_container_build()
        $container->removeEnv();

        $this->assertFalse($container->isReady());
-        $this->assertEquals('nginx:1.4', $container->getImage());
+        $this->assertStringEndsWith('nginx:1.23', $container->getImage());


The change to assertStringEndsWith is useful to ignore the qualified repository bit (regardless of which one it was).

Even in the unqualified case, I could see in KIND that nginx:1.4 would come back with an assertion fault because it got filled in with docker.io/library/nginx:1.4 when queried back out.

cuppett · 2023-03-25T21:13:45Z

tests/CronJobTest.php

-            ->setImage('perl')
-            ->setCommand(['perl',  '-Mbignum=bpi', '-wle', 'print bpi(2000)']);
+            ->setImage('public.ecr.aws/docker/library/perl')
+            ->setCommand(['perl',  '-Mbignum=bpi', '-wle', 'print bpi(200)']);


The newer Perls were throwing errors on 2000 overflowing BigNumber or something.

cuppett · 2023-03-25T21:15:49Z

tests/CronJobTest.php

-        $pi = K8s::container()
-            ->setName('pi')
-            ->setImage('perl', '5.34.0')
-            ->setCommand(['perl',  '-Mbignum=bpi', '-wle', 'print bpi(2000)']);
+        $busybox = K8s::container()
+            ->setName('busybox-exec')
+            ->setImage('public.ecr.aws/docker/library/busybox')
+            ->setCommand(['/bin/sh', '-c', 'sleep 30']);


A change similar to this may be more useful (and save a little CPU) where we don't actually care about burning CPU for the test and just need a pod to run for some predictable period of time.

That's what happened and was needed here; shortening the perl computation meant a Job wasn't running long enough for the test to pick it up and validate it in the way it wanted, so I put it back into running with the persistent sleep I saw in another test.

cuppett · 2023-03-29T23:38:14Z

Happy to have folks try it out and post back how it works:

https://getcomposer.org/doc/05-repositories.md#using-private-repositories

    "require": {
        "renoki-co/php-k8s": "dev-master",
    },
    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/cuppett/php-k8s"
        }
    ]

rennokki · 2023-04-01T11:26:07Z

@cuppett This looks awesome. 👍🏼 Let's wait for the tests to see what's going on and I'll be looking in the diff once again.

I appreciate it a lot! 🥺

codecov · 2023-04-01T11:36:35Z

Codecov Report

Patch coverage: 100.00% and project coverage change: -1.05 ⚠️

Comparison is base (0112d92) 95.21% compared to head (343f9be) 94.17%.

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #325      +/-   ##
============================================
- Coverage     95.21%   94.17%   -1.05%     
- Complexity      698      700       +2     
============================================
  Files            69       70       +1     
  Lines          1631     1837     +206     
============================================
+ Hits           1553     1730     +177     
- Misses           78      107      +29

Impacted Files	Coverage Δ
src/Kinds/K8sHorizontalPodAutoscaler.php	`100.00% <ø> (ø)`
src/Kinds/K8sPodDisruptionBudget.php	`100.00% <ø> (ø)`
src/Kinds/K8sStatefulSet.php	`100.00% <ø> (ø)`
src/Kinds/K8sCronJob.php	`96.15% <100.00%> (+0.15%)`	⬆️
src/Kinds/K8sJob.php	`100.00% <100.00%> (ø)`

... and 26 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

rennokki · 2023-04-01T14:00:54Z

@cuppett Awesome job! Thank you 🙏🏼

cuppett · 2023-04-01T14:15:23Z

@cuppett Awesome job! Thank you 🙏🏼

You bet! Great library, thanks for it.

cuppett marked this pull request as draft February 21, 2023 11:44

cuppett added 5 commits February 21, 2023 20:18

Update: v1beta1/v2beta2 -> v1/v2

f64897a

Fix: Tests should check image and version

ebd47c6

Ensuring the image and version are checked, but not necessarily the resolution or substituions due to full qualification. Saw the following error with KIND: --- Expected +++ Actual @@ @@ -'mysql:5.7' +'docker.io/library/mysql:5.7'

Resolving style issues

2bc06d1

cuppett marked this pull request as ready for review February 22, 2023 12:30

cuppett added 13 commits March 25, 2023 13:51

Add: Timeout for each run of the suite

b440d04

Set timeout to 15.

Explicit registry for pulls

3a4197f

Attempting public.ecr.aws to avoid rate limit on pulls

Updating phpunit.xml

9d8c821

Fix: error with too big number on newer perl

50c3bd3

Can't use an undefined value as an ARRAY reference at /usr/local/lib/perl5/5.36.0/Math/BigInt/Calc.pm line 1049.

Fix: Removing the --no-suggest option

d0836a9

You are using the deprecated option "--no-suggest". It has no effect and will break in Composer 3.

Fix: Missing ext-yaml requirement in composer.json

d321c30

Fix: Unnecessary/error-prone check

c78ffe6

Annotations could get added by different admission plugins or API implementations. No reason to check for empty set here, createOrUpdate() with an empty annotation array won't actually remove any or guarantee that the array back after save is empty.

Bump to 1.26, drop 1.23

70e9c02

* Replace manusa/actions-setup-minikube with medyagh/setup-minikube (from minikube documentation) * Explicit containerd runtime with dockershim removal * Updated README * Bump patch versions of current minor releases

Fix: PodDisruptionBudgetTest Consistent 409s

73acf9a

Issue here likely due to speed of test versus timing in VM. Trying to eliminate this timing issue in this test. Unclear if 409s should be resolved in client code with a loop or in the library.

Fix: K8sJob testing activepods instead of completiontime for completed()

35fcf56

Also, selectors and queries for tier=backend would pick up MySQL pods from other tests. Updated annotation as "compute" instead of "backend" to keep this test separate.

Fix: K8sCronJob needed to find active Job by name and also have runni…

b23f408

…ng job Previous change to simpler Pi calculation exposed timing issue with test. Rather than make VM/CPU work harder or create potential for overflow again, sleep. Then, fixed lookup logic.

StyleCI fixups

c9ba63f

cuppett commented Mar 25, 2023

View reviewed changes

rennokki added 3 commits April 1, 2023 15:31

Updated packages

ed88422

Update ci.yml

61954fa

Update composer.json

343f9be

rennokki changed the title ~~Bump minikube, k8s, php and laravel versions~~ [upgrade] Bump minikube, k8s, PHP and Laravel versions Apr 1, 2023

rennokki merged commit a86dff7 into renoki-co:master Apr 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[upgrade] Bump minikube, k8s, PHP and Laravel versions #325

[upgrade] Bump minikube, k8s, PHP and Laravel versions #325

cuppett commented Feb 21, 2023 •

edited

Loading

cuppett commented Feb 21, 2023

cuppett commented Feb 21, 2023

cuppett commented Feb 22, 2023

Braunson commented Mar 9, 2023

cuppett commented Mar 25, 2023

cuppett commented Mar 25, 2023 •

edited

Loading

cuppett left a comment

cuppett Mar 25, 2023

cuppett Mar 25, 2023

cuppett Mar 25, 2023

cuppett Mar 25, 2023

cuppett commented Mar 29, 2023 •

edited

Loading

rennokki commented Apr 1, 2023

codecov bot commented Apr 1, 2023 •

edited

Loading

rennokki commented Apr 1, 2023

cuppett commented Apr 1, 2023

[upgrade] Bump minikube, k8s, PHP and Laravel versions #325

[upgrade] Bump minikube, k8s, PHP and Laravel versions #325

Conversation

cuppett commented Feb 21, 2023 • edited Loading

cuppett commented Feb 21, 2023

cuppett commented Feb 21, 2023

cuppett commented Feb 22, 2023

Braunson commented Mar 9, 2023

cuppett commented Mar 25, 2023

cuppett commented Mar 25, 2023 • edited Loading

cuppett left a comment

Choose a reason for hiding this comment

cuppett Mar 25, 2023

Choose a reason for hiding this comment

cuppett Mar 25, 2023

Choose a reason for hiding this comment

cuppett Mar 25, 2023

Choose a reason for hiding this comment

cuppett Mar 25, 2023

Choose a reason for hiding this comment

cuppett commented Mar 29, 2023 • edited Loading

rennokki commented Apr 1, 2023

codecov bot commented Apr 1, 2023 • edited Loading

Codecov Report

rennokki commented Apr 1, 2023

cuppett commented Apr 1, 2023

cuppett commented Feb 21, 2023 •

edited

Loading

cuppett commented Mar 25, 2023 •

edited

Loading

cuppett commented Mar 29, 2023 •

edited

Loading

codecov bot commented Apr 1, 2023 •

edited

Loading