Optimize CI pipeline

[beni0888]

Description of the change

This PR contains changes that make it possible to split the local_e2e_tests job into multiple parallelizable chunks. From now on, the e2e_test.sh script accepts an environment variable called TESTS_GROUPS that indicates the group of tests (the chunk) to execute and falls back to all the tests in case the env var is not provided. This way, all the bootstrapping part of the script is run for whatever group of tests we want to execute, but the specific test logic of every group is only executed in case we want to execute that group.

Benefits

With these changes, we can parallelize the execution of every group of tests and reduce the total amount of time consumed by the local_e2e_tests job, and in consequence, by the whole pipeline. So far, the execution time of the local_e2e_tests job (and the whole workflow) has been reduced by ~10min in the GHA workflow (the CircleCI pipeline has not been adapted to take advantage of these changes).

Possible drawbacks

The logic of the e2e_tests.sh script is slightly more complex.

Applicable issues

• related to Migrate CI to Github actions #4436

Additional information

[netlify]

✅ Deploy Preview for kubeapps-dev canceled.

Built without sensitive environment variables

Name	Link
🔨 Latest commit	d40a654
🔍 Latest deploy log	https://app.netlify.com/sites/kubeapps-dev/deploys/6347ba313f808c000a5e35c1

[absoludity]

Thanks @beni0888 , it'll be great to have faster CI! Code looks great, though my main question is: are the test groups running concurrently against the same cluster? If so, are we sure that they cannot interfere with each other? (ie. all namespaces/deployments etc. use unique identifiers).

[absoludity]

Guessing this needs to be removed before you land? (assuming it's just for testing this branch?)

[beni0888]

Yeah, you're right

[absoludity]

We should be safe to use something like https://stackoverflow.com/a/15394738 rather than looping, shouldn't we?

[beni0888]

Yeah, I think that way the code would be more concise. I didn't know about that solution for searching items in an array, thanks for sharing!

[beni0888]

Thanks @beni0888 , it'll be great to have faster CI! Code looks great, though my main question is: are the test groups running concurrently against the same cluster? If so, are we sure that they cannot interfere with each other? (ie. all namespaces/deployments etc. use unique identifiers).

The thing is, every test group runs in a different Kind cluster spun up in a different virtual machine, so there's no risk of interference between them.