feat: parallelize nodes update #1133

AhmedGrati · 2023-04-14T09:28:51Z

Resolves #1096.

netlify · 2023-04-14T09:28:58Z

✅ Deploy Preview for kubernetes-sigs-nfd ready!

Name	Link
🔨 Latest commit	`b3cfe17`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-nfd/deploys/6479c77565ad3a000871bb62
😎 Deploy Preview	https://deploy-preview-1133--kubernetes-sigs-nfd.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

marquiz

Thanks @AhmedGrati for the PR.

I think we need to throttle the max parallelism (i.e. how many concurrent go routines are generated) somehow. Say we have a cluster with 2000 nodes we don't probably want the fire 2000 goroutines at the same time

/milestone v0.14.0

PiotrProkop · 2023-04-14T10:26:32Z

Can we encorporate workqueue here? Have a flag than configures the number of consumers(which will update specific node) and we will just workqueue.Add nodes when we want to update them.

We can use https://pkg.go.dev/k8s.io/client-go/util/workqueue for example.

marquiz · 2023-04-14T14:29:44Z

Can we encorporate workqueue here?

Yeah, for example something like this

AhmedGrati · 2023-04-17T21:56:06Z

Hey @PiotrProkop, I've just come up with this design to include workqueue, and to have a goroutines pool responsible for labeling nodes.

But at the same time @marquiz, suggested that we can use SizedWaitGroup.
I wanted to gather your input here, on what should we do next.

Keep it simple as is now, and add the SizedWaitGroup.
Implement a more sophisticated solution and add the workqueue package.

AhmedGrati · 2023-04-19T08:13:23Z

ping @marquiz @PiotrProkop

PiotrProkop · 2023-04-19T09:51:02Z

I think decoupling labeling into separate workqueue is better approach for the future. It would move labeling responsibility into separate component and we would have to just Add labeling request from the main loop.

On the other hand I know that implementing it is non trivial and I think SizedWaitGroup will also work but we would just have to make sure that we move all labeling into single SizedWaitGroup or single ErrGroup.

@marquiz WDYT?

marquiz · 2023-04-22T08:50:37Z

I think decoupling labeling into separate workqueue is better approach for the future. It would move labeling responsibility into separate component and we would have to just Add labeling request from the main loop.

On the other hand I know that implementing it is non trivial and I think SizedWaitGroup will also work but we would just have to make sure that we move all labeling into single SizedWaitGroup or single ErrGroup.

I agree with @PiotrProkop here. Using workqueues would be more future proof and architecturally cleaner for the long run.

However, as @PiotrProkop said it's not trivial and I think lot more work (including tests) than the simple-and-stupid SizedWaitGroup. Using SizedWaitGroup should be safe and simple (currently) as there is only one place (and goroutine) from where we do labeling.

Thus, I'm fine with either approach. You can choose 😉

AhmedGrati · 2023-04-22T16:36:25Z

Thanks for your input @marquiz @PiotrProkop, I'll implement it using workqueue.

pkg/nfd-master/node-updater-pool.go

AhmedGrati · 2023-06-02T09:13:51Z

@marquiz any idea why is this failing?

pkg/nfd-master/node-updater-pool.go

pkg/nfd-master/nfd-master.go

AhmedGrati · 2023-06-02T09:36:04Z

/retest-required

pkg/nfd-master/nfd-master.go

marquiz · 2023-06-02T09:42:03Z

@AhmedGrati also one with: could you clean up the commit message 😇

pkg/nfd-master/node-updater-pool.go

This PR aims to optimize the process of updating nodes with corresponding features. In fact, previously, we were updating nodes sequentially even though they are independent from each other. Therefore, we integrated new components: LabelersNodePool which is responsible for spininng a goroutine whenever there's a request for updating nodes, and a Workqueue which is responsible for holding nodes names that should be updated. Signed-off-by: AhmedGrati <ahmedgrati1999@gmail.com>

marquiz

Good job and thanks for the persistence on this @AhmedGrati. I don't think I have anything to add for now -> approved

@PiotrProkop could you take look, with fresh eyes?

marquiz

Good job and thanks for the persistence on this @AhmedGrati. I don't think I have anything to add for now -> approved

@PiotrProkop could you take look, with fresh eyes?

k8s-ci-robot · 2023-06-02T10:44:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AhmedGrati, marquiz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [marquiz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2023-06-02T10:44:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AhmedGrati, marquiz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [marquiz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov · 2023-06-02T11:07:30Z

Codecov Report

Merging #1133 (b3cfe17) into master (d64398f) will increase coverage by 0.49%.
The diff coverage is 68.83%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1133      +/-   ##
==========================================
+ Coverage   28.72%   29.22%   +0.49%     
==========================================
  Files          51       52       +1     
  Lines        7139     7219      +80     
==========================================
+ Hits         2051     2110      +59     
- Misses       4871     4883      +12     
- Partials      217      226       +9

Impacted Files	Coverage Δ
pkg/nfd-master/node-updater-pool.go	`57.69% <57.69%> (ø)`
pkg/nfd-master/nfd-master.go	`43.81% <92.00%> (+1.92%)`	⬆️

PiotrProkop

/lgtm

PiotrProkop · 2023-06-02T12:25:56Z

pkg/nfd-master/node-updater-pool.go

+
+	defer queue.Done(nodeName)
+
+	if err := u.nfdMaster.nfdAPIUpdateOneNode(nodeName.(string)); err != nil {


Ok, we should create issue to track this work.

k8s-ci-robot · 2023-06-02T12:27:33Z

LGTM label has been added.

Git tree hash: 046a11ad55b9f94015cfdd23dff74551e04792d3

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [node-feature-discovery](https://github.com/kubernetes-sigs/node-feature-discovery) | minor | `0.13.4` -> `0.14.0` | --- ### Release Notes <details> <summary>kubernetes-sigs/node-feature-discovery (node-feature-discovery)</summary> ### [`v0.14.0`](https://github.com/kubernetes-sigs/node-feature-discovery/releases/tag/v0.14.0) [Compare Source](kubernetes-sigs/node-feature-discovery@v0.13.4...v0.14.0) #### What's new ##### NodeFeature API The [NodeFeature](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/usage/custom-resources.html#nodefeature) API is now enabled by default. The new CRD-based API replaces the previous gRPC-based communication between nfd-master and nfd-worker and, reducing network traffic and allows changes in NodeFeatureRules to take effect immediately (independent of the sleep-interval of nfd-worker). NodeFeature API can also be used to implement 3rd party extensions, see [customization guide](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/usage/customization-guide#nodefeature-custom-resource) for more details. Garbage collection of stale NodeFeature objects was added in the form of nfd-gc daemon. The gRPC API is now deprecated and will be removed in a future release. The related command-line flags are also deprecated (and don't have any effect when NodeFeature API is in use): - nfd-master: `-ca-file`, `-cert-file`, `-key-file`, `-port`, `-verify-node-name` - nfd-worker: `-ca-file`, `-cert-file`, `-key-file`, `-server`, `-server-name-override` ##### Metrics NFD now provides Prometheus metrics for better observability. Also, the Helm and kustomize deployments support enabling metrics collection with the [Prometheus operator](https://github.com/prometheus-operator/prometheus-operator). See the [documentation](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/deployment/metrics.html) for more information about the available metrics and deployment instructions. ##### Hooks disabled by default The deprecation of nfd-worker [hooks](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/usage/customization-guide.html#hooks) continues, disabling them by default in v0.14. Potential users of hooks are encouraged to switch to use the NFD CRDs ([NodeFeature](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/usage/customization-guide.html#nodefeature-custom-resource) and [NodeFeatureRule](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/usage/customization-guide.html#nodefeaturerule-custom-resource)) or [feature files](https://kubernetes-sigs.github.io/node-feature-discovery/master/usage/customization-guide.html#feature-files). Hooks can still be enabled with the [`sources.local.hooksEnabled`](https://kubernetes-sigs.github.io/node-feature-discovery/master/reference/worker-configuration-reference.html#sourceslocalhooksenabled) configuration option. ##### Feature files **Expiry time:** NFD now supports specifying an expiry time for the features specified in a feature file, providing better lifecycle management for the feature labels. See the [documentation](https://kubernetes-sigs.github.io/node-feature-discovery/master/usage/customization-guide.html#input-format) for more details. **Size limit:** There is now a 64kB size limit for feature files. ##### Miscellaneous ##### NodeFeatureRule API Dynamic values for labels is now supported by using the `@` notation, see [documentation](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/usage/customization-guide.html#labels) for more details. ##### NFD-Master - support for leader election was added, enabling high-availability deployments with multiple-replicas of nfd-master (with the NodeFeature API enabled) - dynamically configurable logging parameters via the config file - configurable resync period for the CRD controller - parallelized node updates, speeding up simultaneous updates of large number of nodes (e.g. update in NodeFeatureRules in a big cluster), can be controlled with the [`-nfd-api-parallelism`](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/reference/master-commandline-reference.html#-nfd-api-parallelism) flag ##### CPU features Detection of Intel TDX guests is now supported. ##### Logging The project was migrated structured logging, making log messages more consistent, better machine parseable and enables future improvements in logging. ##### Support policy The project now officially documented it's supported versions and deprecation policy, see the [documentation](https://kubernetes-sigs.github.io/node-feature-discovery/v0.14/reference/versions.html) for details. #### List of PRs - test/e2e: use proper context ([#1154](kubernetes-sigs/node-feature-discovery#1154)) - deps: Update kubernetes to v1.27.1 ([#1155](kubernetes-sigs/node-feature-discovery#1155)) - generate: update k8s code-generator to v0.27.1 ([#1156](kubernetes-sigs/node-feature-discovery#1156)) - generate: update protoc to v22.3 ([#1157](kubernetes-sigs/node-feature-discovery#1157)) - generate: update controller-gen to v0.11.3 ([#1158](kubernetes-sigs/node-feature-discovery#1158)) - generate: update mockery to v2.25.1 ([#1159](kubernetes-sigs/node-feature-discovery#1159)) - nfd-master: support noPublish with -prune ([#1161](kubernetes-sigs/node-feature-discovery#1161)) - nfd-master: fix -prune ([#1160](kubernetes-sigs/node-feature-discovery#1160)) - nfd-master: don't create emtpy annotations ([#1166](kubernetes-sigs/node-feature-discovery#1166)) - nfd-master: fix a crash when processing NodeFeatureRules ([#1173](kubernetes-sigs/node-feature-discovery#1173)) - pkg/nfd-master/nfd-master.go: Fix typo ([#1171](kubernetes-sigs/node-feature-discovery#1171)) - nfd-master: reject malformed extended resource dynamic capacity assignment ([#1169](kubernetes-sigs/node-feature-discovery#1169)) - go.mod: update deps ([#1178](kubernetes-sigs/node-feature-discovery#1178)) - OWNERS: add ArangoGutierrez as an approver ([#1180](kubernetes-sigs/node-feature-discovery#1180)) - feat: add master resync period configurability ([#1139](kubernetes-sigs/node-feature-discovery#1139)) - nfd-topology-updater: fix wrong kubelet_internal_checkpoint path and compare basename to full path ([#1167](kubernetes-sigs/node-feature-discovery#1167)) - docs: add missing .md suffix to internal references ([#1189](kubernetes-sigs/node-feature-discovery#1189)) - nfd-master: log node name when processing NodeFeatureRules ([#1191](kubernetes-sigs/node-feature-discovery#1191)) - scripts/test-infra: provide PR info to codecov ([#1194](kubernetes-sigs/node-feature-discovery#1194)) - Match usage and example for prepare-release.sh ([#1196](kubernetes-sigs/node-feature-discovery#1196)) - apis/nfd: add unit tests for Feature type ([#1190](kubernetes-sigs/node-feature-discovery#1190)) - Update README to v0.13.1 ([#1197](kubernetes-sigs/node-feature-discovery#1197)) - scripts/test-infra: provide PR base SHA to codecov ([#1199](kubernetes-sigs/node-feature-discovery#1199)) - codecov: drop required minimum coverage ratio of a commit to 0% ([#1200](kubernetes-sigs/node-feature-discovery#1200)) - codecov: drop required minimum coverage ratio at patch level ([#1201](kubernetes-sigs/node-feature-discovery#1201)) - nfd-master: refactor api-controller object handling ([#1198](kubernetes-sigs/node-feature-discovery#1198)) - nfd-master: refactor filtering of labels, taints and ERs ([#1202](kubernetes-sigs/node-feature-discovery#1202)) - helm: fix mount for nfd-master config ([#1204](kubernetes-sigs/node-feature-discovery#1204)) - nfd-master: fix resync period config option ([#1185](kubernetes-sigs/node-feature-discovery#1185)) - deployment/helm: fix default for kubeletStateDir parameter ([#1207](kubernetes-sigs/node-feature-discovery#1207)) - deployment/kustomize: drop pod-resources mount for topology-updater ([#1208](kubernetes-sigs/node-feature-discovery#1208)) - test/e2e: refactor matching of node properties ([#1184](kubernetes-sigs/node-feature-discovery#1184)) - deployment/helm: avoid overlapping mount paths on topology-updater ([#1212](kubernetes-sigs/node-feature-discovery#1212)) - deployment/helm: user dedicated serviceaccount for topology-updater ([#1213](kubernetes-sigs/node-feature-discovery#1213)) - deployment/helm: improve handling of topologyUpdater.kubeletStateFiles ([#1211](kubernetes-sigs/node-feature-discovery#1211)) - topology-updater: use node IP in the default configz URI ([#1218](kubernetes-sigs/node-feature-discovery#1218)) - e2e: delete CRs only if found ([#1221](kubernetes-sigs/node-feature-discovery#1221)) - Add leader election for nfd-master ([#1219](kubernetes-sigs/node-feature-discovery#1219)) - Fixed typo in Header under deployment/kustomize.md ([#1222](kubernetes-sigs/node-feature-discovery#1222)) - nfd-master: use close for stop channel ([#1227](kubernetes-sigs/node-feature-discovery#1227)) - scripts/test-infra: bump golangci-lint to v1.52.2 ([#1230](kubernetes-sigs/node-feature-discovery#1230)) - nfd-master: add validation of label names and values ([#1228](kubernetes-sigs/node-feature-discovery#1228)) - Migrate to structured logging ([#1223](kubernetes-sigs/node-feature-discovery#1223)) - scripts/test-infra: add logcheck to verify script ([#1235](kubernetes-sigs/node-feature-discovery#1235)) - Update README to v0.13.2 ([#1238](kubernetes-sigs/node-feature-discovery#1238)) - github: update new-release issue template ([#1239](kubernetes-sigs/node-feature-discovery#1239)) - feat: support dynamic values for labels in the NodeFeatureRule ([#1226](kubernetes-sigs/node-feature-discovery#1226)) - feat: parallelize nodes update ([#1133](kubernetes-sigs/node-feature-discovery#1133)) - cpu: Discover TDX guests based on cpuid information ([#1240](kubernetes-sigs/node-feature-discovery#1240)) - deployment/kustomize: use a named port for nfd gRPC service ([#1243](kubernetes-sigs/node-feature-discovery#1243)) - Fix missing apostrophe for jq ([#1245](kubernetes-sigs/node-feature-discovery#1245)) - Fix a typo on nfd-master cmd ([#1244](kubernetes-sigs/node-feature-discovery#1244)) - Removal of the bases field as it is deprecated by kustomize ([#1246](kubernetes-sigs/node-feature-discovery#1246)) - Docs: Fix typo on customization-guide ([#1247](kubernetes-sigs/node-feature-discovery#1247)) - hooks: disable hooks by default from v0.14 ([#1182](kubernetes-sigs/node-feature-discovery#1182)) - Remove pkg's imported twice ([#1248](kubernetes-sigs/node-feature-discovery#1248)) - fix typo in helm chart ([#1253](kubernetes-sigs/node-feature-discovery#1253)) - Stop ticker in time to avoid memory leak ([#1255](kubernetes-sigs/node-feature-discovery#1255)) - nfd-master: check for nil references in nfdAPIUpdateAllNodes ([#1258](kubernetes-sigs/node-feature-discovery#1258)) - cpu: Take cgroupsv1 into account when reading misc.capacity ([#1265](kubernetes-sigs/node-feature-discovery#1265)) - go.mod: update kubernetes to v1.27.4 ([#1268](kubernetes-sigs/node-feature-discovery#1268)) - github: update assignees in new-release issue template ([#1274](kubernetes-sigs/node-feature-discovery#1274)) - Enable metrics via prometheus operator ([#1242](kubernetes-sigs/node-feature-discovery#1242)) - README: update to v0.13.3 ([#1276](kubernetes-sigs/node-feature-discovery#1276)) - docs: document version and deprecation policy ([#1279](kubernetes-sigs/node-feature-discovery#1279)) - docs: fix toc of topology-updater and topology-gc reference ([#1278](kubernetes-sigs/node-feature-discovery#1278)) - docs: remove useless TOCs ([#1280](kubernetes-sigs/node-feature-discovery#1280)) - Add optional labels to the podmonitor ([#1282](kubernetes-sigs/node-feature-discovery#1282)) - docs: describe supported Kubernetes versions ([#1277](kubernetes-sigs/node-feature-discovery#1277)) - docs: deprecation policy for Helm chart params ([#1283](kubernetes-sigs/node-feature-discovery#1283)) - Fix Topology Manager policy and scope not being updated after NRT creation ([#1256](kubernetes-sigs/node-feature-discovery#1256)) - generate: bump tools to their latest versions ([#1284](kubernetes-sigs/node-feature-discovery#1284)) - Improve metrics ([#1288](kubernetes-sigs/node-feature-discovery#1288)) - docs: align metrics documentation with latest changes on naming ([#1289](kubernetes-sigs/node-feature-discovery#1289)) - docs: unify formatting of NOTEs ([#1292](kubernetes-sigs/node-feature-discovery#1292)) - source/local: trim whitespace from input ([#1293](kubernetes-sigs/node-feature-discovery#1293)) - source/local: support comments in input ([#1294](kubernetes-sigs/node-feature-discovery#1294)) - nfd-master: use term node update instead of labeling ([#1291](kubernetes-sigs/node-feature-discovery#1291)) - docs: document -metrics flag in command line reference ([#1296](kubernetes-sigs/node-feature-discovery#1296)) - fix empty hugepages in some numa nodes caused no such file or directory errors ([#1287](kubernetes-sigs/node-feature-discovery#1287)) - scripts/test-infra: update logcheck tool to v0.6.0 ([#1299](kubernetes-sigs/node-feature-discovery#1299)) - scripts/test-infra: bump golangci-lint to v1.54.0 ([#1300](kubernetes-sigs/node-feature-discovery#1300)) - Update kubernetes to v1.28.0 ([#1302](kubernetes-sigs/node-feature-discovery#1302)) - docs: update github-pages gem to v228 ([#1303](kubernetes-sigs/node-feature-discovery#1303)) - topology-gc: fix Stop ([#1306](kubernetes-sigs/node-feature-discovery#1306)) - topology-gc: rename run() ([#1309](kubernetes-sigs/node-feature-discovery#1309)) - topology-gc: rename runGC to garbageCollect() ([#1310](kubernetes-sigs/node-feature-discovery#1310)) - nfd-topology-updater: add metrics support ([#1295](kubernetes-sigs/node-feature-discovery#1295)) - topology-gc: refactor unit tests ([#1307](kubernetes-sigs/node-feature-discovery#1307)) - topology-gc: move initial GC out of startNodeInformer() ([#1308](kubernetes-sigs/node-feature-discovery#1308)) - topology-gc: simplify listing of node objects ([#1311](kubernetes-sigs/node-feature-discovery#1311)) - metrics: additional metrics for nfd-master ([#1290](kubernetes-sigs/node-feature-discovery#1290)) - Garbage collection of NodeFeature objects ([#1305](kubernetes-sigs/node-feature-discovery#1305)) - topology-updater: make -version always runnable ([#1297](kubernetes-sigs/node-feature-discovery#1297)) - go.mod: update kubernetes to v1.28.1 ([#1315](kubernetes-sigs/node-feature-discovery#1315)) - Makefile: increase golangci-lint timeout to 10min ([#1320](kubernetes-sigs/node-feature-discovery#1320)) - docs: use ruby docker image for building docs ([#1319](kubernetes-sigs/node-feature-discovery#1319)) - README: update to v0.13.4 ([#1324](kubernetes-sigs/node-feature-discovery#1324)) - test: add node updater pool unit tests ([#1252](kubernetes-sigs/node-feature-discovery#1252)) - docs: nfd-updater: clarify accounting ([#1321](kubernetes-sigs/node-feature-discovery#1321)) - nfd-updater: events: enable timer-only flow ([#1325](kubernetes-sigs/node-feature-discovery#1325)) - docs: demote hooks in the customization guide ([#1326](kubernetes-sigs/node-feature-discovery#1326)) - Feat: add expiry date for feature files ([#1285](kubernetes-sigs/node-feature-discovery#1285)) - Dockerfile: bump grpc-health-probe to v0.4.19 ([#1327](kubernetes-sigs/node-feature-discovery#1327)) - e2e/test: make the nfd-gc test pass on one-node cluster ([#1328](kubernetes-sigs/node-feature-discovery#1328)) - Enable NodeFeature API by default ([#1329](kubernetes-sigs/node-feature-discovery#1329)) - tls.md: Add note ([#1332](kubernetes-sigs/node-feature-discovery#1332)) - nfd_gc_test.go: fix multiple import of same pkg ([#1333](kubernetes-sigs/node-feature-discovery#1333)) - feat: add feature file size limit ([#1335](kubernetes-sigs/node-feature-discovery#1335)) - sources/custom: convert static rules to new format ([#1336](kubernetes-sigs/node-feature-discovery#1336)) - nfd-master: add config file options for logging ([#1338](kubernetes-sigs/node-feature-discovery#1338)) - Deprecate gRPC API ([#1334](kubernetes-sigs/node-feature-discovery#1334)) - Helm: conditionally add annotations if defined ([#1331](kubernetes-sigs/node-feature-discovery#1331)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.home/nrdufour/home-ops/pulls/78 Co-authored-by: Renovate <renovate@ptinem.io> Co-committed-by: Renovate <renovate@ptinem.io>

k8s-ci-robot requested review from jjacobelli and kad April 14, 2023 09:28

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Apr 14, 2023

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 14, 2023

marquiz reviewed Apr 14, 2023

View reviewed changes

k8s-ci-robot added this to the v0.14.0 milestone Apr 14, 2023

AhmedGrati changed the title ~~feat: parallelize nodes update~~ [WIP] feat: parallelize nodes update Apr 15, 2023

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 15, 2023

AhmedGrati marked this pull request as draft May 4, 2023 14:08

AhmedGrati force-pushed the feat-parallelize-nodes-update branch from de0c0cf to d908a70 Compare May 12, 2023 21:15

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 12, 2023

AhmedGrati force-pushed the feat-parallelize-nodes-update branch 2 times, most recently from dca45f6 to 64a284e Compare May 12, 2023 21:16

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels May 12, 2023

AhmedGrati force-pushed the feat-parallelize-nodes-update branch 6 times, most recently from bdea8c1 to 2906059 Compare May 13, 2023 20:02

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 1, 2023