cluster/config_manager: fix double apply of configs

[andijcr]

previously, to support validation, the code would call a method and then call it again with a smp::invoke_on_all.

this pr preserve this behavior but uses invoke_on_others to eliminate the double-apply on the current shard.

to support this, the lambdas needs to be no_except_copy/move, so the parameters are taken by pointer, since they live in the parent scope

Fixes #13362
Fixes #13787

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v23.2.x
• v23.1.x
• v22.3.x

Release Notes

• none

• none

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/37979#018ae223-7888-42d3-9a05-1300186a436f

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/37979#018ae223-788b-4ca8-a781-0d18fd796c7a

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/37979#018ae223-7881-4bd5-9291-da9743a6bfdc

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/37979#018ae223-7885-4745-b692-3c8c3f2b8e92

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38095#018af071-e6b2-4eab-ab56-942be25f59ad

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38138#018af273-57f1-4502-bd0f-684480f4b8bb

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38138#018af273-57e6-4b30-a351-89b2467ba0f8

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38138#018af27d-647b-4c53-a47c-45a12e7b7690

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38138#018af27d-6472-40ec-98e0-b4b1b25c3e24

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38240#018afa53-d51d-461d-9f9c-d487df8aa5d0

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38240#018afa62-c855-4ed8-8396-1685e226142a

[andijcr]

note: i've been following admin_sever::patch_cluster_config_handler and there are some issues there as well, seems like a configuration is available in the shard before it has been propagated, and that is in contrast with the purpose of api validation

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38257#018afb0e-e104-4284-8695-32547bcd473b

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38257#018afb1c-61f5-40c9-a2b2-db177cb30801

[andijcr]

Found the manifestation of the alias not persisting after a reboot

In this excerpt, after a restart, the node rejoins and applies a snapshot of configs.

From the log, delete_retention_ms has the new value, but just after that, another entry has log_retention_ms set to the old value, and since it comes after, it overwrites the value.

The name log_retention_ms appears even though the test only applied the old name. This is because when dumping the properties, the new name is used, and somehow, this was merged with a dump where the old names appear (and the old name actually has the new value!)

This means that there is a need to normalize these values, and that in the context of partial upgrades + rollback this might result in the original cluster losing some properties, so a test needs to be crafted for this scenario

 530 
 531 
 532 Welcome to the Redpanda community!
 533 
 534 Documentation: https://docs.redpanda.com - Product documentation site
 535 GitHub Discussion: https://github.com/redpanda-data/redpanda/discussions - Longer, more involved discussions
 536 GitHub Issues: https://github.com/redpanda-data/redpanda/issues - Report and track issues with the codebase
 537 Support: https://support.redpanda.com - Contact the support team privately
 538 Product Feedback: https://redpanda.com/feedback - Let us know how we can improve your experience
 539 Slack: https://redpanda.com/slack - Chat about all things Redpanda. Join the conversation!
 540 Twitter: https://twitter.com/redpandadata - All the latest Redpanda news!
 541 
 542 
 543 INFO  2023-10-06 10:11:43,633 seastar - Reactor backend: linux-aio
 544 INFO  2023-10-06 10:11:43,641 [shard 0:main] seastar - Created fair group io-queue-0 for 2 queues, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000, per tick grab 6291456
 545 INFO  2023-10-06 10:11:43,641 [shard 0:main] seastar - IO queue uses 0.75ms latency goal for device 0
 546 INFO  2023-10-06 10:11:43,641 [shard 0:main] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647
 547 INFO  2023-10-06 10:11:43,641 [shard 0:main] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000
 548 INFO  2023-10-06 10:11:43,642 [shard 0:main] main - application.cc:351 - Redpanda v23.3.0-dev-766-gb7ba0a8c37 - b7ba0a8c379e1c00095174499d3c43fcfe6158f8-dirty
 549 INFO  2023-10-06 10:11:43,642 [shard 0:main] main - application.cc:359 - kernel=6.2.0-34-generic, nodename=057835480b2c, machine=x86_64
 550 INFO  2023-10-06 10:11:43,642 [shard 0:main] main - application.cc:287 - System resources: { cpus: 2, available memory: 2.000GiB, reserved memory: 0.000bytes}
 551 INFO  2023-10-06 10:11:43,642 [shard 0:main] main - application.cc:295 - File handle limit: 65535/65535
 552 INFO  2023-10-06 10:11:43,645 [shard 0:main] cluster - config_manager.cc:313 - Loaded property cluster_id=f826043f-afed-4074-87cd-27363ab446c8 from local cache
 553 INFO  2023-10-06 10:11:43,645 [shard 0:main] cluster - config_manager.cc:313 - Loaded property default_topic_partitions=4 from local cache
 554 INFO  2023-10-06 10:11:43,645 [shard 0:main] cluster - config_manager.cc:313 - Loaded property delete_retention_ms=5678 from local cache
 555 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property enable_metrics_reporter=false from local cache
 556 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property join_retry_timeout_ms=200 from local cache
 557 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property kafka_connections_max=2048 from local cache
 558 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property kafka_connections_max_overrides=[1.2.3.4:5] from local cache
 559 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property kafka_connections_max_per_ip=1024 from local cache
 560 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property log_retention_ms=1234 from local cache
 561 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property log_segment_size_jitter_percent=0 from local cache
 562 INFO  2023-10-06 10:11:43,646 [shard 0:main] cluster - config_manager.cc:313 - Loaded property superusers=[admin] from local cache

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38464#018b05d8-0db4-443d-a5be-fc8211fb3040

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38464#018b05d8-0dad-4426-a6e7-7d46d0323da2

[vbotbuildovich]

ducktape was retried in job https://buildkite.com/redpanda/redpanda/builds/38464#018b05e0-91a6-4dac-b5c5-64ec820d0904

[piyushredpanda]

@andijcr : Is this PR still valid? What is next here?