v24.1.1

New Features

• Adds new cluster and topic level configurations for write caching feature. by @bharathv in #16924
• PR #17009 write caching - raft implementation by @bharathv
• Enables write caching by default in dev container mode. by @bharathv in #17677
• Add rpk security roles, a new command space to manage your Redpanda roles. by @r-vasquez in #17538
• Introduce --allow-role and --deny-role flags for rpk acl commands by @oleiman in #17416
• Introduces GET /v1/security/users/roles (Admin API) by @oleiman in #17155
• Introduces /v1/security/roles/{role}/members Admin API endpoint for reading and updating RBAC role members. by @oleiman in #17153
• #17679 rpk security acl list now supports --format=json by @rockwotj in #17684
• Data Transforms now support writing to multiple output topics. The REDPANDA_OUTPUT_TOPIC environment variable exposed in transforms is now removed for REDPANDA_OUTPUT_TOPIC_%d for each output topic specified. by @rockwotj in #16946
• rpk transform deploy now supports multiple output topics by @rockwotj in #16950
• The golang transform-sdk gains the ability to write to multiple output topics.
  This feature can only be used in Redpanda v24.1.x or newer. by @rockwotj in #16978
• The rust transform-sdk gains the ability to write to multiple output topics.
  This feature can only be used in Redpanda v24.1.x or newer. by @rockwotj in #17007
• Publish log (i.e. stderr/stdout) output from data transforms exclusively to an internally managed Redpanda topic (_redpanda.transform_logs). Data transform logs will no longer appear in broker logs. by @oleiman in #16485
• Introduce rpk transform logs NAME to view logs for a transform by @rockwotj in #16923
• #16075 Data Transform's Rust SDK now supports a Schema Registry Client. by @rockwotj in #16464
• Topic-aware partition balancing, which attempts to spread partition replicas topic-wise across a cluster. This behavior is controlled by the partition_autobalancing_topic_aware config property (enabled by default). by @ztlpn in #17263
• Tiered Storage now supports using Azure VM user-assigned managed identities for securely accessing
  Azure Blob Storage @andijcr in #17157
• Topic recovery and ‘whole-cluster restore’ from Tiered Storage now perform integrity checks on metadata to ensure that each partition can be recovered successfully by @andijcr in #16915
• You can now create namespaces in Redpanda Cloud using rpk cloud namespace. by @r-vasquez in #16685
• #13175 rpk debug bundle now includes a CPU profile of the requested nodes. by @r- vasquez in #16414
• #16107 You can print a schema now using rpk registry schema get --print-schema. by @r-vasquez in #16109
• #16623 rpk redpanda config bootstrap now supports bootstrapping your advertised addresses configuration. by @r-vasquez in #16652
• new metric vectorized_storage_log_compacted_away_bytes for compaction observability in local storage added by @andijcr in #17579
• new public metric redpanda_cluster_latest_cluster_metadata_manifest_age to track the age of the cluster_metadata_manifest in cloud storage added by @andijcr in #17404

Bug Fixes

• Aggregates partitions in some cloud storage metrics when the aggregate_metrics cluster config is set to true. by @ballard26 in #16336
• Fix a bug that could lead to raft log inconsistencies when 2 out of 3 nodes in a configuration are changed. by @ztlpn in #17675
• Fix a bug that resulted in Redpanda ignoring until the next restart config values that were reset to their defaults. by @ztlpn in #16504
• Fix a bug where logging in a transform could cause the transform to not make progress. by @rockwotj in #17186
• Fix a crash that happened when a cluster that was partially in recovery mode tried to upload consumer offsets to cloud storage. by @ztlpn in #17013
• Fix a memory leak when using transactions with many different producer IDs. by @rockwotj in #15797
• Fix a potential cloud storage cache access time tracker file corruption during shutdown. by @nvartolomei in #16648
• Fix a race condition between suffix truncation / delete records and adjacent segment compaction that can lead to crashes and data-loss. by @ nvartolomei in #17019
• Fix a rare bug where http client connections would vanish from the connection pool leading to various operations hanging while waiting for an http client. by @nvartolomei in #15681
• Fix an issue where rpk transform logs waits for records without the --follow flag specified. by @rockwotj in #17832
• Fix an issue with Cargo.toml when initializing a Rust Data Transform project via rpk transform init by @rockwotj in #15934
• Fix initial_leader_epoch/KIP-320 handling in fetch requests. It was ignored until now which prevented consumers to correctly detect suffix truncation. For Redpanda (and Raft), this is a minor problem since suffix truncation is a very improbable event. by @nvartolomei in #17674
• Fix internal RPC client connection stall after more than 2^32 requests are sent. by @ztlpn in #16156
• Fix large allocation in partition manifest. by @dotnwat in #16160
• Fix oversized allocation in storage. by @Lazin in #16642
• Fix the starter code for Rust projects in rpk transform init by @rockwotj in #16180
• Fix tiered-storage housekeeping problem that may cause replaced segments to pile up if the spillover is enabled. by @Lazin in #16163
• Fixed a few oversized allocations for some admin server endpoints. by @rockwotj in #16551
• Fixed the values for the rpc client in/out bytes metric by @ballard26 in #17933
• Fixes rpk transform init --install-deps so that an explicit true value is not needed. by @rockwotj in #17831
• Fixes a bug in windowed compaction that could cause Redpanda to crash when an error occurs while reading batches. by @andrwng in #16928
• Fixes a bug of config_frontend methods getting called on shards other than the controller shard. by @pgellert in #17088
• Fixes a bug that may prevent redpanda from shutting down cleanly when auditing is enabled by @graphcareful in #16315
• Fixes a concurrency issue in transform offset commits pertaining to taking/applying snapshots. by @bharathv in #17383
• Fixes a crash if a WebAssembly function is deployed that immediately crashes. by @rockwotj in #15939
• Fixes a crash that could happen when reading from local storage with a large number of segments that all do not contain user data. by @andrwng in #18075
• Fixes a plausible correctness issue with idempotent requests during replication failures. by @bharathv in #16706
• Fixes a race between compaction and Raft recovery for compacted topics that could result in aborted transactional data batches being visible. by @andrwng in #16295
• Fixes an an improper initialization of metrics related to controller snapshot uploads. by @andrwng in #16070
• Fixes an issue where using the CPU profiler with running Data Transforms could cause the process to deadlock. by @rockwotj in #17877
• Fixes issue that causes the connection to hang when an unsupported compression type is passed via an incremental_alter_configs request by @graphcareful in #16399
• Fixes lock starvation during transform offset commits. by @bharathv in #17402
• Have fetch handler ensure rack awareness is enabled before performing follower fetching by @michael-redpanda in #15883
• Prevent an assertion from being triggered when Wasm VMs fail immediately. by @rockwotj in #15933
• Prevent detecting leader epoch advancement when state is not up to date by @mmaslankaprv in #16560
• Prevent reactor stalls querying leadership information for large clusters by @rockwotj in #17473
• Protect against a very rare scenario where after node restart, some of the partition replicas hosted on that node could not take part in leader elections. by @ztlpn in #16068
• Redpanda used to accept an empty string in redpanda.rack in node config. This would cause issues in Kafka operations. Redpanda will now error on startup if redpanda.rack is set to an empty string. by @michael-redpanda in #15835
• Redpanda will now correctly handle an empty rack ID provided in a fetch request by @michael-redpanda in #15846
• Reduces maximum log line size from 1MiB to 128KiB to reduce occurrences of memory allocation failures by @michael-redpanda in #17922
• Report runtime public metrics by task queue for all cores, not just core 0 by @rockwotj in #16154
• Return a HTTP 400 error code when deploying a transform to a topic that doesn't exist instead of a 500 by @rockwotj in #17011
• Schema Registry: Deleted schemas no longer reappear after certain compaction patterns on the _schemas topic. by @BenPope in #17091
• #15042 Fixes a bug in the tiered storage time-based query implementation that could result in a consumer hang when consuming very old data. by @andrwng in #16645
• #15201 Fix assertion triggered by interleaving of log flush and log truncation followed by append by @Lazin in #16105
• #15603 cluster config aliases are accepted while reading from yaml by @andijcr in #15605
• #15674 Fix an issue where new configs would continually revert to legacy defaults after an upgrade. by @oleiman in #15761
• #15722 #7946 Fix an issue where create topics responses would show incorrect partition count and replication factor by @oleiman in #16410* #15811 Several additional metrics will have their "partition" label aggregated away (i.e., into a single series per remaining label set with no partition label,...

v23.3.13

Bug Fixes

• fixed a problem leading to UAF error while calculating cloud stage usage by @mmaslankaprv in [#17980](https://github.com/ /pull/17980)
• prevents partial consumer group recovery by @mmaslankaprv in #18016

Improvements

• Changes what the kafka_latency_fetch_latency metric measures to be the time the first fetch_ntps_in_parallel takes. by @ ballard26 in #17977
• largely reduced number of health report copies by @mmaslankaprv in #18017## None
  No release notes explicitly specified.
• PR #17917 [v23.3.x] archival: Start housekeeping jobs only on a leader by @Lazin
• PR #17927 [v23.3.x] c/hm_backend: cache the collected report by @ mmaslankaprv
• PR #17960 [v23.3.x] rptest: fix test_exceed_broker_limit flake by @ travisdowns
• PR #17979 [v23.3.x] [CORE-2400] kafka/server: Disable quota balancer by @BenPope
• PR #18019 [v23.3.x] CORE-1752: cst: Downgrade error logs to debug by @ abhijat
• PR #18023 [v23.3.x] Backport of #16243 by @mmaslankaprv
• PR #18039 [v23.3.x] c/controller_backend: try to force-abort reconfiguration only on leaders by @ztlpn
• PR #18053 [v23.3.x] Address oversized allocs across kafka API and schema registry by @oleiman

Full Changelog: v23.3.12...v23.3.13

v23.2.28

Bug Fixes

• Fix a race condition between suffix truncation / delete records and adjacent segment compaction that can lead to crashes and data-loss. by @ nvartolomei in #17254
• Fix initial_leader_epoch/KIP-320 handling in fetch requests. It was ignored until now which prevented consumers to correctly detect suffix truncation. For Redpanda (and Raft), this is a minor problem since suffix truncation is a very improbable event. by @nvartolomei in #17727
• Fixes a bug of config_frontend methods getting called on shards other than the controller shard. by @pgellert in [#17211](https://github.com/ /pull/17211)
• Prevent detecting leader epoch advancement when state is not up to date by @mmaslankaprv in [#17882](https://github.com/redpanda-data/ redpanda/pull/17882)
• Reduces maximum log line size from 1MiB to 128KiB to reduce occurrences of memory allocation failures by @michael-redpanda in #17924
• #16612 fixes small inconsistency between Kafka and Redpanda when trying to query end_offset of an empty log by @mmaslankaprv in #17881
• #17238 Fixes a bug in CreateTopicsResponse to now return all the configs of the topic, not just the topic-specific override configs. by @pgellert in #17241
• #17790 Fix a bug that could lead to raft log inconsistencies when 2 out of 3 nodes in a configuration are changed. by @ztlpn in #17797
• prevents partial consumer group recovery by @mmaslankaprv in #17882
• PR #17160 [v23.2.x] compression: Allocate memory for LZ4_compressEnd by @abhijat
• PR #17826 [v23.2.x] CORE-1722: compression: Use preallocated decompression buffers for lz4 by @abhijat
• PR #17881 [v23.2.x] k/replicated_partition: fixed querying end offset of an empty log by @nvartolomei
• PR #17882 Backport of #17673 #17498 #16560 by @mmaslankaprv

Improvements

• Adds a new public metric redpanda_raft_recovery_partition_movement_consumed_bandwidth that tracks how much bandwidth is currently in use for raft recovery. This helps tune raft_learner_recovery_rate. by @bharathv in #17217

• PR #17397 [v23.2.x] k/group: recover leader epoch on leader change by @nvartolomei * PR #17448 [v23.2.x] tx: fix param ordering in log statement by @nvartolomei

• PR #17577 [v23.2.x] c/topics_frontend: break the loop when dispatching to current leader by @mmaslankaprv

Full Changelog: v23.2.27...v23.2.28

v23.3.12

Bug Fixes

• Fix an issue where rpk transform logs waits for records without the --follow flag specified. by @rockwotj in #17837
• Fixes rpk transform init --install-deps so that an explicit true value is not needed. by @rockwotj in [#17867](https:// github.com//pull/17867)
• Fixes a crash when data transforms error and restart by @rockwotj in [#17696](https://github.com/redpanda-data/redpanda/pull/ 17696)
• Reduces maximum log line size from 1MiB to 128KiB to reduce occurrences of memory allocation failures by @michael-redpanda in #17923
• #16612 fixes small inconsistency between Kafka and Redpanda when trying to query end_offset of an empty log by @mmaslankaprv in #17809
• #17718 Fix reported config source for cleanup.policy by reporting DEFAULT_CONFIG instead of DYNAMIC_TOPIC_CONFIG for the default value. by @pgellert in [#17719](https://github.com/redpanda-data/ redpanda/pull/17719)
• #17791 Fix a bug that could lead to raft log inconsistencies when 2 out of 3 nodes in a configuration are changed. by @ztlpn in #17796
• #17817 Fix problem in Tiered-Storage that could potentially cause consumers to get stuck by @Lazin in #17818
• #17891 fix a race between eviction and producer registration that results in an invalid transaction state. by @bharathv in #17900

Improvements

• Handle missing data transform logs topic in rpk transform logs by @rockwotj in [#17835](https://github.com/redpanda-data/ redpanda/pull/17835)
• #17197 more accurate node status reporting by @mmaslankaprv in #17698
• skipping overhead of collecting node health report for each node separately. by @mmaslankaprv in [#17864](https://github.com/ /pull/17864)
• PR #17756 [v23.3.x] kafka: chunked_vector for config responses by @ pgellert
• PR #17792 [v23.3.x] CORE-1752: cst: improved logging by @abhijat
• PR #17825 [v23.3.x] CORE-1722: compression: Use preallocated decompression buffers for lz4 by @abhijat
• PR #17888 [v23.3.x] CORE-2365: storage: increase size of offset key map fragment size by @dotnwat

Full Changelog: v23.3.11...v23.3.12

v23.3.11

Features

• Introduce "trust_file_crc32c" metric to export a checksum for each trust file in the system. by @oleiman in #17587

Bug Fixes

• #16650 Fix oversized allocation in storage. by @Lazin in #17541
• #17459 Fixes a bug with TLS metrics where expiration timestamps would not advance on certificate reload by @oleiman in #17460
• rpk: prevent a segfault when creating a profile from a cloud that is not in ready state. by @r-vasquez in #17585
• PR #17435 [v23.3.x] c/frag_vector: added get_allocator() method to fragmented vector by @mmaslankaprv
• PR #17449 [v23.3.x] tx: fix param ordering in log statement by @ nvartolomei
• PR #17572 [v23.3.x] Fixed node_hash_map caused oversized allocations in cluster module by @mmaslankaprv
• PR #17573 [v23.3.x] use chunked vector as batches cache in raft:: replicate_batcher by @mmaslankaprv
• PR #17576 [v23.3.x] c/topics_frontend: break the loop when dispatching to current leader by @mmaslankaprv
• PR #17578 [v23.3.x] rm_stm: do not hold producer lock for the duration of the barrier by @bharathv
• PR #17584 [v23.3.x] k/group_manager: used chunked_vector when cleaning groups by @mmaslankaprv

Improvements

• #17428 Improves error feedback when Redpanda is given an invalid number of partitions during either topic creation or when the partition count for a topic is increased. by @michael-redpanda in #17431
• PR #17574 [v23.3.x] Improved validation of Fetch requests when reading from follower by @mmaslankaprv

Full Changelog: v23.3.10...v23.3.11

v23.3.10

Features

• #16075 Data Transform's Rust SDK now supports a Schema Registry Client. by @rockwotj in #17174
• #17235 rpk: support running rpk transform against Redpanda cloud clusters. by @r-vasquez in #17252

Bug Fixes

• Fix a bug where logging in a transform could cause the transform to not make progress. by @rockwotj in #17192
• Fix a race condition between suffix truncation / delete records and adjacent segment compaction that can lead to crashes and data-loss. by @nvartolomei in #17253
• Fixes a bug of config_frontend methods getting called on shards other than the controller shard. by @pgellert in #17184
• Fixes a bug that would prevent the transaction coordinator topic from reclaiming disk space. by @andrwng in #17380
• Fixes a concurrency issue in transform offset commits pertaining to taking/applying snapshots. by @bharathv in #17400
• #17237 Fixes a bug in CreateTopicsResponse to now return all the configs of the topic, not just the topic-specific override configs. by @pgellert in #17239
• #17419 fixed enabling cloud storage in existing clusters by @mmaslankaprv in #17420

Improvements

• Adds a new public metric redpanda_raft_recovery_partition_movement_consumed_bandwidth that tracks how much bandwidth is currently in use for raft recovery. This helps tune raft_learner_recovery_rate. by @bharathv in #17216
• less overhead of health report collection by @mmaslankaprv in #17360

Full Changelog: v23.3.9...v23.3.10

v23.3.9

Features

• Introduce rpk transform logs NAME to view logs for a transform by @rockwotj in #17148

Full Changelog: v23.3.8...v23.3.9

v23.2.27

Bug Fixes

• Schema Registry: Deleted schemas no longer reappear after certain compaction patterns on the _schemas topic. by @BenPope in #17095
• #16129 Fixes a bug in SASL user deletion and update where usernames with a + symbol in the username were prevented from being deleted by @pgellert in #16813
• #16716 Fixed a few oversized allocations for some admin server endpoints. by @rockwotj in #16747

Improvements

• #16995 Added EHOSTUNREACH to retry-able error code list by @michael-redpanda in #16996

Full Changelog: v23.2.26...v23.2.27

v23.3.8

Features

• #16941 rpk redpanda config bootstrap now support bootstrapping your advertised addresses configuration. by @r-vasquez in #16942

Bug Fixes

• Fix a crash that happened when a cluster that was partially in recovery mode tried to upload consumer offsets to cloud storage. by @ ztlpn in #17022
• Return a HTTP 400 error code when deploying a transform to a topic that doesn't exist instead of a 500 by @rockwotj in #17018
• Schema Registry: Deleted schemas no longer reappear after certain compaction patterns on the _schemas topic. by @BenPope in #17094
• #16679 Retains control batches from transactions to preserve transaction boundaries. This prevents some (very unlikely) scenarios where aborted data is read. by @bharathv in [#17100](https://github.com/ /pull/17100)
• PR #17093 [v23.3.x] c/topic_table: replaced partition metadata map with chunked_vector by @mmaslankaprv
• PR #17099 [v23.3.x] storage: ensure monotonic stable offset updates by @ nvartolomei
• PR #17111 [v23.3.x] cloud_storage_clients: classify request_timeout as retriable by @nvartolomei

Improvements

• #16815 Node-wide throughput throttling is now fair an responsive. by @ BenPope in #16848
• #16993 cluster: Avoid oversize allocs for topic creation and configuration by @BenPope in #17012
• #17107 rpk profile has been reworked in an attempt to be simpler; see PR #17038 for more detail by @twmb in #17108
• PR #17115 [v23.3.x] Using contiguous_range_map in partition_leaders_table by @mmaslankaprv
• PR #17120 [v23.3.x] rpk profile: a few more fixes by @twmb

Full Changelog: v23.3.7...v23.3.8

v23.3.7

Features

• You can create namespaces in Redpanda cloud using rpk cloud namespace. by @r-vasquez in [#16777](https://github.com/redpanda- data/redpanda/pull/16777)
• #16570 [#16572](https://github.com/redpanda-data/redpanda/issues/ 16572) Publish log (i.e. stderr/stdout) output from data transforms exclusively to an internally managed Redpanda topic (_redpanda.transform_logs). Data transform logs will no longer appear in broker logs. by @oleiman in [#16663](https://github. com//pull/16663)
• #16895 Add Prometheus metrics for data transforms logging by @ oleiman in #16913

Bug Fixes

• Fixes a plausible correctness issue with idempotent requests during replication failures. by @bharathv in [#16749](https:// github.com//pull/16749)
• #16129 Fixes a bug in SASL user deletion and update where usernames with a + symbol in the username were prevented from being deleted by @pgellert in [#16811](https://github.com/redpanda-data/ redpanda/pull/16811)
• #16659 Fixes a bug in the tiered storage time-based query implementation that could result in a consumer hang when consuming very old data. by @andrwng in [#16660](https://github.com/ /pull/16660)
• #16717 Fixed a few oversized allocations for some admin server endpoints. by @rockwotj in #16719
• #16884 Fixed deleting Data Transforms with names that had URL unsafe characters by @rockwotj in #16885
• #16937 Fixes a bug in windowed compaction that could cause Redpanda to crash when an error occurs while reading batches. by @andrwng in [#16940](https://github.com/redpanda-data/redpanda/pull/ 16940)

Improvements

• Adds observability into producer evictions in each shard. by @bharathv in [#16839](https://github.com/redpanda-data/redpanda/ pull/16839)
• Fix large wasm module deployments by @rockwotj in #16767
• Increase data_transforms_logging_buffer_capacity_bytes from 100KiB to 500KiB by @oleiman in [#16977](https://github.com/ /pull/16977)
• Large allocations are now logged by default (similar to reactor stalls) by @StephanDollberg in [#16844](https://github.com/ /pull/16844)
• #16795 Added ability to change transactional manage topic properties by @mmaslankaprv in #16968
• #16831 get_cluster_uuid returns a correctly formatted string by @ andijcr in #16832
• #16888 Data Transform builds in rpk now uses tinygo v0.31.1 by @ rockwotj in #16889
• #16947 better control of memory usage in storage layer. by @ mmaslankaprv in #16963
• #16997 Added EHOSTUNREACH to retry-able error code list by @ michael-redpanda in #16998
• optimized updating leadership metadata with health reports by @mmaslankaprv in [#16709](https://github.com/redpanda-data/ redpanda/pull/16709)
• preventing large allocation in partition balancer code by @mmaslankaprv in [#16939](https://github.com/redpanda-data/ redpanda/pull/16939)
• rpk: Remove 10s timeout in rpk profile create by @r-vasquez in [#16852](https://github.com/redpanda-data/redpanda/pull/ 16852)
• PR #16682 [v23.3.x] Implement async_for_each by @travisdowns
• PR #16688 [v23.3.x] Add forward iterator to async_for_each by @ travisdowns
• PR #16691 [v23.3.x] Rethrow on unknown exceptions in fetch handler by @ballard26
• PR #16784 [v23.3.x] c/leaders: trigger leadership notification when term changes by @mmaslankaprv
• PR #16801 [v23.3.x] c/topic_table_probe: use btree_map in topic table probe by @mmaslankaprv
• PR #16829 [v23.3.x] rpk: update help text of decommission-status by @ daisukebe
• PR #16891 [v23.3.x] cmake: upgrade tinygo compiler by @rockwotj
• PR #16894 [v23.3.x] cloud_storage: Improve stale_reader test by @Lazin
• PR #16897 [v23.3.x] Fixed background apply fiber race condition in raft::state_machine_manager by @mmaslankaprv
• PR #16903 [v23.3.x] cloud_storage: various non-functional changes by @andrwng
• PR #16908 Revert "[v23.3.x] rm_stm/idempotency: fix the producer lock scope" by @bharathv
• PR #16935 [v23.3.x] fix for cluster_config_test.py::test_aliasing by @andijcr
• PR #16965 [v23.3.x] Ensure fragment_vector fragments are always <= 128KiB by @ballard26

Full Changelog: v23.3.6...v23.3.7