diff --git a/modules/develop/pages/kafka-clients.adoc b/modules/develop/pages/kafka-clients.adoc index fc0bdeef6..d51032c7a 100644 --- a/modules/develop/pages/kafka-clients.adoc +++ b/modules/develop/pages/kafka-clients.adoc @@ -47,6 +47,6 @@ Redpanda does not currently support the following Apache Kafka features: * Managing SASL users with Kafka APIs: DescribeUserSCRAMCredentialsRequest and AlterUserSCRAMCredentialsRequest APIs. * HTTP Proxy (pandaproxy): Unlike other REST proxy implementations in the Kafka ecosystem, Redpanda HTTP Proxy does not support topic and ACLs CRUD through the HTTP Proxy. HTTP Proxy is designed for clients producing and consuming data that do not perform administrative functions. -* Quotas per user for bandwidth and API request rates (though xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group (per CPU core)] are supported). +* Quotas per user for bandwidth and API request rates. However, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[quotas per client and per client group] using AlterClientQuotas and DescribeClientQuotas APIs are supported. If you have any issues while working with a Kafka tool, you can https://github.com/redpanda-data/redpanda/issues/new[file an issue^]. diff --git a/modules/get-started/pages/whats-new.adoc b/modules/get-started/pages/whats-new.adoc index dce1966ec..e146785f9 100644 --- a/modules/get-started/pages/whats-new.adoc +++ b/modules/get-started/pages/whats-new.adoc @@ -25,6 +25,13 @@ You can now deploy data transform functions that xref:develop:data-transforms/de The docs now also include an xref:develop:data-transforms/index.adoc[expanded guide] designed to help you master the creation, deployment, and management of data transforms in Redpanda. + +== Client throughput management + +Redpanda now includes `rpk` and Redpanda Console support for managing xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[client throughput limits]. You can set client throughput quotas using `rpk` for an individual client based on client ID, as well as for a group of clients based on ID prefix matching. New xref:manage:cluster-maintenance/manage-throughput.adoc#monitor-client-throughput[metrics] are available for insights on client throughput utilization. + +Client throughput quotas, previously applied on a per-shard basis, now apply on a per-broker basis. Cluster configuration properties for managing client quotas are xref:upgrade:deprecated/index.adoc[deprecated], including `target_quota_byte_rate` which is disabled by default with the value `0`. + == Next steps * xref:install-beta.adoc[] \ No newline at end of file diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 07ab7b4a9..c3e5ac824 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -6,18 +6,20 @@ Redpanda supports applying throughput throttling on both ingress and egress inde == Throughput throttling enforcement +NOTE: As of v24.2, Redpanda enforces all throughput limits per broker, including client throughput. + Throughput limits are enforced by applying backpressure to clients. When a connection is in breach of the throughput limit, the throttler advises the client about the delay (throttle time) that would bring the rate back to the allowed level. Redpanda starts by adding a `throttle_time_ms` field to responses. If that isn't honored, delays are inserted on the connection's next read operation. The throttling delay may not exceed the limit set by xref:reference:tunable-properties.adoc#max_kafka_throttle_delay_ms[`max_kafka_throttle_delay_ms`]. == Broker-wide throughput limits -Broker-wide throughput limits account for all Kafka API traffic going in to or out of the broker. The limit values represent the allowed rate of data in bytes per second passing through in each direction. Redpanda also provides administrators the ability to exclude clients from throughput throttling and to fine-tune which Kafka request types are subject to throttling limits. +Broker-wide throughput limits account for all Kafka API traffic going into or out of the broker, as data is produced to or consumed from a topic. The limit values represent the allowed rate of data in bytes per second passing through in each direction. Redpanda also provides administrators the ability to exclude clients from throughput throttling and to fine-tune which Kafka request types are subject to throttling limits. === Broker-wide throughput limit properties The properties for broker-wide throughput quota balancing are configured at the cluster level, for all brokers in a cluster: |=== -| Property | Description +| Cluster configuration property | Description | xref:reference:cluster-properties.adoc#kafka_throughput_limit_node_in_bps[kafka_throughput_limit_node_in_bps] | A broker's total throughput limit for ingress Kafka traffic. @@ -43,48 +45,200 @@ The properties for broker-wide throughput quota balancing are configured at the == Client throughput limits -Similar to <> but for clients, Redpanda provides configurable throughput quotas that apply to an individual client or a group of clients. +Redpanda provides configurable throughput quotas that apply to an individual client or a group of clients. You can apply a quota for an individual client based on an exact match with its `client_id`, or a group of clients based on IDs that start with a given prefix. + +As of v24.2, client throughput quotas are compatible with the https://cwiki.apache.org/confluence/display/KAFKA/KIP-546%3A+Add+Client+Quota+APIs+to+the+Admin+Client[AlterClientQuotas and DescribeClientQuotas^] Kafka APIs, and are separate from quotas configured through cluster configuration in earlier Redpanda versions. The client throughput quotas no longer apply on a per-shard basis, and now limit the rates across a Redpanda broker's node. The quotas are neither shared nor balanced between brokers. + +Redpanda supports the following Kafka API-based quota types on clients: + +|=== +| Quota type | Description + +| `producer_byte_rate` +| Limit throughput of produce requests + +| `consumer_byte_rate` +| Limit throughput of fetch requests + +| `controller_mutation_rate` +| Limit rate of topic mutation requests, including create, add, and delete partition, in number of partitions per second + +|=== + +You can also apply a default quota for all other client requests that don't have a specific quota based on an exact match or `client_id` prefix. + +It is possible to create conflicting quotas if you configure the same quotas through both the Kafka API and a cluster configuration. Redpanda resolves these conflicts by following an order of preference in finding a matching quota for a request: -NOTE: The client throughput quotas limit the rates within each shard (logical CPU core) of a Redpanda broker's node. The quotas are neither shared nor balanced between shards and brokers. +. Quota configured through the Kafka API for an exact match on `client_id` +. Quota configured through the Kafka API for a prefix match on `client_id` +. Quota configured through cluster configuration properties (`kafka_client_group_byte_rate_quota`, `kafka_client_group_fetch_byte_rate_quota`, xref:upgrade:deprecated/index.adoc[deprecated starting in v24.2]) for a prefix match on `client_id` +. Default quota configured through the Kafka API on `client_id` +. Default quota configured through cluster configuration properties (`target_quota_byte_rate`, `target_fetch_quota_byte_rate`, `kafka_admin_topic_api_rate`, xref:upgrade:deprecated/index.adoc[deprecated starting in v24.2]) on `client_id` + +Redpanda recommends <> over from cluster configuration-managed quotas to Kafka-compatible quotas. You can re-create the configuration-based quotas with `rpk`, and then remove the cluster configurations. === Individual client throughput limit -To set throughput quota for a single client, configure the cluster properties xref:reference:cluster-properties.adoc#target_quota_byte_rate[`target_quota_byte_rate`] and xref:reference:cluster-properties.adoc#target_fetch_quota_byte_rate[`target_fetch_quota_byte_rate`]. +To view current throughput quotas set through the Kafka API, run xref:reference:rpk/rpk-cluster/rpk-cluster-quotas-describe.adoc[`rpk cluster quotas describe`]. -* The `target_quota_byte_rate` property applies to a producer client that isn't a member of a client group configured by xref:reference:cluster-properties.adoc#kafka_client_group_byte_rate_quota[`kafka_client_group_byte_rate_quota`]. It sets the maximum throughput quota of a client sending to a Redpanda broker. -* The `target_fetch_quota_byte_rate` property applies to a consumer client that isn't a member of a client group configured by xref:reference:cluster-properties.adoc#kafka_client_group_fetch_byte_rate_quota[`kafka_client_group_fetch_byte_rate_quota`]. It sets the maximum throughput quota of a client fetching from a Redpanda broker. +For example, to see the quotas for client ID `consumer-1`: -The values of both `target_quota_byte_rate` and `target_fetch_quota_byte_rate` are throughput rate limits within a shard, in bytes per second. +[,bash] +---- +rpk cluster quotas describe --name client-id=consumer-1 +---- + +[,bash,role=no-copy] +---- +client-id=consumer-1 + producer_byte_rate=140000 +---- + +To set a throughput quota for a single client, use the xref:reference:rpk/rpk-cluster/rpk-cluster-quotas-alter.adoc[`rpk cluster quotas alter`] command. + +[,bash] +---- +rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id=consumer-1 +---- + +[,bash,role=no-copy] +---- +ENTITY STATUS +client-id=consumer-1 OK +---- === Group of clients throughput limit -To set throughput quota for a group of clients, configure the cluster properties xref:reference:cluster-properties.adoc#kafka_client_group_byte_rate_quota[`kafka_client_group_byte_rate_quota`] and xref:reference:cluster-properties.adoc#kafka_client_group_fetch_byte_rate_quota[`kafka_client_group_fetch_byte_rate_quota`]. +Alternatively, you can view or configure throughput quotas for a group of clients based on a match on client ID prefix. The following example sets the `consumer_byte_rate` quota to client IDs prefixed with `consumer-`: + +[,bash] +---- +rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer- +---- + +NOTE: A client group specified with `client-id-prefix` is not the equivalent of a Kafka consumer group. It is used only to match requests based on the `client_id` prefix. The `client_id` field is typically a configurable property when you create a client with Kafka libraries. + +=== Default client throughput limit + +You can apply default throughput limits to clients. Redpanda applies the default limits if no quotas are configured for a specific `client_id` or prefix. + +To specify a produce quota of 1 GBps through the Kafka API (which means a 1 GBps limit across all produce requests sent to a single Redpanda broker), run: + +[,bash] +---- +rpk cluster quotas alter --default client-id --add producer_byte_rate=1000000000 +---- + +=== Bulk manage client throughput limits -* The `kafka_client_group_byte_rate_quota` property applies to producer clients. It sets a maximum throughput quota of traffic sent to Redpanda from each producer in the group named by the property. -* The `kafka_client_group_fetch_byte_rate_quota` property applies to consumer clients. It sets a maximum throughput quota of traffic fetched from Redpanda by each consumer in the group named by the property. +To more easily manage multiple quotas, you can use the `cluster quotas describe` and xref:reference:rpk/rpk-cluster/rpk-cluster-quotas-import.adoc[`cluster quotas import`] commands to do a bulk export and update. -Both `kafka_client_group_byte_rate_quota` and `kafka_client_group_fetch_byte_rate_quota` have the configuration value fields: +For example, to export all client quotas in JSON format: -* `group_name`: a name for the group of clients. -* `clients_prefix`: a client belonging to the group has this prefix in its `client_id` property. -* `quota`: the maximum throughput rate of *each client in the group* in bytes per second. +[,bash] +---- +rpk cluster quotas describe --format json +---- -NOTE: For `kafka_client_group_fetch_byte_rate_quota`, a group of consumer clients is not a Kafka consumer group, and the `group_name` field is not a Kafka consumer group ID. Instead, it is the set of clients fetching from a Redpanda broker that are configured by the property to be throughput limited. +`rpk cluster quotas import` accepts the output string from `rpk cluster quotas describe --format `: -An example configuration of `kafka_client_group_byte_rate_quota` for two groups of producers: +[,bash] +---- +rpk cluster quotas import --from '{"quotas":[{"entity":[{"name":"foo","type":"client-id"}],"values":[{"key":"consumer_byte_rate","values":"12123123"}]},{"entity":[{"name":"foo-","type":"client-id-prefix"}],"values":[{"key":"producer_byte_rate","values":"12123123"},{"key":"consumer_byte_rate","values":"4444444"}]}]}' +---- + +You can also save the JSON or YAML output to a file and pass the file path in the `--from` flag. + +[[migrate]] +=== Migrate cluster configuration quotas to Kafka API-based quotas + +. Use xref:reference:rpk/rpk-cluster/rpk-cluster-config-get.adoc[`rpk cluster config get`] to view current client quotas managed with cluster configuration. The following example shows how to retrieve the `kafka_client_group_byte_rate_quota` for two groups of producers: ++ +[,bash] +---- +rpk cluster config get kafka_client_group_byte_rate_quota -[,json] ---- -"kafka_client_group_byte_rate_quota": [ - { - "group_name": "group_1", - "clients_prefix": "producer_group_alone_producer", - "quota": 10240 - }, - { - "group_name": "group_2", - "clients_prefix": "producer_group_multiple", - "quota": 20480 - }, ++ +[,bash,role=no-copy] +---- +"kafka_client_group_byte_rate_quota": [ + { + "group_name": "group_1", + "clients_prefix": "producer_group_alone_producer", + "quota": 10240 + }, + { "group_name": "group_2", + "clients_prefix": "producer_group_multiple", + "quota": 20480 + }, ] ---- + +. Each client quota cluster property (xref:upgrade:deprecated/index.adoc[deprecated in v24.2]) corresponds to a quota type in Kafka. Check the corresponding `rpk` arguments to use when setting the new quota values: ++ +|=== +| Cluster configuration property | `rpk cluster quotas` arguments + +| `target_quota_byte_rate` +| `--default client-id --add producer_byte_rate=` + +| `target_fetch_quota_byte_rate` +| `--default client-id --add consumer_byte_rate=` + +| `kafka_admin_topic_api_rate` +| `--default client-id --add controller_mutation_rate=` + +| `kafka_client_group_byte_rate_quota` +| `--name client-id-prefix= --add producer_byte_rate=` + +| `kafka_client_group_fetch_byte_rate_quota` +| `--name client-id-prefix= --add consumer_byte_rate=` + +|=== ++ +The client throughput quotas set through the Kafka API apply per broker, so you must convert the cluster configuration values that were applied on a per-shard (logical CPU core) basis. For example, if you set `target_fetch_quota_byte_rate` to 100 MBps/shard, and you run Redpanda on 16-core brokers, you can set the new consumer_byte_rate quota to 100 * 16 = 1600 MBps. + +. Use `rpk cluster quotas alter` to set the corresponding client throughput quotas based on the Kafka API: ++ +[,bash] +---- +rpk cluster quotas alter --name client-id-prefix=producer_group_alone_producer --add producer_byte_rate= +rpk cluster quotas alter --name client-id-prefix=producer_group_multiple --add producer_byte_rate= +---- ++ +Replace the placeholder values with the new quota values, accounting for the conversion to per-broker limits. For example, 10240 * broker core count = new quota. + +. Use xref:reference:rpk/rpk-cluster/rpk-cluster-config-set.adoc[`rpk cluster config set`] to remove the configuration-based quotas: ++ +[,bash] +---- +rpk cluster config set kafka_client_group_byte_rate_quota= +---- + +=== View throughput limits in Redpanda Console + +You can also use Redpanda Console to view enforced limits. In the menu, go to **Quotas**. + +=== Monitor client throughput + +The following metrics are available on both the `/public_metrics` and `/metrics` endpoints to provide insights into client throughput quota usage: + +* Client quota throughput per rule and quota type: +** `/public_metrics` - xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throughput[`redpanda_kafka_quotas_client_quota_throughput`] +** `/metrics` - xref:reference:internal-metrics-reference.adoc#vectorized_kafka_quotas_client_quota_throughput[`vectorized_kafka_quotas_client_quota_throughput`] +* Client quota throttling delay per rule and quota type, in seconds: +** `/public_metrics` - xref:reference:public-metrics-reference.adoc#redpanda_kafka_quotas_client_quota_throttle_time[`redpanda_kafka_quotas_client_quota_throttle_time`] +** `/metrics` - xref:reference:internal-metrics-reference.adoc#vectorized_kafka_quotas_client_quota_throttle_time[`vectorized_kafka_quotas_client_quota_throttle_time`] + +The `kafka_quotas` logger provides details at the trace level on client quota throttling: + +[,bash] +---- +TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns +TRACE 2024-06-14 15:36:05,240 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:51256] throttle request:{snc:0, client:184}, enforce:{snc:-365123762, client:-365123762}, key:0, request_size:1316 +TRACE 2024-06-14 15:37:44,835 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 119, delay:0ns, capped_delay:0ns +TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - quota_manager.cc:361 - request: ctx:{quota_type: produce_quota, client_id: {rpk}}, key:k_client_id{rpk}, value:{limit: {1111}, rule: kafka_client_default}, bytes: 1316, delay:184518451ns, capped_delay:184518451ns +TRACE 2024-06-14 15:37:59,195 [shard 2:main] kafka_quotas - connection_context.cc:605 - [127.0.0.1:58636] throttle request:{snc:0, client:184}, enforce:{snc:-14359, client:-14359}, key:0, request_size:1316 +---- + diff --git a/modules/reference/pages/internal-metrics-reference.adoc b/modules/reference/pages/internal-metrics-reference.adoc index 4f9da8275..22a9cfc44 100644 --- a/modules/reference/pages/internal-metrics-reference.adoc +++ b/modules/reference/pages/internal-metrics-reference.adoc @@ -87,9 +87,35 @@ Current effective quota for the quota balancer, in bytes per second. --- +=== vectorized_kafka_quotas_client_quota_throttle_time + +Client quota throttling delay, in seconds, per rule and quota type based on xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[client throughput limits]. + +*Type*: histogram + +*Labels*: + +* `quota_rule=("not_applicable" | "kafka_client_default" | "cluster_client_default" | "kafka_client_prefix" | "cluster_client_prefix" | "kafka_client_id")` +* `quota_type=("produce_quota" | "fetch_quota" | "partition_mutation_quota")` + +--- + +=== vectorized_kafka_quotas_client_quota_throughput + +Client quota throughput per rule and quota type. + +*Type*: histogram + +*Labels*: + +* `quota_rule=("not_applicable" | "kafka_client_default" | "cluster_client_default" | "kafka_client_prefix" | "cluster_client_prefix" | "kafka_client_id")` +* `quota_type=("produce_quota" | "fetch_quota" | "partition_mutation_quota")` + +--- + === vectorized_kafka_quotas_throttle_time -Histogram of throttle times, in seconds. +Histogram of throttle times, in seconds, based on xref:manage:cluster-maintenance/manage-throughput.adoc#broker-wide-throughput-limits[broker-wide throughput limits]. *Type*: histogram diff --git a/modules/reference/pages/properties/cluster-properties.adoc b/modules/reference/pages/properties/cluster-properties.adoc index 239eeaac2..33e19412a 100644 --- a/modules/reference/pages/properties/cluster-properties.adoc +++ b/modules/reference/pages/properties/cluster-properties.adoc @@ -1674,30 +1674,6 @@ Time between cluster join retries in milliseconds. --- -=== kafka_admin_topic_api_rate - -Target quota rate for partition mutations per <>. If `null`, the property is disabled, and no quota rate is applied. - -*Unit*: partition mutations per `default_window_second` - -*Related properties*: - -* <> - -*Requires restart:* No - -*Optional:* No - -*Visibility:* `user` - -*Type:* integer - -*Accepted values:* [`0`, `4294967295`] - -*Default:* `null` - ---- - === kafka_batch_max_bytes Maximum size of a batch processed by the server. If the batch is compressed, the limit applies to the compressed batch size. @@ -1716,58 +1692,6 @@ Maximum size of a batch processed by the server. If the batch is compressed, the --- -=== kafka_client_group_byte_rate_quota - -A map specifying the produce-rate quota per client group. The configurable fields: - -Example: `([{'group_name': 'first_group','clients_prefix': 'group_1','quota': 10240}])` - -*Related topics*: - -* xref:manage:cluster-maintenance/manage-throughput.adoc#client-group-throughput-limits[Client group throughput limits] - -*Requires restart:* No - -*Optional:* Yes - -*Visibility:* `user` - -*Accepted Values:* - -* `group_name`: Name of a client group -* `clients_prefix`: Prefix to prepend to the name of each client belonging to the group specified by `group_name` -* `quota`: Produce-rate quota of each client in bytes per second - -*Default:* `null` - ---- - -=== kafka_client_group_fetch_byte_rate_quota - -A map specifying the fetch-rate quota per client group. The configurable fields: - -Example: `([{'group_name': 'first_group','clients_prefix': 'group_1','quota': 10240}])` - -*Related topics*: - -* xref:manage:cluster-maintenance/manage-throughput.adoc#client-group-throughput-limits[Client group throughput limits] - -*Requires restart:* No - -*Optional:* Yes - -*Visibility:* `user` - -*Accepted Values:* - -* `group_name`: Name of a client group -* `clients_prefix`: Prefix to prepend to the name of each client belonging to the group specified by `group_name` -* `quota`: Produce-rate quota of each client in bytes per second - -*Default:* `null` - ---- - === kafka_connection_rate_limit Maximum connections per second for one core. If `null` (the default), then the number of connections per second is unlimited. @@ -5236,54 +5160,6 @@ List of superuser usernames. --- -=== target_fetch_quota_byte_rate - -Target fetch-size quota byte rate. If `null`, the property is disabled, and no quota byte rate is applied. - -*Unit*: bytes per second - -*Requires restart:* No - -*Optional:* No - -*Visibility:* `user` - -*Type:* integer - -*Accepted values:* [`0`, `4294967295`] - -*Default:* `null` - ---- - -=== target_quota_byte_rate - -Target quota byte rate. - -The `target_quota_byte_rate` property applies to a producer client that isn't a member of a client group configured by <>. It sets the maximum throughput quota of a client sending to a Redpanda broker node. - -When set to 0, this property is disabled. - -*Unit*: bytes per second - -*Requires restart:* No - -*Optional:* Yes - -*Visibility:* `user` - -*Type:* integer - -*Accepted values:* [`0`, `4294967295`] - -*Default:* `0` - -*Related topics*: - -* xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[Client throughput limits] - ---- - === tm_sync_timeout_ms Transaction manager's synchronization timeout. Maximum time to wait for internal state machine to catch up before rejecting a request. diff --git a/modules/reference/pages/public-metrics-reference.adoc b/modules/reference/pages/public-metrics-reference.adoc index d76099c45..b23e382da 100644 --- a/modules/reference/pages/public-metrics-reference.adoc +++ b/modules/reference/pages/public-metrics-reference.adoc @@ -497,6 +497,32 @@ Latency of produce/consume requests per broker. This duration measures from when --- +=== redpanda_kafka_quotas_client_quota_throttle_time + +Client quota throttling delay, in seconds, per rule and quota type. + +*Type*: histogram + +*Labels*: + +* `quota_rule=("not_applicable" | "kafka_client_default" | "cluster_client_default" | "kafka_client_prefix" | "cluster_client_prefix" | "kafka_client_id")`` +* `quota_type=("produce_quota" | "fetch_quota" | "partition_mutation_quota")` + +--- + +=== redpanda_kafka_quotas_client_quota_throughput + +Client quota throughput per rule and quota type. + +*Type*: histogram + +*Labels*: + +* `quota_rule=("not_applicable" | "kafka_client_default" | "cluster_client_default" | "kafka_client_prefix" | "cluster_client_prefix" | "kafka_client_id")` +* `quota_type=("produce_quota" | "fetch_quota" | "partition_mutation_quota")` + +--- + == Consumer group metrics === redpanda_kafka_consumer_group_committed_offset diff --git a/modules/upgrade/pages/deprecated/index.adoc b/modules/upgrade/pages/deprecated/index.adoc index 12ff1f1cc..76df67434 100644 --- a/modules/upgrade/pages/deprecated/index.adoc +++ b/modules/upgrade/pages/deprecated/index.adoc @@ -9,9 +9,19 @@ This index helps you to identify deprecated features in Redpanda releases and pl |=== | Deprecated in | Feature | Details +| 24.2 +a| Client throughput quota cluster configuration: + +- `kafka_client_group_byte_rate_quota` +- `kafka_client_group_fetch_byte_rate_quota` +- `target_quota_byte_rate` +- `target_fetch_quota_byte_rate` +- `kafka_admin_topic_api_rate` +| Use xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`] to manage xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[client throughput limits] based on the Kafka API. + | 23.3.2 | `space_management_enable_override` -| This property was a workaround in older version of Redpanda. It is no longer required and the property is ignored. +| This property was a workaround in an older version of Redpanda. It is no longer required and the property is ignored. | 23.3.1 | xref:reference:cluster-properties.adoc#delete_retention_ms[`delete_retention_ms`] @@ -189,6 +199,12 @@ This is an exhaustive list of all the deprecated properties. - id_allocator_replication +- kafka_admin_topic_api_rate + +- kafka_client_group_byte_rate_quota + +- kafka_client_group_fetch_byte_rate_quota + - max_version - min_version @@ -203,6 +219,10 @@ This is an exhaustive list of all the deprecated properties. - seq_table_min_size +- target_fetch_quota_byte_rate + +- target_quota_byte_rate + - tm_violation_recovery_policy - transaction_coordinator_replication diff --git a/modules/upgrade/pages/rolling-upgrade.adoc b/modules/upgrade/pages/rolling-upgrade.adoc index c7861d19f..d43cac80d 100644 --- a/modules/upgrade/pages/rolling-upgrade.adoc +++ b/modules/upgrade/pages/rolling-upgrade.adoc @@ -21,6 +21,8 @@ include::partial$rolling-upgrades/upgrade-limitations.adoc[] === Review incompatible changes +Starting in version 24.2, xref:manage:cluster-maintenance/manage-throughput.adoc#client-throughput-limits[client throughput limits] are compatible with the https://cwiki.apache.org/confluence/display/KAFKA/KIP-546%3A+Add+Client+Quota+APIs+to+the+Admin+Client[AlterClientQuotas and DescribeClientQuotas^] Kafka APIs. Redpanda determines client throughput limits on a per-broker basis. In earlier versions, client throughput quotas were applied from cluster configurations on a per-shard basis. + Patch releases in https://github.com/redpanda-data/redpanda/discussions/9522[22.3.14^] and https://github.com/redpanda-data/redpanda/discussions/9523[23.1.2^] changed the behavior when remote read is disabled and the requested Raft term falls below the local log's beginning. In earlier versions, Redpanda returned an offset -1. With the patch, when you request a value older than the lowest offset, Redpanda returns the lowest offset, not -1. == Find a new version diff --git a/package-lock.json b/package-lock.json index 290e474ca..df89ac68e 100644 --- a/package-lock.json +++ b/package-lock.json @@ -12,7 +12,7 @@ "@antora/cli": "3.1.2", "@antora/site-generator": "3.1.2", "@asciidoctor/tabs": "^1.0.0-beta.5", - "@redpanda-data/docs-extensions-and-macros": "^3.5.5" + "@redpanda-data/docs-extensions-and-macros": "latest" }, "devDependencies": { "@web/dev-server": "^0.2.1",