[improve] [pip] PIP-373: Add a topic's system prop that indicates whe…

…ther users have published TXN messages in before. (#23210)

[improve] [pip] PIP-373: Add a topic's system prop that indicates whether users have published TXN messages in before. (#23210)

pip/pip-373.md

@@ -0,0 +1,78 @@
+# PIP-373: Add a topic's system prop that indicates whether users have published TXN messages in before.
+
+# Background knowledge
+
+**Transaction Buffer**
+- It maintains `aborted` messages indexes and `ongoing` TXN messages indexes in memory, we call them `Two Indexes` in the following sections.
+- It helps to filter out the messages that have been aborted or are still binding with an `on-going` TXN, to avoid consuming aborted messages or TXN in-progress messages.
+
+**Transaction Buffer initializes when a topic is loading up**
+- It reads messages that contain TXN state from the original topic, to re-build `Two Indexes` into memory.
+
+**Transaction Buffer closing when a topic is unloading**
+- It takes a snapshot for `Two Indexes` to rebuild the states faster next loading up.
+
+# Motivation
+
+- Since TXN is a feature that across multiple namespaces, it can be enabled or disabled at the cluster level.
+- Transaction Buffer will try to re-build `Two Indexes` for every topic if you enable TXN.
+- If you have a huge number of topics in a namespace, the task that re-build `Two Indexes` costs huge resources(CPU, Memory).
+
+We'd better skip Transaction Buffer re-building `Two Indexes` if the topic it is related to does not contain TXN messages to save resources usage.
+
+# Goals
+
+Skip Transaction Buffer re-building `Two Indexes` if the topic it is related to does not contain TXN messages to save resources usage.
+
+## In Scope
+
+This PIP only focuses on improving the scenario in which users have never published TXN messages on a topic.
+
+## Out of Scope
+
+This PIP does not focus on the scenario that follows.
+- Published TXN messages.
+- Consumed all TXN messages that were sent before.
+- Transaction Buffer keeps re-building `Two Indexes` even if there are no TXN messages in the topic anymore, and it costs resources.
+
+# Detailed Design
+
+## Design & Implementation Details
+
+- Add a topic-level system property named `__contains_txn_messages`, the default value is `false`, and it will be set to `true` when the first TXN message is publishing.
+- Transaction Buffer skips re-building `Two Indexes` if the property is `false`.
+
+## Public-facing Changes
+
+The topic property `__contains_txn_messages` becomes to a system property, it can never be used by users anymore. 
+
+### Public API
+- You will get a `NotAllowedException` when you try to set/remove a topic property named `__contains_txn_messages` by the API `pulsar-admin topics update-properties/remove-properties`.
+- The value of the property `__contains_txn_messages` can be queried by `pulsar-admin topics get-properties`.
+
+
+### Metrics
+| Name                                       | Description                                         | Attributes | Units|
+|--------------------------------------------|-----------------------------------------------------| --- | --- |
+| `pulsar_broker_using_txn_topics_count` | Counter. The number of topics contains TXN messages. | cluster | - |
+
+# Backward & Forward Compatibility
+
+## Upgrade
+
+There are `3` scenarios that the topic's property `__contains_txn_messages` is when users try to upgrade.
+- `__contains_txn_messages` is empty: broker initializes it by confirming whether there is TXN messages or not.
+- **(Highlight)** `__contains_txn_messages` is not empty and is not typed `boolean`, rollback to the original behavior that always re-building `Two Indexes`, but you can never modify it anymore.
+- **(Highlight)** `__contains_txn_messages` is not empty and is typed `boolean`, but it is a users' property, broker assumed that it is a system property.
+
+## Downgrade / Rollback
+
+You can downgrade or roll back gracefully.
+
+## Pulsar Geo-Replication Upgrade & Downgrade/Rollback Considerations
+
+The PIP does not affect Geo-Replication.
+
+# Links
+* Mailing List discussion thread: https://lists.apache.org/thread/7mblhyvsrw5zybo0gs5512xg8f9sm67v
+* Mailing List voting thread: https://lists.apache.org/thread/s6z5gcjyw081cxf9pwz361r8dt2k8gvl