[improve] [pip] PIP-370: configurable remote topic creation in geo-replication

pip/pip-370.md

@@ -0,0 +1,48 @@
+# PIP-370: An optional flag: disable the replicators to automatically trigger topic creation
+
+# Background knowledge
+
+Users using Geo-Replication backup data across multi clusters, and Admin APIs related to Geo-Replication and internal replicators of brokers will trigger topics auto-creation between clusters.
+- For partitioned topics.
+  - After enabled namespace-level Geo-Replication: broker will create topics on the remote cluster automatically when calling `pulsar-admin topics create-partitioned-topic`. It does not depend on enabling `allowAutoTopicCreation`.
+  - When enabling topic-level Geo-Replication on a partitioned topic: broker will create topics on the remote cluster automatically. It does not depend on enabling `allowAutoTopicCreation`.
+- For non-partitioned topics and partitions of partitioned topics.
+  - The internal Geo-Replicator will trigger topics auto-creation for remote clusters. **(Highlight)** It depends on enabling `allowAutoTopicCreation`. In fact, this behavior is not related to Geo-Replication, it is the behavior of the internal producer of Geo Replicator,   
+
+# Motivation
+
+But if users want to maintain pulsar resources manually, there is no choice that prevent pulsar resources creation affects each other between clusters.
+
+**Passed solution**: disable `allowAutoTopicCreation`, the APIs `pulsar-admin topics create-partitioned-topic` still create topics on the remote cluster when enabled namespace level replication, the API `enable topic-level replication` still create topics, And the internal replicator will keep printing error logs due to a not found error.
+
+# Goals
+
+Introduce a flag that used to disable the replicators to automatically trigger topic creation.
+
+# Detailed Design
+
+## Configuration
+
+**broker.conf**
+```properties
+# It is not a dynamic config, default value is "true".
+# See details below.
+replicationTriggerRemoteTopicCreation=true
+```
+
+## Design & Implementation Details
+
+After disabled `replicationTriggerRemoteTopicCreation`.
+- After enabled namespace-level Geo-Replication: broker will not create topics on the remote cluster automatically when calling `pulsar-admin topics create-partitioned-topic`.
+- When enabling topic-level Geo-Replication on a partitioned topic: broker will not create topics on the remote cluster automatically.
+- The internal Geo-Replicator will not trigger topics auto-creation for remote clusters, it just keeps retrying to check if the topic exists on the remote cluster, once the topic was created, the replicator start.
+- It does not change the behavior that creating subscriptions after enable `enableReplicatedSubscriptions`, the subscription will also be created on the remote cluster after users enabled `enableReplicatedSubscriptions`.
+
+# Backward & Forward Compatibility
+
+The feature that disable `replicationTriggerRemoteTopicCreation` depends on the API `PulsarClient.getPartitionsForTopic(String topic, boolean metadataAutoCreationEnabled)`, which was introduced by [PIP-344](https://github.com/apache/pulsar/blob/master/pip/pip-344.md).
+- Only when user's all clusters support PIP-344, users can disable `replicationTriggerRemoteTopicCreation`( `>=3.0.6` or `>=3.3.1` ).
+
+# Links
+* Mailing List discussion thread:
+* Mailing List voting thread: