[BACKPORT pg15-cherrypicks] all: Bulk port from master - 99

Summary:
 aec9a66 [doc][xcluster] Truncate limitation (#23833)
 fe8890d [PLAT-13984] Change default metric graph points count from 100 to 250 + make runtime configurable.
 7d9b57b [#23869] YSQL: Fix one type of ddl atomicity stress test flakiness
 7c1bca8 [PLAT-14052][PLAT-15237] :Add advanced date-time option,Restrict CP for K8s
 afce6ad [PLAT-14158] Support multiple transactional xCluster configs per universe on YBA UI
 33342b3 [PLAT-15101] Add runtime config to turn on/off rollN for k8s
 361a99a [#23809] CDCSDK: Filter out records corresponding to index tables in colocated DBs
 8bbdf66 [docs] changed date (#23885)
 ee639f4 [#22104,#23506] YSQL: auto analyze service collects mutation counts from DDL and skips indexes
 3dbf6da Delete architecture/design/multi-region-xcluster-async-replication.md
 e013578 [#23864] DocDB: Move cluster_uuid to server common flags
 2b6a2d3 [PLAT-15062][PLAT-15071] Support DB scoped on UI and display schema change mode information
 8260075 [PLAT-15079] Treat dropped on target tables as unconfigured but preselected
 f5169ca DocDB: Follow redirects for callhome, and fix URL
 eb61ef6 [PLAT-15228] Update package installation command for YBA
 e791c40 [#18822] YSQL: Add serialization/deserialization mechanism for update optimization metadata
 e69d8cb [doc] Backups and DDLs (#23840)
 e72ae64 [PLAT-14552]filter support bundle core files by date
 8796c83 [PLAT-15274]: BackFill Pitr params for DR config and add not-null constraints in DB.
 dc9cc67 Fixed NPM test:ci build error
 2e5ebef [PLAT-15287]: Add PITR Column and Restore Window Information to Backup List
 d053e45 [#238989]yugabyted: Node doesn't join using `--join` flag
 da4da45 [#23399] DocDB: Fix StopActiveTxnsPriorTo from using local variables in the call back

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Differential Revision: https://phorge.dev.yugabyte.com/D38008

bin/yugabyted

@@ -8541,10 +8541,10 @@ class YBAdminProxy(object):
             YBAdminProxy.cmd_args.append('--certs_dir_name={}'.format(certs_dir_name[0].group(1)))
 
     @staticmethod
-    def add_master(master_addrs, new_master_ip, new_master_rpc_port, timeout=10):
+    def add_master(master_addrs, new_master_ip, new_master_rpc_port, timeout=30):
         cmd = YBAdminProxy.cmd_args + ["--init_master_addrs", master_addrs,
               "change_master_config", "ADD_SERVER", new_master_ip, str(new_master_rpc_port)]
-        out, err, ret_code = run_process(cmd, timeout=timeout, log_cmd=True)
+        out, err, ret_code = run_process_with_retries(cmd=cmd, timeout=timeout, log_cmd=True)
         return (0 == ret_code)
 
     @staticmethod
@@ -9625,6 +9625,29 @@ def run_process_checked(cmd, timeout=None, log_cmd=True, env_vars=None):
         Output.log_error_and_exit("Error: {}".format(err))
     return out
 
+def run_process_with_retries(cmd, encrypted_cmd=None, timeout=None, log_cmd=False, env_vars=None,
+                             shell=False, retries=10):
+    start_time = time.time()
+    now = start_time
+    try_count = 0
+    while True:
+        try_count+=1
+        if log_cmd:
+            Output.log("Running {}. Total retries: {}, Timeout: {}, Try count: {}".format(cmd,
+                                                                    retries, timeout, try_count))
+        out, err, retcode = run_process(cmd=cmd, encrypted_cmd=encrypted_cmd, timeout=timeout,
+                                        log_cmd=log_cmd, env_vars=env_vars, shell=shell)
+        now = time.time()
+        if retcode:
+            if now - start_time > timeout:
+                return (out, err, retcode)
+            elif try_count == retries:
+                return (out, err, retcode)
+            else:
+                time.sleep(0.2)
+        else:
+            return (out, err, retcode)
+
 def rmcontents(dirname, exclude_names=[]):
     for f in os.listdir(dirname):
         if f in exclude_names:

docs/content/preview/architecture/docdb-replication/async-replication.md

@@ -167,7 +167,7 @@ xCluster currently supports active-active single-master and active-active multi-
 
 ### Active-active single-master
 
-Here the replication is unidirectional from a source universe to a target universe. The target universe is typically located in data centers or regions that are different from the source universe. The source universe can serve both reads and writes. The target universe can only serve reads. Since only the nodes in one universe can take writes this mode is referred to as single master. Note that within the source universe all nodes can serve writes.
+In this setup the replication is unidirectional from a source universe to a target universe. The target universe is typically located in data centers or regions that are different from the source universe. The source universe can serve both reads and writes. The target universe can only serve reads. Since only the nodes in one universe can take writes this mode is referred to as single master. Note that within the source universe all nodes can serve writes.
 
 Usually, such deployments are used for serving low-latency reads from the target universes, as well as for disaster recovery purposes.  When used primarily for disaster recovery purposes, these deployments are also called active-standby because the target universe stands by to take over if the source universe is lost.
 
@@ -243,7 +243,7 @@ After losing one universe, the other universe may be left with torn transactions
 
 ### Transactional-mode limitations
 
-With transactional mode,
+Transactional mode has the following limitations:
 
 - No writes are allowed in the target universe
 - Active-active multi-master is not supported
@@ -259,7 +259,7 @@ When the source universe is lost, an explicit decision must be made to switch ov
 ### DDL changes
 
 - Currently, DDL changes are not automatically replicated.  Applying commands such as `CREATE TABLE`, `ALTER TABLE`, and `CREATE INDEX` to the target universes is your responsibility.
-- `DROP TABLE` is not supported.  You must first disable replication for this table.
+- `DROP TABLE` is not supported in YCQL.  You must first disable replication for this table.
 - `TRUNCATE TABLE` is not supported.  This is an underlying limitation, due to the level at which the two features operate.  That is, replication is implemented on top of the Raft WAL files, while truncate is implemented on top of the RocksDB SST files.
 - In the future, it will be possible to propagate DDL changes safely to other universes.  This is tracked in [#11537](https://github.com/yugabyte/yugabyte-db/issues/11537).
 

docs/content/preview/deploy/multi-dc/async-replication/_index.md

@@ -16,29 +16,18 @@ By default, YugabyteDB provides synchronous replication and strong consistency a
 
 For information on xCluster deployment architecture, replication scenarios, and limitations, refer to [xCluster replication](../../../architecture/docdb-replication/async-replication/).
 
-<div class="row">
-  <div class="col-12 col-md-6 col-lg-12 col-xl-6">
-    <a class="section-link icon-offset" href="async-deployment/">
-      <div class="head">
-        <img class="icon" src="/images/section_icons/explore/planet_scale.png"  aria-hidden="true" />
-        <div class="title">Deploy xCluster</div>
-      </div>
-      <div class="body">
-        Set up unidirectional or bidirectional replication.
-      </div>
-    </a>
-  </div>
+{{<index/block>}}
 
-  <div class="col-12 col-md-6 col-lg-12 col-xl-6">
-    <a class="section-link icon-offset" href="async-replication-transactional">
-      <div class="head">
-        <img class="icon" src="/images/section_icons/explore/planet_scale.png"  aria-hidden="true" />
-        <div class="title">Deploy transactional xCluster</div>
-      </div>
-      <div class="body">
-        Set up transactional unidirectional replication.
-      </div>
-    </a>
-  </div>
+  {{<index/item
+    title="Deploy xCluster"
+    body="Set up unidirectional or bidirectional replication."
+    href="async-deployment/"
+    icon="fa-light fa-copy">}}
 
-</div>
+  {{<index/item
+    title="Deploy transactional xCluster"
+    body="Set up transactional unidirectional replication."
+    href="async-replication-transactional/"
+    icon="fa-light fa-money-from-bracket">}}
+
+{{</index/block>}}

docs/content/preview/deploy/multi-dc/async-replication/async-replication-transactional.md

@@ -35,8 +35,8 @@ xCluster safe time is the transactionally consistent time across all tables in a
 
 Transactional xCluster can be set up in the following ways:
 
-- [Manual mode](../async-transactional-setup/).
 - [Semi-automatic mode](../async-transactional-setup-dblevel/), providing operationally simpler setup and management of replication, as well as simpler steps for performing DDL changes.
+- [Manual mode](../async-transactional-setup/).
 
 ## Limitations
 

docs/content/preview/manage/_index.md

@@ -5,8 +5,6 @@ linkTitle: Manage
 description: Manging YugabyteDB deployments
 image: /images/section_icons/quick_start/sample_apps.png
 headcontent: Manage your YugabyteDB deployment
-aliases:
-  - /preview/manage/
 menu:
   preview:
     identifier: manage
@@ -20,7 +18,7 @@ type: indexpage
     title="Back up and restore"
     body="Back up and restore data in YugabyteDB."
     href="backup-restore/"
-    icon="/images/section_icons/manage/backup.png">}}
+    icon="fa-light fa-life-ring">}}
 
   {{<index/item
     title="Migrate data"

docs/content/preview/manage/backup-restore/_index.md

@@ -3,7 +3,7 @@ title: Back up and restore data
 headerTitle: Backup and restore
 linkTitle: Backup and restore
 description: Back up and restore YugabyteDB
-image: /images/section_icons/manage/enterprise.png
+image: fa-light fa-life-ring
 headcontent: Create backups and restore your data
 aliases:
   - /manage/backup-restore/
@@ -20,28 +20,39 @@ Backup and restoration is the process of creating and storing copies of your dat
 
 Unlike traditional single-instance databases, YugabyteDB is designed for fault tolerance. By maintaining at least three copies of your data across multiple data regions or multiple clouds, it makes sure no losses occur if a single node or single data region becomes unavailable. Thus, with YugabyteDB, you would mainly use backups to:
 
-* Recover from a user or software error, such as accidental table removal.
-* Recover from a disaster scenario, like a full cluster failure or a simultaneous outage of multiple data regions. Even though such scenarios are extremely unlikely, it's still a best practice to maintain a way to recover from them.
-* Maintain a remote copy of data, as required by data protection regulations.
+- Recover from a user or software error, such as accidental table removal.
+- Recover from a disaster scenario, like a full cluster failure or a simultaneous outage of multiple data regions. Even though such scenarios are extremely unlikely, it's still a best practice to maintain a way to recover from them.
+- Maintain a remote copy of data, as required by data protection regulations.
+
+## Best practices
+
+- Don't perform cluster operations at the same time as your scheduled backup.
+- Configure your maintenance window and backup schedule so that they do not conflict.
+- Performing a backup or restore incurs a load on the cluster. Perform backup operations when the cluster isn't experiencing heavy traffic. Backing up during times of heavy traffic can temporarily degrade application performance and increase the length of time of the backup.
+- Avoid running a backup during or before a scheduled maintenance.
+
+{{< warning title="Backups and high DDL activity" >}}
+In some circumstances, a backup can fail during high DDL activity. Avoid performing major DDL operations during scheduled backups or while a backup is in progress.
+{{< /warning >}}
 
 {{<index/block>}}
 
   {{<index/item
     title="Export and import"
     body="Export and import data using SQL or CQL scripts."
     href="export-import-data/"
-    icon="/images/section_icons/manage/export_import.png">}}
+    icon="fa-light fa-file-import">}}
 
   {{<index/item
     title="Distributed snapshots"
     body="Back up and restore data using distributed snapshots."
     href="snapshot-ysql/"
-    icon="/images/section_icons/manage/backup.png">}}
+    icon="fa-light fa-camera">}}
 
   {{<index/item
     title="Point-in-time recovery"
     body="Restore data to a particular point in time."
     href="point-in-time-recovery/"
-    icon="/images/section_icons/manage/pitr.png">}}
+    icon="fa-light fa-timeline-arrow">}}
 
 {{</index/block>}}

docs/content/preview/releases/techadvisories/_index.md

@@ -23,7 +23,7 @@ It is strongly recommended that you take appropriate measures as outlined in the
 |YCQL currenttimestamp() precision
 | {{<product "ycql">}}
 | {{<release "All">}}
-| {{<nobreak "01 Januaury 2024">}}
+| {{<nobreak "27 August 2024">}}
 |
 | {{<ta 22935>}}
 |Potential issues with server-side sequence caching in multi-database clusters

docs/content/preview/releases/techadvisories/ta-23476.md

@@ -1,7 +1,7 @@
 ---
 title: TA-23476
 headerTitle: YCQL currenttimestamp() precision
-headcontent: 01 Jan 2024
+headcontent: 27 Aug 2024
 type: docs
 showRightNav: true
 cascade:

docs/content/preview/yugabyte-cloud/cloud-clusters/backup-clusters.md

@@ -53,6 +53,10 @@ You can't restore a backup to a cluster with an version of YugabyteDB that is ea
 
 Backups are not supported for Sandbox clusters.
 
+{{< warning title="Backups and high DDL activity" >}}
+In some circumstances, a backup can fail during high DDL activity. Avoid performing major DDL operations during scheduled backups or while a backup is in progress.
+{{< /warning >}}
+
 ## Recommendations
 
 - Don't perform cluster operations at the same time as your scheduled backup.

docs/content/preview/yugabyte-platform/back-up-restore-universes/_index.md

@@ -3,7 +3,7 @@ title: Back up and restore universes
 headerTitle: Back up and restore universes
 linkTitle: Back up universes
 description: Use YugabyteDB Anywhere to back up and restore YugabyteDB universe data.
-image: /images/section_icons/manage/backup.png
+image: fa-light fa-life-ring
 headcontent: Use YugabyteDB Anywhere to back up and restore YugabyteDB universes and data
 aliases:
   - /preview/manage/enterprise-edition/backup-restore
@@ -32,42 +32,53 @@ You can use YugabyteDB to schedule and manage backups of your universe data. Thi
 Configurable performance parameters and incremental backups are mediated using the yb-controller process, which is only available in YugabyteDB Anywhere v2.16 or later for universes with YugabyteDB version 2.16 or later.
 {{< /note >}}
 
+## Best practices
+
+- Don't perform cluster operations at the same time as your scheduled backup.
+- Configure your maintenance window and backup schedule so that they do not conflict.
+- Performing a backup or restore incurs a load on the cluster. Perform backup operations when the cluster isn't experiencing heavy traffic. Backing up during times of heavy traffic can temporarily degrade application performance and increase the length of time of the backup.
+- Avoid running a backup during or before a scheduled maintenance.
+
+{{< warning title="Backups and high DDL activity" >}}
+In some circumstances, a backup can fail during high DDL activity. Avoid performing major DDL operations during scheduled backups or while a backup is in progress. To view active tasks, navigate to **Tasks**.
+{{< /warning >}}
+
 {{<index/block>}}
 
   {{<index/item
     title="Configure backup storage"
     body="Configure the storage location for your backups."
     href="configure-backup-storage/"
-    icon="/images/section_icons/manage/backup.png">}}
+    icon="fa-light fa-bucket">}}
 
   {{<index/item
     title="Schedule universe data backups"
     body="Create backup schedules to regularly back up universe data."
     href="schedule-data-backups/"
-    icon="/images/section_icons/explore/high_performance.png">}}
+    icon="fa-light fa-calendar">}}
 
   {{<index/item
     title="Back up universe data"
     body="Back up universes and create incremental backups."
     href="back-up-universe-data/"
-    icon="/images/section_icons/manage/backup.png">}}
+    icon="fa-light fa-down-to-bracket">}}
 
   {{<index/item
     title="Restore universe data"
     body="Restore from full and incremental backups."
     href="restore-universe-data/"
-    icon="/images/section_icons/manage/backup.png">}}
+    icon="fa-light fa-up-to-bracket">}}
 
   {{<index/item
     title="Perform point-in-time recovery"
     body="Recover universe data from a specific point in time."
     href="pitr/"
-    icon="/images/section_icons/manage/pitr.png">}}
+    icon="fa-light fa-timeline-arrow">}}
 
   {{<index/item
     title="Disaster recovery"
     body="Fail over to a backup universe in case of unplanned outages."
     href="disaster-recovery/"
-    icon="/images/section_icons/manage/pitr.png">}}
+    icon="fa-light fa-sun-cloud">}}
 
 {{</index/block>}}

docs/content/preview/yugabyte-platform/back-up-restore-universes/back-up-universe-data.md

@@ -134,9 +134,9 @@ s3://user_bucket
 
 | Component | Description |
 | :-------- | :---------- |
-| Storage address | The name of the bucket as specified in the [backup configuration](../configure-backup-storage/) that was used for the backup. |
+| Storage address | The name of the bucket as specified in the [storage configuration](../configure-backup-storage/) that was used for the backup. |
 | Sub-directories | The path of the sub-folders (if any) in a bucket. |
-| Universe UUID | The UUID of the universe that was backed up. You can move this folder to different a location, but to successfully restore, do not modify this folder or any of its contents. |
+| Universe UUID | The UUID of the universe that was backed up. You can move this folder to different a location, but to successfully restore, do not modify this folder, or any of its contents. |
 | Backup series name and UUID | The name of the backup series and YBA-generated UUID. The UUID ensures that YBA can correctly identify the appropriate folder. |
 | Backup type | `full` or `incremental`. Indicates whether the subfolders contain full or incremental backups. |
 | Creation time | The time the backup was started. |