diff --git a/.github/vale-styles/Yugabyte/spelling-exceptions.txt b/.github/vale-styles/Yugabyte/spelling-exceptions.txt index e278e0a4185c..47e474924869 100644 --- a/.github/vale-styles/Yugabyte/spelling-exceptions.txt +++ b/.github/vale-styles/Yugabyte/spelling-exceptions.txt @@ -445,6 +445,7 @@ Patroni performant PgBouncer pgLoader +pg_cron pg_recvlogical Phabricator phaser diff --git a/docs/content/preview/architecture/docdb-replication/async-replication.md b/docs/content/preview/architecture/docdb-replication/async-replication.md index dbeb3197126b..53322f2675d6 100644 --- a/docs/content/preview/architecture/docdb-replication/async-replication.md +++ b/docs/content/preview/architecture/docdb-replication/async-replication.md @@ -161,12 +161,11 @@ Today, this is done by backing up the source universe and restoring it to the ta Ongoing work, [#17862](https://github.com/yugabyte/yugabyte-db/issues/17862), will replace using backup and restore here with directly copying RocksDB files between the source and target universes. This will be more performant and flexible and remove the need for external storage like S3 to set up replication. - ## Supported deployment scenarios xCluster currently supports active-active single-master and active-active multi-master deployments. -### Active- active single-master +### Active-active single-master Here the replication is unidirectional from a source universe to a target universe. The target universe is typically located in data centers or regions that are different from the source universe. The source universe can serve both reads and writes. The target universe can only serve reads. Since only the nodes in one universe can take writes this mode is referred to as single master. Note that within the source universe all nodes can serve writes. diff --git a/docs/content/preview/explore/ysql-language-features/pg-extensions/_index.md b/docs/content/preview/explore/ysql-language-features/pg-extensions/_index.md index e4d3ebee66b8..cd15fec115f9 100644 --- a/docs/content/preview/explore/ysql-language-features/pg-extensions/_index.md +++ b/docs/content/preview/explore/ysql-language-features/pg-extensions/_index.md @@ -59,6 +59,7 @@ YugabyteDB supports the following additional extensions, some of which you must | [HypoPG](extension-hypopg/) | Pre-bundled | Create hypothetical indexes to test whether an index can increase performance for problematic queries without consuming any actual resources. | | Orafce | Pre-bundled | Provides compatibility with Oracle functions and packages that are either missing or implemented differently in YugabyteDB and PostgreSQL. This compatibility layer can help you port your Oracle applications to YugabyteDB.
For more information, see the [Orafce](https://github.com/orafce/orafce) documentation. | | [PGAudit](../../../secure/audit-logging/audit-logging-ysql/) | Pre-bundled | The PostgreSQL Audit Extension (pgaudit) provides detailed session and/or object audit logging via the standard PostgreSQL logging facility. | +| [pg_cron](extension-pgcron/) | Pre-bundled | Provides a cron-based job scheduler that runs inside the database. | | [pg_hint_plan](../../query-1-performance/pg-hint-plan/#root) | Pre-bundled | Tweak execution plans using "hints", which are descriptions in the form of SQL comments.
For more information, see the [pg_hint_plan](https://pghintplan.osdn.jp/pg_hint_plan.html) documentation. | | pg_stat_monitor | Pre-bundled | A PostgreSQL query performance monitoring tool, based on the PostgreSQL pg_stat_statements module.
For more information, see the [pg_stat_monitor](https://docs.percona.com/pg-stat-monitor/index.html) documentation. | | [pgvector](extension-pgvector) | Pre-bundled | Allows you to store and query vectors, for use in vector similarity searching. | diff --git a/docs/content/preview/explore/ysql-language-features/pg-extensions/extension-pgcron.md b/docs/content/preview/explore/ysql-language-features/pg-extensions/extension-pgcron.md new file mode 100644 index 000000000000..6b2dfe9301ff --- /dev/null +++ b/docs/content/preview/explore/ysql-language-features/pg-extensions/extension-pgcron.md @@ -0,0 +1,66 @@ +--- +title: pg_cron extension +headerTitle: pg_cron extension +linkTitle: pg_cron +description: Using the pg_cron extension in YugabyteDB +techPreview: /preview/releases/versioning/#feature-availability +menu: + preview: + identifier: extension-pgcron + parent: pg-extensions + weight: 20 +type: docs +--- + +The [pg_cron](https://github.com/citusdata/pg_cron) extension provides a cron-based job scheduler that runs inside the database. It uses the same syntax as regular cron, and allows you to schedule YSQL commands directly from the database. You can also use '[1-59] seconds' to schedule a job based on an interval. + +YugabyteDB supports all features of the pg_cron extension. Although YugabyteDB is a distributed database that operates on multiple nodes, pg_cron only runs on one of these nodes, called the pg_cron leader. Only the pg_cron leader schedules and runs the cron jobs. The queries executed by jobs do take advantage of all available resources in the cluster. + +If the pg_cron leader node fails, another node is automatically elected as the new leader to ensure it is highly available. This process is transparent, and you can connect to any node in a cluster to schedule jobs. + +## Set up pg_cron + +pg_cron in YugabyteDB is {{}}. Before you can use the feature, you must enable it by setting the `enable_pg_cron` flag. To do this, add `enable_pg_cron` to the `allowed_preview_flags_csv` flag and set the `enable_pg_cron` flag to true on all YB-Masters and YB-TServers. + +The pg_cron extension is installed on only one database, which stores the extension data. The default cron database is `yugabyte`. You can change it by setting the `ysql_cron_database_name` flag on all YB-TServers. You can create the database after setting the flag. + +For example, to create a single-node [yugabyted](../../../../reference/configuration/yugabyted/) cluster with pg_cron on database 'db1', use the following command: + +```sh +./bin/yugabyted start --master_flags "allowed_preview_flags_csv={enable_pg_cron},enable_pg_cron=true" --tserver_flags "allowed_preview_flags_csv={enable_pg_cron},enable_pg_cron=true,ysql_cron_database_name=db1" --ui false +``` + +To change the database after the extension is created, you must first drop the extension and then change the flag value. + +## Enable pg_cron + +Create the extension as superuser on the cron database. + +```sql +CREATE EXTENSION pg_cron; +``` + +You can grant access to other users to use the extension. For example: + +```sql +GRANT USAGE ON SCHEMA cron TO elephant; +``` + +## Use pg_cron + +YugabyteDB supports all features and syntax of the pg_cron extension. + +For example, the following command calls a stored procedure every five seconds: + +```sql +SELECT cron.schedule('process-updates', '5 seconds', 'CALL process_updates()'); +``` + +If you need to run jobs in multiple databases, use `cron.schedule_in_database()`. + +When running jobs, keep in mind the following: + +- It may take up to 60 seconds for job changes to get picked up by the pg_cron leader. +- When a new pg_cron leader node is elected, no jobs are run for the first minute. Any job that were in flight on the failed node will not be retried, as their outcome is not known. + +For more information on how to schedule jobs, refer to the [pg_cron documentation](https://github.com/yugabyte/yugabyte-db/blob/master/src/postgres/third-party-extensions/pg_cron/README.md). diff --git a/docs/content/preview/reference/configuration/yb-tserver.md b/docs/content/preview/reference/configuration/yb-tserver.md index 11cebc2cda23..56f7a9867369 100644 --- a/docs/content/preview/reference/configuration/yb-tserver.md +++ b/docs/content/preview/reference/configuration/yb-tserver.md @@ -855,6 +855,14 @@ Default: `-1` (disables logging statement durations) Specifies the lowest YSQL message level to log. +##### --ysql_cron_database_name + +Specifies the database where pg_cron is to be installed. You can create the database after setting the flag. + +The [pg_cron extension](../../../explore/ysql-language-features/pg-extensions/extension-pgcron/) is installed on only one database (by default, `yugabyte`). + +To change the database after the extension is created, you must first drop the extension and then change the flag value. + ##### --ysql_output_buffer_size Size of YSQL layer output buffer, in bytes. YSQL buffers query responses in this output buffer until either a buffer flush is requested by the client or the buffer overflows. diff --git a/docs/content/preview/releases/yba-releases/v2024.1.md b/docs/content/preview/releases/yba-releases/v2024.1.md index 9cc8da06d704..7c16c2af89da 100644 --- a/docs/content/preview/releases/yba-releases/v2024.1.md +++ b/docs/content/preview/releases/yba-releases/v2024.1.md @@ -27,6 +27,171 @@ If you have a Replicated installation, you must migrate from Replicated to YBA I YugabyteDB 2024.1.0.0 and newer releases do not support v7 Linux versions (CentOS7, Red Hat Enterprise Linux 7, Oracle Enterprise Linux 7.x), Amazon Linux 2, and Ubuntu 18. If you're currently using one of these Linux versions, upgrade to a supported OS version before installing YugabyteDB v2024.1.0. Refer to [Operating system support](/stable/reference/configuration/operating-systems/) for the complete list of supported operating systems. {{}} +## v2024.1.1.0 - July 31, 2024 {#v2024.1.1.0} + +**Build:** `2024.1.1.0-b137` + +**Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.1.0/yugabytedb-2024.1.1.0-b137-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.1.0/yugabytedb-anywhere-2024.1.1.0-b137-third-party-licenses.html) + +### Download + + + +### Highlights + +**CDC Observability** {{}} + +YugabyteDB Anywhere supports monitoring the status of replication slots when using [PostgreSQL Logical Replication Protocol](/stable/explore/change-data-capture/using-logical-replication/) for Change Data Capture (CDC). + +For more information, refer to [CDC Observability](/stable/yugabyte-platform/alerts-monitoring/cdc-observability/). + +### New features + +* [Provider validation](/stable/yugabyte-platform/configure-yugabyte-platform/aws/#create-a-provider). When creating public cloud providers (for AWS, Azure, GCP) and Kubernetes providers, YugabyteDB Anywhere now performs validation checks on some user-inputted fields, reducing downstream errors and speeding the configuration process. +* [Support for Exporting Metrics to a Separate, External Prometheus](/stable/yugabyte-platform/alerts-monitoring/prometheus-custom/). YugabyteDB Anywhere supports export of universe time-series metrics to an external Prometheus instance. The documentation shows how to do this for both VM-based and Kubernetes-based universes. + +
+ View the detailed changelog + +### Improvements + +* Changes the default permission to `600` for the backup manifest file to accommodate immutable NAS devices. PLAT-13578 +* Adds a toggle in the UI to suppress health check notifications during maintenance windows. PLAT-13856 +* Allows Ansible tasks to run seamlessly even with very long host names by using hash for Ansible/SSH control path. PLAT-13938 +* Allows setting up of YugabyteDB in AWS Singapore Government's GCC Plus environment by extending AZ name limit from 25 to 100 characters. PLAT-13212 +* Introduces a new feature, `tablet guardrail mechanism`, that triggers a warning when the ratio of live tablet peers to the supportable tablet peers is more than 0.9, and a severe alert when it's more than 1.0. Available from 2024.1 onwards. PLAT-13520 +* Turns off "assign public IP" option in Azure create universe by default. PLAT-13948 + +### Bug fixes + +* Enables usage of underscores in GCS bucket names during GCP Backup configuration. PLAT-13266 +* Enables consistent generation of new incremental backup times in the event of clock skewness. PLAT-13375 +* Allows users to filter out resolved alert notifications via WebHook channel configuration. PLAT-13506 +* Stops health check alerts during an active maintenance window by introducing a new parameter, `SuppressHealthCheckNotificationsConfig`, to the `MaintenanceWindow` model class and its APIs. PLAT-13518 +* Allows deletion of expired, aborted, or failed backups, removes redundant backups when a schedule is deleted, retries backup deletion before marking it as `Failed To Delete`, and queues ongoing deletions for later when YBA restarts. PLAT-13750 +* Lets users rotate node-to-node certificates alone without client-to-node encryption enabled. PLAT-13806 +* Unsnoozes all universe level health check notifications to encourage use of the maintenance window function. PLAT-13928 +* Corrects the checksum mismatch in the V342 migration to ensure successful upgrades from 2.14 to later branches. PLAT-13977 +* Automatically re-fetches access tokens before expiring for continuous user sessions when `offline_access` scope is set. PLAT-14042,PLAT-11246 +* Allows runtime configuration get API to return the correct inherited value, not just the parent scope value. PLAT-14090 +* Conceals DB user's password to prevent exposure in the application log during the upgrade procedure. PLAT-14286 +* Adjusts yml task to retain the last old release during the release GC process. PLAT-14368 +* Corrects the calculation of affected nodes in the certificate alerts message. PLAT-14385 +* Reduces security risks by storing hashed API tokens instead of actual tokens in the users table. PLAT-8028 +* Allows backing up, avoiding repetitive full backup tasks and updating incremental backup time only after passing all validation checks. PLAT-14497 +* Fixes the issue of tests failing at create universe on Itests Alma8 AMI due to expired GPG keys. PLAT-14136 +* Eliminates file descriptor leaks enhancing database stability. PLAT-13665 +* Removes the misuse of defaultImageBundle in universe when custom AMI is specified using YBA's machineImage. PLAT-13800 +* Allows for updated machineImage passing from nodeDetails in disk operations, preventing edit universe tasks failure due to missing AMIs in AWS clusters. PLAT-13808 +* Upgrades PostgreSQL version to the latest 42.3.x addressing critical vulnerabilities. PLAT-13824 +* Redirects stderr logs in yb_backup.py to prevent kubectl warn logs from disrupting remote command executions. PLAT-14012 +* Allows error-free query for releases with artifacts of a specific deployment type by excluding artifacts without a linked release. PLAT-14057 +* Ensures Edit Kubernetes Universe tasks only re-run after validating previous task parameters. PLAT-14203 +* Allows the collect_metrics.timer and bind_check.service to operate independently, avoiding system hang-ups due to cyclic dependency issues. PLAT-14293 +* Corrects the failure of Azure Provider validation due to unexpected mandatory fields. PLAT-14530 +* Repairs build failure in CentOS 7 pex/yugabundle builder Docker image. PLAT-14543 +* Corrects the failure of Azure Provider validation due to unexpected mandatory fields. PLAT-14530 +* Fixes software upgrade process to ensure master nodes are correctly categorized as `inactive` or `active`. PLAT-14561,PLAT-14153 +* Adjusts missing backport in version 2024.1.1 to correct discrepancy in default core file component value label from bytes to GB. PLAT-14593 +* Upgrades go etcd and cyphar dependencies in yba-installer, enhancing security by fixing vulnerabilities. PLAT-12335 +* Upgrades mina-core package to a secure version 2.2.3 and paramiko to a non-vulnerable version 3.4.0. PLAT-12336 +* Eliminates duplicate `exported_instance` label from Prometheus targets of DB exported metrics. PLAT-12808 +* Changes `localProvider` key from `task` to `input` for its availability during sbt tasks. PLAT-13367 +* Updates AWS metadata to include new regions. PLAT-13623 +* Removes internal flags related to providers now enabled by default. PLAT-13743 +* Lets you store node metrics in the yb_home directory instead of the /tmp directory. PLAT-13755 +* Hides autoflags from the display when listing gflags in the user interface. PLAT-13794 +* Upgrades python cryptography to 42.0.4 and setuptools to 65.5.1, enhancing security. PLAT-13836,PLAT-13835 +* Ensures CPU architecture selection is enabled for all providers, not just AWS, improving DB Versions API usage. PLAT-13852 +* Alters snooze alert behavior to also suppress universe health check alerts during maintenance windows. PLAT-13857 +* Fixes an issue that caused data from newer backups to be wrongly restored during the restoration of older backups. PLAT-13905 +* Allows normal workflows like systemd upgrade to function even when AMI is deleted from the cloud console. PLAT-13971 +* Resolves the issue of universe creation failure due to incorrect Image Bundle UUID by using the specified ec2-user. PLAT-14004 +* Nullifies possibility of Null Pointer Exception when using old storage config based proxy without username. PLAT-14143 +* Allows default use of M-series instance types on AWS. PLAT-14196 +* Prevents removal of overrides during volume resizing in Kubernetes. PLAT-14198 +* Changes permissions to allow traverse through `yb_home` directory, facilitating the collection of custom metrics. PLAT-14216 +* Ensures keyspace is not left empty during restore API requests, preventing restoration issues. PLAT-14221 +* Adds ConnectOnly role to LDAP group table constraint for better access control. PLAT-14230 +* Enables setting of sshUser/Port from the overrides for backward compatibility, fixing Provider Edit to successfully update image Bundle. PLAT-14244 +* Allows handling of large output in remote commands to prevent hanging. PLAT-14342 +* Fixes the `current lag` stat in xCluster to be table & stream specific, not influenced by other universes. PLAT-14425 +* Enhances yba installer migrations to skip certain versions during backporting which can be applied later on upgrades. PLAT-14511 +* Fixes issue where clicking preview clears data and doesn't display correct information when setting up ysql_ident or ysql_hba multiline flags. PLAT-14515 +* Allows customization of the yba installer timeout duration to avoid process failure due to long startups. PLAT-14443 +* Allows processing of all local releases during an import without failing due to local file issues. PLAT-14532 +* Allows fetching of static flags metadata for version 2.16+ from DB package when editing gflags via UI. PLAT-14533 +* Allows YBA to call `need_bootstrap` for older universes without the `indexed_table_id`, preventing errors. PLAT-14540 +* Removes SystemdUpgrade from IN_TRANSIT list to address failures on -gcp-rf3 on master build. PLAT-13770 +* Refines empty list component styling and deactivates the action button on linux version catalog when no versions are present for consistency. PLAT-13776,PLAT-13807 +* Allows increasing tserver volume size in edit universe mode for K8 and enables resize of master volumes. PLAT-13920 +* Allows display of total CPU usage graph in YBA metrics pages alongside user and system CPU usage. PLAT-14025 +* Introduces `follower_lag_ms` metric to the dashboard for easier identification of lagging masters and struggling tservers. PLAT-14254 +* Adds validation to disallow the non-restart upgrade option during rollback. PLAT-14390 +* Eliminates deprecated `vnetName/securityGroupId` fields from the region object during provider editing. PLAT-14802 +* Reduces local Provider's flakiness by using dedicated temporary directories and separate nfs backup directory. PLAT-13252 +* Allows safe extraction of DB files in a multi-thread environment by synchronizing conflicting buffer reads. PLAT-14160 +* Resolves an issue in yb_backup.py where the `stderr` keyword argument was incorrectly passed. PLAT-14208 +* Facilitates handling multiple comma-separated hostnames in YBA installer, enhancing template files, status commands, and reconfiguration wait times. PLAT-13096 +* Increases YBC client and server version to 2.1.0.1-b1, optimizing catalog version mechanism. PLAT-14575 +* Adjusts missing backport in version 2024.1.1 to correct discrepancy in default core file component value label from bytes to GB. PLAT-14593 +* Fixes user interface issues concerning the releases UX design, ensuring consistency across various modal and panel displays. PLAT-14607 +* Updates the log file names for YB-Controller logs and adds missing symlink `yb-controller-server.{INFO|WARN|ERROR}` to enhance troubleshooting. PLAT-14609,PLAT-14594 +* Allows preserving the uploaded YBDB builds by relocating the directory, solving the issue of directory deletion after container restarts. PLAT-14655 +* Upgrades go etcd and cyphar dependencies in yba-installer, enhancing security by fixing vulnerabilities. PLAT-12335 +* Upgrades mina-core package to a secure version 2.2.3 and paramiko to a non-vulnerable version 3.4.0. PLAT-12336 +* Eliminates duplicate `exported_instance` label from Prometheus targets of DB exported metrics. PLAT-12808 +* Changes `localProvider` key from `task` to `input` for its availability during sbt tasks. PLAT-13367 +* Updates AWS metadata to include new regions. PLAT-13623 +* Removes internal flags related to providers now enabled by default. PLAT-13743 +* Lets you store node metrics in the yb_home directory instead of the /tmp directory. PLAT-13755 +* Hides autoflags from the display when listing gflags in the user interface. PLAT-13794 +* Upgrades python cryptography to 42.0.4 and setuptools to 65.5.1, enhancing security. PLAT-13836,PLAT-13835 +* Ensures CPU architecture selection is enabled for all providers, not just AWS, improving DB Versions API usage. PLAT-13852 +* Alters snooze alert behavior to also suppress universe health check alerts during maintenance windows. PLAT-13857 +* Fixes an issue that caused data from newer backups to be wrongly restored during the restoration of older backups. PLAT-13905 +* Allows normal workflows like systemd upgrade to function even when AMI is deleted from the cloud console. PLAT-13971 +* Resolves the issue of universe creation failure due to incorrect Image Bundle UUID by using the specified ec2-user. PLAT-14004 +* Nullifies possibility of Null Pointer Exception when using old storage config based proxy without username. PLAT-14143 +* Allows default use of M-series instance types on AWS. PLAT-14196 +* Prevents removal of overrides during volume resizing in Kubernetes. PLAT-14198 +* Changes permissions to allow traverse through `yb_home` directory, facilitating the collection of custom metrics. PLAT-14216 +* Ensures keyspace is not left empty during restore API requests, preventing restoration issues. PLAT-14221 +* Adds ConnectOnly role to LDAP group table constraint for better access control. PLAT-14230 +* Enables setting of sshUser/Port from the overrides for backward compatibility, fixing Provider Edit to successfully update image Bundle. PLAT-14244 +* Allows handling of large output in remote commands to prevent hanging. PLAT-14342 +* Fixes the `current lag` stat in xCluster to be table & stream specific, not influenced by other universes. PLAT-14425 +* Enhances yba installer migrations to skip certain versions during backporting which can be applied later on upgrades. PLAT-14511 +* Fixes issue where clicking preview clears data and doesn't display correct information when setting up ysql_ident or ysql_hba multiline flags. PLAT-14515 +* Allows customization of the yba installer timeout duration to avoid process failure due to long startups. PLAT-14443 +* Allows processing of all local releases during an import without failing due to local file issues. PLAT-14532 +* Allows fetching of static flags metadata for version 2.16+ from DB package when editing gflags via UI. PLAT-14533 +* Allows YBA to call `need_bootstrap` for older universes without the `indexed_table_id`, preventing errors. PLAT-14540 +* Upgrades YBC client and server version to 2.1.0.1-b3 for better compatibility with both Alma 8.9 and centOS 7 universes. PLAT-14722 +* Removes SystemdUpgrade from IN_TRANSIT list to address failures on -gcp-rf3 on master build. PLAT-13770 +* Refines empty list component styling and deactivates the action button on linux version catalog when no versions are present for consistency. PLAT-13776,PLAT-13807 +* Allows increasing tserver volume size in edit universe mode for K8 and enables resize of master volumes. PLAT-13920 +* Allows display of total CPU usage graph in YBA metrics pages alongside user and system CPU usage. PLAT-14025 +* Introduces `follower_lag_ms` metric to the dashboard for easier identification of lagging masters and struggling tservers. PLAT-14254 +* Adds validation to disallow the non-restart upgrade option during rollback. PLAT-14390 +* Eliminates deprecated `vnetName/securityGroupId` fields from the region object during provider editing. PLAT-14802 +* Updates task_uuid as a key label for proper in-memory updates, enhancing task analysis capabilities. PLAT-14017 +* Reduces local Provider's flakiness by using dedicated temporary directories and separate nfs backup directory. PLAT-13252 +* Allows safe extraction of DB files in a multi-thread environment by synchronizing conflicting buffer reads. PLAT-14160 +* Resolves an issue in yb_backup.py where the `stderr` keyword argument was incorrectly passed. PLAT-14208 +* Facilitates handling multiple comma-separated hostnames in YBA installer, enhancing template files, status commands, and reconfiguration wait times. PLAT-13096 +* Increases YBC client and server version to 2.1.0.1-b1, optimizing catalog version mechanism. PLAT-14575 +* Ensures nodes no longer get stuck in "VM image upgrade" state after upgrading the Linux version. PLAT-14731 + +
+ ## v2024.1.0.0 - June 4, 2024 {#v2024.1.0.0} **Build:** `2024.1.0.0-b129` diff --git a/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16-anywhere.md b/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16-anywhere.md index 0a4d0510f888..d21b03a17568 100644 --- a/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16-anywhere.md +++ b/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16-anywhere.md @@ -34,10 +34,6 @@ type: docs ## Release notes -{{< tip title="YugabyteDB Anywhere release notes have moved" >}} -The release notes for YugabyteDB Anywhere have moved here. The [YugabyteDB v2.16 release notes](../../ybdb-releases/v2.16/) (and all other releases) are still available in their original location. -{{< /tip >}} - Included here are the release notes for all releases in the **YugabyteDB Anywhere** (YBA) v2.16 series. Content will be added as new notable features and changes are available in the patch releases of the YBA v2.16 series. For an RSS feed of all release series to track the latest product updates, point your feed reader to the [RSS feed for releases](../../index.xml). @@ -54,7 +50,7 @@ If you're using a previous v2.16 database version, you won't be able to successf **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.8.0/yugabytedb-2.16.9.0-b67-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.9.0/yugabytedb-anywhere-2.16.9.0-b67-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### New features @@ -82,7 +78,7 @@ For instructions on downloading and installing YugabyteDB Anywhere, refer to [In **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.8.0/yugabytedb-2.16.8.0-b16-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.8.0/yugabytedb-anywhere-2.16.8.0-b16-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### Improvements @@ -187,7 +183,7 @@ N/A **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.7.0/yugabytedb-2.16.7.0-b59-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.7.0/yugabytedb-anywhere-2.16.7.0-b59-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### Improvements @@ -241,7 +237,7 @@ N/A **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.6.1/yugabytedb-2.16.6.1-b1-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.6.1/yugabytedb-anywhere-2.16.6.1-b1-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### Bug fixes @@ -253,7 +249,7 @@ For instructions on downloading and installing YugabyteDB Anywhere, refer to [In **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.6.0/yugabytedb-2.16.6.0-b43-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.6.0/yugabytedb-anywhere-2.16.6.0-b43-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### New features @@ -312,7 +308,7 @@ N/A **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.5.0/yugabytedb-2.16.5.0-b24-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.5.0/yugabytedb-anywhere-2.16.5.0-b24-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### Improvements @@ -347,7 +343,7 @@ N/A -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### New features @@ -393,7 +389,7 @@ N/A **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.3.0/yugabytedb-2.16.3.0-b43-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.3.0/yugabytedb-anywhere-2.16.3.0-b43-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### New features @@ -440,7 +436,7 @@ N/A **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.2.0/yugabytedb-2.16.2.0-b41-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.2.0/yugabytedb-anywhere-2.16.2.0-b41-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### Improvements @@ -498,7 +494,7 @@ This is a database-only release, with no changes to YugabyteDB Anywhere. **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.1.0/yugabytedb-2.16.1.0-b50-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.1.0/yugabytedb-anywhere-2.16.1.0-b50-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### New features @@ -564,7 +560,7 @@ N/A **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.0.1/yugabytedb-2.16.0.1-b7-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.0.1/yugabytedb-anywhere-2.16.0.1-b7-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### Improvements @@ -587,7 +583,7 @@ N/A **Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2.16.0.0/yugabytedb-2.16.0.0-b90-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2.16.0.0/yugabytedb-anywhere-2.16.0.0-b90-third-party-licenses.html) -For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../yugabyte-platform/install-yugabyte-platform/). +For instructions on downloading and installing YugabyteDB Anywhere, refer to [Install YugabyteDB Anywhere](../../../../yugabyte-platform/install-yugabyte-platform/). ### New features diff --git a/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16.md b/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16.md index 10457223e098..d3f20b36f99c 100644 --- a/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16.md +++ b/docs/content/preview/releases/ybdb-releases/end-of-life/v2.16.md @@ -34,14 +34,8 @@ type: docs ## Release notes -{{< tip title="YugabyteDB Anywhere release notes have moved" >}} -The [release notes for YugabyteDB Anywhere](../../yba-releases/) have moved to [their own page](../../yba-releases/v2.16/). -{{< /tip >}} - What follows are the release notes for the YugabyteDB v2.16 series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.16 series. -For an RSS feed of all release series to track the latest product updates, point your feed reader to the [RSS feed for releases](../index.xml). - {{< warning title="Technical Advisory: Use database version 2.16.1.1 or later" >}} YugabyteDB releases in the v2.16 series prior to v2.16.1.1 contain a backup issue: if a database contains a pre-split range-partitioned table whose _primary key constraint column ordering_ isn't the same as the _table's column ordering_. @@ -340,7 +334,7 @@ docker pull yugabytedb/yugabyte:2.16.6.1-b1 ### Bug fixes -This release includes a [fix for YugabyteDB Anywhere](../../yba-releases/v2.16/#v2.16.6.1) and no changes to YugabyteDB (the core database). +This release includes a [fix for YugabyteDB Anywhere](../v2.16-anywhere/#v2.16.6.1) and no changes to YugabyteDB (the core database). ## v2.16.6.0 - July 20, 2023 {#v2.16.6.0} diff --git a/docs/content/preview/releases/ybdb-releases/v2024.1.md b/docs/content/preview/releases/ybdb-releases/v2024.1.md index e0e85aa079ed..ad14c7c11363 100644 --- a/docs/content/preview/releases/ybdb-releases/v2024.1.md +++ b/docs/content/preview/releases/ybdb-releases/v2024.1.md @@ -25,7 +25,7 @@ YugabyteDB 2024.1.0.0 and newer releases do not support v7 Linux versions (CentO YugabyteDB uses [memory division flags](/stable/reference/configuration/yb-master/#memory-division-flags) to specify how memory should be divided between different processes (for example, [YB-TServer](/stable/architecture/yb-tserver/) versus [YB-Master](/stable/architecture/yb-master/)) on a YugabyteDB node as well as within processes. Using these flags, you can better allocate memory for PostgreSQL, making it more suitable for a wider range of use cases. -For _new_ v2024.1.x universes, if you are expecting to use any nontrivial amount of [YSQL](/stable/api/ysql/), it is strongly recommended to set [ ‑‑use_memory_defaults_optimized_for_ysql](/stable/reference/configuration/yb-tserver/#use-memory-defaults-optimized-for-ysql). This changes the memory division defaults to better values for YSQL usage, and optimizes memory for the node size. +For _new_ v2024.1.x universes, if you are expecting to use any nontrivial amount of [YSQL](/stable/api/ysql/), it is strongly recommended to set [‑‑use_memory_defaults_optimized_for_ysql](/stable/reference/configuration/yb-tserver/#use-memory-defaults-optimized-for-ysql). This changes the memory division defaults to better values for YSQL usage, and optimizes memory for the node size. If you are _upgrading_ a universe, you may want to instead review your memory division settings and adjust them if desired; see [best practices](/stable/develop/best-practices-ysql/#minimize-the-number-of-tablets-you-need). @@ -33,6 +33,169 @@ In future releases, the memory division settings will be used to determine how m {{< /tip >}} +## v2024.1.1.0 - July 31, 2024 {#v2024.1.1.0} + +**Build:** `2024.1.1.0-b137` + +**Third-party licenses:** [YugabyteDB](https://downloads.yugabyte.com/releases/2024.1.1.0/yugabytedb-2024.1.1.0-b137-third-party-licenses.html), [YugabyteDB Anywhere](https://downloads.yugabyte.com/releases/2024.1.1.0/yugabytedb-anywhere-2024.1.1.0-b137-third-party-licenses.html) + +### Downloads + + + +**Docker:** + +```sh +docker pull yugabytedb/yugabyte:2024.1.1.0-b137 +``` + +### Highlights + +**PostgreSQL Logical Replication Protocol Support** {{}} + +We're excited to announce in the 2024.1.1.0 release support for the PostgreSQL Logical Replication Protocol for Change Data Capture (CDC), in addition to the existing native [gRPC Replication protocol](/stable/explore/change-data-capture/using-yugabytedb-grpc-replication/). +This feature allows you to manage CDC streams using [Publications](https://www.postgresql.org/docs/11/sql-createpublication.html) and [Replication Slots](https://www.postgresql.org/docs/current/logicaldecoding-explanation.html#LOGICALDECODING-REPLICATION-SLOTS), similar to native PostgreSQL. Additionally, a [new connector](/stable/explore/change-data-capture/using-logical-replication/get-started/#get-started-with-yugabytedb-connector) is introduced that utilizes the logical replication protocol to consume the CDC streams via [Replication slots](https://www.postgresql.org/docs/current/logicaldecoding-explanation.html#LOGICALDECODING-REPLICATION-SLOTS). + +For more information, refer to [logical replication protocol](/stable/explore/change-data-capture/using-logical-replication/). + +### New features + +- [Automated SQL/CQL Shell binary](/stable/admin/ysqlsh/#installation). Along with full binary, added separate downloadable SQL/CQL Shell binary. + +- [Voyager assessment visualisation in yugabyted UI](/preview/yugabyte-voyager/migrate/assess-migration/#visualize-the-migration-assessment-report). Yugabyted UI provides a dashboard to allow the users to effectively plan the migrations based on the complexity and also be able to monitor the progress of each migration {{}} + +### Change log + +
+ View the detailed changelog + +### Improvements + +#### YSQL + +* Eradicates misleading log message during table creation with DDL atomicity enabled. {{}} +* Removes the unused `keep_order` field from `YbctidGenerator` for cleaner results processing. {{}} +* Updates read time for each operation to simplify code and avoid applying used read time from obsolete operations. {{}} +* Enhances logging for DDL atomicity and corrects a loop index in cross-db unit test for clearer test logs. {{}} + +#### DocDB + +* Offers a new stack trace tracking framework for improved visibility of disk I/O operations, configurable through the `track_stack_traces` gflag. {{}} +* Introduces utility to dump top heap stacks when memory limit is exceeded for enhanced debugging. {{}} +* Adds `SCHECK_PB_FIELDS_NOT_EMPTY` macro for validating non-empty fields, with individual checks on list elements. {{}} +* Disables the unused and unreliable NamespaceReplication tests temporarily for seamless operation. {{}} +* Eliminates potential deadlock issue in DeleteOrHideTabletsAndSendRequests by removing unnecessary shared lock, and introduces a test for bug 22608. {{}} +* Refreshes stack trace tracking UI endpoints to enable per-column sorting and optimize sorting script. {{}} +* Introduces two new `SOURCE_UNREACHABLE` and `SYSTEM_ERROR` enums to enable more detailed error reporting from the Poller. {{}} +* Refreshes stack trace tracking UI endpoints to enable per-column sorting and optimize sorting script. {{}} +* Introduces two new `SOURCE_UNREACHABLE` and `SYSTEM_ERROR` enums to enable more detailed error reporting from the Poller. {{}} +* Shifts tcmalloc profiling code to the Utils folder. {{}} +* Enhances visibility of the `Hidden` state in Master/Tserver Tables UI by shifting its position more prominently to the `State` column. {{}} + +#### CDC + +* Prevents addition of tables with enum array column to the CDC stream to avoid crashes during consumption. {{}} +* Transforms the flag `yb_enable_cdc_consistent_snapshot_streams` from a preview into a default true auto flag. {{}} +* Introduces three new yb-admin commands to remove a user table from a CDCSDK stream, descend dynamic table addition in a CDC stream, and validate CDC state for a particular stream, enhancing control over CDC streams. {{}},{{}} +* Adds a tserver flag, `ysql_yb_default_replica_identity`, for customizing default replica identity at table creation. {{}} +* Introduces `cdcsdk_enable_dynamic_table_addition` flag to manage dynamic table additions in replication slot consumption model. {{}} +* Introduces replication slot name for internal distinction between two consumption models in the code. {{}} +* Allows creating an old model stream via yb-admin in upgraded environments, ensuring only one stream type per database. {{}} +* Eliminates unnecessary NOTICE messages when setting yb_read_time from walsender, reducing message clutter. {{}} + +#### yugabyted + +* Allows to specify multiple data directories using the new `additional_data_dir` configuration. {{}} + +### Bug fixes + +#### YSQL + +* Fixes an error that occurs when decoding null values from a boolean column sorted as NULLS LAST in a secondary index. {{}} +* Fixes YSQL upgrade single connection mode error preventing new connection attempts before the old ones are released. {{}} +* Fixes memory leaks in pg_constraint/pg_attrdef local cache by adding a missing `hash_destroy` call in `YbCleanupTupleCache`. {{}} +* Fixes a memory leak in catalog cache refresh during upgrade that led to higher memory consumption. {{}} +* Resolves remaining memory leaks in CacheMemoryContext to stabilize cache memory after every catalog cache refresh. {{}} +* Stops Batched Nest Loop (BNL) crashes by ensuring better indexing condition checks. {{}} +* Refines the logic to accurately push down join clauses to batched index scans without causing conflicts. {{}} +* Reenables rechecking for RowCompareExpressions to accurately handle NULL inputs in scan bound calculations. {{}} +* Grants BNL hashtable its own expression context to prevent data overwrites during query execution. {{}} +* Allows for an enhanced readability and performance of yb_cost_index code, aiding in merging with the pg15 branch. {{}} +* Fixes a bug in the index tuple width calculation for better YB base scans cost model. {{}} +* Makes `yb_get_range_split_clause` robust using PG TRY CATCH block, ensuring YB backup doesn't fail. {{}} +* Prevents coredumps by ensuring YSQL webserver destruction upon receiving a termination signal. {{}} +* Reduces test iterations of `PgCatalogVersionTest.SqlCrossDBLoadWithDDL` in tsan/asan build to prevent timeouts. {{}} +* Fixes memory leaks in ybcFetchNextHeapTuple by properly freeing the YBCStatus. {{}} +* Replaces `TRUNCATE` with `DELETE FROM` in TestPgTransparentRestarts, enhancing its stability and reducing flakiness. {{}} +* Allows `CreateNamespaceIfNotExists` function to retry on "already exists" error, preventing race conditions. {{}} +* Removes the unused function `Catalocustomeranager::WaitForDdlVerificationToFinish` for clarity. {{}} +* Resolves potential database OID collision with `system_postgres` by excluding reserved OID 65535 in allocation. {{}} +* Prevents the `IN` expressions on single column from wrongly taking the tuple path, ensuring correct data processing. {{}} +* Ensures bitmap scans correctly recheck all results and avoid excluding rows, improving accuracy of outcomes. {{}} +* Fixes incorrect access to the scan plan's bind descriptor during tuple IN condition rechecks. {{}} +* Allows the creation of new shared relations during YSQL upgrade to have a global impact by incrementing the catalog version across every database. {{}} +* Corrects the YbGetOrdinaryColumnsNeedingPgRecheck function to return table column numbers instead of index numbers, preventing unnecessary data fetches and potential crashes or errors after dropping a column. {{}} +* Fixes the issue when a separately created and later attached partition does not properly inherit the parent's primary key using `ALTER TABLE ...ATTACH PARTITION`. {{}} +* Now allows for correct backward prefix-based scanning by eliminating the problematic `kGroupEnd` marker that was leading to inaccurate seek results. {{}} +* Reduces unexpected log messages by not invoking `YsqlDdlTxnCompleteCallback` if all table 'pb_txn_id's in the DDL transaction verifier state are already cleared, avoiding potential deadlock situations in DDL atomicity. {{}} +* Reduces sequence cache collision by incorporating both database and sequence OIDs as the entry key. {{}} +* Resolves a detected deadlock during ALTER TABLE operations, enhancing test stability. {{}} +* Prevents a crash related to memory release associated with TupleTableSlots in SubPlans during a Values Scan. {{}} +* Eliminates the occurrence of "schema version mismatch" error following a DROP INDEX statement by introducing a delay in index deletion. {{}} + +#### DocDB + +* Eliminates possible deadlock during setup replication by fixing the order in which locks are acquired. {{}} +* Removes unnecessary flush during snapshot deletion, preventing write blocks. {{}} +* Resolves a heartbeat metrics issue ensuring full xCluster error information is sent to the new master even during a leader failover, and makes `tserver_heartbeat_metrics_interval_ms` runtime updatable. {{}} +* Converted the `ysql_skip_row_lock_for_update` to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{}} +* Prevents duplicate entries in the sequence data filter used by Snapshot schedules, ensuring successful operations. {{}} +* Resolves the issue of `pg_locks` query failure due to missing host node UUID in distributed transactions. {{}} +* Clarifies memory division flags to reflect they are percentage of the process's hard memory limit, not total available memory. {{}} +* Eliminates latency spikes in conflicting workloads by preventing redundant ProbeTransactionDeadlock rpcs. {{}} +* Prevents premature metric destruction during Prometheus scrapes, resolving non-UTF8 character issues. {{}} +* Ensures failed xCluster setup if the xcluster stream state update to `ACTIVE` does not occur. {{}} + +#### CDC + +* Ensures deletion of MemoryContext after each GetChanges RPC to prevent memory leaks. {{}} +* Prevents newly created indexes, materialized views, and non-user tables from being added to the Chang Data Capture (CDC) stream metadata. {{}} +* Reduces resource usage by removing non-eligible tables, like indexes, from existing CDC SDK stream metadata, and releasing retention barriers. This change requires the master flag `enable_cleanup_of_non_eligible_tables_from_cdcsdk_stream` and limits processing to two non-eligible tables per namespace per run with `cdcsdk_table_processing_limit_per_run`. Introduces three yb-admin commands for managing CDC streams. {{}},{{}},{{}} +* Fixes the issue of getting either `0` or a random time as the server's system clock in XLogData from the logical replication stream. {{}} +* Fixes the segmentation fault in walsender for dynamic table addition by refreshing stored replica identities and preventing a race condition when creating dynamic tables. {{}} +* Solves an issue where CDCSDK incorrectly deduces tablets as not interesting for stream before reaching the configured time limit. {{}} +* Refines the logic to remove `BEGIN` record when no DML records are added, preventing potential virtual WAL crashes. {{}} +* Addresses a race condition in dynamic table creation, enhancing stability during table and tablet initialization. {{}} +* Increases timeout for deletion of slot entry from the state table to correct test failures in TSAN builds. {{}} +* Resolves "could not open relation" error by updating slot creation method and simplifying yb_read_time logic. {{}} +* Adds support for data types with dynamic oids in CDC to prevent crashes and allows tables with such columns to join the stream. {{}} + +#### yugabyted + +* Eliminates inconsistent failures in TestSessionParameters.java by resetting roles to default instead of dropping them. {{}} +* Prevents SyntaxWarning and exceptions when incorrect advertise_address is given by adjusting string literals and adding check for errors. {{}},{{}},{{}} + +
+ ## v2024.1.0.0 - June 4, 2024 {#v2024.1.0.0} **Build:** `2024.1.0.0-b129` diff --git a/docs/content/preview/yugabyte-voyager/reference/assess-migration.md b/docs/content/preview/yugabyte-voyager/reference/assess-migration.md index 173a36f47ac4..7d9c5766022f 100644 --- a/docs/content/preview/yugabyte-voyager/reference/assess-migration.md +++ b/docs/content/preview/yugabyte-voyager/reference/assess-migration.md @@ -32,7 +32,7 @@ The valid *arguments* for assess migration are described in the following table: | --iops-capture-interval | Interval (in seconds) at which Voyager will gather IOPS metadata from the source database for the given schema(s).
Default: 120 | | --oracle-db-sid | Oracle System Identifier you can use while exporting data from Oracle instances. Oracle migrations only. | | --oracle-home | Path to set `$ORACLE_HOME` environment variable. `tnsnames.ora` is found in `$ORACLE_HOME/network/admin`. Oracle migrations only. | -| [--oracle-tns-alias](../../yb-voyager-cli/#oracle-options) | TNS (Transparent Network Substrate) alias configured to establish a secure connection with the server. Oracle migrations only. | +| [--oracle-tns-alias](../yb-voyager-cli/#oracle-options) | TNS (Transparent Network Substrate) alias configured to establish a secure connection with the server. Oracle migrations only. | | --send-diagnostics | Enable or disable sending [diagnostics](../../diagnostics-report/) information to Yugabyte.
Default: true
Accepted parameters: true, false, yes, no, 0, 1 | | --source-db-host | Domain name or IP address of the machine on which the source database server is running.
Default: localhost | | --source-db-name | Source database name. | diff --git a/docs/content/stable/architecture/docdb-replication/async-replication.md b/docs/content/stable/architecture/docdb-replication/async-replication.md index 5a27fc3661fa..ee49096eedca 100644 --- a/docs/content/stable/architecture/docdb-replication/async-replication.md +++ b/docs/content/stable/architecture/docdb-replication/async-replication.md @@ -159,12 +159,11 @@ Today, this is done by backing up the source universe and restoring it to the ta Ongoing work, [#17862](https://github.com/yugabyte/yugabyte-db/issues/17862), will replace using backup and restore here with directly copying RocksDB files between the source and target universes. This will be more performant and flexible and remove the need for external storage like S3 to set up replication. - ## Supported deployment scenarios xCluster currently supports active-active single-master and active-active multi-master deployments. -### Active- active single-master +### Active-active single-master Here the replication is unidirectional from a source universe to a target universe. The target universe is typically located in data centers or regions that are different from the source universe. The source universe can serve both reads and writes. The target universe can only serve reads. Since only the nodes in one universe can take writes this mode is referred to as single master. Note that within the source universe all nodes can serve writes. diff --git a/docs/content/v2.20/architecture/docdb-replication/async-replication.md b/docs/content/v2.20/architecture/docdb-replication/async-replication.md index 66a6f5cb10ff..d5f17e08e8de 100644 --- a/docs/content/v2.20/architecture/docdb-replication/async-replication.md +++ b/docs/content/v2.20/architecture/docdb-replication/async-replication.md @@ -159,12 +159,11 @@ Today, this is done by backing up the source universe and restoring it to the ta Ongoing work, [#17862](https://github.com/yugabyte/yugabyte-db/issues/17862), will replace using backup and restore here with directly copying RocksDB files between the source and target universes. This will be more performant and flexible and remove the need for external storage like S3 to set up replication. - ## Supported deployment scenarios xCluster currently supports active-active single-master and active-active multi-master deployments. -### Active- active single-master +### Active-active single-master Here the replication is unidirectional from a source universe to a target universe. The target universe is typically located in data centers or regions that are different from the source universe. The source universe can serve both reads and writes. The target universe can only serve reads. Since only the nodes in one universe can take writes this mode is referred to as single master. Note that within the source universe all nodes can serve writes. diff --git a/docs/data/currentVersions.json b/docs/data/currentVersions.json index b9ac51386077..1ce359688696 100644 --- a/docs/data/currentVersions.json +++ b/docs/data/currentVersions.json @@ -15,9 +15,9 @@ "series": "v2024.1", "alias": "stable", "display": "v2024.1 (STS)", - "version": "2024.1.0.0", - "versionShort": "2024.1.0", - "appVersion": "2024.1.0.0-b129", + "version": "2024.1.1.0", + "versionShort": "2024.1.1", + "appVersion": "2024.1.1.0-b137", "isStable": true, "isSTS": true, "initialRelease": "2024-06-04", diff --git a/docs/netlify.toml b/docs/netlify.toml index 34c83c4c29c4..95cdb2d53f91 100644 --- a/docs/netlify.toml +++ b/docs/netlify.toml @@ -1354,8 +1354,6 @@ https://www.yugabyte.com https://api.segment.io/ https://cdn.segment.com/ - https://c.6sc.co/ - https://ipv6.6sc.co/ https://ingesteer.services-prod.nsvcs.net/rum_collection ; @@ -1381,7 +1379,6 @@ https://x.clearbitjs.com https://js.driftt.com/ https://cdn.segment.com/ - https://j.6sc.co/ ; style-src 'self' 'unsafe-inline' @@ -1416,7 +1413,6 @@ https://www.yugabyte.com https://www.gstatic.com https://www.google.com/s2/favicons - https://b.6sc.co/ ; child-src 'self' 'unsafe-inline' diff --git a/managed/RUNTIME-FLAGS.md b/managed/RUNTIME-FLAGS.md index 4c9225b91ef6..736672e9db1e 100644 --- a/managed/RUNTIME-FLAGS.md +++ b/managed/RUNTIME-FLAGS.md @@ -8,6 +8,7 @@ | "Max Number of Customer Tasks to fetch" | "yb.customer_task_db_query_limit" | "CUSTOMER" | "Knob that can be used when there are too many customer tasks overwhelming the server" | "Integer" | | "Show costs in UI" | "yb.ui.show_cost" | "CUSTOMER" | "Option to enable/disable costs in UI" | "Boolean" | | "Helm chart http download timeout" | "yb.releases.download_helm_chart_http_timeout" | "CUSTOMER" | "The timeout for downloading the Helm chart while importing a release using HTTP" | "Duration" | +| "Enable downloading metrics as a PDF" | "yb.ui.metrics.enable_download_pdf" | "CUSTOMER" | "When enabled, the download metrics option is shown on the universe metrics page." | "Boolean" | | "Use Redesigned Provider UI" | "yb.ui.feature_flags.provider_redesign" | "CUSTOMER" | "The redesigned provider UI adds a provider list view, a provider details view and improves the provider creation form for AWS, AZU, GCP, and K8s" | "Boolean" | | "Enable partial editing of in use providers" | "yb.ui.feature_flags.edit_in_use_provider" | "CUSTOMER" | "A subset of fields from in use providers can be edited. Users can edit in use providers directly through the YBA API. This config is used to enable this functionality through YBA UI as well." | "Boolean" | | "Show underlying xCluster configs from DR setup" | "yb.ui.xcluster.dr.show_xcluster_config" | "CUSTOMER" | "YBA creates an underlying transactional xCluster config when setting up an active-active single-master disaster recovery (DR) config. During regular operation you should manage the DR config through the DR UI instead of the xCluster UI. This feature flag serves as a way to expose the underlying xCluster config for troubleshooting." | "Boolean" | diff --git a/managed/devops/opscli/ybops/cloud/azure/utils.py b/managed/devops/opscli/ybops/cloud/azure/utils.py index 91a5add216f1..57a9822658c0 100644 --- a/managed/devops/opscli/ybops/cloud/azure/utils.py +++ b/managed/devops/opscli/ybops/cloud/azure/utils.py @@ -850,32 +850,58 @@ def create_or_update_vm(self, vm_name, zone, num_vols, private_key_file, volume_ local_compute_client = self.compute_client else: local_compute_client = ComputeManagementClient( - self.credentials, fields['subscription_id'], self.get_retry_policy()) + self.credentials, fields['subscription_id']) - image_identifier = local_compute_client.gallery_images.get( + image_name = fields["image_definition_name"] + "/versions/" + fields["version_id"] + gallery_image = local_compute_client.gallery_images.get( fields['resource_group'], fields['gallery_name'], - fields['image_definition_name']).as_dict().get('identifier') + image_name) + image_tags = gallery_image.tags + logging.info("Gallery Image tags = " + str(image_tags)) # When creating VMs with images that are NOT from the marketplace, # the creator of the VM needs to provide the plan information. - # We try to extract this info from the publisher, offer, sku fields - # of the image definition. - logging.info("Image parameters: {}, publisher = {}, offer={}, sku={}".format( - image_identifier, - image_identifier['publisher'], - image_identifier['offer'], - image_identifier['sku'])) - - if (image_identifier is not None - and image_identifier['publisher'] is not None - and image_identifier['offer'] is not None - and image_identifier['sku'] is not None): + # For images created via packer, this info is added to the tags. + # Otherwise, we try to extract this info from the purchase plan + # or identifier of the image definition. + if (image_tags is not None + and image_tags['PlanPublisher'] is not None + and image_tags['PlanProduct'] is not None + and image_tags['PlanInfo'] is not None): plan = { - "publisher": image_identifier['publisher'], - "product": image_identifier['offer'], - "name": image_identifier['sku'], + "publisher": image_tags['PlanPublisher'], + "product": image_tags['PlanProduct'], + "name": image_tags['PlanInfo'], } + # Try to use purchase plan if info is absent from tags. + if plan is None: + image_purchase_plan = gallery_image.purchase_plan + logging.info("Gallery Image purchase plan = " + str(image_purchase_plan)) + if (image_purchase_plan is not None + and image_purchase_plan.publisher is not None + and image_purchase_plan.product is not None + and image_purchase_plan.name is not None): + plan = { + "publisher": image_purchase_plan.publisher, + "product": image_purchase_plan.product, + "name": image_purchase_plan.name, + } + # Try to fetch info from identifier. + if plan is None: + image_identifier = gallery_image.as_dict().get('identifier') + logging.info("Gallery Image identifier = " + str(image_identifier)) + if (image_identifier is not None + and image_identifier["publisher"] is not None + and image_identifier["offer"] is not None + and image_identifier["sku"] is not None): + plan = { + "publisher": image_identifier["publisher"], + "product": image_identifier["offer"], + "name": image_identifier["sku"], + } + if plan is None: + logging.warn("Plan info absent from the following VM image: " + str(image)) else: # machine image URN - "OpenLogic:CentOS:7_8:7.8.2020051900" pub, offer, sku, version = image.split(':') diff --git a/managed/devops/opscli/ybops/cloud/common/method.py b/managed/devops/opscli/ybops/cloud/common/method.py index ba902421bf26..1c5b31a34ddc 100644 --- a/managed/devops/opscli/ybops/cloud/common/method.py +++ b/managed/devops/opscli/ybops/cloud/common/method.py @@ -152,8 +152,8 @@ def add_extra_args(self): else: self.parser.add_argument("search_pattern", nargs="?") self.parser.add_argument("-t", "--type", default=self.YB_SERVER_TYPE) - self.parser.add_argument('--tags', action='append', default=None) - self.parser.add_argument("--skip_tags", action='append', default=None) + self.parser.add_argument('--tags', action='append', default=[]) + self.parser.add_argument("--skip_tags", action='append', default=[]) # If we do not have this entry from ansible.env, then set a None default, else, assume the # pem file is in the same location as the ansible.env file. @@ -793,7 +793,7 @@ def callback(self, args): use_default_ssh_port = not ssh_port_updated # Check if secondary subnet is present. If so, configure it. - if host_info.get('secondary_subnet'): + if "systemd_upgrade" not in args.tags and host_info.get('secondary_subnet'): # Wait for host to be ready to run ssh commands. self.wait_for_host(args, use_default_ssh_port) server_ports = self.get_server_ports_to_check(args) diff --git a/managed/devops/roles/configure-cluster-server/tasks/prepare-configure-server.yml b/managed/devops/roles/configure-cluster-server/tasks/prepare-configure-server.yml index 8d102a746e2e..766c02c7aeb7 100644 --- a/managed/devops/roles/configure-cluster-server/tasks/prepare-configure-server.yml +++ b/managed/devops/roles/configure-cluster-server/tasks/prepare-configure-server.yml @@ -40,6 +40,7 @@ mode: 0755 tags: - install-software + - systemd_upgrade - name: Setup | Create directory to land core dumps file: @@ -67,7 +68,9 @@ dest: "{{ yb_bin_dir }}/clean_cores.sh" owner: "{{ user_name }}" mode: 0755 - tags: yb-prebuilt-ami + tags: + - yb-prebuilt-ami + - systemd_upgrade - name: Configure | Add purge logs script template: @@ -75,7 +78,9 @@ dest: "{{ yb_bin_dir }}/zip_purge_yb_logs.sh" owner: "{{ user_name }}" mode: 0755 - tags: yb-prebuilt-ami + tags: + - yb-prebuilt-ami + - systemd_upgrade - name: Configure | Add collect metrics wrapper script template: @@ -83,7 +88,9 @@ dest: "{{ yb_bin_dir }}/collect_metrics_wrapper.sh" owner: "{{ user_name }}" mode: 0755 - tags: yb-prebuilt-ami + tags: + - yb-prebuilt-ami + - systemd_upgrade - set_fact: cron_result: {} @@ -284,7 +291,7 @@ state: present mode: 0644 - tags: + tags: - systemd_upgrade - install-software when: (systemd_option and not ((ansible_os_family == 'RedHat' and ansible_distribution_major_version == '7') diff --git a/managed/devspace.yaml b/managed/devspace.yaml index ee276dba2c76..7bd2871c8ec8 100755 --- a/managed/devspace.yaml +++ b/managed/devspace.yaml @@ -45,7 +45,9 @@ deployments: tag: ${YB_IMAGE_TAG} yugaware: service: - type: ClusterIP + annotations: + 'networking.gke.io/load-balancer-type': Internal + type: LoadBalancer storage: 100Gi storageClass: yb-standard # valuesFiles: diff --git a/managed/src/main/java/com/yugabyte/yw/common/LocalNodeManager.java b/managed/src/main/java/com/yugabyte/yw/common/LocalNodeManager.java index 6752bd24fbfe..b7cc2dd0635d 100644 --- a/managed/src/main/java/com/yugabyte/yw/common/LocalNodeManager.java +++ b/managed/src/main/java/com/yugabyte/yw/common/LocalNodeManager.java @@ -793,8 +793,9 @@ private void writeGFlagsToFile( if (additionalGFlags != null && serverType != UniverseTaskBase.ServerType.CONTROLLER) { gflagsToWrite.putAll(additionalGFlags.getPerProcessFlags().value.get(serverType)); } - log.debug("Write gflags {} to file {}", gflagsToWrite, serverType); - File flagFileTmpPath = new File(getNodeGFlagsFile(userIntent, serverType, nodeInfo)); + String fileName = getNodeGFlagsFile(userIntent, serverType, nodeInfo); + log.debug("Write gflags {} for {} to file {}", gflagsToWrite, serverType, fileName); + File flagFileTmpPath = new File(fileName); if (!flagFileTmpPath.exists()) { flagFileTmpPath.getParentFile().mkdirs(); flagFileTmpPath.createNewFile(); @@ -898,7 +899,11 @@ private ShellResponse successResponse(JsonNode jsonNode) { public String getNodeRoot(UniverseDefinitionTaskParams.UserIntent userIntent, NodeInfo nodeInfo) { String binDir = getCloudInfo(userIntent).getDataHomeDir(); - return binDir + "/" + nodeInfo.ip + "-" + nodeInfo.name.substring(nodeInfo.name.length() - 2); + String suffix = nodeInfo.name.substring(nodeInfo.name.length() - 2); + if (nodeInfo.name.contains("readonly")) { + suffix = "rr-" + suffix; + } + return binDir + "/" + nodeInfo.ip + "-" + suffix; } public String getNodeRoot(UniverseDefinitionTaskParams.UserIntent userIntent, String nodeName) { diff --git a/managed/src/main/java/com/yugabyte/yw/common/config/CustomerConfKeys.java b/managed/src/main/java/com/yugabyte/yw/common/config/CustomerConfKeys.java index dd625a176f7f..604d95c1b3f4 100644 --- a/managed/src/main/java/com/yugabyte/yw/common/config/CustomerConfKeys.java +++ b/managed/src/main/java/com/yugabyte/yw/common/config/CustomerConfKeys.java @@ -109,6 +109,15 @@ public class CustomerConfKeys extends RuntimeConfigKeysModule { ConfDataType.DurationType, ImmutableList.of(ConfKeyTags.PUBLIC)); + public static final ConfKeyInfo enableDownloadMetricsPdf = + new ConfKeyInfo<>( + "yb.ui.metrics.enable_download_pdf", + ScopeType.CUSTOMER, + "Enable downloading metrics as a PDF", + "When enabled, the download metrics option is shown on the universe metrics page.", + ConfDataType.BooleanType, + ImmutableList.of(ConfKeyTags.PUBLIC)); + public static final ConfKeyInfo useNewProviderUI = new ConfKeyInfo<>( "yb.ui.feature_flags.provider_redesign", diff --git a/managed/src/main/java/com/yugabyte/yw/common/gflags/SpecificGFlags.java b/managed/src/main/java/com/yugabyte/yw/common/gflags/SpecificGFlags.java index 85e82a31be2e..99b1653d9ed8 100644 --- a/managed/src/main/java/com/yugabyte/yw/common/gflags/SpecificGFlags.java +++ b/managed/src/main/java/com/yugabyte/yw/common/gflags/SpecificGFlags.java @@ -89,6 +89,7 @@ public SpecificGFlags clone() { newValue.perAZ.put(k, clone(v)); }); } + newValue.gflagGroups = new ArrayList<>(gflagGroups == null ? new ArrayList<>() : gflagGroups); return newValue; } diff --git a/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesController.java b/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesController.java index db426227f117..e240a8c564cc 100644 --- a/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesController.java +++ b/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesController.java @@ -48,8 +48,7 @@ @Api( value = "New Release management", - authorizations = @Authorization(AbstractPlatformController.API_KEY_AUTH), - hidden = true) + authorizations = @Authorization(AbstractPlatformController.API_KEY_AUTH)) @Slf4j public class ReleasesController extends AuthenticatedController { @Inject ReleasesUtils releasesUtils; @@ -58,8 +57,7 @@ public class ReleasesController extends AuthenticatedController { value = "Create a release", response = YBPCreateSuccess.class, nickname = "createNewRelease", - notes = "YbaApi Internal new releases list", - hidden = true) // TODO: remove hidden once complete. + notes = "WARNING: This is a preview API that could change: create a ybdb release") @ApiImplicitParams({ @ApiImplicitParam( name = "Release", @@ -74,7 +72,7 @@ public class ReleasesController extends AuthenticatedController { @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) - @YbaApi(visibility = YbaApiVisibility.INTERNAL, sinceYBAVersion = "2.21.1.0") + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result create(UUID customerUUID, Http.Request request) { Customer.getOrBadRequest(customerUUID); CreateRelease reqRelease = @@ -132,15 +130,14 @@ public Result create(UUID customerUUID, Http.Request request) { response = ResponseRelease.class, responseContainer = "List", nickname = "listNewReleases", - notes = "YbaApi Internal new releases list", - hidden = true) // TODO: Remove hidden once complete + notes = "WARNING: This is a preview API that could change: list ybdb releases") @AuthzPath({ @RequiredPermissionOnResource( requiredPermission = - @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), + @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.READ), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) - @YbaApi(visibility = YbaApiVisibility.INTERNAL, sinceYBAVersion = "2.21.1.0") + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result list(UUID customerUUID, @Nullable String deploymentType, Http.Request request) { Customer.getOrBadRequest(customerUUID); @@ -181,14 +178,14 @@ public Result list(UUID customerUUID, @Nullable String deploymentType, Http.Requ value = "Get a release", response = ResponseRelease.class, nickname = "getNewRelease", - notes = "YbaApi Internal new release get", - hidden = true) + notes = "WARNING: This is a preview API that could change: get a specific ybdb release") @AuthzPath({ @RequiredPermissionOnResource( requiredPermission = - @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), + @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.READ), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result get(UUID customerUUID, UUID releaseUUID, Http.Request request) { Customer.getOrBadRequest(customerUUID); Release release = Release.getOrBadRequest(releaseUUID); @@ -201,14 +198,14 @@ public Result get(UUID customerUUID, UUID releaseUUID, Http.Request request) { value = "delete a release", response = YBPSuccess.class, nickname = "deleteNewRelease", - notes = "YbaApi Internal new release delete", - hidden = true) + notes = "WARNING: This is a preview API that could change: delete a ybdb release") @AuthzPath({ @RequiredPermissionOnResource( requiredPermission = - @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), + @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.DELETE), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result delete(UUID customerUUID, UUID releaseUUID, Http.Request request) { Customer.getOrBadRequest(customerUUID); Release release = Release.get(releaseUUID); @@ -235,8 +232,7 @@ public Result delete(UUID customerUUID, UUID releaseUUID, Http.Request request) value = "Update a release", response = YBPSuccess.class, nickname = "updateNewRelease", - notes = "YbaApi Internal new releases update", - hidden = true) // TODO: remove hidden once complete. + notes = "WARNING: This is a preview API that could change: update or edit a ybdb release") @ApiImplicitParams({ @ApiImplicitParam( name = "Release", @@ -248,10 +244,10 @@ public Result delete(UUID customerUUID, UUID releaseUUID, Http.Request request) @AuthzPath({ @RequiredPermissionOnResource( requiredPermission = - @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), + @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.UPDATE), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) - @YbaApi(visibility = YbaApiVisibility.INTERNAL, sinceYBAVersion = "2.21.1.0") + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result update(UUID customerUUID, UUID releaseUUID, Http.Request request) { Customer.getOrBadRequest(customerUUID); Release release = Release.getOrBadRequest(releaseUUID); diff --git a/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesExtractMetadataController.java b/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesExtractMetadataController.java index 0203cf09b310..4ed6bf02dc24 100644 --- a/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesExtractMetadataController.java +++ b/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesExtractMetadataController.java @@ -12,6 +12,8 @@ import com.yugabyte.yw.forms.PlatformResults.YBPCreateSuccess; import com.yugabyte.yw.forms.PlatformResults.YBPSuccess; import com.yugabyte.yw.models.Customer; +import com.yugabyte.yw.models.common.YbaApi; +import com.yugabyte.yw.models.common.YbaApi.YbaApiVisibility; import com.yugabyte.yw.rbac.annotations.AuthzPath; import com.yugabyte.yw.rbac.annotations.PermissionAttribute; import com.yugabyte.yw.rbac.annotations.RequiredPermissionOnResource; @@ -37,8 +39,7 @@ @Api( value = "Extract metadata from remote tarball", - authorizations = @Authorization(AbstractPlatformController.API_KEY_AUTH), - hidden = true) + authorizations = @Authorization(AbstractPlatformController.API_KEY_AUTH)) @Slf4j public class ReleasesExtractMetadataController extends AuthenticatedController { @@ -56,8 +57,9 @@ public class ReleasesExtractMetadataController extends AuthenticatedController { value = "helper to extract release metadata from a remote tarball", response = YBPSuccess.class, nickname = "extractMetadata", - notes = "YbaApi Internal extract metadata", - hidden = true) + notes = + "WARNING: This is a preview API that could change: start extracting metadata from a" + + " remote tgz url") @ApiImplicitParams({ @ApiImplicitParam( name = "Release URL", @@ -72,6 +74,7 @@ public class ReleasesExtractMetadataController extends AuthenticatedController { @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result extract_metadata(UUID customerUUID, Http.Request request) { Customer.getOrBadRequest(customerUUID); ExtractMetadata em = @@ -115,14 +118,16 @@ public void run() { value = "get the extract release metadata from a remote tarball", response = ResponseExtractMetadata.class, nickname = "extractMetadata", - notes = "YbaApi Internal extract metadata", - hidden = true) + notes = + "WARNING: This is a preview API that could change: Get extract metadata and its" + + " progress.") @AuthzPath({ @RequiredPermissionOnResource( requiredPermission = - @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), + @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.READ), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result getMetadata(UUID customerUUID, UUID metadataUUID, Http.Request request) { ResponseExtractMetadata metadata = metadataMap.get(metadataUUID); Customer.getOrBadRequest(customerUUID); diff --git a/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesUploadController.java b/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesUploadController.java index c313ddfe39bb..5adf57cd2088 100644 --- a/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesUploadController.java +++ b/managed/src/main/java/com/yugabyte/yw/controllers/ReleasesUploadController.java @@ -35,8 +35,7 @@ @Api( value = "Upload Release packages", - authorizations = @Authorization(AbstractPlatformController.API_KEY_AUTH), - hidden = true) + authorizations = @Authorization(AbstractPlatformController.API_KEY_AUTH)) @Slf4j public class ReleasesUploadController extends AuthenticatedController { @@ -52,15 +51,14 @@ public ReleasesUploadController(Config appConfig, ReleasesUtils releasesUtils) { @ApiOperation( value = "upload a release tgz", nickname = "uploadRelease", - notes = "YbaApi Internal upload release", - hidden = true) // TODO: remove hidden once complete. + notes = "WARNING: This is a preview API that could change: upload release tgz file") @AuthzPath({ @RequiredPermissionOnResource( requiredPermission = @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) - @YbaApi(visibility = YbaApiVisibility.INTERNAL, sinceYBAVersion = "2.21.1.0") + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result upload(UUID customerUUID, Http.Request request) { Customer.getOrBadRequest(customerUUID); @@ -98,16 +96,15 @@ public Result upload(UUID customerUUID, Http.Request request) { @ApiOperation( value = "get an uploaded release metadata", nickname = "getUploadRelease", - notes = "YbaApi Internal get uploaded release metadata", - response = ResponseExtractMetadata.class, - hidden = true) // TODO: remove hidden once complete. + notes = "WARNING: This is a preview API that could change: get uploaded release metadata", + response = ResponseExtractMetadata.class) @AuthzPath({ @RequiredPermissionOnResource( requiredPermission = - @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.CREATE), + @PermissionAttribute(resourceType = ResourceType.OTHER, action = Action.READ), resourceLocation = @Resource(path = Util.CUSTOMERS, sourceType = SourceType.ENDPOINT)) }) - @YbaApi(visibility = YbaApiVisibility.INTERNAL, sinceYBAVersion = "2.21.1.0") + @YbaApi(visibility = YbaApiVisibility.PREVIEW, sinceYBAVersion = "2024.2.0.0") public Result get(UUID customerUUID, UUID fileUUID, Http.Request request) { Customer.getOrBadRequest(customerUUID); ReleaseLocalFile rlf = ReleaseLocalFile.getOrBadRequest(fileUUID); diff --git a/managed/src/main/java/com/yugabyte/yw/controllers/apiModels/ResponseRelease.java b/managed/src/main/java/com/yugabyte/yw/controllers/apiModels/ResponseRelease.java index bbe9677d9217..b644515de075 100644 --- a/managed/src/main/java/com/yugabyte/yw/controllers/apiModels/ResponseRelease.java +++ b/managed/src/main/java/com/yugabyte/yw/controllers/apiModels/ResponseRelease.java @@ -1,8 +1,10 @@ package com.yugabyte.yw.controllers.apiModels; +import com.fasterxml.jackson.annotation.JsonFormat; import com.fasterxml.jackson.annotation.JsonInclude; import com.yugabyte.yw.cloud.PublicCloudConstants; import com.yugabyte.yw.models.ReleaseArtifact; +import io.swagger.annotations.ApiModelProperty; import java.util.Date; import java.util.List; import java.util.UUID; @@ -33,6 +35,9 @@ public static class Artifact { public static class Universe { public UUID uuid; public String name; + + @ApiModelProperty(value = "Universe creation date", example = "2024-07-28T01:02:03Z") + @JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss'Z'") public Date creation_date; public Universe(UUID uuid, String name, Date creation_date) { diff --git a/managed/src/main/java/com/yugabyte/yw/forms/ResizeNodeParams.java b/managed/src/main/java/com/yugabyte/yw/forms/ResizeNodeParams.java index 829ef4aee4ef..30d6b808e934 100644 --- a/managed/src/main/java/com/yugabyte/yw/forms/ResizeNodeParams.java +++ b/managed/src/main/java/com/yugabyte/yw/forms/ResizeNodeParams.java @@ -51,7 +51,8 @@ public class ResizeNodeParams extends UpgradeWithGFlags { Common.CloudType.gcp, Common.CloudType.aws, Common.CloudType.kubernetes, - Common.CloudType.azu); + Common.CloudType.azu, + Common.CloudType.local); private boolean forceResizeNode; diff --git a/managed/src/main/resources/health/node_health.py.template b/managed/src/main/resources/health/node_health.py.template index 1b09c0a791b1..d05ae37e92d7 100755 --- a/managed/src/main/resources/health/node_health.py.template +++ b/managed/src/main/resources/health/node_health.py.template @@ -1733,8 +1733,9 @@ class NodeChecker(): pid_list.append(root_pid) if process_name == TSERVER: postgre_pid = self.get_process_pid_by_name(POSTMASTER) - pid_list.remove(postgre_pid) - pid_list = [p for p in pid_list if p not in self._get_subprocess_pids(postgre_pid)] + if postgre_pid is not None: + pid_list.remove(postgre_pid) + pid_list = [p for p in pid_list if p not in self._get_subprocess_pids(postgre_pid)] total_stat = self._get_empty_proc_results() prev_process_map = {} if process_name in self.prev_process_results.keys(): diff --git a/managed/src/main/resources/reference.conf b/managed/src/main/resources/reference.conf index 5cc6449decda..5639f7a5a2ef 100644 --- a/managed/src/main/resources/reference.conf +++ b/managed/src/main/resources/reference.conf @@ -193,6 +193,10 @@ yb { show_xcluster_config = false } } + + metrics: { + enable_download_pdf=false + } } edit_provider { diff --git a/managed/src/main/resources/swagger-strict.json b/managed/src/main/resources/swagger-strict.json index ddfeea0c5fc2..88f7508fd6ba 100644 --- a/managed/src/main/resources/swagger-strict.json +++ b/managed/src/main/resources/swagger-strict.json @@ -1338,6 +1338,30 @@ }, "type" : "object" }, + "Artifact" : { + "properties" : { + "architecture" : { + "enum" : [ "x86_64", "aarch64" ], + "type" : "string" + }, + "package_file_id" : { + "format" : "uuid", + "type" : "string" + }, + "package_url" : { + "type" : "string" + }, + "platform" : { + "enum" : [ "LINUX", "KUBERNETES" ], + "type" : "string" + }, + "sha256" : { + "type" : "string" + } + }, + "required" : [ "architecture", "package_file_id", "package_url", "platform", "sha256" ], + "type" : "object" + }, "Audit" : { "description" : "Audit logging for requests and responses", "properties" : { @@ -4072,6 +4096,42 @@ "required" : [ "creatingUser", "platformUrl", "platformVersion", "sleepAfterMasterRestartMillis", "sleepAfterTServerRestartMillis" ], "type" : "object" }, + "CreateRelease" : { + "description" : "Release metadata required to create a new release", + "properties" : { + "artifacts" : { + "items" : { + "$ref" : "#/definitions/Artifact" + }, + "type" : "array" + }, + "release_date_msecs" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_tag" : { + "type" : "string" + }, + "release_type" : { + "type" : "string" + }, + "release_uuid" : { + "format" : "uuid", + "type" : "string" + }, + "version" : { + "type" : "string" + }, + "yb_type" : { + "type" : "string" + } + }, + "required" : [ "artifacts", "release_date_msecs", "release_notes", "release_tag", "release_type", "release_uuid", "version", "yb_type" ], + "type" : "object" + }, "CreateTablespaceParams" : { "properties" : { "tablespaceInfos" : { @@ -5232,6 +5292,20 @@ }, "type" : "object" }, + "ExtractMetadata" : { + "description" : "url to release TGZ to extract metadata from", + "properties" : { + "url" : { + "type" : "string" + }, + "uuid" : { + "format" : "uuid", + "type" : "string" + } + }, + "required" : [ "url", "uuid" ], + "type" : "object" + }, "FailedSubtasks" : { "description" : "Failed Subtasks", "properties" : { @@ -10125,6 +10199,92 @@ "required" : [ "resourceDefinitionSet" ], "type" : "object" }, + "ResponseExtractMetadata" : { + "properties" : { + "architecture" : { + "enum" : [ "x86_64", "aarch64" ], + "type" : "string" + }, + "metadata_uuid" : { + "format" : "uuid", + "type" : "string" + }, + "platform" : { + "enum" : [ "LINUX", "KUBERNETES" ], + "type" : "string" + }, + "release_date_msecs" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_type" : { + "type" : "string" + }, + "sha256" : { + "type" : "string" + }, + "status" : { + "enum" : [ "waiting", "running", "success", "failure" ], + "type" : "string" + }, + "version" : { + "type" : "string" + }, + "yb_type" : { + "enum" : [ "YBDB" ], + "type" : "string" + } + }, + "required" : [ "architecture", "metadata_uuid", "platform", "release_date_msecs", "release_notes", "release_type", "sha256", "status", "version", "yb_type" ], + "type" : "object" + }, + "ResponseRelease" : { + "properties" : { + "artifacts" : { + "items" : { + "$ref" : "#/definitions/Artifact" + }, + "type" : "array" + }, + "release_date_msecs" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_tag" : { + "type" : "string" + }, + "release_type" : { + "type" : "string" + }, + "release_uuid" : { + "format" : "uuid", + "type" : "string" + }, + "state" : { + "type" : "string" + }, + "universes" : { + "items" : { + "$ref" : "#/definitions/Universe" + }, + "type" : "array" + }, + "version" : { + "type" : "string" + }, + "yb_type" : { + "type" : "string" + } + }, + "required" : [ "artifacts", "release_date_msecs", "release_notes", "release_tag", "release_type", "release_uuid", "state", "universes", "version", "yb_type" ], + "type" : "object" + }, "RestartBootstrapParams" : { "description" : "Bootstrap parameters for restarting", "properties" : { @@ -13460,6 +13620,25 @@ "required" : [ "apiToken", "customerUUID", "metricsScrapePeriodSecs", "metricsUrl", "tpUrl", "uuid", "ybaUrl" ], "type" : "object" }, + "Universe" : { + "properties" : { + "creation_date" : { + "description" : "Universe creation date", + "example" : "2024-07-28T01:02:03Z", + "format" : "date-time", + "type" : "string" + }, + "name" : { + "type" : "string" + }, + "uuid" : { + "format" : "uuid", + "type" : "string" + } + }, + "required" : [ "name", "uuid" ], + "type" : "object" + }, "Universe performance advisor status" : { "properties" : { "customerUUID" : { @@ -14476,6 +14655,32 @@ "required" : [ "firstTry", "name", "retryLimit", "sleepMultiplier", "taskInfo" ], "type" : "object" }, + "UpdateRelease" : { + "description" : "Release metadata required to create a new release", + "properties" : { + "artifacts" : { + "items" : { + "$ref" : "#/definitions/Artifact" + }, + "type" : "array" + }, + "release_date" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_tag" : { + "type" : "string" + }, + "state" : { + "type" : "string" + } + }, + "required" : [ "artifacts", "release_date", "release_notes", "release_tag", "state" ], + "type" : "object" + }, "UpgradeTaskParams" : { "properties" : { "allowInsecure" : { @@ -15929,6 +16134,17 @@ }, "type" : "object" }, + "YBPCreateSuccess" : { + "properties" : { + "resourceUUID" : { + "description" : "UUID of the successfully created resource", + "format" : "uuid", + "readOnly" : true, + "type" : "string" + } + }, + "type" : "object" + }, "YBPError" : { "description" : "Generic error response from the YugabyteDB Anywhere API", "properties" : { @@ -27976,6 +28192,402 @@ "tags" : [ "Asynchronous Replication" ] } }, + "/api/v1/customers/{cUUID}/ybdb_release" : { + "get" : { + "description" : "WARNING: This is a preview API that could change: list ybdb releases", + "operationId" : "listNewReleases", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "default" : "null", + "in" : "query", + "name" : "deployment_type", + "required" : false, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "items" : { + "$ref" : "#/definitions/ResponseRelease" + }, + "type" : "array" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "items" : { + "$ref" : "#/definitions/ResponseRelease" + }, + "type" : "array" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "List releases", + "tags" : [ "New Release management" ] + }, + "post" : { + "description" : "WARNING: This is a preview API that could change: create a ybdb release", + "operationId" : "createNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + }, { + "description" : "Release data to be created", + "in" : "body", + "name" : "Release", + "required" : true, + "schema" : { + "$ref" : "#/definitions/CreateRelease" + } + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPCreateSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPCreateSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "Create a release", + "tags" : [ "New Release management" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/extract_metadata" : { + "post" : { + "description" : "WARNING: This is a preview API that could change: start extracting metadata from a remote tgz url", + "operationId" : "extractMetadata", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + }, { + "description" : "Release URL to extract metadata from", + "in" : "body", + "name" : "Release URL", + "required" : true, + "schema" : { + "$ref" : "#/definitions/ExtractMetadata" + } + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "helper to extract release metadata from a remote tarball", + "tags" : [ "Extract metadata from remote tarball" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/extract_metadata/{rUUID}" : { + "get" : { + "description" : "WARNING: This is a preview API that could change: Get extract metadata and its progress.", + "operationId" : "extractMetadata", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "get the extract release metadata from a remote tarball", + "tags" : [ "Extract metadata from remote tarball" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/upload" : { + "post" : { + "description" : "WARNING: This is a preview API that could change: upload release tgz file", + "operationId" : "uploadRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "default" : { + "description" : "successful operation" + } + }, + "responsesObject" : { + "default" : { + "description" : "successful operation" + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "upload a release tgz", + "tags" : [ "Upload Release packages" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/upload/{rUUID}" : { + "get" : { + "description" : "WARNING: This is a preview API that could change: get uploaded release metadata", + "operationId" : "getUploadRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "get an uploaded release metadata", + "tags" : [ "Upload Release packages" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/{rUUID}" : { + "delete" : { + "description" : "WARNING: This is a preview API that could change: delete a ybdb release", + "operationId" : "deleteNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "delete a release", + "tags" : [ "New Release management" ] + }, + "get" : { + "description" : "WARNING: This is a preview API that could change: get a specific ybdb release", + "operationId" : "getNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseRelease" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseRelease" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "Get a release", + "tags" : [ "New Release management" ] + }, + "put" : { + "description" : "WARNING: This is a preview API that could change: update or edit a ybdb release", + "operationId" : "updateNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + }, { + "description" : "Release data to be updated", + "in" : "body", + "name" : "Release", + "required" : true, + "schema" : { + "$ref" : "#/definitions/UpdateRelease" + } + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "Update a release", + "tags" : [ "New Release management" ] + } + }, "/api/v1/customers/{cUUID}/zones/{azUUID}/nodes" : { "post" : { "description" : "", @@ -28735,6 +29347,8 @@ "name" : "Disaster Recovery" }, { "name" : "Encryption at rest" + }, { + "name" : "Extract metadata from remote tarball" }, { "name" : "Grafana Dashboard" }, { @@ -28753,6 +29367,8 @@ "name" : "Maintenance windows" }, { "name" : "Metrics" + }, { + "name" : "New Release management" }, { "name" : "Node Agents" }, { @@ -28789,6 +29405,8 @@ "name" : "Universe performance suggestions" }, { "name" : "UniverseClusterMutations" + }, { + "name" : "Upload Release packages" }, { "name" : "User management" }, { diff --git a/managed/src/main/resources/swagger.json b/managed/src/main/resources/swagger.json index 7aa55067f307..0192753be567 100644 --- a/managed/src/main/resources/swagger.json +++ b/managed/src/main/resources/swagger.json @@ -1350,6 +1350,30 @@ }, "type" : "object" }, + "Artifact" : { + "properties" : { + "architecture" : { + "enum" : [ "x86_64", "aarch64" ], + "type" : "string" + }, + "package_file_id" : { + "format" : "uuid", + "type" : "string" + }, + "package_url" : { + "type" : "string" + }, + "platform" : { + "enum" : [ "LINUX", "KUBERNETES" ], + "type" : "string" + }, + "sha256" : { + "type" : "string" + } + }, + "required" : [ "architecture", "package_file_id", "package_url", "platform", "sha256" ], + "type" : "object" + }, "Audit" : { "description" : "Audit logging for requests and responses", "properties" : { @@ -4107,6 +4131,42 @@ "required" : [ "creatingUser", "platformUrl", "platformVersion", "sleepAfterMasterRestartMillis", "sleepAfterTServerRestartMillis" ], "type" : "object" }, + "CreateRelease" : { + "description" : "Release metadata required to create a new release", + "properties" : { + "artifacts" : { + "items" : { + "$ref" : "#/definitions/Artifact" + }, + "type" : "array" + }, + "release_date_msecs" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_tag" : { + "type" : "string" + }, + "release_type" : { + "type" : "string" + }, + "release_uuid" : { + "format" : "uuid", + "type" : "string" + }, + "version" : { + "type" : "string" + }, + "yb_type" : { + "type" : "string" + } + }, + "required" : [ "artifacts", "release_date_msecs", "release_notes", "release_tag", "release_type", "release_uuid", "version", "yb_type" ], + "type" : "object" + }, "CreateTablespaceParams" : { "properties" : { "tablespaceInfos" : { @@ -5267,6 +5327,20 @@ }, "type" : "object" }, + "ExtractMetadata" : { + "description" : "url to release TGZ to extract metadata from", + "properties" : { + "url" : { + "type" : "string" + }, + "uuid" : { + "format" : "uuid", + "type" : "string" + } + }, + "required" : [ "url", "uuid" ], + "type" : "object" + }, "FailedSubtasks" : { "description" : "Failed Subtasks", "properties" : { @@ -10229,6 +10303,92 @@ "required" : [ "resourceDefinitionSet" ], "type" : "object" }, + "ResponseExtractMetadata" : { + "properties" : { + "architecture" : { + "enum" : [ "x86_64", "aarch64" ], + "type" : "string" + }, + "metadata_uuid" : { + "format" : "uuid", + "type" : "string" + }, + "platform" : { + "enum" : [ "LINUX", "KUBERNETES" ], + "type" : "string" + }, + "release_date_msecs" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_type" : { + "type" : "string" + }, + "sha256" : { + "type" : "string" + }, + "status" : { + "enum" : [ "waiting", "running", "success", "failure" ], + "type" : "string" + }, + "version" : { + "type" : "string" + }, + "yb_type" : { + "enum" : [ "YBDB" ], + "type" : "string" + } + }, + "required" : [ "architecture", "metadata_uuid", "platform", "release_date_msecs", "release_notes", "release_type", "sha256", "status", "version", "yb_type" ], + "type" : "object" + }, + "ResponseRelease" : { + "properties" : { + "artifacts" : { + "items" : { + "$ref" : "#/definitions/Artifact" + }, + "type" : "array" + }, + "release_date_msecs" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_tag" : { + "type" : "string" + }, + "release_type" : { + "type" : "string" + }, + "release_uuid" : { + "format" : "uuid", + "type" : "string" + }, + "state" : { + "type" : "string" + }, + "universes" : { + "items" : { + "$ref" : "#/definitions/Universe" + }, + "type" : "array" + }, + "version" : { + "type" : "string" + }, + "yb_type" : { + "type" : "string" + } + }, + "required" : [ "artifacts", "release_date_msecs", "release_notes", "release_tag", "release_type", "release_uuid", "state", "universes", "version", "yb_type" ], + "type" : "object" + }, "RestartBootstrapParams" : { "description" : "Bootstrap parameters for restarting", "properties" : { @@ -13568,6 +13728,25 @@ "required" : [ "apiToken", "customerUUID", "metricsScrapePeriodSecs", "metricsUrl", "tpUrl", "uuid", "ybaUrl" ], "type" : "object" }, + "Universe" : { + "properties" : { + "creation_date" : { + "description" : "Universe creation date", + "example" : "2024-07-28T01:02:03Z", + "format" : "date-time", + "type" : "string" + }, + "name" : { + "type" : "string" + }, + "uuid" : { + "format" : "uuid", + "type" : "string" + } + }, + "required" : [ "name", "uuid" ], + "type" : "object" + }, "Universe performance advisor status" : { "properties" : { "customerUUID" : { @@ -14584,6 +14763,32 @@ "required" : [ "firstTry", "name", "retryLimit", "sleepMultiplier", "taskInfo" ], "type" : "object" }, + "UpdateRelease" : { + "description" : "Release metadata required to create a new release", + "properties" : { + "artifacts" : { + "items" : { + "$ref" : "#/definitions/Artifact" + }, + "type" : "array" + }, + "release_date" : { + "format" : "int64", + "type" : "integer" + }, + "release_notes" : { + "type" : "string" + }, + "release_tag" : { + "type" : "string" + }, + "state" : { + "type" : "string" + } + }, + "required" : [ "artifacts", "release_date", "release_notes", "release_tag", "state" ], + "type" : "object" + }, "UpgradeTaskParams" : { "properties" : { "allowInsecure" : { @@ -16078,6 +16283,17 @@ }, "type" : "object" }, + "YBPCreateSuccess" : { + "properties" : { + "resourceUUID" : { + "description" : "UUID of the successfully created resource", + "format" : "uuid", + "readOnly" : true, + "type" : "string" + } + }, + "type" : "object" + }, "YBPError" : { "description" : "Generic error response from the YugabyteDB Anywhere API", "properties" : { @@ -29484,6 +29700,402 @@ "tags" : [ "Asynchronous Replication" ] } }, + "/api/v1/customers/{cUUID}/ybdb_release" : { + "get" : { + "description" : "WARNING: This is a preview API that could change: list ybdb releases", + "operationId" : "listNewReleases", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "default" : "null", + "in" : "query", + "name" : "deployment_type", + "required" : false, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "items" : { + "$ref" : "#/definitions/ResponseRelease" + }, + "type" : "array" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "items" : { + "$ref" : "#/definitions/ResponseRelease" + }, + "type" : "array" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "List releases", + "tags" : [ "New Release management" ] + }, + "post" : { + "description" : "WARNING: This is a preview API that could change: create a ybdb release", + "operationId" : "createNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + }, { + "description" : "Release data to be created", + "in" : "body", + "name" : "Release", + "required" : true, + "schema" : { + "$ref" : "#/definitions/CreateRelease" + } + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPCreateSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPCreateSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "Create a release", + "tags" : [ "New Release management" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/extract_metadata" : { + "post" : { + "description" : "WARNING: This is a preview API that could change: start extracting metadata from a remote tgz url", + "operationId" : "extractMetadata", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + }, { + "description" : "Release URL to extract metadata from", + "in" : "body", + "name" : "Release URL", + "required" : true, + "schema" : { + "$ref" : "#/definitions/ExtractMetadata" + } + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "helper to extract release metadata from a remote tarball", + "tags" : [ "Extract metadata from remote tarball" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/extract_metadata/{rUUID}" : { + "get" : { + "description" : "WARNING: This is a preview API that could change: Get extract metadata and its progress.", + "operationId" : "extractMetadata", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "get the extract release metadata from a remote tarball", + "tags" : [ "Extract metadata from remote tarball" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/upload" : { + "post" : { + "description" : "WARNING: This is a preview API that could change: upload release tgz file", + "operationId" : "uploadRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "default" : { + "description" : "successful operation" + } + }, + "responsesObject" : { + "default" : { + "description" : "successful operation" + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "upload a release tgz", + "tags" : [ "Upload Release packages" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/upload/{rUUID}" : { + "get" : { + "description" : "WARNING: This is a preview API that could change: get uploaded release metadata", + "operationId" : "getUploadRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseExtractMetadata" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "get an uploaded release metadata", + "tags" : [ "Upload Release packages" ] + } + }, + "/api/v1/customers/{cUUID}/ybdb_release/{rUUID}" : { + "delete" : { + "description" : "WARNING: This is a preview API that could change: delete a ybdb release", + "operationId" : "deleteNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "delete a release", + "tags" : [ "New Release management" ] + }, + "get" : { + "description" : "WARNING: This is a preview API that could change: get a specific ybdb release", + "operationId" : "getNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseRelease" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/ResponseRelease" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "Get a release", + "tags" : [ "New Release management" ] + }, + "put" : { + "description" : "WARNING: This is a preview API that could change: update or edit a ybdb release", + "operationId" : "updateNewRelease", + "parameters" : [ { + "format" : "uuid", + "in" : "path", + "name" : "cUUID", + "required" : true, + "type" : "string" + }, { + "format" : "uuid", + "in" : "path", + "name" : "rUUID", + "required" : true, + "type" : "string" + }, { + "in" : "query", + "name" : "request", + "required" : false + }, { + "description" : "Release data to be updated", + "in" : "body", + "name" : "Release", + "required" : true, + "schema" : { + "$ref" : "#/definitions/UpdateRelease" + } + } ], + "responses" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "responsesObject" : { + "200" : { + "description" : "successful operation", + "schema" : { + "$ref" : "#/definitions/YBPSuccess" + } + } + }, + "security" : [ { + "apiKeyAuth" : [ ] + } ], + "summary" : "Update a release", + "tags" : [ "New Release management" ] + } + }, "/api/v1/customers/{cUUID}/zones/{azUUID}/nodes" : { "post" : { "description" : "", @@ -30243,6 +30855,8 @@ "name" : "Disaster Recovery" }, { "name" : "Encryption at rest" + }, { + "name" : "Extract metadata from remote tarball" }, { "name" : "Grafana Dashboard" }, { @@ -30261,6 +30875,8 @@ "name" : "Maintenance windows" }, { "name" : "Metrics" + }, { + "name" : "New Release management" }, { "name" : "Node Agents" }, { @@ -30299,6 +30915,8 @@ "name" : "Universe performance suggestions" }, { "name" : "UniverseClusterMutations" + }, { + "name" : "Upload Release packages" }, { "name" : "User management" }, { diff --git a/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/GFlagsUpgradeLocalTest.java b/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/GFlagsUpgradeLocalTest.java index 9fa88d6911fd..3986d18b047e 100644 --- a/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/GFlagsUpgradeLocalTest.java +++ b/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/GFlagsUpgradeLocalTest.java @@ -6,18 +6,22 @@ import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.containsString; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; import com.yugabyte.yw.commissioner.tasks.CommissionerBaseTest; import com.yugabyte.yw.commissioner.tasks.UniverseTaskBase; import com.yugabyte.yw.common.LocalNodeManager; import com.yugabyte.yw.common.PlacementInfoUtil; import com.yugabyte.yw.common.config.UniverseConfKeys; +import com.yugabyte.yw.common.gflags.GFlagGroup; import com.yugabyte.yw.common.gflags.GFlagsUtil; import com.yugabyte.yw.common.gflags.SpecificGFlags; import com.yugabyte.yw.common.utils.Pair; import com.yugabyte.yw.controllers.UniverseControllerRequestBinder; import com.yugabyte.yw.forms.GFlagsUpgradeParams; +import com.yugabyte.yw.forms.ResizeNodeParams; import com.yugabyte.yw.forms.UniverseDefinitionTaskParams; import com.yugabyte.yw.forms.UniverseResp; import com.yugabyte.yw.forms.UpgradeTaskParams; @@ -108,6 +112,55 @@ public void testRollingUpgradeWithRRInherited() throws InterruptedException { verifyPayload(); } + @Test + public void testResizeWithRR() throws InterruptedException { + UniverseDefinitionTaskParams.UserIntent userIntent = getDefaultUserIntent(); + userIntent.specificGFlags.setGflagGroups( + Collections.singletonList(GFlagGroup.GroupName.ENHANCED_POSTGRES_COMPATIBILITY)); + Universe universe = createUniverse(userIntent); + UniverseDefinitionTaskParams.UserIntent rrIntent = getDefaultUserIntent(); + rrIntent.numNodes = 1; + rrIntent.replicationFactor = 1; + rrIntent.specificGFlags = + SpecificGFlags.construct( + Collections.singletonMap("max_log_size", "1805"), + Collections.singletonMap("log_max_seconds_to_retain", "86333")); + addReadReplica(universe, rrIntent); + universe = Universe.getOrBadRequest(universe.getUniverseUUID()); + UniverseDefinitionTaskParams.Cluster rrCluster = + universe.getUniverseDetails().getReadOnlyClusters().get(0); + NodeDetails primary = + universe.getNodesByCluster(universe.getUniverseDetails().getPrimaryCluster().uuid).get(0); + NodeDetails rrNode = universe.getNodesByCluster(rrCluster.uuid).get(0); + + assertTrue( + getVarz(primary, universe, UniverseTaskBase.ServerType.TSERVER) + .containsKey("yb_enable_read_committed_isolation")); + assertTrue( + getVarz(rrNode, universe, UniverseTaskBase.ServerType.TSERVER) + .containsKey("yb_enable_read_committed_isolation")); + + ResizeNodeParams resizeParams = + getUpgradeParams( + universe, UpgradeTaskParams.UpgradeOption.ROLLING_UPGRADE, ResizeNodeParams.class); + resizeParams.clusters = Collections.singletonList(rrCluster); + rrCluster.userIntent.instanceType = instanceType2.getInstanceTypeCode(); + GFlagsUtil.removeGFlag( + rrCluster.userIntent, "log_max_seconds_to_retain", UniverseTaskBase.ServerType.TSERVER); + + TaskInfo taskInfo = + waitForTask( + upgradeUniverseHandler.resizeNode( + resizeParams, customer, Universe.getOrBadRequest(universe.getUniverseUUID())), + universe); + assertEquals(TaskInfo.State.Success, taskInfo.getTaskState()); + universe = Universe.getOrBadRequest(universe.getUniverseUUID()); + Map newValues = + getDiskFlags(rrNode, universe, UniverseTaskBase.ServerType.TSERVER); + assertTrue(newValues.containsKey("yb_enable_read_committed_isolation")); + assertFalse(newValues.containsKey("log_max_seconds_to_retain")); + } + @Test public void testRollingUpgradeWithRR() throws InterruptedException { Universe universe = createUniverse(getDefaultUserIntent()); diff --git a/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/LocalProviderUniverseTestBase.java b/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/LocalProviderUniverseTestBase.java index a4cae753d90d..35c74b0aa575 100644 --- a/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/LocalProviderUniverseTestBase.java +++ b/managed/src/test/java/com/yugabyte/yw/commissioner/tasks/local/LocalProviderUniverseTestBase.java @@ -134,9 +134,13 @@ public abstract class LocalProviderUniverseTestBase extends PlatformGuiceApplica private static final String DEFAULT_BASE_DIR = "/tmp/local"; protected static String YBC_VERSION; - public static String DB_VERSION = "2.20.5.0-b72"; + public static String DB_VERSION = "2024.1.0.0-b129"; private static final String DOWNLOAD_URL = - "https://downloads.yugabyte.com/releases/2.20.5.0/" + "yugabyte-2.20.5.0-b72-%s-%s.tar.gz"; + "https://downloads.yugabyte.com/releases/2024.1.0.0/" + + "yugabyte-" + + DB_VERSION + + "-%s-%s.tar.gz"; + private static final String YBC_BASE_S3_URL = "https://downloads.yugabyte.com/ybc/"; private static final String YBC_BIN_ENV_KEY = "YBC_PATH"; private static final boolean KEEP_FAILED_UNIVERSE = true; diff --git a/managed/ui/package-lock.json b/managed/ui/package-lock.json index 2d3b7b851f69..161e749b52fb 100644 --- a/managed/ui/package-lock.json +++ b/managed/ui/package-lock.json @@ -28,6 +28,7 @@ "ace-builds": "1.4.12", "axios": "0.21.3", "bootstrap": "3.4.1", + "canvg": "4.0.2", "clsx": "1.1.1", "copy-to-clipboard": "3.3.1", "cron-parser": "2.16.3", @@ -40,6 +41,7 @@ "intl": "1.2.5", "js-cookie": "2.2.1", "js-yaml": "4.1.0", + "jspdf": "2.5.1", "leaflet": "^1.0.2", "leaflet.markercluster": "1.4.1", "lodash": "4.17.21", @@ -9222,6 +9224,11 @@ "integrity": "sha512-FGa1F62FT09qcrueBA6qYTrJPVDzah9a+493+o2PCXsesWHIn27G98TsSMs3WPNbZIEj4+VJf6saSFpvD+3Zsw==", "dev": true }, + "node_modules/@types/raf": { + "version": "3.4.3", + "resolved": "https://registry.npmjs.org/@types/raf/-/raf-3.4.3.tgz", + "integrity": "sha512-c4YAvMedbPZ5tEyxzQdMoOhhJ4RD3rngZIdwC2/qDN3d7JpEhB6fiBRKVY1lg5B7Wk+uPBjn5f39j1/2MY1oOw==" + }, "node_modules/@types/range-parser": { "version": "1.2.4", "resolved": "https://registry.npmjs.org/@types/range-parser/-/range-parser-1.2.4.tgz", @@ -11226,6 +11233,15 @@ "node": ">=0.10.0" } }, + "node_modules/base64-arraybuffer": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-1.0.2.tgz", + "integrity": "sha512-I3yl4r9QB5ZRY3XuJVEPfc2XhZO6YweFPI+UovAzn+8/hb3oJ6lnysaFcjVpkCPfVWFUDvoZ8kmVDP7WyRtYtQ==", + "optional": true, + "engines": { + "node": ">= 0.6.0" + } + }, "node_modules/base64-js": { "version": "1.5.1", "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", @@ -11724,7 +11740,6 @@ "version": "1.2.1", "resolved": "https://registry.npmjs.org/btoa/-/btoa-1.2.1.tgz", "integrity": "sha512-SB4/MIGlsiVkMcHmT+pSmIPoNDoHg+7cMzmt3Uxt628MTz2487DKSqK/fuhFBrkuqrYv5UCEnACpF4dTFNKc/g==", - "dev": true, "bin": { "btoa": "bin/btoa.js" }, @@ -12353,6 +12368,21 @@ "element-size": "^1.1.1" } }, + "node_modules/canvg": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/canvg/-/canvg-4.0.2.tgz", + "integrity": "sha512-/7kIZger/mdFci4KXdtMr+NQB4GU1InkJ4RwSyDBRcvy4BUlg1hD+ZUWo550sWPyWaKZ8purqby6kjf09qVriw==", + "dependencies": { + "@types/raf": "^3.4.0", + "raf": "^3.4.1", + "rgbcolor": "^1.0.1", + "stackblur-canvas": "^2.0.0", + "svg-pathdata": "^6.0.3" + }, + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/capture-exit": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/capture-exit/-/capture-exit-2.0.0.tgz", @@ -13650,6 +13680,15 @@ "hyphenate-style-name": "^1.0.3" } }, + "node_modules/css-line-break": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/css-line-break/-/css-line-break-2.1.0.tgz", + "integrity": "sha512-FHcKFCZcAha3LwfVBhCQbW2nCNbkZXn7KVUJcsT5/P8YmfsVja0FMPJr0B903j/E69HUphKiV9iQArX8SDYA4w==", + "optional": true, + "dependencies": { + "utrie": "^1.0.2" + } + }, "node_modules/css-loader": { "version": "4.3.0", "resolved": "https://registry.npmjs.org/css-loader/-/css-loader-4.3.0.tgz", @@ -14963,6 +15002,12 @@ "url": "https://github.com/fb55/domhandler?sponsor=1" } }, + "node_modules/dompurify": { + "version": "2.5.6", + "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-2.5.6.tgz", + "integrity": "sha512-zUTaUBO8pY4+iJMPE1B9XlO2tXVYIcEA4SNGtvDELzTSCQO7RzH+j7S180BmhmJId78lqGU2z19vgVx2Sxs/PQ==", + "optional": true + }, "node_modules/domutils": { "version": "2.8.0", "resolved": "https://registry.npmjs.org/domutils/-/domutils-2.8.0.tgz", @@ -17088,6 +17133,11 @@ "integrity": "sha512-3yurQZ2hD9VISAhJJP9bpYFNQrHHBXE2JxxjY5aLEcDi46RmAzJE2OC9FAde0yis5ElW0jTTzs0zfg/Cca4XqQ==", "dev": true }, + "node_modules/fflate": { + "version": "0.4.8", + "resolved": "https://registry.npmjs.org/fflate/-/fflate-0.4.8.tgz", + "integrity": "sha512-FJqqoDBR00Mdj9ppamLa/Y7vxm+PRmNWA67N846RvsoYVMKB4q3y/de5PA7gUmRMYK/8CMz2GDZQmCRN1wBcWA==" + }, "node_modules/figgy-pudding": { "version": "3.5.2", "resolved": "https://registry.npmjs.org/figgy-pudding/-/figgy-pudding-3.5.2.tgz", @@ -19521,6 +19571,19 @@ "node": ">=6" } }, + "node_modules/html2canvas": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/html2canvas/-/html2canvas-1.4.1.tgz", + "integrity": "sha512-fPU6BHNpsyIhr8yyMpTLLxAbkaK8ArIBcmZIRiBLiDhjeqvXolaEmDGmELFuX9I4xDcaKKcJl+TKZLqruBbmWA==", + "optional": true, + "dependencies": { + "css-line-break": "^2.1.0", + "text-segmentation": "^1.0.3" + }, + "engines": { + "node": ">=8.0.0" + } + }, "node_modules/htmlparser2": { "version": "6.1.0", "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-6.1.0.tgz", @@ -24212,6 +24275,53 @@ "node": "*" } }, + "node_modules/jspdf": { + "version": "2.5.1", + "resolved": "https://registry.npmjs.org/jspdf/-/jspdf-2.5.1.tgz", + "integrity": "sha512-hXObxz7ZqoyhxET78+XR34Xu2qFGrJJ2I2bE5w4SM8eFaFEkW2xcGRVUss360fYelwRSid/jT078kbNvmoW0QA==", + "dependencies": { + "@babel/runtime": "^7.14.0", + "atob": "^2.1.2", + "btoa": "^1.2.1", + "fflate": "^0.4.8" + }, + "optionalDependencies": { + "canvg": "^3.0.6", + "core-js": "^3.6.0", + "dompurify": "^2.2.0", + "html2canvas": "^1.0.0-rc.5" + } + }, + "node_modules/jspdf/node_modules/canvg": { + "version": "3.0.10", + "resolved": "https://registry.npmjs.org/canvg/-/canvg-3.0.10.tgz", + "integrity": "sha512-qwR2FRNO9NlzTeKIPIKpnTY6fqwuYSequ8Ru8c0YkYU7U0oW+hLUvWadLvAu1Rl72OMNiFhoLu4f8eUjQ7l/+Q==", + "optional": true, + "dependencies": { + "@babel/runtime": "^7.12.5", + "@types/raf": "^3.4.0", + "core-js": "^3.8.3", + "raf": "^3.4.1", + "regenerator-runtime": "^0.13.7", + "rgbcolor": "^1.0.1", + "stackblur-canvas": "^2.0.0", + "svg-pathdata": "^6.0.3" + }, + "engines": { + "node": ">=10.0.0" + } + }, + "node_modules/jspdf/node_modules/core-js": { + "version": "3.37.1", + "resolved": "https://registry.npmjs.org/core-js/-/core-js-3.37.1.tgz", + "integrity": "sha512-Xn6qmxrQZyB0FFY8E3bgRXei3lWDJHhvI+u0q9TKIYM49G8pAr0FgnnrFRAmsbptZL1yxRADVXn+x5AGsbBfyw==", + "hasInstallScript": true, + "optional": true, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/core-js" + } + }, "node_modules/jss": { "version": "10.10.0", "resolved": "https://registry.npmjs.org/jss/-/jss-10.10.0.tgz", @@ -32804,6 +32914,14 @@ "resolved": "https://registry.npmjs.org/rgba-regex/-/rgba-regex-1.0.0.tgz", "integrity": "sha512-zgn5OjNQXLUTdq8m17KdaicF6w89TZs8ZU8y0AYENIU6wG8GG6LLm0yLSiPY8DmaYmHdgRW8rnApjoT0fQRfMg==" }, + "node_modules/rgbcolor": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/rgbcolor/-/rgbcolor-1.0.1.tgz", + "integrity": "sha512-9aZLIrhRaD97sgVhtJOW6ckOEh6/GnvQtdVNfdZ6s67+3/XwLS9lBcQYzEEhYVeUowN7pRzMLsyGhK2i/xvWbw==", + "engines": { + "node": ">= 0.8.15" + } + }, "node_modules/right-align": { "version": "0.1.3", "resolved": "https://registry.npmjs.org/right-align/-/right-align-0.1.3.tgz", @@ -34710,6 +34828,14 @@ "node": ">=10" } }, + "node_modules/stackblur-canvas": { + "version": "2.7.0", + "resolved": "https://registry.npmjs.org/stackblur-canvas/-/stackblur-canvas-2.7.0.tgz", + "integrity": "sha512-yf7OENo23AGJhBriGx0QivY5JP6Y1HbrrDI6WLt6C5auYZXlQrheoY8hD4ibekFKz1HOfE48Ww8kMWMnJD/zcQ==", + "engines": { + "node": ">=0.1.14" + } + }, "node_modules/stackframe": { "version": "1.3.4", "resolved": "https://registry.npmjs.org/stackframe/-/stackframe-1.3.4.tgz", @@ -35553,6 +35679,14 @@ "svg-path-bounds": "^1.0.1" } }, + "node_modules/svg-pathdata": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/svg-pathdata/-/svg-pathdata-6.0.3.tgz", + "integrity": "sha512-qsjeeq5YjBZ5eMdFuUa4ZosMLxgr5RZ+F+Y1OrDhuOCEInRMA3x74XdBtggJcj9kOeInz0WE+LgCPDkZFlBYJw==", + "engines": { + "node": ">=12.0.0" + } + }, "node_modules/svgo": { "version": "1.3.2", "resolved": "https://registry.npmjs.org/svgo/-/svgo-1.3.2.tgz", @@ -36500,6 +36634,15 @@ "vectorize-text": "^3.2.1" } }, + "node_modules/text-segmentation": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/text-segmentation/-/text-segmentation-1.0.3.tgz", + "integrity": "sha512-iOiPUo/BGnZ6+54OsWxZidGCsdU8YbE4PSpdPinp7DeMtUJNJBoJ/ouUSTJjHkh1KntHaltHl/gDs2FC4i5+Nw==", + "optional": true, + "dependencies": { + "utrie": "^1.0.2" + } + }, "node_modules/text-table": { "version": "0.2.0", "resolved": "https://registry.npmjs.org/text-table/-/text-table-0.2.0.tgz", @@ -37628,6 +37771,15 @@ "validate.io-string-primitive": "^1.0.0" } }, + "node_modules/utrie": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/utrie/-/utrie-1.0.2.tgz", + "integrity": "sha512-1MLa5ouZiOmQzUbjbu9VmjLzn1QLXBhwpUa7kdLUQK+KQ5KA9I1vk5U4YHe/X2Ch7PYnJfWuWT+VbuxbGwljhw==", + "optional": true, + "dependencies": { + "base64-arraybuffer": "^1.0.2" + } + }, "node_modules/uuid": { "version": "3.4.0", "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.4.0.tgz", @@ -47278,6 +47430,11 @@ "integrity": "sha512-FGa1F62FT09qcrueBA6qYTrJPVDzah9a+493+o2PCXsesWHIn27G98TsSMs3WPNbZIEj4+VJf6saSFpvD+3Zsw==", "dev": true }, + "@types/raf": { + "version": "3.4.3", + "resolved": "https://registry.npmjs.org/@types/raf/-/raf-3.4.3.tgz", + "integrity": "sha512-c4YAvMedbPZ5tEyxzQdMoOhhJ4RD3rngZIdwC2/qDN3d7JpEhB6fiBRKVY1lg5B7Wk+uPBjn5f39j1/2MY1oOw==" + }, "@types/range-parser": { "version": "1.2.4", "resolved": "https://registry.npmjs.org/@types/range-parser/-/range-parser-1.2.4.tgz", @@ -48902,6 +49059,12 @@ } } }, + "base64-arraybuffer": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/base64-arraybuffer/-/base64-arraybuffer-1.0.2.tgz", + "integrity": "sha512-I3yl4r9QB5ZRY3XuJVEPfc2XhZO6YweFPI+UovAzn+8/hb3oJ6lnysaFcjVpkCPfVWFUDvoZ8kmVDP7WyRtYtQ==", + "optional": true + }, "base64-js": { "version": "1.5.1", "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", @@ -49316,8 +49479,7 @@ "btoa": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/btoa/-/btoa-1.2.1.tgz", - "integrity": "sha512-SB4/MIGlsiVkMcHmT+pSmIPoNDoHg+7cMzmt3Uxt628MTz2487DKSqK/fuhFBrkuqrYv5UCEnACpF4dTFNKc/g==", - "dev": true + "integrity": "sha512-SB4/MIGlsiVkMcHmT+pSmIPoNDoHg+7cMzmt3Uxt628MTz2487DKSqK/fuhFBrkuqrYv5UCEnACpF4dTFNKc/g==" }, "buble": { "version": "0.19.8", @@ -49787,6 +49949,18 @@ "element-size": "^1.1.1" } }, + "canvg": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/canvg/-/canvg-4.0.2.tgz", + "integrity": "sha512-/7kIZger/mdFci4KXdtMr+NQB4GU1InkJ4RwSyDBRcvy4BUlg1hD+ZUWo550sWPyWaKZ8purqby6kjf09qVriw==", + "requires": { + "@types/raf": "^3.4.0", + "raf": "^3.4.1", + "rgbcolor": "^1.0.1", + "stackblur-canvas": "^2.0.0", + "svg-pathdata": "^6.0.3" + } + }, "capture-exit": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/capture-exit/-/capture-exit-2.0.0.tgz", @@ -50861,6 +51035,15 @@ "hyphenate-style-name": "^1.0.3" } }, + "css-line-break": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/css-line-break/-/css-line-break-2.1.0.tgz", + "integrity": "sha512-FHcKFCZcAha3LwfVBhCQbW2nCNbkZXn7KVUJcsT5/P8YmfsVja0FMPJr0B903j/E69HUphKiV9iQArX8SDYA4w==", + "optional": true, + "requires": { + "utrie": "^1.0.2" + } + }, "css-loader": { "version": "4.3.0", "resolved": "https://registry.npmjs.org/css-loader/-/css-loader-4.3.0.tgz", @@ -51910,6 +52093,12 @@ "domelementtype": "^2.2.0" } }, + "dompurify": { + "version": "2.5.6", + "resolved": "https://registry.npmjs.org/dompurify/-/dompurify-2.5.6.tgz", + "integrity": "sha512-zUTaUBO8pY4+iJMPE1B9XlO2tXVYIcEA4SNGtvDELzTSCQO7RzH+j7S180BmhmJId78lqGU2z19vgVx2Sxs/PQ==", + "optional": true + }, "domutils": { "version": "2.8.0", "resolved": "https://registry.npmjs.org/domutils/-/domutils-2.8.0.tgz", @@ -53579,6 +53768,11 @@ "integrity": "sha512-3yurQZ2hD9VISAhJJP9bpYFNQrHHBXE2JxxjY5aLEcDi46RmAzJE2OC9FAde0yis5ElW0jTTzs0zfg/Cca4XqQ==", "dev": true }, + "fflate": { + "version": "0.4.8", + "resolved": "https://registry.npmjs.org/fflate/-/fflate-0.4.8.tgz", + "integrity": "sha512-FJqqoDBR00Mdj9ppamLa/Y7vxm+PRmNWA67N846RvsoYVMKB4q3y/de5PA7gUmRMYK/8CMz2GDZQmCRN1wBcWA==" + }, "figgy-pudding": { "version": "3.5.2", "resolved": "https://registry.npmjs.org/figgy-pudding/-/figgy-pudding-3.5.2.tgz", @@ -55641,6 +55835,16 @@ } } }, + "html2canvas": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/html2canvas/-/html2canvas-1.4.1.tgz", + "integrity": "sha512-fPU6BHNpsyIhr8yyMpTLLxAbkaK8ArIBcmZIRiBLiDhjeqvXolaEmDGmELFuX9I4xDcaKKcJl+TKZLqruBbmWA==", + "optional": true, + "requires": { + "css-line-break": "^2.1.0", + "text-segmentation": "^1.0.3" + } + }, "htmlparser2": { "version": "6.1.0", "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-6.1.0.tgz", @@ -59266,6 +59470,45 @@ "integrity": "sha512-S6cATIPVv1z0IlxdN+zUk5EPjkGCdnhN4wVSBlvoUO1tOLJootbo9CquNJmbIh4yikWHiUedhRYrNPn1arpEmQ==", "dev": true }, + "jspdf": { + "version": "2.5.1", + "resolved": "https://registry.npmjs.org/jspdf/-/jspdf-2.5.1.tgz", + "integrity": "sha512-hXObxz7ZqoyhxET78+XR34Xu2qFGrJJ2I2bE5w4SM8eFaFEkW2xcGRVUss360fYelwRSid/jT078kbNvmoW0QA==", + "requires": { + "@babel/runtime": "^7.14.0", + "atob": "^2.1.2", + "btoa": "^1.2.1", + "canvg": "^3.0.6", + "core-js": "^3.6.0", + "dompurify": "^2.2.0", + "fflate": "^0.4.8", + "html2canvas": "^1.0.0-rc.5" + }, + "dependencies": { + "canvg": { + "version": "3.0.10", + "resolved": "https://registry.npmjs.org/canvg/-/canvg-3.0.10.tgz", + "integrity": "sha512-qwR2FRNO9NlzTeKIPIKpnTY6fqwuYSequ8Ru8c0YkYU7U0oW+hLUvWadLvAu1Rl72OMNiFhoLu4f8eUjQ7l/+Q==", + "optional": true, + "requires": { + "@babel/runtime": "^7.12.5", + "@types/raf": "^3.4.0", + "core-js": "^3.8.3", + "raf": "^3.4.1", + "regenerator-runtime": "^0.13.7", + "rgbcolor": "^1.0.1", + "stackblur-canvas": "^2.0.0", + "svg-pathdata": "^6.0.3" + } + }, + "core-js": { + "version": "3.37.1", + "resolved": "https://registry.npmjs.org/core-js/-/core-js-3.37.1.tgz", + "integrity": "sha512-Xn6qmxrQZyB0FFY8E3bgRXei3lWDJHhvI+u0q9TKIYM49G8pAr0FgnnrFRAmsbptZL1yxRADVXn+x5AGsbBfyw==", + "optional": true + } + } + }, "jss": { "version": "10.10.0", "resolved": "https://registry.npmjs.org/jss/-/jss-10.10.0.tgz", @@ -66098,6 +66341,11 @@ "resolved": "https://registry.npmjs.org/rgba-regex/-/rgba-regex-1.0.0.tgz", "integrity": "sha512-zgn5OjNQXLUTdq8m17KdaicF6w89TZs8ZU8y0AYENIU6wG8GG6LLm0yLSiPY8DmaYmHdgRW8rnApjoT0fQRfMg==" }, + "rgbcolor": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/rgbcolor/-/rgbcolor-1.0.1.tgz", + "integrity": "sha512-9aZLIrhRaD97sgVhtJOW6ckOEh6/GnvQtdVNfdZ6s67+3/XwLS9lBcQYzEEhYVeUowN7pRzMLsyGhK2i/xvWbw==" + }, "right-align": { "version": "0.1.3", "resolved": "https://registry.npmjs.org/right-align/-/right-align-0.1.3.tgz", @@ -67692,6 +67940,11 @@ "escape-string-regexp": "^2.0.0" } }, + "stackblur-canvas": { + "version": "2.7.0", + "resolved": "https://registry.npmjs.org/stackblur-canvas/-/stackblur-canvas-2.7.0.tgz", + "integrity": "sha512-yf7OENo23AGJhBriGx0QivY5JP6Y1HbrrDI6WLt6C5auYZXlQrheoY8hD4ibekFKz1HOfE48Ww8kMWMnJD/zcQ==" + }, "stackframe": { "version": "1.3.4", "resolved": "https://registry.npmjs.org/stackframe/-/stackframe-1.3.4.tgz", @@ -68368,6 +68621,11 @@ "svg-path-bounds": "^1.0.1" } }, + "svg-pathdata": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/svg-pathdata/-/svg-pathdata-6.0.3.tgz", + "integrity": "sha512-qsjeeq5YjBZ5eMdFuUa4ZosMLxgr5RZ+F+Y1OrDhuOCEInRMA3x74XdBtggJcj9kOeInz0WE+LgCPDkZFlBYJw==" + }, "svgo": { "version": "1.3.2", "resolved": "https://registry.npmjs.org/svgo/-/svgo-1.3.2.tgz", @@ -69091,6 +69349,15 @@ "vectorize-text": "^3.2.1" } }, + "text-segmentation": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/text-segmentation/-/text-segmentation-1.0.3.tgz", + "integrity": "sha512-iOiPUo/BGnZ6+54OsWxZidGCsdU8YbE4PSpdPinp7DeMtUJNJBoJ/ouUSTJjHkh1KntHaltHl/gDs2FC4i5+Nw==", + "optional": true, + "requires": { + "utrie": "^1.0.2" + } + }, "text-table": { "version": "0.2.0", "resolved": "https://registry.npmjs.org/text-table/-/text-table-0.2.0.tgz", @@ -70006,6 +70273,15 @@ "validate.io-string-primitive": "^1.0.0" } }, + "utrie": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/utrie/-/utrie-1.0.2.tgz", + "integrity": "sha512-1MLa5ouZiOmQzUbjbu9VmjLzn1QLXBhwpUa7kdLUQK+KQ5KA9I1vk5U4YHe/X2Ch7PYnJfWuWT+VbuxbGwljhw==", + "optional": true, + "requires": { + "base64-arraybuffer": "^1.0.2" + } + }, "uuid": { "version": "3.4.0", "resolved": "https://registry.npmjs.org/uuid/-/uuid-3.4.0.tgz", diff --git a/managed/ui/package.json b/managed/ui/package.json index 4291cfe2ef53..be94a6a14625 100644 --- a/managed/ui/package.json +++ b/managed/ui/package.json @@ -82,6 +82,7 @@ "ace-builds": "1.4.12", "axios": "0.21.3", "bootstrap": "3.4.1", + "canvg": "4.0.2", "clsx": "1.1.1", "copy-to-clipboard": "3.3.1", "cron-parser": "2.16.3", @@ -94,6 +95,7 @@ "intl": "1.2.5", "js-cookie": "2.2.1", "js-yaml": "4.1.0", + "jspdf": "2.5.1", "leaflet": "^1.0.2", "leaflet.markercluster": "1.4.1", "lodash": "4.17.21", diff --git a/managed/ui/src/components/metrics/CustomerMetricsPanel/CustomerMetricsPanel.js b/managed/ui/src/components/metrics/CustomerMetricsPanel/CustomerMetricsPanel.js index 504136735d03..67eba5e92259 100644 --- a/managed/ui/src/components/metrics/CustomerMetricsPanel/CustomerMetricsPanel.js +++ b/managed/ui/src/components/metrics/CustomerMetricsPanel/CustomerMetricsPanel.js @@ -41,7 +41,8 @@ const PanelBody = ({ width, tableName, graph, - customer + customer, + printMode }) => { let result = null; const runtimeConfigs = customer?.runtimeConfigs; @@ -115,66 +116,88 @@ const PanelBody = ({ ) ) { if (!invalidTabType.includes(type)) { - prevTabs.push( - - - - ); + if (printMode) { + prevTabs.push( +
+ +
+ ); + } else { + prevTabs.push( + + + + ); + } } } return prevTabs; }, []); if (origin === MetricOrigin.UNIVERSE && isDrEnabled && hasDrConfig) { - metricTabs.push( - + if (printMode) { + metricTabs.push( See xCluster DR Metrics - + ); + } else { + metricTabs.push( + + + + See xCluster DR Metrics + + + + ); + } + } + if (printMode) { + result = ; + } else { + result = ( + + {metricTabs} + ); } - result = ( - - {metricTabs} - - ); } else if (metricMeasure === MetricMeasure.OUTLIER_TABLES) { - result = ( - - + if (printMode) { + result = ( + + ); + } else { + result = ( + + + + + + ); + } } return result; @@ -216,10 +267,10 @@ export default class CustomerMetricsPanel extends Component { } render() { - const { origin } = this.props; + const { origin, printMode = false } = this.props; return ( - - + + ); } diff --git a/managed/ui/src/components/metrics/GraphPanelHeader/GraphPanelHeader.js b/managed/ui/src/components/metrics/GraphPanelHeader/GraphPanelHeader.js index 66309640e68f..4703c04d6545 100644 --- a/managed/ui/src/components/metrics/GraphPanelHeader/GraphPanelHeader.js +++ b/managed/ui/src/components/metrics/GraphPanelHeader/GraphPanelHeader.js @@ -1,6 +1,9 @@ // Copyright (c) YugaByte, Inc. import React, { Component, Fragment } from 'react'; +import { jsPDF } from 'jspdf'; +import { Canvg, presets } from 'canvg'; +import { Box, Typography } from '@material-ui/core'; import { Link, withRouter, browserHistory } from 'react-router'; import { Dropdown, MenuItem } from 'react-bootstrap'; import momentLocalizer from 'react-widgets-moment'; @@ -22,7 +25,7 @@ import { SplitType } from '../dtos'; import { YBButton, YBButtonLink } from '../../common/forms/fields'; import { YBPanelItem } from '../../panels'; import { FlexContainer, FlexGrow } from '../../common/flexbox/YBFlexBox'; - +import CustomerMetricsPanel from '../CustomerMetricsPanel/CustomerMetricsPanel'; import { getPromiseState } from '../../../utils/PromiseUtils'; import { isValidObject, isNonEmptyObject } from '../../../utils/ObjectUtils'; import { isDedicatedNodePlacement, isKubernetesUniverse } from '../../../utils/UniverseUtils'; @@ -33,10 +36,9 @@ import { NodeTypeSelector } from '../NodeTypeSelector/NodeTypeSelector'; import { CustomDatePicker } from '../CustomDatePicker/CustomDatePicker'; import { MetricsMeasureSelector } from '../MetricsMeasureSelector/MetricsMeasureSelector'; import { OutlierSelector } from '../OutlierSelector/OutlierSelector'; -import { RuntimeConfigKey } from '../../../redesign/helpers/constants'; import { ybFormatDate } from '../../../redesign/helpers/DateUtils'; - import './GraphPanelHeader.scss'; +import { YBModal, YBProgress } from '../../../redesign/components'; require('react-widgets/dist/css/react-widgets.css'); @@ -138,7 +140,10 @@ class GraphPanelHeader extends Component { metricMeasure: metricMeasureTypes[DEFAULT_METRIC_MEASURE_KEY].value, outlierType: outlierTypes[DEFAULT_OUTLIER_TYPE].value, outlierNumNodes: DEFAULT_OUTLIER_NUM_NODES, - isSingleNodeSelected: false + isSingleNodeSelected: false, + openPreviewMetricsModal: false, + pdfDownloadInProgress: false, + downloadPercent: 0 }; this.props.setGraphFilter(defaultFilters); @@ -553,11 +558,11 @@ class GraphPanelHeader extends Component { origin, universe: { currentUniverse }, prometheusQueryEnabled, - customer: { currentUser, runtimeConfigs }, showModal, closeModal, visibleModal, - enableNodeComparisonModal + enableNodeComparisonModal, + printMode } = this.props; const { filterType, @@ -644,208 +649,339 @@ class GraphPanelHeader extends Component { - {this.props.origin === MetricOrigin.CUSTOMER ? ( -

Metrics

- ) : ( - '' - )} - - -
- {universePicker} - {this.props.origin !== MetricOrigin.TABLE && ( - - )} - {this.props.origin !== MetricOrigin.TABLE && isDedicatedNodes && ( - - )} - {this.props.origin !== MetricOrigin.TABLE && ( - - )} - {liveQueriesLink && !universePaused && ( - - - See Queries - - - )} - {enableNodeComparisonModal && ( - showModal('metricsComparisonModal')} - /> - )} -
-
- -
-
-
- Current Timestamp: {ybFormatDate(new Date())} -
-
+ !printMode && ( +
+ {this.props.origin === MetricOrigin.CUSTOMER ? ( +

Metrics

+ ) : ( + '' + )} + + +
+ {universePicker} + {this.props.origin !== MetricOrigin.TABLE && ( + + )} + {this.props.origin !== MetricOrigin.TABLE && isDedicatedNodes && ( + + )} + {this.props.origin !== MetricOrigin.TABLE && ( + + )} + {liveQueriesLink && !universePaused && ( + + + See Queries + + + )} + {enableNodeComparisonModal && ( + showModal('metricsComparisonModal')} + /> + )} +
+
+ + +
+
+ Current Timestamp: {ybFormatDate(new Date())} +
+
+ + + Auto Refresh:  + + {refreshIntervalLabel} + + + {intervalMenuItems} + + +   + { + this.setState({ + openPreviewMetricsModal: true, + pdfDownloadInProgress: false, + downloadPercent: 0 + }); + }} + /> +
- - Auto Refresh:  - - {refreshIntervalLabel} - + + - {intervalMenuItems} + + + VIEW OPTIONS + + + + {prometheusQueryEnabled + ? 'Disable Prometheus query' + : 'Enable Prometheus query'} + + { + self.props + .getGrafanaJson() + .then((response) => { + return new Blob([JSON.stringify(response.data, null, 2)], { + type: 'application/json' + }); + }) + .catch((error) => { + toast.error( + 'Error in downloading Grafana JSON: ' + error.message + ); + return null; + }) + .then((blob) => { + // eslint-disable-next-line eqeqeq + if (blob != null) { + const url = window.URL.createObjectURL(blob); + const a = document.createElement('a'); + a.style.display = 'none'; + a.href = url; + a.download = 'Grafana_Dashboard.json'; + document.body.appendChild(a); + a.click(); + window.URL.revokeObjectURL(url); + a.remove(); + } + }); + }} + > + {'Download Grafana JSON'} + + -
- - - + +
+
+ + + {this.state.currentSelectedUniverse !== MetricConsts.ALL && + this.props.origin !== MetricOrigin.TABLE && ( + + )} + + +
+ {datePicker} + + +   + {this.state.filterLabel} - - - VIEW OPTIONS - - - - {prometheusQueryEnabled - ? 'Disable Prometheus query' - : 'Enable Prometheus query'} - - { - self.props - .getGrafanaJson() - .then((response) => { - return new Blob([JSON.stringify(response.data, null, 2)], { - type: 'application/json' - }); - }) - .catch((error) => { - toast.error( - 'Error in downloading Grafana JSON: ' + error.message - ); - return null; - }) - .then((blob) => { - // eslint-disable-next-line eqeqeq - if (blob != null) { - const url = window.URL.createObjectURL(blob); - const a = document.createElement('a'); - a.style.display = 'none'; - a.href = url; - a.download = 'Grafana_Dashboard.json'; - document.body.appendChild(a); - a.click(); - window.URL.revokeObjectURL(url); - a.remove(); - } - }); - }} - > - {'Download Grafana JSON'} - - + {menuItems}
- -
-
- - - {this.state.currentSelectedUniverse !== MetricConsts.ALL && - this.props.origin !== MetricOrigin.TABLE && ( - - )} - - -
- {datePicker} - - -   - {this.state.filterLabel} - - {menuItems} - -
-
-
- - - {/* Show Outlier Selector component if user has selected Outlier section + + + + + {/* Show Outlier Selector component if user has selected Outlier section or if user has selected TopTables tab in Overall section */} - {currentSelectedUniverse !== MetricConsts.ALL && - (this.state.metricMeasure === MetricMeasure.OUTLIER || - this.state.metricMeasure === MetricMeasure.OUTLIER_TABLES) && ( - - )} - - - {enableNodeComparisonModal ? ( - - ) : ( - '' - )} -
+ {currentSelectedUniverse !== MetricConsts.ALL && + (this.state.metricMeasure === MetricMeasure.OUTLIER || + this.state.metricMeasure === MetricMeasure.OUTLIER_TABLES) && ( + + )} + + + { + this.setState({ + openPreviewMetricsModal: false + }); + }} + footerAccessory={ + this.state.pdfDownloadInProgress && ( + + + Please wait while Download metrics to pdf is in Progress, Closing the + preview modal may abort the operation. + {' '} +     + + + ) + } + onSubmit={async () => { + try { + this.setState({ + pdfDownloadInProgress: true + }); + const tabs = document.getElementById('print-metrics'); + + const pdf = new jsPDF('p', 'pt', 'a4'); + pdf.setFontSize(22); + pdf.text('Metrics', 250, 25, null, null, 'center'); + const allCharts = []; + const charts = tabs.querySelectorAll('.metrics-panel-container'); + for (let j = 0; j < charts.length; j++) { + const svgs = charts[j].querySelectorAll('svg'); + allCharts.push(svgs); + } + + const positions = [ + [0, 0], + [1, 0], + [0, 1], + [1, 1] + ]; + + for (let i = 0; i < allCharts.length; i++) { + if (allCharts[i].type !== 'br') { + if (i !== 0 && i % 4 === 0) { + pdf.addPage(); + } + + for (let j = 0; j < allCharts[i].length; j++) { + const svg = allCharts[i][j]; + const pos = i % 4; + const svgData = new XMLSerializer().serializeToString(svg); + const svgDataBase64 = btoa(unescape(encodeURIComponent(svgData))); + const svgDataUrl = `data:image/svg+xml;charset=utf-8;base64,${svgDataBase64}`; + + const image = new Image(); + image.src = svgDataUrl; + await image.decode(); + + const preset = presets.offscreen(); + + const canvas = new OffscreenCanvas(1500, 1500); + + const dimension = pdf.internal.pageSize.width / 2 - 10; + const ctx = canvas.getContext('2d'); + const v = await Canvg.fromString(ctx, svgData, preset); + v.resize(500, 500, 'xMidYMid meet'); + await v.render(); + + const blob = await canvas.convertToBlob(); + const imgURL = URL.createObjectURL(blob); + + pdf.addImage( + imgURL, + 'PNG', + positions[pos][0] * dimension + 5, + positions[pos][1] * dimension + 40, + dimension, + dimension, + '', + 'FAST' + ); + this.setState({ + downloadPercent: (i / charts.length) * 100 + }); + } + } else { + pdf.addPage(); + } + } + + pdf.save(`metrics-${Date.now()}.pdf`); + this.setState({ + pdfDownloadInProgress: false + }); + } catch (e) { + console.warn(e); + this.setState({ + pdfDownloadInProgress: false + }); + } + }} + > + + + {enableNodeComparisonModal ? ( + + ) : ( + '' + )} +
+ ) } /* React.cloneELement for passing state down to child components in HOC */ body={React.cloneElement(this.props.children, { diff --git a/managed/ui/src/components/metrics/GraphTab/GraphTab.tsx b/managed/ui/src/components/metrics/GraphTab/GraphTab.tsx index e13990302040..41768b75d8d1 100644 --- a/managed/ui/src/components/metrics/GraphTab/GraphTab.tsx +++ b/managed/ui/src/components/metrics/GraphTab/GraphTab.tsx @@ -24,6 +24,7 @@ interface MetricsData { title: string; tableName?: string; isGranularMetricsEnabled: boolean; + printMode: boolean; } export const GraphTab: FC = ({ @@ -33,7 +34,8 @@ export const GraphTab: FC = ({ selectedUniverse, title, tableName, - isGranularMetricsEnabled + isGranularMetricsEnabled, + printMode = false }) => { let tabContent = null; const { currentUser } = useSelector((state: any) => state.customer); @@ -175,7 +177,8 @@ export const GraphTab: FC = ({ title, currentUser, isGranularMetricsEnabled, - updateTimestamp + updateTimestamp, + printMode ); return <>{tabContent}; diff --git a/managed/ui/src/components/metrics/MetricsPanel/MetricsPanel.js b/managed/ui/src/components/metrics/MetricsPanel/MetricsPanel.js index ca157d7730e7..c711322ff54b 100644 --- a/managed/ui/src/components/metrics/MetricsPanel/MetricsPanel.js +++ b/managed/ui/src/components/metrics/MetricsPanel/MetricsPanel.js @@ -57,8 +57,10 @@ export default class MetricsPanel extends Component { metricMeasure, operations = [], metricType, - isMetricLoading = false + isMetricLoading = false, + printMode = false } = this.props; + const newMetricKey = printMode ? `${metricKey}-printMode` : metricKey; const metric = metricMeasure === MetricMeasure.OUTLIER || metricType === MetricTypes.OUTLIER_TABLES ? _.cloneDeep(this.props.metric) @@ -156,7 +158,7 @@ export default class MetricsPanel extends Component { metric.layout.height = layoutHeight; metric.layout.showlegend = true; metric.layout.title = { - text: metric.layout.title, + text: metric?.layout?.title?.text ?? metric.layout.title, x: 0.05, y: 2.2, xref: 'container', @@ -213,7 +215,7 @@ export default class MetricsPanel extends Component { metric.layout.xaxis = { range: [0, 2] }; metric.layout.yaxis = { range: [0, 2] }; } - Plotly.newPlot(metricKey, metric.data, metric.layout, { displayModeBar: false }); + Plotly.newPlot(newMetricKey, metric.data, metric.layout, { displayModeBar: false }); } }; @@ -228,7 +230,7 @@ export default class MetricsPanel extends Component { componentDidUpdate(prevProps) { // Enables user to view granular data based on selected time range within the graph if (this.props.isGranularMetricsEnabled) { - const metricKeyContainer = document.getElementById(prevProps.metricKey); + const metricKeyContainer = document.getElementById(prevProps.newMetricKey); metricKeyContainer?.on('plotly_relayout', function (eventData) { const startTime = Math.floor(new Date(eventData['xaxis.range[0]']).getTime() / 1000); const endTime = Math.floor(new Date(eventData['xaxis.range[1]']).getTime() / 1000); @@ -240,7 +242,7 @@ export default class MetricsPanel extends Component { this.props.containerWidth !== prevProps.containerWidth || this.props.width !== prevProps.width ) { - Plotly.relayout(prevProps.metricKey, { + Plotly.relayout(prevProps.newMetricKey, { width: this.props.width || this.getGraphWidth(this.props.containerWidth) }); } else { @@ -329,9 +331,11 @@ export default class MetricsPanel extends Component { metricOperationsDropdown = operations?.slice(operations.length - numButtonsInDropdown); } const focusedButton = this.state.focusedButton ? this.state.focusedButton : operations?.[0]; - + const newMetricKey = this.props.printMode + ? `${this.props.metricKey}-printMode` + : this.props.metricKey; return ( -
+
{(metricMeasure === MetricMeasure.OUTLIER || metricType === MetricTypes.OUTLIER_TABLES) && operations.length > 0 && diff --git a/managed/ui/src/utils/GraphUtils.tsx b/managed/ui/src/utils/GraphUtils.tsx index 90c80f776e0a..24811d04b1ba 100644 --- a/managed/ui/src/utils/GraphUtils.tsx +++ b/managed/ui/src/utils/GraphUtils.tsx @@ -12,7 +12,8 @@ export const getTabContent = ( title: string, currentUser: any, isGranularMetricsEnabled: boolean, - updateTimestamp: (start: 'object' | number, end: 'object' | number) => void + updateTimestamp: (start: 'object' | number, end: 'object' | number) => void, + printMode: boolean ) => { let tabData: any = ; if (graph.error?.data && !graph.loading) { @@ -65,6 +66,7 @@ export const getTabContent = ( operations={uniqueOperations} isGranularMetricsEnabled={isGranularMetricsEnabled} updateTimestamp={updateTimestamp} + printMode={printMode} /> ) : null; }) diff --git a/src/postgres/src/backend/optimizer/path/costsize.c b/src/postgres/src/backend/optimizer/path/costsize.c index 8d71a3444f80..bff36204d56f 100644 --- a/src/postgres/src/backend/optimizer/path/costsize.c +++ b/src/postgres/src/backend/optimizer/path/costsize.c @@ -173,6 +173,7 @@ double yb_random_block_cost = DEFAULT_RANDOM_PAGE_COST; double yb_docdb_next_cpu_cycles = YB_DEFAULT_DOCDB_NEXT_CPU_CYCLES; double yb_seek_cost_factor = YB_DEFAULT_SEEK_COST_FACTOR; double yb_backward_seek_cost_factor = YB_DEFAULT_BACKWARD_SEEK_COST_FACTOR; +double yb_fast_backward_seek_cost_factor = YB_DEFAULT_FAST_BACKWARD_SEEK_COST_FACTOR; int yb_docdb_merge_cpu_cycles = YB_DEFAULT_DOCDB_MERGE_CPU_CYCLES; int yb_docdb_remote_filter_overhead_cycles = YB_DEFAULT_DOCDB_REMOTE_FILTER_OVERHEAD_CYCLES; double yb_local_latency_cost = YB_DEFAULT_LOCAL_LATENCY_COST; @@ -7811,7 +7812,8 @@ yb_cost_index(IndexPath *path, PlannerInfo *root, double loop_count, if (path->indexscandir == BackwardScanDirection) { - per_next_cost *= yb_backward_seek_cost_factor; + per_next_cost *= YbUseFastBackwardScan() ? + yb_fast_backward_seek_cost_factor : yb_backward_seek_cost_factor; } run_cost += num_seeks * index_per_seek_cost + diff --git a/src/postgres/src/include/optimizer/cost.h b/src/postgres/src/include/optimizer/cost.h index bc4b9e87d430..5251956c6026 100644 --- a/src/postgres/src/include/optimizer/cost.h +++ b/src/postgres/src/include/optimizer/cost.h @@ -50,6 +50,14 @@ #define YB_DEFAULT_SEEK_COST_FACTOR 50 #define YB_DEFAULT_BACKWARD_SEEK_COST_FACTOR 10 +/* + * The value for the fast backward scan seek cost factor has been selected based on the smallest + * improvement (2.8 times) for the backward scan related Order By workloads of Featurebench. It + * might be good to use a different factor for colocated case, where the smallest improvement + * is 3 times higher comparing to non-colocated case; refer to D35894 for the details. + */ +#define YB_DEFAULT_FAST_BACKWARD_SEEK_COST_FACTOR (YB_DEFAULT_BACKWARD_SEEK_COST_FACTOR / 3.0) + /* DocDB row decode and process cost */ #define YB_DEFAULT_DOCDB_MERGE_CPU_CYCLES 50 diff --git a/src/postgres/third-party-extensions/pg_partman/sql/functions/create_partition_time.sql b/src/postgres/third-party-extensions/pg_partman/sql/functions/create_partition_time.sql index db8459ca45b9..2a609c0dde18 100644 --- a/src/postgres/third-party-extensions/pg_partman/sql/functions/create_partition_time.sql +++ b/src/postgres/third-party-extensions/pg_partman/sql/functions/create_partition_time.sql @@ -259,7 +259,7 @@ FOREACH v_time IN ARRAY p_partition_times LOOP END IF; END IF; - IF v_exists IS NULL THEN + IF v_exists = 0 THEN RAISE DEBUG 'create_partition_time v_sql: %', v_sql; EXECUTE v_sql; END IF; diff --git a/src/yb/docdb/cql_operation.cc b/src/yb/docdb/cql_operation.cc index 0c0be6c5cdc7..e7c12fc70396 100644 --- a/src/yb/docdb/cql_operation.cc +++ b/src/yb/docdb/cql_operation.cc @@ -494,7 +494,7 @@ Status QLWriteOperation::ReadColumns(const DocOperationApplyData& data, // Generate hashed / primary key depending on if static / non-static columns are referenced in // the if-condition. RETURN_NOT_OK(InitializeKeys( - !static_projection->columns.empty(), !non_static_projection->columns.empty())); + !request_.column_refs().static_ids().empty(), !non_static_projection->columns.empty())); // Scan docdb for the static and non-static columns of the row using the hashed / primary key. if (hashed_doc_key_) { diff --git a/src/yb/integration-tests/cql-test.cc b/src/yb/integration-tests/cql-test.cc index e3e44429401b..ee68e31fc3bb 100644 --- a/src/yb/integration-tests/cql-test.cc +++ b/src/yb/integration-tests/cql-test.cc @@ -1540,4 +1540,52 @@ TEST_F(CqlTest, RetainSchemaPacking) { LOG(INFO) << "Content: " << content; } +TEST_F(CqlTest, InsertHashAndRangePkWithReturnsStatusAsRow) { + constexpr auto TTL_SECONDS = 2; + constexpr auto INSERT_PHASE_SECONDS = TTL_SECONDS + 3; + + constexpr auto HASH_COLUMN_VALUE = 12345678; + + auto session = ASSERT_RESULT(EstablishSession(driver_.get())); + + ASSERT_OK(session.ExecuteQuery( + "CREATE TABLE test (h int, r int, v int, PRIMARY KEY (h, r)) " + "WITH CLUSTERING ORDER BY (r ASC) AND transactions = {'enabled': 'false'}")); + + auto insert_prepared = ASSERT_RESULT(session.Prepare(Format( + "INSERT INTO test (h, r, v) VALUES (?,?,?) USING TTL $0 RETURNS STATUS AS ROW", + TTL_SECONDS))); + + int row_count = 0; + + auto deadline = CoarseMonoClock::now() + INSERT_PHASE_SECONDS * 1s; + while (CoarseMonoClock::now() < deadline) { + auto stmt = insert_prepared.Bind(); + stmt.Bind(0, HASH_COLUMN_VALUE); + stmt.Bind(1, row_count); + stmt.Bind(2, row_count); + ASSERT_OK(session.Execute(stmt)); + ++row_count; + YB_LOG_EVERY_N_SECS(INFO, 5) << row_count << " rows inserted"; + } + + ASSERT_GT(row_count, 0) << "We expect some rows to be inserted"; + + // Make sure `RETURNS STATUS AS ROW` doesn't iterate over rows (both obsolete and live) related to + // the same hash column but different range columns. + for (auto peer : ListTabletPeers(cluster_.get(), ListPeersFilter::kAll)) { + auto tablet = peer->shared_tablet(); + if (tablet->table_type() != TableType::YQL_TABLE_TYPE) { + break; + } + auto* metrics = tablet->metrics(); + for (auto counter : + {tablet::TabletCounters::kDocDBKeysFound, tablet::TabletCounters::kDocDBObsoleteKeysFound, + tablet::TabletCounters::kDocDBObsoleteKeysFoundPastCutoff}) { + ASSERT_EQ(metrics->Get(counter), 0) + << "Expected " << counter << " to be zero for tablet peer " << peer->LogPrefix(); + } + } +} + } // namespace yb diff --git a/src/yb/master/catalog_manager.cc b/src/yb/master/catalog_manager.cc index 78c093cfb9d0..58b109fd1fe8 100644 --- a/src/yb/master/catalog_manager.cc +++ b/src/yb/master/catalog_manager.cc @@ -3274,6 +3274,26 @@ Status CatalogManager::DdlLog( return sys_catalog_->FetchDdlLog(resp->mutable_entries()); } +Status CatalogManager::StartYsqlMajorVersionUpgradeInitdb( + const StartYsqlMajorVersionUpgradeInitdbRequestPB* req, + StartYsqlMajorVersionUpgradeInitdbResponsePB* resp, + rpc::RpcContext* rpc, const LeaderEpoch& epoch) { + return STATUS(NotSupported, "Ysql major version upgrade is not supported"); +} + +Status CatalogManager::IsYsqlMajorVersionUpgradeInitdbDone( + const IsYsqlMajorVersionUpgradeInitdbDoneRequestPB* req, + IsYsqlMajorVersionUpgradeInitdbDoneResponsePB* resp, rpc::RpcContext* rpc) { + return STATUS(NotSupported, "Ysql major version upgrade is not supported"); +} + +Status CatalogManager::RollbackYsqlMajorVersionUpgrade( + const RollbackYsqlMajorVersionUpgradeRequestPB* req, + RollbackYsqlMajorVersionUpgradeResponsePB* resp, + rpc::RpcContext* rpc, const LeaderEpoch& epoch) { + return STATUS(NotSupported, "Ysql major version upgrade is not supported"); +} + namespace { Status ValidateCreateTableSchema(const Schema& schema, CreateTableResponsePB* resp) { diff --git a/src/yb/master/catalog_manager.h b/src/yb/master/catalog_manager.h index a443a3c7294c..2a34a5056330 100644 --- a/src/yb/master/catalog_manager.h +++ b/src/yb/master/catalog_manager.h @@ -1098,6 +1098,21 @@ class CatalogManager : public tserver::TabletPeerLookupIf, Status DdlLog( const DdlLogRequestPB* req, DdlLogResponsePB* resp, rpc::RpcContext* rpc); + // Not implemented. + Status StartYsqlMajorVersionUpgradeInitdb(const StartYsqlMajorVersionUpgradeInitdbRequestPB* req, + StartYsqlMajorVersionUpgradeInitdbResponsePB* resp, + rpc::RpcContext* rpc, const LeaderEpoch& epoch); + + // Not implemented. + Status IsYsqlMajorVersionUpgradeInitdbDone( + const IsYsqlMajorVersionUpgradeInitdbDoneRequestPB* req, + IsYsqlMajorVersionUpgradeInitdbDoneResponsePB* resp, rpc::RpcContext* rpc); + + // Not implemented. + Status RollbackYsqlMajorVersionUpgrade(const RollbackYsqlMajorVersionUpgradeRequestPB* req, + RollbackYsqlMajorVersionUpgradeResponsePB* resp, + rpc::RpcContext* rpc, const LeaderEpoch& epoch); + // Test wrapper around protected DoSplitTablet method. Status TEST_SplitTablet( const TabletInfoPtr& source_tablet_info, diff --git a/src/yb/master/master_admin.proto b/src/yb/master/master_admin.proto index 5a3c544d7ced..43b8448e1630 100644 --- a/src/yb/master/master_admin.proto +++ b/src/yb/master/master_admin.proto @@ -177,6 +177,29 @@ message DdlLogResponsePB { repeated DdlLogEntryPB entries = 2; } +message StartYsqlMajorVersionUpgradeInitdbRequestPB { +} + +message StartYsqlMajorVersionUpgradeInitdbResponsePB { + optional MasterErrorPB error = 1; +} + +message IsYsqlMajorVersionUpgradeInitdbDoneRequestPB { +} + +message IsYsqlMajorVersionUpgradeInitdbDoneResponsePB { + optional MasterErrorPB error = 1; + optional bool done = 2; + optional MasterErrorPB initdb_error = 3; +} + +message RollbackYsqlMajorVersionUpgradeRequestPB { +} + +message RollbackYsqlMajorVersionUpgradeResponsePB { + optional MasterErrorPB error = 1; +} + message CheckIfPitrActiveRequestPB {} message CheckIfPitrActiveResponsePB { @@ -268,6 +291,13 @@ service MasterAdmin { rpc DdlLog(DdlLogRequestPB) returns (DdlLogResponsePB); + rpc StartYsqlMajorVersionUpgradeInitdb(StartYsqlMajorVersionUpgradeInitdbRequestPB) + returns (StartYsqlMajorVersionUpgradeInitdbResponsePB); + rpc IsYsqlMajorVersionUpgradeInitdbDone(IsYsqlMajorVersionUpgradeInitdbDoneRequestPB) + returns (IsYsqlMajorVersionUpgradeInitdbDoneResponsePB); + rpc RollbackYsqlMajorVersionUpgrade(RollbackYsqlMajorVersionUpgradeRequestPB) + returns (RollbackYsqlMajorVersionUpgradeResponsePB); + rpc CheckIfPitrActive(CheckIfPitrActiveRequestPB) returns (CheckIfPitrActiveResponsePB); diff --git a/src/yb/master/master_admin_service.cc b/src/yb/master/master_admin_service.cc index 417f27e30739..23748b73ec4c 100644 --- a/src/yb/master/master_admin_service.cc +++ b/src/yb/master/master_admin_service.cc @@ -49,6 +49,9 @@ class MasterAdminServiceImpl : public MasterServiceBase, public MasterAdminIf { (GetCompactionStatus) (CreateTransactionStatusTable) (DdlLog) + (StartYsqlMajorVersionUpgradeInitdb) + (IsYsqlMajorVersionUpgradeInitdbDone) + (RollbackYsqlMajorVersionUpgrade) (DeleteNotServingTablet) (FlushSysCatalog) (SplitTablet) diff --git a/src/yb/tools/yb-admin_cli.cc b/src/yb/tools/yb-admin_cli.cc index 84e00b505888..568946ab0dd9 100644 --- a/src/yb/tools/yb-admin_cli.cc +++ b/src/yb/tools/yb-admin_cli.cc @@ -1596,12 +1596,13 @@ Status create_keyspace_snapshot_action( return Status::OK(); } -const auto create_database_snapshot_args = "[ysql.] [retention_duration_hours] " - "(set a <= 0 value to retain the snapshot forever. If not specified " +const auto create_database_snapshot_args = + "[ysql.] [retention_duration_hours] " + "(set retention_duration_hours to 0 to retain the snapshot forever. If not specified " "then takes the default value controlled by gflag default_retention_hours)"; Status create_database_snapshot_action( const ClusterAdminCli::CLIArguments& args, ClusterAdminClient* client) { - if (args.size() != 1) { + if (args.size() < 1 && args.size() > 2) { return ClusterAdminCli::kInvalidArguments; }