v2.4.0.0 (Released Jan 21, 2021)

Release Notes

v2.4.0 - Jan 21, 2021

New Features

Yugabyte Platform

• Data encryption in-transit enhancements:

  • Support for bringing your own CA certificates for an on-premise cloud provider (#5729)
  • Support for rotating custom CA certificates (#5727)
  • Displaying the certificate name in the YB Platform Universe editor
  • Moving of the Certificates page to Cloud Config > Security > Encryption in Transit

• Moving of TLS In-Transit Certificates to Cloud > Security Config

• Support for Rolling restart of a universe (#6323)

• Support for VMware Tanzu as a new cloud provider (#6633)

• Alerts for backup tasks (#5556)

Core Database

• General availability of Automatic Tablet splitting, which enables resharding of data in a cluster automatically while online, and transparently to users, when a specified size threshold has been reached (#1004)
• Introducing support for audit logging in YCQL and YSQL API (#1331, #5887, #6199)
• Ability to log slow running queries in YSQL (#4817)
• Introducing support for LDAP integration in YSQL API (#6088)

Improvements

Yugabyte Platform

• Support for Transactional tables and Cross Cluster Async Replication topology (#5779)

• Support for very large transactions and stability improvements (#1923)

• Displaying an entire query in the detailed view of the live queries tab (#6412)

• Not returning Hashes and Tokens in API responses (#6388)

• Authentication of proxy requests against a valid platform session (#6544)

• Disabling of the unused API endpoint run_query (#6383)

• Improved error handling for user-created concurrent backup tasks (#5888)

• Performance improvements for live queries (#6289)

• Pre-flight checks for Create Universe, Edit Universe, and Add Node operations for an on-premise provider (#6016)

• Disabling of the unused API endpoint run_in_shell (#6384)

• Disabling of the unused API endpoint create_db_credentials (#6385)

• Input validation for Kubernetes configuration path (#6389)

• Input validation for the access_keys API endpoint (#6386)

• Input path validation for backup target paths (#6382)

• Timeout support for busybox (#6652)

• Enabling of CORS policy by default (#6390)

• Deleting backups for TLS-enabled universes (#5980)

Core Database

• DDL consistency (#4710, #3979, #4360)

• DDL performance improvements (#5177, #3503)

• ALTER versions for ORM support (#4424)

• Online index backfill GA (#2301, #4899, #5031)

• Table partitioning (#1126, #5387)

• SQL support improvements:

  • Support for the FROM clause in UPDATE (#738)
  • Support for the USING clause in DELETE (#738)
  • SQL/JSON path language [PG12] (#5408)

• Performance improvements:

  • Sequence cache min-value flag (#6041, #5869)
  • Foreign-key batching (#2951)

• YSQL usability improvements:

  • COPY FROM rows-per-txn option (#2855, #6380)
  • COPY / ysql_dump OOM (#5205, #5453, #5603)
  • Large DML OOM (#5374)
  • Updating pkey values (#659)
  • Restarting write-txns on conflict (#4291)

• YSQL statement execution statistics (#5478)

• Improved raft leader stepdown operations, by waiting for the target follower to catch up. This reduces the unavailability window during cluster operations. (#5570)

• Improved performance during cluster overload:

  • Improved throttling for user-level YCQL RPC (#4973)
  • Lower YCQL live RPC memory usage (#5057)
  • Improved throttling for internal master level RPCs (#5434)

• Improved performance for YCQL and many connections:

  • Cache for generic YCQL system queries (#5043)
  • Cache for auth information for YCQL (#6010)
  • Speedup system.partitions queries (#6394)

• Improved DNS handling:

  • Process-wide cache for DNS queries (#5201)
  • Avoid duplicate DNS requests during YCQL system.partitions queries (#5225)

• Improved master-level load balancer (LB) operations:

  • Throttle tablet moves per table (#4053)
  • New global optimization for tablet balancing -- this also allows the LB to work with colocation (#3079)
  • New global optimization for leader balancing (#5021)
  • Delay LB on master leader failover (#5221)
  • Register temporary TS on master failover to prevent invalid LB operations (#4691)
  • Configurable option to disallow TLS v1.0 and v1.1 protocols (#6865)

• Skip loading deleted table metadata into master memory (#5122)

Bug Fixes

Yugabyte Platform

• Edit Universe did not display the TLS certificate used
• Multi-zone K8s universe creation failed (#5882)
• Tasks page reported an incorrect status of a failed backup (#6210)
• K8s universe upgrade did not wait for each pod to update before moving to the next (#6360)
• Changing node count by AZ was doing a full move (#5335)
• Enabling Encryption-at-Rest without KMS configuration caused create Universe to fail silently (#6228)
• Missing SST size in the Nodes page of a universe with read replicas (#6275)
• Incorrect link in the Live queries dashboard node name (#6458)
• Failure to create a TLS-enabled universe with a custom home directory setting with on-premise provider (#6602)
• Database node health liveliness check was blocked indefinitely (#6301)

Core Database

• Critical fixes for transaction cleanup applicable to aborted transactions (observed frequently as servers reaching soft memory limit)

• Main raft fixes:

  • Incorrect tracking of flushed op id in case of aborted operations (#4150)
  • Va...

v2.5.1.0 (Released Jan 14, 2020)

Yugabyte Release Notes v2.5.1 Jan 14, 2021

New features

Yugabyte Platform

[#5723] Support for custom CA signed certificates for encryption in-flight
[#5556] Add alerts capability for backup tasks

Core Database

[#6010] YCQL: Cache authentication information
[#4874] YSQL: Backup for colocated databases
[#4899] YSQL: Index backfill unique indexes
[#6237] YSQL: add CREATE INDEX NONCONCURRENTLY grammar
[#5982] YSQL: Add support for CREATE TABLE/INDEX ... WITH (table_oid = x)
[#4770] YSQL: Completing feature RANGE Partitioning
Improved Clock Skew Handling
[#6370] Bump default max_clock_skew to 500ms
[#3335] Crash when too big clock skew is detected

Improvements

Yugabyte Platform

[#5888] Better error for handling when user creates concurrent backup tasks
[#6016] Pre-flight checks for create universe / edit universe / add node operations
[#4183] Add error message for invalid ssh keys for onprem providers
[#5848] Onprem provider creation failure leaves incomplete state behind
[#5652] Improve delete universe handling for failed universes
[#6254] Allow user to input multiple GFlag entries at the same time
[#4843] New Health check for Replication status in 2dc setup
[#6268] Utility script to edit universe json from command line
[#4795] Embed http proxy within yb platform to avoid need for deploying nginx
[#5799] Delete platform code related to dev/debug package installs and EPEL repo
[#6228] Enabling Encryption-at-Rest without KMS config causes Create Universe to fail silently
[#6289] Fixed degraded performance on Live Queries tab caused by huge number of DOM nodes in rows
[#6383] Disabled unused API endpoint: run_query
[#6384] Disabled unused API endpoint: run_in_shell
[#6389] Input validation for kubernetes config path
[#6386] Added input validation for access_keys API endpoint
[#6382] Added UI validation for backup target paths
#6175: Use sudo when removing prometheus snapshots during platform backup
[#6602] Fix TLS directory while provisioning YB nodes.
[#6683] Use a timeout mechanism in cluster_health.py that is more compatible with docker's lack of zombie reaping
[#6633] Platform: rename Pivotal to VMware Tanzu, add Red Hat OpenShift cloud provider

Core Database

[#6580] ycql: Log the status when statement fails for system query cache
[#6608, #6609] YCQL: Consolidate authentication error handling and messages
[YCQL][#6374] Upgrade spark-cassandra-connector version to 2.4-yb-3
#5678: [YCQL] Optimize updating non-indexed columns within indexed tables.
[#3329][YSQL] Optimized updates to reflect changes on necessary indexes only
[#5805] [YSQL] Use slots more widely in tuple mapping code and make naming more consistent
[YSQL] Clean up libpq connection code (#6481)
[#6417] [YSQL] Backport 'Skip allocating hash table in EXPLAIN-only mode'
[#6131] [YSQL] Dowgrade permission check for data directory to warning
[YSQL] Bubble up backfill error message (#6292)
[#5805] [YSQL] Don't require return slots for nodes without projection
[#5805] [YSQL] Split ExecStoreTuple into ExecStoreHeapTuple and ExecStoreBufferHeapTuple
[#5805] [YSQL] Error position support for defaults and check constraints
Improvements to 2DC x-cluster async replication
[#6169] Correctly replay write batches with external intents on bootstrap
[#6169] New format for external intents
[#6283] Fix Very Large Metric Lag for 2DC Txns
[#3522] Fix Threading Issues with CDC Consumer Writes
[#6068] Replicate Intents and Apply Messages for 2DC Txns
[#6169] Fix apply order of updated external intents records
[#4516] 2DC: Initial support for colocated databases
Improvements to Tooling
[#6589] docdb: Add option to clear placement info in yb-admin.
[#6223] master UI make hash_split more readable
[#6161] Change modify_table_placement_info to wipe read_replicas and affinitized_leaders
[#5420] ybase: Enhance YMaster admin page to display under-replicated tablets
[#1325] ybase: API for displaying YB version information
Improvements to Tablet splitting
[#4942] docdb: tablet splitting: implemented retries to post-split involved tablets for
#5937: Fixed the case when one of the tablet replicas is down during the split.
#6101 Add the flag to limit number of tablets per table
[#6424] Fix post-split compaction to be async
Improvements to core product security
[#6568] Add flag to force client certificate verification in SSL
[#6266] Replace retry counter with check that data is ready
[#6266] Retry SSL_write on SSL_ERROR_WANT_WRITE
[#6266] Fix handling SSL_write error
[#6394, #6434] docdb: Speedup system.partitions queries
[#3979] Add Transaction Cleanup to Catalog Manager Create DDLs
[#1258]: Send election request from master during table creation
[#6114] docdb: Add metrics for master YCQL system table
[#1259] Speedup DROP TABLE
[#5752] Avoid starving threadpool with run election tasks from FailureDetector
[#5755] Faster cleanup of transactions that failed to commit due to a concurrent abort
(#5996) Messenger::ScheduleOnReactor should break loop finding reactor
[#6305] Adaptive Heartbeat Reporting
[#6445] docdb: Master should rebuild YCQL system.partitions on a background thread
[#6696] Small master perf tweaks

Bug Fixes

Yugabyte Platform

[#6300] Install of s3cmd fails for default GCP OS image when airgapped
[#6085] Fixed the issue where master is brought up in read replica cluster
[#6416] Deletion with flag --node_ip fails for onprem universes.
[#6614] Fixed NPE with full move
[#6252] Fixed NPE on the metrics page
[#5942] Fixed an issue where release instance is not an option for a node should the install fail because of ssh access
[#6275] Fixed missing stats on Nodes page when universe has read replicas
[#6257] Updating user profile when smtp username or password are empty

Core Database

[#6144] YCQL: Fix handling of tablet-lookup errors in Executor::FlushAsync
[#6570] YSQL: Use IsYBRelation instead of IsYugaByteEnabled
[#6492] YSQL Avoid memcpy() with a NULL source pointer and count == 0
[YSQL] Fix IsCreateTableDone for index backfill (#6234)
[#6318, #6334] Call InitThreading in YSQL webserver process
#6317: [YSQL] Fixed SIGSERV in YBPreloadRelCache
[#6364] [YSQL] Replace CurrentMemoryContext with GetCurrentMemoryContext for all pg_extensions
#6219] [YSQL] PRIMARY KEY index in TEMP TABLE is not checked for uniqueness
(#6284) [YSQL] Clear ALTERING when there's no alter
[#5805] [YSQL] Fix run-time partition pruning for appends with multiple source rels
[#6151] [YSQL] Handle rowmark in case of read with batch of ybctids
(#6270) [YSQL] Prevent concurrent backfill index
[#6133] [YSQL] Fix procedure with an INOUT parameter in DO block
[#6061] [YSQL] ysql_dump should consistently use quotes on column names
#6009: [backup][YSQL] Fixed incorrect column-ids in restored YSQL table if original table was altered.
#5954 [YSQL] Check transaction status after read finished
#6430 [YSQL] Refresh YBCache in case postgres clears its internal cache
#6468 [YSQL] Fix read restarts of request with paging state
[#6435] docdb: fixed handling of empty bloom filter key in the write path
[#6435] Fixed bloom filter index generation for range-partitioned tablets
[#6375] Check against TabletPeer returning null tablet pointer in
(#5641) Fix for --cert_node_filename for tservers and --enable_ysql
[#6318, #6334] Call InitThreading in YSQL webserver process
[#4150] Fix incorrect tracking of flushed/synced op id in case of Raft operation abort
(#6278) Master SEGV during LB due to null TSDescriptor
[#6353] Disable rocksdb flush on all DeleteTablet calls
[#6338] Fix crash with redis workloads and snapshot restore
[#6334] Attach/detach Squeasel threads to/from libcds using callbacks
[#6245] Fixed incorrect restored table schema if the table was altered after the backup.
[#6217] Avoid a possible assertion failure in CDSAttacher destructor in Webserver
[#6170] Shutdown status resolver before destroying it in ResolveIntents
[#6482] Fix timeout handling when getting safe time in a RF1 cluster
[#6635] Fix the wrong detection of communication failure when no operations transferred
[#6678] backup: Fix restore of colocated table with table_oid already set

Known Issues

Yugabyte Platform

Azure IaaS orchestration
No pricing information provided (5624)
No support for regions with zero Availability Zones(AZs) (5628)

Core Database

Advisory on clock-skew
After commit a60a4ae, by default, any YB node will explicitly crash if it detects a clock skew higher than the max_clock_skew flag (default 500ms). This can be disabled by setting fail_on_out_of_range_clock_skew=false, but this could lead to consistency issues! The recommendation is that you ensure clocks are synchronized across your cluster.

v2.2.6.0 (Released Dec 28, 2020)

Bug Fixes

Yugabyte Platform

• [#6602] Creating a TLS enabled universe with a custom home dir setting in on-prem provider fails

Core Database

• [#6394, #6434] [docdb]: Speedup system.partitions queries
• [#6435] [docdb]: fixed bloom filter index generation for range-partitioned tablets
• [#6435] [docdb]: fixed handling of empty bloom filter key in the write path
• [#6430] [YSQL] Refresh YBCache in case Postgres clears its internal cache

Known Issues

Yugabyte Platform

• [#6210] Tasks page reports an incorrect status of failed backup

2.5.0.0 (Released Nov 12, 2020)

New features

Yugabyte Platform

• Azure Cloud integration for Yugabyte Platform (in beta):
  Yugabyte Platform is natively integrated with Azure cloud to simplify deploying, monitoring, and managing YugabyteDB deployments. This feature automates a number of operations including orchestration of instances, secure deployments, online software upgrades, and scheduled backups, as well as monitoring and alerting. (6094, 6020)

• Yugabyte Platform operations now allow promoting a Yugabyte TServer only node to run YB-Master and YB-TServer process. (5831)

Core database

• Enhanced multi-region capabilities with geo-partitioning and follower reads
  The YugabyteDB 2.5 release adds row-level geo-partitioning capabilities as well as follower reads to the extensive set of multi-region features that YugabyteDB already had.

• Enterprise-grade security features:
  Authentication using the highly secure SCRAM-SHA-256 is now supported to limit security risks from brute force attacks and sniffing, including LDAP support for better user management and the ability to audit all database operations.

• Table-level partitions
  This allows users to split what is logically one large table into smaller sub-tables, using the following types of table partitioning schemes that PostgreSQL supports: range partitioning, list partitioning, and hash partitioning. Read more about table partitioning in YugabyteDB.

• Event triggers
  This is now supported in addition to regular table-level triggers in YSQL. While regular triggers are attached to a single table and capture only DML events, event triggers are global to a particular database and are capable of capturing DDL events. The event-based trigger framework enables detecting changes made to the data, and automating any subsequent tasks that need to be performed, which are useful in a number of use cases such as implementing a consolidated, central audit table (2379)

• Simplified cluster administration:

• Online rebuild of indexes
  This is supported for both the YSQL and YCQL APIs. This means that new indexes can be added to tables with pre-existing data while concurrent updates are happening on the cluster. The online index rebuild process creates the newly added index in the background, and transactionally enables the index once the rebuild of all the data is completed. This feature allows flexibility of adding indexes as the application needs evolve to keep queries efficient.

• Cluster execution statistics and running queries
  This can be analyzed in detail, allowing administrators to gain insights into how the database is performing. The pg_stat_statements extension, which enables tracking execution statistics of all SQL statements executed by the cluster, is supported and enabled by default. Support for pg_stat_activity has also been added, which shows information related to the activity performed by each connection. Yet another useful feature in this category is the ability to view all the live queries being executed by the cluster at any point in time.

• Detailed query plan and execution analysis
  This can now be performed with commands such as EXPLAIN and EXPLAIN ANALYZE. These commands display the execution plan generated by the planner for a given SQL statement. The execution plan shows details for any SQL statement such as how tables will be scanned (plain sequential scan, index scan), what join algorithms will be used to fetch required rows from the different tables, etc.

Improvements

Yugabyte Platform

• Enhancements to on-premises deployment workflows:
  • Do not fail universe creation if cronjobs can't be created for on-premises. (5939)
  • Remove pre-provision script requirement for airgapped installations. (5929)
  • "Sudo passwordless" in on-premises cloud provider configuration toggle is renamed.
  • Allow yugabyte user to belong to other user groups in Linux. (5943)
  • Added a new "Advanced" section in on-prem cloud provider configuration which includes:
    • Use hostnames
    • Desired home directory
    • Node exporter settings
  • Improvements to installation of Prometheus Node Exporter utility workflow. (5926)
    • The node exporter option is now available in the cloud configuration under advanced settings.
    • Supports bringing your own node exporter user.
• Make YEDIS API optional for new Universes and no change in behavior of existing universes. (5207)
• UI/UX improvements for YB Platform:
  • Add visual feedback when backup or restore is initiated from modal. (5908)
  • Minor fixes to primary cluster widget text issue. (5988)
  • Show pre-provision script in UI for non-passwordless sudo on-premises provider. (5550)
  • Update backup target and backup pages. (5917)
• For YugabyteDB universes with replication factor (RF) > 3, change the default min_num replicas for even distribution of AZs across all regions. (5426)
• Added functionality to create IPv6-enabled universe in Kubernetes. (5309, 5235)

Core database

• Support for SQL/JSON Path Language (jsonb_path_query). (5408)
• Remove range-loop-analysis and fallthrough errors in our code. (5812)
• Incorrect index update if used expression result is still unchanged. (5898)
• As part of the tablet splitting feature:
  • Implemented cleanup of the tablet for which all replicas have been split for. (4929)
  • Compaction improvements (5523)
• Improve performance for sequences by using higher cache value by default. Controlled by a YB-TServer configuration flag --ysql_sequence_cache_minval. (6041)
• Added compatibility mode in the yb_backup.py script for YugabyteDB version earlier than v2.1.4. (5810)
• Stability improvements to core database - make exponential backoff on lagging RAFT followers send NOOP. (5527)
• Added use of separate metrics objects for RegularDB and IntentsDB. (5640)
• Improvements to 2DC (xCluster replication) by adding TLS support for cleaning up cdc_state table. (5905)

Bug fixes

Yugabyte Platform

• Fix for Universe disk usage shows up empty on the universe page. (5548)
• Fix on on-premises backup failures due to file owned by the root user. (6062)
• Fix for a bug where user operation to perform a change to nodes count by AZ was doing a full move. (5335)
• Fixes for Yugabyte Platform data backup script for Replicated-based installations
• Fixes to Client Certificate start time to use UTC during download. (6118)
• Fixes for migration if no access keys exist yet. (6099)
• Fix to resolve issues caused by starting a YB-TServer node when another YB-Master node is down in the universe. (5739)
• Use the correct disk mount point while calculating disk usage of logs. (5983)

Core database

• Fix for bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. (5811)
• Improve fault tolerance of DDLs and reduce version mismatch errors in YSQL. (3979, 4360)
• Fixes to incorrect column-ids in the restored table if the original table was altered. (5958)
• Fixes timeout bug in Yugabyte Platform when there are read replicas. This fix will ignore read replica YB-TServers when running AreLeadersOnPreferredOnly. (6081)
• Fixes to restore of YSQL Backups after dropping and recreating a database. (5651)
• Fixes to a bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. This is fixed by setting the read time for backfill write requests to be the safe time chosen for backfill read requests rather than the hard-coded time 50. (5811)
• Fixes to delete backups for TLS-enabled universes. (5980)

Known Issues

Yugabyte Platform

...

v2.2.5.0 (Released Dec 2, 2020)

Yugabyte Release Notes v2.2.5 Dec 2, 2020

Bug Fixes

Yugabyte Platform

• Fix for updating user profile when SMTP username or password is empty (6257)
• Fix for editing YugabyteDB Universe placement info logic (6250)
• Fixes to delete unnecessary Prometheus snapshot when backing up YB Platform data (6175)
• Fix for an issue where Client Certificates are not valid on download due to incorrect validity time (6118)
• Safeguard extra migration for pre-provisioning (6181)

Core Database

• Fixes to the logic for SSL write to continue retrying, as long as SSL_write produces new data to send (6266)
• Fix TServer crash with YEDIS workloads and snapshot restore (6338)
• Disable Rocksdb flush on all DeleteTablet calls (6353)
• Shutdown status resolver before destroying it in ResolveIntents (6170)

Known Issues

Yugabyte Platform

• Tasks page reports an incorrect status of failed backup (6210)

Platform Support Package Improvements (Only applicable to Non-Replicated installations)

Yugabyte Platform

• It is required to pull the latest platform-support-packages.tar.gz binaries for the fix related to issue (6175) - “Fixes to delete unnecessary Prometheus snapshot when backing up YB Platform data”

v2.2.4.0 (Released Oct 30, 2020)

Yugabyte Release Notes v2.2.4 Oct 30, 2020

New features

Yugabyte Platform

Yugabyte Platform operations now allow promoting a Yugabyte TServer only node to run Yugabyte Master and TServer process (5831)

Improvements

Yugabyte Platform

Enhancements to on-prem Cloud Provider configuration
When creating onprem universes, YW will automatically run provisioning if airgap is enabled now, instead of asking the users to run the pre-provision script. Users will need to run the script only when creating universes without passwordless sudo access.
Do not fail universe creation if cronjobs can't be created for on-prem (5939)
Remove pre-provision script requirement for air-gapped installations (5929)
"Sudo passwordless" in on-prem cloud provider configuration toggle is renamed to “Manually Provision Node”
Added a new "Advanced" section in on-prem cloud provider configuration which includes
Use hostnames
Desired home directory
Node exporter settings
Pre-provisioning of ‘yugabyte’ user is supported for on-prem Cloud Provider. (YB devops script won't run create_user if yugabyte user already exists) (6086)
Improvements to installation of Prometheus Node Exporter utility workflow (5926)
Prometheus Node exporter option is now available in the cloud configuration under advanced settings
Supports bringing your own node exporter user
UI/UX improvements for YB Platform
Add visual feedback when backup or restore is initiated from modal (5908)
Add new button for downloading root cert in Certificates page (5521)
Use the correct disk mount while calculating the disk usage of logs (5983)
Core Database
YSQL Support ROWS_PER_TRANSACTION option for COPY FROM stdin (6069)
Bug Fixes
Yugabyte Platform

[Replicated] Fixed an issue that would leave Replicated in a bad state if the network connection drops while uploading an airgap bundle, preventing the installation from proceeding. (https://release-notes.replicated.com/release-notes/2.49.0/)
Fix for the check if changing node count by AZ affects master node placement and requires a full move (5335)
Fix delete backup failure for TLS enabled universes (5980)
Universe disk usage shows up empty on the universe page (5548)
Fix for starting a YB TServer Node when another YB Master is down causes issues (5739)
Fix platform backup script for replicated installations (6130)

Core Database
[docdb] Fix a rare bug on restore which could clear more metadata than necessary (5788)
[docdb] Fix a rare race condition between transactions and snapshots that could lead to a deadlock (6170)

[YSQL] Fix an issue with setting bind address for YSQL metrics endpoint for IPv6 scenarios (4901)
[YCQL] Fix for bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index column(s) (5811)
[YCQL] Fix a bug with update and expression indexes, where an index row would get deleted if the column is updated without changing the indexed value (expression) (5898)
[YSQL] Fix OOM with \copy or COPY FROM stdin for large files (5603)
Known Issues
Yugabyte Platform
Client Certificates are not valid on download due to incorrect validity time (6118)
Tasks page reports incorrect status of failed backup (6210)

Platform Support Package Improvements
(Only applicable to Non-Replicated installations)
Yugabyte Platform
Fix for On-prem YB Platform backup fails due to file owned by root (6062)

v2.3.3.0 (Released October 26, 2020)

New features

Yugabyte Platform

• Introducing beta support for Azure IaaS orchestration in Yugabyte Platform cloud provider configuration. (6094, 6020)
• Yugabyte Platform operations now allow promoting a Yugabyte TServer only node to run YB-Master and YB-TServer process. (5831)

Core database

• [YSQL] Introducing support for Event Triggers that are global to a particular database and are capable of capturing DDL events. (2379)

Improvements

Yugabyte Platform

• Enhancements to on-premises deployment workflows:
  • Do not fail universe creation if cronjobs can't be created for on-premises. (5939)
  • Remove pre-provision script requirement for airgapped installations. (5929)
  • "Sudo passwordless" in on-premises cloud provider configuration toggle is renamed.
  • Allow yugabyte user to belong to other user groups in Linux. (5943)
  • Added a new "Advanced" section in on-prem cloud provider configuration which includes:
    • Use hostnames
    • Desired home directory
    • Node exporter settings
  • Improvements to installation of Prometheus Node Exporter utility workflow. (5926)
    • The node exporter option is now available in the cloud configuration under advanced settings.
    • Supports bringing your own node exporter user.
• Make YEDIS API optional for new Universes and no change in behavior of existing universes. (5207)
• UI/UX improvements for YB Platform:
  • Add visual feedback when backup or restore is initiated from modal. (5908)
  • Minor fixes to primary cluster widget text issue. (5988)
  • Show pre-provision script in UI for non-passwordless sudo on-premises provider. (5550)
  • Update backup target and backup pages. (5917)
• For YugabyteDB universes with replication factor (RF) > 3, change the default min_num replicas for even distribution of AZs across all regions. (5426)
• Added functionality to create IPv6-enabled universe in Kubernetes. (5309, 5235)

Core database

• Support for SQL/JSON Path Language (jsonb_path_query). (5408)
• Remove range-loop-analysis and fallthrough errors in our code. (5812)
• Incorrect index update if used expression result is still unchanged. (5898)
• Various improvements to RaftConsensus tests. (4548, 5912)
• As part of the tablet splitting feature:
  • Implemented cleanup of the tablet for which all replicas have been split for. (4929)
  • Compaction improvements (5523)
• Improve performance for sequences by using higher cache value by default. Controlled by a YB-TServer configuration flag --ysql_sequence_cache_minval. (6041)
• Added compatibility mode in the yb_backup.py script for YugabyteDB version earlier than v2.1.4. (5810)
• Stability improvements to core database - make exponential backoff on lagging RAFT followers send NOOP. (5527)
• Added use of separate metrics objects for RegularDB and IntentsDB. (5640)
• Improvements to 2DC (xCluster replication) by adding TLS support for cleaning up cdc_state table. (5905)

Bug fixes

Yugabyte Platform

• Fix for Universe disk usage shows up empty on the universe page. (5548)
• Fix on on-premises backup failures due to file owned by the root user. (6062)
• Fix for a bug where user operation to perform a change to nodes count by AZ was doing a full move. (5335)
• Fixes for Yugabyte Platform data backup script for Replicated-based installations
• Fixes to Client Certificate start time to use UTC during download. (6118)
• Fixes for migration if no access keys exist yet. (6099)
• Fix to resolve issues caused by starting a YB-TServer node when another YB-Master node is down in the universe. (5739)
• Use the correct disk mount point while calculating disk usage of logs. (5983)
• Fixes to delete backups for TLS-enabled universes. (5980)

Core database

• Fix for bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. (5811)
• Improve fault tolerance of DDLs and reduce version mismatch errors in YSQL. (3979, 4360)
• Fixes to incorrect column-ids in the restored table if the original table was altered. (5958)
• Fixes timeout bug in Yugabyte Platform when there are read replicas. This fix will ignore read replica YB-TServers when running AreLeadersOnPreferredOnly. (6081)
• Fixes to restore of YSQL Backups after dropping and recreating a database. (5651)
• Fixes to a bug with the duplicate row detection that allows a unique index to get created when the table is not unique on the index columns. This is fixed by setting the read time for backfill write requests to be the safe time chosen for backfill read requests rather than the hard-coded time 50. (5811)

Known Issues

Yugabyte Platform

• Azure IaaS orchestration:
  • No pricing information provided. (5624)
  • No support for regions with zero availability zones (AZs). (5628)

v2.2.3.0 (Released September 30, 2020)

New features

Yugabyte Platform

• Enhancements to RBAC with a new “Backup Admin” role which allows all read-only permissions plus back up operations #5311
• Display table sizes in the Tables page #3581
• On-prem Cloud provider instances table list can be sorted and filtered #4757
• API for disabling auto-generation of database nodes certificate #5545
• Backup retention policy for auto-deletion of older backups #4493

Core database

• [YCQL] Introducing Audit Logging feature for YCQL operations #5887
• [YSQL] Support for ALTER COLUMN type that does not require on-disk changes #4424
• [YSQL] Support for Transactional batch size in COPY FROM command syntax #2855, #5453
• [YCQL] Finer grained permission control for TRUNCATE #5443
• [YSQL] Enable 2DC replication bootstrap for YSQL Tables #5601

Improvements

Yugabyte Platform

• Backup improvements
  • YCQL, YSQL, and YEDIS backups are split into different tabs on the backup UI screen
  • Back up operation creates a single restore object regardless of “transactional” field selection
  • Added a new field parallel thread for multi-threading backups
  • YSQL backups only support namespace level backup
  • Backup table list has been revamped with the new UX and shows more details - expiration time, duration, backup ty
  • Universe can be restored with a single action
  • Backup objects can be manually deleted
  • Scheduled backups cron expression shows the next run time in local browser time and the label displays that cron supports UTC only #4709
• Improve the Replication graph for xDC replication set up to display the graph and metric names cleanly #5429
• UI improvements for displaying On-prem instances
  • Instance ID is now optional and also shown in the nodes page #4760
  • “In Use” column has been renamed to “Universe Name”
  • Supports adding multiple instances in different rows instead of comma-separated entries
• Supports generating API tokens for SSO-enabled users
• Yugabyte Platform restart cancels any tasks that were in progress
• Remove sudo requirement in DB backup script #5440

Core database

• [DocDB] Removal of unreachable (dead) nodes from YugabyteDB UI #4759
• [DocDB] Improvements with selecting better default flags for Index backfill #5494
• [DocDB] Lower the leader load balancer parallelism for performance improvements #5461

Bug fixes

Yugabyte Platform

• [Platform] Security hardening: Added CSRF token support
• [Platform] Support for PingFederate OIDC SSO login for Non Replicated deployments
• [Platform] Allow on-prem nodes to be reused after releasing them from a universe #5703

Core Database

• [YCQL] Fix for CQL Index scans when ORDER BY columns exist in the table but not in chosen index #5690
• [YSQL] Avoids Pushdown down of UPDATE with RETURNING clause #5366
• [YSQL] Fix for UPDATE operation with partial and expression Indexes #4939
• [YSQL] OOM issue fix in COPY FROM query #2855
• [YSQL] OOM issue fix in COPY TO query #5205
• [DocDB] Crash fix for DNS-enabled Yugabyte Universes #5561
• [DocDB] Fix for in-memory state not getting updated correctly when using read-from-follower query semantics #1052
• [DocDB] For Yugabyte Universes using cross-cluster async replication (2DC), added RPC throttling on idle CDC tables to avoid wasting CPU #5472
• [DocDB] Fix for rare deadlock scenario when the node switches from Leader to Follower state of the tablet and there are pending operations on Leader side #5741
• [DocDB] Fix to ensure all Yugabyte Tablet Servers are accounted for when Master fails over #5501

Known Issues

Yugabyte Platform

• Universe disk usage shows up empty on the universe page #5548

Platform Support Package Improvements (Only applicable to Non-Replicated installations)

Yugabyte Platform

• Platform backup script needs to preserve attributes on copy from Prometheus snapshot #5612
• Platform backup fails if /opt/yugabyte/release directory does not exist #5615
• Platform backup script needs to stop Prometheus service during the restore of Prometheus data #5685

Note:

Prior to v2.0, YSQL was still in beta. Upon the release of v2.0, a backward-incompatible file format change was made for YSQL. For existing clusters running pre-2.0 release with YSQL enabled, you cannot upgrade to v2.0 or later. Instead, export your data from existing clusters and then import the data into a new cluster (v2.0 or later).

v2.3.2.0 (Released September 29, 2020)

YSQL

• Enable pg_stat_statements extension by default. #5750
• Clean up unused PgStatement objects on hot path after usage to avoid memory consumption. #5374
• Fix restarting writes (INSERT and UPDATE) on transaction conflicts and avoid surfacing transaction errors to user applications. #4291
• Enable batching of updates for YSQL procedures and functions. #5257

YCQL

• Add additional system queries (SELECT_LOCAL, SELECT_LOCAL_TOKENS, and SELECT_PEERS_TOKENS) for caching. #5753
• Fix crash for YCQL index scan when ORDER BY columns exist in table, but not in the chosen index, and raise an error. #5690

Core database

• Allow setting a custom replacement policy per table through yb_admin. #5368
• Do not clean up transaction that potentially was committed. #5789 #5783 #5755
• Fix CDC shutdown races. Moved thread pool shutdown before other CDC objects cleanup. Added explicit CDC poller clear in CDC consumer shutdown. #5583
• Fix preparer deadlock after switching from leader to follower. #5741
• Fix RaftGroupMetadata construction issues found using PVS Studio. #5578
• Change rlimit flags to have at-least semantics. Only change the process soft limit if it is smaller than the limit specified by the corresponding flag value. #5715
• Add extra logging when initializing syscatalog. #5708
• Set default webserver_interface to rpc_bind_address IP/host component. #5320
• Add environmental variables functionality to yugabyted. New env variables for YSQL (YSQL_USER, YSQL_PASSWORD, and YSQL_DB) and YCQL(YCQL_USER, YCQL_PASSWORD, and YCQL_KEYSPACE). (#5499
• Improve cleanup of aborted transactions. #5669
• Fix dumping an empty batch by updating MaxReplicateOpId. #5679
• Set default staleness for read from followers (using max_stale_read_time_ms) to 60 seconds. Previously, a follower could return requested data no matter how state the data is. #5590
• Calculate the placement UUID from Raft upon tserver registration. #5558
• Enable ignoring deleted tablets. #5122
• Remove unreachable nodes from UI (/tablet-servers page) after nodes are removed from the cluster. #4759

Yugabyte Platform

• Fix issue parsing multiple comma-separated addresses for rpc_bind_addresses. #5634
• Add functionality to delete old backups after a specified time period. In the Create Backup dialog, added Number of Days to Retain Backup field. #4493
• Combine the Platform and community Grafana dashboards. Grafana dashboard shows the node status and other panels correctly. #5707
• Sum table sizes across all nodes of a universe. #5745
• Fixes releasing OnPrem nodes should not delete them from the universe metadata. Fixes releasing a node should only attempt to shut down processes on that node. Nodes can be re-added using Node > Actions > Add flow. #5703 #5791 #5792
• Add additional property to MetricsPanel to handle width of Replication graph and increase height. Aggregates replication metrics values from all tservers into an average and max value graph across all tservers. #5429 #5613 #5632
• Update yb-client JAR version to 0.8.1. #5759
• Stop and restart relevant services during Yugabyte Platform backup. Backups stop and restart services. Restores stop and restart Yugabyte Platform and Prometheus, if necessary. #5685
• Authenticate API tokens even when using OAuth. This allows curl requests. #5587
• Fix Metrics tab and columns misalignment on table page. #5582
• Rename In Use column to Universe Name and show NOT USED if the node is not in use. #5713
• Do not generate node certificates for custom certificates. #5545
• Make instance name field optional for on-premises provider. #5234
• Fix button for Metrics tab to reset active and focus state when key is unpressed. #5620
• Add Docker- and VM-aware options to the backup script and will include Prometheus snapshots. #5120 #5615 #5612 #5614

Note:
Prior to v2.0, YSQL was still in beta. Upon release of v2.0, a backward-incompatible file format change was made for YSQL. For existing clusters running pre-2.0 release with YSQL enabled, you cannot upgrade to v2.0 or later. Instead, export your data from existing clusters and then import the data into a new cluster (v2.0 or later).

v2.3.1.0 (Released September 15, 2020)

Features and changes

YSQL

• Fixes OOM when using COPY <table> FROM <file> to load data from a large file to a table. #5453
• Enable 2DC replication bootstrap for YSQL tables. This allows the producer to know where to start replicating from. #5601
• Fix CREATE TABLE is 4-5x slower using Docker on Mac than not using Docker. Speeds up table creation by buffering writes to postgres system tables, caching pinned objects, and significantly reducing write RPC calls. #3503
• Roll back the catalog version made in commit 46f3701 so that 2.3 upgrades can proceed. #5408

Core database

• Quickly evict known unresponsive tablet servers from the tablet location cache. Applies only to follower reads. For example, when tablet servers are not replying to RPC calls or dead — not sending heartbeats to the master for 5 minutes. This could also happen after decommissioning nodes. #1052

Yugabyte Platform

• Add search input and data sorting to the on-premises instances table list. Click arrows next to column titles to sort. Use the Search form to search multiple columns. #4757
• Add the ability to change the user role (Admin, ReadOnly, or BackupAdmin) from the UI by an admin. Also, fix stale users list after creation or deletion of a user and disable Save buttons at Customer Profile tabs for ReadOnly users. #5311
• Before deleting a universe, stop the master/tserver processes and release the instance so that if started again, the processes are not still running. #4953

Note:

Prior to version 2.0, YSQL was still in beta. As a result, the 2.0 release included a backward-incompatible file format change for YSQL. If you have an existing cluster running releases earlier than 2.0 with YSQL enabled, then you will not be able to upgrade to version 2.0+. Export from your existing cluster and then import into a new cluster (v2.0 or later) to use existing data.