28 Jul 13:50

mattlord

72d943e

Vitess v15.0.4

Release of Vitess v15.0.4

The entire changelog for this release can be found here.

The release includes 33 merged Pull Requests.

Thanks to all our contributors: @GuptaManan100, @app/vitess-bot, @frouioui, @harshit-gangal, @shlomi-noach, @systay

Contributors

systay, shlomi-noach, and 3 other contributors

Assets 5

27 Jun 13:10

mattlord

v17.0.0

70a9466

Vitess v17.0.0

Release of Vitess v17.0.0

Summary

Major Changes

We added options to the TabletPicker that allow for specifying a cell preference in addition to making the default behavior to give priority to the local cell and any alias it belongs to. We are also introducing a new way to select tablet type preference which should eventually replace the in_order: hint currently used as a prefix for tablet types. The signature for creating a new TabletPicker now looks like:

func NewTabletPicker(
	ctx context.Context,
	ts *topo.Server,
	cells []string,
	localCell, keyspace, shard, tabletTypesStr string,
	options TabletPickerOptions,
) (*TabletPicker, error) {...}

Where ctx, localCell, option are all new parameters.

option is of type TabletPickerOptions and includes two fields, CellPreference and TabletOrder.

CellPreference: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client
TabletOrder: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client

See PR 12282 Description for examples on how this changes cell picking behavior.

Default TLS version changed for `vtgr`

When using TLS with vtgr, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.

vtgr is now deprecated as part of v17.0.0, please see the deprecation notice.

Dedicated stats for VTGate Prepare operations

Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (Execute). In v17 Execute and Prepare operations generate stats under independent stats keys.

Here is a (condensed) example of stats output:

{
  "VtgateApi": {
    "Histograms": {
      "Execute.src.primary": {
        "500000": 5
      },
      "Prepare.src.primary": {
        "100000000": 0
      }
    }
  },
  "VtgateApiErrorCounts": {
    "Execute.src.primary.INVALID_ARGUMENT": 3,
    "Execute.src.primary.ALREADY_EXISTS": 1
  }
}

VTAdmin web migrated to vite

Previously, VTAdmin web used the Create React App framework to test, build, and serve the application. In v17, Create React App has been removed, and Vite is used in its place. Some of the main changes include:

Vite uses VITE_* environment variables instead of REACT_APP_* environment variables
Vite uses import.meta.env in place of process.env
Vitest is used in place of Jest for testing
Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
public/index.html has been moved to root directory in web/vtadmin

Keyspace name validation in TopoServer

Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetKeyspace and CreateKeyspace methods return an error if given an invalid name.

Shard name validation in TopoServer

Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetShard and CreateShard methods return an error if given an invalid name.

Compression CLI flags remove from vtctld and vtctldclient binaries

The CLI flags below were mistakenly added to vtctld and vtctldclient in v15. In v17, they are no longer present in those binaries.

--compression-engine-name
--compression-level
--external-compressor
--external-compressor-extension
--external-decompressor

VtctldClient command RestoreFromBackup will now use the correct context

The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.

VTTablet Restore Metrics

As part of the VTTablet Sidecar Schema Maintenance Refactor in v16.0.0, we dropped the local_metadata table from the sidecar database schema. This table was storing a couple of metrics related to restores from backup.
They have now been re-introduced as metrics that can be accessed from /debug/vars.

Vttablet's transaction throttler now also throttles DML outside of `BEGIN; ...; COMMIT;` blocks

Prior to v17, vttablet's transaction throttler (enabled with --enable-tx-throttler) would only throttle requests done inside an explicit transaction, i.e., a BEGIN; ...; COMMIT; block.
In v17 PR#13040, this behavior was being changed so that it also throttles work outside of explicit transactions for INSERT/UPDATE/DELETE/LOAD queries.

New command line flags and behavior

Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size

Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.

In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size.

--builtinbackup-file-read-buffer-size: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.
--builtinbackup-file-write-buffer-size: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)

These flags are applicable to the following programs:

vtbackup
vtctld
vttablet
vttestserver

Manifest backup external decompressor command

Add a new builtin/xtrabackup flag --manifest-external-decompressor. When set the value of that flag is stored in the manifest field ExternalDecompressor. This manifest field may be consulted when decompressing a backup that was c...

Contributors

arthurschreiber, kovyrin, and 46 other contributors

Assets 5

13 Jun 13:37

mattlord

v17.0.0-rc2

4de9709

Vitess v17.0.0-rc2 Pre-release

Pre-release

Release of Vitess v17.0.0-rc2

Summary

Major Changes

Major Changes

Breaking Changes

Default Local Cell Preference for TabletPicker

func NewTabletPicker(
	ctx context.Context,
	ts *topo.Server,
	cells []string,
	localCell, keyspace, shard, tabletTypesStr string,
	options TabletPickerOptions,
) (*TabletPicker, error) {...}

Where ctx, localCell, option are all new parameters.

option is of type TabletPickerOptions and includes two fields, CellPreference and TabletOrder.
CellPreference: "PreferLocalWithAlias" (default) gives preference to vtgate's local cell, or "OnlySpecified" which only picks from the cells explicitly passed in by the client TabletOrder`: "Any" (default) for no ordering or random, or "InOrder" to use the order specified by the client

See PR 12282 Description for examples on how this changes cell picking behavior.

Default TLS version changed for `vtgr`

When using TLS with vtgr, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.

Dedicated stats for VTGate Prepare operations

Prior to v17 Vitess incorrectly combined stats for VTGate Execute and Prepare operations under a single stats key (Execute). In v17 Execute and Prepare operations generate stats under independent stats keys.

Here is a (condensed) example of stats output:

{
  "VtgateApi": {
    "Histograms": {
      "Execute.src.primary": {
        "500000": 5
      },
      "Prepare.src.primary": {
        "100000000": 0
      }
    }
  },
  "VtgateApiErrorCounts": {
    "Execute.src.primary.INVALID_ARGUMENT": 3,
    "Execute.src.primary.ALREADY_EXISTS": 1
  }
}

VTAdmin web migrated to vite

Vite uses VITE_* environment variables instead of REACT_APP_* environment variables
Vite uses import.meta.env in place of process.env
Vitest is used in place of Jest for testing
Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
public/index.html has been moved to root directory in web/vtadmin

Keyspace name validation in TopoServer

Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetKeyspace and CreateKeyspace methods return an error if given an invalid name.

Shard name validation in TopoServer

Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetShard and CreateShard methods return an error if given an invalid name.

Compression CLI flags remove from vtctld and vtctldclient binaries

The CLI flags below were mistakenly added to vtctld and vtctldclient in v15. In v17, they are no longer present in those binaries.

--compression-engine-name
--compression-level
--external-compressor
--external-compressor-extension
--external-decompressor

VtctldClient command RestoreFromBackup will now use the correct context

The VtctldClient command RestoreFromBackup initiates an asynchronous process on the specified tablet to restore data from either the latest backup or the closest one before the specified backup-timestamp.
Prior to v17, this asynchronous process could run indefinitely in the background since it was called using the background context. In v17 PR#12830,
this behavior was changed to use a context with a timeout of action_timeout. If you are using VtctldClient to initiate a restore, make sure you provide an appropriate value for action_timeout to give enough
time for the restore process to complete. Otherwise, the restore will throw an error if the context expires before it completes.

Vttablet's transaction throttler now also throttles DML outside of `BEGIN; ...; COMMIT;` blocks

New command line flags and behavior

Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size

Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.

In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size.

--builtinbackup-file-read-buffer-size: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.
--builtinbackup-file-write-buffer-size: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)

These flags are applicable to the following programs:

vtbackup
vtctld
vttablet
vttestserver

Manifest backup external decompressor command

This feature enables the following flow:

Take a backup using an external compressor

 Backup --compression-engine=external \
        --external-compressor=zstd \
        --manifest-external-decompressor="zstd -d"

Restore that backup with a mere Restore command, without having to specify --external-decompressor.

vttablet --throttler-config-via-topo

This flag was introduced in v16 and defaulted to false. In v17 it defaults to true, and there is no need to supply it.

Note that this flag overrides --enable-lag-throttler and --throttle-threshold, ...

Contributors

arthurschreiber, kovyrin, and 44 other contributors

Assets 5

06 Jun 14:36

mattlord

v17.0.0-rc1

d0caade

Vitess v17.0.0-rc1 Pre-release

Pre-release

Release of Vitess v17.0.0-rc1

Summary

Major Changes

Major Changes

Breaking Changes

Default Local Cell Preference for TabletPicker

func NewTabletPicker(
	ctx context.Context,
	ts *topo.Server,
	cells []string,
	localCell, keyspace, shard, tabletTypesStr string,
	options TabletPickerOptions,
) (*TabletPicker, error) {...}

Where ctx, localCell, option are all new parameters.

See PR 12282 Description for examples on how this changes cell picking behavior.

Default TLS version changed for `vtgr`

When using TLS with vtgr, we now default to TLS 1.2 if no other explicit version is configured. Configuration flags are provided to explicitly configure the minimum TLS version to be used.

Dedicated stats for VTGate Prepare operations

Here is a (condensed) example of stats output:

{
  "VtgateApi": {
    "Histograms": {
      "Execute.src.primary": {
        "500000": 5
      },
      "Prepare.src.primary": {
        "100000000": 0
      }
    }
  },
  "VtgateApiErrorCounts": {
    "Execute.src.primary.INVALID_ARGUMENT": 3,
    "Execute.src.primary.ALREADY_EXISTS": 1
  }
}

VTAdmin web migrated to vite

Vite uses VITE_* environment variables instead of REACT_APP_* environment variables
Vite uses import.meta.env in place of process.env
Vitest is used in place of Jest for testing
Our protobufjs generator now produces an es6 module instead of commonjs to better work with Vite's defaults
public/index.html has been moved to root directory in web/vtadmin

Keyspace name validation in TopoServer

Prior to v17, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetKeyspace and CreateKeyspace methods return an error if given an invalid name.

Shard name validation in TopoServer

Prior to v17, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names are restricted to using only ASCII characters, digits and _ and -. TopoServer's GetShard and CreateShard methods return an error if given an invalid name.

Compression CLI flags remove from vtctld and vtctldclient binaries

The CLI flags below were mistakenly added to vtctld and vtctldclient in v15. In v17, they are no longer present in those binaries.

--compression-engine-name
--compression-level
--external-compressor
--external-compressor-extension
--external-decompressor

VtctldClient command RestoreFromBackup will now use the correct context

Vttablet's transaction throttler now also throttles DML outside of `BEGIN; ...; COMMIT;` blocks

New command line flags and behavior

Backup --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size

Prior to v17 the builtin Backup Engine does not use read buffering for restores, and for backups uses a hardcoded write buffer size of 2097152 bytes.

In v17 these defaults may be tuned with, respectively --builtinbackup-file-read-buffer-size and --builtinbackup-file-write-buffer-size.

--builtinbackup-file-read-buffer-size: read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.
--builtinbackup-file-write-buffer-size: write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)

These flags are applicable to the following programs:

vtbackup
vtctld
vttablet
vttestserver

Manifest backup external decompressor command

This feature enables the following flow:

Take a backup using an external compressor

 Backup --compression-engine=external \
        --external-compressor=zstd \
        --manifest-external-decompressor="zstd -d"

Restore that backup with a mere Restore command, without having to specify --external-decompressor.

vttablet --throttler-config-via-topo

This flag was introduced in v16 and defaulted to false. In v17 it defaults to true, and there is no need to supply it.

Note that this flag overrides --enable-lag-throttler and --throttle-threshold, which now give warnings, and will be removed in v18.

New stats

####...

Contributors

arthurschreiber, kovyrin, and 44 other contributors

Assets 5

09 May 15:15

GuptaManan100

v16.0.2

6076fed

Vitess v16.0.2

Release of Vitess v16.0.2

Known Issues

Schema-initialization stuck on semi-sync ACKs while upgrading to `v16.0.2`

During upgrades from <= v15.x.x to v16.0.2, as part of PromoteReplica call, the schema-init realizes that there are schema diffs to apply and ends up writing to the database if semi-sync is enabled, all of these writes get blocked indefinitely.
Eventually, PromoteReplica fails, and this fails the entire PRS call.

A fix for this issue was merged on release-16.0 in PR#13441, read the corresponding bug report to learn more.

This issue is fixed in v16.0.3 and later patch releases.

Broken downgrade from v17.x.x when super_read_only turned on by default

In v17.x.x super_read_only is turned on by default meaning that downgrading from v17 to v16.0.2 breaks due to init_db.sql needing write access.

This issue is fixed in >= v16.0.3 thanks to PR #13525

Major Changes

Upgrade to `go1.20.3`

Vitess v16.0.2 now runs on go1.20.3.
Below is a summary of this Go patch release. You can learn more here.

go1.20.3 (released 2023-04-04) includes security fixes to the go/parser, html/template, mime/multipart, net/http, and net/textproto packages, as well as bug fixes to the compiler, the linker, the runtime, and the time package. See the Go 1.20.3 milestone on our issue tracker for details.

EffectiveCallerId in Vtgate gRPC calls

A new flag grpc-use-static-authentication-callerid is added to gate the behavior introduced in #12050.
Earlier, we used to automatically set immediateCallerID to user from static authentication context that overrode the EffectiveCallerId.

Shard name validation in TopoServer

Prior to v16.0.2, it was possible to create a shard name with invalid characters, which would then be inaccessible to various cluster management operations.

Shard names may no longer contain the forward slash ("/") character, and TopoServer's CreateShard method returns an error if given such a name.

The entire changelog for this release can be found here.

The release includes 24 commits (excluding merges)

Thanks to all our contributors: @GuptaManan100, @ajm188, @frouioui, @github-actions[bot], @harshit-gangal, @mattlord, @systay, @vitess-bot[bot]

Contributors

systay, mattlord, and 5 other contributors

Assets 5

30 Mar 16:21

GuptaManan100

v16.0.1

d1ba625

Vitess v16.0.1

Release of Vitess v16.0.1

Known Issues

Schema-initialization stuck on semi-sync ACKs while upgrading to `v16.0.1`

During upgrades from <= v15.x.x to v16.0.1, as part of PromoteReplica call, the schema-init realizes that there are schema diffs to apply and ends up writing to the database if semi-sync is enabled, all of these writes get blocked indefinitely.
Eventually, PromoteReplica fails, and this fails the entire PRS call.

A fix for this issue was merged on release-16.0 in PR#13441, read the corresponding bug report to learn more.

This issue is fixed in v16.0.3 and later patch releases.

Broken downgrade from v17.x.x when super_read_only turned on by default

In v17.x.x super_read_only is turned on by default meaning that downgrading from v17 to v16.0.1 breaks due to init_db.sql needing write access.

This issue is fixed in >= v16.0.3 thanks to PR #13525

Major Changes

Upgrade to `go1.20.2`

Vitess v16.0.1 now runs on go1.20.2.
Below is a summary of this Go patch release. You can learn more here.

go1.20.2 (released 2023-03-07) includes a security fix to the crypto/elliptic package, as well as bug fixes to the compiler, the covdata command, the linker, the runtime, and the crypto/ecdh, crypto/rsa, crypto/x509, os, and syscall packages.

Keyspace name validation in TopoServer

Prior to v16.0.1, it was possible to create a keyspace with invalid characters, which would then be inaccessible to various cluster management operations.

Keyspace names may no longer contain the forward slash ("/") character, and TopoServer's GetKeyspace and CreateKeyspace methods return an error if given such a name.

The entire changelog for this release can be found here.

The release includes 39 commits (excluding merges)

Thanks to all our contributors: @GuptaManan100, @ajm188, @frouioui, @github-actions[bot], @mattlord, @rohit-nayak-ps, @rsajwani, @shlomi-noach, @systay, @vitess-bot[bot]

Contributors

systay, mattlord, and 7 other contributors

Assets 5

30 Mar 16:34

mattlord

v15.0.3

652b0da

Vitess v15.0.3

Release of Vitess v15.0.3

The entire changelog for this release can be found here.

The release includes 52 commits (excluding merges)

Thanks to all our contributors: @GuptaManan100, @ajm188, @dbussink, @deepthi, @frouioui, @harshit-gangal, @mattlord, @rsajwani, @shlomi-noach, @systay, @vitess-bot[bot], @vmg

Contributors

dbussink, vmg, and 9 other contributors

Assets 5

30 Mar 16:21

frouioui

v14.0.5

3e5671f

Vitess v14.0.5

Release of Vitess v14.0.5

The entire changelog for this release can be found here.

The release includes 33 commits (excluding merges)

Thanks to all our contributors: @GuptaManan100, @dbussink, @frouioui, @harshit-gangal, @rsajwani, @shlomi-noach, @systay, @vitess-bot[bot], @vmg

Contributors

dbussink, vmg, and 6 other contributors

Assets 5

28 Feb 15:15

GuptaManan100

v16.0.0

bb768df

Vitess v16.0.0

Release of Vitess v16.0.0

Summary

Known Issues
Major Changes
Minor Changes
- Backup compression benchmarks
Refactor
- VTTablet sidecar schema maintenance refactor

Known Issues

MySQL & Xtrabackup known issue

There is a known issue with MySQL's INSTANT DDL combined with Percona XtraBackup, that affects users of Vitess 16.0.
The problem is described in https://docs.percona.com/percona-xtrabackup/8.0/em/instant.html, and the immediate impact is you may not be able to backup your database using XtraBackup under certain conditions.

As of MySQL 8.0.12, the default ALGORITHM for InnoDB's ALTER TABLE is INSTANT. In 8.0.12 only a small number of operations were eligible for INSTANT, but MySQL 8.0.29 added support for more common cases.
Unfortunately, the changes in 8.0.29 affect XtraBackup as follows: if you ALTER TABLE in MySQL 8.0.29, and that ALTER is eligible for INSTANT DDL (e.g. add new table column), then as of that moment, XtraBackup is unable to backup that table, hence your entire database.

It is important to note that even if you then upgrade your MySQL server to, e.g. 8.0.32, the table still cannot be backed up.

Versions where XtraBackup is unable to backup such tables: MySQL 8.0.29 - 8.0.31. This does not apply to Percona Server flavor.

The issue is resolved with Percona XtraBackup 8.0.32 combined with MySQL 8.0.32.

You might be affected if:

You're using MySQL 8.0.29 - 8.0.31 and are using XtraBackup to backup your database
and, you have issued an ALTER TABLE, either directly, or using Online DDL in vitess v16.0 and below

A futures Vitess patch release v16.0.1 will address the issue via Online DDL migrations.

Mitigations

Use Percona XtraBackup 8.0.32 combined with MySQL 8.0.32. To go with this option, you can use the docker image vitess/lite:v16.0.0-mysql-8.0.32.
or, Use a Percona Server flavor
or, always ensure to add ALGORITHM=INPLACE or ALGORITHM=COPY to your ALTER TABLE statements

Workarounds

If you have already been affected, these are the options to be able to backup your database:

Use builtin backups, see https://vitess.io/docs/15.0/user-guides/operating-vitess/backup-and-restore/creating-a-backup/. builting backups are not based on XtraBackup.
Upgrade to MySQL 8.0.32 or above and to Xtrabackup 8.0.32, or switch to Percona Server. To go with this option, you can use the docker image vitess/lite:v16.0.0-mysql-8.0.32. Then rebuild the table directly via:
- OPTIMIZE TABLE your_table
- or, ALTER TABLE your_table ENGINE=INNOB
Upgrade to Vitess patch release v16.0.1, upgrade to MySQL 8.0.32 or above and to Xtrabackup 8.0.32, or switch to Percona Server, and rebuild the table via Online DDL:

$ vtctldclient ApplySchema --skip_preflight --ddl_strategy "vitess" --sql "ALTER TABLE your_table ENGINE=InnoDB" your_keyspace

> SET @@ddl_strategy='vitess';
> ALTER TABLE your_table ENGINE=InnoDB;

VTTablet Restore Metrics

As part of the VTTablet Sidecar Schema Maintenance Refactor in v16.0.0, we dropped the local_metadata table from the sidecar database schema. This table was storing a couple of metrics related to restores from backup, which have now been lost.
They have been re-introduced in v17.0.0 as metrics that can be accessed from /debug/vars.

The original issue can be found here.

Schema-initialization stuck on semi-sync ACKs while upgrading to `v16.0.0`

During upgrades from <= v15.x.x to v16.0.0, as part of PromoteReplica call, the schema-init realizes that there are schema diffs to apply and ends up writing to the database.
The issue is that if semi-sync is enabled, all of these writes get blocked indefinitely.
Eventually, PromoteReplica fails, and this fails the entire PRS call.

A fix for this issue was merged on release-16.0 in PR#13441, read the corresponding bug report to learn more.

This issue is fixed in v16.0.3 and later patch releases.

Broken downgrade from v17.x.x when super_read_only turned on by default

In v17.x.x super_read_only is turned on by default meaning that downgrading from v17 to v16.0.0 breaks due to init_db.sql needing write access.

This issue is fixed in >= v16.0.3 thanks to PR #13525

Major Changes

Breaking Changes

VTGate Advertised MySQL Version

Since Pull Request #11989, VTGate advertises MySQL version 8.0.30. This is a breaking change for clients that rely on the VTGate advertised MySQL version and still use MySQL 5.7.
The users can set the mysql_server_version flag to advertise the correct version.

It is worth noting that the feature to avoid using reserved connections depends on the mysql_server_version CLI flag, which default value has been changed from 5.7.9-vitess to 8.0.30-vitess. We recommend that users running MySQL 5.7 set vtgate's mysql_server_version CLI flag to 5.7.9-vitess to prevent the queries from being unexpectedly rewritten.

Default MySQL version on Docker

The default major MySQL version used by our vitess/lite:latest image is going from 5.7 to 8.0. Additionally, the patch version of MySQL80 has been upgraded from 8.0.23 to 8.0.30.
This change was brought by Pull Request #12252.

⚠️Upgrading to this release with vitess-operator

If you are using the vitess-operator and want to remain on MySQL 5.7, you are required to use the vitess/lite:v16.0.0-mysql57 Docker Image, otherwise the vitess/lite:v16.0.0 image will be on MySQL 80.

However, if you are running MySQL 8.0 on the vitess-operator, with for instance vitess/lite:v15.0.2-mysql80, considering that we are bumping the patch version of MySQL 80 from 8.0.23 to 8.0.30, you will have to manually upgrade:

Add innodb_fast_shutdown=0 to your extra cnf in your YAML file.
Apply this file.
Wait for all the pods to be health...

Contributors

arthurschreiber, dbussink, and 31 other contributors

Assets 5

08 Feb 15:42

GuptaManan100

v16.0.0-rc1

4e32ce1

Vitess v16.0.0-rc1 Pre-release

Pre-release

Release of Vitess v16.0.0-rc1

Summary

Major Changes
Minor Changes
- Backup compression benchmarks
Refactor
- VTTablet sidecar schema maintenance refactor

Major Changes

VReplication

VStream Copy Resume

In PR #11103 we introduced the ability to resume a VTGate VStream copy operation. This is useful when a VStream copy operation is interrupted due to e.g. a network failure or a server restart. The VStream copy operation can be resumed by specifying each table's last seen primary key value in the VStream request. Please see the VStream docs for more details.

VDiff2 GA

We are marking VDiff v2 as Generally Available or production-ready in v16. We now recommend that you use v2 rather than v1 going forward. V1 will be deprecated and eventually removed in future releases.
If you wish to use v1 for any reason, you will now need to specify the --v1 flag.

Tablet throttler

The tablet throttler can now be configured dynamically. Configuration is now found in the topo service, and applies to all tablets in all shards and cells of a given keyspace. For backwards compatibility v16 still supports vttablet-based command line flags for throttler ocnfiguration.

It is possible to enable/disable, to change throttling threshold as well as the throttler query.

See #11604

Incremental backup and point in time recovery

In PR #11097 we introduced native incremental backup and point in time recovery:

It is possible to take an incremental backup, starting with last known (full or incremental) backup, and up to either a specified (GTID) position, or current ("auto") position.
The backup is done by copying binary logs. The binary logs are rotated as needed.
It is then possible to restore a backup up to a given point in time (GTID position). This involves finding a restore path consisting of a full backup and zero or more incremental backups, applied up to the given point in time.
A server restored to a point in time remains in DRAINED tablet type, and does not join the replication stream (thus, "frozen" in time).
It is possible to take incremental backups from different tablets. It is OK to have overlaps in incremental backup contents. The restore process chooses a valid path, and is valid as long as there are no gaps in the backed up binary log content.

Replication manager removal and VTOrc becomes mandatory

VTOrc is now a required component of Vitess starting from v16. If the users want VTOrc to manage replication, then they must run VTOrc.
Replication manager is removed from vttablets since the responsibility of fixing replication lies entirely with VTOrc now.
The flag disable-replication-manager is deprecated and will be removed in a later release.

Breaking Changes

VTGate Advertised MySQL Version

VTGate now advertises MySQL version 8.0.30. This is a breaking change for clients that rely on the VTGate advertised MySQL version and still use MySQL 5.7.
The users can set the mysql_server_version flag to advertise the correct version.

Default MySQL version on Docker

The default major MySQL version used by our vitess/lite:latest image is going from 5.7 to 8.0. Additionally, the patch version of MySQL80 has been upgraded from 8.0.23 to 8.0.30.

Running Vitess on the Operator

If you are using the vitess-operator and want to remain on MySQL 5.7, we invite you to use the vitess/lite:v16.0.0-mysql57 Docker Image.

Add innodb_fast_shutdown=0 to your extra cnf in your YAML file.
Apply this file.
Wait for all the pods to be healthy.
Then change your YAML file to use the new Docker Images (vitess/lite:v16.0.0, defaults to mysql80).
Remove innodb_fast_shutdown=0 from your extra cnf in your YAML file.
Apply this file.

vtctld web UI Removal

In v13, the vtctld UI was deprecated. As of this release, the web/vtctld2 directory is deleted and the UI will no longer be included in any Vitess images going forward. All build scripts and the Makefile have been updated to reflect this change.

However, the vtctld HTTP API will remain at {$vtctld_web_port}/api.

vtctld Flag Deprecation & Deletions

With the removal of the vtctld UI, the following vtctld flags have been deprecated:

--vtctld_show_topology_crud: This was a flag that controlled the display of CRUD topology actions in the vtctld UI. The UI is removed, so this flag is no longer necessary.

The following deprecated flags have also been removed:

--enable_realtime_stats
--enable_vtctld_ui
--web_dir
--web_dir2
--workflow_manager_init
--workflow_manager_use_election
--workflow_manager_disable

Orchestrator Integration Deletion

Orchestrator integration in vttablet was deprecated in the previous release and is deleted in this release.
Consider using VTOrc instead of Orchestrator.

mysqlctl Flags

The mysqlctl command-line client had some leftover (ignored) server flags after the v15 pflag work. Those unused flags have now been removed. If you are using any of the following flags with mysqlctl in your scripts or other tooling, they will need to be removed prior to upgrading to v16:
--port --grpc_auth_static_client_creds --grpc_compression --grpc_initial_conn_window_size --grpc_initial_window_size --grpc_keepalive_time --grpc_keepalive_timeout

Query Serving Errors

In this release, we are introducing a new way to report errors from Vitess through the query interface.
Errors will now have an error code for each error, which will make it easy to search for more information on the issue.
For instance, the following error:

aggregate functions take a single argument 'count(user_id, name)'

Will be transformed into:

VT03001: aggregate functions take a single argument 'count(user_id, name)'

The error code VT03001 can then be used to search or ask for help and report problems.

If you have code searching for error strings from Vitess, this is a breaking change.
Many error st...

Contributors

arthurschreiber, dbussink, and 30 other contributors

Assets 5

Releases: vitessio/vitess

Vitess v15.0.4

Release of Vitess v15.0.4

Contributors

Vitess v17.0.0

Release of Vitess v17.0.0

Summary

Table of Contents

Contributors

Vitess v17.0.0-rc2

Release of Vitess v17.0.0-rc2

Summary

Table of Contents

Contributors

Vitess v17.0.0-rc1

Release of Vitess v17.0.0-rc1

Summary

Table of Contents

Contributors

Vitess v16.0.2

Release of Vitess v16.0.2

Known Issues

Schema-initialization stuck on semi-sync ACKs while upgrading to v16.0.2

Major Changes

Upgrade to go1.20.3

EffectiveCallerId in Vtgate gRPC calls

Shard name validation in TopoServer

Contributors

Vitess v16.0.1

Release of Vitess v16.0.1

Known Issues

Schema-initialization stuck on semi-sync ACKs while upgrading to v16.0.1

Major Changes

Upgrade to go1.20.2

Keyspace name validation in TopoServer

Contributors

Vitess v15.0.3

Release of Vitess v15.0.3

Contributors

Vitess v14.0.5

Release of Vitess v14.0.5

Contributors

Vitess v16.0.0

Release of Vitess v16.0.0

Summary

Table of Contents

Mitigations

Workarounds

Contributors

Vitess v16.0.0-rc1

Release of Vitess v16.0.0-rc1

Summary

Table of Contents

Contributors

Schema-initialization stuck on semi-sync ACKs while upgrading to `v16.0.2`

Upgrade to `go1.20.3`

Schema-initialization stuck on semi-sync ACKs while upgrading to `v16.0.1`

Upgrade to `go1.20.2`