Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add default collation to VTGate and VTTablet #9097

Merged
merged 47 commits into from
Nov 16, 2021

Conversation

frouioui
Copy link
Member

@frouioui frouioui commented Oct 27, 2021

Description

This pull request does two things:

  • Changing the default charset from utf8 to utf8mb4
  • Adding a default collation to both VTGate and VTTablet

We are changing the default charset from utf8 (utf8mb3) to utf8mb4 because utf8mb4 is way more optimized than its predecessor utf8mb3. Another reason, utf8mb4 supports more characters. With utf8mb4 being a superset of utf8mb3, queries that were previously relying on utf8mb3 should not have a problem with this new charset.

Default Collation

We are adding a default collation to both VTGate and VTTablet, this is done to let us do string comparisons at the VTGate level. It also ensures that we are using the same connection collation as the one between VTTablet and MySQL. For both components, the collation can be set using the -collation (for VTGate) and -db_collation (for VTTablet). If the flag is not specified, it will default to the default collation of utf8mb4 in regards to the underlying database version we are running (MySQL57, MySQL80, etc).

The default collation for certain charset varies depending on the MySQL version we are running, for this reason, the VTTablet is responsible for telling VTGate which MySQL version is being used.

VTGate

At start time, VTGate subscribes to a health check and waits until a VTTablet indicates which MySQL version is used. Once the MySQL version is known in VTGate, VTGate will create its own collation environment that contains the default collation to use and the collations.Environment that will allow us to lookup other collations when planning/executing queries.

Before each execution, the default collation is added to the ExecuteOptions so the VTTablet can make sure collations are all the same from the client to MySQL.

VTTablet

When opening a new connection with a MySQL server, VTTablet is responsible for setting the connection collation properly using the collation/charset the user has defined, and again, in regards to the underlying MySQL version we are connecting to.

Documentation

Checklist

  • Tests were added or are not required
  • Documentation was added or is not required

@frouioui frouioui added Type: Enhancement Logical improvement (somewhere between a bug and feature) Component: Cluster management release notes labels Oct 27, 2021
@frouioui frouioui changed the title Adding default charset flag to vtgate and matching vttablet's charset Adding default collation flag to vtgate and vttablet Oct 28, 2021
@frouioui frouioui changed the title Adding default collation flag to vtgate and vttablet Add a collation flag to vtgate and vttablet to set the default collation Oct 28, 2021
@@ -289,6 +279,28 @@ func (c *Conn) clientHandshake(characterSet uint8, params *ConnParams) error {
c.Capabilities = capabilities & (CapabilityClientDeprecateEOF)
}

// The MySQL handshake package uses the "character set" field to define
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️ these comments

@frouioui frouioui force-pushed the default-collation branch 5 times, most recently from 8a8c624 to 935d49d Compare November 3, 2021 13:41
@frouioui frouioui mentioned this pull request Nov 3, 2021
2 tasks
@frouioui frouioui force-pushed the default-collation branch 6 times, most recently from 0d69c32 to 6110fc5 Compare November 9, 2021 06:57
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
…mation all the way to the tablet

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
…ol with MySQL

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
…ion ID overflow in SSLRequest

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
@frouioui frouioui changed the title Add a collation flag to vtgate and vttablet to set the default collation Add default collation to VTGate and VTTablet Nov 10, 2021
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
@frouioui frouioui marked this pull request as ready for review November 10, 2021 18:59
@frouioui frouioui requested a review from vmg November 10, 2021 18:59
go/mysql/conn_params.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@vmg vmg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great Floflo. I don't like the collation environment in the Conn Params though. It doesn't feel right, again, because it doesn't serialize as JSON, which the conn params must do. Let's come up with a different place to store it. Hit me up in Slack if you need help.

go/mysql/fakesqldb/server.go Outdated Show resolved Hide resolved
…arams

Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
@frouioui
Copy link
Member Author

This is looking great Floflo. I don't like the collation environment in the Conn Params though. It doesn't feel right, again, because it doesn't serialize as JSON, which the conn params must do. Let's come up with a different place to store it. Hit me up in Slack if you need help.

Your two comments are addressed via a56fe1d

Copy link
Collaborator

@vmg vmg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paired on this with @frouioui this morning. Lookin' gucci. 🇫🇷👑

go/mysql/conn.go Outdated Show resolved Hide resolved
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Vicent Marti <vmg@strn.cat>
@vmg vmg force-pushed the default-collation branch from 9602fd9 to 5936dff Compare November 16, 2021 11:36
@vmg vmg merged commit 9085201 into vitessio:main Nov 16, 2021
@vmg vmg deleted the default-collation branch November 16, 2021 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Breaking change Component: Cluster management Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants