Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Materialize Command to Vtctldclient #11874

Conversation

rohit-nayak-ps
Copy link
Contributor

@rohit-nayak-ps rohit-nayak-ps commented Dec 4, 2022

Description

When completed, this PR will migrate all current vreplication vtctl-based commands to the new vtctldserver-based approach.

The following commands will be migrated as part of this PR, in order:
Materialize, MoveTables, Reshard, CreateLookupVindex, VDiff, Mount, Migrate

We will move commands one by one to vtctld and copy over all required wrangler related functionality to get each command workfing into the vtctl\workflow package. The vtctl command is then stubbed to call the vtctldserver instead. e2e tests should confirm that this works.

Note that since all the logic is being moved over, we will need to migrate the unit test framework and tests as well. This will be done once all commands have been ported over and the e2e tests pass.

We will also take this opportunity to refactor the code while the new unit tests are being written.

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Documentation was added or is not required

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
…ackage

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
… test passes

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
@rohit-nayak-ps rohit-nayak-ps added Type: Enhancement Logical improvement (somewhere between a bug and feature) Component: VReplication release notes (needs details) This PR needs to be listed in the release notes in a dedicated section (deprecation notice, etc...) NeedsWebsiteDocsUpdate What it says labels Dec 4, 2022
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
@rohit-nayak-ps rohit-nayak-ps removed the release notes (needs details) This PR needs to be listed in the release notes in a dedicated section (deprecation notice, etc...) label Dec 4, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Jan 4, 2023

This PR is being marked as stale because it has been open for 30 days with no activity. To rectify, you may do any of the following:

  • Push additional commits to the associated branch.
  • Remove the stale label.
  • Add a comment indicating why it is not stale.

If no action is taken within 7 days, this PR will be closed.

@github-actions github-actions bot added the Stale Marks PRs as stale after a period of inactivity, which are then closed after a grace period. label Jan 4, 2023
@github-actions
Copy link
Contributor

This PR was closed because it has been stale for 7 days with no activity.

@github-actions github-actions bot closed this Jan 11, 2023
@mattlord mattlord reopened this Jan 23, 2023
@vitess-bot
Copy link
Contributor

vitess-bot bot commented Jan 23, 2023

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

  • Is it really necessary to add this flag?
  • Flag names should be clear and intuitive (as far as possible)
  • Help text should be descriptive.
  • Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow should be required, the maintainer team should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from VTop, if used there.

@mattlord
Copy link
Contributor

@rohit-nayak-ps this will be a project, so IMO we should create a tracking issue for it. We can then repurpose this issue to be about Materialize. If you are good with that, I'll create a tracking issue — where each ticket will include everything for that command, including unit tests and any refactoring — and then start working on another command.

@mattlord mattlord removed the Stale Marks PRs as stale after a period of inactivity, which are then closed after a grace period. label Jan 23, 2023
@rohit-nayak-ps
Copy link
Contributor Author

@rohit-nayak-ps this will be a project, so IMO we should create a tracking issue for it. We can then repurpose this issue to be about Materialize. If you are good with that, I'll create a tracking issue — where each ticket will include everything for that command, including unit tests and any refactoring — and then start working on another command.

Yeah, we can either create a project or a parent task. We should probably only merge the sub-tasks into v17, so that we don't have some commands moved and other not, in v16.

@mattlord mattlord changed the title WIP: Migrate vreplication vtctl commands to vtctld VReplication: Migrate Materialize Command to Vtctldclient Jan 25, 2023
@mattlord mattlord changed the title VReplication: Migrate Materialize Command to Vtctldclient Migrate Materialize Command to Vtctldclient Jan 25, 2023
@github-actions
Copy link
Contributor

This PR is being marked as stale because it has been open for 30 days with no activity. To rectify, you may do any of the following:

  • Push additional commits to the associated branch.
  • Remove the stale label.
  • Add a comment indicating why it is not stale.

If no action is taken within 7 days, this PR will be closed.

@github-actions github-actions bot added the Stale Marks PRs as stale after a period of inactivity, which are then closed after a grace period. label Feb 25, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Mar 5, 2023

This PR was closed because it has been stale for 7 days with no activity.

@github-actions github-actions bot closed this Mar 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: VReplication NeedsWebsiteDocsUpdate What it says Stale Marks PRs as stale after a period of inactivity, which are then closed after a grace period. Type: Enhancement Logical improvement (somewhere between a bug and feature)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants