Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AIP-65: Add versioning to the SerializedDagModel #42547

Closed
wants to merge 9 commits into from
Closed

AIP-65: Add versioning to the SerializedDagModel #42547

wants to merge 9 commits into from

Commits on Oct 3, 2024

  1. AIP-65: Add versioning to the SerializedDagModel 

    This commit adds versioning to the serializedDagModel.

Changes:
Added new columns, id, and version_number to the SDM and made id the
primary key.

Updated the write_dag method of the SDM to add the SDs correctly.

Updated the queries so the scheduler/webserver runs with the latest SDM

The version_number was added to help us track the evolution of a DAG.
Suppose a DAG with dag_hash AB is changed, and the dag_hash becomes CD.
If the change is reverted, we will have a dag_hash of AB again. In this
case, the version_number would still increment, letting us know that the
DAG was changed three times. I feel it's a meaningful way to track the changes,
independent of the id column, which is database internals.
    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    5cbad8a View commit details
    Browse the repository at this point in the history

  2. fix rebase

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    d9977b8 View commit details
    Browse the repository at this point in the history

  3. Fix migration

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    36a04b3 View commit details
    Browse the repository at this point in the history

  4. fixup! Fix migration

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    26c9895 View commit details
    Browse the repository at this point in the history

  5. add tests

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    e181c50 View commit details
    Browse the repository at this point in the history

  6. fixup! add tests

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    9f539ab View commit details
    Browse the repository at this point in the history

  7. Revert change in example dag

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    347595a View commit details
    Browse the repository at this point in the history

  8. a little refactor

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    b2fc6d8 View commit details
    Browse the repository at this point in the history

  9. fixup! a little refactor

    @ephraimbuddy
    ephraimbuddy committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    01a0c77 View commit details
    Browse the repository at this point in the history