Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration script and DAO updates for job renaming and symlinking #1947

Merged
merged 5 commits into from
Apr 20, 2022

Conversation

collado-mike
Copy link
Collaborator

Problem

Part one of #1946
Implements the migration script and changes to JobDao and RunDao to support symlinked jobs.

Note: All database schema changes require discussion. Please link the issue for context.

Checklist

  • You've signed-off your work
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
  • You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)

Signed-off-by: Michael Collado <collado.mike@gmail.com>
Signed-off-by: Michael Collado <collado.mike@gmail.com>
Signed-off-by: Michael Collado <collado.mike@gmail.com>
Signed-off-by: Michael Collado <collado.mike@gmail.com>
@collado-mike collado-mike requested a review from wslulciuc April 12, 2022 23:49
@collado-mike collado-mike added this to the 0.22.0 milestone Apr 12, 2022
@collado-mike collado-mike changed the title Job symlink Migration script and DAO updates for job renaming and symlinking Apr 12, 2022
@codecov
Copy link

codecov bot commented Apr 12, 2022

Codecov Report

Merging #1947 (6b29b55) into main (35e0a45) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 6b29b55 differs from pull request most recent head 1cab6b7. Consider uploading reports for the commit 1cab6b7 to get more accurate results

@@            Coverage Diff            @@
##               main    #1947   +/-   ##
=========================================
  Coverage     78.09%   78.09%           
- Complexity      944      945    +1     
=========================================
  Files           193      193           
  Lines          5249     5251    +2     
  Branches        418      418           
=========================================
+ Hits           4099     4101    +2     
  Misses          705      705           
  Partials        445      445           
Impacted Files Coverage Δ
api/src/main/java/marquez/db/Columns.java 81.81% <ø> (ø)
api/src/main/java/marquez/db/OpenLineageDao.java 96.26% <ø> (ø)
api/src/main/java/marquez/db/RunDao.java 92.59% <ø> (ø)
api/src/main/java/marquez/db/models/JobRow.java 100.00% <ø> (ø)
api/src/main/java/marquez/db/JobDao.java 100.00% <100.00%> (ø)
...src/main/java/marquez/db/mappers/JobRowMapper.java 72.00% <100.00%> (+1.16%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@@ -78,6 +79,9 @@ private Columns() {}
/* STREAM VERSION ROW COLUMNS */
public static final String SCHEMA_LOCATION = "schema_location";

/* JOB ROW COLUMNS */
public static final String SYMLINK_TARGET_UUID = "symlink_target_uuid";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note, I can imagine the renaming of datasets also requiring a symlink target column.

Copy link
Member

@wslulciuc wslulciuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being able to rename jobs using symlinks has been on the roadmap for a while! Great work, @collado-mike! Also, thanks for being so thorough with testing 💯 🥇

@collado-mike collado-mike enabled auto-merge (squash) April 20, 2022 17:27
@collado-mike collado-mike merged commit 2c4a4db into main Apr 20, 2022
@collado-mike collado-mike deleted the job_symlink branch April 20, 2022 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants