Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --metadata option to seed backend with ol events #2082

Merged
merged 2 commits into from
Aug 23, 2022

Conversation

wslulciuc
Copy link
Member

@wslulciuc wslulciuc commented Aug 22, 2022

Problem

Metadata used in the seed cmd is not defined using the OpenLineage standard.

Solution

Update seed cmd to load metadata from a file containing an array of OpenLineage events via the --metadata option:

$ java -jar marquez-api.jar seed -h
usage: java -jar marquez-api.jar
       seed [--url URL] --metadata METADATA [-h] [file]

seeds the HTTP API server with metadata

positional arguments:
  file                   application configuration file

named arguments:
  --url URL              the HTTP API server url (default: http://localhost:8080)
  --metadata METADATA    the path to the metadata file (ex: path/to/metadata.json)
  -h, --help             show this help message and exit

Usage: --metadata

$ java -jar marquez-api.jar seed --url http://localhost:5000 --metadata metadata.json marquez.yml

Screen Shot 2022-08-22 at 4 51 18 PM

Checklist

  • You've signed-off your work
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
  • You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)
  • You've included a header in any source code files (if relevant)

Signed-off-by: wslulciuc <willy@datakin.com>
@wslulciuc wslulciuc requested review from fm100 and mobuchowski August 22, 2022 23:54
@wslulciuc wslulciuc added the review Ready for review label Aug 22, 2022
Signed-off-by: wslulciuc <willy@datakin.com>
@codecov
Copy link

codecov bot commented Aug 23, 2022

Codecov Report

Merging #2082 (b549e1c) into main (4d9475c) will decrease coverage by 1.69%.
The diff coverage is 3.03%.

@@             Coverage Diff              @@
##               main    #2082      +/-   ##
============================================
- Coverage     78.73%   77.04%   -1.70%     
+ Complexity     1014     1013       -1     
============================================
  Files           201      201              
  Lines          5583     4643     -940     
  Branches        423      389      -34     
============================================
- Hits           4396     3577     -819     
+ Misses          733      628     -105     
+ Partials        454      438      -16     
Impacted Files Coverage Δ
api/src/main/java/marquez/cli/SeedCommand.java 7.14% <3.03%> (-76.57%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Collaborator

@fm100 fm100 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@wslulciuc wslulciuc removed the review Ready for review label Aug 23, 2022
@wslulciuc wslulciuc merged commit 95d6de5 into main Aug 23, 2022
@wslulciuc wslulciuc deleted the feature/use-ol-events-in-seed-cmd branch August 23, 2022 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants