Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Remove the deprecated cml-publish command from the dvc-diff workflow #2559

Merged
merged 3 commits into from
Jun 1, 2023
Merged

CI: Remove the deprecated cml-publish command from the dvc-diff workflow #2559

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
aa4bedc
Add/remove/update baseline images for testing the dvc diff workflow
seisman May 29, 2023
a9727dd
Simpilify the dvc diff workflow by removing the deprecated cml-publis…
seisman Jun 1, 2023
49a3558
Revert "Add/remove/update baseline images for testing the dvc diff wo…
seisman Jun 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 15 additions & 21 deletions .github/workflows/dvc-diff.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,6 @@ jobs:
- name: Setup continuous machine learning (CML)
uses: iterative/setup-cml@v1.2.3

- name: Pull image data from cloud storage
run: dvc pull --remote upstream

# workaround from https://github.com/iterative/cml/issues/1377
- name: Setup NodeJS
uses: actions/setup-node@v1
Expand Down Expand Up @@ -65,43 +62,40 @@ jobs:
run: |
echo -e "## Summary of changed images\n" > report.md
echo -e "This is an auto-generated report of images that have changed on the DVC remote\n" >> report.md

# Pull image data from cloud storage
dvc pull --remote upstream
dvc diff --show-md main HEAD >> report.md

# Get just the filename of the added and modified image from the report
awk 'NF==5 && NR>=7 && $2=="added" {print $4}' report.md > added_files.txt
awk 'NF==5 && NR>=7 && $2=="modified" {print $4}' report.md > modified_files.txt

# Backup new images in the baseline-new directory
mkdir pygmt/tests/baseline-new
cp pygmt/tests/baseline/*.png pygmt/tests/baseline-new/
# Pull images in the main branch from cloud storage
git checkout main
dvc pull --remote upstream

# Append each image to the markdown report
echo -e "## Image diff(s)\n" >> report.md
echo -e "<details>\n" >> report.md

# Added images
echo -e "### Added images\n" >> report.md
while IFS= read -r line; do
echo -e "- $line \n" >> report.md
cml-publish --title $line --md "$line" >> report.md < /dev/null
echo -e "- $(basename $line) \n" >> report.md
echo -e "![](${line/baseline/baseline-new})" >> report.md
done < added_files.txt

# Modified images
echo -e "### Modified images\n" >> report.md
# Upload new images
while IFS= read -r line; do
cml-publish --title $line --md "$line" >> modified_images_new.md < /dev/null
done < modified_files.txt

# Pull images in the main branch from cloud storage
git checkout main
dvc pull --remote upstream
# Upload old images
while IFS= read -r line; do
cml-publish --title $line --md "$line" >> modified_images_old.md < /dev/null
done < modified_files.txt

# Append image report for modified images
echo -e "| Path | Old | New |" >> report.md
echo -e "|---|---|---|" >> report.md
paste modified_files.txt modified_images_old.md modified_images_new.md -d"|" |
awk -F"|" 'function basename(file) {sub(".*/", "", file); return file} {printf("| %s | %s | %s |\n", basename($1), $2, $3)}' >> report.md
while IFS= read -r line; do
echo -e "| $(basename $line) | ![]($line) | ![](${line/baseline/baseline-new}) |" >> report.md
done < modified_files.txt

echo -e "</details>\n" >> report.md

Expand Down