Revamp schema versioning and dev version trigger #393

joanise · 2024-08-26T16:50:18Z

PR Goal?

Clean up how schemas a generated and unit tested

Fixes?

Fixes #392

Also fixes the chicken and egg problem when we start working on the next minor: since the setuptools_scm uses tags to determine the current version, you'd have to tag 2.2.0 if the current is 2.1.0 before it would install with the right version, but that's not right since 2.2.0 would not have been published yet. But, in fact, the solution is simple:

when we start 2.2.0, tag 2.2.0.dev0
prevent pythonpublish.yml from actually publishing that to PyPI, since it's just dev (this PR does that with 65499d6)
so ongoing work will now except the schema at 2.2, but the current published version will still be 2.1.x

Feedback sought?

general validation

@roedoejet make sure the schema logic is good
@dhdaines confirm my interpretation of dev tagging with dynamic versioning

Priority?

low

Tests added?

yup, loads!

How to test?

Run g2p update-schema with the current schema being up to date, wrong, a new version without change, a new version with change (unit testing covers all these cases).

Confidence?

moderate

Version change?

No, but will make the next change cleaner.

github-actions · 2024-08-26T17:03:36Z

CLI load time: 0:00.05
Pull Request HEAD: 208a8e0e695550e20b92eed8b7f032de9e26e213
Imports that take more than 0.1 s:
import time: self [us] | cumulative | imported package

codecov · 2024-08-26T17:04:45Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.54%. Comparing base (54d4e18) to head (208a8e0).
Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #393      +/-   ##
==========================================
+ Coverage   93.50%   93.54%   +0.04%     
==========================================
  Files          18       18              
  Lines        2554     2571      +17     
  Branches      576      579       +3     
==========================================
+ Hits         2388     2405      +17     
  Misses         95       95              
  Partials       71       71

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

joanise · 2024-08-26T20:21:15Z

I added one more thing: make the SCM pretend version thing DRY, and validate it so it stays in sync.

dhdaines · 2024-09-12T17:42:09Z

.github/workflows/tests.yml

@@ -9,7 +9,6 @@ on:
 # This **must** match the Major.Minor version of the JSON schema file


probably should remove this comment :)

Right, yes! :)

joanise · 2024-09-12T17:49:17Z

CI was failing because pydantic 2.9 changes the schema. :(
Fixed by excluding it, but that's annoying because we're generally compatible with it, it's just an irrelevant change, but which has visible effect on the schema file.

dhdaines · 2024-09-12T18:15:37Z

.github/workflows/matrix-tests.yml

        run: |
          python -m pip install --upgrade pip
-          pip install -e .[test]
+          SETUPTOOLS_SCM_PRETEND_VERSION=`cat .SETUPTOOLS_SCM_PRETEND_VERSION` pip install -e .[test]


$(cat .SETUPTOOLS_SCM_PRETEND_VERSION) may be a bit more portable than using backticks here (though specifying bash is still a good idea so you don't get, say, PowerShell!)

I was going to use $(...), but I decided backticks were a better choice. Now I don't remember why...
Does $(...) work with POSIX Bourne shell? I might have hesitated because of not being sure that would be the case, but maybe that's misguided.

OK, no, that was misguided, https://stackoverflow.com/questions/22709371/backticks-vs-braces-in-bash says $() is POSIX compliant, and backticks are deprecated! I learned something today.

yeah the nice thing about $() is that it nests, e.g.

echo $(echo $(echo foo) $(echo bar))

dhdaines · 2024-09-12T18:16:40Z

g2p/cli.py

+            print(
+                dedent(
+                    f"""
+                    Schema {schema_path}


these line lengths are rather long!

Adjusted a bit, but they're sized to look decent after dedenting, so they're going to stay somewhat long.
I agree they were too long, though, so I adjusted them somewhat.

dhdaines · 2024-09-12T18:17:28Z

g2p/tests/test_cli.py

    def test_update_schema(self):
+        # It's an error for the currently saved schema to be out of date


I'll just assume all of this works!

It works too well, killed CI when it ran with Pydantic 2.9.1... :(

dhdaines

Ugh, I am not very awake, I clickd "add single comment" when I meant to click "start a review. Sorry about that. This all looks good to me, assuming it passes tests and coverage.

squashed with test: exercise the modified g2p update-schema logic

We had forgotten to bump the heroku pretend version from 2.0 to 2.1. This is going to happen again and again, so let's put a stop to it all now! The new file .SETUPTOOLS_SCM_PRETEND_VERSION has the pretend version we'll use anywhere we have to do these fake mechanics, with the current major.minor, and we'll make it an error in unit testing if it's out of sync. When we start working a the next major or minor version, we'll need that dev tag for all this to work, which is good because this way we won't forget! And if we try to put the new major or minor version and didn't do it, it'll fail to run the tests.

…system

joanise requested review from dhdaines and roedoejet August 26, 2024 16:50

joanise force-pushed the dev.ej/revamp-schema-versioning branch from a4480e4 to f595744 Compare August 26, 2024 19:59

joanise force-pushed the dev.ej/revamp-schema-versioning branch from f595744 to 257b13b Compare September 12, 2024 16:43

joanise added a commit that referenced this pull request Sep 12, 2024

test: troubleshoot PR #393

5a4279b

joanise added a commit that referenced this pull request Sep 12, 2024

test: troubleshoot PR #393

7709ce4

joanise added a commit that referenced this pull request Sep 12, 2024

test: troubleshoot PR #393

9b5b208

dhdaines reviewed Sep 12, 2024

View reviewed changes

dhdaines approved these changes Sep 12, 2024

View reviewed changes

joanise force-pushed the dev.ej/revamp-schema-versioning branch from 92b037c to c75ffa9 Compare September 12, 2024 18:24

joanise added 5 commits September 12, 2024 14:48

feat: revamp schema versioning and update-schema

6eb29ac

squashed with test: exercise the modified g2p update-schema logic

ci: publish only actual versions, not dev, alpha or beta

e68129d

fix: add [tool.setuptools_scm] in pyproject.toml to please the build …

8929608

…system

fix(deps): pydantic 2.9 changes our schemas, so block it

208a8e0

joanise force-pushed the dev.ej/revamp-schema-versioning branch from c75ffa9 to 208a8e0 Compare September 12, 2024 18:50

joanise merged commit 5631210 into main Sep 12, 2024
8 checks passed

joanise deleted the dev.ej/revamp-schema-versioning branch September 12, 2024 18:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revamp schema versioning and dev version trigger #393

Revamp schema versioning and dev version trigger #393

joanise commented Aug 26, 2024

github-actions bot commented Aug 26, 2024 •

edited

Loading

codecov bot commented Aug 26, 2024 •

edited

Loading

joanise commented Aug 26, 2024

dhdaines Sep 12, 2024

joanise Sep 12, 2024

joanise commented Sep 12, 2024

dhdaines Sep 12, 2024

joanise Sep 12, 2024

joanise Sep 12, 2024

dhdaines Sep 12, 2024

dhdaines Sep 12, 2024 •

edited

Loading

joanise Sep 12, 2024

dhdaines Sep 12, 2024

joanise Sep 12, 2024

dhdaines left a comment

		@@ -9,7 +9,6 @@ on:
		# This must match the Major.Minor version of the JSON schema file

		def test_update_schema(self):
		# It's an error for the currently saved schema to be out of date

Revamp schema versioning and dev version trigger #393

Revamp schema versioning and dev version trigger #393

Conversation

joanise commented Aug 26, 2024

PR Goal?

Fixes?

Feedback sought?

Priority?

Tests added?

How to test?

Confidence?

Version change?

github-actions bot commented Aug 26, 2024 • edited Loading

codecov bot commented Aug 26, 2024 • edited Loading

Codecov Report

joanise commented Aug 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joanise commented Sep 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhdaines Sep 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhdaines left a comment

Choose a reason for hiding this comment

github-actions bot commented Aug 26, 2024 •

edited

Loading

codecov bot commented Aug 26, 2024 •

edited

Loading

dhdaines Sep 12, 2024 •

edited

Loading