Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decide what package versions to run tests against #284

Open
bdpedigo opened this issue Dec 12, 2024 · 3 comments
Open

Decide what package versions to run tests against #284

bdpedigo opened this issue Dec 12, 2024 · 3 comments

Comments

@bdpedigo
Copy link
Collaborator

bdpedigo commented Dec 12, 2024

We recently had some issues/discussion around what versions of package dependencies we should be running tests against. There are at least 3 distinct cases:
(1) Running against the versions pinned in the lockfile, which should be the same across platforms,
(2) Running against the oldest versions of packages we say we support,
(3) Running against the newest versions of packages we say we support.

I believe we can configure the actions to do any of these, or some combination.

@fcollman at some point mentioned that the PR tests could be (1) while the daily builds or pre-release builds could use (3). I worry that this division could make errors slower to catch in the development cycle, since they'd only pop up after PRs are merged.

Soliciting thoughts in general!

@bdpedigo
Copy link
Collaborator Author

As an example of why (2) could be helpful, this bugfix was because I made some assumptions about urllib 2 that I didn't realize weren't part of urllib 1 since I never tested against it #239

@bdpedigo
Copy link
Collaborator Author

(2) would require us doing #282

@bdpedigo
Copy link
Collaborator Author

(3) is definitely useful for catching when a package breaks our stuff, so that should at least be some part of the daily IMO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant