Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing "testing" object #1740

Conversation

fabal
Copy link
Contributor

@fabal fabal commented May 1, 2024

Introducing a testing object available in late bindings, similar to the building object, but indicating a rez-test context.

@fabal fabal requested a review from a team as a code owner May 1, 2024 02:42
Copy link

linux-foundation-easycla bot commented May 1, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

@fabal fabal force-pushed the testing_object branch from fc32271 to 110cac2 Compare May 2, 2024 04:49
Copy link

codecov bot commented May 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (testing_object@6c36578). Learn more about missing BASE report.

Additional details and impacted files
@@                Coverage Diff                @@
##             testing_object    #1740   +/-   ##
=================================================
  Coverage                  ?   58.71%           
=================================================
  Files                     ?      126           
  Lines                     ?    17208           
  Branches                  ?     3519           
=================================================
  Hits                      ?    10104           
  Misses                    ?     6394           
  Partials                  ?      710           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@JeanChristopheMorinPerso JeanChristopheMorinPerso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just by looking at this PR, I'm struggling to see if it also applies to early bound functions. The description says that it's only for @late, but we have building in @early.

If the changes inject testing in @early, then the documentation should be adapted as such. If it doesn't then I think we should change the PR so that testing will also be available in @early.

Additionally, I would like to see a test or two.

Thanks!

docs/source/package_commands.rst Outdated Show resolved Hide resolved
@fabal
Copy link
Contributor Author

fabal commented May 6, 2024

I think early bound attributes are not relevant to rez-test scenarios because they're flattened upon install, meaning any logic will go away anyway. Unless I'm missing something?

I thought about adding some unit tests but I couldn't find counterparts for building use cases? Unless there aren't any?

@fnaum
Copy link

fnaum commented May 6, 2024

I also think that testing does not make sense in @early, as we early decorated functions are evaluated at build time.
so even if we add the object there it would always be false.

@JeanChristopheMorinPerso
Copy link
Member

Oh, yeah, you are right about @early, by bad.

As for the tests, I don't think there is any for building and we also don't have tests for rez-test. So it could be a good occasion to add some rez-tests tests.

@fabal
Copy link
Contributor Author

fabal commented May 12, 2024

Sounds good, we've carved out some time to add a few unit tests.

@nca45 nca45 force-pushed the testing_object branch 2 times, most recently from 5fc33eb to 6466a13 Compare June 20, 2024 23:42
Copy link
Member

@JeanChristopheMorinPerso JeanChristopheMorinPerso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! The tests are failing, but I added instructions on how to make them pass.

Also, you will need to run the ./src/build_utils/license/apply_copyright script and fix flake8 warnings.

docs/source/package_commands.rst Outdated Show resolved Hide resolved
docs/source/package_commands.rst Outdated Show resolved Hide resolved

def test_1(self):
"""package.py unit tests are correctly run in a testing environment"""
context = ResolvedContext(["testing_obj"])

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
context = ResolvedContext(["testing_obj"])
self.inject_python_repo()
context = ResolvedContext(["testing_obj"])

This will add a usable python package and make it available to the tests (only this test). Without this, the tests won't be portable across platforms because we can't rely on python being available on the system. We need python to run the test suite, but that doesn't mean that calling python should work because it doesn't have to be on PATH, and by experience, it will sometimes not be on PATH.

The magic happens in

repo = os.path.join(os.getcwd(), "__tests_pkg_repo")
os.makedirs(repo, exist_ok=True)
create_python_package(os.path.join(os.getcwd(), "__tests_pkg_repo"))
os.environ["__REZ_SELFTEST_PYTHON_REPO"] = repo

@JeanChristopheMorinPerso JeanChristopheMorinPerso added this to the Next milestone Jun 22, 2024
@JeanChristopheMorinPerso
Copy link
Member

JeanChristopheMorinPerso commented Jul 3, 2024

@fabal @fnaum @nca45 can one of you enable the "allow maintainers to modify the PR" option please? I'll take care of the failing tests.

@nca45 nca45 force-pushed the testing_object branch from 1320509 to 9a09a46 Compare July 4, 2024 02:23
@fabal fabal force-pushed the testing_object branch 2 times, most recently from 9a09a46 to b40e8ee Compare July 4, 2024 07:39
fabal and others added 7 commits July 4, 2024 17:47
Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
Co-authored-by: Jean-Christophe Morin <38703886+JeanChristopheMorinPerso@users.noreply.github.com>
Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
Signed-off-by: nathan-cheung <nathan.cheung.451@outlook.com>
Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
Signed-off-by: Nathan Cheung <nathan.cheung.451@outlook.com>
Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
Signed-off-by: Nathan Cheung <nathan.cheung.451@outlook.com>
Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
… and 'test' tests (#2)

* add missing python package to test_context.py

Signed-off-by: Nathan Cheung <nathan.cheung.451@outlook.com>

* add missing python pacakge to test_test.py

Signed-off-by: Nathan Cheung <nathan.cheung.451@outlook.com>

---------

Signed-off-by: Nathan Cheung <nathan.cheung.451@outlook.com>
Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
@fabal fabal force-pushed the testing_object branch from b40e8ee to d267a0c Compare July 4, 2024 07:48
@fabal
Copy link
Contributor Author

fabal commented Jul 4, 2024

I can't seem to see or have access to this option @JeanChristopheMorinPerso

@JeanChristopheMorinPerso
Copy link
Member

Ah, that only works for user-owned forks, see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork. This PR comes from AL's fork. That's unfortunate.

I'll find a way to fix the tests and will post the diff.

Copy link
Member

@JeanChristopheMorinPerso JeanChristopheMorinPerso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I had some time to look at the failures. I left some comments. Hopefully that should fix it.

src/rez/tests/test_context.py Outdated Show resolved Hide resolved
src/rez/tests/test_context.py Outdated Show resolved Hide resolved
src/rez/tests/test_context.py Show resolved Hide resolved
Co-authored-by: Jean-Christophe Morin <38703886+JeanChristopheMorinPerso@users.noreply.github.com>
Signed-off-by: Nate Cheung <nathan.cheung.451@outlook.com>
@JeanChristopheMorinPerso
Copy link
Member

Tests are not working on macOS! I can probably merge the PR as is and I'll handle the linter stuff.

@JeanChristopheMorinPerso
Copy link
Member

The windows tests are now failing though. Argh. Anyway, at this point, I think I'll just merge this and will handle the rest in another PR.

@fabal
Copy link
Contributor Author

fabal commented Sep 2, 2024

Hi @JeanChristopheMorinPerso , did you need something from us to be able to merge this?

@JeanChristopheMorinPerso
Copy link
Member

Hi @fabal, thanks for the reminder. The last couple of months have been very busy and intense for us. We didn't have the time to push your PR to the finish line unfortunately.

I'm definitely planning on releasing your PR in the next release.

@JeanChristopheMorinPerso JeanChristopheMorinPerso changed the base branch from main to testing_object October 14, 2024 15:34
@JeanChristopheMorinPerso JeanChristopheMorinPerso merged commit 31d469c into AcademySoftwareFoundation:testing_object Oct 14, 2024
42 of 47 checks passed
JeanChristopheMorinPerso added a commit to JeanChristopheMorinPerso/rez that referenced this pull request Oct 14, 2024
* Introducing 'testing' object

Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
Signed-off-by: Nathan Cheung <nathan.cheung.451@outlook.com>
Co-authored-by: Jean-Christophe Morin <jean_christophe_morin@hotmail.com>
Co-authored-by: Nathan Cheung <nathan.cheung.451@outlook.com>
JeanChristopheMorinPerso added a commit to JeanChristopheMorinPerso/rez that referenced this pull request Oct 14, 2024
* Introducing 'testing' object

Signed-off-by: Fabrice Macagno <fabrice.macagno@al.com.au>
Signed-off-by: Nathan Cheung <nathan.cheung.451@outlook.com>
Co-authored-by: Jean-Christophe Morin <jean_christophe_morin@hotmail.com>
Co-authored-by: Nathan Cheung <nathan.cheung.451@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants