Introducing "testing" object

[fabal]

Introducing a testing object available in late bindings, similar to the building object, but indicating a rez-test context.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: nca45 / name: Nate Cheung (bd74553, d267a0c, 2af057e, 6fbc8e3, 96f40e3)
• ✅ login: fabal / name: Fabrice Macagno (2ba694a, d4ab96d, e49f794)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.71%. Comparing base (e215a77) to head (6fbc8e3).
Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1740      +/-   ##
==========================================
+ Coverage   58.39%   58.71%   +0.32%     
==========================================
  Files         126      126              
  Lines       17205    17208       +3     
  Branches     3519     3519              
==========================================
+ Hits        10047    10104      +57     
+ Misses       6491     6394      -97     
- Partials      667      710      +43     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[JeanChristopheMorinPerso]

Just by looking at this PR, I'm struggling to see if it also applies to early bound functions. The description says that it's only for @late, but we have building in @early.

If the changes inject testing in @early, then the documentation should be adapted as such. If it doesn't then I think we should change the PR so that testing will also be available in @early.

Additionally, I would like to see a test or two.

Thanks!

[fabal]

I think early bound attributes are not relevant to rez-test scenarios because they're flattened upon install, meaning any logic will go away anyway. Unless I'm missing something?

I thought about adding some unit tests but I couldn't find counterparts for building use cases? Unless there aren't any?

[fnaum]

I also think that testing does not make sense in @early, as we early decorated functions are evaluated at build time.
so even if we add the object there it would always be false.

[JeanChristopheMorinPerso]

Oh, yeah, you are right about @early, by bad.

As for the tests, I don't think there is any for building and we also don't have tests for rez-test. So it could be a good occasion to add some rez-tests tests.

[fabal]

Sounds good, we've carved out some time to add a few unit tests.

[JeanChristopheMorinPerso]

Looks great! The tests are failing, but I added instructions on how to make them pass.

Also, you will need to run the ./src/build_utils/license/apply_copyright script and fix flake8 warnings.

[JeanChristopheMorinPerso]

Suggested change

	context = ResolvedContext(["testing_obj"])
	self.inject_python_repo()
	context = ResolvedContext(["testing_obj"])

This will add a usable python package and make it available to the tests (only this test). Without this, the tests won't be portable across platforms because we can't rely on python being available on the system. We need python to run the test suite, but that doesn't mean that calling python should work because it doesn't have to be on PATH, and by experience, it will sometimes not be on PATH.

The magic happens in

rez/src/rez/cli/selftest.py

Lines 89 to 93 in 9094b08

	repo = os.path.join(os.getcwd(), "__tests_pkg_repo")
	os.makedirs(repo, exist_ok=True)
	create_python_package(os.path.join(os.getcwd(), "__tests_pkg_repo"))
	os.environ["__REZ_SELFTEST_PYTHON_REPO"] = repo

[JeanChristopheMorinPerso]

@fabal @fnaum @nca45 can one of you enable the "allow maintainers to modify the PR" option please? I'll take care of the failing tests.

[fabal]

I can't seem to see or have access to this option @JeanChristopheMorinPerso

[JeanChristopheMorinPerso]

Ah, that only works for user-owned forks, see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork. This PR comes from AL's fork. That's unfortunate.

I'll find a way to fix the tests and will post the diff.

[JeanChristopheMorinPerso]

Alright, I had some time to look at the failures. I left some comments. Hopefully that should fix it.

[JeanChristopheMorinPerso]

Tests are not working on macOS! I can probably merge the PR as is and I'll handle the linter stuff.

[JeanChristopheMorinPerso]

The windows tests are now failing though. Argh. Anyway, at this point, I think I'll just merge this and will handle the rest in another PR.

[fabal]

Hi @JeanChristopheMorinPerso , did you need something from us to be able to merge this?

[JeanChristopheMorinPerso]

Hi @fabal, thanks for the reminder. The last couple of months have been very busy and intense for us. We didn't have the time to push your PR to the finish line unfortunately.

I'm definitely planning on releasing your PR in the next release.