Run unit tests against PyTorch nightlies as part of our nightly CI

[ebsmothers]

Our CI was not running our unit tests against PyTorch nightlies at all, just our recipe tests.

Ideally, we should not run against PyTorch nightlies on every PR. This is because if PyTorch nightlies are broken it will block all our PRs until things are resolved. This is the reason that we run our recipe tests against the PyTorch nightlies each night. However, we currently do not run our unit tests against the PyTorch nightlies at all in our CI. This is not good.. one of our flex attention tests was broken for a couple days and there was no means of catching it.

So this PR adds unit tests to our nightly CI job that runs against PyTorch nightlies. We can also consider a job that runs against PyTorch nightlies on push to main too.. that way we can get quicker signal

Also I rename the file, job, etc from "Recipe tests" to "GPU tests" since recipe tests is no longer an accurate description.

Edit: after some discussions with @RdoubleA I am gonna enable testing against nightlies on push to main (but still not on PRs). Oh also now that I realize (a) how to do conditional matrix elements, and (b) how similar our two gpu test workflow files are, I am consolidating them into one.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/1569

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 9c83a1f with merge base 01619ce ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.04%. Comparing base (221031a) to head (b250e03).
Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1569      +/-   ##
==========================================
- Coverage   73.36%   71.04%   -2.32%     
==========================================
  Files         287      289       +2     
  Lines       14142    14179      +37     
==========================================
- Hits        10375    10074     -301     
- Misses       3767     4105     +338     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.