Tell pants about tests that use globs to test with many pack fixtures

[cognifloyd]

Background

This is another part of introducing pants, as discussed in various TSC meetings.

Related PRs can be found in:

• pantsbuild milestone (which includes PRs since Initialize pantsbuild #5713)
• pantsbuild label (which also includes preparatory PRs that came before adding pantsbuild)

Overview of this PR

This PR:

• adds a packs_glob target (in pants-plugins/pack_metadata) with a rule that "infers" dependencies (like a * glob) on all of the subdirectories (unless ignored in the dependencies field).
• creates a python fixture file in st2tests/st2tests/fixtures/packs/all_packs_glob for tests that use globs to discover a bunch of packs. This fixture depends on the packs_glob(name="all_packs") target to facilitate the glob behavior.

This PR is basically a follow-up to the "Use python imports to identify fixtures" PRs: #5699, #5702, #5703, #5704, #5705, and #5706. But, instead of including these fixtures in that series of PRs, I waited to add this until after the pack_metadata targets were added in #5871.

[amanda11]

I am worried that we are going to forget to do this when we add a new test pack.
I am just wondering if we can either discover these or if we need to have a comment in the current test packs that exist that they are registered here. So that if someone copies an existing pack they will see the message that they also have to add here.

[cognifloyd]

Yes. I agree. I already stumbled over that when packs were added after I added this (in my PoC branch awhile ago). So, I want a better solution than just comments.

Dependencies are not allowed to use globs because things like import * are generally a bad idea and make dependency inference much more complicated.

But, I can probably add a new pants plugin, or perhaps extend the pack_metadata plugin, with a packs_glob target. Then I would inject dependencies on all the globbed pack directories. 🤔

Yeah. I'll add it to pack_metadata and see how that works.

[cognifloyd]

Upon closer inspection, I realized I don't understand the one test that would use the child_packs fixture, and it's not really using a glob like the all_packs one, so I removed it.

Also, I updated target(name="all_packs") to use be a new packs_glob target instead. This way we don't have to manually register all of the packs. As soon as I document that in the PR description, I will mark this ready for review again.

[cognifloyd]

This is ready for review now.

[cognifloyd]

Nothing depends on this (and the one in packs_invalid) directly right now. We will need both to include all the packs in the wheel.