How to select from multiple targets in a single environment yaml file?

[cruise-bryan-murdock]

I'd love to use pytest's built in test collection to run all my tests with a single pytest command, but the various tests use different targets. I can only pass one yaml file to pytest with --lg-env so I figured I could put all the targets into the one yaml file and each test could specify which target it wants. The documentation and the structure of the yaml seems to show support for more than one target in the yaml:

targets:
    target_a:
        ...
    target_b:
        ...

But I see no examples of having anything other than a single target named "main" in the documentation or in the labgrid code examples. I tried putting two targets in a single yaml file and it seems like the second gets used by pytest by default. Is there a way to do what I'm trying to do?

[Bastian-Krause]

There is support for multiple targets: A target is constructed from the environment config via Environment.get_target(role='main'). That's where the "main" default is coming from. You're probably using the target fixture of labgrid's pytest plugin:

labgrid/labgrid/pytestplugin/fixtures.py

Lines 119 to 127 in 6872745

	@pytest.fixture(scope="session")
	def target(env):
	"""Return the default target `main` configured in the supplied
	configuration file."""
	target = env.get_target()
	if target is None:
	raise UserError("Using target fixture without 'main' target in config")
	return target

As you can see there, env.get_target() is called without specifying a role, so this only works for the main target. To get hold of another target, you can create your own fixture, e.g.:

 @pytest.fixture(scope="session") 
 def target_a(env): 
     """Return the target `a` configured in the supplied configuration file.""" 
     target = env.get_target("a")
     if target is None: 
         raise UserError("Using target_a fixture without 'a' target in config") 
  
     return target

[cruise-bryan-murdock]

I added the target_a fixture that you suggested in my conftest.py and I changed my test from this:

def test_foo(target):

to

def test_foo(target_a):

but when I run pytest it's still trying to use target_b (which doesn't have the our custom EppCliDriver driver that target_a has).

Here's the full output from pytest:

$ pytest --lg-env=foo_env.yaml test_foo.py
========================================================================================== test session starts ==========================================================================================
platform linux -- Python 3.8.10, pytest-7.2.2, pluggy-1.2.0
rootdir: /home/bryan.murdock/work/, configfile: pytest.ini
plugins: labgrid-23.0.3, flake8-1.0.6
collected 1 item

test_foo.py E                                                                                                                                                                                  [100%]

================================================================================================ ERRORS =================================================================================================
_____________________________________________________________________________________ ERROR at setup of test_foo _____________________________________________________________________________________

request = <SubRequest 'env' for <Function test_foo>>, record_testsuite_property = <function record_testsuite_property.<locals>.record_func at 0x7f940db79550>

    @pytest.fixture(scope="session")
    def env(request, record_testsuite_property):
        """Return the environment configured in the supplied configuration file.
        It contains the targets contained in the configuration file.
        """
        env = request.config._labgrid_env

        if not env:
            pytest.skip("missing environment config (use --lg-env)")

        record_testsuite_property('ENV_CONFIG', env.config_file)
        targets = list(env.config.get_targets().keys())
        record_testsuite_property('TARGETS', targets)

        for target_name in targets:
            try:
>               target = env.get_target(target_name)

../../../../../.local/lib/python3.8/site-packages/labgrid/pytestplugin/fixtures.py:68:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../../../.local/lib/python3.8/site-packages/labgrid/environment.py:50: in get_target
    target = target_factory.make_target(role, config, env=self)
../../../../../.local/lib/python3.8/site-packages/labgrid/factory.py:159: in make_target
    self.make_driver(target, driver, name, args)
../../../../../.local/lib/python3.8/site-packages/labgrid/factory.py:138: in make_driver
    d = cls(target, name, **args)
../../../../../.local/lib/python3.8/site-packages/cruise_labgrid/strategy/linux_boot.py:62: in __attrs_post_init__
    super().__attrs_post_init__()
../../../../../.local/lib/python3.8/site-packages/labgrid/strategy/common.py:28: in __attrs_post_init__
    super().__attrs_post_init__()
../../../../../.local/lib/python3.8/site-packages/labgrid/driver/common.py:24: in __attrs_post_init__
    super().__attrs_post_init__()
../../../../../.local/lib/python3.8/site-packages/labgrid/binding.py:55: in __attrs_post_init__
    target.bind(self)
../../../../../.local/lib/python3.8/site-packages/labgrid/target.py:423: in bind
    return self.bind_driver(bindable)
../../../../../.local/lib/python3.8/site-packages/labgrid/target.py:372: in bind_driver
    raise errors[0]
../../../../../.local/lib/python3.8/site-packages/labgrid/target.py:362: in bind_driver
    self.get_driver(requirement, name=supplier_name, activate=False),
../../../../../.local/lib/python3.8/site-packages/labgrid/target.py:230: in get_driver
    return self._get_driver(cls, name=name, activate=activate)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Target(name='target_b', env=Environment(config_file='foo_env.yaml')), cls = <class 'cruise_labgrid.driver.epp_cli_driver.EppCliDriver'>

    def _get_driver(self, cls, *, name=None, activate=True, active=False):
        assert not (activate is True and active is True)

        found = []
        other_names = []
        if isinstance(cls, str):
            cls = target_factory.class_from_string(cls)

        for drv in self.drivers:
            if not isinstance(drv, cls):
                continue
            if name and drv.name != name:
                other_names.append(drv.name)
                continue
            if active and drv.state != BindingState.active:
                continue
            found.append(drv)
        if not found:
            name_msg = f" named '{name}'" if name else ""
            if other_names:
                raise NoDriverFoundError(
                    "no {active}{cls} driver{name} found in {target}, matching resources with other names: {other_names}".format(  # pylint: disable=line-too-long
                        active="active " if active else "", cls=cls, name=name_msg, target=self,
                        other_names=other_names)
                )

>           raise NoDriverFoundError(
                f"no {'active ' if active else ''}{cls} driver{name_msg} found in {self}"
E               labgrid.exceptions.NoDriverFoundError: no <class 'cruise_labgrid.driver.epp_cli_driver.EppCliDriver'> driver found in Target(name='target_b', env=Environment(config_file='foo_env.yaml'))

../../../../../.local/lib/python3.8/site-packages/labgrid/target.py:183: NoDriverFoundError
======================================================================================== short test summary info ========================================================================================
ERROR test_foo.py::test_foo - labgrid.exceptions.NoDriverFoundError: no <class 'cruise_labgrid.driver.epp_cli_driver.EppCliDriver'> driver found in Target(name='target_b', env=Environment(config_file='foo_env.yaml'))
=========================================================================================== 1 error in 0.65s ============================================================================================

[cruise-bryan-murdock]

More information. Each target has a RemotePlace that I want to template, like this:

  target_b:
    resources:
      RemotePlace:
        name: !template $LG_PLACE

That place will have resources not every test will need, so they aren't all specified in each target...but I guess that's not really allowed? This raises some other higher-level questions about what I'm trying to do. I will open a new discussion for that.

[Bastian-Krause]

I can't follow. Most resources should come from the RemotePlace, so they don't have to be listed here.

Yes, please open a new discussion, describe what you want to achieve and add a complete config. Thanks!