Question for write a Test with pytest in Plone

[1letter]

@davisagli @ericof
Where is the best place for questions about pytest-plone? here or the community forum?

I play a little bit with the new pytest layer in an addon created with cookieplone

My goal: a functional test setup in a class with some sample content:

With the following setup all is fine. but the fixture is firing up in every method of my TestMyView class. This is not what i want. as far as I understand it, the reason is the scope of the fixture. in this case the "function scope".

# conftest.py

# global fixture to provide base structure for content
# can use in tests
@pytest.fixture
def contents_payload() -> list:
    """Payload to create two content items."""
    return [
        {
            "type": "Folder",
            "id": "test-folder",
            "title": "Test Folder",
            "description": "A Test Folder",
        },
        {
            "type": "Document",
            "id": "test-doc",
            "title": "Test Document",
            "description": "A Test Document",
        },
    ]

# fixture portal for functional test with test content
@pytest.fixture()
def portal(functional, content_payload):
    # the portal object for functional tests
    portal = functional["portal"]
    with api.env.adopt_roles(["Manager",]):
      for data in contents_payload:
        api.content.create(container=portal, **data)

    transaction.commit()
    return portal

# my Test Class
class TestMyView:
  def test_my_view1(self, portal):
    # do something in a functional test
    pass

  def test_my_view2(self, portal):
    # do something other stuff in a functional test
     pass

Now i changed the code to:

# fixture portal for functional test with test content
@pytest.fixture(scope="class")
def portal(functional_class, content_payload):
    # the portal object for functional tests
    portal = functional_class["portal"]
    with api.env.adopt_roles(["Manager",]):
      for data in contents_payload:
        api.content.create(container=portal, **data)

    transaction.commit()
    return portal

But this ends with an error:

self = <Layer 'collective.addon.testing.Collective.AddonLayer:FunctionalTesting'>, key = 'portal'

    def __getitem__(self, key):
        item = self.get(key, _marker)
        if item is _marker:
>           raise KeyError(key)
E           KeyError: 'portal'

.tox/test/lib/python3.12/site-packages/plone/testing/layer.py:28: KeyError

Where is my mistake? I thought, i can switch easily the scope, and no more changes are needed.

I'm new in the pytest world, please bear with me.

[stevepiercy]

Where is the best place for questions about pytest-plone? here or the community forum?

@1letter both! Thank you for bringing this up. I assume many developers are struggling through the same challenges that you face. A question in the Community Forum exposes this to a broader audience than just this repo, and increases the probability of getting answers that we can put into documentation. Cross-link between that post and this issue so we can track it.

[ericof]

@stevepiercy, either way this is a very good question. I was planning to address this during my talk at the conference, but @sneridagh pushed me to get better documentation for pytest-plone sooner than that.
My question here would be: Should I add it here to this repo, to the testing training or elsewhere?

[stevepiercy]

@ericof I think the Plone 6 documentation is the most important place to put it. That gets the most exposure and has the best SEO, and it is maintained perpetually instead of once per PloneConf.

FYI, we recently revised the Install entry point to the documentation. One of the options is now Create a project with Volto (development or pre-release).

We need similar documentation, "Create a project with Classic UI (development or pre-release)". It should target developers or integrators, especially those who currently use buildout to make the transition. There may be some snippets that you can reuse from that Volto guide. In fact you could probably copy that file and remove any Volto-specific things such as Node.js prerequisites, and your work might be already halfway done for you.

Install should be the foundation. I expect that the usual development tasks—create and manage add-ons, tests, code quality—will be separate pages or topics.

Imagine with that documentation in place, you could refer to that in your talk and training.

As always, don't worry about proper English or MyST spelling, grammar, and syntax. The most important thing is to document the steps that a developer would follow to start from nothing and end with a working Classic UI environment to develop and test an add-on or project. I'm always in Discord in the Documentation channel, if you have questions or need a helping hand.

[davisagli]

@stevepiercy I'm a little confused by your comment. This issue is about how to document writing tests for Plone backend packages using pytest, but you're talking about documentation for how to create a Classic UI project.

The choice of whether to test your Python packages using unittest and the Zope testrunner or using pytest is orthogonal to the choice of whether to use the Classic UI or Volto as your frontend. You can use either unittest or pytest to write tests for any Python package, including both for Classic UI projects and for the backend portion of a Volto project or add-on.

I think we probably want a conceptual guide which gives a very high level overview of different types of testing, and then a Testing subsection of the new Development Guide you started in plone/documentation#1731, structured something like this:

• Testing Volto add-ons
  • Unit testing with Jest (similar to https://6.docs.plone.org/volto/contributing/testing.html but updated so it's not just about contributing to Volto core)
    • Write a test
    • Run a test
    • Test setup
    • Assertions
    • Updating snapshots
  • Acceptance testing with Cypress (similar to https://6.docs.plone.org/volto/contributing/acceptance-tests.html)
    • Run a test
    • Write a test
    • Available commands
    • Assertions
  • Accessibility testing (needs to be written, unless I missed something)
• Testing Python packages
  • Unit testing with unittest and the Zope testrunner (some overlap with https://5.docs.plone.org/develop/testing/unit_testing.html and https://github.com/plone/plone.app.testing/tree/master/docs)
    • Write a test
    • Run a test
    • Test setup
    • Test layers
    • Test coverage
    • Test outgoing email
  • Unit testing with pytest (needs to be written)
    • Write a test
    • Run a test
    • Fixtures
    • Using Plone test layers
    • Test coverage
    • Test outgoing email
  • Acceptance testing
    • REST API client
    • Robot Framework (can link to https://github.com/plone/plone.app.robotframework/blob/master/docs/source/index.rst, but it might need updates)
    • Zope testbrowser (can link to https://5.docs.plone.org/develop/testing/functional_testing.html)

[stevepiercy]

@davisagli I assumed that the choice of Plone installation method would determine which test runner to use, as Cookieplone currently uses pytest for the default.

In any case, I think your outline is excellent, with possibly the exception of the Robot Framework. Isn't that being replaced with Playwright in Plone 6.1?

[davisagli]

I assumed that the choice of Plone installation method would determine which test runner to use, as Cookieplone currently uses pytest for the default.

There's no fundamental reason that has to be the case. The cookiecutter-plone-starter template offered a choice between unittest and pytest, and the new cookieplone template could as well with a little work.

In any case, I think your outline is excellent, with possibly the exception of the Robot Framework. Isn't that being replaced with Playwright in Plone 6.1?

Not exactly. There are efforts to rewrite Plone core's Robot Framework tests to use a new set of commands that is implemented with Playwright instead of selenium, but it's still Robot Framework, and it's still possible to use the old commands. plone/Products.CMFPlone#3813