Add documentation for how to test a Briefcase-packaged app

[freakboy3742]

The following exchange occurred in our Gitter chat room. A formalized version of this should be incorporated into our documentation as a topic guide on testing:

@tmontes: Given that briefcase leans towards a src-based layout -- which I like, for many reasons -- how are people running tests against their application code? My previous experience, as is the case with briefcase 0.2, is mostly based on setuptools+setup.py where I'd install the application in editable mode (with pip install -e .) and from there, running tests would be "fine" because the application modules/packages would be importable.

@freakboy3742: There’s two options. The first is to change the sources definition. Although the default Briefcase project uses the src prefix for all it’s code, it doesn’t have to; you can specify any folder you want as a source. If you drop the src prefix from the default sources definition, you can use a “prefixless” source.

The other option is to use a test runner that supports different source prefixes. pytest, for example, has a pytest-pythonpath plugin that lets you add a python_paths option in the [tool:pytest] section of setup.cfg so you can add other prefixes to your source path.

---

We should include details for how to use unittest, pytest and as a stretch goal - trial.

[freakboy3742]

The docs should also mention pytest-briefcase

[saroad2]

The docs should also mention pytest-briefcase

The link is broken. Is this package exists yet?

[freakboy3742]

You've beaten me to it :-)

I posted that comment thinking I was moments away from pushing to pypi... and then realized I needed new Briefcase release before I can push pytest-briefcase.

Hopefully that can happen in the next day or two; in the meantime, the WIP link is https://github.com/beeware/pytest-briefcase

[GenevieveBuckley]

Does pytest-briefcase allow you to run the test suite on the bundled app/environment? Or is it equivalent to pytest with a testpaths argument (i.e. same as regular development testing)?

Right now I'm using a pytest.inito specify that pytest should only look for tests in the source code.

[pytest]
testpaths = myprojectname/

This ignores any packaged bundles (macOS, windows, linux) that might also exist in the directory. This is good, now our CI pipeline won't break.

But it'd also be good if we could run tests on the actual bundled app (or run tests on the source from an environment that exactly matches all the dependency versions that were bundled). Is that what pytest-briefcase is about?

cc @tlambert03

[freakboy3742]

@GenevieveBuckley pytest-briefcase is orthogonal to testpaths; testpaths defines where your tests are; pytest-briefcase makes sure that the code required by your app is as defined in your briefcase configuration. At present, it's not much more than a tool for PYTHONPATH manipulation.

A briefcase project that isn't using a src folder and contains a single app won't really need pytest-briefcase (or, at least, won't notice a significant change). However, if you've got an app that contains:

    myproject/
        src/
            myapp/
            otherapp/
        utils/
            otherstuff

and your briefcase configurations defines:

[tool.briefcase.app.myapp]
source = ['src/myapp', 'utils/otherstuff']

[tool.briefcase.app.secondapp]
source = ['src/secondapp', 'utils/otherstuff']

pytest-briefcase makes sure that the PYTHONPATH is correctly configured so that you can import myapp and import otherstuff when testing MyApp.

It currently runs entirely in the local development environment, assuming that your test environment contains all the relevant dependencies. I'd definitely like to extend it to making sure (or, at least, provide an ability to ensure) that the test environment has the dependencies specified by the Briefcase configuration. However, I'm not 100% sure if that might be better served as a feature of briefcase - e.g., a briefcase dev --no-run option, for example, that will enable you to ensure that dependencies are installed, but won't actually run the app. That command could then be incorporated into your CI config in the same way that pip install -e . is often used for testing libraries.

Running the test suite with the actual packaged Python and environment would be an interesting long term goal, but there's a bunch of complications. Firstly, the "deployed" environment won't have pytest in it, so you'd need to find a way to inject it. Secondly, how do you run the test suite for iOS or Android apps? These are probably solvable problems, but might need a briefcase test option to allow for setting up the test environment.

[freakboy3742]

For those watching this ticket; pytest-briefcase has now been released to PyPI.

[pratheekcr]

Hi, I am first timer and would like to contribute to this repo, is this still open?

[freakboy3742]

The ticket is open, so yes, there's still work to be done, so this is somewhere you could contribute if you want.

As with ticket #394 in which you also expressed an interest - We don't a strong formal process for "taking" a ticket - just drop a comment on the ticket you're planning to work on, and submit a PR; as long as it looks like you're making active progress on the issue, we'll discourage anyone else from working on the same ticket.

[mhsmith]

Running the test suite with the actual packaged Python and environment would be an interesting long term goal, but there's a bunch of complications. Firstly, the "deployed" environment won't have pytest in it, so you'd need to find a way to inject it. Secondly, how do you run the test suite for iOS or Android apps? These are probably solvable problems, but might need a briefcase test option to allow for setting up the test environment.

Android Studio has support for running Java unit tests within a running app. This obviously won't carry over directly to Python unit tests, but to get ideas it still might be useful to look into how it controls the tests and extracts the results.

[mhsmith]

The documentation is now here:

• https://briefcase.readthedocs.io/en/stable/reference/commands/run.html#test-mode
• https://docs.beeware.org/en/latest/tutorial/tutorial-9.html