Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: optionally install Scenario with ops[testing] and expose the names in ops.testing #1381

Merged
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
21c0396
Offer pip install ops[testing] and if available expose Scenario in op…
tonyandrewmeyer Sep 18, 2024
cd0ecbf
Use an explicit reference to help Sphinx out.
tonyandrewmeyer Sep 18, 2024
4eedb72
Create a new class rather than inheriting from both the originals.
tonyandrewmeyer Sep 18, 2024
b61469b
Add types in the test class.
tonyandrewmeyer Sep 18, 2024
7ee4fe5
Use explicit references to ops so that copying the docstrings works.
tonyandrewmeyer Sep 19, 2024
40323f2
CharmEvents should be exposed in ops.testing (maybe also in scenario …
tonyandrewmeyer Sep 19, 2024
55f0910
Add Scenario and Harness explicitly rather than via automodule, to wo…
tonyandrewmeyer Sep 19, 2024
e7870cd
Adjust the monkeypatching to also include runtime.
tonyandrewmeyer Sep 19, 2024
41a9854
Fix markup. Note that Harness is included by default.
tonyandrewmeyer Sep 19, 2024
2b60277
Merge in the Scenario customisations.
tonyandrewmeyer Sep 20, 2024
c647130
Re-pin with ops-scenario added.
tonyandrewmeyer Sep 20, 2024
5f21f94
Add ops-scenario to the docs dependencies.
tonyandrewmeyer Sep 20, 2024
d6c7d2d
Use __all__ and move ActionFailed to Harness.
tonyandrewmeyer Sep 23, 2024
8905747
Add a test to ensure that we don't forget to document ops.testing cla…
tonyandrewmeyer Sep 23, 2024
98f0e30
Minor tweaks, mostly based on review.
tonyandrewmeyer Sep 23, 2024
3decc03
Update docs/index.rst
tonyandrewmeyer Sep 23, 2024
3a6c58f
Update docs/index.rst
tonyandrewmeyer Sep 23, 2024
b92d4e3
Update docs/index.rst
tonyandrewmeyer Sep 23, 2024
cbf52b3
Tweak, per review.
tonyandrewmeyer Sep 23, 2024
67b5a7a
WiP docs on separate pages.
tonyandrewmeyer Sep 23, 2024
1f5801e
Split the testing classes into separate pages.
tonyandrewmeyer Sep 23, 2024
151ec9f
Fix merge.
tonyandrewmeyer Sep 23, 2024
04bac0b
Update test for doc split.
tonyandrewmeyer Sep 23, 2024
08c95f8
Fix rebase.
tonyandrewmeyer Sep 24, 2024
cf61118
Allow 'pip install ops[harness], which is the same as pip install ops…
tonyandrewmeyer Sep 24, 2024
8b1d1ee
Typo from previous PR.
tonyandrewmeyer Sep 24, 2024
d0ce232
Add a PendingDeprecationWarning for Harness.
tonyandrewmeyer Sep 24, 2024
24686d3
Tweaks per review.
tonyandrewmeyer Sep 24, 2024
bac91d7
Update ops/_private/harness.py
tonyandrewmeyer Sep 24, 2024
2472e7c
Update docs/index.rst
tonyandrewmeyer Sep 24, 2024
bc05df8
Move some index content to the harness page.
tonyandrewmeyer Sep 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions docs/harness.rst
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
:orphan:

.. _harness:

Harness Unit Test Framework
===========================
Harness (legacy unit testing)
=============================

.. deprecated:: 2.17
The Harness framework is deprecated and will be moved out of the base
Expand All @@ -23,6 +21,11 @@ The Harness framework includes:
- :attr:`~ops.testing.Harness.model` attribute, which exposes e.g. the
:attr:`~ops.Model.unit` attribute for detailed assertions on the unit's state.

.. note::
Unit testing is only one aspect of a comprehensive testing strategy. For more
on testing charms, see `Charm SDK | Testing <https://juju.is/docs/sdk/testing>`_.


.. autoclass:: ops.testing.ActionFailed
:noindex:
.. autoclass:: ops.testing.ActionOutput
Expand Down
31 changes: 10 additions & 21 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,18 @@ The `ops` library is a Python framework for writing and testing Juju charms.

The library (`available on PyPI`_) provides:
tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved

- :ref:`ops_main_entry_point`, used to initialise and run your charm;
- :ref:`ops_module`, the API to respond to Juju events and manage the application;
- :ref:`ops_main_entry_point`, used to initialise and run your charm;
- :ref:`ops_pebble_module`, the Pebble client, a low-level API for Kubernetes containers;
- :ref:`ops_testing_module` frameworks for unit testing charms in a simulated environment;
- the APIs for unit testing charms in a simulated environment:

- :doc:`State-transition testing </state-transition-testing>`. This is the
recommended approach.
tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
- :doc:`Harness </harness>`. This is a deprecated framework, and has issues,
particularly with resetting the charm state between Juju events. It will be
moved out of the base ``ops`` install in an ops release in the future. Charm
tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
authors that don't want to upgrade will still be able to use it with
``pip install ops[harness]``.

You can structure your charm however you like, but with the `ops` library, you
get a framework that promotes consistency and readability by following best
Expand Down Expand Up @@ -60,25 +68,6 @@ ops.pebble

.. _ops_testing_module:

Testing
-------

Two frameworks for unit testing charms in a simulated Juju environment are
available:

* :doc:`State-transition testing </state-transition-testing>`, which tests the charm's state transitions in response
to events. This is the recommended approach. Install ops with the ``testing``
extra to use this framework; for example: ``pip install ops[testing]``
* :doc:`Harness </harness>`, which provides an API reminiscent of the Juju CLI. This is a
deprecated framework, and has issues, particularly with resetting the charm
state between Juju events. It will be moved out of the base ``ops`` install in
a future release.


.. note::
Unit testing is only one aspect of a comprehensive testing strategy. For more
on testing charms, see `Charm SDK | Testing <https://juju.is/docs/sdk/testing>`_.


Indices
=======
Expand Down
24 changes: 15 additions & 9 deletions docs/state-transition-testing.rst
Original file line number Diff line number Diff line change
@@ -1,22 +1,23 @@
:orphan:

.. _state-transition-tests:

Unit Test Framework
===================
Unit testing (was: Scenario)
============================

Install ops with the ``testing`` extra to use this API; for example:
``pip install ops[testing]``

State-transition tests expect you to define the Juju state all at once, define
the Juju context against which to test the charm, and fire a single event on the
charm to execute its logic. The tests can then assert that the Juju state has
changed as expected.
State-transition tests, previously known as 'Scenario', expect you to define the
Juju state all at once, define the Juju context against which to test the charm,
and fire a single event on the charm to execute its logic. The tests can then
assert that the Juju state has changed as expected.

A very simple test, where the charm has no config, no integrations, the unit
is the leader, and has a `start` handler that sets the status to active might
look like this:

.. code-block:: python

from ops import testing
from ops import testing

def test_base():
ctx = testing.Context(MyCharm)
Expand Down Expand Up @@ -63,6 +64,11 @@ A test consists of three broad steps:
- verify that the output state is what you expect it to be
- verify that the charm has seen a certain sequence of statuses, events, and `juju-log` calls

.. note::
Unit testing is only one aspect of a comprehensive testing strategy. For more
on testing charms, see `Charm SDK | Testing <https://juju.is/docs/sdk/testing>`_.


..
_The list here is manually maintained, because the `automodule` directive
expects to document names defined in the module, and not imported ones, and
Expand Down
2 changes: 1 addition & 1 deletion ops/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

"""The API to respond to Juju events and manage the application.

This module provides code features to your charm, including:
This module provides core features to your charm, including:

- :class:`~ops.CharmBase`, the base class for charms and :class:`~ops.Object`,
the base class for charm libraries.
Expand Down
9 changes: 8 additions & 1 deletion ops/_private/harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ def __init__(
state: typing.Optional['State'] = None,
):
self.message = message
self.output = ActionOutput([], {}) if output is None else output
self.output = output or ActionOutput([], {})
self.state = state

def __str__(self):
Expand Down Expand Up @@ -314,6 +314,13 @@ def __init__(
juju_debug_at=self._juju_context.debug_at,
)

warnings.warn(
'Harness is deprecated; we recommend using state transition testing '
"(previously known as 'Scenario') instead",
PendingDeprecationWarning,
stacklevel=2,
)

def _event_context(self, event_name: str):
"""Configures the Harness to behave as if an event hook were running.

Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ docs = [
testing = [
"ops-scenario>=7.0.5,<8",
]
# Empty for now, because Harness is bundled with the base install, but allow
# specifying the extra to ease transition later.
harness = []

[project.urls]
"Homepage" = "https://juju.is/docs/sdk"
Expand Down
Loading