test: refactor test_framework from unittest to pytest style

[IronCore864]

Refactor test_framework.py to pytest style.

Major Changes

• remove unittest.TestCase from the test classes
• remove setup/teardown
• parametrize
• logging related assertions

Before Refactor

$ python -m tox -e unit -- test/test_framework.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_framework.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator
collected 72 items
====================================================================== 72 passed in 1.63s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
ops/__init__.py               10      0      0      0   100%
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61     51     28      0    11%   42-62, 74-114, 122-129
ops/_private/yaml.py           9      2      0      0    78%   28, 33
ops/charm.py                 643    260    177     17    55%   50-80, 147, 211, 219, 227, 270, 288, 421, 451-464, 471-479, 486-505, 580-582, 589-592, 603-605, 612-613, 647-648, 655-660, 667-686, 735-737, 744-747, 754-758, 782-783, 790-795, 802-806, 817-819, 824-825, 832, 839-840, 870, 888-889, 894, 901-903, 910-911, 923-924, 929, 936-938, 945-946, 954, 1021-1031, 1167-1168, 1174-1179, 1182-1184, 1187-1188, 1191-1193, 1198, 1203, 1208, 1213, 1218, 1226-1235, 1343, 1345, 1347, 1383-1396, 1401, 1404, 1407, 1410, 1413, 1416, 1418->1420, 1438-1444, 1465, 1505-1521, 1555-1572, 1594-1597, 1610-1611, 1659-1663, 1672-1678, 1703, 1731-1743, 1763, 1772-1782, 1799-1800, 1804, 1809, 1819-1821
ops/framework.py             685     62    270     40    87%   62->exit, 64->exit, 65->exit, 66->exit, 123, 126, 164->169, 166->169, 170, 280, 309->311, 318, 391-392, 431, 492, 495, 515-516, 519, 534, 559, 562, 597-598, 671->675, 722, 746, 780, 795, 905-907, 924, 927->948, 1062-1066, 1085, 1089, 1094, 1153, 1160, 1181, 1210, 1226, 1257, 1260, 1264, 1268, 1292, 1306, 1310, 1314, 1318, 1322, 1326, 1330, 1334, 1338, 1342, 1392, 1396, 1399-1404, 1408, 1412
ops/jujuversion.py            80     57     48      0    26%   40-49, 52-58, 61-67, 75-95, 100-103, 107, 111, 115, 124, 130, 135-142
ops/lib/__init__.py          176    176     77      0     0%   15-282
ops/log.py                    27     17      4      0    32%   29-30, 38, 55-75
ops/main.py                  262    213     92      0    14%   49-52, 56-62, 75-100, 118-122, 132-143, 147, 152-220, 240-247, 251-264, 272-301, 305-308, 316-317, 328-336, 345, 351-368, 375-376, 406-424, 427-429, 432-439, 442-475, 483-494, 504-511, 515, 519-523, 539-546, 555
ops/model.py                1532    941    628     16    34%   100->exit, 101->exit, 146, 158, 163, 172, 177, 189, 197, 205, 225, 245, 259, 279-285, 300, 305, 317->exit, 319->exit, 326, 357, 384-395, 399-413, 432-436, 439, 465-473, 478-485, 515->exit, 520, 544-552, 556-566, 569, 579-584, 594-597, 606-608, 616-619, 633-641, 668, 696, 700, 724-735, 757-759, 796, 800-803, 806, 809, 812, 815, 818, 821, 843-844, 852, 855, 858, 861-871, 880, 883-905, 924-938, 942, 945, 948, 958-961, 964, 969-978, 985-989, 1026-1042, 1052-1055, 1066-1069, 1091-1112, 1138-1143, 1148-1154, 1164, 1191-1198, 1201-1206, 1211-1214, 1219-1242, 1262, 1281-1289, 1325, 1338-1341, 1349, 1356, 1369-1373, 1392-1397, 1414-1416, 1432-1434, 1451-1453, 1461-1463, 1508-1536, 1539, 1558-1568, 1573, 1576, 1579, 1582, 1585, 1595-1598, 1605, 1609-1613, 1618-1657, 1667-1697, 1703-1705, 1708, 1713-1719, 1722-1723, 1727, 1730-1733, 1736-1740, 1753, 1769-1771, 1774-1776, 1779, 1795-1799, 1805, 1825, 1842, 1845, 1870, 1921-1925, 1951-1953, 1965, 1968, 1971, 1974-1984, 1995-1998, 2005, 2015-2018, 2023, 2028-2029, 2034, 2039-2042, 2051, 2072-2073, 2076, 2079, 2107-2112, 2130-2145, 2149, 2153, 2157-2160, 2164-2177, 2181-2184, 2201, 2210, 2218-2220, 2228-2233, 2247-2248, 2256-2261, 2265, 2269, 2289, 2326, 2347, 2399-2432, 2484-2506, 2511-2534, 2556-2570, 2590-2596, 2600-2606, 2610-2616, 2642, 2661, 2683, 2705, 2737-2742, 2771-2774, 2782-2787, 2802, 2813, 2827, 2830, 2833, 2836, 2847, 2850, 2853, 2856, 2859, 2870, 2873, 2876, 2879, 2882, 2894-2899, 2969-2995, 3018, 3039, 3042, 3044->3048, 3050-3051, 3052->exit, 3054, 3060, 3064, 3067-3069, 3072-3079, 3083-3104, 3108-3127, 3130-3150, 3153-3154, 3161-3175, 3178-3179, 3183-3196, 3205-3230, 3241-3243, 3246-3248, 3251-3261, 3264-3269, 3272-3274, 3283-3284, 3287, 3290, 3293, 3303-3308, 3312-3313, 3322-3331, 3335-3350, 3354, 3366-3373, 3377, 3384-3402, 3406-3411, 3416-3424, 3432-3445, 3453-3469, 3472-3475, 3478-3481, 3484-3487, 3490-3491, 3494-3495, 3502-3511, 3515-3525, 3528-3535, 3542-3543, 3553-3554, 3559-3560, 3567-3574, 3580-3585, 3610-3618, 3621-3622, 3626-3627, 3630-3634, 3661, 3705
ops/pebble.py               1279    890    431     12    26%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 309-314, 318-323, 330-331, 335, 345, 350-352, 359, 385-386, 389, 392, 412-416, 419, 433-434, 437-449, 452, 491-494, 497-506, 528, 533, 536, 551-556, 561, 572, 591-593, 598, 605, 616, 634-642, 647, 661, 678, 697-706, 711, 726, 748-759, 769, 777, 785, 789-795, 799, 804-808, 830-842, 847, 851-859, 862, 866-871, 880-902, 906-930, 938-946, 949, 953-958, 985-987, 991, 996-1004, 1011, 1022-1047, 1051-1063, 1066, 1070-1075, 1097-1106, 1110-1118, 1121, 1124-1129, 1190-1199, 1204-1208, 1222, 1279-1283, 1288-1296, 1305, 1377-1381, 1452-1466, 1469-1471, 1483-1485, 1488-1520, 1534-1561, 1570-1577, 1582-1588, 1597-1611, 1617-1637, 1644, 1648, 1652-1655, 1659, 1666-1668, 1672, 1676-1705, 1709, 1739-1746, 1751-1756, 1772-1781, 1790-1795, 1804-1832, 1836-1837, 1841-1843, 1847-1849, 1855-1859, 1863-1864, 1868-1870, 1887, 1905, 1926, 1947, 1968, 1974-1990, 2015-2019, 2023-2044, 2048-2063, 2068-2077, 2089-2109, 2113-2114, 2122-2126, 2130, 2134, 2156-2194, 2198-2204, 2237-2256, 2264-2275, 2281-2317, 2338-2348, 2375-2384, 2400-2408, 2430, 2452, 2596-2709, 2712-2717, 2720-2722, 2736-2749, 2767-2773, 2790-2800, 2808-2809, 2842-2852, 2859-2882, 2885-2904, 2907-2925, 2929-2931, 2935, 2938-2940, 2943, 2947, 2951, 2955-2962, 2990-3011, 3015-3057, 3067-3092
ops/storage.py               159     45     30      0    71%   44-47, 76-77, 82-83, 221, 243, 254-258, 265, 269-271, 275-277, 289-293, 301-307, 315, 355-356, 376-382, 393-396, 406
ops/testing.py              1589   1589    768      0     0%   15-3568
ops/version.py                 2      0      0      0   100%
----------------------------------------------------------------------
TOTAL                       6514   4303   2553     85    30%
  unit: OK (2.33=setup[0.02]+cmd[1.83,0.48] seconds)
  congratulations :) (2.36 seconds)

After Refactor

$ python -m tox -e unit -- test/test_framework.py
unit: commands[0]> coverage run --source=ops/ -m pytest --ignore=test/smoke -v --tb native test/test_framework.py
===================================================================== test session starts ======================================================================
platform darwin -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0 -- /Users/tiexin/work/operator/.tox/unit/bin/python
cachedir: .tox/unit/.pytest_cache
rootdir: /Users/tiexin/work/operator
collected 96 items
====================================================================== 96 passed in 1.22s ======================================================================
unit: commands[1]> coverage report
Name                       Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------
ops/__init__.py               10      0      0      0   100%
ops/_private/__init__.py       0      0      0      0   100%
ops/_private/timeconv.py      61     51     28      0    11%   42-62, 74-114, 122-129
ops/_private/yaml.py           9      2      0      0    78%   28, 33
ops/charm.py                 643    260    177     17    55%   50-80, 147, 211, 219, 227, 270, 288, 421, 451-464, 471-479, 486-505, 580-582, 589-592, 603-605, 612-613, 647-648, 655-660, 667-686, 735-737, 744-747, 754-758, 782-783, 790-795, 802-806, 817-819, 824-825, 832, 839-840, 870, 888-889, 894, 901-903, 910-911, 923-924, 929, 936-938, 945-946, 954, 1021-1031, 1167-1168, 1174-1179, 1182-1184, 1187-1188, 1191-1193, 1198, 1203, 1208, 1213, 1218, 1226-1235, 1343, 1345, 1347, 1383-1396, 1401, 1404, 1407, 1410, 1413, 1416, 1418->1420, 1438-1444, 1465, 1505-1521, 1555-1572, 1594-1597, 1610-1611, 1659-1663, 1672-1678, 1703, 1731-1743, 1763, 1772-1782, 1799-1800, 1804, 1809, 1819-1821
ops/framework.py             685     62    270     40    87%   62->exit, 64->exit, 65->exit, 66->exit, 123, 126, 164->169, 166->169, 170, 280, 309->311, 318, 391-392, 431, 492, 495, 515-516, 519, 534, 559, 562, 597-598, 671->675, 722, 746, 780, 795, 905-907, 924, 927->948, 1062-1066, 1085, 1089, 1094, 1153, 1160, 1181, 1210, 1226, 1257, 1260, 1264, 1268, 1292, 1306, 1310, 1314, 1318, 1322, 1326, 1330, 1334, 1338, 1342, 1392, 1396, 1399-1404, 1408, 1412
ops/jujuversion.py            80     57     48      0    26%   40-49, 52-58, 61-67, 75-95, 100-103, 107, 111, 115, 124, 130, 135-142
ops/lib/__init__.py          176    176     77      0     0%   15-282
ops/log.py                    27     17      4      0    32%   29-30, 38, 55-75
ops/main.py                  262    213     92      0    14%   49-52, 56-62, 75-100, 118-122, 132-143, 147, 152-220, 240-247, 251-264, 272-301, 305-308, 316-317, 328-336, 345, 351-368, 375-376, 406-424, 427-429, 432-439, 442-475, 483-494, 504-511, 515, 519-523, 539-546, 555
ops/model.py                1532    941    628     16    34%   100->exit, 101->exit, 146, 158, 163, 172, 177, 189, 197, 205, 225, 245, 259, 279-285, 300, 305, 317->exit, 319->exit, 326, 357, 384-395, 399-413, 432-436, 439, 465-473, 478-485, 515->exit, 520, 544-552, 556-566, 569, 579-584, 594-597, 606-608, 616-619, 633-641, 668, 696, 700, 724-735, 757-759, 796, 800-803, 806, 809, 812, 815, 818, 821, 843-844, 852, 855, 858, 861-871, 880, 883-905, 924-938, 942, 945, 948, 958-961, 964, 969-978, 985-989, 1026-1042, 1052-1055, 1066-1069, 1091-1112, 1138-1143, 1148-1154, 1164, 1191-1198, 1201-1206, 1211-1214, 1219-1242, 1262, 1281-1289, 1325, 1338-1341, 1349, 1356, 1369-1373, 1392-1397, 1414-1416, 1432-1434, 1451-1453, 1461-1463, 1508-1536, 1539, 1558-1568, 1573, 1576, 1579, 1582, 1585, 1595-1598, 1605, 1609-1613, 1618-1657, 1667-1697, 1703-1705, 1708, 1713-1719, 1722-1723, 1727, 1730-1733, 1736-1740, 1753, 1769-1771, 1774-1776, 1779, 1795-1799, 1805, 1825, 1842, 1845, 1870, 1921-1925, 1951-1953, 1965, 1968, 1971, 1974-1984, 1995-1998, 2005, 2015-2018, 2023, 2028-2029, 2034, 2039-2042, 2051, 2072-2073, 2076, 2079, 2107-2112, 2130-2145, 2149, 2153, 2157-2160, 2164-2177, 2181-2184, 2201, 2210, 2218-2220, 2228-2233, 2247-2248, 2256-2261, 2265, 2269, 2289, 2326, 2347, 2399-2432, 2484-2506, 2511-2534, 2556-2570, 2590-2596, 2600-2606, 2610-2616, 2642, 2661, 2683, 2705, 2737-2742, 2771-2774, 2782-2787, 2802, 2813, 2827, 2830, 2833, 2836, 2847, 2850, 2853, 2856, 2859, 2870, 2873, 2876, 2879, 2882, 2894-2899, 2969-2995, 3018, 3039, 3042, 3044->3048, 3050-3051, 3052->exit, 3054, 3060, 3064, 3067-3069, 3072-3079, 3083-3104, 3108-3127, 3130-3150, 3153-3154, 3161-3175, 3178-3179, 3183-3196, 3205-3230, 3241-3243, 3246-3248, 3251-3261, 3264-3269, 3272-3274, 3283-3284, 3287, 3290, 3293, 3303-3308, 3312-3313, 3322-3331, 3335-3350, 3354, 3366-3373, 3377, 3384-3402, 3406-3411, 3416-3424, 3432-3445, 3453-3469, 3472-3475, 3478-3481, 3484-3487, 3490-3491, 3494-3495, 3502-3511, 3515-3525, 3528-3535, 3542-3543, 3553-3554, 3559-3560, 3567-3574, 3580-3585, 3610-3618, 3621-3622, 3626-3627, 3630-3634, 3661, 3705
ops/pebble.py               1279    890    431     12    26%   172->exit, 184->exit, 186->exit, 190->exit, 192->exit, 194->exit, 204-267, 283->exit, 285->exit, 287->exit, 289->exit, 291->exit, 309-314, 318-323, 330-331, 335, 345, 350-352, 359, 385-386, 389, 392, 412-416, 419, 433-434, 437-449, 452, 491-494, 497-506, 528, 533, 536, 551-556, 561, 572, 591-593, 598, 605, 616, 634-642, 647, 661, 678, 697-706, 711, 726, 748-759, 769, 777, 785, 789-795, 799, 804-808, 830-842, 847, 851-859, 862, 866-871, 880-902, 906-930, 938-946, 949, 953-958, 985-987, 991, 996-1004, 1011, 1022-1047, 1051-1063, 1066, 1070-1075, 1097-1106, 1110-1118, 1121, 1124-1129, 1190-1199, 1204-1208, 1222, 1279-1283, 1288-1296, 1305, 1377-1381, 1452-1466, 1469-1471, 1483-1485, 1488-1520, 1534-1561, 1570-1577, 1582-1588, 1597-1611, 1617-1637, 1644, 1648, 1652-1655, 1659, 1666-1668, 1672, 1676-1705, 1709, 1739-1746, 1751-1756, 1772-1781, 1790-1795, 1804-1832, 1836-1837, 1841-1843, 1847-1849, 1855-1859, 1863-1864, 1868-1870, 1887, 1905, 1926, 1947, 1968, 1974-1990, 2015-2019, 2023-2044, 2048-2063, 2068-2077, 2089-2109, 2113-2114, 2122-2126, 2130, 2134, 2156-2194, 2198-2204, 2237-2256, 2264-2275, 2281-2317, 2338-2348, 2375-2384, 2400-2408, 2430, 2452, 2596-2709, 2712-2717, 2720-2722, 2736-2749, 2767-2773, 2790-2800, 2808-2809, 2842-2852, 2859-2882, 2885-2904, 2907-2925, 2929-2931, 2935, 2938-2940, 2943, 2947, 2951, 2955-2962, 2990-3011, 3015-3057, 3067-3092
ops/storage.py               159     45     30      0    71%   44-47, 76-77, 82-83, 221, 243, 254-258, 265, 269-271, 275-277, 289-293, 301-307, 315, 355-356, 376-382, 393-396, 406
ops/testing.py              1589   1589    768      0     0%   15-3568
ops/version.py                 2      0      0      0   100%
----------------------------------------------------------------------
TOTAL                       6514   4303   2553     85    30%

Test cases become more because of parametrize, but total coverage does not change.

[IronCore864]

I've left some comments to help review this PR.

[benhoyt]

Looking good. Just a few initial comments.

[tonyandrewmeyer]

Looking good - the description still says this is a draft, is that still the case?

[benhoyt]

Looks good, just a couple of minor style comments.

[benhoyt]

Nice, thanks!

[tonyandrewmeyer]

This still says "Draft PR" in the description - if it's not draft any more, can that get removed? (Also, it's probably easiest to just use GitHub's draft functionality for drafts).

[tonyandrewmeyer]

Looking good - thanks for all the work on these!