Add web UI notification for under voltage/throttling events #1203

benjamink · 2023-08-10T10:37:49Z

Adds a banner under the header banner showing when an active or previous under voltage or other throttling events have occured on the Raspberry Pi. The changes use the vcgencmd Python module that provides an interface for the underlying vcgencmd utility that comes with the Rasperry Pi OS distribution.

The modes that are displayed can be configured via the python/web/src/settings.py file.

Example with current styling...

Classic Theme:

Modern Theme:

Potential TODOs

Documentation on configuration (in wiki?)
Translations for mode messages
Improved CSS
Unit tests
Mock vcgencmd for Docker development environment (non-Raspberry Pi environment)

benjamink · 2023-08-10T23:38:31Z

Fixes #1199

rdmark · 2023-08-11T01:06:00Z

python/common/src/piscsi/sys_cmds.py

+        state_msgs = {
+            "0": ("error", "Under-voltage detected"),
+            "1": ("warning", "Arm frequency capped"),
+            "2": ("error", "Currently throttled"),
+            "3": ("warning", "Soft temperature limit active"),
+            "16": ("warning", "Under-voltage has occurred"),
+            "17": ("warning", "Arm frequency capping has occurred"),
+            "18": ("warning", "Throttling has occurred"),
+            "19": ("warning", "Soft temperature limit has occurred"),
+        }


As mentioned in Discord, these strings need to be internationalized with gettext before release (i.e. using the _() alias function). The preferred solution in this codebase is to use return code constants that are interpreted by the implementing client, and then define the English language strings in the client code.

I'm not sure how to go about doing that since the current implementation is to return a list of tuples rather than a single return code. I'm open to ideas though.

I just committed a different approach which just uses the _() alias in the base.html template where the message is rendered. Is that sufficient or do you want me to refactor the function itself to somehow provide a return code?

Do the translations NEED to exist for a wrapped string to be rendered or will it just default to the original English message if no translation exists? If the translations are required then I'll need someone to create those for this.

Using a tuple is perfectly fine, since what you do with return codes is to have a constant that resolves to a unique integer ID, which is then mapped by the client to a human readable string.

The codes are defined in:
https://github.com/PiSCSI/piscsi/blob/main/python/common/src/piscsi/return_codes.py
Then used for instance like this:

piscsi/python/common/src/piscsi/piscsi_cmds.py

Line 264 in 57fa874

"return_code": ReturnCodes.ATTACHIMAGE_COULD_NOT_ATTACH,

Then strings are mapped like this in the client: https://github.com/PiSCSI/piscsi/blob/main/python/web/src/return_code_mapper.py
Mapper is called like this:

piscsi/python/web/src/web.py

Line 464 in 57fa874

process = ReturnCodeMapper.add_msg(process)

I pretty sure your approach here to internationalize the variable messages will not work, since gettext will interpret the variable name as a string and not the value of the variable. You can test this by running the python/web/translation_update.sh script. The script will call gettext to extract all internationalized strings and generate a messages.pot file. If you inspect messages.pot you'll see all the extracted strings for translation, which validates whether your approach works for translations or not.

And to answer your last question, gettext will always fall back to the original (English) string in the code in absence of a translation. We have a process of announcing a code freeze to translators 2 weeks before a release, so that they can update their translations, so you don't have to worry about translating strings that you add yourself.

I THINK I might have gotten it reworked with the internalization working in the last 2 commits. Would love for you to check it out. I could be doing something incredibly wrong/improper but the translation script does update the translation files.

Scratch that - I still have bugs…

Aug 14 00:18:59 octoprint PISCSIWEB[1336]: [2023-08-14 00:18:59,701] [ERROR] app.py:1741 Exception on / [GET] Aug 14 00:18:59 octoprint PISCSIWEB[1336]: Traceback (most recent call last): Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/venv/lib/python3.9/site-packages/flask/app.py", line 2525, in wsgi_app Aug 14 00:18:59 octoprint PISCSIWEB[1336]: response = self.full_dispatch_request() Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/venv/lib/python3.9/site-packages/flask/app.py", line 1822, in full_dispatch_request Aug 14 00:18:59 octoprint PISCSIWEB[1336]: rv = self.handle_user_exception(e) Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/venv/lib/python3.9/site-packages/flask/app.py", line 1820, in full_dispatch_request Aug 14 00:18:59 octoprint PISCSIWEB[1336]: rv = self.dispatch_request() Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/venv/lib/python3.9/site-packages/flask/app.py", line 1796, in dispatch_request Aug 14 00:18:59 octoprint PISCSIWEB[1336]: return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/src/web.py", line 260, in index Aug 14 00:18:59 octoprint PISCSIWEB[1336]: return response( Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/src/web.py", line 165, in response Aug 14 00:18:59 octoprint PISCSIWEB[1336]: env_info = get_env_info() Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/src/web.py", line 115, in get_env_info Aug 14 00:18:59 octoprint PISCSIWEB[1336]: [(s[0], ReturnCodeMapper.add_msg(s[1])) for s in throttled_statuses] Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/src/web.py", line 115, in <listcomp> Aug 14 00:18:59 octoprint PISCSIWEB[1336]: [(s[0], ReturnCodeMapper.add_msg(s[1])) for s in throttled_statuses] Aug 14 00:18:59 octoprint PISCSIWEB[1336]: File "/home/pi/piscsi/python/web/src/return_code_mapper.py", line 76, in add_msg Aug 14 00:18:59 octoprint PISCSIWEB[1336]: if "return_code" not in payload: Aug 14 00:18:59 octoprint PISCSIWEB[1336]: TypeError: argument of type 'int' is not iterable

Ok, I think it’s fixed now. Please let me know if there’s a more proper/cleaner way to do this.

rdmark · 2023-08-11T01:14:20Z

It looks like a clean and light-weight solution. Great job!

The one major missing piece is the internationalization of the strings.

You may also want to look into adding tests to the integration test suite. The main technical hurdle there would be the mocking of the undervoltage states, I think.

rdmark · 2023-08-14T02:21:31Z

python/common/src/piscsi/return_codes.py

+    ARM_FREQUENCY_CAPPED = 101
+    CURRENTLY_THROTTLED = 102
+    SOFT_TEMPERATURE_LIMIT_ACTIVE = 103
+    UNDER_VOLTAGE_HAS_OCCURED = 116


Nitpick: Spelling of "OCCURRED"

This was misspelled in a few places. They should all be fixed now.

rdmark · 2023-08-14T02:23:43Z

python/web/src/web.py

@@ -69,6 +69,8 @@
    TEMPLATE_THEMES,
    TEMPLATE_THEME_DEFAULT,
    TEMPLATE_THEME_LEGACY,
+    THROTTLE_NOTIFY_MODES,
+    THROTTLE_TEST_MODES


Nitpick: It is the coding convention in this project to add a trailing comma to the final element in a list.

Fixed this to follow convention in a few areas.

rdmark · 2023-08-14T02:28:30Z

Great job getting the i18n job done! I think this will work, although I haven't tested it myself yet.

Two things:
Please remove the updated pot files from the changeset. Those are updated by the translators later on.
And, look into the feasibility of adding an integration test to https://github.com/PiSCSI/piscsi/tree/main/python/web/tests/api
Since you have the mock in place now, it should be doable I think!

This reverts commit 1bcd64b.

benjamink · 2023-08-14T12:28:46Z

Two things: Please remove the updated pot files from the changeset. Those are updated by the translators later on.

I committed a revert that should reset the files back to what they were.

And, look into the feasibility of adding an integration test to https://github.com/PiSCSI/piscsi/tree/main/python/web/tests/api Since you have the mock in place now, it should be doable I think!

How can I modify values from the settings.py for a unit test? I need to set THROTTLE_TEST_MODES = [“0”, “16”] so the UI will display notifications. Or will it just use the output of the mock script I created for the Docker environment? How can I run the unit tests to ensure they are doing what I expect? Not familiar with how that works yet.

rdmark · 2023-08-15T00:22:59Z

python/web/src/return_code_mapper.py

+        ReturnCodes.SOFT_TEMPERATURE_LIMIT_ACTIVE:
+            _("Soft-temperature limit active"),
+        ReturnCodes.UNDER_VOLTAGE_HAS_OCCURRED:
+            _("Under voltage has occured"),


One remaining typo here! (should say "occurred")

rdmark · 2023-08-15T00:23:23Z

python/web/src/return_code_mapper.py

+        ReturnCodes.UNDER_VOLTAGE_DETECTED:
+            _("Under voltage detected"),
+        ReturnCodes.ARM_FREQUENCY_CAPPED:
+            _("Arm frequency capped"),


Should "Arm" be all uppercase?

rdmark · 2023-08-15T00:38:44Z

Regarding tests, a simplistic approach would be to run the throttling test only when running in Docker, so that you leverage your mock. You can use os.getenv("DOCKER") to detect if you're running in Docker or not!

benjamink · 2023-08-30T17:57:21Z

As per Discord, the tests may not be overly beneficial for this change & thus it might be best to table them. I'll let the maintainers decide if I should pursue it more or not.

Marking this PR as ready to review so others might test it out.

rdmark · 2023-08-31T05:02:03Z

I check out and ran the code. It basically works well!

One thing I noticed is that I had to put the vcgencmd library into web/requirements.txt for pip to actually install the library. It didn't pick it up from common/requirements.txt. Has this worked for you so far?

Another piece of feedback is that I think a more actionable message might be needed. What do you want users to do when they see those messages? Can you give them a little more context what this state actually means?

And finally (this is kind of minor) since only two of the states are actually used right now, I don't think we should mark the rest for gettext i18n. It's kind of rude to ask translators to translate a bunch of obscure strings that will never be used, don't you think. ;)

benjamink · 2023-08-31T11:58:09Z

One thing I noticed is that I had to put the vcgencmd library into web/requirements.txt for pip to actually install the library. It didn't pick it up from common/requirements.txt. Has this worked for you so far?

I did have some issues getting the module in the right place but I thought I had it. I'll make sure it's added to web/requirements.txt.

Another piece of feedback is that I think a more actionable message might be needed. What do you want users to do when they see those messages? Can you give them a little more context what this state actually means?

I've added more descriptive messages that look like this:

And finally (this is kind of minor) since only two of the states are actually used right now, I don't think we should mark the rest for gettext i18n. It's kind of rude to ask translators to translate a bunch of obscure strings that will never be used, don't you think. ;)

I think that since we make it configurable which errors to display that they should all be configured with potential translations. I was under the understanding that if no translation is done for an item it will default to the English version anyhow so I'm not sure what we gain from hardcoding the English versions & complicating the logic with inconsistency. I think all the errors are potentially useful if someone wants full insight even though we only enable 2 of them by default.

benjamink · 2023-08-31T12:01:09Z

Should the vcgencmd Python requirement be in BOTH web/requirements.txt AND common/requirements.txt? I figured it should be since it's used by common/src/piscsi/sys_cmds.py

rdmark · 2023-09-01T06:17:16Z

Should the vcgencmd Python requirement be in BOTH web/requirements.txt AND common/requirements.txt? I figured it should be since it's used by common/src/piscsi/sys_cmds.py

I'm not sure to be frank... I'm not entirely sure when common/requirements.txt is actually needed.

rdmark · 2023-09-01T06:19:56Z

I think that since we make it configurable which errors to display that they should all be configured with potential translations. I was under the understanding that if no translation is done for an item it will default to the English version anyhow so I'm not sure what we gain from hardcoding the English versions & complicating the logic with inconsistency. I think all the errors are potentially useful if someone wants full insight even though we only enable 2 of them by default.

Fair enough, let's do it your way. :)

benjamink added 2 commits August 10, 2023 06:30

add web UI notification for under voltage/throttling events

3749d15

fix branch name in readme

9e75ffb

benjamink requested review from akuker, erichelgeson and rdmark as code owners August 10, 2023 10:37

benjamink marked this pull request as draft August 10, 2023 10:38

rdmark reviewed Aug 11, 2023

View reviewed changes

benjamink added 5 commits August 11, 2023 06:51

support internalization of messages

1d42500

rework for internalization

5c5adb0

update translation messages

1bcd64b

fix ReturnCodeMapper lookup

34edf69

add mock command for vcgencmd

8793ad0

rdmark reviewed Aug 14, 2023

View reviewed changes

benjamink added 5 commits August 14, 2023 07:57

Revert "update translation messages"

280736f

This reverts commit 1bcd64b.

fix syntax/typos

e19fa08

fix syntax/typos

355ca21

fix syntax/typos

e9417e1

fix syntax

3999f38

add throttle notification unittest

0bd2a4b

rdmark requested changes Aug 15, 2023

View reviewed changes

fix return code text

d12424b

benjamink marked this pull request as ready for review August 30, 2023 17:57

rdmark self-requested a review August 31, 2023 05:02

make messages more descriptive

620a7cc

add vcgencmd to web/requirements.txt

041ad46

rdmark approved these changes Sep 1, 2023

View reviewed changes

rdmark merged commit b32027f into PiSCSI:develop Sep 1, 2023

benjamink mentioned this pull request Sep 2, 2023

Fix test failures in low-voltage PR #1207

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add web UI notification for under voltage/throttling events #1203

Add web UI notification for under voltage/throttling events #1203

benjamink commented Aug 10, 2023 •

edited

Loading

benjamink commented Aug 10, 2023

rdmark Aug 11, 2023

benjamink Aug 11, 2023

rdmark Aug 11, 2023 •

edited

Loading

benjamink Aug 13, 2023

benjamink Aug 13, 2023

benjamink Aug 13, 2023

rdmark commented Aug 11, 2023

rdmark Aug 14, 2023

benjamink Aug 14, 2023

rdmark Aug 14, 2023

benjamink Aug 14, 2023

rdmark commented Aug 14, 2023

benjamink commented Aug 14, 2023

rdmark Aug 15, 2023

rdmark Aug 15, 2023

rdmark commented Aug 15, 2023 •

edited

Loading

benjamink commented Aug 30, 2023

rdmark commented Aug 31, 2023

benjamink commented Aug 31, 2023

benjamink commented Aug 31, 2023

rdmark commented Sep 1, 2023

rdmark commented Sep 1, 2023

Add web UI notification for under voltage/throttling events #1203

Add web UI notification for under voltage/throttling events #1203

Conversation

benjamink commented Aug 10, 2023 • edited Loading

Potential TODOs

benjamink commented Aug 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rdmark Aug 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rdmark commented Aug 11, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rdmark commented Aug 14, 2023

benjamink commented Aug 14, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rdmark commented Aug 15, 2023 • edited Loading

benjamink commented Aug 30, 2023

rdmark commented Aug 31, 2023

benjamink commented Aug 31, 2023

benjamink commented Aug 31, 2023

rdmark commented Sep 1, 2023

rdmark commented Sep 1, 2023

benjamink commented Aug 10, 2023 •

edited

Loading

rdmark Aug 11, 2023 •

edited

Loading

rdmark commented Aug 15, 2023 •

edited

Loading