fix(asyncudp): Fixes and implements tcpip thread locking #10415

HamzaHajeir · 2024-10-03T11:32:31Z

Description of Change

Implements and fixes TCPIP Core locking in AsyncUDP when enabled.

AsyncUDP calls Raw LwIP APIs that are not safe to call from other threads without locking. While some raw APIs are being called by using tcpip_api_call(), other calls are not considering this method, as [1], [2], [3], and [4].

When CONFIG_LWIP_CHECK_THREAD_SAFETY is enabled, the system gets aborted when the lwip core is not locked; Therefore exposing a source of bugs.

Any call to tcpip_api_call() in tcpip core locked results in a deadlock, thus removed the corresponding calls.

I've prefered to check against the state before locking and unlocking, helpful when any concerned AsyncUDP API being called from within LwIP thread, highly through callbacks, if any.

This will not affect the direct Arduino users but until esp32-arduino-libs-builder PR gets merged it will fix a source of bugs.

Currently: Arduino-as-ESPIDF-component users can easily have this fix by enabling CONFIG_LWIP_TCPIP_CORE_LOCKING option.

Related links

https://www.nongnu.org/lwip/2_1_x/multithreading.html

#10391 (comment)

CLAassistant · 2024-10-03T11:32:38Z

All committers have signed the CLA.

github-actions · 2024-10-03T11:33:08Z

	Warnings
⚠️	Some issues found for the commit messages in this PR: the commit message `"fix(asyncudp): Adds missing unlock"`: summary looks too short Please fix these commit messages - here are some basic tips: follow Conventional Commits style correct format of commit message should be: `<type/action>(<scope/component>): <summary>`, for example `fix(esp32): Fixed startup timeout issue` allowed types are: `change,ci,docs,feat,fix,refactor,remove,revert,test` sufficiently descriptive message summary should be between 20 to 72 characters and start with upper case letter avoid Jira references in commit messages (unavailable/irrelevant for our customers) `TIP:` Install pre-commit hooks and run this check when committing (uses the Conventional Precommit Linter).

	Messages
📖	You might consider squashing your 3 commits (simplifying branch history).

👋 Hello HamzaHajeir, we appreciate your contribution to this project!

Click to see more instructions ...

This automated output is generated by the PR linter DangerJS, which checks if your Pull Request meets the project's requirements and helps you fix potential issues.

DangerJS is triggered with each push event to a Pull Request and modify the contents of this comment.

Please consider the following:
- Danger mainly focuses on the PR structure and formatting and can't understand the meaning behind your code or changes.
- Danger is not a substitute for human code reviews; it's still important to request a code review from your colleagues.
- Resolve all warnings (⚠️ ) before requesting a review from human reviewers - they will appreciate it.
- Addressing info messages (📖) is strongly recommended; they're less critical but valuable.
- To manually retry these Danger checks, please navigate to the Actions tab and re-run last Danger workflow.

Review and merge process you can expect ...

We do welcome contributions in the form of bug reports, feature requests and pull requests.

1. An internal issue has been created for the PR, we assign it to the relevant engineer.
2. They review the PR and either approve it or ask you for changes or clarifications.
3. Once the GitHub PR is approved we do the final review, collect approvals from core owners and make sure all the automated tests are passing.
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
4. If the change is approved and passes the tests it is merged into the default branch.

Generated by 🚫 dangerJS against 848b0ae

github-actions · 2024-10-04T08:26:26Z

Memory usage test (comparing PR against master branch)

The table below shows the summary of memory usage change (decrease - increase) in bytes and percentage for each target.

Memory	FLASH [bytes]		FLASH [%]		RAM [bytes]		RAM [%]
Target	DEC	INC	DEC	INC	DEC	INC	DEC	INC
ESP32S3	0	0	0.00	0.00	0	0	0.00	0.00
ESP32S2	0	0	0.00	0.00	0	0	0.00	0.00
ESP32C3	0	0	0.00	0.00	0	0	0.00	0.00
ESP32C6	0	0	0.00	0.00	0	0	0.00	0.00
ESP32	0	0	0.00	0.00	0	0	0.00	0.00

Click to expand the detailed deltas report [usage change in BYTES]

Target	ESP32S3		ESP32S2		ESP32C3		ESP32C6		ESP32
Example	FLASH	RAM	FLASH	RAM	FLASH	RAM	FLASH	RAM	FLASH	RAM
AsyncUDP/examples/AsyncUDPClient	0	0	0	0	0	0	0	0	0	0
AsyncUDP/examples/AsyncUDPMulticastServer	0	0	0	0	0	0	0	0	0	0
AsyncUDP/examples/AsyncUDPServer	0	0	0	0	0	0	0	0	0	0

github-actions · 2024-10-04T08:28:12Z

Test Results

56 files - 59 56 suites - 59 4m 12s ⏱️ - 30m 21s
21 tests - 7 21 ✅ + 4 0 💤 ±0 0 ❌ ±0
135 runs - 72 135 ✅ - 16 0 💤 ±0 0 ❌ ±0

Results for commit 848b0ae. ± Comparison against base commit 733373a.

This pull request removes 9 and adds 2 tests. Note that renamed tests count towards both.

performance.coremark.test_coremark ‑ test_coremark
performance.fibonacci.test_fibonacci ‑ test_fibonacci
performance.psramspeed.test_psramspeed ‑ test_psramspeed
performance.ramspeed.test_ramspeed ‑ test_ramspeed
performance.superpi.test_superpi ‑ test_superpi
validation.periman.test_periman ‑ test_periman
validation.timer.test_timer ‑ test_timer
validation.uart.test_uart ‑ test_uart
validation.unity.test_unity ‑ test_unity

validation.democfg.test_democfg ‑ test_cfg
validation.wifi.test_wifi ‑ test_wifi

♻️ This comment has been updated with latest results.

TD-er · 2024-10-04T08:48:23Z

libraries/AsyncUDP/src/AsyncUDP.cpp

@@ -473,12 +485,13 @@ bool AsyncUDP::_init() {
  if (_pcb) {
    return true;
  }
+  UDP_MUTEX_LOCK();
  _pcb = udp_new();
  if (!_pcb) {
    return false;


Shouldn't there be an unlock before the return?

HamzaHajeir · 2024-10-04T10:39:17Z

Well spotted, missed to consider that path. Now should I add another commit or amend the commit?

…

On Fri, Oct 4, 2024, 11:48 TD-er ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In libraries/AsyncUDP/src/AsyncUDP.cpp <#10415 (comment)> : > @@ -473,12 +485,13 @@ bool AsyncUDP::_init() { if (_pcb) { return true; } + UDP_MUTEX_LOCK(); _pcb = udp_new(); if (!_pcb) { return false; Shouldn't there be an unlock before the return? — Reply to this email directly, view it on GitHub <#10415 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH3O7J2WS6AW55EUKMG375LZZZI63AVCNFSM6AAAAABPJS4YGOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGNBXGQ4TCNBRHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

me-no-dev · 2024-10-04T10:51:15Z

Now should I add another commit or amend the commit?

just add another commit to this PR

HamzaHajeir · 2024-10-04T11:19:55Z

just add another commit to this PR

Done

VojtechBartoska · 2024-10-07T07:10:28Z

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

HamzaHajeir · 2024-10-07T09:30:24Z

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

That bureaucratic :)

Couldn't find that document, can you guide how to?

TD-er · 2024-10-07T09:40:16Z

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

That bureaucratic :)

Couldn't find that document, can you guide how to?

See the 2nd post in this PR: #10415 (comment)

HamzaHajeir · 2024-10-07T09:46:00Z

Hello @HamzaHajeir, please sign CLA without that we are not able to merge this PR. Thanks!

That bureaucratic :)
Couldn't find that document, can you guide how to?

See the 2nd post in this PR: #10415 (comment)

Thanks, done!

fix(asyncudp): Fixes and implements tcpip thread locking

5bbb5d5

me-no-dev approved these changes Oct 4, 2024

View reviewed changes

TD-er reviewed Oct 4, 2024

View reviewed changes

fix(asyncudp): Adds missing unlock

be56836

VojtechBartoska added the Status: Pending CLA ⚠️ Contributor is required to sign the CLA label Oct 7, 2024

me-no-dev added Status: Pending Merge Pull Request is ready to be merged and removed Status: Pending CLA ⚠️ Contributor is required to sign the CLA labels Oct 7, 2024

ci(pre-commit): Apply automatic fixes

848b0ae

me-no-dev merged commit f083e2d into espressif:master Oct 7, 2024
42 checks passed

mathieucarbou mentioned this pull request Oct 26, 2024

Crash with error message: "Required to lock TCPIP core functionality" #10526

Closed

1 task

HamzaHajeir mentioned this pull request Nov 1, 2024

Make LwIP Core Locking available espressif/esp32-arduino-lib-builder#186

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(asyncudp): Fixes and implements tcpip thread locking #10415

fix(asyncudp): Fixes and implements tcpip thread locking #10415

HamzaHajeir commented Oct 3, 2024

CLAassistant commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 4, 2024

github-actions bot commented Oct 4, 2024 •

edited

Loading

TD-er Oct 4, 2024

HamzaHajeir commented Oct 4, 2024 via email

me-no-dev commented Oct 4, 2024

HamzaHajeir commented Oct 4, 2024

VojtechBartoska commented Oct 7, 2024

HamzaHajeir commented Oct 7, 2024

TD-er commented Oct 7, 2024

HamzaHajeir commented Oct 7, 2024

fix(asyncudp): Fixes and implements tcpip thread locking #10415

fix(asyncudp): Fixes and implements tcpip thread locking #10415

Conversation

HamzaHajeir commented Oct 3, 2024

Description of Change

Related links

CLAassistant commented Oct 3, 2024 • edited Loading

github-actions bot commented Oct 3, 2024 • edited Loading

github-actions bot commented Oct 4, 2024

Memory usage test (comparing PR against master branch)

github-actions bot commented Oct 4, 2024 • edited Loading

Test Results

TD-er Oct 4, 2024

Choose a reason for hiding this comment

HamzaHajeir commented Oct 4, 2024 via email

me-no-dev commented Oct 4, 2024

HamzaHajeir commented Oct 4, 2024

VojtechBartoska commented Oct 7, 2024

HamzaHajeir commented Oct 7, 2024

TD-er commented Oct 7, 2024

HamzaHajeir commented Oct 7, 2024

CLAassistant commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 3, 2024 •

edited

Loading

github-actions bot commented Oct 4, 2024 •

edited

Loading