Shimmable suspend-CONT API for async esp_yield() / esp_schedule() "stop the Arduino world" metaphor

[dok-net]

In order to let Arduino-like scheduling libraries, that shim yield() and/or delay(), make use of CPU cycles that are available while async callbacks are awaited, the CONT task must not be completely stopped.
This PR changes the way async callbacks are handled in ESP8266WiFi, currently the only code that works with NonOS async callbacks in ESP8266 Arduino core and core libraries according to Unix grep.
Fixes #7969

[dok-net]

FYI: Alters part of #6107. Conflicts/supersedes #6212. Of interest besides the obvious Arduino incompatibility, in my mind is the question of scheduling efficiency: does the esp scheduler internally consume a significantly different number of cycles for a suspended (esp_yield()) CONT task while a Timer is active, vs. starting, then suspending again, a busy-locked CONT task that checks a condition continually. And if so, are those cycles put to any better use.

[devyte]

This will require extensive testing for functional correctness in different scenarios, behavior if new code vs. current,. and power usage.
About that last, it is my understanding that a loop-yield is more cpu intensive than a loop-delay (n). Also, the sweet spot for such loops is delay(5), determined both from past experiences and from independent conclusions reached recently by @Tech-TX.

At a glance, I see at least one code change that changes behavior: replacing delay(0) by yield(). They are not equivalent in this context. I suspect apps that scan from SYS, e. g. from callbacks that run there, would break.

I'll leave specific comments once I review in detail.

[dok-net]

@devyte Delay(0) vs yield() in master, actual code path unrolled, illegal (panic) path and do nothing pruned:

delay(0) =>

esp_schedule();
cont_yield(g_pcont);
run_scheduled_recurrent_functions(); 

yield() =>

esp_schedule();
cont_yield(g_pcont);
run_scheduled_recurrent_functions();

[dok-net]

esp_schedule() can be un-removed in the CBs if there is no ill effect in calling it on non-suspended CONT, just to make the change smaller. Reverting to delay(x) (x > 0) though turns this PR on its head.

[dok-net]

Before you dive in, let me digest the input you have given me w.r.t. power and performance. On a hunch, I think I have an idea involving just a rename and weak bindings, that leave behavior unchanged without shims, AND let's me recover those coveted cycles for MT when needed. I'll let you know. Thanks for the valuable feedback.

[devyte]

No hurry, this is one of those cases where extremely careful consideration and thought is needed, or you risk breaking/worsening very subtle things that eventually blow things up in seemingly unrelated places. I suggest taking your time, doing broad and careful testing, and reporting your findings/progress back here for further discussion.

[dok-net]

With the latest commit, this PR is supposed to introduce basically no difference in behavior over master, when the weak bindings are not touched.
One pattern is required and implemented, though.
For any delay() that can be resumed by esp_schedule(), a guard loop is required, like so:

//delay(timeout_ms);
auto op_start_time = millis();
// will continue on timeout or when wifi_dns_found_callback fires
while (millis() - op_start_time < timeout_ms && _dns_lookup_pending) {
    esp_delay(timeout_ms);
}

Use esp_delay(1) in order to "give scheduled functions a chance to run (e.g. Ethernet uses recurrent)".

For any esp_yield() that is resumed by esp_schedule(), a guard is also required, for example:

//esp_yield();
_wps_config_pending = true;
// will continue when wifi_wps_status_cb fires
while (_wps_config_pending) {
    esp_yield();
}
_wps_config_pending = false;

[dok-net]

This PR was included in merge c312a2e, and is therefore withdrawn.