Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

For chip/lgtm env #4

Closed
wants to merge 581 commits into from
Closed

For chip/lgtm env #4

wants to merge 581 commits into from

Conversation

mspang
Copy link
Owner

@mspang mspang commented Oct 1, 2020

Problem

Summary of Changes

<what's in this PR>

fixes #<issue number, or numbers. if no issue, please create one>

e-rk and others added 30 commits August 31, 2020 13:54
* [gn] add nRF Connect SDK support with gn

* [nrfconnect] include missing header in device layer

* [core] fix signed/unsigned comparison

* [thread] use proper sprintf format for uint32_t

* [zephyr] remove uses of deprecated types and fix casting

* [nrfconnect] clean up unused include

* [github] move nRF Connect build configuration
* Update ReadMe

* Restyled by shellharden

* Fix typo

Co-authored-by: Restyled.io <commits@restyled.io>
* WIP: Demonstration of action()-based binary flashing.

Strawman for one method of building flashable binaries and a consistent
method of flashing them, for parity with some existing `make flash`
commands and ultimately for automated testing.

This is temporarily wholly inside examples/lock-app/nrf5 in order to
experiment with minimal disruption but is intended to become generic.

issue project-chip#1520 GN build parity - flashing

* Remove include dependency on underlying crypto libraries (project-chip#2310)

* Remove include dependency on underlying crypto libraries

* Restyled by clang-format

* Use nlassert, cpp style cast

* Restyled by clang-format

* use inline methods to cleanup the code

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>

* [shell] add missing inttypes.h header (project-chip#2357)

* Address feedback and generalize.

* WIP: Demonstration of action()-based binary flashing.

Strawman for one method of building flashable binaries and a consistent
method of flashing them, for parity with some existing `make flash`
commands and ultimately for automated testing.

This is temporarily wholly inside examples/lock-app/nrf5 in order to
experiment with minimal disruption but is intended to become generic.

issue project-chip#1520 GN build parity - flashing

* Address feedback and generalize.

* Add missing (ignored) symbolic link for `build`.

* Remove unused import from gn_run_binary.py

* Fix paths in flashing rules

* Remove the build symlinks

On second thought, we need to wait for automake removal for this.

* Revised flashing script generation and execution.

* Streamlined scripts and BUILD.gn files.

* Script/generator cleanup and description.

* Restyled by gn

Co-authored-by: Pankaj Garg <pgarg2@apple.com>
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: Jiacheng Guo <gjc@google.com>
Co-authored-by: Michael Spang <spang@google.com>
* Add Silicon Labs ZCL framework files

* Add CHIP license to all C files

* Add missing level-control content

* Add missing content in several plugins

* restyled

* add clusters directories to CHIPv1Clusters build

* add stubbed out cluster callbacks

* whoops

* restyle

* update gen_config.h

* remove .mak file

* move reporting

* update endpoint_config.h to include clusters

* remove duplicate on-off

* update znet-bookkeeping.c

* remove unneeded xml, propoerties, info files
add copyrights to all the source files

* * remove poll control cluster
* update boolean -> bool

* fixup

* fix indexing into attribute data

* update comments, too

* turn on DEFINES that pull in cluster plugins

Co-authored-by: Sergei Lissianoi <selissia@silabs.com>
Setting up the LEDs is using BoltLockMgr() before its initialized.
Reorder these operations.
add copyrights to all the source files
* update boolean -> bool
Disabling advertising via SetAdvertisingEnabled(false) does not
currently work because kFlag_Advertising is not set, and the wrong flag
is checked in DriveBLEState, and HandleConnectionCloseEvent may throw a
wrench in things by messing directly with the advertising flag.
* fix build esp32 with gn

* sh doesn't like '[[' on Linux
* Add ability to generate jar files (tested at least on android)

* A few comments were updated

* Support extra arguments for the javac runner. Code is cleaner now

* conditionally compile java

* Place java build rules in separate directory

* Add build file for java

* Compile JNI so

* Compile for jni cpp files seems to work now

* Run gn format

* Run gn format on gni files as well

* Moved the java compile to be android specific

* Add libSetupPayloadParser as a java build

* Reformat build.gn files

* Fix LGTM.com warning for python package not used

* Add setuppayload dependency to java setup payload, to match autotools deps

* Fix LWIP compile error by including build config from the chip generated code

* Add JDK11 to chip-build-android Dockerfile

* Added a workflow that compiles things for android

* Add path to the gn build step as well

* Revert openthread repo update. Not sure what caused that

* Added arm64 support for android builds.

I needed this to be able to test that GN-build libraries actually work
on my phone.

* Add a arch-specific folder for the output of andoid libs.

Android apps need folders to match the architecture for JNI load to
work. This makes it easier to setup android builds.

* Run gn format on gn and gni files

* Make android builds compile both arm and arm64 builds

* Fix quoting for gn args

* Update base on code reveiew comments

* Ran gn format on gni files

* Fix device layer include on Android

Co-authored-by: Michael Spang <spang@google.com>
mspang and others added 19 commits September 29, 2020 11:20
We're actually taking the first conditional, since "android" !=
"freertos". Reorder these conditions so that Android logging is used on
Android.
* Cleanup unused WiFi interfaces

* Cleanup unused WiFi local varibles
…hip#2888)

Discriminator can be 12-bit long while ChipBLEDeviceIdentificationInfo
used 7-bit mask (0x7f) in the getter.
The return value for mbedtls_x509write_csr_pem was being used to set the length, whereas this is 0, or error code.
PacketBuffer uses uint16_t for all of its sizes already.  It's easier
to tell that we're not overflowing the buffer if we know our header
sizes also fit in uint16_t.
…oject-chip#2832)

* Added a compilable start of a message cache

* Rename to generic cache since we will need addresses as well as identifiers (exchange id). More unit tests added

* Added Add and Remove methods plus unit tests for them

* Add remove matching method and unit tests

* Remove peer address as a dependency - things are generic now

* Remove transport build dependency for now

* Add core dependency for CHIP_ERROR

* Add find method and unit tests

* gn format files

* Add ending newline to Cache.h

* Add another ending newline

* Update the name of the include guard

* Update based on code review comments.

* Run gn format on gn and gni files

* Remove unused define, updated TestCache comments a bit

* Update casts for testcache - android compiler is stricter than the gcc I oritinally used for wconversion tests
* [zephyr] Align BLE advertising data with the spec

* [nRF Connect] Add rendezvous info to QR Code payload

* Also move QR Code printing code to examples/common to be
  shared across all examples.
* Restyled by whitespace
* [example] Add rendezvous session to nrf apps

* add bypass

* revert id change

* check PIN code

* add license header
* [GN] ESP32 flashing support

#### Problem

ESP32 is missing .flash.py support equivalent to EFR32 and NRF5.

#### Summary of Changes

- Generate flashing wrapper script from ESP32 wifi-echo Makefile.
- Added esp32_firmware_utils.
- Added subclass hooks for Flasher.make_wrapper() and .parse_argv().
- Added Flasher.format_command() for tool command argument lists.
- Use argparse.Namespace directly instead of converting to a dict.
- Minor text fixes.

Flashing wrapper generation from GN is not included here since there
is no GN binary target yet, but would be easy to add based on one of
the existing //third_party/*_sdk/*_executable.gni.

* [cirque] Add requests to py requirements (project-chip#2906)

* Restyled by prettier-markdown

Co-authored-by: Song Guo <songguo@google.com>
Co-authored-by: Restyled.io <commits@restyled.io>
* Delete a bunch of automake files, remove autotools from CI

* Remove makefile-bootstrap as well

* Remove autotools CI for reals

* Add lgtm YML file to try to control LGTM

* Use 4 space intent for yml

* Another update to match prettier

* Ask for gcc9 for LGTM.com

* Fix target os for gn configure - it is linux, not linux_x64

* Remove target os flag from gn gen
…p#2934)

* Remove unused code from wifi-echo.cpp to avoid confusion

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>
@restyled-io restyled-io bot mentioned this pull request Oct 1, 2020
@mspang mspang closed this Oct 1, 2020
mspang added a commit that referenced this pull request Feb 17, 2021
Android isn't able to unwind the CHIP native stack, which interferes
with debugging. Turn on unwind tables to fix this.  Unclear if we want
this in optimized builds, but Android OS seems to be able to unwind all
of frameworks and OS libraries, so for now assume we do. There's only a
space cost, and we can revisit that later.

This is the difference between this:

backtrace:
02-16 20:49:34.628  3872  3872 F DEBUG   : backtrace:
02-16 20:49:34.628  3872  3872 F DEBUG   :       #00 pc 000000000008246c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: 5812256023147338b8a9538321d4c456)
02-16 20:49:34.628  3872  3872 F DEBUG   :       #1 pc 00000000000a836c  /data/app/com.google.chip.chiptool-rPVLWEFRvE413khV9YptWg==/base.apk (offset 0x97a000) (chip::NetworkProvisioning::SendNetworkCredentials(char const*, char const*)+96)

and this:
02-16 20:56:04.323  5040  5040 F DEBUG   : backtrace:
02-16 20:56:04.323  5040  5040 F DEBUG   :       #00 pc 000000000008246c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: 5812256023147338b8a9538321d4c456)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #1 pc 00000000000a839c  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (chip::NetworkProvisioning::SendNetworkCredentials(char const*, char const*)+96)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #2 pc 000000000009bb58  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (chip::RendezvousSession::SendNetworkCredentials(char const*, char const*)+44)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #3 pc 000000000009bb94  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (non-virtual thunk to chip::RendezvousSession::SendNetworkCredentials(char const*, char const*)+44)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #4 pc 0000000000051488  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (AndroidDeviceControllerWrapper::SendNetworkCredentials(char const*, char const*)+128)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #5 pc 0000000000054188  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (Java_chip_devicecontroller_ChipDeviceController_sendWiFiCredentials+188)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #6 pc 000000000013f350  /apex/com.android.runtime/lib64/libart.so (art_quick_generic_jni_trampoline+144) (BuildId: ccd73e8ae9b59d5596b3b8aeef234d43)
<snip>
02-16 20:56:04.327  5040  5040 F DEBUG   :       #75 pc 00000000000be560  /system/lib64/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+116) (BuildId: e5b25f8fb9f6bb45ccbeca8c07061dad)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #76 pc 00000000000c13d0  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vector<android::String8> const&, bool)+776) (BuildId: e5b25f8fb9f6bb45ccbeca8c07061dad)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #77 pc 00000000000034e0  /system/bin/app_process64 (main+1168) (BuildId: ade4367f7cc82a88f668180d34ce79fe)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #78 pc 000000000007dc24  /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+108) (BuildId: 5812256023147338b8a9538321d4c456)
mspang added a commit that referenced this pull request Feb 17, 2021
Android isn't able to unwind the CHIP native stack, which interferes
with debugging. Turn on unwind tables to fix this.  Unclear if we want
this in optimized builds, but Android OS seems to be able to unwind all
of its frameworks and native libraries, so for now assume we do. There's
only a space cost, and we can revisit that later.

This is the difference between this:

backtrace:
02-16 20:49:34.628  3872  3872 F DEBUG   : backtrace:
02-16 20:49:34.628  3872  3872 F DEBUG   :       #00 pc 000000000008246c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: 5812256023147338b8a9538321d4c456)
02-16 20:49:34.628  3872  3872 F DEBUG   :       #1 pc 00000000000a836c  /data/app/com.google.chip.chiptool-rPVLWEFRvE413khV9YptWg==/base.apk (offset 0x97a000) (chip::NetworkProvisioning::SendNetworkCredentials(char const*, char const*)+96)

and this:
02-16 20:56:04.323  5040  5040 F DEBUG   : backtrace:
02-16 20:56:04.323  5040  5040 F DEBUG   :       #00 pc 000000000008246c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: 5812256023147338b8a9538321d4c456)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #1 pc 00000000000a839c  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (chip::NetworkProvisioning::SendNetworkCredentials(char const*, char const*)+96)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #2 pc 000000000009bb58  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (chip::RendezvousSession::SendNetworkCredentials(char const*, char const*)+44)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #3 pc 000000000009bb94  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (non-virtual thunk to chip::RendezvousSession::SendNetworkCredentials(char const*, char const*)+44)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #4 pc 0000000000051488  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (AndroidDeviceControllerWrapper::SendNetworkCredentials(char const*, char const*)+128)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #5 pc 0000000000054188  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (Java_chip_devicecontroller_ChipDeviceController_sendWiFiCredentials+188)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #6 pc 000000000013f350  /apex/com.android.runtime/lib64/libart.so (art_quick_generic_jni_trampoline+144) (BuildId: ccd73e8ae9b59d5596b3b8aeef234d43)
<snip>
02-16 20:56:04.327  5040  5040 F DEBUG   :       #75 pc 00000000000be560  /system/lib64/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+116) (BuildId: e5b25f8fb9f6bb45ccbeca8c07061dad)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #76 pc 00000000000c13d0  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vector<android::String8> const&, bool)+776) (BuildId: e5b25f8fb9f6bb45ccbeca8c07061dad)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #77 pc 00000000000034e0  /system/bin/app_process64 (main+1168) (BuildId: ade4367f7cc82a88f668180d34ce79fe)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #78 pc 000000000007dc24  /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+108) (BuildId: 5812256023147338b8a9538321d4c456)
mspang added a commit that referenced this pull request Feb 23, 2021
* Fix unwinding on Android

Android isn't able to unwind the CHIP native stack, which interferes
with debugging. Turn on unwind tables to fix this.  Unclear if we want
this in optimized builds, but Android OS seems to be able to unwind all
of its frameworks and native libraries, so for now assume we do. There's
only a space cost, and we can revisit that later.

This is the difference between this:

backtrace:
02-16 20:49:34.628  3872  3872 F DEBUG   : backtrace:
02-16 20:49:34.628  3872  3872 F DEBUG   :       #00 pc 000000000008246c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: 5812256023147338b8a9538321d4c456)
02-16 20:49:34.628  3872  3872 F DEBUG   :       #1 pc 00000000000a836c  /data/app/com.google.chip.chiptool-rPVLWEFRvE413khV9YptWg==/base.apk (offset 0x97a000) (chip::NetworkProvisioning::SendNetworkCredentials(char const*, char const*)+96)

and this:
02-16 20:56:04.323  5040  5040 F DEBUG   : backtrace:
02-16 20:56:04.323  5040  5040 F DEBUG   :       #00 pc 000000000008246c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: 5812256023147338b8a9538321d4c456)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #1 pc 00000000000a839c  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (chip::NetworkProvisioning::SendNetworkCredentials(char const*, char const*)+96)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #2 pc 000000000009bb58  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (chip::RendezvousSession::SendNetworkCredentials(char const*, char const*)+44)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #3 pc 000000000009bb94  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (non-virtual thunk to chip::RendezvousSession::SendNetworkCredentials(char const*, char const*)+44)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #4 pc 0000000000051488  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (AndroidDeviceControllerWrapper::SendNetworkCredentials(char const*, char const*)+128)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #5 pc 0000000000054188  /data/app/com.google.chip.chiptool-dz3iwqwmItgQDBBaEcevJw==/base.apk (offset 0x97a000) (Java_chip_devicecontroller_ChipDeviceController_sendWiFiCredentials+188)
02-16 20:56:04.323  5040  5040 F DEBUG   :       #6 pc 000000000013f350  /apex/com.android.runtime/lib64/libart.so (art_quick_generic_jni_trampoline+144) (BuildId: ccd73e8ae9b59d5596b3b8aeef234d43)
<snip>
02-16 20:56:04.327  5040  5040 F DEBUG   :       #75 pc 00000000000be560  /system/lib64/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+116) (BuildId: e5b25f8fb9f6bb45ccbeca8c07061dad)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #76 pc 00000000000c13d0  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vector<android::String8> const&, bool)+776) (BuildId: e5b25f8fb9f6bb45ccbeca8c07061dad)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #77 pc 00000000000034e0  /system/bin/app_process64 (main+1168) (BuildId: ade4367f7cc82a88f668180d34ce79fe)
02-16 20:56:04.327  5040  5040 F DEBUG   :       #78 pc 000000000007dc24  /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+108) (BuildId: 5812256023147338b8a9538321d4c456)

* Make it an argument
mspang added a commit that referenced this pull request Sep 22, 2021
…aligned with the spec (project-chip#9455)

* RotatingId: version0

* RotatingId: version1

* RotatingId: version0

* RotatingId: version1

* Fix Darwin host build (project-chip#3990)

#### Problem

Some conversions to use PacketBufferHandle (project-chip#3909) broke Darwin builds,
which aren't currently run in CI.

#### Summary of Changes

Fix src/platform/Darwin/BleConnectionDelegateImpl.mm to match the API
change in project-chip#3909.

* Add '-Wextra' to compiler flags (project-chip#3902)

* Implement Level Control Cluster (project-chip#3806)

* New seekbar in Android CHIPTool
* Sample usage in lighting-app/nrfconnect

Signed-off-by: Markus Becker <markus.becker@tridonic.com>

* Fix Rendezvous over BLE after recent changes (project-chip#4012)

PR project-chip#3704 introduced a change that the BLE transport in
RendezvousSession is only initialized when PeerAddress
in RendezvousParams is of type BLE. However, PeerAddress
isn't initialized anywhere. As a result Rendezvous over BLE
stopped working between Android CHIPTool and accessories.

Btw, remove an assert related to the storage delegate
as it seems an optional member of the device controller.

* [thread] fix invalid configuration of active dataset (project-chip#4008)

* Fix data loss or crash in TCPEndPoint with LwIP (project-chip#4022)

* Fix data loss or crash in TCPEndPoint with LwIP

#### Problem

Under the configuration CHIP_SYSTEM_CONFIG_USE_LWIP, in some cases where
the data size exceeds the TCP window size, TCPEndPoint can either die or
lose data when accounting of un-acked data falls out of sync.

#### Summary of Changes

Imported fix from Weave:

* This change removes separate accounting of the unsent
  data position and replaces it with simple counting of
  sent-but-not-acked data and a skip-loop at the start
  of DriveSending().

Fixes project-chip#4013 - Data loss or crash in TCPEndPoint with LwIP

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>

* Update lighting-app gen/ folder with ZAP generated content (project-chip#4010)

* Fix segmentation fault error in response echo message (project-chip#3984)

* Add back Android default build coverage & fix the build (project-chip#3966)

mDNS doesn't build with no device layer. Remove it from the build and
add back the coverage that would catch this that was lost in project-chip#3340.

* Update all-clusters-app gen/ folder with ZAP generated content (project-chip#3963)

* Move src/inet/tests to auto-test-driver generation (project-chip#3997)

* Rename TestUtils to UnitTestRegistration. (project-chip#4021)

Looking to remove usage of 'Utils' unless we have really no choice.
'UnitTestRegistration' seems clearer in what it does compared to
'TestUtils'.

* Update src/lib/core/tests to auto-test-driver generation (project-chip#3991)

* Cleanup zap chip-helper.js (project-chip#3973)

* Cleanup zap chip-helper.js

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>

* Move src/transport/tests to auto-test-driver generation (project-chip#3999)

* Move src/transport/tests to auto-test-driver generation

* Add relevant libraries (and more test-capable libs) to nrf.

* Refactor inet test helpers (to not include actual inet tests), try to make qemu allow better linkage but still failed for transport tests so disabled for now

* Added more tests on esp32 qemu

* Restyle fixes

* Fix cast errors in InetCommon

* Disable raw tests from zephyr: somehow they fail running out of endpoints

* Disable DNS test on zephyr

* Remove inet endpoint test from zephyr

* Remove inet endpoint test from zephyr - fix again

* Modify gitignore

* Restyle fixes

* Use CHIPDeviceController instead of CHIPDeviceController_deprecated (project-chip#3979)

* Implement the missing part of Exchange Header in Transport layer (project-chip#4017)

* Implement the missing part of Exchange Header in Transport layer

* Revert comment 'if' back to 'iff'("if and only if")

* Remove duplicated send flag defines and put ExchangeMgr/ExchangeConte… (project-chip#3994)

* Remove duplicated send flag defines and put ExchangeMgr/ExchangeContext under the same namespace as CRMP

* Rename kSendFlag_Default to kSendFlag_None

* Move src/lib/asn1/tests and src/ble/tests to auto-test-driver generation (project-chip#3998)

* Move src/lib/asn1/tests and src/ble/tests to auto-test-driver generation

* Remove one more unused file

* Attempt to enable asn1 and ble tests in esp32 - see if they pass or not

* Fix merge error

* Update include header for ASN1 test

* Include  ASN1 in libCHIP

* Some conversions to use PacketBufferHandle (project-chip#4011)

* Some conversions to use PacketBufferHandle

#### Problem

Code should use `PacketBufferHandle` rather than `PacketBuffer *`.

#### Summary of Changes

- Converts remaining receive path in //src/inet and //src/transport.
- Converts most of //src/ble.
- Introduces Handle versions of the `AddToEnd`/`DetachTail` pair.

Part of issue project-chip#2707 - Figure out a way to express PacketBuffer ownership in the type system

* Restyled by clang-format

* review

* revive BtpEngine::Clear[TR]xPacket()
* simplify conditional
* (void) message.DetachTail() → message.FreeHead()
* remove ExchangeContext::kSendFlag_RetainBuffer
* missed pBuf.IsNull()
* DetachHead() → PopTail()
* typos

Co-authored-by: Restyled.io <commits@restyled.io>

* Move src/system/tests to auto-test-driver generation (project-chip#4000)

* Move src/system/tests to auto-test-driver generation

* Remove a TCP/IP init call that was killing qemu

* Remove explicit "all" target from root build file (project-chip#3967)

The "all" target exists implicitly and contains everything. We don't
need to specify one, and it's misleading to do so specifying deps has no
effect.

* Make src/setup_payload compile with -Werror=conversion (project-chip#4032)

* Add SSID and password to chip-tool pairing (project-chip#4054)

* Get temperature-measurement and all-clusters-app to use examples/common/chip-app-server (project-chip#4039)

#### Problem

PR project-chip#3704 introduced a change where a `PeerAddress` is now required in order to start `RendezvousSession`.
Sadly the multiple code paths bootstrapping `RendezvousSession` has not been updated.

PR project-chip#4012 add a fix for some of the `examples/` but not for the `all-clusters-app` nor the `temperature-measurement-app`.

To avoid such situation, this PR merge `examples/common/chip-app-server` and the custom code from `all-clusters-app` and `temperature-measurement-app`.

One of the more discutable change of this PR (imo) is the code that moves the custom `Echo` mechanism from the `all-clusters-app` to `chip-app-server`. I was hoping to get rid of it before doing this change but the `all-clusters-app` and the `temperature-measurement-app` are broken since project-chip#3704 and this PR should fix that.
Also I have a PR (mostly) ready once project-chip#3979 lands to get rid of this `Echo` specific code and replace it by a manufacturer specific `ping` command.

 #### Summary of Changes
 * Remove `EchoServer.cpp`, `RendezvousDeviceDelegate.cpp` and `include/RendezvousDeviceDelegate.h` from `all-clusters-app`
 * Remove `ResponseServer.cpp`, `RendezvousDeviceDelegate.cpp` and `include/RendezvousDeviceDelegate.h` from `temperature-measurement-app`
 * Introduce `chip-app-server/include/AppDelegate.h` in order to keep the behavior the `all-clusters-app` that turns on/off leds on different events. Maybe it should be converted to some types of `ChipDeviceEvent` or `CHIPCallback` at some point.
 * Fix `chip-app-server` to accomodate for the specifics of `all-clusters-app`

* Add all Thread ULA addresses to the lwip interface (project-chip#4053)

ULA prefixes will used for CHIP network so we need to add all these
addresses to the interface.

* Remove src/lib/message. (project-chip#4055)

* Remove src/lib/message.

Updated messaging implementation lives in src/messaging and the
src/lib/message is not currently compiled or linked in.

This saves us from looking at this code when some refactoring is needed
(e.g. the SystemPacketBuffer changes).

* Remove one more unused file

* [ChipTool] Add Payload Parse Command (project-chip#3696)

* [ChipTool] Add Payload Parse Command

* [ChipTool] Add Payload Parse Command

* [ChipTool] Restyle issues resolved

* Restyled by whitespace

* Resolve build errors caused by Command.h schema change

Co-authored-by: lijayaku <lijayaku@amazon.com>
Co-authored-by: Restyled.io <commits@restyled.io>

* rename ParseCommand to QRCodeParseCommand

* adding AdditionalDataParseCommand version 0

* Adding parsing logic + logging

* adding another parsing method

* Basic Parsing is DONE

* fixing memory issue

* removing some logs

* removing more logs

* minor update

* Add RotatingDeviceId to DNS-SD

* Revert "Merge pull request #4 from hnnajh/rotating-id-test"

This reverts commit 0235d05, reversing
changes made to 3e1a4b9.

* Storing RI in Octet String + Adding Binary format for BLE

* Fixing rotating id parser + adding unittests

* restyling

* refactoring rotating id unit tests

* Added more unit tests for Rotating Device Id

* updated styling

* refactor RI tests

* Added RI Unittest + more validation

* applying restyling

* Fix CI

* update styling

* Fix CI

* Update Styling

* Fix CI

* Restyling

* Fixing nits

* Using MutableByteSpan in RI generation

* Fixing nits

Co-authored-by: Kevin Schoedel <67607049+kpschoedel@users.noreply.github.com>
Co-authored-by: Vivien Nicolas <vnicolas@apple.com>
Co-authored-by: Markus Becker <Markus.Becker@tridonic.com>
Co-authored-by: Damian Królik <66667989+Damian-Nordic@users.noreply.github.com>
Co-authored-by: Łukasz Duda <lukasz.duda@nordicsemi.no>
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: Yufeng Wang <44623591+yufengwangca@users.noreply.github.com>
Co-authored-by: Michael Spang <spang@google.com>
Co-authored-by: Andrei Litvin <andrei@andy314.com>
Co-authored-by: jepenven-silabs <67962328+jepenven-silabs@users.noreply.github.com>
Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>
Co-authored-by: Jiacheng Guo <gjc@google.com>
Co-authored-by: Liju Jayakumar <26148162+lijujayakumar@users.noreply.github.com>
Co-authored-by: lijayaku <lijayaku@amazon.com>
mspang added a commit that referenced this pull request Jan 25, 2022
* RotatingId: version0

* RotatingId: version1

* RotatingId: version0

* RotatingId: version1

* Fix Darwin host build (project-chip#3990)

#### Problem

Some conversions to use PacketBufferHandle (project-chip#3909) broke Darwin builds,
which aren't currently run in CI.

#### Summary of Changes

Fix src/platform/Darwin/BleConnectionDelegateImpl.mm to match the API
change in project-chip#3909.

* Add '-Wextra' to compiler flags (project-chip#3902)

* Implement Level Control Cluster (project-chip#3806)

* New seekbar in Android CHIPTool
* Sample usage in lighting-app/nrfconnect

Signed-off-by: Markus Becker <markus.becker@tridonic.com>

* Fix Rendezvous over BLE after recent changes (project-chip#4012)

PR project-chip#3704 introduced a change that the BLE transport in
RendezvousSession is only initialized when PeerAddress
in RendezvousParams is of type BLE. However, PeerAddress
isn't initialized anywhere. As a result Rendezvous over BLE
stopped working between Android CHIPTool and accessories.

Btw, remove an assert related to the storage delegate
as it seems an optional member of the device controller.

* [thread] fix invalid configuration of active dataset (project-chip#4008)

* Fix data loss or crash in TCPEndPoint with LwIP (project-chip#4022)

* Fix data loss or crash in TCPEndPoint with LwIP

#### Problem

Under the configuration CHIP_SYSTEM_CONFIG_USE_LWIP, in some cases where
the data size exceeds the TCP window size, TCPEndPoint can either die or
lose data when accounting of un-acked data falls out of sync.

#### Summary of Changes

Imported fix from Weave:

* This change removes separate accounting of the unsent
  data position and replaces it with simple counting of
  sent-but-not-acked data and a skip-loop at the start
  of DriveSending().

Fixes project-chip#4013 - Data loss or crash in TCPEndPoint with LwIP

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>

* Update lighting-app gen/ folder with ZAP generated content (project-chip#4010)

* Fix segmentation fault error in response echo message (project-chip#3984)

* Add back Android default build coverage & fix the build (project-chip#3966)

mDNS doesn't build with no device layer. Remove it from the build and
add back the coverage that would catch this that was lost in project-chip#3340.

* Update all-clusters-app gen/ folder with ZAP generated content (project-chip#3963)

* Move src/inet/tests to auto-test-driver generation (project-chip#3997)

* Rename TestUtils to UnitTestRegistration. (project-chip#4021)

Looking to remove usage of 'Utils' unless we have really no choice.
'UnitTestRegistration' seems clearer in what it does compared to
'TestUtils'.

* Update src/lib/core/tests to auto-test-driver generation (project-chip#3991)

* Cleanup zap chip-helper.js (project-chip#3973)

* Cleanup zap chip-helper.js

* Restyled by clang-format

Co-authored-by: Restyled.io <commits@restyled.io>

* Move src/transport/tests to auto-test-driver generation (project-chip#3999)

* Move src/transport/tests to auto-test-driver generation

* Add relevant libraries (and more test-capable libs) to nrf.

* Refactor inet test helpers (to not include actual inet tests), try to make qemu allow better linkage but still failed for transport tests so disabled for now

* Added more tests on esp32 qemu

* Restyle fixes

* Fix cast errors in InetCommon

* Disable raw tests from zephyr: somehow they fail running out of endpoints

* Disable DNS test on zephyr

* Remove inet endpoint test from zephyr

* Remove inet endpoint test from zephyr - fix again

* Modify gitignore

* Restyle fixes

* Use CHIPDeviceController instead of CHIPDeviceController_deprecated (project-chip#3979)

* Implement the missing part of Exchange Header in Transport layer (project-chip#4017)

* Implement the missing part of Exchange Header in Transport layer

* Revert comment 'if' back to 'iff'("if and only if")

* Remove duplicated send flag defines and put ExchangeMgr/ExchangeConte… (project-chip#3994)

* Remove duplicated send flag defines and put ExchangeMgr/ExchangeContext under the same namespace as CRMP

* Rename kSendFlag_Default to kSendFlag_None

* Move src/lib/asn1/tests and src/ble/tests to auto-test-driver generation (project-chip#3998)

* Move src/lib/asn1/tests and src/ble/tests to auto-test-driver generation

* Remove one more unused file

* Attempt to enable asn1 and ble tests in esp32 - see if they pass or not

* Fix merge error

* Update include header for ASN1 test

* Include  ASN1 in libCHIP

* Some conversions to use PacketBufferHandle (project-chip#4011)

* Some conversions to use PacketBufferHandle

#### Problem

Code should use `PacketBufferHandle` rather than `PacketBuffer *`.

#### Summary of Changes

- Converts remaining receive path in //src/inet and //src/transport.
- Converts most of //src/ble.
- Introduces Handle versions of the `AddToEnd`/`DetachTail` pair.

Part of issue project-chip#2707 - Figure out a way to express PacketBuffer ownership in the type system

* Restyled by clang-format

* review

* revive BtpEngine::Clear[TR]xPacket()
* simplify conditional
* (void) message.DetachTail() → message.FreeHead()
* remove ExchangeContext::kSendFlag_RetainBuffer
* missed pBuf.IsNull()
* DetachHead() → PopTail()
* typos

Co-authored-by: Restyled.io <commits@restyled.io>

* Move src/system/tests to auto-test-driver generation (project-chip#4000)

* Move src/system/tests to auto-test-driver generation

* Remove a TCP/IP init call that was killing qemu

* Remove explicit "all" target from root build file (project-chip#3967)

The "all" target exists implicitly and contains everything. We don't
need to specify one, and it's misleading to do so specifying deps has no
effect.

* Make src/setup_payload compile with -Werror=conversion (project-chip#4032)

* Add SSID and password to chip-tool pairing (project-chip#4054)

* Get temperature-measurement and all-clusters-app to use examples/common/chip-app-server (project-chip#4039)

#### Problem

PR project-chip#3704 introduced a change where a `PeerAddress` is now required in order to start `RendezvousSession`.
Sadly the multiple code paths bootstrapping `RendezvousSession` has not been updated.

PR project-chip#4012 add a fix for some of the `examples/` but not for the `all-clusters-app` nor the `temperature-measurement-app`.

To avoid such situation, this PR merge `examples/common/chip-app-server` and the custom code from `all-clusters-app` and `temperature-measurement-app`.

One of the more discutable change of this PR (imo) is the code that moves the custom `Echo` mechanism from the `all-clusters-app` to `chip-app-server`. I was hoping to get rid of it before doing this change but the `all-clusters-app` and the `temperature-measurement-app` are broken since project-chip#3704 and this PR should fix that.
Also I have a PR (mostly) ready once project-chip#3979 lands to get rid of this `Echo` specific code and replace it by a manufacturer specific `ping` command.

 #### Summary of Changes
 * Remove `EchoServer.cpp`, `RendezvousDeviceDelegate.cpp` and `include/RendezvousDeviceDelegate.h` from `all-clusters-app`
 * Remove `ResponseServer.cpp`, `RendezvousDeviceDelegate.cpp` and `include/RendezvousDeviceDelegate.h` from `temperature-measurement-app`
 * Introduce `chip-app-server/include/AppDelegate.h` in order to keep the behavior the `all-clusters-app` that turns on/off leds on different events. Maybe it should be converted to some types of `ChipDeviceEvent` or `CHIPCallback` at some point.
 * Fix `chip-app-server` to accomodate for the specifics of `all-clusters-app`

* Add all Thread ULA addresses to the lwip interface (project-chip#4053)

ULA prefixes will used for CHIP network so we need to add all these
addresses to the interface.

* Remove src/lib/message. (project-chip#4055)

* Remove src/lib/message.

Updated messaging implementation lives in src/messaging and the
src/lib/message is not currently compiled or linked in.

This saves us from looking at this code when some refactoring is needed
(e.g. the SystemPacketBuffer changes).

* Remove one more unused file

* [ChipTool] Add Payload Parse Command (project-chip#3696)

* [ChipTool] Add Payload Parse Command

* [ChipTool] Add Payload Parse Command

* [ChipTool] Restyle issues resolved

* Restyled by whitespace

* Resolve build errors caused by Command.h schema change

Co-authored-by: lijayaku <lijayaku@amazon.com>
Co-authored-by: Restyled.io <commits@restyled.io>

* rename ParseCommand to QRCodeParseCommand

* adding AdditionalDataParseCommand version 0

* Adding parsing logic + logging

* adding another parsing method

* Basic Parsing is DONE

* fixing memory issue

* removing some logs

* removing more logs

* minor update

* Add RotatingDeviceId to DNS-SD

* Revert "Merge pull request #4 from hnnajh/rotating-id-test"

This reverts commit 0235d05, reversing
changes made to 3e1a4b9.

* increment lifetime counter

* Incrementing lifetime counter at each BLE/MDNS advertisement

* address comments

* Update src/include/platform/ConfigurationManager.h

Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>

* removing unnecessary lines

* nit

* Update src/app/server/Dnssd.cpp

Co-authored-by: chrisdecenzo <61757564+chrisdecenzo@users.noreply.github.com>

* restyling

* Update src/app/server/Dnssd.cpp

Co-authored-by: chrisdecenzo <61757564+chrisdecenzo@users.noreply.github.com>

* Update src/app/server/Dnssd.cpp

Co-authored-by: chrisdecenzo <61757564+chrisdecenzo@users.noreply.github.com>

* fix comments

* incrementing lifetimecounter once

* remove unnecessary include

Co-authored-by: Kevin Schoedel <67607049+kpschoedel@users.noreply.github.com>
Co-authored-by: Vivien Nicolas <vnicolas@apple.com>
Co-authored-by: Markus Becker <Markus.Becker@tridonic.com>
Co-authored-by: Damian Królik <66667989+Damian-Nordic@users.noreply.github.com>
Co-authored-by: Łukasz Duda <lukasz.duda@nordicsemi.no>
Co-authored-by: Restyled.io <commits@restyled.io>
Co-authored-by: Yufeng Wang <44623591+yufengwangca@users.noreply.github.com>
Co-authored-by: Michael Spang <spang@google.com>
Co-authored-by: Andrei Litvin <andrei@andy314.com>
Co-authored-by: jepenven-silabs <67962328+jepenven-silabs@users.noreply.github.com>
Co-authored-by: Boris Zbarsky <bzbarsky@apple.com>
Co-authored-by: Jiacheng Guo <gjc@google.com>
Co-authored-by: Liju Jayakumar <26148162+lijujayakumar@users.noreply.github.com>
Co-authored-by: lijayaku <lijayaku@amazon.com>
Co-authored-by: chrisdecenzo <61757564+chrisdecenzo@users.noreply.github.com>
mspang pushed a commit that referenced this pull request Feb 18, 2022
* Fix shell app build; separate project configs for each project; add QPG shell app to CI

* Update qpg_sdk with changed file

* Remove last old config include points

* Disable stack lock tracking in shell app (#3)

* Fix chip-gn build (#4)

* Disable stack lock tracking in shell app

* Fix include for lib build

Co-authored-by: Timothy Maes <timothy.maes@qorvo.com>
mspang added a commit that referenced this pull request Jul 7, 2022
It's not safe to access line editing state from the IO thread while
inside readline() on the main thread.

Remove the code that attempts to redraw readline after printing logs.
This avoids segfaults during logging at the cost of those logs
overwriting the prompt (this is not trivial to fix as readline
is a blocking API).

==================
WARNING: ThreadSanitizer: data race (pid=63005)
  Write of size 1 at 0x55f81c7745ff by main thread:
    #0 InteractiveStartCommand::ParseCommand(char*) ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 (chip-tool+0x874911)
    #1 InteractiveStartCommand::RunCommand() ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:85 (chip-tool+0x874594)
    #2 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:408 (chip-tool+0x83e478)
    #3 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #4 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #5 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #6 main <null> (chip-tool+0x569c0a)

  Previous read of size 1 at 0x55f81c7745ff by thread T5 (mutexes: write M185):
    #0 LoggingCallback ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:46 (chip-tool+0x874479)
    #1 chip::Logging::LogV(unsigned char, unsigned char, char const*, __va_list_tag*) ../../src/lib/support/logging/CHIPLogging.cpp:221 (chip-tool+0x8ee4dc)
    #2 chip::Logging::Log(unsigned char, unsigned char, char const*, ...) ../../src/lib/support/logging/CHIPLogging.cpp:172 (chip-tool+0x8ee30a)
    #3 chip::app::ReadClient::RefreshLivenessCheckTimer() <null> (chip-tool+0x8b1746)
    #4 chip::app::ReadClient::ProcessSubscribeResponse(chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:845 (chip-tool+0x8b20ec)
    #5 chip::app::ReadClient::OnMessageReceived(chip::Messaging::ExchangeContext*, chip::PayloadHeader const&, chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:409 (chip-tool+0x8ae2a4)
    #6 chip::Messaging::ExchangeContext::HandleMessage(unsigned int, chip::PayloadHeader const&, chip::BitFlags<chip::Messaging::MessageFlagValues, unsigned int>, chip::System::PacketBufferHandle&&) <null> (chip-tool+0xa0517a)
    #7 operator()<chip::Messaging::ExchangeContext> ../../src/messaging/ExchangeMgr.cpp:219 (chip-tool+0xa08c73)
    #8 Call ../../src/lib/support/Pool.h:126 (chip-tool+0xa0912d)
    #9 chip::internal::HeapObjectList::ForEachNode(void*, chip::Loop (*)(void*, void*)) ../../src/lib/support/Pool.cpp:127 (chip-tool+0x8ee05a)
    #10 ForEachActiveObject<chip::Messaging::ExchangeManager::OnMessageReceived(const chip::PacketHeader&, const chip::PayloadHeader&, const chip::SessionHandle&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&)::<lambda(auto:2*)> > ../../src/lib/support/Pool.h:396 (chip-tool+0xa08d10)
    #11 chip::Messaging::ExchangeManager::OnMessageReceived(chip::PacketHeader const&, chip::PayloadHeader const&, chip::SessionHandle const&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&) ../../src/messaging/ExchangeMgr.cpp:212 (chip-tool+0xa07e91)
    #12 chip::SessionManager::SecureUnicastMessageDispatch(chip::PacketHeader const&, chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:616 (chip-tool+0xa1548b)
    #13 chip::SessionManager::OnMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:443 (chip-tool+0xa14426)
    #14 chip::TransportMgrBase::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/TransportMgrBase.cpp:76 (chip-tool+0xa17dfa)
    #15 chip::Transport::Base::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/raw/Base.h:102 (chip-tool+0xb19728)
    #16 chip::Transport::UDP::OnUdpReceive(chip::Inet::UDPEndPoint*, chip::System::PacketBufferHandle&&, chip::Inet::IPPacketInfo const*) ../../src/transport/raw/UDP.cpp:122 (chip-tool+0xb1a48b)
    #17 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>) ../../src/inet/UDPEndPointImplSockets.cpp:688 (chip-tool+0xb00aa0)
    #18 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>, long) ../../src/inet/UDPEndPointImplSockets.cpp:569 (chip-tool+0xafff89)
    #19 chip::System::LayerImplSelect::HandleEvents() ../../src/system/SystemLayerImplSelect.cpp:406 (chip-tool+0xb07563)
    #20 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:181 (chip-tool+0x98a227)
    #21 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #22 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Location is global '(anonymous namespace)::gIsCommandRunning' of size 1 at 0x55f81c7745ff (chip-tool+0x000000c485ff)

  Mutex M185 (0x55f81c776180) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4240 (libtsan.so.0+0x4f30a)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_LockChipStack() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:78 (chip-tool+0x989e90)
    #2 chip::DeviceLayer::PlatformManager::LockChipStack() ../../src/include/platform/PlatformManager.h:410 (chip-tool+0x988fa5)
    #3 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:170 (chip-tool+0x98a147)
    #4 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #5 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Thread T5 (tid=63013, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x5ad75)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_StartEventLoopTask() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:231 (chip-tool+0x98a40a)
    #2 chip::DeviceLayer::PlatformManager::StartEventLoopTask() ../../src/include/platform/PlatformManager.h:375 (chip-tool+0xaacca2)
    #3 chip::Controller::DeviceControllerFactory::ServiceEvents() ../../src/controller/CHIPDeviceControllerFactory.cpp:331 (chip-tool+0xab0417)
    #4 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:403 (chip-tool+0x83e353)
    #5 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #6 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #7 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #8 main <null> (chip-tool+0x569c0a)

SUMMARY: ThreadSanitizer: data race ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 in InteractiveStartCommand::ParseCommand(char*)
==================
mspang added a commit that referenced this pull request Jul 11, 2022
It's not safe to access line editing state from the IO thread while
inside readline() on the main thread.

Remove the code that attempts to redraw readline after printing logs.
This avoids segfaults during logging at the cost of those logs
overwriting the prompt (this is not trivial to fix as readline
is a blocking API).

==================
WARNING: ThreadSanitizer: data race (pid=63005)
  Write of size 1 at 0x55f81c7745ff by main thread:
    #0 InteractiveStartCommand::ParseCommand(char*) ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 (chip-tool+0x874911)
    #1 InteractiveStartCommand::RunCommand() ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:85 (chip-tool+0x874594)
    #2 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:408 (chip-tool+0x83e478)
    #3 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #4 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #5 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #6 main <null> (chip-tool+0x569c0a)

  Previous read of size 1 at 0x55f81c7745ff by thread T5 (mutexes: write M185):
    #0 LoggingCallback ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:46 (chip-tool+0x874479)
    #1 chip::Logging::LogV(unsigned char, unsigned char, char const*, __va_list_tag*) ../../src/lib/support/logging/CHIPLogging.cpp:221 (chip-tool+0x8ee4dc)
    #2 chip::Logging::Log(unsigned char, unsigned char, char const*, ...) ../../src/lib/support/logging/CHIPLogging.cpp:172 (chip-tool+0x8ee30a)
    #3 chip::app::ReadClient::RefreshLivenessCheckTimer() <null> (chip-tool+0x8b1746)
    #4 chip::app::ReadClient::ProcessSubscribeResponse(chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:845 (chip-tool+0x8b20ec)
    #5 chip::app::ReadClient::OnMessageReceived(chip::Messaging::ExchangeContext*, chip::PayloadHeader const&, chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:409 (chip-tool+0x8ae2a4)
    #6 chip::Messaging::ExchangeContext::HandleMessage(unsigned int, chip::PayloadHeader const&, chip::BitFlags<chip::Messaging::MessageFlagValues, unsigned int>, chip::System::PacketBufferHandle&&) <null> (chip-tool+0xa0517a)
    #7 operator()<chip::Messaging::ExchangeContext> ../../src/messaging/ExchangeMgr.cpp:219 (chip-tool+0xa08c73)
    #8 Call ../../src/lib/support/Pool.h:126 (chip-tool+0xa0912d)
    #9 chip::internal::HeapObjectList::ForEachNode(void*, chip::Loop (*)(void*, void*)) ../../src/lib/support/Pool.cpp:127 (chip-tool+0x8ee05a)
    #10 ForEachActiveObject<chip::Messaging::ExchangeManager::OnMessageReceived(const chip::PacketHeader&, const chip::PayloadHeader&, const chip::SessionHandle&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&)::<lambda(auto:2*)> > ../../src/lib/support/Pool.h:396 (chip-tool+0xa08d10)
    #11 chip::Messaging::ExchangeManager::OnMessageReceived(chip::PacketHeader const&, chip::PayloadHeader const&, chip::SessionHandle const&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&) ../../src/messaging/ExchangeMgr.cpp:212 (chip-tool+0xa07e91)
    #12 chip::SessionManager::SecureUnicastMessageDispatch(chip::PacketHeader const&, chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:616 (chip-tool+0xa1548b)
    #13 chip::SessionManager::OnMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:443 (chip-tool+0xa14426)
    #14 chip::TransportMgrBase::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/TransportMgrBase.cpp:76 (chip-tool+0xa17dfa)
    #15 chip::Transport::Base::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/raw/Base.h:102 (chip-tool+0xb19728)
    #16 chip::Transport::UDP::OnUdpReceive(chip::Inet::UDPEndPoint*, chip::System::PacketBufferHandle&&, chip::Inet::IPPacketInfo const*) ../../src/transport/raw/UDP.cpp:122 (chip-tool+0xb1a48b)
    #17 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>) ../../src/inet/UDPEndPointImplSockets.cpp:688 (chip-tool+0xb00aa0)
    #18 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>, long) ../../src/inet/UDPEndPointImplSockets.cpp:569 (chip-tool+0xafff89)
    #19 chip::System::LayerImplSelect::HandleEvents() ../../src/system/SystemLayerImplSelect.cpp:406 (chip-tool+0xb07563)
    #20 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:181 (chip-tool+0x98a227)
    #21 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #22 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Location is global '(anonymous namespace)::gIsCommandRunning' of size 1 at 0x55f81c7745ff (chip-tool+0x000000c485ff)

  Mutex M185 (0x55f81c776180) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4240 (libtsan.so.0+0x4f30a)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_LockChipStack() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:78 (chip-tool+0x989e90)
    #2 chip::DeviceLayer::PlatformManager::LockChipStack() ../../src/include/platform/PlatformManager.h:410 (chip-tool+0x988fa5)
    #3 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:170 (chip-tool+0x98a147)
    #4 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #5 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Thread T5 (tid=63013, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x5ad75)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_StartEventLoopTask() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:231 (chip-tool+0x98a40a)
    #2 chip::DeviceLayer::PlatformManager::StartEventLoopTask() ../../src/include/platform/PlatformManager.h:375 (chip-tool+0xaacca2)
    #3 chip::Controller::DeviceControllerFactory::ServiceEvents() ../../src/controller/CHIPDeviceControllerFactory.cpp:331 (chip-tool+0xab0417)
    #4 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:403 (chip-tool+0x83e353)
    #5 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #6 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #7 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #8 main <null> (chip-tool+0x569c0a)

SUMMARY: ThreadSanitizer: data race ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 in InteractiveStartCommand::ParseCommand(char*)
==================
mspang pushed a commit that referenced this pull request May 29, 2024
* Rerouted tracing macros to Darwin signposts

* Initial framework for logging scalar data event

* Handled the new metrics event changes in collector

* Modified VerifyOrExit macro to accept an optional metric key as third argument

* Removed direct use of chrono in metrics_event.h
Switched MTRMetrics to vend dictionary for metric keys

* Modified SuccessOrExit to optionally accept metric key

* Moved metric keys to separate header
Reworked metric_event names for more clarity

* Switched MATTER_TRACE_METRIC usage to MATTER_LOG_METRIC

* Restyle fixes

* Fixed unit tests

* Fixed build failure due to MetricEvent hidden inside tracing enabled

* Fixing one source of build error

* Fixing darwin build failure

* Code Review: Rename LogMetric to LogMetricEvent

* Code Review Suggestions:

1. Metric Macros take full string constants and no longer use
   preprocessor to prefix. Allows free flowing strings
2. Reworked MetricEvent class and documented
3. Handled LogEventMetric for Darwin, ESP32, Perfetto, JSON to account
   for all types
4. Removed timePoint from MetricEvent class. Timestamps and duration
   calculation is now responsibility for the handlers of the event
5. Reverted BUILD.gn in system to not break out SystemClock.h

* Code Review Feedback #2:

1. Added SuccessOrExitWithMetric and VerifyOrExitWithMetric
2. Cleaned up support .gn to remove dependedency on metrics

* Code Review Feedback #3:

1. Added ScopedMetricEvent to use RAII to track begin and end within a
   scope

* Code Review #4:

Reverted an accidental removal

* Added MTRMetricData to description as per review comment

* Restyler fixes

* Sample code of how Begin and End log metrics can be used

* Fixed compilation error when tracing is disabled

* Fixes for build failures when tracing is disabled

* Picked up code review suggestion accidently dropped

* Code Review Feedback:

1. Begin metric does not take value
2. Allow undefined value for metric
3. Misc other feedback

* Handle undefined value and error value

* Revert a comment change

* Review Feedback: Changed ScopedMetricEvent to capture error by reference

* Fixed another build failure

* Reverting usage of LOG_METRICS

* Review feedback: Fix incorrect documentation

* Code Review Feedback: Remove access to Value in MetricEvent to avoid incorrect access

* Restyler fixes

* Unregistering backend in Darwin shutdown

* Resytler fixes...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.