make test target broken

[grevaillot]

Hi,

Apparently, unitary tests are broken - i've no idea of the current level of brokenness, but, at least they do not run anymore due to some binary path change. I think this was a recent modification.

% make test
[DEBUG]
[ 22%] Built target stlink
[ 27%] Built target st-info
[ 36%] Built target st-flash
[ 58%] Built target stlink-static
[ 69%] Built target st-util
[ 75%] Built target stlink-gui
[ 80%] Built target stlink-gui-local
[ 88%] Built target flash
[ 94%] Built target sg
[100%] Built target usb
Running tests...
Test project /home/moot/Developpement/tools/stlink/build/Debug
Start 1: usb
Could not find executable /home/moot/Developpement/tools/stlink/build/Debug/tests/usb
Looked in the following places:
/home//Developpement/tools/stlink/build/Debug/tests/usb
/home//Developpement/tools/stlink/build/Debug/tests/usb
/home/*/Developpement/tools/stlink/build/Debug/tests/Release/usb
...
Unable to find executable: /home/moot/Developpement/tools/stlink/build/Debug/tests/usb

etc..

i'm running current develop branch on debian testing.

[chenguokai]

Reproducible on macOS.
The master branch is only broken with Test 3, while the develop branch is much more broken.

[Nightwalker-87]

Thx for reporting. Interesting... I'll do some research with older commits to find out. No clue yet where it exactly happened and why.

[chenguokai]

On the develop branch, the built test binaries resident in build/Debug/bin.
On the master branch, they are in build/Debug/tests.

[Nightwalker-87]

To me it would actually make sense to place them in build/Debug/bin/tests.

Can't really spot a reason for this in /tests/CMakeLists.txt or our Makefile which was my initial guess.

[Nightwalker-87]

Test 3 already fails on v1.3.1, so nobody seems to have taken any notice thereof... No clue on that.

This merge-commit 946b7f3 introduced the failure of tests 1 and 2 on develop. Sry for that. 😞 Further investigation reveals commit 587fb48 causes the problem, more precisely /CMakeLists.txt with the lines:

#set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
#set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
#set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)

They should either be deleted again or related settings aligned to them in order to match.

[Nightwalker-87]

The above lines are commented-out in branch "build-settings" and thus the two first tests are functional again. Had a look at flash.c again and found the following in /build/Debug/Testing/Temporary/LastTest.log:

3/3 Testing: flash
3/3 Test: flash
Command: "/$PATH$/git/stlink/build/Debug/tests/flash"
Directory: /$PATH$/git/stlink/build/Debug/tests
"flash" start time: Apr 17 02:07 CEST
Output:
----------------------------------------------------------
*** Error: Expected args for this command: read <path> <addr> <size>
*** Error: Expected args for this command: write <path> <addr>
*** Error: Expected args for this command: write <path> <addr>
*** Error: Expected args for this command: write <path>
*** Error: Invalid value for addr
[OK] (-1) 
[ERROR] (-1) --debug --reset read /dev/sg0 test.bin 0x80000000 0x1000
[ERROR] (-1) --debug --reset write /dev/sg0 test.bin 0x80000000
[OK] (-1) --serial A1020304 /dev/sg0 erase
[ERROR] (-1) /dev/sg0 erase
[OK] (0) --debug --reset read test.bin 0x80000000 0x1000
[OK] (0) --debug --reset write test.bin 0x80000000
[OK] (0) erase
[OK] (0) --debug --reset --format=ihex write test.hex
[OK] (-1) --debug --reset --format=binary write test.hex
[OK] (-1) --debug --reset --format=ihex write test.hex 0x80000000
[OK] (-1) --debug --reset write test.hex sometext
[ERROR] (0) --serial A1020304 erase
[ERROR] (0) --serial=A1020304 erase
<end of output>
Test time =   0.00 sec

Can't go back any further than v1.3.1 to determine exactly where it first appeared as build fails on v1.3.0 and v1.2.0 used autotools for building. However this shows that arguments are missing (maybe due to a change in the code where someone forgot to update the code for the files in /tests as well...)

Who can help fixing this?

[chenguokai]

Maybe I have figured out the breakage.

The first two breakages are caused by one commit between master and develop.

[ERROR] (-1) --debug --reset read /dev/sg0 test.bin 0x80000000 0x1000
[ERROR] (-1) --debug --reset write /dev/sg0 test.bin 0x80000000

The possible reason is the removal of actual usage of devname in struct flash_opts. In master branch, src/tools/flash_opts.c and src/tools/flash.c deal with devname. In the develop branch, they are removed.

Edit: commit that cause this breakage found:

git bisect bad
fc507fd4842cd9b4719378d0ee79a6a123eecae3 is the first bad commit
commit fc507fd4842cd9b4719378d0ee79a6a123eecae3
Author: Guillaume Revaillot <g.revaillot@ppp0.dev>
Date:   Sun Apr 12 15:58:17 2020 +0200

    flash.c: stlink_open_usb is actually able to open v1 and v2 (and v3 ..) devices.
    
    no need to check that _not used_ o.devname ..
    no need to specify devname in commandline
    
    get rid of the devname cmdline stuff, and update comments

:040000 040000 4be448e0310e8e7bdef30fd43af625fe84a37110 ad7c5a6150fcbb736bf2431130ae2f17fa2d943f M	src

The last two breakages are caused by the change of serial argument layout in memory. The latest serial buffer begins with no leading zeros which the test expects.

With this patch, all tests passed on master branch.

[Nightwalker-87]

Good job @chenguokai! I can verify that you are right with the first two failures and also tested the second issue successfully. So indeed this fixes that bug. I'll include the fix soon. Thank you.