Support for cross-platform "1200bps touch" using libserialport

[MCUdude]

Now that #1498 is pretty much complete and hopefully ready to be merged soon, the next step involving libserialport would be to finally support devices that require a serial port "touch" before communication can be established.

To perform "touch" (usually 1200 baud) is as simple as opening the serial port, and waiting a little, just to make sure that the decide in the other end has enough time to jump into "bootloader mode" and close it again.

Some boards, like the Arduino Leonardo or the Arduino Micro, will re-appear with a different USB VID/PID, which may on some OSes result in a new /dev path or COM port number. other boards, like the Arduino Nano Every will not appear as a new device.

Here's how I imagine it can be done if we add a new command line flag that specifies the "touch" baud rate, for instance -r 1200 (please suggest a better-suited flag if you have one. -r just happens to be available).

• The user tries to connect to a serial port and has specified -r, for instance:
  • avrdude -c avr109 -p atmega32u4 -P usb:2341:0058 -b 57600 -r 1200
• Before opening any serial ports, Avrdude creates a list of all available serial ports using libserialport, that contains, their path, USB VID, USB PID, and USB SN.
• If -r is specified, Avrdude opens the serial port (specified using -P) with the baud rate specified by -r.
• Avrdude waits a little, closes the port, and waits a little more.
• Avrdude creates a new list of the available serial ports.
• If a new port has appeared (when compared to the list created earlier), connect to this port using the default baud rate (115200) or the user-specified one (-b). If there are no changes, connect to the previously used port

I doubt this would be very difficult to implement, now that #1498 contains some of the boilerplate code needed to implement this feature.

Any thoughts or ideas?

[mcuee]

Sounds like a great idea and good improvement to avrdude.

Existing discussion:

• Flashing an Arduino Leonardo with AVRDUDE is troublesome #1443

[stefanrueger]

"touch" baud rate

The touch serial interface with specific baud rate seems to be a specific thing for some Arduino boards. Is that necessary for programming these boards irrespective of the programmer or is it tied to a specific way of programming (say, though a particular bootloader)? I am trying to figure out what the best way of specifying this peculiarity is in terms, eg, of

• Command line option
• Separate programmer
• Programmer option
• Using two ids for the same programmer and depending on which id was used doing the outlined touch algorithm or not

Using a new option -r may well be the best way, but I wanted to make sure we thought about all possible alternatives.

[mcuee]

It is tied to a specific type of bootloaders or programmers. It is basically to trigger the reset so that the bootloader or programmer can work.

If you use the other programmers you do not really need the trick, when you have the access to the reset line.

1. Arduino Micro and Leonardo and a few other boards -- with on-board bootloader (avr109 compatible Arduino caterina bootloader). The serial port number will change under Windows due to USB VID/PID change (to be more accurate: USB VID does not change but USB PID changes). The serial port number may or may not change under Linux and macOS.
   FW: https://github.com/arduino/ArduinoCore-avr/tree/master/bootloaders/caterina (USB to Serial + AVR109)

2. Arduino Nano Every -- with on-board jtag2updi programmer. The serial port number will not change under Windows. The FW is rather problematic (buggy) though. It is better to use external programmer.
   FW: https://github.com/arduino/ArduinoCore-megaavr/tree/master/firmwares/MuxTO

3) Potentially other AVR109 bootloaders (plain serial, serial port number will not change) (No such implementation as of now).

Existing discussions:

• Flashing an Arduino Leonardo with AVRDUDE is troublesome #1443
• avrdude with Arduino Nano Every #980

Background info:
https://arduino.github.io/arduino-cli/0.34/platform-specification/#1200-bps-bootloader-reset

[stefanrueger]

@mcuee Thanks, really useful info. It's probably best to follow the lead of @MCUdude in this matter.

[mcuee]

BTW, Arduino Nano ATmega4808 clone does not need this trick, unlike official Arduino Nano Every (ATmega4809).

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_pr1507v5 -C .\avrdude_pr1507v5.conf -c jtag2updi -p m4808 -P ch340
avrdude_pr1507v5: AVR device initialized and ready to accept instructions
avrdude_pr1507v5: device signature = 0x1e9650 (probably m4808)

avrdude_pr1507v5 done.  Thank you.

PS C:\work\avr\avrdude_test\avrdude_bin> .\avrdude_git -c jtag2updi -p m4808 -P ch340
avrdude_git: AVR device initialized and ready to accept instructions
avrdude_git: device signature = 0x1e9650 (probably m4808)

avrdude_git done.  Thank you.

[MCUdude]

@mcuee I can test the Arduino Nano Every on my Windows computer, but I've never built Avrdude on Windows before. You mentioned that you built using MinGW in order to get libserialport working. How did you install libserialport on Windows, and how did you manage to build Avrdude?

I have a "stock" Windows 11 computer I'm using for this, so don't expect anything to be pre-installed.

[mcuee]

@MCUdude

Here is the binary you can use. avrdude Windows binaries are static-linked so it should work for you under Windows 11.
avrdude_pr1507v5.zip

$ ldd ./avrdude_pr1507v5
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffc22ef0000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffc20c80000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffc205c0000)
        ADVAPI32.dll => /c/WINDOWS/System32/ADVAPI32.dll (0x7ffc20d50000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffc22dc0000)
        sechost.dll => /c/WINDOWS/System32/sechost.dll (0x7ffc20ee0000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7ffc22020000)
        SETUPAPI.dll => /c/WINDOWS/System32/SETUPAPI.dll (0x7ffc228d0000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7ffc22720000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7ffc20590000)
        HID.DLL => /c/WINDOWS/SYSTEM32/HID.DLL (0x7ffc1eae0000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7ffc20fb0000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7ffc20470000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7ffc20310000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffc209f0000)
        WS2_32.dll => /c/WINDOWS/System32/WS2_32.dll (0x7ffc20e60000)
        cfgmgr32.DLL => /c/WINDOWS/SYSTEM32/cfgmgr32.DLL (0x7ffc1ff90000)

I will update the Wiki to include libserialport support for most of the build, except MSVC for Windows.

[mcuee]

@MCUdude

Wiki updated.

If you want, you can install MSYS2 mingw64 and then see if you can build the mingw64 binaries under Windows by yourself.
https://github.com/avrdudes/avrdude/wiki/Building-AVRDUDE-for-Windows-using-MSYS2

Installation of MSYS2.
https://www.msys2.org/wiki/MSYS2-installation/

[mcuee]

@MCUdude

I have pushed two commits to MinGW32/64bit github actions so that you can get x86 and x64 version of avrdude binaries under Windows from now on, with libserialport and GNU Readline support.

[MCUdude]

@mcuee I'll see what I can do. I'm currently installing minGW on a Windows computer in order to get the binaries from the CI to work. Since you already have a working Windows setup, can you try to reduce delay from then the port is closed until it's re-opened again? I tried the statically built binary you provide earlier, and it took quite a while for Avrdude to "give up" waiting for a new port to appear. It may be that the Nano Every times out in the mean time.

[mcuee]

@mcuee I'll see what I can do. I'm currently installing minGW on a Windows computer in order to get the binaries from the CI to work. Since you already have a working Windows setup, can you try to reduce delay from then the port is closed until it's re-opened again? I tried the statically built binary you provide earlier, and it took quite a while for Avrdude to "give up" waiting for a new port to appear. It may be that the Nano Every times out in the mean time.

I have tried that but it does not work.

In my Windows 11 system, avrdude exits from the loop very fast. I have tried the default 400ms delay, 200ms delay, 100ms delay and no delay, as well as longer 800ms delay, all have the same results, which means the delay loop is not the issue.

Example: 100ms initial delay
#1507 (comment)

avrdude_pr1507v5_100ms_delay: touching serial port COM6 at 1200 baud
avrdude_pr1507v5_100ms_delay: waiting for new port... using same port COM6

The output is the same for other delays I have tried, which means the for loop is immediately out. So this part of the codes is not the issue, rather somehow Arduino Nano Every is disturbed and becomes in a bad state, after this part of the codes.

  const int nloops = 32, nap = 50;
#if (defined(__arm__) || defined(__aarch64__)) && !defined(__APPLE__)
  nwaits += 2;
#endif
  pmsg_info("waiting for new port...");
  usleep(400*1000*nwaits);
  for(i = nloops; i > 0; i--) {
    usleep(nap*1000);
    if((sp2 = get_libserialport_data(&n2))) {
      diff = sa_spa_not_spb(sp2, n2, sp1, n1);
      if(*diff && diff[0]->port && !diff[1]) { // Exactly one new port sprung up
        pmsg_notice("new port %s discovered\n", (*diff)->port);
        if(*portp)
          free(*portp);
        *portp = cfg_strdup(__func__, (*diff)->port);
        msg_info(" %d ms:", (nloops-i+1)*nap + nwaits*400);
        i = -1;                 // Leave loop
      }
      free(diff); 
      free_libserialport_data(sp2, n2);
    }
  }
  free_libserialport_data(sp1, n1);
  msg_info(" using %s port %s\n", i<0? "new": "same", *portp);

[mcuee]

@MCUdude

It is strange that you mention avrdude tries quite a bit before finding new port, are you testing with Arduino Nano Every when you see this happening?

I got my Arduino Nano Every back in the end of May 2022. But I can not find the date code on the board.

Please help to post your debug log as well, thanks if you are using Arduino Nano Every and see different behavior under your Windows computer. Thanks.

[mcuee]

@MCUdude

Binaries for you to try.

1. no initial 400ms delay at all
   avrdude_pr1507v5_no400ms.zip

$ git diff
diff --git a/src/serialadapter.c b/src/serialadapter.c
index 21a350a9..41ebc8d0 100644
--- a/src/serialadapter.c
+++ b/src/serialadapter.c
@@ -330,7 +330,7 @@ int touch_serialport(char **portp, int baudrate, int nwaits) {
   nwaits += 2;
 #endif
   pmsg_info("waiting for new port...");
-  usleep(400*nwaits*1000);
+  //usleep(400*nwaits*1000);
   for(i = nloops; i > 0; i--) {
     usleep(nap*1000);
     if((sp2 = get_libserialport_data(&n2))) {

2. Not looking for new port at all, no delay.
   avrdude_pr1507v5_no_delay.zip

$ git diff
diff --git a/src/serialadapter.c b/src/serialadapter.c
index 21a350a9..8fb92048 100644
--- a/src/serialadapter.c
+++ b/src/serialadapter.c
@@ -324,7 +324,7 @@ int touch_serialport(char **portp, int baudrate, int nwaits) {
   }
   serial_set_dtr_rts(&fd, 0);
   serial_rawclose(&fd);
-
+/*
   int nloops = 32, nap = 50;
 #if (defined(__arm__) || defined(__aarch64__)) && !defined(__APPLE__)
   nwaits += 2;
@@ -349,7 +349,7 @@ int touch_serialport(char **portp, int baudrate, int nwaits) {
   }
   free_libserialport_data(sp1, n1);
   msg_info(" using %s port %s\n", i<0? "new": "same", *portp);
-
+*/
   return 0;
 }

[mcuee]

@MCUdude

BTW, you can use github actions to build MinGW32/64 binaries with libserialport support.

You may have to merge the git main changes to your branch.

[mcuee]

I have also looked into the history of the commits again and verfied that they are not working with Arduino Nano Every with the on-board jtag2updi programmer, under Windows.

Tested three prior commits and they results are the same.

1. The first commit: 6e4c005

2. Then: efe0649

3. And then: 354a5bb

[MCUdude]

Thanks for trying older commits as well. It is strange that this is only an issue on Windows.
So it didn't even work when using libserialport to open and close the port?
It may be that the way Avrdude closes serial ports under Windows is different than how other programs do it. I have no idea.

Maybe you can look at the Nano Every jtag2updi source code to see if you can spot any obvious things?
https://github.com/arduino/ArduinoCore-megaavr/blob/master/firmwares/MuxTO/MuxTO.ino

[mcuee]

@MCUdude and @stefanrueger

As mentioned in the above, I was very puzzled why this PR does not work, then I looked at the working method again closely, it is :

mode COM12 baud=12 dtr=on

And actually the method mentioned before is actually:

mode COM12 baud=12 dtr=on && mode COM12 baud=12 dtr=off

Then I compare the codes in this PR and it only has

serial_set_dtr_rts(&fd, 0);

Change the line to the following fixed the issue for me.

$ git diff
diff --git a/src/serialadapter.c b/src/serialadapter.c
index 6a5f9d2e..448424a0 100644
--- a/src/serialadapter.c
+++ b/src/serialadapter.c
@@ -322,6 +322,7 @@ int touch_serialport(char **portp, int baudrate, int nwaits) {
     pmsg_error("%s() failed to open port %s at %d baud\n", __func__, *portp, baudrate);
     return -1;
   }
+  serial_set_dtr_rts(&fd, 1);
   serial_set_dtr_rts(&fd, 0);
   serial_rawclose(&fd);

PS> .\avrdude_pr1507v5mod -C .\avrdude_pr1507v5mod.conf -c jtag2updi -p m4809 -P COM12 -r
avrdude_pr1507v5mod: touching serial port COM12 at 1200 baud
avrdude_pr1507v5mod: waiting for new port... using same port COM12
avrdude_pr1507v5mod: AVR device initialized and ready to accept instructions
avrdude_pr1507v5mod: device signature = 0x1e9651 (probably m4809)

avrdude_pr1507v5mod done.  Thank you.

[mcuee]

The change is also good for Arduino Leonardo.

PS> .\avrdude_pr1507v5mod -C .\avrdude_pr1507v5mod.conf -c avr109 -p m32u4 -P COM5 -r
avrdude_pr1507v5mod: touching serial port COM5 at 1200 baud
avrdude_pr1507v5mod: waiting for new port... 650 ms: using new port COM6
avrdude_pr1507v5mod: AVR device initialized and ready to accept instructions
avrdude_pr1507v5mod: device signature = 0x1e9587 (probably m32u4)

avrdude_pr1507v5mod done.  Thank you.

[mcuee]

The change is also good for jtag2updi using Uno CH340 Clone.

PS> .\avrdude_pr1507v5mod -C .\avrdude_pr1507v5mod.conf -c jtag2updi -p m4808 -P COM7 -r
avrdude_pr1507v5mod: touching serial port COM7 at 1200 baud
avrdude_pr1507v5mod: waiting for new port... using same port COM7
avrdude_pr1507v5mod: AVR device initialized and ready to accept instructions
avrdude_pr1507v5mod: device signature = 0x1e9650 (probably m4808)

avrdude_pr1507v5mod done.  Thank you.