This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Flashing an Arduino Leonardo with AVRDUDE is troublesome #788
Comments
dl8dtl
added
enhancement
|
The way the second device comes up is very OS-specific. I'd like to avoid that kind of OS hacks in AVRDUDE itself, as it opens a can of worms. So far, AVRDUDE suffers only from very few (low-level) things that are really OS specific, as for most of the required abstractions, it can sit on top of the respective backend libraries (libusb, hidapi and such).
Obviously, that script is inherently OS specific. |
|
Hi @dl8dtl , You basically suggest to defer the first two steps (1. resetting device and 2. waiting for it to return with a new serial port) to an OS-specific script outside AVRDUDE. A good idea, as this keeps AVRDUDE clean from OS-specific stuff. A few questions come to my mind:
Kind regards, |
|
Hi Kristof, while I agree that it would be a nice addition to AVRDUDE, it is not quite trivial to do this cross-platform. I did this for Windows, check out The code for doing the auto-reset is here, it is just about 1k lines of non-portable code: |
|
Hi @mariusgreuel , |
Puh. I'm not sure whether I'd like to have that in a supposed-to-be portable application. |
mariusgreuel
changed the title
|
As a compromise, would it be an idea to create a separate "1200 bps touch" handler that would contain all the non-portable code one could call along with avrdude? The boards I know of that are currently "impossible" to flash directly using Avrdude because of this is the Arduino Leonardo, Arduino Micro/Pro Micro, and the Arduino Nano Every. I believe making this work on Windows is the most difficult part, but @mariusgreuel has already figured out this part. At least on my mac, the serial port always has the same name after a disconnect. The only time the name changes is when I use a different USB port. The command below is just to illustrate what I mean. The idea Here's the python based logic PlatformIO uses: before_ports = get_serial_ports()
if upload_options.get("use_1200bps_touch", False):
env.TouchSerialPort("$UPLOAD_PORT", 1200)
if upload_options.get("wait_for_upload_port", False):
env.Replace(UPLOAD_PORT=env.WaitForNewSerialPort(before_ports))def TouchSerialPort(env, port, baudrate):
port = env.subst(port)
print("Forcing reset using %dbps open/close on port %s" % (baudrate, port))
try:
s = Serial(port=port, baudrate=baudrate)
s.setDTR(False)
s.close()
except: # pylint: disable=bare-except
pass
sleep(0.4) # DO NOT REMOVE THAT (required by SAM-BA based boards)
def WaitForNewSerialPort(env, before):
print("Waiting for the new upload port...")
prev_port = env.subst("$UPLOAD_PORT")
new_port = None
elapsed = 0
before = [p["port"] for p in before]
while elapsed < 5 and new_port is None:
now = [p["port"] for p in util.get_serial_ports()]
for p in now:
if p not in before:
new_port = p
break
before = now
sleep(0.25)
elapsed += 0.25
if not new_port:
for p in now:
if prev_port == p:
new_port = p
break
try:
s = Serial(new_port)
s.close()
except SerialException:
sleep(1)
if not new_port:
sys.stderr.write(
"Error: Couldn't find a board on the selected port. "
"Check that you have the correct port selected. "
"If it is correct, try pressing the board's reset "
"button after initiating the upload.\n"
)
env.Exit(1)
return new_port |
I have the Arduino Leonardo and I am getting the Arduino Pro Micro and Arduino Nano Every soon. So this is interesting. |
|
This is what Arduino is doing: run log under Windows with a clone Pro Micro with ATmega32U4. |
|
Ref: https://www.avrfreaks.net/forum/uploading-code-new-arduino-nano-every More details here: |
|
@mariusgreuel I tried your branch ( https://github.com/mariusgreuel/avrdude) and it seems to work for me for the Pro Micro clone. Run log with a Pro Micro clone. |
|
@mariusgreuel I tried your branch ( https://github.com/mariusgreuel/avrdude) under Windows but somehow it does not work for my official Arduino Leonardo as it stuck there forever. It does work under Arduino. |
|
@MCUdude It seems that Arduino does more things for Arduino Leonardo than to the Arduino Nano Every. The reason is to switch COM port, that is not necessary for Arduino Nano Every. But you are right, macOS will not change the serial port number name. Arduino Nano Every (official version): Arduino Leonardo (official version) |
|
This is Arduino is doing under macOS (Mac Mini M1). |
|
Ref: this one seems to offer different solutions for different OS. Under macOS, the following shell script file works for me. It also has the script for Linux using Python/pyserial. This does not seem to work for me under macOS. Minor modification makes the above work under both macOS (Mac Mini M1) and Linux (Ubuntu 20.04 x86_64).. Windows batch file using WMIC -- not working for me yet, need to try again. |
|
It would be really nice if we somehow could find a way to implement the whole |
Nice idea. Not so sure if extra dependancies like libserialport can be useful here.. If this is not acceptable, another way is do document the scripts or application to be used to get avrdude to work under different OS like Windows, Linux, macOS and FreeBSD. |
|
The following mod of the Windows batch file below seems to work. The trick is to add a delay to wait for Windows to re-enumerate the new bootloader COM port. |
|
RUN log using the above modified batch file. |
|
Found another batch file which may work better. Run log: |
@mariusgreuel Maybe the above a false alarm. I tried again today under Windows 11 and your repo works fine. |
|
c/c++ based 1200bps helper for Windows and Linux |
I looked at the Windows implementation, and I have to say it is a very poor one: It is simply looking at the available COM ports, and not trying to find a matching one, not even looking at the VID/PID. For instance, on my PC I have a serial port on my motherboard, so when plugging in a Leonardo I would have two COM ports, which breaks the whole thing... |
|
Curious, I'm quite a bit away from these Arduinos … Would it somehow help to have a hook in AVRDUDE that allows for calling an external program (in the background?) right before AVRDUDE is going to take over the device? That could then e.g. be a Python script, which is not strictly a part of AVRDUDE, but we could ship it along if needed. |
I am a little reluctant to add device-specific hacks like that. I'm somewhat afraid of running into a maintenance mess, because each supported target board would then have the same right to get whatever is needed for it added. That's why my idea of an external program, that's at least generic enough to handle whatever prerequisite any environment would need. However, I realize that a simple shell script / batch file cannot handle serial ports very well. |
What about a small, external C program? |
Certainly a good idea, and it can easily be made a different one for Windows vs. Posix APIs. If I understand it correctly, some USB stuff is needed, so for the Posix world, it's easy enough to use libusb, while for Windows, I think Marius has already the needed bits and pieces together. |
|
I'd vote for a small program, written in C or similar. (I've no experience with the Go language, but it looks like there's a |
The Windows implementation in my repo may be a little overly complicated, because I tried to not change the avrdude codebase too much, plus I reused some C++ code I had written previously.
I guess it depends on what implementation we pick. I might be jumping ahead here, but I can think of various implementations to do the 1200 baud reset-to-bootloader trick: A)
This requires getting the link between serial devices and USB devices. I think libusb is of limited help here, not sure. B)
In my avrdude repo, I picked A, plus I used a heuristic that tries to infer whether the reset-to-bootloader is needed, based on the fact whether its a composite device (sketch) or not (bootloader). @mcuee did break the heuristic when he used a sketch as an ISP programmer, where you do not want to reset the device, but my heuristic did. So, an explicit option is needed here. B is probably easier to implement and still sufficiently robust. I think finding the serial devices requires a bit of thought, because it requires implicit knowledge of all possible device names. Again, libusb is of limited help here (I think). @mcuee Does libusb provide properties about the USB device that would help getting the mapping between serial devices and the corresponding USB ACM devices? Can libusb provide USB location properties, and if so, for which systems? |
|
@mariusgreuel Personally I like cross-platform stuff and so far I found that pyserial to be the best for this task. The next one is probably libusbp (which may not support FreeBSD and may need some work for MSVC). I do not know if libserial and libserialport can do the job or not. I think maybe we still need two implementation if we do not want to use pyserial, one for Posix and one for Windows. As for Option A, at least under Linux, we may be able to utilize libudev. In the end, Option B may be a simpler approach. |
|
Sigrok libserialport example output. Looks like it can do the job and it support msvc (better than libusbp). It supports Windows, Linux, macOS and FreeBSD. |
Yes.
That seems to be nice but I think C/C++ is probably more preferred than Go. Even Python is more preferred than Go if you asked me. Still it is pretty good as the Go serial library is cross platform (Windows, Linux, macOS, FreeBSD and OpenBSD). |
|
The You already have something written for this exact purpose, actively maintained by the primary manufacturer of the boards that use it and the primary consumer of the package, which is used by Arduino IDE 2.x, and Arduino Create Agent (the local tool used by Arduino Web Editor for facilitating uploads). Why would you want to reinvent the wheel and then maintain it? Go is well suited for this type of application. The amount of wrapper code required to make an application around this package is likely very minimal. |
I vote against using Go, primarily because that would exclude many of us (including me) from being able to work with this code.
Seems like the Arduino people implemented B. Note all the Sleeps() :-). Personally, I think this functionality should be integrated in avrdude. If we are talking external application, we might ditch the entire idea and say 'go and use the Arduino CLI instead'. If we do B, the avrdude codebase already includes most of the code. The only issue left is being able to enumerate all serial devices. I think that is doable. |
|
@mariusgreuel I agree. Even though Go might provide what we need, I'l have to learn Go before being able to contribute.
I'll assume resolving/implementing the feature mentioned in #907 is a part of this task? |
|
My previous comment was specific to the proposal of creating a standalone "1200 bps touch" tool. Obviously it is not relevant to the proposal of providing this capability in AVRDUDE itself.
Is it really more effort than what would be required to reinvent the wheel? |
I think B is good enough. I tend to think sigrok libserialport has all the things we need (to combine the following three examples, for example). The question is whether we want to add one more dependency. |
For that purpose, I'd be fine with it.
Which is not all that of a bad idea, given that this hack is only required for one (or just a few) Arduinos. The "use an external tool" option has the merit that people with specific requirements for activating their own bootloaders could benefit from that as well, so it's not just targeting one Arduino platform only. |
|
arduino-cli is kind of interesting. I have never used it before. I can see from Windows Device Manager that the port changed from COM5 --> COM8 --> COM5. |
|
I'm leaving this here in case I ever forget. For those of you that just need a simple 1200bps touch command that will work on macOS and other *nix systems, EDIT: Or perhaps even better: |
|
Another discussion here: Interesting |
|
For those who are using Arduino Nano Every official version, you may encounter issues like this under Linux. |
|
Not exactly related to this issue but for those who are using Uno WiFi Rev2 (with Arduino custom FW xplainedmini_udpi) due to FW bug. |
|
In the end, I think we will not include this in avrdude. Therefore I will only keep the documentation label. |
|
As for the documentation, since this is very specific to a paticular board, I am thinking that Wiki entry is good enough. I will close this as an avrdude issue and move this to discussion. |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
The
ATmega32u4microcontroller is present on a couple of Arduino boards, such as the Arduino Leonardo and Arduino Micro. The flash procedure for these boards is quite complicated. I'll first review how it works precisely, then I'll explain why this is a problem for many users of AVRDUDE. Finally we can look into potential solutions.1. The flash procedure explained
Unlike others, the Arduino Leonardo and Arduino Micro boards identify themselves through one port (eg.
COM9) while you need to flash them through another (eg.COM10) which is only available for a couple of seconds just after a reset. I observed how the Arduino IDE deals with this case:Open the serial port through which the board has identified itself (eg.
COM9) at baudrate 1200 and close it immediately thereafter. This basically triggers a board reset.Observe the serial ports from the OS (Windows/Linux). As soon as the board has completed the reset, it should present itself with the serial port for flashing (eg.
COM10). Store that port name and go on to the next step.Flash the firmware through the new port (eg.
COM10) with AVRDUDE. Do it quickly, because this serial port is only active for a couple of seconds.Arduino IDE does it this way. PlatformIO also.
2. What is the problem?
The flash procedure explained above consists of three steps. I know how to put the third step in a makefile, but I have no idea how to implement the first two.
I could just hardcode the first two steps in Embeetle IDE and run them behind the scenes before launching the third step. I actually just did that, and it works. But it feels wrong. At Embeetle, we have the desire to keep our users informed and free. In other words: if one of our users decides to work on a project outside Embeetle IDE, he/she is free to do so. We achieve that in two ways:
All Embeetle projects are makefile-based. The user simply needs to invoke
clean,buildandflashtargets to do all the magic.We provide a makefile to each sample project our users can download, but it is not touched thereafter. The makefile is well-explained, with plenty of comments.
Hardcoding (part of) the flash procedure in the IDE itself defeats this goal. The user cannot run his project outside our IDE anymore.
3. Potential solutions
I see two solutions to this dilemma:
We find a way to implement the first two steps in the makefile anyhow. I looked into this with my colleague Johan - who is a true makefile wizard - but we couldn't figure it out. If you have an idea, please let me know.
We improve the AVRDUDE software such that it can handle this particular situation.
I believe the second solution is the cleanest and will help most AVRDUDE users - also those that have nothing to do with Embeetle IDE. I would love to help with the implementation, but I'm afraid I need a little help and guidance, as I'm not familiar with the source code.
Kind regards,
Kristof Mulier
The text was updated successfully, but these errors were encountered: