Programming AVR128DB128 from another chip

[dpetican]

I've previously decoded the STK500v2 protocol and programmed an ESP8266 to program an ATMEGA2560 via bootloader over serial. The v2 message format is complex compared to v1. I now want to do the same for an AVR128DB128 from an ESP32. I've looked at the Optiboot source code and figure I can do this fairly easily. For the ATMEGA extended addressing was required because 256K.

I'm probably just not remembering something because its been almost 4 years since I touched the previous code. The v1 protocol uses only 2 bytes for addresses whereas the v2 protocol uses 4. But I also remember that the ATMEGA1280 used the v1 protocol. Reading the source code for Optiboot leads me to believe that I don't need extended addressing for 128K. But that doesn't seem correct 16bits != 128K of addresses.

If I can figure that out I could probably do this without decoding the serial stream as I upload from Arduino. That was a nightmare. Even though the memory layout of the 128K parts is more complex than an ATMEGA I'm assuming I would just send the bytes as if the address space was fully mapped along with the correct starting address of each 256 byte page (extended if required) and the bootloader would figure out the rest? Thanks.

[SpenceKonde]

The flash is word addressed not byte addressed. So you have 64k words not 64k bytes max with v1. There was some line in avrdude.conf that was needed to make it do that correctly, but shouldn't be a problem for optiboot. That said, i would definitely meditate on why I wasn't using updi directly

…

____________ Spence Konde Azzy’S Electronics New products! Check them out at tindie.com/stores/DrAzzy GitHub: github.com/SpenceKonde ATTinyCore: Arduino support for almost every ATTiny microcontroller Contact: ***@***.***

On Thu, Jul 14, 2022, 14:03 dpetican ***@***.***> wrote: I've previously decoded the STK500v2 protocol and programmed an ESP8266 to program an ATMEGA2560 via bootloader over serial. The v2 message format is complex compared to v1. I now want to do the same for an AVR128DB128 from an ESP32. I've looked at the Optiboot source code and figure I can do this fairly easily. For the ATMEGA extended addressing was required because 256K. I'm probably just not remembering something because its been almost 4 years since I touched the previous code. The v1 protocol uses only 2 bytes for addresses whereas the v2 protocol uses 4. But I also remember that the ATMEGA1280 used the v1 protocol. Reading the source code for Optiboot leads me to believe that I don't need extended addressing for 128K. But that doesn't seem correct 16bits != 128K of addresses. If I can figure that out I could probably do this without decoding the serial stream as I upload from Arduino. That was a nightmare. Even though the memory layout of the 128K parts is more complex than an ATMEGA I'm assuming I would just send the bytes as if the address space was fully mapped along with the correct starting address of each 256 byte page (extended if required) and the bootloader would figure out the rest? Thanks. — Reply to this email directly, view it on GitHub <#298>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTXEWYHMKWLXABRD2RLBTDVUBI7TANCNFSM53TCV73Q> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[dpetican]

The word addressing was the basic thing I forgot. That helps tremendously. Thank you.

As for UPDI vs bootloading my current production boards do have a circuit that allows for various types of resistor/diode combinations for programming the DB from the ESP32. However, its a matter of code re-use and time. Yes UPDI is a cleaner solution and I would prefer it. Next revision of the board is not until the late fall. I will have more time to experiment.

I've looked at the Optiboot code and since I got v2 to work then v1 should be possible. In fact I now realize that even though I was using the original bootloader with the ATMEGA's I probably implemented way too much such as reading fuses etc. BTW the prevous revision has been in production for 3+ years and remote flashing (bootloading) has yet to fail.

BTW you said this regarding bootloading vs UPDI in a previous discussion, but that may have been before the UPDI thing matured:

[https://github.com//discussions/245#discussioncomment-2086444]

[SpenceKonde]

I have updated that comment considerably,in large part due to clear need to make a standalone uploader that would run on another AVR.

[SpenceKonde]

As in, I updated it just now.

[ObviousInRetrospect]

I have a working system updating an avr64db32 from optiboot on an esp32. LMK if you want my optiboot client library. No ESP dependency in particular although I am lazy and have a buffer that holds the full target flash in SRAM.

I spent several days trying to get UPDI to work (I agree it is cleaner. It also saves two pins on the avr) but the Adafruit AVRPRog UPDI library gave me nothing but a wide variety of physical interface errors when running either on an esp32 or a avr64db.

[dpetican]

Thats tremendous. Thank you. On Friday I did a serial dump of avrdude programming the AVR128DB28. For the most part I saw what I expected. However, the actual binary data (page write command payload) for the program itself (not stk commands) looked strange.

Is "strange" expected or some anomoly in my com port monitoring software? Last time I tried this was with the v2 protocol, but that was 4 yesrs ago so...

Also, for AVR128 do I need concern myself where constants are stored in NVRAM? Or is the binary as opposed to intel hex file always monolithic?

[ObviousInRetrospect]

So I pulled this code out of a bigger project. I've used it to do firmware updates over lora a few dozen times at this point, works well. No real trouble once I got the protocol right.

https://github.com/ObviousInRetrospect/OptibootClient

I've so far avoided the 128 so I am afraid I can't answer much there.

[dpetican]

Thanks @ObviousInRetrospect I was able to use your code as a guide to modify my ESP32 program to flash the AVR128DB28... sometimes. A small 3K program (read that this is less than 64K) flashed via ESP32 okay. However, a larger 65K program did not.

[deleted all previous comments]

I think I see the problem. I captured the serial output (directly not through ESP32) while uploading the 65K program and noticed this:

LOAD_ADDRESS: 55 00 FE 20
PROG_PAGE: command plus data
STK_UNIVERSAL: 56 4D 00 01 00 20 (extended address)
LOAD_ADDRESS: 55 00 00 20
PROG_PAGE: command plus data

I read the comments in the Optiboot source code and thought I read extended addressing is not required for 128K parts. My binary is 130 pages of 512 bytes = 66560 and I saw that 2 pages were programmed after the extended address command. However, shouldn't that be 3 pages after since the bootloader takes up the first page? I counted several times and saw only 129 pages being programmed, But it is late now so... I just need to format the debug output from my ESP32 to look a little bit more like the serial capture and do some more debugging after implementing the extended addressing. That should get me further along.

In summary it seems only 64K can be programmed without loading an extended address. In the V2 protocol with a page size of 128 extended addressing is needed for > 128K while with the V1 protocol its needed for > 64K with page size of 512.

BTW I noticed an interesting thing while getting to this point. Trying to explain my process in writing actually helped with the debugging. This was a 2+ hour editing session. Oy!

[ObviousInRetrospect]

You might see only 129 pages being programmed because you don't reprogram the bootloader.

I thought I remembered my inputs to load_address ranging up to 64kish when programming 64k so I was surprised load extended wasn't needed.

What are you using to get the firmware into esp memory? if you are using the hex loader I provided with a hex built against optiboot the offsets will be correct and the first page of the buffer never written. Not sure what the .bin start offset is. If you want it I have a much less readable streaming parser that can process http uploads a chunk at a time.

You might also check whether the contents of page 2 is page 2 or page 3? Or more generally program from the esp then read out the flash from either updi or optiboot running against a computer and then erase, program from the computer, read out, and diff. (The symmetry on program then read will level out the fact that your readout will include the bootloader).

If you get this working I would be interested in seeing what changes were needed as I am running out of avr64db's but can still get 128's.

Re serial ports I happen to run uart0 pointed at the esp and uart1 pointed at a computer for debugging on the db. On the DD (once it actually works in dxcore) I'll run the bootloader off one alt mapping and the debug port off another. At one point I had a mux to allow the ESP to connect to the debug port but I dropped it as it was unused.

[dpetican]

I am using my own library modified from V2 back to V1. Its quite elegant IMHO ;) and integrates with reading pages from the SD card WiFi webserver and sends lots of debugging and status messages back to the SD or to my connected app. Once I get my production code stable I will spin off both the V1 and V2 code into libraries and upload them to Github.

The basic issue I ran into last night was that I was trying to flash (from the ESP32) using a binary built for without Optiboot. Derp! I fixed that today and it works using extended addressing. [deleted memcpy_P comments]

I'm fairly certain that Optiboot will program flash memory up to 128K with AND without using the extended addressing command. I will test this to be sure. But...

I'm still wanting to understand what the comments in the bootloader code really mean. The confusion for me being the word more at the top and the word less at the bottom. Since I'm resetting the address to 0x0000 after loading an extended address of 0x01 RAMPZ must be in effect, but the comments in Optiboot... I don't know really. For some reason I can't paste the code properly but the load extended address command starts at line 533 in https://github.com/SpenceKonde/DxCore/blob/master/megaavr/bootloaders/optiboot_dx/optiboot_dx.c

As for the serial ports I use the exact opposite because I take advantage of MVIO so I don't need level converters for the ESP32 or the I2C GPIO expander I have connected.

Going from a 64K to a 128K part might require either no mods or just one extended address command which is effectively like a bank switch command.

Can you get AVR128DB28 in DIP? Or were you referring to SMT? My next revision of this board will use a larger SMT part and get rid of the GPIO expander anyway, but curious as the DIP parts are hard to come by and they are easier to use for rapid iteration.

[ObviousInRetrospect]

you still need PROGMEM on parts with >32k flash - see https://github.com/SpenceKonde/DxCore/blob/master/megaavr/extras/Ref_PROGMEM.md

I'm a bit confused on the addressing and didn't understand those comments either. SpenceKonde said above that its word addressed but I looked and the load addresses I am using were byte addressed and they are working.

I use the 32-pin tqfp. For quick prototyping I just solder them to a generic breakout that has the decoupling caps + some qwiic ports & serial ports wired. In my case I am running the esp & avr at the same voltage so mvio isn't needed (the current draw at 5mhz+sleep is low enough that a second buck down to 1.8v didn't seem worth the effort, or do you use 5v).

[dpetican]

[deleted memcpy_P comments] Using just const ... PROGMEM as opposed to the other possibilities is desirable as it puts the constants near the start of the binary which should always generate a linear hex file/binary for flashing purposes.

After generating the correct binary for Optiboot usage I was able to program a much greater than 64K program and it ran after flashing. I want to see if I can do the same thing without using extended addressing. The result of this will tell me what the bootloader is expecting. Deciphering the bootloader code is redundant at that point. I'll report back in a day or two on this experiment. I love serial protocol decoding. Especially on 30+ year old CNC machines :D

I'll digress a little bit further into my hardware design in case you are interested since you seem to be making similar boards. My PCB is about 120x180cm and is 4 layers with internal ground and 5V power planes. Power is supplied by a switching regulator module (24V-5V) of the "linear regulator replacement" type with EMC filtering on the input and a 47uF tantalum cap on the output. A linear regulator would generate way too much heat at the 24-5 voltage spread. All digital IC power pins are decoupled with 1.0/0.1uF caps. Adding a 10uF cap would have been pedantic for this board. The ESP32 module is actually the official devkit so I supply it with 5V from the power plane and it uses the onboard regulator to generate 3.3V. The GPIO expander is powered by a 3.3V TO-92 type linear regulator. Of course I ran out of pins and the DB28 which is why I am using the expander, but as it happens the MVIO pins encompass both I2C and serial1. Very convenient. The ESP32 is connected to an SD card reader and RTC for logging messages from the AVR.

I need all the extra space for the control and sensor circuits. This things runs a big machine. The next all SMT board will do away with the GPIO expander and the devkit. I will just use an ESP32 SMT module. So why not go all 3.3V instead of 5V? Well some of my sensors need a 5V reference and power supply. However, everything I have read about the ESP8266/32 says that the ESP current draw varies widely with WiFi power usage. Putting 3.3V on a big power plane might cause a voltage drop to appear at the ESP32 power input if the plane does not have enough capacitance. I might have to sprinkle caps around the board although a huge power plane itself has significant capacitance which is why I always spring for 4 layers for non-trivial designs. 3.3V on the power plane could also generate EMC issues which could affect my sensor circuits or external devices (but unlikely considering the usage environment). Also running an ESP from a switching regulator is not recommended for the same reasons. Better to use a local linear (5V-3.3V/1A) regulator with say a 10uF ceramic on both the input and output with extra smaller caps for decoupling if I want to be pedantic. So I will still use MVIO for the serial connection. Seems logical to me and. What do you think?

I make no warranty about my entire board being EMC compliant but all the I/O and power is routed through one 32pin connector on the long edge. Not input on one side and output on the other which is bad for EMC. So it should be fairly compliant. I hope :D I can't afford EMC testing so I'm just using best practices I've researched on the net even if they are overkill in the end.