SerialUPDI and PlatformIO (long post)

[hexeguitar]

Hey all!
Here is something that some of you might find useful:
platformio.ini file and a custom script that makes automated use of the built in (not implemented in pio, though) prog.py uploader.
Both uploading the firmware and setting the fuses works.

Files

platformio.ini

; PlatformIO template configuration file for megaTinyCore
; https://github.com/SpenceKonde/megaTinyCore/
;
;   Build options: build flags, source filter
;   Hardware options: oscillator type, BOD, EEPROM retain
;   Upload options: custom upload port, speed, and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options
; https://github.com/SpenceKonde/megaTinyCore/blob/master/PlatformIO.md
; https://docs.platformio.org/page/projectconf.html
; https://docs.platformio.org/en/latest/platforms/atmelmegaavr.html

[platformio]
; Default build target
default_envs = Upload_UPDI
; default_envs = set_fuses

; Parameters used for all environments
[env]
platform = atmelmegaavr
framework = arduino

; Chip in use
board = ATtiny806
; Clock frequency in [Hz]
board_build.f_cpu = 20000000L
; Oscillator type (internal or external)
board_hardware.oscillator = internal

; Upload protocol for UPDI upload
; upload_protocol = atmelice_updi
; "custom" has to be used for SerialPDI
upload_protocol = custom
; set the port here, ie COM3 on windows, ttyUSBx on linux
upload_port = /dev/ttyUSB2
upload_speed = 256000

; Unflag build flags
build_unflags =
; Extra build flags
build_flags =

; Monitor port settings
monitor_port = /dev/ttyUSB2
; Serial monitor baud rate
monitor_speed = 115200
monitor_rts = 0

extra_scripts = updi_upload.py

; Run the following command to upload with this environment
; pio run -e Upload_UPDI -t upload
[env:Upload_UPDI]


; run the following command to set fuses
; pio run -t fuses -e set_fuses
[env:set_fuses]
; Hardware settings
board_hardware.bod = 2.7v
board_hardware.eesave = yes
board_hardware.updipin = updi

updi_upload.py

"""
Custom script for uploading and setting the fuses using SerialUPDI method.
(c) 2022 by P.Z.

"""
import serial
import time
from os.path import join

Import("env")

platform = env.PioPlatform()
board = env.BoardConfig()
build_core = board.get("build.core", "")
# mcu type
mcu = env.GetProjectOption("board").lower()
# upload port 
up_port = env.GetProjectOption("upload_port")
# upload port speed
baud =  env.GetProjectOption("upload_speed")

PYEXE = env.subst("$PYTHONEXE")

FRAMEWORK_DIR = platform.get_package_dir("framework-arduino-megaavr")
if build_core != "arduino":
    FRAMEWORK_DIR = platform.get_package_dir(
        "framework-arduino-megaavr-%s" % build_core.lower())
# path to the programmer
PROG_PATH = join(FRAMEWORK_DIR, "tools", "prog.py")
# prog.py command
CMD =  f'{PYEXE} {PROG_PATH} -t uart -u {up_port} -b {baud} -d {mcu} '

# callback for upload
def on_upload(source, target, env):
    firmware_path = str(source[0]) 
    # write the hex file
    env.Execute(CMD + f'-a write -f{firmware_path}')

env.Replace(UPLOADCMD=on_upload)

# callback for fuse setting, programming the lockbit is not implemented
def on_setFuses(source, target, env):
    fuses = ""
    fs = env.subst("$FUSESFLAGS").split(" ")
    for fuse in fs:
        if fuse.startswith('-Ufuse' ):
            # print(f'fuse no {int(fuse[6])} = {fuse[10:14]}')
            fuses += f' {fuse[6]}:{fuse[10:14]}'
    env.Execute(CMD + f' --fuses {fuses}')

env.Replace(SETFUSESCMD=on_setFuses)

How to use

place both files in the root directory of the project, where platformio.ini normally resides.
Open the platformio.ini and edit any MCU related settings as usually for a new project.
Ontop of that:

• determine the COM port of the serial dongle used as updi programmer and update the corresponding setting, ie.
  upload_port = /dev/ttyUSB2 or upload_port = COM3
• set the upload speed. With my CH330 based programmer i can easily achieve 961000 baud. The max value depends on the usb-serial dongle type and the updi implementation. 115200 should always work.
• change the monitor port to the same value if using the same uart for tracing.

To set the fuses invoke as normally:
pio run -t fuses -e set_fuses
To upload the firmware, just hit the upload button.

How it works

The important part is to replace the built in stock programming method with a custom one. This is done in the ini file by setting the
upload_protocol = custom
and providing an extra python script:
extra_scripts = updi_upload.py
The script does the following:

• grabs the serial port settings and the MCU type from the platformio.ini file
• finds the path to the pio's python path (pyenv)
• finds the path to the prog.py, which is installed together with the core files
• using the above invokes the prog.py with imported settings using the correct python path, basically automating the manual way presented in the readme file here
  https://github.com/SpenceKonde/megaTinyCore/blob/master/megaavr/extras/PlatformIO.md#using-serialupdi-from-platformio

Fuses

Programming fuses is also implemented. The fuse values are calculated when using the command:
pio run -t fuses -e set_fuses
The returned values are formatted for avrdude, the scripts reformats them into offset:value pairs accepted by the prog.py. Programming the lockbits is not implemented, because the avrdude format does not provide the offset value.

---

I have tested it with a few of ATTINY806 based projects and my own CH330 based programmer.
Here the console output for the programming operation:

/home/user/.platformio/penv/bin/python /home/pio/.platformio/packages/framework-arduino-megaavr-megatinycore/tools/prog.py -t uart -u /dev/ttyUSB2 -b 960000 -d attiny806 -a write -f.pio/build/Upload_UPDI/firmware.hex
SerialUPDI
UPDI programming for Arduino using a serial adapter
Based on pymcuprog, with significant modifications
By Quentin Bolsee and Spence Konde
Version 1.2.0 - June 2021
Using serial port /dev/ttyUSB2 at 960000 baud.
Target: attiny806
Action: write
File: .pio/build/Upload_UPDI/firmware.hex
Pinging device...
Ping response: 1E9324
Chip/Bulk erase,
Memory type eeprom is conditionally erased (depending upon EESAVE fuse setting)
Memory type flash is always erased
Memory type internal_sram is always erased
Memory type lockbits is always erased
...
Erased.
Action took 0.01s
Writing from hex file...
Writing flash...
[                                                  ]
[=                                                 ] 2/85
[==                                                ] 4/85
[===                                               ] 6/85
...
[================================================= ] 84/85
[==================================================] 85/85
Action took 0.88s
Verifying...
[                                                  ]
[====                                              ] 1/11
[=========                                         ] 2/11
...
[==================================================] 11/11
Verify successful. Data in flash matches data in specified hex-file
Action took 0.14s
================= [SUCCESS] Took 3.79 seconds =========================

Setting the fuses:

TARGET CONFIGURATION:
------------------------
Target = attiny806
Clock speed = 20000000L
Oscillator = internal
BOD level = 2.7v
Save EEPROM = yes
UPDI pin mode = updi
------------------------
Warning: The `custom` upload protocol is used! The upload and fuse flags may conflict!
More information: https://docs.platformio.org/en/latest/platforms/atmelavr.html#overriding-default-fuses-command


Selected fuses:
------------------------
[fuse0 / WDTCFG  = 0x00]
[fuse1 / BODCFG  = 0x00]
[fuse2 / OSCCFG  = 0x02]
[fuse4 / TCD0CFG = 0x00]
[fuse5 / SYSCFG0 = 0xC5]
[fuse6 / SYSCFG1 = 0x06]
[fuse7 / APPEND  = 0x00]
[fuse8 / BOOTEND = 0x00]
[lfuse / LOCKBIT = 0xC5]
------------------------

Setting fuses...
/home/user/.platformio/penv/bin/python /home/pio/.platformio/packages/framework-arduino-megaavr-megatinycore/tools/prog.py -t uart -u /dev/ttyUSB2 -b 960000 -d attiny806 --fuses 0:0x00 1:0x00 2:0x02 4:0x00 5:0xC5 6:0x06 7:0x00 8:0x00
SerialUPDI
UPDI programming for Arduino using a serial adapter
Based on pymcuprog, with significant modifications
By Quentin Bolsee and Spence Konde
Version 1.2.0 - June 2021
Using serial port /dev/ttyUSB2 at 960000 baud.
Target: attiny806
Set fuses: ['0:0x00', '1:0x00', '2:0x02', '4:0x00', '5:0xC5', '6:0x06', '7:0x00', '8:0x00']
Action: 
Pinging device...
Ping response: 1E9324
Setting fuse 0x0=0x0
Writing literal values...
Verifying literal values...
Action took 0.03s
Setting fuse 0x1=0x0
Writing literal values...
Verifying literal values...
Action took 0.03s
Setting fuse 0x2=0x2
Writing literal values...
Verifying literal values...
Action took 0.03s
Setting fuse 0x4=0x0
Writing literal values...
Verifying literal values...
Action took 0.03s
Setting fuse 0x5=0xc5
Writing literal values...
Verifying literal values...
Action took 0.03s
Setting fuse 0x6=0x6
Writing literal values...
Verifying literal values...
Action took 0.03s
Setting fuse 0x7=0x0
Writing literal values...
Verifying literal values...
Action took 0.03s
Setting fuse 0x8=0x0
Writing literal values...
Verifying literal values...
Action took 0.03s
Finished writing fuses.
=============== [SUCCESS] Took 2.85 seconds 

[apws]

nice, thanks, I am playing with jetbrains CLion for a while too

[SpenceKonde]

Don't suppose you could submit a pr for adding that to PlatformIO.md documentation? I lack the background to fully integrate this into the documentation

…

____________ Spence Konde Azzy’S Electronics New products! Check them out at tindie.com/stores/DrAzzy GitHub: github.com/SpenceKonde ATTinyCore: Arduino support for almost every ATTiny microcontroller Contact: ***@***.***

On Tue, Jan 11, 2022, 08:39 Piotr Zapart ***@***.***> wrote: Hey all! Here is something that some of you might find useful: platformio.ini file and a custom script that makes automated use of the built in (not implemented in pio, though) prog.py uploader. Both uploading the firmware and setting the fuses works. Files platformio.ini ; PlatformIO template configuration file for megaTinyCore ; https://github.com/SpenceKonde/megaTinyCore/ ; ; Build options: build flags, source filter ; Hardware options: oscillator type, BOD, EEPROM retain ; Upload options: custom upload port, speed, and extra flags ; Library options: dependencies, extra library storages ; Advanced options: extra scripting ; ; Please visit documentation for the other options ; https://github.com/SpenceKonde/megaTinyCore/blob/master/PlatformIO.md ; https://docs.platformio.org/page/projectconf.html ; https://docs.platformio.org/en/latest/platforms/atmelmegaavr.html [platformio] ; Default build target default_envs = Upload_UPDI ; default_envs = set_fuses ; Parameters used for all environments [env] platform = atmelmegaavr framework = arduino ; Chip in use board = ATtiny806 ; Clock frequency in [Hz] board_build.f_cpu = 20000000L ; Oscillator type (internal or external) board_hardware.oscillator = internal ; Upload protocol for UPDI upload ; upload_protocol = atmelice_updi ; "custom" has to be used for SerialPDI upload_protocol = custom ; set the port here, ie COM3 on windows, ttyUSBx on linux upload_port = /dev/ttyUSB2 upload_speed = 256000 ; Unflag build flags build_unflags = ; Extra build flags build_flags = ; Monitor port settings monitor_port = /dev/ttyUSB2 ; Serial monitor baud rate monitor_speed = 115200 monitor_rts = 0 extra_scripts = updi_upload.py ; Run the following command to upload with this environment ; pio run -e Upload_UPDI -t upload [env:Upload_UPDI] ; run the following command to set fuses ; pio run -t fuses -e set_fuses [env:set_fuses] ; Hardware settings board_hardware.bod = 2.7v board_hardware.eesave = yes board_hardware.updipin = updi updi_upload.py """Custom script for uploading and setting the fuses using SerialUPDI method.(c) 2022 by P.Z."""import serialimport timefrom os.path import join Import("env") platform = env.PioPlatform()board = env.BoardConfig()build_core = board.get("build.core", "")# mcu typemcu = env.GetProjectOption("board").lower()# upload port up_port = env.GetProjectOption("upload_port")# upload port speedbaud = env.GetProjectOption("upload_speed") PYEXE = env.subst("$PYTHONEXE") FRAMEWORK_DIR = platform.get_package_dir("framework-arduino-megaavr")if build_core != "arduino": FRAMEWORK_DIR = platform.get_package_dir( "framework-arduino-megaavr-%s" % build_core.lower())# path to the programmerPROG_PATH = join(FRAMEWORK_DIR, "tools", "prog.py")# prog.py commandCMD = f'{PYEXE} {PROG_PATH} -t uart -u {up_port} -b {baud} -d {mcu} ' # callback for uploaddef on_upload(source, target, env): firmware_path = str(source[0]) # write the hex file env.Execute(CMD + f'-a write -f{firmware_path}') env.Replace(UPLOADCMD=on_upload) # callback for fuse setting, programming the lockbit is not implementeddef on_setFuses(source, target, env): fuses = "" fs = env.subst("$FUSESFLAGS").split(" ") for fuse in fs: if fuse.startswith('-Ufuse' ): # print(f'fuse no {int(fuse[6])} = {fuse[10:14]}') fuses += f' {fuse[6]}:{fuse[10:14]}' env.Execute(CMD + f' --fuses {fuses}') env.Replace(SETFUSESCMD=on_setFuses) How to use place both files in the root directory of the project, where platformio.ini normally resides. Open the platformio.ini and edit any MCU related settings as usually for a new project. Ontop of that: - determine the COM port of the serial dongle used as updi programmer and update the corresponding setting, ie. upload_port = /dev/ttyUSB2 or upload_port = COM3 - set the upload speed. With my CH330 based programmer i can easily achieve 961000 baud. The max value depends on the usb-serial dongle type and the updi implementation. 115200 should always work. - change the monitor port to the same value if using the same uart for tracing. To set the fuses invoke as normally: pio run -t fuses -e set_fuses To upload the firmware, just hit the upload button. How it works The important part is to replace the built in stock programming method with a custom one. This is done in the ini file by setting the upload_protocol = custom and providing an extra python script: extra_scripts = updi_upload.py The script does the following: - grabs the serial port settings and the MCU type from the platformio.ini file - finds the path to the pio's python path (pyenv) - finds the path to the prog.py, which is installed together with the core files - using the above invokes the prog.py with imported settings using the correct python path, basically automating the manual way presented in the readme file here https://github.com/SpenceKonde/megaTinyCore/blob/master/megaavr/extras/PlatformIO.md#using-serialupdi-from-platformio Fuses Programming fuses is also implemented. The fuse values are calculated when using the command: pio run -t fuses -e set_fuses The returned values are formatted for avrdude, the scripts reformats them into offset:value pairs accepted by the prog.py. Programming the lockbits is not implemented, because the avrdude format does not provide the offset value. ------------------------------ I have tested it with a few of ATTINY806 based projects and my own CH330 based programmer. Here the console output for the programming operation: /home/user/.platformio/penv/bin/python /home/pio/.platformio/packages/framework-arduino-megaavr-megatinycore/tools/prog.py -t uart -u /dev/ttyUSB2 -b 960000 -d attiny806 -a write -f.pio/build/Upload_UPDI/firmware.hex SerialUPDI UPDI programming for Arduino using a serial adapter Based on pymcuprog, with significant modifications By Quentin Bolsee and Spence Konde Version 1.2.0 - June 2021 Using serial port /dev/ttyUSB2 at 960000 baud. Target: attiny806 Action: write File: .pio/build/Upload_UPDI/firmware.hex Pinging device... Ping response: 1E9324 Chip/Bulk erase, Memory type eeprom is conditionally erased (depending upon EESAVE fuse setting) Memory type flash is always erased Memory type internal_sram is always erased Memory type lockbits is always erased ... Erased. Action took 0.01s Writing from hex file... Writing flash... [ ] [= ] 2/85 [== ] 4/85 [=== ] 6/85 ... [================================================= ] 84/85 [==================================================] 85/85 Action took 0.88s Verifying... [ ] [==== ] 1/11 [========= ] 2/11 ... [==================================================] 11/11 Verify successful. Data in flash matches data in specified hex-file Action took 0.14s ================= [SUCCESS] Took 3.79 seconds ========================= Setting the fuses: TARGET CONFIGURATION: ------------------------ Target = attiny806 Clock speed = 20000000L Oscillator = internal BOD level = 2.7v Save EEPROM = yes UPDI pin mode = updi ------------------------ Warning: The `custom` upload protocol is used! The upload and fuse flags may conflict! More information: https://docs.platformio.org/en/latest/platforms/atmelavr.html#overriding-default-fuses-command Selected fuses: ------------------------ [fuse0 / WDTCFG = 0x00] [fuse1 / BODCFG = 0x00] [fuse2 / OSCCFG = 0x02] [fuse4 / TCD0CFG = 0x00] [fuse5 / SYSCFG0 = 0xC5] [fuse6 / SYSCFG1 = 0x06] [fuse7 / APPEND = 0x00] [fuse8 / BOOTEND = 0x00] [lfuse / LOCKBIT = 0xC5] ------------------------ Setting fuses... /home/user/.platformio/penv/bin/python /home/pio/.platformio/packages/framework-arduino-megaavr-megatinycore/tools/prog.py -t uart -u /dev/ttyUSB2 -b 960000 -d attiny806 --fuses 0:0x00 1:0x00 2:0x02 4:0x00 5:0xC5 6:0x06 7:0x00 8:0x00 SerialUPDI UPDI programming for Arduino using a serial adapter Based on pymcuprog, with significant modifications By Quentin Bolsee and Spence Konde Version 1.2.0 - June 2021 Using serial port /dev/ttyUSB2 at 960000 baud. Target: attiny806 Set fuses: ['0:0x00', '1:0x00', '2:0x02', '4:0x00', '5:0xC5', '6:0x06', '7:0x00', '8:0x00'] Action: Pinging device... Ping response: 1E9324 Setting fuse 0x0=0x0 Writing literal values... Verifying literal values... Action took 0.03s Setting fuse 0x1=0x0 Writing literal values... Verifying literal values... Action took 0.03s Setting fuse 0x2=0x2 Writing literal values... Verifying literal values... Action took 0.03s Setting fuse 0x4=0x0 Writing literal values... Verifying literal values... Action took 0.03s Setting fuse 0x5=0xc5 Writing literal values... Verifying literal values... Action took 0.03s Setting fuse 0x6=0x6 Writing literal values... Verifying literal values... Action took 0.03s Setting fuse 0x7=0x0 Writing literal values... Verifying literal values... Action took 0.03s Setting fuse 0x8=0x0 Writing literal values... Verifying literal values... Action took 0.03s Finished writing fuses. =============== [SUCCESS] Took 2.85 seconds — Reply to this email directly, view it on GitHub <#606>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTXEWZO3CMR2AWW5KY26ZTUVQXIPANCNFSM5LWKWHCQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[hexeguitar]

I think it should be tested more before submitting a pr. I don't have too many of the new AVRs to test it, esp the fuse programming part.

[apws]

I will try to test this deeply as CLion/platformio is way to go forward with more chips... In fact, I LOVE java, so IntelliJ and so CLion ... best ide, period.

[ve7bwn]

Very interesting. I'll have to try all that out.

Alternatively, here's what I did inside a project's platformio.ini file (I use Mac OS X):

[env:ATtiny414]
platform = atmelmegaavr
board = ATtiny414
framework = arduino
upload_protocol = custom
upload_port = /dev/cu.usbserial-*
upload_speed = 230400
upload_flags =
-t
uart
-u
$UPLOAD_PORT
-b
$UPLOAD_SPEED
-d
$BOARD
--fuses
"2:0x02 6:0x04 8:0x00"
upload_command = ~/.platformio/penv/bin/python3 -u ~/.platformio/packages/framework-arduino-megaavr-megatinycore/tools/prog.py $UPLOAD_FLAGS -f$SOURCE -a write

I was pleased to find that platformio did the right thing with upload_port = /dev/cu.usbserial-* and found the port my UPDI CH340 was plugged into.

With my low volume of projects, I imagine I'll just cut-and-paste the custom upload part into each platformio.ini at need.

[SpenceKonde]

Nice, this sort of thing is great. I can't competently wrap that up into a documentation file (remember, I have never installed nor opened platform IO. At the rate things are goingI might be abke to get to it in the summer, which is actually a drastic improvement from what I was extimating next fall, which was sometime after retirement age.(I;m 35 now, it would be a long wait). But I'd rather the documentation on it not wait until summer even. Lots of the people using my cores seem to be on PIO

[hexeguitar]

Just tested the script on VSCode+PlatformIO on Win10. It worked for both, firmware upload and setting the fuses.
@ve7bwn would be nice if you could try it on macOSX.
What the script does is very similar to what you manually entered in the ini file, just automates finding the paths and variables. Thus, i hope, making it a cross platform solution.

[apws]

hi, fact is that setup platformio alone is just 3secs of adding extension to vscode, but then it pulls SDKs from net on demand; I will just start this journey as I had lot of complains to vscode messy approach, all the plugins for everything, somewhat conflicting if its too easy ... but I recommend also to spence to start with plain clean vscode + arduino ms extension to start with the beast ... this approach is only wrapper to tools installed through arduino ide, nothing more, only better project handling, persistence of board settings and some intellisense help (also to catch issues in libs, probably). After mastering this vscode/arduino-ext combo it will be far easier to move to platformio, IMHO. For those comfortable with vscode, enough... in case you want faster/native/(well, fast java) IDE, go to CLion for platformio integration, my final goal... but its true, that there is nothing faster and sexier than segger IDE :-) (sorry for OT, just trying everything possible around...)