Support COM port discovery via USB VID/PID #907
Comments
|
The crucial part is figuring out the link between the USB device and the serial device. I never looked into non-Windows implementations, but on Windows there are two parts of metadata one can use: 1) You can enumerate through the device tree, and find the parent of the serial device, which will be the USB device. That is tricky if the USB device is a composite device. 2) Windows actually maintains a bunch of properties for every device, and Portname does what I needed. |
|
For a completely different example, on FreeBSD, they can only be examined by These attach to the So question is to find out how to get this information on MacOS. Downside: it adds a lot of per-OS code to AVRDUDE. |
|
Thank you for the details! I'll do a bit of research this evening to see if I can figure it out on MacOS.
Good point. But on the other side, this (and probably the "Arduino Leonardo style") is IMO functionality that should be considered even though OSes handles this differently. The end-user won't notice anything, and it's very convenient! I'll agree that adding functionality to one "OS build" that wouldn't work for the others is a bad idea, but if we can figure out how to do it for Windows, MacOS, FreeBSD and other UNIX compatible OSes, I think we should consider it for the sake of the convenience this functionality adds. |
|
I guess finding a way for Linux will be possible as well. |
Is I agree that it would make sense to "separate" OS-specific code into a separate API in order to not "pollute" existing code. The good this about this is that it makes it easier to add more OS-specific code later on. We should, however, be restrictive and not just throw in all the bells whistles just because we can. At the moment I can't think of other "nice to have" features that rely on OS-specific code other than port discovery via VID/PID and "1200bps touch" used on various Arduino boards. EDIT: |
|
@dl8dtl It looks like First command: Second command: |
|
Now, form that as an algorithm in C ;-) |
Challenge accepted! Here's a proof of concept. Lots of code borrowed from here. Note that it will only recognize "modems", so from my understanding, serial devices only. I've tried to plug in various other USB equipment, but my USB to serial devices are the only ones that show up. Output: Build command: usb_vid_pid_test.c #include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <errno.h>
#include <paths.h>
#include <termios.h>
#include <sysexits.h>
#include <sys/param.h>
#include <sys/select.h>
#include <sys/time.h>
#include <time.h>
#include <CoreFoundation/CoreFoundation.h>
#include <IOKit/IOKitLib.h>
#include <IOKit/serial/IOSerialKeys.h>
#include <IOKit/serial/ioss.h>
#include <IOKit/IOBSD.h>
// Function prototypes
static kern_return_t findModems(io_iterator_t *matchingServices);
static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize);
// Returns an iterator across all known modems. Caller is responsible for
// releasing the iterator when iteration is complete.
static kern_return_t findModems(io_iterator_t *matchingServices)
{
kern_return_t kernResult;
CFMutableDictionaryRef classesToMatch;
// Serial devices are instances of class IOSerialBSDClient.
// Create a matching dictionary to find those instances.
classesToMatch = IOServiceMatching(kIOSerialBSDServiceValue);
if (classesToMatch == NULL) {
printf("IOServiceMatching returned a NULL dictionary.\n");
}
else {
// Look for devices that claim to be modems.
CFDictionarySetValue(classesToMatch,
CFSTR(kIOSerialBSDTypeKey),
CFSTR(kIOSerialBSDAllTypes));
}
// Get an iterator across all matching devices.
kernResult = IOServiceGetMatchingServices(kIOMasterPortDefault, classesToMatch, matchingServices);
if (KERN_SUCCESS != kernResult)
printf("IOServiceGetMatchingServices returned %d\n", kernResult);
}
return kernResult;
}
static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize)
{
io_object_t modemService;
kern_return_t kernResult = KERN_FAILURE;
bool modemFound = false;
int vid;
int pid;
// Initialize the returned path
*bsdPath = '\0';
// Iterate across all modems found. In this example, we bail after finding the first modem.
while ((modemService = IOIteratorNext(serialPortIterator))) {
// Variable declaration
int pid, vid;
CFTypeRef bsdPathAsCFString, cf_vendor, cf_product;
cf_vendor = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane,
CFSTR("idVendor"),
kCFAllocatorDefault,
kIORegistryIterateRecursively
| kIORegistryIterateParents);
cf_product = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane,
CFSTR("idProduct"),
kCFAllocatorDefault,
kIORegistryIterateRecursively
| kIORegistryIterateParents);
bsdPathAsCFString = IORegistryEntryCreateCFProperty(modemService,
CFSTR(kIOCalloutDeviceKey),
kCFAllocatorDefault,
0);
// Decode & print port, VID & PID
if (cf_vendor && cf_product && bsdPathAsCFString &&
CFNumberGetValue(cf_vendor , kCFNumberIntType, &vid) &&
CFNumberGetValue(cf_product, kCFNumberIntType, &pid) &&
CFStringGetCString(bsdPathAsCFString, bsdPath, maxPathSize, kCFStringEncodingUTF8)) {
printf("Found modem. Port: %s USB VID: 0x%04X PID: 0x%04X\n", bsdPath, vid, pid);
modemFound = true;
kernResult = KERN_SUCCESS;
}
// Release CFTypeRef
if (cf_vendor) CFRelease(cf_vendor);
if (cf_product) CFRelease(cf_product);
if (bsdPathAsCFString) CFRelease(bsdPathAsCFString);
}
return kernResult;
}
int main(int argc, const char * argv[])
{
kern_return_t kernResult;
io_iterator_t serialPortIterator;
char bsdPath[MAXPATHLEN];
kernResult = findModems(&serialPortIterator);
if (KERN_SUCCESS != kernResult) {
printf("No modems were found.\n");
}
kernResult = getModemPath(serialPortIterator, bsdPath, sizeof(bsdPath));
if (KERN_SUCCESS != kernResult) {
printf("Could not get path for modem.\n");
}
IOObjectRelease(serialPortIterator);
return EX_OK;
} |
|
Here's another example where the USB VID and PID is passed as arguments, and the program will tell if the device is present or not: Output: Build command: Source: #include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <errno.h>
#include <paths.h>
#include <termios.h>
#include <sysexits.h>
#include <sys/param.h>
#include <sys/select.h>
#include <sys/time.h>
#include <time.h>
#include <CoreFoundation/CoreFoundation.h>
#include <IOKit/IOKitLib.h>
#include <IOKit/serial/IOSerialKeys.h>
#include <IOKit/serial/ioss.h>
#include <IOKit/IOBSD.h>
// Function prototypes
static kern_return_t findModems(io_iterator_t *matchingServices);
static bool getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize, int usb_vid, int usb_pid);
// Returns an iterator across all known modems. Caller is responsible for
// releasing the iterator when iteration is complete.
static kern_return_t findModems(io_iterator_t *matchingServices)
{
kern_return_t kernResult;
CFMutableDictionaryRef classesToMatch;
// Serial devices are instances of class IOSerialBSDClient.
// Create a matching dictionary to find those instances.
classesToMatch = IOServiceMatching(kIOSerialBSDServiceValue);
if (classesToMatch == NULL) {
printf("IOServiceMatching returned a NULL dictionary.\n");
}
else {
// Look for devices that claim to be modems.
CFDictionarySetValue(classesToMatch,
CFSTR(kIOSerialBSDTypeKey),
CFSTR(kIOSerialBSDAllTypes));
}
// Get an iterator across all matching devices.
kernResult = IOServiceGetMatchingServices(kIOMasterPortDefault, classesToMatch, matchingServices);
if (KERN_SUCCESS != kernResult) {
printf("IOServiceGetMatchingServices returned %d\n", kernResult);
}
return kernResult;
}
static bool getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize, int usb_vid, int usb_pid)
{
io_object_t modemService;
int vid;
int pid;
// Initialize the returned path
*bsdPath = '\0';
// Iterate across all modems found. In this example, we bail after finding the first modem.
while ((modemService = IOIteratorNext(serialPortIterator))) {
// Variable declaration
int pid, vid;
CFTypeRef bsdPathAsCFString, cf_vendor, cf_product;
cf_vendor = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane,
CFSTR("idVendor"),
kCFAllocatorDefault,
kIORegistryIterateRecursively
| kIORegistryIterateParents);
cf_product = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane,
CFSTR("idProduct"),
kCFAllocatorDefault,
kIORegistryIterateRecursively
| kIORegistryIterateParents);
bsdPathAsCFString = IORegistryEntryCreateCFProperty(modemService,
CFSTR(kIOCalloutDeviceKey),
kCFAllocatorDefault,
0);
// Decode & print port, VID & PID
if (cf_vendor && cf_product && bsdPathAsCFString &&
CFNumberGetValue(cf_vendor , kCFNumberIntType, &vid) &&
CFNumberGetValue(cf_product, kCFNumberIntType, &pid) &&
CFStringGetCString(bsdPathAsCFString, bsdPath, maxPathSize, kCFStringEncodingUTF8)){
if(usb_vid == vid && usb_pid == pid) {
printf("USB device found. Port: %s USB VID: 0x%04X PID: 0x%04X\n", bsdPath, vid, pid);
return true;
}
}
// Release CFTypeRef
if (cf_vendor) CFRelease(cf_vendor);
if (cf_product) CFRelease(cf_product);
if (bsdPathAsCFString) CFRelease(bsdPathAsCFString);
}
return false;
}
int main(int argc, const char * argv[])
{
kern_return_t kernResult;
io_iterator_t serialPortIterator;
char bsdPath[MAXPATHLEN];
char * end_ptr;
int usb_vid;
int usb_pid;
char* token;
usb_vid = strtol(argv[1], &end_ptr, 0);
if (*end_ptr || (end_ptr == argv[1])) {
printf("Could not parse argument %s\n", argv[1]);
return 0;
}
usb_pid = strtol(argv[2], &end_ptr, 0);
if (*end_ptr || (end_ptr == argv[1])) {
printf("Could not parse argument %s\n", argv[2]);
return 0;
}
kernResult = findModems(&serialPortIterator);
if(KERN_SUCCESS != kernResult || !getModemPath(serialPortIterator, bsdPath, sizeof(bsdPath), usb_vid, usb_pid)) {
printf("USB device not found.\n");
}
IOObjectRelease(serialPortIterator);
return EX_OK;
} |
|
Cool I wouldn't consider it for v7.0 though, there are enough other things still on the plate, and I'd rather have a release anytime soon than deferring it further by having to implement and debug a completely new feature. |
Very well, let's continue the discussion after 7.0 is released. Is there anything I or we can do to help out preparing for a release? |
Well, you already did a great job by walking through the old issues. |
|
I remember python-serial has already got this done. Example run log under Windows. Unfortunately it does not work under FreeBSD, just as what the document says. |
|
Is this something that should be considered for the 7.1 release? We could perhaps support the following syntaxes:
I'm having a hard time figuring out how to deal with config_gram.y, but we would probably need to extract typedef struct serialport_t {
LISTID id;
const char *desc;
int usbvid;
LISTID usbpid;
char path[MAXLEN];
char serno[MAXLEN];
} SERIALPORT;Output from MacOS test program: |
Looks like a good proposal. The path will be OS specific. It is kind of also similar to pyserial output. |
|
I would use int usbvid;
LISTID usbpid;
const char *usbdev;
const char *usbsn;
const char *usbvendor;
const char *usbproduct;I know little about USB (both hardware and software); is |
Great, just needs to add two entries. The path may not be const since it may change for some use cases. USB serial number needs to read from run-time and some device may not have serial number. And by USB spec the device either has no serial number or unique serial number (but some devices may violate this spec). ABCs of USB: |
|
Edit: mcuee was faster ;-) |
|
Thanks @mcuee and @dl8dtl. Some programmers already read in the serial number at run time. So, It's likely that |
|
It is perhaps a good idea to move the OS specific things into separate .c and .h files. Perhaps io.c/h or iousb.c/h? For reference, here's an updated version of the MacOS test program that finds the path and serial number for all connected serial devices. It is a result of some copy-pasting, but it works as a proof of concept. Perhaps we can figure out a way for Linux and Windows as well? MacOS USB VID/PID test program#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <errno.h>
#include <paths.h>
#include <termios.h>
#include <sysexits.h>
#include <sys/param.h>
#include <sys/select.h>
#include <sys/time.h>
#include <time.h>
#include <CoreFoundation/CoreFoundation.h>
#include <IOKit/IOKitLib.h>
#include <IOKit/serial/IOSerialKeys.h>
#include <IOKit/serial/ioss.h>
#include <IOKit/IOBSD.h>
#include <IOKit/IOKitLib.h>
#include <IOKit/usb/IOUSBLib.h>
#include <IOKit/hid/IOHIDKeys.h>
// Function prototypes
static kern_return_t findModems(io_iterator_t *matchingServices);
static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize);
CFStringRef find_serial(int idVendor, int idProduct)
{
CFMutableDictionaryRef matchingDictionary = IOServiceMatching(kIOUSBDeviceClassName);
CFNumberRef numberRef;
numberRef = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &idVendor);
CFDictionaryAddValue(matchingDictionary, CFSTR(kUSBVendorID), numberRef);
CFRelease(numberRef);
numberRef = 0;
numberRef = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &idProduct);
CFDictionaryAddValue(matchingDictionary, CFSTR(kUSBProductID), numberRef);
CFRelease(numberRef);
numberRef = 0;
io_iterator_t iter = NULL;
if (IOServiceGetMatchingServices(kIOMasterPortDefault, matchingDictionary, &iter) == KERN_SUCCESS) {
io_service_t usbDeviceRef;
if ((usbDeviceRef = IOIteratorNext(iter))) {
CFMutableDictionaryRef dict = NULL;
if (IORegistryEntryCreateCFProperties(usbDeviceRef, &dict, kCFAllocatorDefault, kNilOptions) == KERN_SUCCESS) {
CFTypeRef obj = CFDictionaryGetValue(dict, CFSTR(kIOHIDSerialNumberKey));
if (!obj) {
obj = CFDictionaryGetValue(dict, CFSTR(kUSBSerialNumberString));
}
if (obj) {
return CFStringCreateCopy(kCFAllocatorDefault, (CFStringRef)obj);
}
}
}
}
return NULL;
}
// Returns an iterator across all known modems. Caller is responsible for
// releasing the iterator when iteration is complete.
static kern_return_t findModems(io_iterator_t *matchingServices)
{
kern_return_t kernResult;
CFMutableDictionaryRef classesToMatch;
// Serial devices are instances of class IOSerialBSDClient.
// Create a matching dictionary to find those instances.
classesToMatch = IOServiceMatching(kIOSerialBSDServiceValue);
if (classesToMatch == NULL) {
printf("IOServiceMatching returned a NULL dictionary.\n");
}
else {
// Look for devices that claim to be modems.
CFDictionarySetValue(classesToMatch,
CFSTR(kIOSerialBSDTypeKey),
CFSTR(kIOSerialBSDAllTypes));
}
// Get an iterator across all matching devices.
kernResult = IOServiceGetMatchingServices(kIOMasterPortDefault, classesToMatch, matchingServices);
if (KERN_SUCCESS != kernResult)
printf("IOServiceGetMatchingServices returned %d\n", kernResult);
return kernResult;
}
static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize)
{
io_object_t modemService;
kern_return_t kernResult = KERN_FAILURE;
bool modemFound = false;
int vid;
int pid;
// Initialize the returned path
*bsdPath = '\0';
// Iterate across all modems found. In this example, we bail after finding the first modem.
while ((modemService = IOIteratorNext(serialPortIterator))) {
// Variable declaration
int pid, vid;
CFTypeRef bsdPathAsCFString, cf_vendor, cf_product;
cf_vendor = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane,
CFSTR("idVendor"),
kCFAllocatorDefault,
kIORegistryIterateRecursively
| kIORegistryIterateParents);
cf_product = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane,
CFSTR("idProduct"),
kCFAllocatorDefault,
kIORegistryIterateRecursively
| kIORegistryIterateParents);
bsdPathAsCFString = IORegistryEntryCreateCFProperty(modemService,
CFSTR(kIOCalloutDeviceKey),
kCFAllocatorDefault,
0);
// Decode & print port, VID & PID
if (cf_vendor && cf_product && bsdPathAsCFString &&
CFNumberGetValue(cf_vendor , kCFNumberIntType, &vid) &&
CFNumberGetValue(cf_product, kCFNumberIntType, &pid) &&
CFStringGetCString(bsdPathAsCFString, bsdPath, maxPathSize, kCFStringEncodingUTF8))
{
CFStringRef obj = find_serial(vid, pid);
char serial[256] = {0x00};
if (obj)
CFStringGetCString(obj, serial, 256, CFStringGetSystemEncoding());
printf("Found modem. Port: %s USB VID: 0x%04X PID: 0x%04X, S/N: %s\n", bsdPath, vid, pid, serial);
modemFound = true;
kernResult = KERN_SUCCESS;
}
// Release CFTypeRef
if (cf_vendor) CFRelease(cf_vendor);
if (cf_product) CFRelease(cf_product);
if (bsdPathAsCFString) CFRelease(bsdPathAsCFString);
}
return kernResult;
}
int main(int argc, const char * argv[])
{
kern_return_t kernResult;
io_iterator_t serialPortIterator;
char bsdPath[MAXPATHLEN];
kernResult = findModems(&serialPortIterator);
if (KERN_SUCCESS != kernResult) {
printf("No modems were found.\n");
}
kernResult = getModemPath(serialPortIterator, bsdPath, sizeof(bsdPath));
if (KERN_SUCCESS != kernResult) {
printf("Could not get path for modem.\n");
}
IOObjectRelease(serialPortIterator);
return EX_OK;
}Output: |
Thanks. I am more talking about the following situation. You can see that the COM port assignment, USB PID and Path can change during run time (basically two different devices already). Maybe we have to treat them as two devices. In that case, I think |
|
Just wondering if you can give libserialport a try. Sigrok libserialport supports Windows (MSVC and mingw), Linux, macOS and FreeBSD. |
|
@mcuee sorry, I totally forgot to reply!
Wow, libserialport is a really nice tool! I think libserialport can make serial port discovery much easier! The provided examples are also very straightforward and easy to follow. Just what we need. Here's the output from the And here is the output from the As you can see, libserialport gives us the /dev path and the USB VID/PID. Exactly what we need! |
|
As @dl8dtl mentioned, this may add too many platform specific codes to avrdude, do we really want to persue this or we can leave it outside of avrdude? |
|
Seems like there is almost a solution that can be abstracted away. We need a champion, though. Sounds like this might be @MCUdude? If not I'd be fine with dropping this. |
|
We use Avrdude for batch programming at work. I decided to stick with the USBasp programmer (USBISP hardware running modified USBasp firmware), since the COM port number on the Windows computer we use tends to change occasionally, even though there's only one USB to serial adapter connected. If we instead could specify the USB vid/pid, or even better, the chip name itself, it would be much easier to deal with USB to serial devices when using a script to execute an Avrdude command. ( Libserialport seems like the perfect match. It supports all major operating systems and has simple examples that do what we want. I can create a test program that takes a chip name or a USB VID/PID and outputs the serial port. However, I'm not all that good at integrating with avrdude.conf and how integrating the libserialport source code. But I think this would be a very useful addition to Avrdude! |
|
But then, you get compile errors from serialadapter.c. |
If you want a list of PIDs, please have a look at the "programmer" code. It takes a little more than just an assignment there. |
|
Or, just start out with a single PID, and get the remainder running. Turning that into a list of PIDs could be handled later. |
|
|
With this couple of changes, it compiles and links. |
|
For some reason, I'm now getting a |
|
Hmm, will have a look at it, but not sure whether I can manage it tonight. |
|
Ahh progress... @MCUdude Try this 0001-Fix-serialadapter-grammar-and-parsing.zip patch which should resolve the |
|
Thanks for the patch, it certainly did the trick. I've applied the patch and pushed the changes. There seems to be an issue with the CI, but I'm not sure what's wrong. I can build on my computer using make and cmake/build.sh without any errors. |
|
@MCUdude BTW, this won't compile when libserialport isn't installed and therefore |
|
Indeed it will not build when libserialport is not installed. After the installation of libserialport it will build. The warning is still there though. |
|
| an issue with the CI @MCUdude I had a look and I predict it is the wrongly placed |
|
Agreed.
|
|
I have a mild preference for 2. It is a bit of a pain to have The (by far) easier solution for the grammar for serial adapters would have been to do what C++ did with classes and structs: They are internally virtually identical (except for that inheritance for one is by default private for one and public for the other) but their use in practice is very different. Same could be done here: the C structure for serial adapters could simply be a one-line typedef of the programmer structure (actually, serial adapters only need a subset of the programmer components) and |
Easier alternative algos:
|
As I've mentioned before, dealing with the grammar is what I think is the most difficult thing with the Avrdude project. If you have suggestions on how the work-in-progress can be improved, I'm more than willing to accept a patch or a PR if that makes the code neater and/or easier to maintain. Actually, feel free to do as much or as little as you want. I'm a little busy with other things next week, so I can't promise much progress over the next couple of days. I'm just glad the idea of having |
@stefanrueger |
|
This needs a design decision. | suggestions on how the work-in-progress can be improved I would make minimal changes to the grammar and use the programmer structure for the serial adapters. Let me create a branch based on that idea, and then you can compare your current approach (which isn't finished as it misses treatment of the developer options) and my approach and weigh up the advantages and disadvantages of either. Ideally, I'd merge the current two PRs first, as to minimise the potential for merge conflicts. OK? |
|
Yes please 😁 |
I am genuinely not aware why one would be more complicated than the other. In either case the code has to consider that the library is there or not, both so it can be compiled in either case and that the functionality changes accordingly. |
Yes I think you can merge the two remaing PRs. |
@stefanrueger and @MCUdude Hopefully it may attract a few more users to come and review or test. |
I don't think it is. It's just a matter of where the |
|
Somehow I got segmentation fault under Windows. I will check again. No issue with git main. |
|
The result is then used without checking for |
Apparently, @mariusgreuel's Avrdude fork for windows supports COM port discovery via USB VID/PID. This is actually very neat if you're using a UART-based programmer that has a tendency to bump the COM port number. I have a similar issue on my mac as well; The serial port (/dev/cu.*) gets a different name depending on which USB port I connect it to.
Would it be possible, within a reasonable amount of time, to make this work on non-windows systems as well?
The text was updated successfully, but these errors were encountered: