[L0] UR_PROGRAM_INFO_DEVICES is unimplementable for urProgramCreateWithNativeHandle

[pbalcer]

enumerator UR_PROGRAM_INFO_DEVICES
[ur_device_handle_t[]] Return list of devices associated with Program.

This is currently unimplementable for programs that are created using urProgramCreateWithNativeHandle because there doesn't appear to be a way to retrieve a list of devices from L0 for a module (see L0 docs) nor does this function take a list of devices we could just store.

As far as I can tell, we don't even have device information in SYCL. Instead, SYCL relies on exactly UR_PROGRAM_INFO_DEVICES to populate its list of devices in the program_impl class.

I don't think there are any user-visible issues, performance or functional, related to this in SYCL right now, but we might want to consider returning 0 devices or some error status for this case.

[kbenzie]

urProgramCreateWithNativeHandle takes a ur_context_handle_t which could be used to populate the device list.

[kbenzie]

If that's not the desired set of devices, then we'd likely need to require the user to provide the device list when calling urProgramCreateWithNativeHandle.

[pbalcer]

But can't a module be built for a subset of devices in a context? @nrspruit can you chime in?

[pbalcer]

If that's not the desired set of devices, then we'd likely need to require the user to provide the device list when calling urProgramCreateWithNativeHandle.

I think that would require the SYCL API to change.

[kbenzie]

But can't a module be built for a subset of devices in a context?

It can, that's different to the list of devices associated with the program though. The list of associated devices is all the devices the program can be built for.

[pbalcer]

But can't a module be built for a subset of devices in a context?

It can, that's different to the list of devices associated with the program though. The list of associated devices is all the devices the program can be built for.

Gotcha, I was misled by this comment "return true number of devices this program exists for." in the current L0 implementation. So, generally, is UR_PROGRAM_INFO_DEVICES just a shorthand for UR_PROGRAM_INFO_CONTEXT -> UR_CONTEXT_INFO_DEVICES ?

We found this with @nrspruit in a real application when SYCL was hitting this assert: https://github.com/intel/llvm/blob/sycl/sycl/source/detail/program_impl.cpp#L164

The problem ended up being that MContext->getPlatformImpl()->get_devices(); didn't return the correct devices, but this code seems to suggest that there might be some difference between the set of devices in the platform, the devices in the context (which makes sense), and finally the devices associated with the program. Otherwise, why not just use MContext->get_devices() and skip the whole remove_if ?

[pbalcer]

CL_PROGRAM_DEVICES

cl_device_id[]

Return the list of devices associated with the program object. This can be the devices associated with context on which the program object has been created or can be a subset of devices that are specified when a program object is created using clCreateProgramWithBinary.

from https://registry.khronos.org/OpenCL/sdk/3.0/docs/man/html/clGetProgramInfo.html

OpenCL has better docs than UR :-)

Based on this, I believe UR_PROGRAM_INFO_DEVICES should return all devices in the context for urProgramCreateWithNativeHandle.

[kbenzie]

We will clarify the UR spec to align with the OpenCL spec on this.