UCX protocols

[lijh5]

Set UCX_PROTO_ENABLE=y upon application launch to enable the newer “v2” UCX protocols, the older v1 protocols are needed, one can set UCX_PROTO_ENABLE=n in the environment to override this setting.

Is there any explanation for the new UCX v2 protocol, what is the difference from v1, and how is it implemented?
Thank you.

[ivankochin]

AFAIK there is no detailed design documentation about proto selection.

The main difference between v1 and v2 is that v1 chooses protocol basing on calculated thresholds + hardcoded order that defines which protocol should be preferred. v2 builds mathematical performance prediction model for each protocol which estimates how much time would certain operation take and chooses protocol with fastest estimation.

You can examine how does v2 works in details by setting UCX_PROTO_INFO=y and UCX_PROTO_INFO_DIR=<patch_to_log_dir> variables.

[Yangfisher1]

@ivankochin Hi, Ivan. I have a question about the v2 protocol. Is the mathematical performance prediction model determined at the time of connection establishment and cannot be modified later? For example, in the case where one NIC is overloaded, another idle NIC should be used.

[yosefe]

@Yangfisher1 yes, the prediction is determined according to theoretical NIC performance estimation.

[Yangfisher1]

@yosefe Thank you!