Fix custom imports for both distributed and single device

Fix custom imports for both distributed and single device #5247