[Core] enable lora for sdxl adapters too and add slow tests.

[Core] enable lora for sdxl adapters too and add slow tests. #16404