Fix caching issues which render remote ref caching ineffective

Fix caching issues which render remote ref caching ineffective #1234