Fix panics in `try_cast_slice_box` etc #536

Job	Run time
Test with miri	1m 15s
Test on i686-unknown-linux-gnu with cross	1m 4s
Test Rust 1.34.0 on ubuntu-latest	1m 34s
Test with -Zsanitizer=address	36s
Test Rust stable on ubuntu-latest	7s
Test with -Zsanitizer=memory	39s
Test Rust beta on ubuntu-latest	15s
Test with -Zsanitizer=leak	38s
Test Rust nightly on ubuntu-latest	20s
Test Rust stable on macos-latest	11s
Test Rust stable on windows-latest	32s
Test Rust stable-x86_64-gnu on windows-latest	39s
Test Rust stable-i686-msvc on windows-latest	28s
Test Rust stable-i686-gnu on windows-latest	30s
	8m 48s

Provide feedback