[experiment] break as many Zip side-effects as we can

[the8472]

This is removes all the side-effect-preserving code from iter::Zip. On top it converts some Iterator internal iteration implementations to counted loops to skip additional side-effects on the default implementation of zip. That's a version of some optimizations I wanted to try but modified to increase coverage. Some of those already ran afoul of the *_sideffectful() tests.

It's designed to increase impact in a crater run for #83791 to see if anyone relies on side-effects.

This is a somewhat bigger change than merely omitting side-effects on the last next() during fowards because it can skips many steps worth of side-effects at the end of an iterator when using backwards iteration, which is normally necessary to bring the iterators to equal length before doing a backwards zip.
Omitting the side-effects during a forward pass are easily justified and safely implementable with counted loops. There currently isn't anything safe or stable that would justify skipping during backwards iteration. On the other hand the zip documentaion doesn't spend a single word on its DoubleEndedIterator behavior.

@rustbot label +S-experimental

[rust-highfive]

r? @kennytm

(rust-highfive has picked a reviewer for you, use r? to override)

[the8472]

r? @m-ou-se

[m-ou-se]

@bors try

[bors]

⌛ Trying commit bed379a with merge 46592174fb13958bb9a2aa0d7af6c1224e1d7143...

[bors]

☀️ Try build successful - checks-actions
Build commit: 46592174fb13958bb9a2aa0d7af6c1224e1d7143 (46592174fb13958bb9a2aa0d7af6c1224e1d7143)

[m-ou-se]

@craterbot run mode=build-and-test

[craterbot]

👌 Experiment pr-83796 created and queued.
🤖 Automatically detected try build 46592174fb13958bb9a2aa0d7af6c1224e1d7143
⚠️ Try build based on commit d408fdd, but latest commit is bed379a. Did you forget to make a new try build?
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-83796 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-83796 is completed!
📊 58 regressed and 60 fixed (156780 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[the8472]

My first time using crater, analyzing the results will take some time.

[JohnCSimon]

@rustbot label: -S-waiting-on-review +S-waiting-on-author

[JohnCSimon]

ping from triage:
@the8472 - still analyzing?

[the8472]

Haven't had the time yet, I'll take a look on monday.

[the8472]

Local testing setup:

• spin up container
• install rustup
• use rustup-toolchain-installer-master to fetch 46592174fb13958bb9a2aa0d7af6c1224e1d7143
• switch default toolchain to 46592174fb13958bb9a2aa0d7af6c1224e1d7143
• download regressed crates with cargo clone <crate> --vers <version> or git clone <crate> + git checkout <rev>
• run cargo test in each crate

Results:

Build failed cases

• rocurley.jed.26c9dae4c85b52b32a50876d1ab1cbe697d1c6a0

  getting the same results with d408fdd and 46592174fb13958bb9a2aa0d7af6c1224e1d7143, i.e. it builds but 3 tests fail

• paranoia-0.1.4

  fails to build with the same linker errors but I get exactly the same errors when running cargo +d408fdd4a82bc3e7ea61dd81bc9a8781b2bf939d test, so I don't know why that passed on crater in the first place.
  The log does contain this warning, which seems relevant

  warning: The package `paranoia_sys` provides no linkable target. The compiler might raise an error while compiling `paranoia_caller`. Consider adding 'dylib' or 'rlib' to key `crate-type` in `paranoia_sys`'s Cargo.toml. This warning might turn into a hard error in the future.
  

Test failed cases

can't reproduce, test passed on 1st try

aristeia-0.2.3
async_smux-0.2.1
beats-0.1.3
bus_queue-0.5.3
cargo-open-0.4.0
crux-0.3.0
env_assert-0.1.4
ffi_helpers-0.2.0
filecmp-0.2.0
fluvio-storage-0.3.0
fzq-0.1.0
mobc-0.7.2
phi-detector-0.3.0
r2d2-0.8.9
rast-0.1.0-alpha.0
ringu-0.1.1
safina-net-0.1.3
secc-0.0.10
skiplist-0.3.0
snooze-rs-0.0.3
solana-streamer-1.6.4 - has "outdated" any significance in the crater report?
strip-ansi-escapes-0.1.0
tokio-by-hand-0.0.0
vented-0.11.7
JMurph2015.rust-vebtrees.0120b1332b903167260df47a8b43430faca81f9c
Robbe7730.MCRust.8d13c37a88d59a4ba60f9a9df215e92dbe4746b3
appaquet.extsort-rs.ff85698c25436131e1d79c5d4f8c46805d47a50f
arron-speake-bluefruit.phal.1c6ad989c565242dafcbcdb16bf779ad1e177cee
baoziv587.LoxLang.c5f72cf61fa1d6b4fc5cc4f69385090c8930202d
benj2468.cards.46cc1eff859e6a969cad231368b2d41683c7240c
black-binary.async-smux.cf4a14c19263cda719bcb7d4117841dbc5f8237
cryptonote-rust.account.a05482ccd191259374fdc7ad7f1d1980863e003f
duriantang.nickel_cors.5685cd9045bc264276689410a4052e3a8ed0a8e1
ecsumed.wrench.e291faf00d511fe1c06d16dd7a9769e6a3a22ce9
exclave.exclave.df7bda920fed6bd274616d848fb90a535ec6fc81
garbados.roll-rs.e41d0395472322b676edf169600e64d452980a91
keiya01.kmonkey.f23bda01226b035f832c53d6c72947564d7af669
richardanaya.globals.d7a07df2847c8e41c0d46ab80f7157feafb8fa5b
rogithub.rust-patterns.eb7f8f161633858888b8f829f84ed0ae7da85119
spebern.futures-delay-queue.739c55a1d486fd2a24ea5657b3853e9098ac0aae
ycatbink0t.smallest_enclosing_ring.13e654db002d9882d0e4a1c237bd6c5eccfda6c5

tests fail, but also fail on d408fdd.

I don't know what crater does differently that they pass there

ezhook-0.2.2
mime-detective-1.0.0
atsmtat.lox-vm.855cad798fa90d0a1457460de998778abeb33130

tests fail intermittently

ipc-orchestrator-0.3.3
persistence-0.0.5
qjsonrs-0.1.0
quanta-0.7.2 - lib contains global state and tests race
rapid-0.0.1
19h.dualshock4-rs.ce3c8f4f0c97117f3b28ad6ccb9929bb0195c828
ciusji.eureka.6601fb97857aae23fefabc258effd5900b2dda6f
krisvanrens.rust-workshop-accu2019.e96db277ae8c993b90a90bf182e3a84f235fdf03
sts10.fgift.8190734406b2ceb5ba9ed7713442bb364a32821b

Other

notp-2.1.0 - test fails on first run due to filesystem state, passes on subsequent runs because ~/.config was created
Wilfred.bfc.9344deefdca47efea3a62dd4862ccdd2e06811ff - crate build failed due to wrong version llvm dependency
philippkeller.rexpect.db3b17d33af0db39134d82010f0e848a2b404769 - tests interact with various CLI from the linux distro, probably wrong versions installed

[the8472]

So unless I missed any critical step in reproducing the issues it appears that none of the failures (at least those that I could test) seem related to skipping side-effects on the iterators in question.

Of course this does not prove that nobody is relying on those properties, but at least there is a chance that we could eliminate that logic and thus simplify iter:Zip.

In the meantime #85342 has been suggested, which offers a more conservative path: We implement zip_exact, remove the specializations from zip and put the simplified versions of the specializations in zip_exact which don't need to concern themselves with preserving side-effects because the _exact property will already guarantee that they are not needed.

Tangentially to the Zip question this crater run also tested specializing the iterator default implementations for fold and rfold to use counted loops which skip the last next() call since they know when it would be None. That may lead to faster compile times due to less IR, but will have to be evaluated in a perf run that doesn't mix in the other changes on this branch.

ping @m-ou-se

@rustbot label +S-waiting-on-review -S-waiting-on-author

[crlf0710]

@the8472 Would you mind removing the [experiment] part from title (and optionally give it a more descriptive title), if this is ready for review? Thanks!

[the8472]

@crlf0710 this isn't meant to be merged as-is and more meant to inform a decision on #83791. Although I might pull out a subset of the changes into another PR later.

I guess I'll just close this and link here from #83791