Zip's __iterator_get_unchecked doesn't preserve side-effects

[SkiFire13]

I tried this code:

fn main() {
    let a = vec![String::new(), String::new()];
    let b = [()];
    
    let mut counter = 0;
    
    a.into_iter() 
        .map(|i| {
            counter += 1;
            i
        })
        .zip(&b)
        .zip(&b)
        .for_each(drop);
        
    assert_eq!(counter, 2);
    
    let a = vec![String::new(), String::new()];
    let b = [()];
    
    let mut counter = 0;
    
    a.iter()
        .map(|i| {
            counter += 1;
            i
        })
        .zip(&b)
        .zip(&b)
        .for_each(drop);
        
    assert_eq!(counter, 2); // this fails, even though it succeed in the non-specialized version
}

Playground link

I would expect the two iterators to behave the same, so either both asserts fail (actually the first should stop the program) or neither of them. Instead, the second iterator doesn't preserve side-effects. causing the second assert (and only that one) to fail.

I don't think there's a way to solve this without completly changing the TrustedRandomAccess trait to use some type level trickery to replace may_have_side_effect. I wonder if there's even a point to keep the simulated side effects in the specialized ZipImpl::next if it doesn't cover all cases.

[the8472]

I would expect the two iterators to behave the same

We could also update the documentation to clarify that next() might not always be called on empty iterators.

For example a valid implementation would be checking size_hint() before iterating and bailing out early in case the upper bound is 0. It should be permissible for safe code to make these kinds of checks, size_hint only can't be relied on to uphold unsafe contracts. Whether side-effects happen would then depend on the tightness of the hint bounds.

Note that this isn't specific to Zip. Another adapter wrapping zip could introduce this kind of behavior.

[the8472]

In the meantime I have introduced some additional specializations relying on TrustedRandomAccess and I just realized that they all suffer from similar problems when a zip exists somewhere earlier in the iterator pipeline. I think that is fixable for those cases, but...

I wonder if there's even a point to keep the simulated side effects in the specialized ZipImpl::next if it doesn't cover all cases.

Indeed, I would be in favor of getting rid of it and rewording the documentation instead.

[the8472]

Original discussion about side-effects: #33090 (comment)
And the PR implementing it for the first time: #37230

I don't find those to be particularly strong arguments, primarily concerns about consistency between default and specialized code.
As far as side-effects are concerned this consistency arguably does not exist in the first place given the the different ways iterators can be driven: counted loops vs. checking for next() == None. Only the latter triggers the side-effect behavior.

[steffahn]

The documentation of Zip says

If either iterator returns None , next from the zipped iterator will return None . If the first iterator returns None , zip will short-circuit and next will not be called on the second iterator.

Which pretty clearly describes the behavior of Zip. Following this description one would not necessarily expect the first iterator not to be advanced just because the second iterator has size 0.

Edit: I suppose that’s exactly what #83791 tries to address.

[the8472]

I believe with #83791 merged this can be closed since it turns it into a non-promise

[Dylan-DPC]

Closing this as this isn't promised.