fetch offset does not always work

[gaborfeher]

I have an example where offset values larger than 1000 are still handled as if they were only set to 1000. For a demonstration I have reused the same project that I was using for #386 . So I guess the example could be a bit simpler but hopefully it's simple enough: https://github.com/gaborfeher/ndb-testbed/tree/offset-issue

The short summary is that the below 3 queries are yielding the same set of results for me. I've only tried this in a GAE project, not locally.

ProductCopy5.query().fetch(limit=10, offset=1000)
ProductCopy5.query().fetch(limit=10, offset=2000)
ProductCopy5.query().fetch(limit=10, offset=3000)

[gaborfeher]

I've managed to simplify my example. It happens both in Datastore and Firestore:

def demo():
    class EmptyStuff(ndb.Model):
        pass

    ndb.put_multi(
        EmptyStuff(key=ndb.Key('EmptyStuff', i+1))
        for i in range(1100))

    q = EmptyStuff.query()
    findings = [
        (i, q.fetch(limit=1, offset=i)[0].key.id())
        for i in range(995, 1006)
    ]
    return str(findings) + '\n'

Expected output:

[(995, 996), (996, 997), (997, 998), (998, 999), (999, 1000), (1000, 1001), (1001, 1001), (1002, 1002), (1003, 1003), (1004, 1004), (1005, 1005)]

Observed output:

[(995, 996), (996, 997), (997, 998), (998, 999), (999, 1000), (1000, 1001), (1001, 1001), (1002, 1001), (1003, 1001), (1004, 1001), (1005, 1001)]