You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The stats are missing (empty string) from DatasetIterators resulting from streaming_split(), after execution. See below for reproducible example.
Versions / Dependencies
Ray master, Python 3.9
Reproduction script
def pause(x):
time.sleep(.0001)
return x
ds = ray.data.range(10000)
ds = ds.map(lambda x: x)
ds = ds.map(pause)
@ray.remote
def consume(p):
for x in p.iter_batches():
pass
print("Finish consume")
stats = p.stats()
print(f"Emit DatasetStats: {stats}")
print(type(stats))
a, b = ds.streaming_split(2)
ray.get([consume.remote(a), consume.remote(b)])
Issue Severity
None
The text was updated successfully, but these errors were encountered:
scottjlee
added
bug
Something that is supposed to be working; but isn't
triage
Needs triage (eg: priority, bug/not-bug, and owning component)
P1
Issue that should be fixed within a few weeks
data
Ray Data-related issues
and removed
triage
Needs triage (eg: priority, bug/not-bug, and owning component)
labels
Jun 2, 2023
What happened + What you expected to happen
The stats are missing (empty string) from
DatasetIterator
s resulting fromstreaming_split()
, after execution. See below for reproducible example.Versions / Dependencies
Ray master, Python 3.9
Reproduction script
Issue Severity
None
The text was updated successfully, but these errors were encountered: