[data] move many datasource classes for internal #46169
Conversation
can-anyscale
force-pushed
the
can-dd02
branch
6 times, most recently
from
9425e8e
to
3ee0736
Compare
can-anyscale
requested review from
ericl,
scv119,
c21,
amogkam,
scottjlee,
bveeramani,
raulchen,
stephanie-wang and
omatthew98
as code owners
| @@ -1,12 +1,14 @@ | |||
| import logging | |||
There was a problem hiding this comment.
For consistency with the Datasource implementations, could we move the Datasink implementations to _internal and also remove leading underscores from the names (e.g., move ray.data.datasource.mongo_datasink._MongoDatasink to ray.data._internal.datasource.mongo_datasink.MongoDatasink?
python/ray/data/read_api.py
Outdated
| Connection, | ||
| CSVDatasource, | ||
| Datasource, | ||
| ImageDatasource, |
There was a problem hiding this comment.
ImageDatasource, LancecDatasource, TFRecordDatasource, and TorchDatasource are annotated as developer APIs but they should be internal. Could we remove the annotations and also move them to _internal? (Saw there's already a PR for TFRecordDatasource: #46171)
There was a problem hiding this comment.
actually if it's ok let me do it in a follow up PR to reduce the size of this one
There was a problem hiding this comment.
Move datasink classes to internal, as discussed in #46169 Test: - CI --------- Signed-off-by: can <can@anyscale.com>
Signed-off-by: can <can@anyscale.com>
Move three developerapi datasource classes to internal Test: - CI --------- Signed-off-by: can <can@anyscale.com>
Move many data source classes internal, according to https://docs.google.com/spreadsheets/d/1wQALnsUuFN4dMgmGITfiVX41fCc6P6FEjYVO6jDWGtM/edit#gid=1683753102
Test: