-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Cloud support for NDJSON #17717
Conversation
2f1f064
to
ca3a5fe
Compare
ca3a5fe
to
ab84f0a
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #17717 +/- ##
==========================================
+ Coverage 80.39% 80.42% +0.03%
==========================================
Files 1501 1501
Lines 196758 196830 +72
Branches 2793 2794 +1
==========================================
+ Hits 158178 158306 +128
+ Misses 38067 38010 -57
- Partials 513 514 +1 ☔ View full report in Codecov by Sentry. |
@@ -344,7 +344,8 @@ pub(crate) fn into_py(py: Python<'_>, plan: &IR) -> PyResult<PyObject> { | |||
#[cfg(feature = "ipc")] | |||
FileScan::Ipc { .. } => return Err(PyNotImplementedError::new_err("ipc scan")), | |||
#[cfg(feature = "json")] | |||
FileScan::NDJson { options } => { | |||
FileScan::NDJson { options, .. } => { | |||
// TODO: Also pass cloud_options |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is for the GPU engine, this probably needs updates on that side as well so I've left a todo
It is quite a common format, especially in LLM datasets