feat: Implement support for multi-character comments in read_csv
#12519
Conversation
|
Sorry, I don't want this in polars. I am sorry about this one. This had to be documented better. |
|
Ok, I see this one is only comments. This should be acceptable. |
demetribu
force-pushed
the
feature/multi-char-comment-support-10583
branch
from
60a8de2
to
8df1694
Compare
demetribu
requested review from
stinodego,
alexander-beedie and
MarcoGorelli
as code owners
demetribu
force-pushed
the
feature/multi-char-comment-support-10583
branch
from
8df1694
to
c1d79fb
Compare
demetribu
changed the title
demetribu
changed the title
github-actions
bot
added
the
enhancement
|
Thanks, can you also expose it to python as well? |
crates/polars-lazy/src/scan/csv.rs
Outdated
| #[must_use] | ||
| pub fn with_comment_char(mut self, comment_char: Option<u8>) -> Self { | ||
| self.comment_char = comment_char; | ||
| /// Set a single byte character as the comment character. |
There was a problem hiding this comment.
Should update the comment here (it mentions 'single-byte character')
crates/polars/tests/it/io/csv.rs
Outdated
| assert_eq!(df.shape(), (3, 5)); | ||
|
|
||
| let csv = r"1,2,3,4,5 | ||
| ### this is a comment |
There was a problem hiding this comment.
This would work equally well with comment_char being a single "#" character; could you update the example to something that can only work with a multi-character comment prefix?
|
We should also probably rename the |
stinodego
changed the title
read_csv
github-actions
bot
added
python
crates/polars-lazy/src/scan/csv.rs
Outdated
| if c.is_ascii() { | ||
| Some(CommentChar::Single(c as u8)) | ||
| } else { | ||
| None // Or handle non-ASCII characters as needed |
There was a problem hiding this comment.
Shouldn't that fallback to Multi then?
|
@alexander-beedie regarding comment char/prefix. I've considered this approach as well. However, these changes will impact the user experience. If the Polars team is in agreement, I am ready to implement it. |
demetribu
force-pushed
the
feature/multi-char-comment-support-10583
branch
4 times, most recently
from
c7184d9
to
850a61c
Compare
You can use the decorator |
demetribu
force-pushed
the
feature/multi-char-comment-support-10583
branch
from
850a61c
to
22b1f5d
Compare
This pull request introduces the capability to handle multi-character and single-byte comment identifiers in the read_csv function, addressing the feature request in issue #10583.