Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Export query result supports standard CSV format #6954

Open
2 of 3 tasks
CUITCHE opened this issue Oct 28, 2021 · 1 comment
Open
2 of 3 tasks

[Feature] Export query result supports standard CSV format #6954

CUITCHE opened this issue Oct 28, 2021 · 1 comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@CUITCHE
Copy link

CUITCHE commented Oct 28, 2021

Search before asking

  • I had searched in the issues and found no similar issues.

Description

At present, when exporting CSV data, it is simply separated by fields with specified separator, rather than strictly according to the standard CSV format.
For example, if the field contains a comma, Doris will not use the double quotation mark wrapper field when exporting, resulting in wrong columns in the final result set.

For Table(t1) data storage is ['x1,', 'x2', f2, f3].
Exporting text is x1,,x2,f2,f3. Obviously, not what we expected.

Use case

In CSV format, we could add a key-value to PROPERTIES to indicate that the exporting action will be exported by standard CSV format.

The key-value may be "csv.format.standard" = "true"

Example

SELECT * FROM tbl
INTO OUTFILE "hdfs:/path/to/result_"
FORMAT AS CSV
PROPERTIES
(
    "broker.name" = "my_broker",
    "broker.hadoop.security.authentication" = "kerberos",
    "broker.kerberos_principal" = "doris@YOUR.COM",
    "broker.kerberos_keytab" = "/home/doris/my.keytab",
    "column_separator" = ",",
    "line_delimiter" = "\n",
    "max_file_size" = "100MB",
    "csv.format.standard" = "true"
);

Related issues

#7552

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@CUITCHE CUITCHE added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 28, 2021
@geoffreytran
Copy link

I'm also running into this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

2 participants