Add this week in Databend 44

[PsiACE]

ty!

---

COPY INTO Databend with COMPRESSION Option

After PR 5655, Databend started to support decompression reads in COPY INTO and streaming leading.

Loading Compressed Files from Amazon S3

Try to load data from a gzip compressed csv and insert into mytable.

COPY INTO mytable
  FROM s3://mybucket/data.csv.gz
  credentials=(aws_key_id='<AWS_ACCESS_KEY_ID>' aws_secret_key='<AWS_SECRET_ACCESS_KEY>')
  FILE_FORMAT = (type = "CSV" field_delimiter = ',' record_delimiter = '\n' 
  skip_header = 1 compression = GZIP) size_limit=10;

COMPRESSION Option

The COMPRESSION option is a string that represents the compression algorithm.

| Values        | Notes                                                           |
| ------------- | --------------------------------------------------------------- |
| `AUTO`        | Auto detect compression via file extensions                     |
| `GZIP`        |                                                                 |
| `BZ2`         |                                                                 |
| `BROTLI`      | Must be specified if loading/unloading Brotli-compressed files. |
| `ZSTD`        | Zstandard v0.8 (and higher) is supported.                       |
| `DEFLATE`     | Deflate-compressed files (with zlib header, RFC1950).           |
| `RAW_DEFLATE` | Deflate-compressed files (without any header, RFC1951).         |
| `NONE`        | Indicates that the files have not been compressed.              |

Learn more:

• Doc | COPY INTO <table>
• PR 5655 | Add decompress support for COPY INTO and streaming loading

[ericseppanen]

Looks like this missed the cutoff for last week's issue; I'll move this to the 2022-06-08 draft.