Warn when download: warn remote file backup extension 2022-05-16T19-00-08/shadow/mydatabase/mytable/default_202202_24_24_1.tar.gz not equal with gzip

[fzyzcjy]

Hi thanks for the library! However it has warnings when downloading

Error log: warn remote file backup extension 2022-05-16T19-00-08/shadow/mydatabase/mytable/default_202202_24_24_1.tar.gz not equal with gzip

[Slach]

Could you share
clickhouse-backup print-config results without your credentials?

[fzyzcjy]

Thanks for the reply! Sure:

general:
  remote_storage: s3
  max_file_size: 0
  disable_progress_bar: true
  backups_to_keep_local: 0
  backups_to_keep_remote: 0
  log_level: info
  allow_empty_backups: false
  download_concurrency: 2
  upload_concurrency: 2
  restore_schema_on_cluster: ""
  upload_by_part: true
  download_by_part: true
clickhouse:
  username: default
  password: ...
  host: clickhouse
  port: 9000
  disk_mapping: {}
  skip_tables:
  - system.*
  - tutorial.*
  timeout: 5m
  freeze_by_part: false
  secure: false
  skip_verify: false
  sync_replicated_tables: false
  log_sql_queries: true
  config_dir: /etc/clickhouse-server/
  restart_command: systemctl restart clickhouse-server
  ignore_not_exists_error_during_freeze: true
  tls_key: ""
  tls_cert: ""
  tls_ca: ""
  debug: false
s3:
  access_key: ..
  secret_key: ..
  bucket: ..
  endpoint: ..
  region: ..
  acl: private
  assume_role_arn: ""
  force_path_style: false
  path: clickhouse
  disable_ssl: false
  compression_level: 1
  compression_format: gzip
  sse: ""
  disable_cert_verification: false
  storage_class: STANDARD
  concurrency: 1
  part_size: 0
  max_parts_count: 10000
  debug: false
gcs:
  credentials_file: ""
  credentials_json: ""
  bucket: ""
  path: ""
  compression_level: 1
  compression_format: tar
  debug: false
  endpoint: ""
cos:
  url: ""
  timeout: 2m
  secret_id: ""
  secret_key: ""
  path: ""
  compression_format: tar
  compression_level: 1
  debug: false
api:
  listen: 0.0.0.0:7171
  enable_metrics: true
  enable_pprof: false
  username: ""
  password: ""
  secure: false
  certificate_file: ""
  private_key_file: ""
  create_integration_tables: false
  allow_parallel: false
ftp:
  address: ""
  timeout: 2m
  username: ""
  password: ""
  tls: false
  path: ""
  compression_format: tar
  compression_level: 1
  concurrency: 2
  debug: false
sftp:
  address: ""
  port: 22
  username: ""
  password: ""
  key: ""
  path: ""
  compression_format: tar
  compression_level: 1
  concurrency: 1
  debug: false
azblob:
  endpoint_suffix: core.windows.net
  account_name: ""
  account_key: ""
  sas: ""
  use_managed_identity: false
  container: ""
  path: ""
  compression_level: 1
  compression_format: tar
  sse_key: ""
  buffer_size: 0
  buffer_count: 3
  max_parts_count: 0

[fzyzcjy]

@Slach The fix is so quick, great job :)

Btw it is having an error here: #442

[Slach]

check it
https://github.com/AlexAkulov/clickhouse-backup/releases/tag/v1.4.0

[fzyzcjy]

Well with 1.4.0 it still exists:

2022/05/17 08:14:16.315455 debug finish download from 2022-05-16T19-00-08/shadow/yplusplus_prod_op_main/request_info/default_202202_20_20_19935.tar.gz
2022/05/17 08:14:16.315517 debug start download from 2022-05-16T19-00-08/shadow/yplusplus_prod_op_main/request_info/default_202110_12_12_3242.tar.gz
2022/05/17 08:14:16.364123  warn remote file backup extension 2022-05-16T19-00-08/shadow/yplusplus_prod_op_main/request_info/default_202110_12_12_3242.tar.gz not equal with gzip

[Slach]

could you share
clickhouse-backup --version ? i'm published 1.4.0 twice (first time without fix)

[fzyzcjy]

Will check it. thanks

[fzyzcjy]

bash-5.1# clickhouse-backup --version
Version:         1.4.0
Git Commit:      81babc91fd2ec43488cebb68c80e8ff7f9f66f33
Build Date:      2022-05-17

[Slach]

as quick workaround you could use
compression_format: gz in your config s3 section

fix will prepared in next release

[fzyzcjy]

I see. Thanks!

[fzyzcjy]

Well,

bash-5.1# clickhouse-backup list local
2022/05/19 09:42:26.564986 error 'gz' is unsupported compression format

bash-5.1# clickhouse-backup --version
Version:         1.4.0
Git Commit:      81babc91fd2ec43488cebb68c80e8ff7f9f66f33
Build Date:      2022-05-17