Problem after restart clickhouse-backup with backups

[ByJacob]

Error after restart when have one backup

1. Error log

   2023/03/23 13:09:56.714562  info SELECT count() AS is_macros_exists FROM system.tables WHERE database='system' AND name='macros' logger=clickhouse
   panic: interface conversion: interface {} is []interface {}, not []string
   
   goroutine 27 [running]:
   github.com/AlexAkulov/clickhouse-backup/pkg/server.(*APIServer).ResumeOperationsAfterRestart(0xc0001b40e0)
           /home/runner/work/clickhouse-backup/clickhouse-backup/pkg/server/server.go:1543 +0x10b1
   github.com/AlexAkulov/clickhouse-backup/pkg/server.Run.func1()
           /home/runner/work/clickhouse-backup/clickhouse-backup/pkg/server/server.go:111 +0x26
   created by github.com/AlexAkulov/clickhouse-backup/pkg/server.Run
           /home/runner/work/clickhouse-backup/clickhouse-backup/pkg/server/server.go:110 +0x56f
   
   

2. clickhouse-backup print-config

   bash-5.1# clickhouse-backup print-config
   general:
       remote_storage: sftp
       max_file_size: 0
       disable_progress_bar: true
       backups_to_keep_local: 7
       backups_to_keep_remote: 30
       log_level: info
       allow_empty_backups: false
       download_concurrency: 2
       upload_concurrency: 2
       use_resumable_state: true
       restore_schema_on_cluster: ""
       upload_by_part: true
       download_by_part: true
       restore_database_mapping: {}
       retries_on_failure: 3
       upload_retries_pause: 30s
       watch_interval: 1h
       full_interval: 24h
       watch_backup_name_template: shard{shard}-{type}-{time:20060102150405}
       retriesduration: 30s
       watchduration: 1h0m0s
       fullduration: 24h0m0s
   clickhouse:
       username: username
       password: ""
       host: 192.168.50.20
       port: 9000
       disk_mapping: {}
       skip_tables:
           - system.*
           - INFORMATION_SCHEMA.*
           - information_schema.*
           - _temporary_and_external_tables.*
       timeout: 5m
       freeze_by_part: true
       freeze_by_part_where: ""
       use_embedded_backup_restore: false
       embedded_backup_disk: ""
       secure: false
       skip_verify: false
       sync_replicated_tables: false
       log_sql_queries: true
       config_dir: /etc/clickhouse-server/
       restart_command: systemctl restart clickhouse-server
       ignore_not_exists_error_during_freeze: true
       check_replicas_before_attach: true
       tls_key: ""
       tls_cert: ""
       tls_ca: ""
       debug: false
   s3:
       access_key: ""
       secret_key: ""
       bucket: ""
       endpoint: ""
       region: us-east-1
       acl: private
       assume_role_arn: ""
       force_path_style: false
       path: ""
       disable_ssl: false
       compression_level: 1
       compression_format: tar
       sse: ""
       disable_cert_verification: false
       use_custom_storage_class: false
       storage_class: STANDARD
       custom_storage_class_map: {}
       concurrency: 1
       part_size: 0
       max_parts_count: 10000
       allow_multipart_download: false
       object_labels: {}
       debug: false
   gcs:
       credentials_file: ""
       credentials_json: ""
       credentials_json_encoded: ""
       bucket: ""
       path: ""
       compression_level: 1
       compression_format: tar
       debug: false
       endpoint: ""
       storage_class: STANDARD
       object_labels: {}
       custom_storage_class_map: {}
   cos:
       url: ""
       timeout: 2m
       secret_id: ""
       secret_key: ""
       path: ""
       compression_format: tar
       compression_level: 1
       debug: false
   api:
       listen: 192.168.50.20:7171
       enable_metrics: true
       enable_pprof: false
       username: apu_username
       password: password
       secure: false
       certificate_file: ""
       private_key_file: ""
       create_integration_tables: false
       integration_tables_host: ""
       allow_parallel: false
       complete_resumable_after_restart: true
   ftp:
       address: ""
       timeout: 2m
       username: ""
       password: ""
       tls: false
       path: ""
       compression_format: tar
       compression_level: 1
       concurrency: 2
       debug: false
   sftp:
       address: 192.168.50.20
       port: 4433
       username: ftp_username
       password: ""
       key: /id_ed25519
       path: /upload
       compression_format: tar
       compression_level: 1
       concurrency: 1
       debug: false
   azblob:
       endpoint_schema: https
       endpoint_suffix: core.windows.net
       account_name: ""
       account_key: ""
       sas: ""
       use_managed_identity: false
       container: ""
       path: ""
       compression_level: 1
       compression_format: tar
       sse_key: ""
       buffer_size: 0
       buffer_count: 3
       max_parts_count: 10000
       timeout: 15m
   custom:
       upload_command: ""
       download_command: ""
       list_command: ""
       delete_command: ""
       command_timeout: 4h
       commandtimeoutduration: 4h0m0s
   

Workaround: delete all states file and set USE_RESUMABLE_STATE=false

clickhouse-backup version: 2.2.0
clickhouse-version: 23.2.4.12

[Slach]

thanks for reporting, will fix it asap

[krgn]

I'm experiencing the same problem. Might there be a workaround that doesn't require deleting the state?

[Slach]

@ByJacob and @krgn, could you check altinity/clickhouse-backup:master docker image and confirm issue is resolved?

[krgn]

Hi @Slach - the fix is working for us, thank you! Would be great to cut a release from this soon (so we don't keep tracking :master)

[Slach]

@krgn just use 2.2.1, which released yesterday

[krgn]

Ah I see, I had not noticed since it wasn't a GH release. Thanks 👍