Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download from Azure Blob storage times out in 60 seconds #467

Closed
mxalis opened this issue Jul 7, 2022 · 3 comments · Fixed by #509
Closed

Download from Azure Blob storage times out in 60 seconds #467

mxalis opened this issue Jul 7, 2022 · 3 comments · Fixed by #509
Assignees
Milestone

Comments

@mxalis
Copy link

mxalis commented Jul 7, 2022

When downloading backups from Azure Blob storage, the default go library uses a 60 seconds timeout. This will cause downloads to fail (and therefore backup restore) if either your connection is slow or the file is large.

Can be fixed by adding the TryTimeout option to https://github.com/AlexAkulov/clickhouse-backup/blob/820961e12158a39772b370dd8460f408c4833f70/pkg/new_storage/azblob.go#L99

eg. increasing it to 15 minutes

s.Container = azblob.NewServiceURL(*u, azblob.NewPipeline(credential, azblob.PipelineOptions{
                                Retry: azblob.RetryOptions{
			                    TryTimeout: 15 * time.Minute,
			                    },
		                  })).NewContainerURL(s.Config.Container)

It would be great to have the timeout configurable with a reasonable default (eg 5 mins). Unfortunately I am a golang noob and cannot help you with a pull request

Discussion relating to the timeout: Azure/azure-storage-blob-go#60

@Slach Slach self-assigned this Jul 12, 2022
@Slach Slach added this to the 2.0.0 milestone Jul 12, 2022
@k0t3n
Copy link

k0t3n commented Jul 28, 2022

Faced same problem.
@Slach is it possible to apply fix for v1 version?

@Slach
Copy link
Collaborator

Slach commented Jul 28, 2022

@k0t3n
Copy link

k0t3n commented Jul 29, 2022

@Slach works like a charm, thanks!

@Slach Slach closed this as completed Jul 29, 2022
@Slach Slach mentioned this issue Sep 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants