Resilient connection

[chkeita]

Summary of the Pull Request

Using an exponential backoff strategy for the http calls in the agent.

PR Checklist

• Applies to work item: supervisor connection to storage queue is not resilient  #79
• CLA signed. If not, go over here and sign the CLI.
• Tests added/passed
• Requires documentation to be updated
• I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Info on Pull Request

What does this include?

Validation Steps Performed

How does someone test & validate?

[bmc-msft]

In general, this is extremely positive change.

I didn't know defining traits for third-party structs was allowed. This is significantly better than the exponential backoff code added to AVML

Failures on CICD

[bmc-msft]

Every task in testing is failing with

[
'task failed. exit_status:code=1 signal=None success=False', 
'', 
"[2020-10-26T22:43:50Z ERROR onefuzz_agent] error running task: Couldn't upload file.  path:/onefuzz/3f059c05-2b1b-44b9-985f-ae1c6135ad1b/task_inputs_2/eb408ddc4fa484e6befdf5954e56a2198c7a9fab dir:task_inputs_2 err:This request cannot be retried because it cannot be cloned\nError: Couldn't upload file.  path:/onefuzz/3f059c05-2b1b-44b9-985f-ae1c6135ad1b/task_inputs_2/eb408ddc4fa484e6befdf5954e56a2198c7a9fab dir:task_inputs_2 err:This request cannot be retried because it cannot be cloned\n"
]

[bmc-msft]

Performing integration testing now