Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Host plugin updates #505

Merged
merged 2 commits into from
Dec 6, 2016
Merged

Conversation

hglkrijger
Copy link
Member

These changes allow the agent to function normally even while outbound connectivity to the public internet is blocked, via NSG for example.

  • enables host plugin downloads for extension and agent manifests and packages
  • fixes an issue uploading the extension health via the host plugin
  • fixes issues whereby the agent manifest and package are downloaded multiple times
  • refactor and cleanup
  • adds unit tests

/cc @jinhyunr @brendandixon

@hglkrijger hglkrijger added this to the v2.2.2 milestone Dec 6, 2016
Copy link
Contributor

@jinhyunr jinhyunr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments

@@ -282,16 +282,16 @@ def get_ext_handlers(self):
def get_ext_handler_pkgs(self, extension):
raise NotImplementedError()

def get_in_vm_artifacts_profile(self):
def get_artifacts_profile(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if in_vm_artifacts_profile equals to artifacts_profile.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are just so many similar names which contains artifacts or profile(i.e. extension artifacts) so I am just worried that there might be another thing called 'artifact profile' different from 'in_vm_artifacts_profile'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can always change it, but at the moment this is the only artifact profile. It just seems unnecessary to qualify it as in_vm_.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, leaving as is.

url = URI_FORMAT_PUT_VM_STATUS.format(self.endpoint, HOST_PLUGIN_PORT)
logger.verbose("Posting VM status to host plugin")
status = textutil.b64encode(status_blob.data)
blob_type = status_blob.type if status_blob.type else config_blob_type
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be better to use get_blob_type() here and let the function handle this fallback.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this could be improved, but I think the correct change is to remove get_blob_type since the HEAD call is not actually necessary anymore. For this PR I did not want to make more changes than necessary, but if you feel we should include that let me know.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, I am going to add an event to be emitted which we can use to determine how reliable the given value is before we switch over to it.

else:
host.manifest_uri = version.uri
logger.verbose("Manifest downloaded successfully from host plugin")
if response:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

else?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

else we move on to the next uri, line 632.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline

if not response:
logger.info("Retry fetch in {0} seconds",
LONG_WAITING_INTERVAL)
time.sleep(LONG_WAITING_INTERVAL)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the retry work here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fetch calls http_get which has retries built-in.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline

try:
uploaded = self.status_blob.upload(ext_conf.status_upload_blob)
except (HttpError, ProtocolError) as e:
# errors have already been logged
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we let StatusBlob to use blobtype from extensionConfig as a fallback when it cannot retrieve one by making a request, status_blob.upload() should not throw HttpError/ProtocolError.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if we did that the actual upload would subsequently fail and still throw the protocol error.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline

uri, headers = host.get_artifact_request(blob)
profile = self.decode_config(self.fetch(uri, headers))

if profile and not profile.isspace():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use textutil.is_str_none_or_whitespace() instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, updated

if not textutil.is_str_none_or_whitespace(in_vm_artifacts_profile_json):
self.__dict__.update(parse_json(in_vm_artifacts_profile_json))
def __init__(self, artifacts_profile):
if artifacts_profile and not artifacts_profile.isspace():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason for reverting it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, bad merge - thanks for catching.

@brendandixon
Copy link
Contributor

@hglkrijger LGTM. Nice.

@hglkrijger hglkrijger force-pushed the host-plugin-artifacts branch 2 times, most recently from 85d5f3a to f177e92 Compare December 6, 2016 21:38
@hglkrijger hglkrijger merged commit 92fe3f2 into Azure:master Dec 6, 2016
@hglkrijger hglkrijger deleted the host-plugin-artifacts branch December 6, 2016 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants