Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse statusUploadBlob in vmSettings #2382

Merged
merged 10 commits into from
Oct 20, 2021

Conversation

narrieta
Copy link
Member

No description provided.

@codecov
Copy link

codecov bot commented Oct 18, 2021

Codecov Report

Merging #2382 (4e3f55b) into develop (e65b6d5) will increase coverage by 0.05%.
The diff coverage is 79.50%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2382      +/-   ##
===========================================
+ Coverage    71.09%   71.14%   +0.05%     
===========================================
  Files           97       97              
  Lines        14302    14384      +82     
  Branches      2064     2076      +12     
===========================================
+ Hits         10168    10234      +66     
- Misses        3677     3689      +12     
- Partials       457      461       +4     
Impacted Files Coverage Δ
azurelinuxagent/ga/exthandlers.py 85.74% <75.00%> (ø)
...inuxagent/common/protocol/extensions_goal_state.py 86.98% <78.50%> (-3.11%) ⬇️
azurelinuxagent/common/protocol/wire.py 80.44% <87.50%> (-0.07%) ⬇️
azurelinuxagent/common/exception.py 97.00% <100.00%> (+0.06%) ⬆️
azurelinuxagent/ga/collect_telemetry_events.py 90.41% <0.00%> (+0.68%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e65b6d5...4e3f55b. Read the comment docs.

kevinclark19a
kevinclark19a previously approved these changes Oct 19, 2021
Does validations common to vmSettings and ExtensionsConfig
"""
if self._status_upload_blob_type not in ["BlockBlob", "PageBlob"]:
logger.info("Status Blob type '{0}' is ot valid, assuming BlockBlob", self._status_upload_blob)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo , 'not'

dhivyaganesan
dhivyaganesan previously approved these changes Oct 19, 2021
kevinclark19a
kevinclark19a previously approved these changes Oct 20, 2021
dhivyaganesan
dhivyaganesan previously approved these changes Oct 20, 2021
Copy link
Contributor

@larohra larohra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments else LGTM

try:
self._do_common_validations()
except Exception as e:
raise ExtensionsConfigError("Error parsing ExtensionsConfig (incarnation: {0}): {1}\n{2}".format(incarnation, format_exception(e), xml_text))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we trim the xml text? Incase its very big? Is there a chance xml_text can contain secrets? We should redact it in that case too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These errors should be rare (don't remember ever seeing one in the wild) and the files are usually a few KB long. I would not worry about disk space on the log or event files; telemetry will truncate the message before sending it so no concern there,

Since they are rare, it is best not to truncate in order to have as much debugging info as possible.

The text should be redacted, though, thanks for pointing that out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while at it, i also changed get_redacted_text to return an empty string instead of None for empty extensions config.


self.in_vm_gs_metadata.parse_node(find(xml_doc, "InVMGoalStateMetaData"))

self._do_common_validations()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for validating in 2 different places? (Line 133 and 169)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to be really, really sure the text is valid...

No, seriously, the second call is a leftover, will remove it. Thanks

status_upload_blob = vm_settings.get("statusUploadBlob")
if status_upload_blob is None:
raise Exception("Missing statusUploadBlob")
self._status_upload_blob = status_upload_blob["value"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also check if value is present in the status_upload_blob and raise appropriate Exception if not (else a KeyError might be raised)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, should be a call to get(), not to the [] operator. Will fix it.

self._status_upload_blob = status_upload_blob["value"]
if self._status_upload_blob is None:
raise Exception("Missing statusUploadBlob.value")
self._status_upload_blob_type = status_upload_blob["statusBlobType"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

#
# TODO: The current implementation of the vmSettings API uses inconsistent cases on the names of the json items it returns.
# To work around that, we use _CaseFoldedDict to query those json items in a case-insensitive matter, Do not use
# _CaseFoldedDict for other purposes. Remove it once the vmSettings API is updated.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might never be able to remove this given the long tail nature of the HostGA deployments and also the fact that some hosts are never updated/not updated frequently. Just a thought to keep in mind when considering removing this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A future version of the API will include a version number and we'll send it over telemetry. That'll help us drive the removal/update of many of those TODO comments I've been adding.

@narrieta narrieta merged commit 87c9e1d into Azure:develop Oct 20, 2021
@narrieta narrieta deleted the parse-basic-settings branch October 20, 2021 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants