Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up supporting code to compare ExtensionsConfig with vmSettings #2378

Merged
merged 4 commits into from
Oct 8, 2021

Conversation

narrieta
Copy link
Member

@narrieta narrieta commented Oct 7, 2021

This PR prepares the code to compare ExtensionsConfig with vmSettings.

For now, it only compares the RequiredFeatures; subsequent PRs will parse and compare other attributes.

@@ -82,13 +82,6 @@ def __init__(self):
self.vmAgentManifests = DataContractList(VMAgentManifest)


class RequiredFeature(DataContract):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vmSettings does not expose a Value. Since it is not used anyway, I simplified the code to use simple strings instead of instances of RequiredFeature

if force_update or self._goal_state is None or self._goal_state.incarnation != goal_state.incarnation:
if force_update or \
self._goal_state is None or self._extensions_goal_state is None or \
self._goal_state.incarnation != goal_state.incarnation:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a check on self._extensions_goal_state is None. Checking only on self._goal_state is not enough since an exception may leave the latter initialized and the former uninitialized.

@codecov
Copy link

codecov bot commented Oct 7, 2021

Codecov Report

Merging #2378 (e73455c) into develop (6884f7c) will increase coverage by 0.10%.
The diff coverage is 90.19%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2378      +/-   ##
===========================================
+ Coverage    70.89%   70.99%   +0.10%     
===========================================
  Files           97       97              
  Lines        14107    14142      +35     
  Branches      2029     2034       +5     
===========================================
+ Hits         10001    10040      +39     
+ Misses        3657     3654       -3     
+ Partials       449      448       -1     
Impacted Files Coverage Δ
azurelinuxagent/common/protocol/restapi.py 97.38% <ø> (-0.06%) ⬇️
azurelinuxagent/common/protocol/wire.py 78.52% <81.81%> (-0.08%) ⬇️
...inuxagent/common/protocol/extensions_goal_state.py 90.08% <92.30%> (+0.55%) ⬆️
azurelinuxagent/common/event.py 85.32% <100.00%> (+0.03%) ⬆️
azurelinuxagent/ga/exthandlers.py 86.25% <100.00%> (ø)
azurelinuxagent/ga/collect_telemetry_events.py 92.12% <0.00%> (+2.39%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6884f7c...e73455c. Read the comment docs.

self._save_cache(redacted_vm_settings_text, VM_SETTINGS_FILE_NAME.format(self._vm_settings_etag))
if self._extensions_goal_state is not None:
# disable=assignment-from-none: E1128: Assigning result of a function call, where the function returns None
text = self._extensions_goal_state.get_redacted_text() # pylint: disable=assignment-from-none
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_redacted_text() can return None; suppressing warning

@narrieta narrieta changed the title Setup supporting code to compare ExtensionsConfig with vmSettings Set up supporting code to compare ExtensionsConfig with vmSettings Oct 7, 2021
Copy link
Contributor

@larohra larohra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good, just a couple of small comments?
Maybe add a test case for ensuring we compare the attributes and report it properly on mismatch? (I didn't see a test case here, not sure if already added)

@staticmethod
def compare(from_extensions_config, from_vm_settings):
"""
Compares the current instance against 'other' and logs a GoalStateMismatch message if they are different
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: The comment suggests that the self will be compared with other but the function definition is a static class comparing 2 different arguments.

Maybe modify the comment a bit to make it a bit more clearer?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, initially this was a member function and i made it static without updating the comment; will do that

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

super(_ExtensionsGoalStateFromExtensionsConfig, self).__init__()
self._id = incarnation
self._text = xml_text
self._type = 'ExtensionsConfig'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make the type an Enum - ExtensionsConfig vs VmSettings?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may removed this. I mean to use for logging purposes, but the name of the class is already pretty descriptive so I think I'll just use that

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if required_features is not None:
if not isinstance(required_features, list):
raise Exception("requiredFeatures should be an array")
self.required_features.append(required_features)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be extend over append?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, will fix it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

azurelinuxagent/common/protocol/wire.py Show resolved Hide resolved
ExtensionsConfig when the goal state is retrieved from the WireServe or from vmSettings when it is retrieved from
the HostGAPlugin.

NOTE: Do not instantiate this class directly, use one of the create_* methods instead.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be worth making an interface and providing the create classes there? i.e. making this a private class so that we don't have to rely on this docstring?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure i follow your suggestion, how would you express that in code?

@@ -1204,6 +1204,42 @@ def test_update_goal_state_should_not_persist_the_protected_settings(self):
len(protected_settings),
"Could not find the expected number of redacted settings. Expected {0}.\n{1}".format(len(protected_settings), extensions_config))

def test_update_goal_state_should_save_goal_state(self):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved from test_goal_state.py

@@ -31,71 +20,3 @@ def test_fetch_goal_state_should_raise_on_incomplete_goal_state(self, _):
GoalState(protocol.client)
self.assertEqual(_NUM_GS_FETCH_RETRIES, mock_sleep.call_count, "Unexpected number of retries")

def test_update_goal_state_should_save_goal_state(self, _):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to test_wire.py

for e in extensions:
self.assertEqual(e["settings"][0]["protectedSettings"], "*** REDACTED ***", "The protected settings for {0} were not redacted".format(e["name"]))

def test_update_vm_settings_should_should_retry_on_resource_gone_error(self, _):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved to test_extensions_goal_state.py



class ExtensionsGoalStateTestCase(HttpRequestPredicates, AgentTestCase):
def test_fetch_vm_settings_should_should_retry_on_resource_gone_error(self):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved from test_goal_state.py

self.assertEqual("GET_VM_SETTINGS_TEST_CONTAINER_ID", request_headers[1][hostplugin._HEADER_CONTAINER_ID], "The retry request did not include the expected header for the ContainerId")
self.assertEqual("GET_VM_SETTINGS_TEST_ROLE_CONFIG_NAME", request_headers[1][hostplugin._HEADER_HOST_CONFIG_NAME], "The retry request did not include the expected header for the RoleConfigName")

def test_compare_should_report_mismatches_between_extensions_config_and_vm_settings(self):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new test


def test_compare_should_report_mismatches_between_extensions_config_and_vm_settings(self):
from_vm_settings = ExtensionsGoalState.create_from_vm_settings("123", fileutil.read_file(os.path.join(data_dir, "hostgaplugin/vm_settings.json")))
from_extensions_config = ExtensionsGoalState.create_from_vm_settings("123", fileutil.read_file(os.path.join(data_dir, "hostgaplugin/vm_settings.json")))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be from extension config?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should have a jar and put 1 dollar for each darn copy and paste bug

then have a big party on each release

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@narrieta narrieta merged commit dbe9b30 into Azure:develop Oct 8, 2021
@narrieta narrieta deleted the parse-vm-settings branch October 8, 2021 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants