Skip to content
This repository has been archived by the owner on May 21, 2024. It is now read-only.

fix/ota-3865/Log connectivity restored after an interruption #1463

Closed
wants to merge 1 commit into from

Conversation

Zee314159
Copy link
Contributor

Draft code.

auto manifest = AssembleManifest();
if (custom != Json::nullValue) {
manifest["custom"] = custom;
}
auto signed_manifest = uptane_manifest.signManifest(manifest);
HttpResponse response = http->put(config.uptane.director_server + "/manifest", signed_manifest);
if (response.isOk()) {
if (!connect) {
LOG_INFO << "Connectivity is restored.";
sendEvent<event::LogConnectivityRestored>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just logging is enough. This is probably easier than we were thinking. Sending an event isn't a bad idea but for now we can skip it.

EXPECT_NO_THROW(sota_client->initialize());
EXPECT_TRUE(sota_client->putManifestSimple());
EXPECT_EQ(num_events_LogConnectivityRestored, 1);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This general test format looks fine, although capturing logging output is slightly challenging, and I'd suggest putting this in uptane_network_test.cc, although to be honest this isn't really Uptane-specific.

@codecov-io
Copy link

codecov-io commented Dec 1, 2019

Codecov Report

Merging #1463 into master will increase coverage by 0.05%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1463      +/-   ##
==========================================
+ Coverage   80.51%   80.56%   +0.05%     
==========================================
  Files         184      184              
  Lines       11083    11087       +4     
==========================================
+ Hits         8923     8932       +9     
+ Misses       2160     2155       -5
Impacted Files Coverage Δ
src/libaktualizr/primary/sotauptaneclient.h 100% <ø> (ø) ⬆️
src/libaktualizr/primary/sotauptaneclient.cc 88.85% <100%> (+0.06%) ⬆️
src/libaktualizr/package_manager/ostreemanager.cc 77.52% <0%> (-1.5%) ⬇️
src/libaktualizr/storage/sqlstorage.cc 76.89% <0%> (+0.22%) ⬆️
src/aktualizr_info/main.cc 90.69% <0%> (+0.46%) ⬆️
src/libaktualizr/crypto/crypto.cc 86.85% <0%> (+0.79%) ⬆️
src/libaktualizr/storage/sqlstorage_base.cc 79.05% <0%> (+2.7%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b1da3e...d829078. Read the comment docs.

@Zee314159 Zee314159 force-pushed the fix/ota-3865 branch 4 times, most recently from 2ef60e1 to bad609d Compare December 2, 2019 11:39
@Zee314159
Copy link
Contributor Author


[2019-12-02 11:56:38.874602] [0x000000000404fcc0] [info]    Bootstrap empty SQL storage
4401
[2019-12-02 11:56:38.896849] [0x000000000404fcc0] [info]    Bootstraping DB to version 22
4402
[2019-12-02 11:56:39.130130] [0x000000000404fcc0] [trace]   meta with role root in repo director not present in db
4403
[2019-12-02 11:56:39.157554] [0x000000000404fcc0] [trace]   meta with role root in repo image not present in db
4404
[2019-12-02 11:56:39.214023] [0x000000000404fcc0] [trace]   device_id not present in db
4405
[2019-12-02 11:56:39.277134] [0x000000000404fcc0] [trace]   client_pkey not present in db
4406
[2019-12-02 11:56:39.316517] [0x000000000404fcc0] [trace]   client_cert not present in db
4407
[2019-12-02 11:56:39.347844] [0x000000000404fcc0] [trace]   ca_cert not present
4408
[2019-12-02 11:56:39.376881] [0x000000000404fcc0] [trace]   client_pkey not present in db
4409
[2019-12-02 11:56:39.837780] [0x000000000404fcc0] [trace]   post request body:{
4410
	"deviceId" : "tasty-raclette-116",
4411
	"ttl" : "logconnect"
4412
}
4413
[2019-12-02 11:56:39.858029] [0x000000000404fcc0] [trace]   response http code: 404
4414
[2019-12-02 11:56:39.858286] [0x000000000404fcc0] [trace]   response: 
4415
[2019-12-02 11:56:39.859687] [0x000000000404fcc0] [error]   Shared credential provisioning failed, response: 
4416
[2019-12-02 11:56:39.859988] [0x000000000404fcc0] [error]   Shared credential provisioning failed. Aborting initialization.
4417
../src/libaktualizr/uptane/uptane_network_test.cc:163: Failure
4418
Value of: result
4419
  Actual: false
4420
Expected: true

Any tips why my test failed at the first stage? I followed the "noerrors" test case to write this one. And testing::internal::CaptureStdout seems not like a right choice to capture LOG_INFO. Any other suggestions?

Copy link
Collaborator

@pattivacek pattivacek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

../src/libaktualizr/uptane/uptane_network_test.cc:163: Failure

From what I see that points to https://github.com/advancedtelematic/aktualizr/pull/1463/files#diff-8b2accdd7accfa663ac3e253db791a7eR163 which means initialization failed. CaptureStdout looks like it might be exactly what you want, but it's probably not the cause of the failure here.

*/
TEST(UptaneNetwork, LogConnectivityRestored) {
// testing::internal::CaptureStdout();
RecordProperty("zephyr_key", "OTA-991,TST-158");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can leave out this RecordProperty thing for now. Maybe someday that'll be important again, but for now, it isn't.

conf.provision.server = conf.provision.server.substr(conf.provision.server.size() - 2) + "11";
KeyManager keys(store, conf.keymanagerConfig());
Initializer initializer(conf.provision, store, http, keys, {});
result = initializer.isSuccessful();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea of feigning a broken connection by changing the server URL. However, does it require re-initialization every time? I was hoping we could just test this with calls to CheckUpdates. At any rate, we almost certainly can avoid extra instantiations of KeyManager.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At any rate, we almost certainly can avoid extra instantiations of KeyManager.

But Initializer initializer(conf.provision, store, http, keys, {}); need keys. And from doTestInit it looks re-initialization is needed. I'm not sure though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fairly confident that you can reuse the keys if not the entire initialization. To simulate a more realistic scenario, we should only initialize once and merely call CheckUpdates multiple times under different network conditions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should only initialize once and merely call CheckUpdates multiple times under different network conditions.

How can I get under different network conditions if I don't use initialize? I didn't find the function I need.

@Zee314159 Zee314159 force-pushed the fix/ota-3865 branch 6 times, most recently from 1e035c1 to d6c3ac2 Compare December 4, 2019 08:30
auto sota_client = std_::make_unique<SotaUptaneClient>(conf, store, http);
EXPECT_NO_THROW(sota_client->initialize());
EXPECT_NO_THROW(sota_client->fetchMeta());
EXPECT_NO_THROW(sota_client->checkUpdates());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're using the SotaUptaneClient functions directly, you probably want to just call initialize once and then fetchMeta each time. fetchMeta is actually what Aktualizr::CheckUpdates calls, and fetchMeta calls SotaUptaneClient::checkUpdates. It also is what calls putManifestSimple, which is where your change is located, so just calling SotaUptaneClient::checkUpdates won't trigger it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW if you make this test case a FRIEND_TEST of SotaUptaneClient (see sotauptaneclient.h for other examples) you can directly manipulate the Config object stored in SotaUptaneClient and then you can bypass needing to re-instantiate or re-initialize SotaUptaneClient.

@Zee314159 Zee314159 force-pushed the fix/ota-3865 branch 4 times, most recently from ffd825e to ea74192 Compare December 5, 2019 06:41

auto sota_client = std_::make_unique<SotaUptaneClient>(conf, store, http);
EXPECT_NO_THROW(sota_client->initialize());
result::UpdateCheck result = sota_client->fetchMeta();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got this failure HTTP/1.0 501 Unsupported method ('PUT'). Is auto http = std::make_shared<HttpClient>(); a susceptive target?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HttpFake seems to make no difference in local test.

@lbonn
Copy link
Contributor

lbonn commented Dec 5, 2019

It looks like it's failing right now because the "hasupdates" HttpFake should be used with a secondary.

Adding this line makes the fetchMeta call succeed. It would even be better to use makeTestConfig from uptane_test_common.h instead of this shared object between all tests.

UptaneTestCommon::addDefaultSecondary(conf, temp_dir, "secondary_ecu_serial", "secondary_hw");

@Zee314159
Copy link
Contributor Author

It looks like it's failing right now because the "hasupdates" HttpFake should be used with a secondary.
Adding this line makes the fetchMeta call succeed. It would even be better to use makeTestConfig from uptane_test_common.h instead of this shared object between all tests.
UptaneTestCommon::addDefaultSecondary(conf, temp_dir, "secondary_ecu_serial", "secondary_hw");

Thanks Laurent@lbonn ,yes, I tried your method, the fetchMeta call succeed, but when I try to simulate a broken connection, it seems still connected, and failed to directly manipulate the Config object stored in SotaUptaneClient. Any tips?

@lbonn
Copy link
Contributor

lbonn commented Dec 6, 2019

@Zee314159 Modifying the config object might work, or not if it has been copied before by an internal component.

The most straightforward thing that comes to my mind is to subclass HttpFake and add a new method to toggle a switch that causes all the requests to fail.

@Zee314159 Zee314159 force-pushed the fix/ota-3865 branch 3 times, most recently from 1b348cb to 5cbc053 Compare December 8, 2019 03:39
@Zee314159 Zee314159 force-pushed the fix/ota-3865 branch 7 times, most recently from 0f0dfad to bd82eb0 Compare December 8, 2019 16:26
(void)data;
return HttpResponse(url, 503, CURLE_OK, "");
} else {
if (url.find("/devices") != std::string::npos || url.find("/director/ecus") != std::string::npos || url.empty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we can't call HttpFake::post() here? Copying this logic here, would complicate maintenance.

Copy link
Contributor Author

@Zee314159 Zee314159 Dec 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, but the test will fail if I use the original. I'll have a test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

file served: "/tmp/aktualizr-4d8c-c7cb-fcae-9383/619b-47da-dir/repo/targets_hasupdates.json"
4429
[2019-12-09 12:59:48.635639] [0x000000000404fcc0] [info]    2 new updates found in Uptane metadata.
4430
[2019-12-09 12:59:48.659888] [0x000000000404fcc0] [info]    got UpdateCheckComplete event
4431
[2019-12-09 12:59:50.232787] [0x000000000404fcc0] [error]   Failed to update director metadata: 
4432
[2019-12-09 12:59:50.236768] [0x000000000404fcc0] [info]    got UpdateCheckComplete event
4433
../src/libaktualizr/uptane/uptane_network_test.cc:217: Failure
4434
Expected: (std::string::npos) != (testing::internal::GetCapturedStdout().find("Connectivity is restored.")), actual: 18446744073709551615 vs 1844674407370955161

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, you still need to handle the failure case. In the last version you've pushed, put and post are the same as in HttpFake.

If you follow the same pattern that what you did for get(), it should work.

e.g:

HttpResponse post(const std::string &url, const Json::Value &data) override {
 if (!connectSwitch) {
   (void)data;
   return HttpResponse(url, 503, CURLE_OK, "");
 } else {
   return HttpFake::post(url, data);
  }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see, thank you!

(void)data;
return HttpResponse(url, 504, CURLE_OK, "");
} else {
last_manifest = data;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also call HttpFake::put()?

@lbonn
Copy link
Contributor

lbonn commented Dec 9, 2019

Looking good apart from the two small comments. Also, before we merge, could you edit the commit message so that it describes the changes better?

Signed-off-by: Zee314159 <252806294@qq.com>
@lbonn
Copy link
Contributor

lbonn commented Dec 9, 2019

It merged in master but the PR didn't get closed somehow 👐

@lbonn lbonn closed this Dec 9, 2019
atsjenkins referenced this pull request in uptane/aktualizr Dec 9, 2019
fix/ota-3865/Log connectivity restored after an interruption
@pattivacek pattivacek deleted the fix/ota-3865 branch December 12, 2019 10:14
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants