migration-helpers: AddSettingsMigration, RemoveSettingsMigration for migrating multiple settings at once

[etungsten]

Issue number:
N/A

Description of changes:

Author: Erikson Tung <etung@amazon.com>
Date:   Thu Jun 25 17:05:41 2020 -0700

    migration-helpers: `RemoveSettingsMigration` to remove multiple settings at once
    
    Adds new migration helper `RemoveSettingsMigration` to remove multiple
    settings at once.
    
    Original `RemoveSettingMigration` now wraps `RemoveSettingsMigration`
    and is now deprecated.

Author: Erikson Tung <etung@amazon.com>
Date:   Thu Jun 25 13:42:16 2020 -0700

    migration-helpers: AddSettingsMigration handle more than one setting
    
    Adds new migration helper `AddSettingsMigration`.
    `AddSettingsMigration` handles migrating more than one setting.
    
    Original `AddSettingMigration` now wraps `AddSettingsMigration` and is
    now deprecated

Testing done:
Created my own datastore locally with storewolf.
Running apiserver locally, committed initial bottlerocket-launch transaction with settings-commiter.
The source datastore has the following content for update settings:

$ ls -al ds/current/live/settings/updates
total 32
drwxr-xr-x 2 etung domain^users 4096 Jun 25 14:05 .
drwxr-xr-x 6 etung domain^users 4096 Jun 25 14:05 ..
-rw-r--r-- 1 etung domain^users    5 Jun 25 14:05 ignore-waves
-rw-r--r-- 1 etung domain^users   46 Jun 25 14:01 metadata-base-url.setting-generator
-rw-r--r-- 1 etung domain^users   80 Jun 25 14:01 metadata-base-url.template
-rw-r--r-- 1 etung domain^users   11 Jun 25 14:01 seed.setting-generator
-rw-r--r-- 1 etung domain^users   43 Jun 25 14:05 targets-base-url
-rw-r--r-- 1 etung domain^users    8 Jun 25 14:05 version-lock

Tried the following with a migration using this helper:
Forward migration:

Running `add-version-lock-ignore-waves --source-datastore ~/thar/testing/ds/current --target-datastore ~/thar/testing/ds/next --forward`
AddSettingsMigration(["settings.updates.version-lock", "settings.updates.ignore-waves"]) has no work to do on upgrade.

Observed that the temporary datastore created matches the source datastore. Which is expected

Backward migration:

     Running `add-version-lock-ignore-waves --source-datastore ~/thar/testing/ds/current --target-datastore ~/thar/testing/ds/next --backward`
Removed settings.updates.version-lock, which was set to '"latest"'
Removed settings.updates.ignore-waves, which was set to 'false'

Observed that the temporary datastore no longer has the version-lock and ignore-waves settings.

$ ls -al ds/next/live/settings/updates
total 24
drwxr-xr-x 2 etung domain^users 4096 Jun 25 15:03 .
drwxr-xr-x 6 etung domain^users 4096 Jun 25 15:03 ..
-rw-r--r-- 1 etung domain^users   46 Jun 25 15:03 metadata-base-url.setting-generator
-rw-r--r-- 1 etung domain^users   80 Jun 25 15:03 metadata-base-url.template
-rw-r--r-- 1 etung domain^users   11 Jun 25 15:03 seed.setting-generator
-rw-r--r-- 1 etung domain^users   43 Jun 25 15:03 targets-base-url

Testing the original AddSettingMigration (singular setting)

Backwards migration to remove a single version-lock setting:

Running `add-version-lock-ignore-waves --source-datastore ~/thar/testing/ds/current --target-datastore ~/thar/testing/ds/next --backward`
Removed settings.updates.version-lock, which was set to '"latest"'

Resultant temporary datastore where version-lock no longer exists.

$ ls -al ds/next/live/settings/updates
total 28
drwxr-xr-x 2 etung domain^users 4096 Jun 25 15:07 .
drwxr-xr-x 6 etung domain^users 4096 Jun 25 15:07 ..
-rw-r--r-- 1 etung domain^users    5 Jun 25 15:07 ignore-waves
-rw-r--r-- 1 etung domain^users   46 Jun 25 15:07 metadata-base-url.setting-generator
-rw-r--r-- 1 etung domain^users   80 Jun 25 15:07 metadata-base-url.template
-rw-r--r-- 1 etung domain^users   11 Jun 25 15:07 seed.setting-generator
-rw-r--r-- 1 etung domain^users   43 Jun 25 15:07 targets-base-url

For RemoveSettingsMigration, I inverted the forward/backward testing from the steps above and they work as expected.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[tjkirch]

I think we should do the same for removing settings with a RemoveSettingsMigration; the same problem would happen there, in reverse.

[etungsten]

Push above addresses @tjkirch 's comments.

• Adds a new RemoveSettingsMIgration that remove multiple settings at once.
• Deprecated RemoveSettingMigration

Tested the migrations and they still work as expected.