Zero downtime on non-upgrade deployments

[davidalger]

The current deployment process uses bin/magento setup:upgrade for the purpose of upgrading the schema and data in a Magento 2 installation if needed. This command also re-generates the app/etc/config.php file. Due to the errors Magento 2 will throw up if a site has modules requiring schema and/or data updates installed, the site must be put into maintenance mode before linking the new release and running setup:upgrade. This results in a downtime between 5 and 8 seconds on average (by my rather unscientific estimation based on real-life experience… ;)).

Stated goal of this task: eliminate all downtime for deployments which do not trigger schema or data upgrades.

Additionally, as the app/etc/config.php file is regenerated by the setup:ugprade command, this causes a potential fringe situation on a multi-node cluster deployment (as of 0.5.3) where setup:upgrade updates this file on one (but not all) nodes in the deployment. This is only an issue if the config.php file re-created when this runs is different (for some reason or other) than the one committed to the repository. Eliminating the use of setup:upgrade will have the effect of eliminating this potential issue as well.

At the suggestion of @hostep over on issue #33 I went looking into the use of setup:db:status and found the following:

1. The built-in setup:db:status command will report on whether or not any installed module(s) require schema and/or data upgrade. Example output of this command:

   All modules are up to date.
   

   or

   The module code base doesn't match the DB schema and data.
            Magento_Cms     schema:       2.0.0  ->  2.0.1      
         Magento_Bundle       data:       2.0.0  ->  2.0.2      
   Run 'setup:upgrade' to update your DB schema and data.
   

The command returns the same exit code of 0 regardless of the db status.

1. There are also two other interesting built-in console commands, setup:db-schema:upgrade and setup:db-data:upgrade. Upon my investigation into the code, the difference between these commands and setup:upgrade is the following:

   setup:upgrade calls updateModulesSequence, installSchema, and installDataFixtures whereas the other two commands only call installSchema and installDataFixtures respectively.

2. These additional commands are available in versions 2.0 and 2.1 making no special version checks necessary.

The result of this task will be to cease use of setup:upgrade and transition to only calling the schema/data upgrade specific tasks on deploy, eliminating the potential fringe issue with a deploy creating unexpected changes to config.php as well as eliminate all downtime on deploys which do not introduce module (schema and/or data) version changes.

[michael811]

When will you integrate this enhancement to deploy.rake?

[davidalger]

@michael811 Unfortunately it's still a work-in-progress and I have no ETA, but I hope to have it complete in the next couple of weeks. Been working on it a good bit over the last few days, and am making progress though! :)

[PascalBrouwers]

@davidalger care to look another look at this feature? Would really love to see it.

[alinalexandru]

At this moment setup:db-schema:upgrade and setup:db-data:upgrade are executed at each deploy, even though it is not necessary.
Some how the setup:db:status is not executed to check if it's still necessary to do db changes.

[adamj88]

@alinalexandru - it's mentioned in the original issue why this is the case.

setup:db:status - returns a successful exit code every time it's ran, regardless if the DB requires upgrading.

It's not possible to know programatically if the upgrade commands are needed or not, so they will run every time to ensure the database is upgraded when needed.

[alinalexandru]

I saw that the output is checked. So there is a way to check if is needed or not

[davidalger]

Alright everybody, @roman204 sent in PR #104 which got me started on this more in earnest. I'm polishing it up and it'll be out soon as v0.8.0! Once it's done, you'll see something like the following in a deployment:

04:13 magento:maintenance:check
      Checking database status...
      All modules are up to date. No database updates should occur during this release.

      <release_path>/app/etc/config.php hash: 46441a938bf279b86f5cf23ca07d0244
      <current_path>/app/etc/config.php hash: 46441a938bf279b86f5cf23ca07d0244

      Disabling use of maintenance mode for a zero-down deployment.

Maintenance mode disablement is going to work by setting :magento_deploy_maintenance to false when the checks indicates maintenance mode is not required. Both of the following will need to pass for this to happen:

1. Magento CLI must report "All modules are up to date" in the setup:db:status output.

2. The md5sum hash of the app/etc/config.php must match between the new and current release (if they differ, the app:config:import is necessary and cause for maintenance mode).

[davidalger]

And an example of where the config.php has not changed, but some modules are requiring db updates:

04:17 magento:maintenance:check
      Checking database status...
      The module code base doesn't match the DB schema and data.
         Magento_Directory     schema:       2.0.1  ->  2.0.2      
         Magento_Directory       data:       2.0.1  ->  2.0.2      
          Magento_Customer     schema:      2.0.12  ->  2.0.13     
          Magento_Customer       data:      2.0.12  ->  2.0.13     
           Magento_Catalog     schema:       2.2.3  ->  2.2.4      
           Magento_Catalog       data:       2.2.3  ->  2.2.4      
             Magento_Quote     schema:       2.0.6  ->  2.0.7      
             Magento_Quote       data:       2.0.6  ->  2.0.7      
             Magento_Sales     schema:       2.0.7  ->  2.0.9      
             Magento_Sales       data:       2.0.7  ->  2.0.9      
        Magento_Newsletter     schema:       2.0.0  ->  2.0.1      
        Magento_Newsletter       data:       2.0.0  ->  2.0.1      
             Magento_Vault     schema:       2.0.2  ->  2.0.3      
             Magento_Vault       data:       2.0.2  ->  2.0.3      
      
      <release_path>/app/etc/config.php hash: 46441a938bf279b86f5cf23ca07d0244
      <current_path>/app/etc/config.php hash: 46441a938bf279b86f5cf23ca07d0244

      Maintenance mode usage will be enforced per :magento_deploy_maintenance (enabled by default)

[davidalger]

Finally the test output when db changes are not made, but the config.php file differs:

04:11 magento:maintenance:check
      Checking database status...
      All modules are up to date. No database updates should occur during this release.

      <release_path>/app/etc/config.php hash: 46441a938bf279b86f5cf23ca07d0244
      <current_path>/app/etc/config.php hash: 136ab0193d460e18c19b26a6c8ebf446

      Configuration hashes do not match. Maintenance mode will be used by default.
      Maintenance mode usage will be enforced per :magento_deploy_maintenance (enabled by default)

[dverkade]

@davidalger awesome feature! Thank you very much, this helps us a lot.