diff --git a/docs/glossary.rst b/docs/glossary.rst index 7e16b569e8..16a719e102 100644 --- a/docs/glossary.rst +++ b/docs/glossary.rst @@ -1206,3 +1206,6 @@ Glossary context manager A context manager is an object that defines the runtime context to be established when executing a :ref:`with ` statement in Python. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code. Context managers are normally invoked using the ``with`` statement, but can also be used by directly invoking their methods. Pyramid adds context managers for :class:`pyramid.config.Configurator`, :meth:`pyramid.interfaces.IRouter.request_context`, :func:`pyramid.paster.bootstrap`, :func:`pyramid.scripting.prepare`, and :func:`pyramid.testing.testConfig`. See also the Python documentation for :ref:`With Statement Context Managers ` and :pep:`343`. + + Alembic + `Alembic `_ is a lightweight database migration tool for usage with the SQLAlchemy Database Toolkit for Python. diff --git a/docs/tutorials/wiki2/authentication.rst b/docs/tutorials/wiki2/authentication.rst index 85977d1be2..3f2fcec83f 100644 --- a/docs/tutorials/wiki2/authentication.rst +++ b/docs/tutorials/wiki2/authentication.rst @@ -122,15 +122,15 @@ Remember our goals: * Only allow ``editor`` users and the page creator (possibly a ``basic`` user) to edit pages. -Open the file ``tutorial/views/default.py`` and fix the following imports: +Open the file ``tutorial/views/default.py`` and fix the following import: .. literalinclude:: src/authentication/tutorial/views/default.py - :lines: 5-13 + :lines: 5-9 :lineno-match: - :emphasize-lines: 2,9 + :emphasize-lines: 2 :language: python -Change the two highlighted lines. +Change the highlighted line. In the same file, now edit the ``edit_page`` view function: diff --git a/docs/tutorials/wiki2/definingmodels.rst b/docs/tutorials/wiki2/definingmodels.rst index fd1967d99d..c22fd2b617 100644 --- a/docs/tutorials/wiki2/definingmodels.rst +++ b/docs/tutorials/wiki2/definingmodels.rst @@ -33,7 +33,7 @@ Open ``tutorial/setup.py`` and edit it to look like the following: .. literalinclude:: src/models/setup.py :linenos: - :emphasize-lines: 12 + :emphasize-lines: 13 :language: python Only the highlighted line needs to be added. @@ -70,7 +70,7 @@ like the following. .. code-block:: text - Successfully installed bcrypt-3.1.2 cffi-1.9.1 pycparser-2.17 tutorial + Successfully installed bcrypt-3.1.4 cffi-1.11.5 pycparser-2.18 tutorial Remove ``mymodel.py`` @@ -158,56 +158,76 @@ the following: Here we align our imports with the names of the models, ``Page`` and ``User``. -Edit ``scripts/initializedb.py`` -================================ +.. _wiki2_migrate_database_alembic: -We haven't looked at the details of this file yet, but within the ``scripts`` -directory of your ``tutorial`` package is a file named ``initializedb.py``. -Code in this file is executed whenever we run the ``initialize_tutorial_db`` -command, as we did in the installation step of this tutorial. +Migrate the database with Alembic +================================= -.. note:: +Now that we have written our models, we need to modify the database schema to reflect the changes to our code. Let's generate a new revision, then upgrade the database to the latest revision (head). - The command is named ``initialize_tutorial_db`` because of the mapping defined in the ``[console_scripts]`` entry point of our project's ``setup.py`` file. +On UNIX: -Since we've changed our model, we need to make changes to our -``initializedb.py`` script. In particular, we'll replace our import of -``MyModel`` with those of ``User`` and ``Page``. We'll also change the very end -of the script to create two ``User`` objects (``basic`` and ``editor``) as well -as a ``Page``, rather than a ``MyModel``, and add them to our ``dbsession``. +.. code-block:: bash -Open ``tutorial/scripts/initializedb.py`` and edit it to look like the -following: + $ $VENV/bin/alembic -c development.ini revision --autogenerate \ + -m "use new models Page and User" + $ $VENV/bin/alembic -c development.ini upgrade head -.. literalinclude:: src/models/tutorial/scripts/initializedb.py - :linenos: - :language: python - :emphasize-lines: 18,44-57 +On Windows: -Only the highlighted lines need to be changed. +.. code-block:: doscon + c:\tutorial> %VENV%\Scripts\alembic -c development.ini revision \ + --autogenerate -m "use new models Page and User" + c:\tutorial> %VENV%\Scripts\alembic -c development.ini upgrade head -Installing the project and re-initializing the database -======================================================= +Success executing these commands will generate output similar to the following. -Because our model has changed, and in order to reinitialize the database, we -need to rerun the ``initialize_tutorial_db`` command to pick up the changes -we've made to both the models.py file and to the initializedb.py file. See -:ref:`initialize_db_wiki2` for instructions. +.. code-block:: text -Success will look something like this: + 2018-06-29 01:28:42,407 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 + 2018-06-29 01:28:42,407 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-29 01:28:42,408 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 + 2018-06-29 01:28:42,408 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-29 01:28:42,409 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA table_info("alembic_version") + 2018-06-29 01:28:42,409 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,410 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT alembic_version.version_num + FROM alembic_version + 2018-06-29 01:28:42,410 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,411 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT name FROM sqlite_master WHERE type='table' ORDER BY name + 2018-06-29 01:28:42,412 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,413 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA table_info("models") + 2018-06-29 01:28:42,413 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,414 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT sql FROM (SELECT * FROM sqlite_master UNION ALL SELECT * FROM sqlite_temp_master) WHERE name = 'models' AND type = 'table' + 2018-06-29 01:28:42,414 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,414 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA foreign_key_list("models") + 2018-06-29 01:28:42,414 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,414 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT sql FROM (SELECT * FROM sqlite_master UNION ALL SELECT * FROM sqlite_temp_master) WHERE name = 'models' AND type = 'table' + 2018-06-29 01:28:42,415 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,416 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA index_list("models") + 2018-06-29 01:28:42,416 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,416 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA index_info("my_index") + 2018-06-29 01:28:42,416 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,417 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA index_list("models") + 2018-06-29 01:28:42,417 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,417 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA index_info("my_index") + 2018-06-29 01:28:42,417 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:28:42,417 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT sql FROM (SELECT * FROM sqlite_master UNION ALL SELECT * FROM sqlite_temp_master) WHERE name = 'models' AND type = 'table' + 2018-06-29 01:28:42,417 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + Generating //tutorial/tutorial/alembic/versions/20180629_23e9f8eb6c28.py ... done -.. code-block:: bash +.. code-block:: text - 2016-12-20 02:51:11,195 INFO [sqlalchemy.engine.base.Engine:1235][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 - 2016-12-20 02:51:11,195 INFO [sqlalchemy.engine.base.Engine:1236][MainThread] () - 2016-12-20 02:51:11,195 INFO [sqlalchemy.engine.base.Engine:1235][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 - 2016-12-20 02:51:11,195 INFO [sqlalchemy.engine.base.Engine:1236][MainThread] () - 2016-12-20 02:51:11,196 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] PRAGMA table_info("pages") - 2016-12-20 02:51:11,196 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] () - 2016-12-20 02:51:11,196 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] PRAGMA table_info("users") - 2016-12-20 02:51:11,197 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] () - 2016-12-20 02:51:11,197 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] + 2018-06-29 01:29:37,957 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 + 2018-06-29 01:29:37,958 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-29 01:29:37,958 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 + 2018-06-29 01:29:37,958 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-29 01:29:37,960 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA table_info("alembic_version") + 2018-06-29 01:29:37,960 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:29:37,960 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT alembic_version.version_num + FROM alembic_version + 2018-06-29 01:29:37,960 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:29:37,963 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] CREATE TABLE users ( id INTEGER NOT NULL, name TEXT NOT NULL, @@ -218,30 +238,106 @@ Success will look something like this: ) - 2016-12-20 02:51:11,197 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] () - 2016-12-20 02:51:11,198 INFO [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT - 2016-12-20 02:51:11,199 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] + 2018-06-29 01:29:37,963 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:29:37,966 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + 2018-06-29 01:29:37,968 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] CREATE TABLE pages ( id INTEGER NOT NULL, name TEXT NOT NULL, data TEXT NOT NULL, creator_id INTEGER NOT NULL, CONSTRAINT pk_pages PRIMARY KEY (id), - CONSTRAINT uq_pages_name UNIQUE (name), - CONSTRAINT fk_pages_creator_id_users FOREIGN KEY(creator_id) REFERENCES users (id) + CONSTRAINT fk_pages_creator_id_users FOREIGN KEY(creator_id) REFERENCES users (id), + CONSTRAINT uq_pages_name UNIQUE (name) ) - 2016-12-20 02:51:11,199 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] () - 2016-12-20 02:51:11,200 INFO [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT - 2016-12-20 02:51:11,755 INFO [sqlalchemy.engine.base.Engine:679][MainThread] BEGIN (implicit) - 2016-12-20 02:51:11,755 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] INSERT INTO users (name, role, password_hash) VALUES (?, ?, ?) - 2016-12-20 02:51:11,755 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] ('editor', 'editor', '$2b$12$ds7h2Zb7.l6TEFup5h8f4ekA9GRfEpE1yQGDRvT9PConw73kKuupG') - 2016-12-20 02:51:11,756 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] INSERT INTO users (name, role, password_hash) VALUES (?, ?, ?) - 2016-12-20 02:51:11,756 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] ('basic', 'basic', '$2b$12$KgruXP5Vv7rikr6dGB3TF.flGXYpiE0Li9K583EVomjY.SYmQOsyi') - 2016-12-20 02:51:11,757 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] INSERT INTO pages (name, data, creator_id) VALUES (?, ?, ?) - 2016-12-20 02:51:11,757 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] ('FrontPage', 'This is the front page', 1) - 2016-12-20 02:51:11,757 INFO [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT + 2018-06-29 01:29:37,968 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:29:37,969 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + 2018-06-29 01:29:37,969 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] + DROP INDEX my_index + 2018-06-29 01:29:37,969 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:29:37,970 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + 2018-06-29 01:29:37,970 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] + DROP TABLE models + 2018-06-29 01:29:37,970 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:29:37,971 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + 2018-06-29 01:29:37,972 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] UPDATE alembic_version SET version_num='23e9f8eb6c28' WHERE alembic_version.version_num = 'b6b22ae3e628' + 2018-06-29 01:29:37,972 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-29 01:29:37,972 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + + +.. _wiki2_alembic_overview: + +Alembic overview +---------------- + +Let's briefly discuss our configuration for Alembic. + +In the alchemy cookiecutter's ``development.ini`` file, the setting for ``script_location`` configures Alembic to look for the migration script in the directory ``tutorial/alembic``. +By default Alembic stores the migration files one level deeper in ``tutorial/alembic/versions``. +These files are generated by Alembic, then executed when we run upgrade or downgrade migrations. +The setting ``file_template`` provides the format for each migration's file name. +We've configured the ``file_template`` setting to make it somewhat easy to find migrations by file name. + +At this point in this tutorial, we have two migration files. +Examine them to see what Alembic will do when you upgrade or downgrade the database to a specific revision. +Notice the revision identifiers and how they relate to one another in a chained sequence. + +.. seealso:: For further information, see the `Alembic documentation `_. + + +Edit ``scripts/initialize_db.py`` +================================= + +We haven't looked at the details of this file yet, but within the ``scripts`` +directory of your ``tutorial`` package is a file named ``initialize_db.py``. +Code in this file is executed whenever we run the ``initialize_tutorial_db`` +command, as we did in the installation step of this tutorial. + +.. note:: + + The command is named ``initialize_tutorial_db`` because of the mapping defined in the ``[console_scripts]`` entry point of our project's ``setup.py`` file. + +Since we've changed our model, we need to make changes to our +``initialize_db.py`` script. In particular, we'll replace our import of +``MyModel`` with those of ``User`` and ``Page``. We'll also change the the script to create two ``User`` objects (``basic`` and ``editor``) as well +as a ``Page``, rather than a ``MyModel``, and add them to our ``dbsession``. + +Open ``tutorial/scripts/initialize_db.py`` and edit it to look like the +following: + +.. literalinclude:: src/models/tutorial/scripts/initialize_db.py + :linenos: + :language: python + :emphasize-lines: 11-24 + +Only the highlighted lines need to be changed. + + +Populating the database +======================= + +Because our model has changed, and to repopulate the database, we +need to rerun the ``initialize_tutorial_db`` command to pick up the changes +we've made to the initialize_db.py file. See :ref:`initialize_db_wiki2` for instructions. + +Success will look something like this: + +.. code-block:: text + + 2018-06-29 01:30:39,326 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 + 2018-06-29 01:30:39,326 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-29 01:30:39,327 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 + 2018-06-29 01:30:39,327 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-29 01:30:39,328 INFO [sqlalchemy.engine.base.Engine:682][MainThread] BEGIN (implicit) + 2018-06-29 01:30:39,329 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] INSERT INTO users (name, role, password_hash) VALUES (?, ?, ?) + 2018-06-29 01:30:39,329 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] ('editor', 'editor', '$2b$12$PlaJSN7goVbyx8OFs8yAju9n5gHGdI6PZ2QRJGM2jDCiEU4ItUNxy') + 2018-06-29 01:30:39,330 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] INSERT INTO users (name, role, password_hash) VALUES (?, ?, ?) + 2018-06-29 01:30:39,330 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] ('basic', 'basic', '$2b$12$MvXdM8jlkbjEyPZ6uXzRg.yatZZK8jCwfPaM7kFkmVJiJjRoCCvmW') + 2018-06-29 01:30:39,331 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] INSERT INTO pages (name, data, creator_id) VALUES (?, ?, ?) + 2018-06-29 01:30:39,331 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] ('FrontPage', 'This is the front page', 1) + 2018-06-29 01:30:39,332 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT View the application in a browser @@ -250,11 +346,11 @@ View the application in a browser We can't. At this point, our system is in a "non-runnable" state; we'll need to change view-related files in the next chapter to be able to start the application successfully. If you try to start the application (see -:ref:`wiki2-start-the-application`), you'll wind up with a Python traceback on +:ref:`wiki2-start-the-application`) and visit http://localhost:6543, you'll wind up with a Python traceback on your console that ends with this exception: .. code-block:: text - ImportError: cannot import name MyModel + AttributeError: module 'tutorial.models' has no attribute 'MyModel' This will also happen if you attempt to run the tests. diff --git a/docs/tutorials/wiki2/definingviews.rst b/docs/tutorials/wiki2/definingviews.rst index 3c343e1553..2eb78cb004 100644 --- a/docs/tutorials/wiki2/definingviews.rst +++ b/docs/tutorials/wiki2/definingviews.rst @@ -126,7 +126,7 @@ edit it to look like the following: .. literalinclude:: src/views/tutorial/views/default.py :linenos: :language: python - :emphasize-lines: 1-9,12- + :emphasize-lines: 1-9,14- The highlighted lines need to be added or edited. diff --git a/docs/tutorials/wiki2/design.rst b/docs/tutorials/wiki2/design.rst index 515aff2766..e3b35d24af 100644 --- a/docs/tutorials/wiki2/design.rst +++ b/docs/tutorials/wiki2/design.rst @@ -19,7 +19,7 @@ Models ====== We'll be using an SQLite database to hold our wiki data, and we'll be using -:term:`SQLAlchemy` to access the data in this database. +:term:`SQLAlchemy` to access the data in this database. We will also use :term:`Alembic` for database migrations, including initialization of the SQLite database. Within the database, we will define two tables: diff --git a/docs/tutorials/wiki2/installation.rst b/docs/tutorials/wiki2/installation.rst index 5a935f7fca..9b89327cb0 100644 --- a/docs/tutorials/wiki2/installation.rst +++ b/docs/tutorials/wiki2/installation.rst @@ -198,6 +198,143 @@ Testing requirements are defined in our project's ``setup.py`` file, in the ``te :lines: 48-50 +.. _initialize_db_wiki2: + +Initialize and upgrade the database using Alembic +------------------------------------------------- + +We use :term:`Alembic` to manage our database initialization and migrations. + +Generate your first revision. + +On UNIX +^^^^^^^ + +.. code-block:: bash + + $ $VENV/bin/alembic -c development.ini revision --autogenerate -m "init" + +On Windows +^^^^^^^^^^ + +.. code-block:: doscon + + c:\tutorial> %VENV%\Scripts\alembic -c development.ini revision --autogenerate -m "init" + +The output to your console should be something like this: + +.. code-block:: text + + 2018-06-22 17:57:31,587 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 + 2018-06-22 17:57:31,587 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-22 17:57:31,588 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 + 2018-06-22 17:57:31,588 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-22 17:57:31,589 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA table_info("alembic_version") + 2018-06-22 17:57:31,589 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:31,590 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA table_info("alembic_version") + 2018-06-22 17:57:31,590 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:31,590 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] + CREATE TABLE alembic_version ( + version_num VARCHAR(32) NOT NULL, + CONSTRAINT alembic_version_pkc PRIMARY KEY (version_num) + ) + + + 2018-06-22 17:57:31,591 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:31,591 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + 2018-06-22 17:57:31,594 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT name FROM sqlite_master WHERE type='table' ORDER BY name + 2018-06-22 17:57:31,594 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + Generating //tutorial/alembic/versions/20180622_bab5a278ce04.py ... done + +Upgrade to that revision. + +On UNIX +^^^^^^^ + +.. code-block:: bash + + $ $VENV/bin/alembic -c development.ini upgrade head + +On Windows +^^^^^^^^^^ + +.. code-block:: doscon + + c:\tutorial> %VENV%\Scripts\alembic -c development.ini upgrade head + +The output to your console should be something like this: + +.. code-block:: text + + 2018-06-22 17:57:37,814 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 + 2018-06-22 17:57:37,814 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-22 17:57:37,814 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 + 2018-06-22 17:57:37,814 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-22 17:57:37,816 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA table_info("alembic_version") + 2018-06-22 17:57:37,816 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:37,817 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] SELECT alembic_version.version_num + FROM alembic_version + 2018-06-22 17:57:37,817 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:37,817 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] PRAGMA table_info("alembic_version") + 2018-06-22 17:57:37,817 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:37,819 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] + CREATE TABLE models ( + id INTEGER NOT NULL, + name TEXT, + value INTEGER, + CONSTRAINT pk_models PRIMARY KEY (id) + ) + + + 2018-06-22 17:57:37,820 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:37,822 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + 2018-06-22 17:57:37,824 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] CREATE UNIQUE INDEX my_index ON models (name) + 2018-06-22 17:57:37,824 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:37,825 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + 2018-06-22 17:57:37,825 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] INSERT INTO alembic_version (version_num) VALUES ('bab5a278ce04') + 2018-06-22 17:57:37,825 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] () + 2018-06-22 17:57:37,825 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + + +.. _load_data_wiki2: + +Load default data +----------------- + +Load default data into the database using a :term:`console script`. Type the following command, making sure you are still in the ``tutorial`` directory (the directory with a ``development.ini`` in it): + +On UNIX +^^^^^^^ + +.. code-block:: bash + + $ $VENV/bin/initialize_tutorial_db development.ini + +On Windows +^^^^^^^^^^ + +.. code-block:: doscon + + c:\tutorial> %VENV%\Scripts\initialize_tutorial_db development.ini + +The output to your console should be something like this: + +.. code-block:: bash + + 2018-06-22 17:57:46,241 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 + 2018-06-22 17:57:46,241 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-22 17:57:46,242 INFO [sqlalchemy.engine.base.Engine:1254][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 + 2018-06-22 17:57:46,242 INFO [sqlalchemy.engine.base.Engine:1255][MainThread] () + 2018-06-22 17:57:46,243 INFO [sqlalchemy.engine.base.Engine:682][MainThread] BEGIN (implicit) + 2018-06-22 17:57:46,244 INFO [sqlalchemy.engine.base.Engine:1151][MainThread] INSERT INTO models (name, value) VALUES (?, ?) + 2018-06-22 17:57:46,245 INFO [sqlalchemy.engine.base.Engine:1154][MainThread] ('one', 1) + 2018-06-22 17:57:46,246 INFO [sqlalchemy.engine.base.Engine:722][MainThread] COMMIT + +Success! You should now have a ``tutorial.sqlite`` file in your current +working directory. This is an SQLite database with a single table defined in it +(``models``) and single record inside of that. + + .. _sql_running_tests: Run the tests @@ -260,27 +397,28 @@ If successful, you will see output something like this: .. code-block:: bash ======================== test session starts ======================== - platform Python 3.6.0, pytest-3.0.5, py-1.4.31, pluggy-0.4.0 - rootdir: /Users/stevepiercy/tutorial, inifile: - plugins: cov-2.4.0 + platform Python 3.6.5, pytest-3.6.2, py-1.5.3, pluggy-0.6.0 + rootdir: //tutorial, inifile: pytest.ini + plugins: cov-2.5.1 collected 2 items tutorial/tests.py .. - ------------------ coverage: platform Python 3.6.0 ------------------ - Name Stmts Miss Cover Missing - ---------------------------------------------------------------- - tutorial/__init__.py 8 6 25% 7-12 - tutorial/models/__init__.py 22 0 100% - tutorial/models/meta.py 5 0 100% - tutorial/models/mymodel.py 8 0 100% - tutorial/routes.py 3 2 33% 2-3 - tutorial/scripts/__init__.py 0 0 100% - tutorial/scripts/initializedb.py 26 16 38% 22-25, 29-45 - tutorial/views/__init__.py 0 0 100% - tutorial/views/default.py 12 0 100% - tutorial/views/notfound.py 4 2 50% 6-7 - ---------------------------------------------------------------- - TOTAL 88 26 70% + ------------------ coverage: platform Python 3.6.5 ------------------ + Name Stmts Miss Cover Missing + ----------------------------------------------------------------- + tutorial/__init__.py 8 6 25% 7-12 + tutorial/models/__init__.py 24 0 100% + tutorial/models/meta.py 5 0 100% + tutorial/models/mymodel.py 8 0 100% + tutorial/routes.py 3 3 0% 1-3 + tutorial/scripts/__init__.py 0 0 100% + tutorial/scripts/initialize_db.py 24 24 0% 1-34 + tutorial/views/__init__.py 0 0 100% + tutorial/views/default.py 12 0 100% + tutorial/views/notfound.py 4 4 0% 1-7 + ----------------------------------------------------------------- + TOTAL 88 37 58% + ===================== 2 passed in 0.57 seconds ====================== Our package doesn't quite have 100% test coverage. @@ -319,71 +457,6 @@ coverage. ``py.test -h`` to see its full set of options. -.. _initialize_db_wiki2: - -Initializing the database -------------------------- - -We need to use the ``initialize_tutorial_db`` :term:`console script` to -initialize our database. - -.. note:: - - The ``initialize_tutorial_db`` command does not perform a migration, but - rather it simply creates missing tables and adds some dummy data. If you - already have a database, you should delete it before running - ``initialize_tutorial_db`` again. - -Type the following command, making sure you are still in the ``tutorial`` -directory (the directory with a ``development.ini`` in it): - -On UNIX -^^^^^^^ - -.. code-block:: bash - - $ $VENV/bin/initialize_tutorial_db development.ini - -On Windows -^^^^^^^^^^ - -.. code-block:: doscon - - c:\tutorial> %VENV%\Scripts\initialize_tutorial_db development.ini - -The output to your console should be something like this: - -.. code-block:: bash - - 2016-12-18 21:30:08,675 INFO [sqlalchemy.engine.base.Engine:1235][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1 - 2016-12-18 21:30:08,675 INFO [sqlalchemy.engine.base.Engine:1236][MainThread] () - 2016-12-18 21:30:08,676 INFO [sqlalchemy.engine.base.Engine:1235][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1 - 2016-12-18 21:30:08,676 INFO [sqlalchemy.engine.base.Engine:1236][MainThread] () - 2016-12-18 21:30:08,676 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] PRAGMA table_info("models") - 2016-12-18 21:30:08,676 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] () - 2016-12-18 21:30:08,677 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] - CREATE TABLE models ( - id INTEGER NOT NULL, - name TEXT, - value INTEGER, - CONSTRAINT pk_models PRIMARY KEY (id) - ) - - - 2016-12-18 21:30:08,677 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] () - 2016-12-18 21:30:08,678 INFO [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT - 2016-12-18 21:30:08,679 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] CREATE UNIQUE INDEX my_index ON models (name) - 2016-12-18 21:30:08,679 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] () - 2016-12-18 21:30:08,679 INFO [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT - 2016-12-18 21:30:08,681 INFO [sqlalchemy.engine.base.Engine:679][MainThread] BEGIN (implicit) - 2016-12-18 21:30:08,682 INFO [sqlalchemy.engine.base.Engine:1140][MainThread] INSERT INTO models (name, value) VALUES (?, ?) - 2016-12-18 21:30:08,682 INFO [sqlalchemy.engine.base.Engine:1143][MainThread] ('one', 1) - 2016-12-18 21:30:08,682 INFO [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT - -Success! You should now have a ``tutorial.sqlite`` file in your current -working directory. This is an SQLite database with a single table defined in it -(``models``). - .. _wiki2-start-the-application: Start the application @@ -445,6 +518,10 @@ assumptions: - You are willing to use :term:`SQLAlchemy` for a database access tool. +- You are willing to use :term:`Alembic` for a database migrations tool. + +- You are willing to use a :term:`console script` for a data loading tool. + - You are willing to use :term:`URL dispatch` to map URLs to code. - You want to use zope.sqlalchemy_, pyramid_tm_, and the transaction_ packages diff --git a/docs/tutorials/wiki2/src/authentication/.gitignore b/docs/tutorials/wiki2/src/authentication/.gitignore new file mode 100644 index 0000000000..1853d983c7 --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/.gitignore @@ -0,0 +1,21 @@ +*.egg +*.egg-info +*.pyc +*$py.class +*~ +.coverage +coverage.xml +build/ +dist/ +.tox/ +nosetests.xml +env*/ +tmp/ +Data.fs* +*.sublime-project +*.sublime-workspace +.*.sw? +.sw? +.DS_Store +coverage +test diff --git a/docs/tutorials/wiki2/src/authentication/README.txt b/docs/tutorials/wiki2/src/authentication/README.txt index 7b33da6104..5d5133e34a 100644 --- a/docs/tutorials/wiki2/src/authentication/README.txt +++ b/docs/tutorials/wiki2/src/authentication/README.txt @@ -20,7 +20,17 @@ Getting Started env/bin/pip install -e ".[testing]" -- Configure the database. +- Initialize and upgrade the database using Alembic. + + - Generate your first revision. + + env/bin/alembic -c development.ini revision --autogenerate -m "init" + + - Upgrade to that revision. + + env/bin/alembic -c development.ini upgrade head + +- Load default data into the database using a script. env/bin/initialize_tutorial_db development.ini diff --git a/docs/tutorials/wiki2/src/authentication/development.ini b/docs/tutorials/wiki2/src/authentication/development.ini index cc2a5586e3..d76a6cd72c 100644 --- a/docs/tutorials/wiki2/src/authentication/development.ini +++ b/docs/tutorials/wiki2/src/authentication/development.ini @@ -28,6 +28,12 @@ auth.secret = seekrit # wsgi server configuration ### +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + [server:main] use = egg:waitress#main listen = localhost:6543 diff --git a/docs/tutorials/wiki2/src/authentication/production.ini b/docs/tutorials/wiki2/src/authentication/production.ini index 759807abf7..c46adb7b54 100644 --- a/docs/tutorials/wiki2/src/authentication/production.ini +++ b/docs/tutorials/wiki2/src/authentication/production.ini @@ -22,6 +22,12 @@ auth.secret = real-seekrit # wsgi server configuration ### +[alembic] +# path to migration scripts +script_location = tutorial/alembic +file_template = %%(year)d%%(month).2d%%(day).2d_%%(rev)s +# file_template = %%(rev)s_%%(slug)s + [server:main] use = egg:waitress#main listen = *:6543 diff --git a/docs/tutorials/wiki2/src/authentication/pytest.ini b/docs/tutorials/wiki2/src/authentication/pytest.ini index 8b76bc4108..a3489cdf86 100644 --- a/docs/tutorials/wiki2/src/authentication/pytest.ini +++ b/docs/tutorials/wiki2/src/authentication/pytest.ini @@ -1,3 +1,3 @@ [pytest] testpaths = tutorial -python_files = *.py +python_files = test*.py diff --git a/docs/tutorials/wiki2/src/authentication/setup.py b/docs/tutorials/wiki2/src/authentication/setup.py index abc24876de..28b766cbec 100644 --- a/docs/tutorials/wiki2/src/authentication/setup.py +++ b/docs/tutorials/wiki2/src/authentication/setup.py @@ -9,10 +9,11 @@ CHANGES = f.read() requires = [ + 'alembic', 'bcrypt', 'docutils', 'plaster_pastedeploy', - 'pyramid >= 1.9a', + 'pyramid >= 1.9', 'pyramid_debugtoolbar', 'pyramid_jinja2', 'pyramid_retry', @@ -56,7 +57,7 @@ 'main = tutorial:main', ], 'console_scripts': [ - 'initialize_tutorial_db = tutorial.scripts.initializedb:main', + 'initialize_tutorial_db = tutorial.scripts.initialize_db:main', ], }, ) diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/alembic/env.py b/docs/tutorials/wiki2/src/authentication/tutorial/alembic/env.py new file mode 100644 index 0000000000..ba116d0f35 --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tutorial/alembic/env.py @@ -0,0 +1,58 @@ +"""Pyramid bootstrap environment. """ +from alembic import context +from pyramid.paster import get_appsettings, setup_logging +from sqlalchemy import engine_from_config + +from tutorial.models.meta import Base + +config = context.config + +setup_logging(config.config_file_name) + +settings = get_appsettings(config.config_file_name) +target_metadata = Base.metadata + + +def run_migrations_offline(): + """Run migrations in 'offline' mode. + + This configures the context with just a URL + and not an Engine, though an Engine is acceptable + here as well. By skipping the Engine creation + we don't even need a DBAPI to be available. + + Calls to context.execute() here emit the given string to the + script output. + + """ + context.configure(url=settings['sqlalchemy.url']) + with context.begin_transaction(): + context.run_migrations() + + +def run_migrations_online(): + """Run migrations in 'online' mode. + + In this scenario we need to create an Engine + and associate a connection with the context. + + """ + engine = engine_from_config(settings, prefix='sqlalchemy.') + + connection = engine.connect() + context.configure( + connection=connection, + target_metadata=target_metadata + ) + + try: + with context.begin_transaction(): + context.run_migrations() + finally: + connection.close() + + +if context.is_offline_mode(): + run_migrations_offline() +else: + run_migrations_online() diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/alembic/script.py.mako b/docs/tutorials/wiki2/src/authentication/tutorial/alembic/script.py.mako new file mode 100644 index 0000000000..2c0156303a --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tutorial/alembic/script.py.mako @@ -0,0 +1,24 @@ +"""${message} + +Revision ID: ${up_revision} +Revises: ${down_revision | comma,n} +Create Date: ${create_date} + +""" +from alembic import op +import sqlalchemy as sa +${imports if imports else ""} + +# revision identifiers, used by Alembic. +revision = ${repr(up_revision)} +down_revision = ${repr(down_revision)} +branch_labels = ${repr(branch_labels)} +depends_on = ${repr(depends_on)} + + +def upgrade(): + ${upgrades if upgrades else "pass"} + + +def downgrade(): + ${downgrades if downgrades else "pass"} diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/alembic/versions/README.txt b/docs/tutorials/wiki2/src/authentication/tutorial/alembic/versions/README.txt new file mode 100644 index 0000000000..09ed32c8db --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tutorial/alembic/versions/README.txt @@ -0,0 +1 @@ +Placeholder for alembic versions \ No newline at end of file diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py b/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py index 3c9ba8e546..a4209a6e97 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py +++ b/docs/tutorials/wiki2/src/authentication/tutorial/models/__init__.py @@ -5,8 +5,8 @@ # import or define all models here to ensure they are attached to the # Base.metadata prior to any initialization routines -from .page import Page # noqa -from .user import User # noqa +from .page import Page # flake8: noqa +from .user import User # flake8: noqa # run configure_mappers after defining all of the models to ensure # all relationships can be setup diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initialize_db.py b/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initialize_db.py new file mode 100644 index 0000000000..231d5d44b5 --- /dev/null +++ b/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initialize_db.py @@ -0,0 +1,56 @@ +import os +import sys + +from pyramid.paster import bootstrap, setup_logging +from sqlalchemy.exc import OperationalError + +from .. import models + + +def setup_models(dbsession): + editor = models.User(name='editor', role='editor') + editor.set_password('editor') + dbsession.add(editor) + + basic = models.User(name='basic', role='basic') + basic.set_password('basic') + dbsession.add(basic) + + page = models.Page( + name='FrontPage', + creator=editor, + data='This is the front page', + ) + dbsession.add(page) + + +def usage(argv): + cmd = os.path.basename(argv[0]) + print('usage: %s \n' + '(example: "%s development.ini")' % (cmd, cmd)) + sys.exit(1) + + +def main(argv=sys.argv): + if len(argv) != 2: + usage(argv) + config_uri = argv[1] + setup_logging(config_uri) + env = bootstrap(config_uri) + + try: + with env['request'].tm: + dbsession = env['request'].dbsession + setup_models(dbsession) + except OperationalError: + print(''' +Pyramid is having a problem using your SQL database. The problem +might be caused by one of the following things: + +1. You may need to initialize your database tables with `alembic`. + Check your README.txt for description and try to run it. + +2. Your database server may not be running. Check that the + database server referred to by the "sqlalchemy.url" setting in + your "development.ini" file is running. + ''') diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initializedb.py b/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initializedb.py deleted file mode 100644 index f3c0a6fef4..0000000000 --- a/docs/tutorials/wiki2/src/authentication/tutorial/scripts/initializedb.py +++ /dev/null @@ -1,57 +0,0 @@ -import os -import sys -import transaction - -from pyramid.paster import ( - get_appsettings, - setup_logging, - ) - -from pyramid.scripts.common import parse_vars - -from ..models.meta import Base -from ..models import ( - get_engine, - get_session_factory, - get_tm_session, - ) -from ..models import Page, User - - -def usage(argv): - cmd = os.path.basename(argv[0]) - print('usage: %s [var=value]\n' - '(example: "%s development.ini")' % (cmd, cmd)) - sys.exit(1) - - -def main(argv=sys.argv): - if len(argv) < 2: - usage(argv) - config_uri = argv[1] - options = parse_vars(argv[2:]) - setup_logging(config_uri) - settings = get_appsettings(config_uri, options=options) - - engine = get_engine(settings) - Base.metadata.create_all(engine) - - session_factory = get_session_factory(engine) - - with transaction.manager: - dbsession = get_tm_session(session_factory, transaction.manager) - - editor = User(name='editor', role='editor') - editor.set_password('editor') - dbsession.add(editor) - - basic = User(name='basic', role='basic') - basic.set_password('basic') - dbsession.add(basic) - - page = Page( - name='FrontPage', - creator=editor, - data='This is the front page', - ) - dbsession.add(page) diff --git a/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 b/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 index 9b2dc82fc6..4016b26c9f 100644 --- a/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 +++ b/docs/tutorials/wiki2/src/authentication/tutorial/templates/layout.jinja2 @@ -16,7 +16,7 @@ - + - + - + - + - + - + - +