Merge branch 'release/1.5.0'

.gitignore

@@ -15,3 +15,4 @@ TAGS
 .envrc
 TAGS
 .cache
+.pytest_cache

.travis.yml

@@ -5,14 +5,19 @@ python:
   - "3.4"
   - "3.5"
   - "3.6"
+
+matrix:
+  include:
+    - python: "3.6"
+      env: TRAVIS_PYLINT=true
+
 # command to run tests, e.g. python setup.py test
 install: 
   - pip install virtualenv
   - pip install codecov
 script: 
   - make env
-  - make test
-  - if [[ $TRAVIS_PYTHON_VERSION == '3.6' ]]; then make pylint; fi
+  - if [[ $TRAVIS_PYLINT != 'true' ]]; then make test; else make pylint; fi
 after_success:
   - codecov
 after_failure:

Makefile

@@ -13,7 +13,7 @@ env: .env/.up-to-date
 
 .env/.up-to-date: setup.py Makefile
 	python -m virtualenv .env
-	.env/bin/pip install -e .[testing] -r doc/pip_requirements.txt
+	.env/bin/pip install -e .[testing,doc]
 	touch .env/.up-to-date
 
 doc: env

doc/api.rst

@@ -75,6 +75,7 @@ Warnings
   :members:
   :special-members: 
 
+.. autofunction:: slash.ignore_warnings
 
 Hooks
 -----

doc/changelog.rst

@@ -1,6 +1,40 @@
 Changelog
 =========
 
+* :release:`1.5.0 <7-3-2018>`
+* :feature:`697` Added ``slash.before_interactive_shell`` hook
+* :feature:`-` Added a configuration option preventing ``slash.g`` from being available in interactive namespaces
+* :feature:`664` Added ``metadata.set_file_path``, allowing integrations to set a custom file path to be associated with a loaded test
+* :feature:`752` Added ``slash.ignore_warnings`` to filter unwanted warnings during sessions
+* :feature:`757` ``slash list tests`` now accepts the ``--warnings-as-errors`` flag, making it treat warnings it encounters as errors
+* :feature:`755` ``timestamp`` can now be used when formatting log path names
+* :feature:`747` session.results.global_result.is_success() now returns False if any test in the session isn't successful
+* :feature:`-` Add ``slash rerun`` - given a session_id, run all the tests of this session
+* :feature:`740` session.results.current is now a complete synonym for slash.context.result
+* :feature:`702` Rename log.traceback_level to log.console_traceback_level
+* :feature:`681` Added a new hook, ``log_file_closed``, and added configuration ``log.cleanup`` to enable removing log files after they are closed
+* :feature:`719` Added log.core_log_level, allowing limiting the verbosity of logs initiated from Slash itself
+* :feature:`-` ``-X`` can now be used to turn off stop-on-error behavior. Useful if you have it on by default through a configuration file
+* :feature:`711` Logs can now optionally be compressed on-the-fly through the ``log.compression.enabled`` configuration parameter
+* :feature:`723` Add configuration for resume state path location
+* :bug:`721 major` Add timeout to sending emails through SMTP
+* :feature:`-` Support fixture keyword arguments for ``generator_fixture``
+* :feature:`712` Added ``--pdb-filter`` - a new command-line flag that allows the user to enter pdb only on specific caught exceptions, based on pattern matching (similar to ``-k``)
+* :bug:`714 major` Session cleanups now happen under the global result object
+* :bug:`669 major` Session-scoped fixtures now properly register cleanups on session scope as expected
+* :bug:`710 major` Fix sorting when repeat-all option is use
+* :feature:`698` By setting ``log.traceback_variables`` to ``True``, traceback variable values will now be written to the debug log upon failures/errors
+* :feature:`704` Error objects now have their respective ``exc_info`` attribute containing the exception info for the current info (if available). This deprecates the use of the ``locals``/``globals`` attributes on traceback frames.
+* :feature:`-` During the execution of ``error_added`` hooks, traceback frame objects now have ``python_frame``, containing the original Pythonic frame that yielded them. Those are cleared soon after the hook is called.
+* :feature:`-` Suite files can now have a ``repeat: X`` marker to make the test run multiple times (Thanks @pierreluctg!)
+* :bug:`671 major` Help for ``slash resume`` is now more helpful
+* :feature:`685` use.X is now a shortcut for use('x') for fixture annotations
+* :feature:`692` Enhance errors summary log to session highlights log (configuration changed: ``log.errors_subpath`` -> ``log.highlights_subpath``)
+* :feature:`658` Deprecate ``PluginInterface.get_config()`` and rename it to ``PluginInterface.get_default_config()``
+* :bug:`- major` Fix tests loading order for some FS types
+* :feature:`689` Added a new hook, ``interruption_added``, for registering exceptions which cause test/session interruptions
+* :feature:`686` ``assert_raises`` raises ``ExpectedExceptionNotCaught`` if exception wasn't caught also allowing inspection of the expected exception object
+* :bug:`684 major` Optimize test loading with ``--repeat-each`` and ``--repeat-all``
 * :bug:`679 major` Fix coloring console for non TTY stdout
 * :feature:`675` Emit native python warnings for logbook warning level
 * :feature:`661` Support PDB notifications by notifications plugin

doc/cookbook.rst

@@ -19,6 +19,31 @@ Slash offers a hook called ``tests_loaded`` which can be used, among else, to co
 The above code is best placed in a ``slashconf.py`` file at the root of your test repository.
 
 
+Interactive Tests
+-----------------
+
+.. _cookbook-interactive-namespace:
+
+Controlling Interactive Namespaces from Plugins
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can customize the namespace available by default to interactive tests run with Slash (like ``slash run -i``) using the special hook :ref:`hooks.before_interactive_shell`:
+
+.. code-block:: python
+
+                class MyPlugin(PluginInterface):
+
+                    ...
+                    def before_interactive_shell(self, namespace):
+                        namespace['lab_name'] = 'MicrowaveLab'
+
+Now when running your session interactively you'll get::
+
+  $ slash run -i
+  In [1]: lab_name
+  Out[1]: 'MicrowaveLab'
+
+
 Logging
 -------
 
@@ -52,9 +77,9 @@ You can also configure extre session paths, for example from plugins:
     class MyPlugin(slash.plugins.PluginInterface):
 
         def get_name(self):
-            return "my_plugin"
+            return "my plugin"
 
-        def get_config(self):
+        def get_default_config(self):
             retrun {'extra_log_path': ''}
 
         def session_start(self):

doc/customizing_slash.rst

@@ -134,7 +134,7 @@ In the real world, you want to test integrated products. These are often physica
 		class ProductTestingPlugin(slash.plugins.PluginInterface):
 
 		    def get_name(self):
-		        return 'your-product'
+		        return 'your product'
 
 		    def configure_argument_parser(self, parser):
 		        parser.add_argument('-t', '--target', 
@@ -166,18 +166,17 @@ Slash supports a hierarchical configuration facility, described in :ref:`the rel
 		@slash.plugins.active
 		class ProductTestingPlugin(slash.plugins.PluginInterface):
 		    ...
-		    def activate(self):
-		        slash.config.extend({
-			    'product': {
-			        'api_timeout_seconds': 50
-			    }
-			})
+		    def get_name(self):
+		        return 'your product'
+
+		    def get_default_config(self):
+		        return {'api_timeout_seconds': 50}
 
 		    ...
 		    def session_start(self):
 		        slash.g.target = Target(
 			    self.target_address, 
-			    timeout=slash.config.root.product.api_timeout_seconds)
+			    timeout=slash.config.root.plugin_config.your_product.api_timeout_seconds)
 		    
 
 We use the :func:`slash.plugins.PluginInterface.activate` method to control what happens when our plugin is **activated**. Note that this happens very early in the execution phase - even before tests are loaded to be executed.
@@ -223,14 +222,10 @@ Below is the final code for the ``.slashrc`` file for our project:
         class ProductTestingPlugin(slash.plugins.PluginInterface):
         
             def get_name(self):
-                return 'your-product'
+                return 'your product'
         
-            def activate(self):
-                slash.config.extend({
-                    'product': {
-                        'api_timeout_seconds': 50
-                    }
-                })
+            def get_default_config(self):
+                return {'api_timeout_seconds': 50}
         
             def configure_argument_parser(self, parser):
                 parser.add_argument('-t', '--target',
@@ -241,6 +236,6 @@ Below is the final code for the ``.slashrc`` file for our project:
         
             def session_start(self):
                 slash.g.target = Target(
-                    self.target_address, timeout=slash.config.root.product.api_timeout_seconds)
+                    self.target_address, timeout=slash.config.root.plugin_config.your_product.api_timeout_seconds)
 
 

doc/development.rst

@@ -16,4 +16,5 @@ Special thanks go to these people for taking the time in improving Slash and pro
 
 * Alon Horev (@alonho)
 * Omer Gertel
+* Pierre-Luc Tessier Gagné
 

doc/errors.rst

@@ -51,15 +51,15 @@ One case that is not easily covered by the assert statement is asserting Excepti
 
 	  assert caught.exception.param == 'some_value'
 
-:func:`slash.assert_raises` will raise ``TestFailed`` exception in case the expected exception was not raised:
+:func:`slash.assert_raises` will raise ``ExpectedExceptionNotCaught`` exception in case the expected exception was not raised:
 
 .. code:: python
 
 	 >>> with slash.assert_raises(Exception) as caught: # doctest: +IGNORE_EXCEPTION_DETAIL
 	 ...    pass
 	 Traceback (most recent call last):
 	     ...
-	 TestFailed: ...
+	 ExpectedExceptionNotCaught: ...
 
 In a case where the test author wants to allow a specific exception but not to enforce its propagation (e.g. allowing a timing issue to be present), :func:`slash.allowing_exceptions` can be used.
 

doc/hook_list_doc.py

@@ -1,19 +1,18 @@
 from docutils import nodes
-from docutils.parsers.rst import directives
+from docutils.parsers.rst import directives, Directive
 from slash import hooks
 import gossip
 
-from sphinx.util.compat import Directive
 
 class HookListDoc(Directive):
     has_content = False
     required_arguments = 0
     optional_arguments = 0
     def run(self):
         returned = []
-        all_hooks = []
         for hook in sorted(gossip.get_group("slash").get_hooks(), key=lambda hook:hook.name):
-            section = nodes.section(ids=[hook.name])
+            section = nodes.section(ids=[hook.name], names=['hooks.{}'.format(hook.name)])
+            self.state.document.note_explicit_target(section)
             returned.append(section)
             title = "slash.hooks.{0}".format(hook.name)
             args = hook.get_argument_names()

doc/index.rst

@@ -38,6 +38,7 @@ Table Of Contents
    tags
    fixtures
    errors
+   warnings
    customizing_slash
    configuration
    logging

doc/logging.rst

@@ -33,19 +33,35 @@ You can use :attr:`slash.core.metadata.Metadata.test_index0` to include an ordin
 
     {context.session.id}/{context.test.__slash__.test_index0:03}-{context.test.id}.log
 
+Timestamps
+~~~~~~~~~~
+
+The current timestamp can also be used when formatting log paths. This is useful if you want to create log directories named according to the current date/time::
+
+  logs/{timestamp:%Y%m%d-%H%M%S}.log
+
 The Session Log
 ~~~~~~~~~~~~~~~
 
 Another important config path is :ref:`conf.log.session_subpath`. In this subpath, a special log file will be kept logging all records that get emitted when there's no active test found. This can happen between tests or on session start/end.
 
 The session log, by default, does not contain logs from tests, as they are redirected to test log files. However, setting the :ref:`conf.log.unified_session_log` to ``True`` will cause the session log to contain *all* logs from all tests.
 
-The Error Log
-~~~~~~~~~~~~~
+The Highlights Log
+~~~~~~~~~~~~~~~~~~
+
+Slash allows you to configure a separate log file to receive "highlight" logs from your sessions. This isn't necessarily related to the log level, as any log emitted can be marked as a "highlight". This is particularly useful if you have infrequent operations that you'd like to track and skim occasionally.
+
+To configure a log location for your highlight logs, set the :ref:`conf.log.highlights_subpath` configuration path. To emit a highlight log, just pass ``{'highlight': True}`` to the required log's ``extra`` dict:
+
+.. code-block:: python
+       
+   slash.logger.info("hey", extra={"highlight": True})
+
+.. tip:: The :ref:`conf.log.highlights_subpath` configuration path is treated just like other logging subpaths, and thus supports all substitutions and formatting mentioned above
 
-You can optionally control a separate log file in which only errors and failures are to be logged, through the :ref:`conf.log.errors_subpath` configuration variable. 
+.. note:: All errors emitted in a session are automatically added to the highlights log
 
-If set, this variable will hold the subpath (with optional formatting as described above) for a file which will contain only added errors throughout the tests and/or session. This is useful to quickly sift through your runs to only spot the errors, without having to skim through overly verbose debug logs.
 
 Last Log Symlinks
 -----------------

doc/parameters.rst

@@ -83,6 +83,48 @@ You can specify dependent parameters in a way that forces them to receive relate
        def test_fruits(fruit, color):
            ... # <-- this never gets a yellow apple
 
+Labeling Parameters
+-------------------
+
+By default, parameters are being designated by their ordinal number, starting with zero. This means that the following test:
+
+.. code-block:: python
+
+     @slash.parametrize('param', [Object1(), Object2()])
+     def test_something(param):
+         ...
+
+This will generate tests named ``test_something(param=param0)`` and ``test_something(param=param1)``. This is not very useful for most cases -- as the tests should be indicative of their respective parametrization flavors.
+
+To cope with this, Slash supports *parametrization labels*. This can be done as follows:
+
+.. code-block:: python
+
+     @slash.parametrize('param', [
+       slash.param('first', Object1()),
+       slash.param('second', Object2()),
+     ])
+     def test_something(param):
+         ...
+
+The above will generate tests named ``test_something(param=first)`` and ``test_something(param=second)``, which, given descriptive labels, should differentiate the cases more clearly.
+
+The labeling mechanism has a second possible syntactic shortcut, for developers preferring the value to appear first:
+
+.. code-block:: python
+
+     @slash.parametrize('param', [
+       Object1() // slash.param('first'),
+       Object2() // slash.param('second'),
+     ])
+     def test_something(param):
+         ...
+
+The two forms are functionally equivalent.
+
+.. note:: Label names are limited to 30 characters, and are under the same naming constraints as Python variables. This is intentional, and is intended to avoid abuse and keep labels concise. 
+
+
 Excluding Parameter Values
 --------------------------
 

doc/pip_requirements.txt

@@ -1,5 +1 @@
-# These are needed to build the documentation
--e .
-releases
-alabaster
-Sphinx>1.4
+-e .[doc]

doc/plugins.rst

@@ -119,13 +119,13 @@ In many cases you would like to receive options from the command line. Plugins c
 Plugin Configuration
 --------------------
 
-Plugins can override the :func:`config <slash.plugins.PluginInterface.get_config>` method to provide configuration to be placed under ``plugin_config.<plugin name>``:
+Plugins can override the :func:`config <slash.plugins.PluginInterface.get_default_config>` method to provide configuration to be placed under ``plugin_config.<plugin name>``:
 
 .. code-block:: python
 
  class LogCollectionPlugin(PluginInterface):
 
-     def get_config(self):
+     def get_default_config(self):
          return {
              'log_destination': '/some/default/path'
          }