Merge pull request #199 from giffels/release-0.6.0

Release 0.6.0

docs/source/changelog.rst

@@ -1,23 +1,30 @@
-.. Created by changelog.py at 2021-08-05, command
+.. Created by changelog.py at 2021-08-10, command
    '/Users/giffler/.cache/pre-commit/repor6pnmwlm/py_env-python3.9/bin/changelog docs/source/changes compile --output=docs/source/changelog.rst'
    based on the format of 'https://keepachangelog.com/'
 
 #########
 CHANGELOG
 #########
 
-[Unreleased] - 2021-08-05
-=========================
+[0.6.0] - 2021-08-09
+====================
 
 Added
 -----
 
+* Added support for Kubernetes horizontal pod autoscaler
+* Enable support for `msub` command line options in the Moab site adapter
 * An optional and per site configurable drone heartbeat interval has been added
+* Added support for executors in batch system adapters
+* Added a new site adapter to use Kubernetes clusters as resource provider
+* Added TARDIS docker images to matterminers@dockerhub
 
 Fixed
 -----
 
+* Fixed pypy support of TARDIS
 * Fixes a bug that get_resource_ratios raised a ValueError
+* Fixed installation issues on Centos 7
 * Fixes a bug that the drone_minimum_lifetime parameter is not working as described in the documentation
 * Fixes a bug in the HTCondor Site Adapter which leads to wrong requirements when using non HTCondor OBS
 

docs/source/changes/161.add_moab_cmdline_options.yaml

@@ -0,0 +1,8 @@
+category: added
+summary: "Enable support for `msub` command line options in the Moab site adapter"
+description: |
+  `msub` command line option can now be added to the `MachineTypeConfiguration` of the
+  Moab site adapter. `short` and `long` option are supported via yaml MappingNodes.
+pull requests:
+- 161
+version: 0.6.0

docs/source/changes/163.add_kubernetes_site_adapter.yaml

@@ -0,0 +1,10 @@
+category: added
+summary: "Added a new site adapter to use Kubernetes clusters as resource provider"
+description: |
+  A new Kubernetes site adapter has been added to `TARDIS` using Kubernetes deployments to deploy Drone on Kubernetes
+  clusters using its API.
+issues:
+- 50
+pull requests:
+- 163
+version: 0.6.0

docs/source/changes/166.add_drone_heartbeat_interval.yaml

@@ -1,9 +1,11 @@
 category: added
-summary: "An optional and per site configurable drone heartbeat interval has been added"
+summary: "An optional and per site configurable drone heartbeat interval has been\
+  \ added"
 description: |
   Add an optional and per site configurable drone heartbeat interval to TARDIS. The heartbeat interval is defined as
   the time between two consecutive calls of the drones run method. The heartbeat interval defaults to 60s.
 issues:
 - 165
 pull requests:
 - 166
+version: 0.6.0

docs/source/changes/169.fix_drone_lifetime.yaml

@@ -1,11 +1,13 @@
 category: fixed
-summary: "Fixes a bug that the drone_minimum_lifetime parameter is not working as described in the documentation"
+summary: "Fixes a bug that the drone_minimum_lifetime parameter is not working as\
+  \ described in the documentation"
 description: |
-    The `drone_minimum_lifetime` parameter is not working as expected and described in the documentation.
-    `drone_minimum_lifetime` is meant to be a generic site parameter. However the code is trying to look it up in the
-    site adapter specific section of the configuration. Since default values are applied if the parameter is not
-    present, it remains probably unnoted by users.
-pull requests:
-  - 169
+  The `drone_minimum_lifetime` parameter is not working as expected and described in the documentation.
+  `drone_minimum_lifetime` is meant to be a generic site parameter. However the code is trying to look it up in the
+  site adapter specific section of the configuration. Since default values are applied if the parameter is not
+  present, it remains probably unnoted by users.
 issues:
-  - 167
+- 167
+pull requests:
+- 169
+version: 0.6.0

docs/source/changes/173.fix_meta_data_translation_htcondor.yaml

@@ -1,12 +1,15 @@
 category: fixed
-summary: "Fixes a bug in the HTCondor Site Adapter which leads to wrong requirements when using non HTCondor OBS"
+summary: "Fixes a bug in the HTCondor Site Adapter which leads to wrong requirements\
+  \ when using non HTCondor OBS"
 description: |
   The HTCondor Site Adapter takes a wrong `machine_meta_data_translation_mapping` into account in some circumstances.
   Due to a bug introduced in #157, the HTCondor Site Adapter uses the `machine_meta_data_translation_mapping` of the
   Batchsystem Adapter (OBS). In case the OBS is also HTCondor or the OBS has the same translations it does not have any
-  affect. However, in case the OBS is using different units for memory and disk space --hence different translation
+  effect. However, in case the OBS is using different units for memory and disk space --hence different translation
   mappings-- the requested Drones have wrong requirements.
-pull requests:
-  - 173
 issues:
-  - 170
+- 170
+- 153
+pull requests:
+- 173
+version: 0.6.0

docs/source/changes/175.fix_resource_ratios.yaml

@@ -1,10 +1,11 @@
 category: fixed
 summary: "Fixes a bug that get_resource_ratios raised a ValueError"
 description: |
-    In case one of the resource ratios is `undefined` or even has the value `error`
-    a `ValueError` or `TypeError` could occur. In case one of those errors occurs,
-    an empty list is returned.
-pull requests:
-  - 175
+  In case one of the resource ratios is `undefined` or even has the value `error`
+  a `ValueError` or `TypeError` could occur. In case one of those errors occurs,
+  an empty list is returned.
 issues:
-  - 168
+- 168
+pull requests:
+- 175
+version: 0.6.0

docs/source/changes/180.fix_pypy_support.yaml

@@ -0,0 +1,9 @@
+category: fixed
+summary: "Fixed pypy support of TARDIS"
+description: |
+  Refactoring the SQlite plugin code in order to fix the pypy support of TARDIS. Enabling unittests running on pypy.
+issues:
+- 179
+pull requests:
+- 180
+version: 0.6.0

docs/source/changes/182.add_k8s_horizontal_pod_autoscaling.yaml

@@ -0,0 +1,7 @@
+category: added
+summary: "Added support for Kubernetes horizontal pod autoscaler"
+description: |
+  Enable automated scaling of Kubernetes deployments using the horizontal pod autoscaler.
+pull requests:
+- 182
+version: 0.6.0

docs/source/changes/185.add_tardis_docker_images.yaml

@@ -0,0 +1,7 @@
+category: added
+summary: "Added TARDIS docker images to matterminers@dockerhub"
+description: |
+  TARDIS docker images have been added to simplify deployments and tests.
+pull requests:
+- 185
+version: 0.6.0

docs/source/changes/186.fix_centos7_installation.yaml

@@ -0,0 +1,9 @@
+category: fixed
+summary: "Fixed installation issues on Centos 7"
+description: |
+  Pinned version of the cryptography package to support OpenSSL < 1.1.0 and aiohttp to support Python 3.6.
+issues:
+- 184
+pull requests:
+- 186
+version: 0.6.0

docs/source/changes/198.add_support_executors_batchsystem_adapters.yaml

@@ -0,0 +1,8 @@
+category: added
+summary: "Added support for executors in batch system adapters"
+description: |
+  Allows users to specify an executor (ShellExecutor or SSHExecutor, default is the first) for the Slurm and HTCondor
+  batch system adapters.
+pull requests:
+- 198
+version: 0.6.0

docs/source/changes/versions.yaml

@@ -1,3 +1,5 @@
+- semver: 0.6.0
+  date: '2021-08-09'
 - semver: 0.5.0
   date: '2020-12-09'
 - semver: 0.4.0

setup.py

@@ -46,6 +46,7 @@ def get_cryptography_version():
         "Programming Language :: Python :: 3.6",
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",
     ],
     entry_points={
         "cobald.config.yaml_constructors": [

tardis/__about__.py

@@ -3,8 +3,8 @@
 __summary__ = "Transparent Adaptive Resource Dynamic Integration System"
 __url__ = "https://github.com/matterminers/tardis"
 
-__version__ = "0.5.0"
+__version__ = "0.6.0"
 __author__ = "Manuel Giffels, Matthias Schnepf"
 __email__ = "giffels@gmail.com"
-__copyright__ = "2018 - 2020 %s and Contributors" % __author__
+__copyright__ = "2018 - 2021 %s and Contributors" % __author__
 __keywords__ = "asyncio tardis cloud scheduler"

tardis/utilities/utils.py

@@ -71,10 +71,11 @@ def machine_meta_data_translation(
     """
     Helper function to translate units of the machine_meta_data to match the
     units required by the overlay batch system
+
     :param machine_meta_data: Machine Meta Data (Cores, Memory, Disk)
     :param meta_data_translation_mapping: Map used for the translation of meta
            data, contains conversion factors
-    :return:
+    :return: Converted meta data with units expected by the OBS
     :rtype: dict
     """
     try: