AWS ParallelCluster v3.0.3

We're excited to announce the release of AWS ParallelCluster Cookbook 3.0.3

This is associated with AWS ParallelCluster v3.0.3

CHANGES

• Disable log4j-cve-2021-44228-hotpatch service on Amazon Linux to avoid incurring in potential performance degradation.

AWS ParallelCluster v2.11.4

We're excited to announce the release of AWS ParallelCluster Cookbook 2.11.4

This is associated with AWS ParallelCluster v2.11.4

CHANGES

• CentOS 8 is no longer supported (EOL on December 31st, 2021).
• Upgrade Slurm to version 20.11.8.
• Upgrade Cinc Client to version 17.2.29.
• Upgrade NICE DCV to version 2021.2-11190.
• Upgrade NVIDIA driver to version 470.82.01.
• Upgrade CUDA library to version 11.4.3.
• Upgrade NVIDIA Fabric manager to 470.82.01.
• Disable unattended packages update on Ubuntu.
• Install Python 3 version of aws-cfn-bootstrap scripts on CentOS 7 and Ubuntu 18.04, aligning with Ubuntu 20.04 and Amazon Linux 2.

AWS ParallelCluster v3.0.2

We're excited to announce the release of AWS ParallelCluster Cookbook 3.0.2

This is associated with AWS ParallelCluster v3.0.2

3.0.2

CHANGES

• Upgrade EFA installer to version 1.14.1. Thereafter, EFA enables GDR support by default on supported instance type(s).
  ParallelCluster does not reinstall EFA during node start. Previously, EFA was reinstalled if GdrSupport had been
  turned on in the configuration file.
  • EFA configuration: efa-config-1.9-1
  • EFA profile: efa-profile-1.5-1
  • EFA kernel module: efa-1.14.2
  • RDMA core: rdma-core-37.0
  • Libfabric: libfabric-1.13.2
  • Open MPI: openmpi40-aws-4.1.1-2

BUG FIXES

• Fix issue that is preventing cluster names to start with parallelcluster- prefix.

AWS ParallelCluster v2.11.3

We're excited to announce the release of AWS ParallelCluster Cookbook 2.11.3

This is associated with AWS ParallelCluster v2.11.3

2.11.3

CHANGES

• Upgrade EFA installer to version 1.14.1. Thereafter, EFA enables GDR support by default on supported instance type(s).
  ParallelCluster does not reinstall EFA during node start. Previously, EFA was reinstalled if enable_efa_gdr had been
  turned on in the configuration file.
  • EFA configuration: efa-config-1.9-1
  • EFA profile: efa-profile-1.5-1
  • EFA kernel module: efa-1.14.2
  • RDMA core: rdma-core-37.0
  • Libfabric: libfabric-1.13.2
  • Open MPI: openmpi40-aws-4.1.1-2

BUG FIXES

• Fix failure when building AMI, due to SGE sources not available at arc.liv.ac.uk
• Fix cluster update when using proxy setup.
• Update ca-certificates package during AMI build time and prevent Chef from using outdated/distrusted CA certificates.

AWS ParallelCluster v3.0.1

We're excited to announce the release of AWS ParallelCluster Cookbook 3.0.1

This is associated with AWS ParallelCluster v3.0.1

CHANGES

• Change supervisord service script from SysVinit to Systemd.
• Drop support for SysVinit. Only Systemd is supported.

BUG FIXES

• Fix supervisord service not enabled on Ubuntu.
• Update ca-certificates package during AMI build time and prevent Chef from using outdated/distrusted CA certificates.

AWS ParallelCluster v3.0.0

We're excited to announce the release of AWS ParallelCluster Cookbook 3.0.0

This is associated with AWS ParallelCluster v3.0.0

3.0.0

ENHANCEMENTS

• Support restart/reboot for instance type with instance store (ephemeral drives).
• Compile Slurm with jobcomp/elasticsearch support.

CHANGES

• Drop support for SGE and Torque schedulers.
• Drop support for CentOS8.
• Remove nodewatcher, sqswatcher, jobwatcher related code.
• Remove Ganglia support.
• Install ParallelCluster AWS Batch CLI at AMI build time.
• Run daemons as cluster admin user (not root).
• Add explicit assignment of names, uids, gids for slurm, munge and dcvextauth users.
• Remove packer.
• Restrict access to IMDS to root and cluster admin users, only.
• Make PATH include required directories for every user and recipes context.
• Fail cluster creation when IMDS lockdown is not working correctly.
• Make sudoers secure_path include the same directories in every platform.
• Remove option for instance store software encryption (encrypted_ephemeral).
• Add support for iptables restore on instance reboot.
• Allow IMDS access for dcv user when dcv is enabled.
• Restore noatime option, which has positive impact on the performances of NFS filesystem
• Upgrade NICE DCV to version 2021.1-10851.
• Upgrade Slurm to version 20.11.8
• Upgrade Cinc Client to version 17.2.29.
• Upgrade NVIDIA driver to version 470.57.02.
• Upgrade CUDA library to version 11.4.0.
• Avoid installing MPICH and FFTW packages.
• Upgrade EFA installer to version 1.13.0
  • Update rdma-core to v35.0.
  • Update libfabric to v1.13.0amzn1.0.

BUG FIXES

• Fix cluster update when using proxy setup

AWS ParallelCluster v2.11.2

We're excited to announce the release of AWS ParallelCluster Cookbook 2.11.2

This is associated with AWS ParallelCluster v2.11.2

2.11.2

CHANGES

• When using a custom AMI with a preinstalled EFA package, no actions are taken at node bootstrap time in case GPUDirect RDMA is enabled. The original EFA package deployment is preserved as during the createami process.
• Upgrade EFA installer to version 1.13.0
  • Update rdma-core to v35.0.
  • Update libfabric to v1.13.0amzn1.0.

BUG FIXES

• Lock the version of nvidia-fabricmanager package to the installed NVIDIA drivers to prevent updates and misalignments.

AWS ParallelCluster v2.11.1

We're excited to announce the release of AWS ParallelCluster Cookbook 2.11.1

This is associated with AWS ParallelCluster v2.11.1

ENHANCEMENTS

• Retry failed installations of aws-parallelcluster package on head node of clusters using AWS Batch as the scheduler.

CHANGES

• Restore noatime option, which has positive impact on the performances of NFS filesystem.

BUG FIXES

• Pin to version 1.247347 of the CloudWatch agent due to performance impact of latest CW agent version 1.247348.
• Avoid failures when building SGE using instance type with vCPU >=32.

AWS ParallelCluster v2.11.0

We're excited to announce the release of AWS ParallelCluster Cookbook 2.11.0

This is associated with AWS ParallelCluster v2.11.0

ENHANCEMENTS

• Add support for Ubuntu 20.04.
• Add support for using FSx Lustre in subnet with no internet access.
• Add support for building custom Centos 7 AMIs on ARM.
• Make sure slurmd service is only enabled after post-install process, which will prevent user from unintentionally making compute node available during post-install process.
• Change to ssh_target_checker.sh syntax that makes the script compatible with pdsh.
• Add possibility to use a post installation script when building Centos 8 AMI.
• Install SSM agent on CentOS 7 and 8.
• Transition from IMDSv1 to IMDSv2.
• Add support for security_group_id in packer custom builders. Customers can export AWS_SECURITY_GROUP_ID environment variable to specify security group for custom builders when building custom AMIs.
• Configure the following default gc_thresh values for performance at scale.
  • net.ipv4.neigh.default.gc_thresh1 = 0
  • net.ipv4.neigh.default.gc_thresh2 = 15360
  • net.ipv4.neigh.default.gc_thresh3 = 16384

CHANGES

• Ubuntu 16.04 is no longer supported.
• Amazon Linux is no longer supported.
• Upgrade EFA installer to version 1.12.2
  • EFA configuration: efa-config-1.8-1 (from efa-config-1.7)
  • EFA profile: efa-profile-1.5-1 (from efa-profile-1.4)
  • EFA kernel module: efa-1.12.3 (from efa-1.10.2)
  • RDMA core: rdma-core-32.1amzn (from rdma-core-31.2amzn)
  • Libfabric: libfabric-1.11.2amzon1.1-1 (from libfabric-1.11.1amzn1.0)
  • Open MPI: openmpi40-aws-4.1.1-2 (from openmpi40-aws-4.1.0)
• Increase timeout when attaching EBS volumes from 3 to 5 minutes.
• Retry berkshelf installation up to 3 times.
• Root volume size increased from 25GB to 35GB on all AMIs. Minimum root volume size is now 35GB.
• Upgrade Slurm to version 20.11.7.
  • Update slurmctld and slurmd systemd unit files according to latest provided by slurm
  • Add new SlurmctldParameters, power_save_min_interval=30, so power actions will be processed every 30 seconds
  • Specify instance GPU model as GRES GPU Type in gres.conf, instead of previous hardcoded value Type=tesla for all GPU
• Upgrade Arm Performance Libraries (APL) to version 21.0.0
• Upgrade NICE DCV to version 2021.1-10557.
• Upgrade NVIDIA driver to version 460.73.01.
• Upgrade CUDA library to version 11.3.0.
• Upgrade NVIDIA Fabric manager to nvidia-fabricmanager-460.
• Install ParallelCluster AWSBatch CLI in dedicated python3 virtual env.
• Upgrade Python version used in ParallelCluster virtualenvs from version 3.6.13 to version 3.7.10.
• Upgrade Cinc Client to version 16.13.16.
• Upgrade third-party cookbook dependencies:
  • apt-7.4.0 (from apt-7.3.0)
  • iptables-8.0.0 (from iptables-7.1.0)
  • line-4.0.1 (from line-2.9.0)
  • openssh-2.9.1 (from openssh-2.8.1)
  • pyenv-3.4.2 (from pyenv-3.1.1)
  • selinux-3.1.1 (from selinux-2.1.1)
  • ulimit-1.1.1 (from ulimit-1.0.0)
  • yum-6.1.1 (from yum-5.1.0)
  • yum-epel-4.1.2 (from yum-epel-3.3.0)
• Drop lightdm package install from Ubuntu 18.04 DCV installation process.
• Update default NFS options used by Compute nodes to mount shared filesystem from head node.
  • Drop intr option, which is deprecated since kernel 2.6.25
  • Drop noatime option, which is not relevant for NFS mount

AWS ParallelCluster v2.10.4

We're excited to announce the release of AWS ParallelCluster Cookbook 2.10.4

This is associated with AWS ParallelCluster v2.10.4

CHANGES

• Upgrade Slurm to version 20.02.7