Skip to content

Releases: fermitools/jobsub_lite

1.8.1

11 Jun 16:11
36a26d0
Compare
Choose a tag to compare
  • If cigetcert fails for any reason, raise PermissionError (#572). This fixes an issue with the error message emitted by jobsub_submit when it tried to obtain a VOMS proxy. The message would sometimes mask the real underlying issue; for example if jobsub could not obtain the initial X509 proxy that is used to generate a VOMS proxy.

1.8

30 Apr 19:14
115727d
Compare
Choose a tag to compare
1.8

Features

  • Update condor_vault_storer to reflect HTCondor 23 release changes (#563)
  • Cleanup copies of weakened tokens after job submission (#565)

Bugfixes

  • Make gfal operations for web sandbox creation check for BEARER_TOKEN (#562)
  • When --global-pool flag is passed, make sure we save the BEARER_TOKEN_FILE and X509_USER_PROXY into the restored environment post-submission (#568)

Other code improvements

  • Delete test file if it's already there (for example if a previous test run didn't finish unlinking the file (#566)
  • Clear up escape warning in tests (#567)

1.7.2

02 Apr 20:06
2d2b350
Compare
Choose a tag to compare
  • Increase dagbegin.cmd and dagend.cmd memory requests to 500 Mb (#558)

1.7.1

01 Apr 16:50
Compare
Choose a tag to compare
  • Fixes a bug introduced in jobsub_lite 1.7 wrt POMS submissions (#557)

1.7

26 Mar 17:47
94aedf2
Compare
Choose a tag to compare
1.7

Features

  • Changed DEFAULT_SINGULARITY_IMAGE to /cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-el9:latest (#545)
  • Use spack in wrapper scripts (#536)

Bugfixes

  • If a disk quota check errors out, we shouldn't fail the submission. We removed the quota check implemented in 1.6, and instead improved the error handling in the case of a "no space left on device" error. (#539)
  • Make sure --global-pool flag doesn't clear the pre-existing environment (#551)

1.6

20 Feb 15:55
e8a0245
Compare
Choose a tag to compare
1.6

Features

  • Improvements to TarfilePublisherHandler (#520)
    • Use same RCDS publish server for (1) first call to see if a tarball exists, and (2) to publish the tarball if it doesn't already exist
    • After publish, if exists call fails, immediately retry, and if that fails, then wait for the retry interval
  • Check if we have enough disk space at strategic points (job definition file generation, tarball generation, etc.) (#521, #522, #534)
  • Added skip-check option called disk_space to skip disk quota checking (#523)

Bugfixes

  • Check token for experiment and role when BEARER_TOKEN_FILE is set by user (#509, #533)
  • Don't assume we have GROUP set in the environment (#511)
  • jobsub_q bugfix with totals (#517)
  • Fix cyclic import in condor.py (#519)

Other Code Improvements

  • Run pylint in pre-commit (#513)
  • Run pylint in github action (#515)
  • Build jobsub RPM in github action from master branch (#526-#530)

Full Changelog: 1.5.1...1.6

1.5.1

12 Dec 18:33
f3685ef
Compare
Choose a tag to compare

User-facing

  • Workaround scitokens library bug where empty scitokens keycache file breaks jobsub commands (#507)
  • Fixed bug where RCDS errors leave generated tarballs lying around (#502)

Internal

  • Repository Makefile now can build release candidates as well as final releases (#504)

1.5

14 Nov 16:02
5e3b72c
Compare
Choose a tag to compare
1.5

Features

  • New executable, jobsub, which currently lists all the supported jobsub_* commands (#471)
  • AL9 support (#466)
  • Any time a tarball is being uploaded to the Rapid Code Distribution Service (RCDS), we will now show the hash RCDS uses to publish the tarball in CVMFS in stdout (#460)
  • If a tarball to be uploaded to RCDS contains a file larger than 1GiB, we will make the submission fail, since RCDS would silently (to users) fail during publishing if we allowed the submission to proceed (#470)
  • Send CVMFS revision info to Elasticsearch (via ifdh log) at end of job (#489)
  • Debug output includes timestamp and submit hostname (#496)

Bugfixes

  • Using officially supported scitokens python libraries for token decoding so users' IFDH setup does not interfere with token parsing operations of jobsub_lite (#480)
  • Credentials not obtained for file upload for the pnfs dropbox case (--tar-file-name tardir/dropbox://) (#477, #479)
  • TRACEPARENT formatting in v1.4 preventing environment variables from being passed to lens (#473, #479)
  • jobsub_q totals off by one (#494)

Other Code Improvements

  • Remove -global where applicable in jobsub_cmd --> condor_cmd translation (#458)
  • Adding jobsub_cleanup_cred script (#486)
  • Fixed tokens and get_parser unit tests to work with new scitokens library (#490)

1.4.1

04 Oct 16:17
e58fd08
Compare
Choose a tag to compare

Emergency patch release that removes the osg-ca-certs RPM as a dependency for jobsub_lite. There are no code changes in this release.

1.4

22 Aug 15:18
8c6753b
Compare
Choose a tag to compare
1.4

Features

  • Make obtaining proxy optional in job submission . This feature allows for users to opt out of obtaining a proxy in their jobs submission. (#459)
    - To opt-out of getting a proxy, pass the flag --auth-methods token to jobsub_submit before the file:// URI.
    - The default will remain to send both a proxy and token with the submitted job.
    - More details can be found on the github wiki here: https://github.com/fermitools/jobsub_lite/wiki#jobsub_submit
    - Enable tracing to assist in troubleshooting. This feature allows users and admins to follow the code paths for a given command.(#131)
  • Environment variable LC_CTYPE is now cleared out in the jobsub wrapper. This can be overridden by exporting LC_CTYPE in the user environment, then submitting with jobsub_submit -e LC_CTYPE <other_args> (#450)
  • --blacklist is now --blocklist. We will retain the ability to use the previous spelling for backward-compatibility. (#449)

Bugfixes

  • Get rid of tarfile size memory limits by streaming upload (#452)
  • Changed --append-condor-requirements to save repeated invocations into a list (#462)