Initial steps to modernizing task_schedule

[taldcroft]

Cuts the cruft that no longer applies and makes this work on a standalone MacOS platform.

Substantive code changes are:

• Ensure that the SKA_ARCH_OS env var is always defined, even in a (non-production) conda env.
• If a non-absolute executable is specified as a command, then check if it exists in bin_dir. ONLY if it exists and is executable then prepend bin_dir to the executable path. This was previously done regardless of whether the executable was found in bin_dir. This new logic allows specifying an executable that is in the global search PATH. This is more common now with the use of entry point scripts, e.g. cheta_sync or kadi_update_cmds.

This takes a different approach to unit testing and just installs locally in a fixed directory, then overrides SKA. For these tests SKA is the root for both data and bin.

Testing

Created a testing Ska3 environment on linux and MacOS

conda create -n ska3-kady-task -c $ska3conda -c $ska3conda/core-pkg-repo ska3-flight ska3-perl
source activate ska3-kady-task
# Note that the kadi => kady machine migration has gotten into my fingers
# and so the env name is a little misleading.

conda uninstall --force ska3-flight ska3-core ska3-perl
conda uninstall --force kadi task_schedule watch_cron_logs

# Then from respective modernize branches install task_schedule and
# watch_cron_logs with `make install`.
# Install kadi `py3-cmds-task` branch (version 4.19) with pip install.
# Also make Ska $CONDA_PREFIX/data/ directory with kadi and cmd_states dirs.
# kadi is new but cmd_states is a link.

task_schedule self-test

Then from the task_schedule git repo, each of the following gave expected results

make test_basic
make test_exception
make test_basic_full
make test_fail
make test_send_mail  # Does not send mail on MacOS
make test_send_mail_quiet  # Does not send mail on MacOS

Note that make test had problems because it seemed to start the exception test before the basic test was done since this forks off processes.

Test with kadi 4.19 (py3-cmds-task branch) on linux and MacOS

skare task_schedule3.pl -config kadi/task_schedule_cmds.cfg

Expected results:

• Saw process running (the cfg file was locally modified to have loud=1)
• Saw expected output in the log file in ${CONDA_PREFIX}/data/kadi/logs/kadi_cmds.log

[jeanconn]

If there are @cmd_args, where do they go? I don't see that handled.

[taldcroft]

Yeah... cripes. I just came to the same conclusion the hard way by testing and looking at logs in debug mode.

[jeanconn]

Yeah, sorry I missed that in my review. Though I note it seems to be working fine for arc which I thought was a stressing case (though maybe it doesn't use this bin_dir section).

[taldcroft]

Clearly the test coverage is inadequate here to have caught this. And of course in Python my editor would have complained that the new variable is never used.

[taldcroft]

arc3 uses a better idiom, calling out the path instead of magically finding it in share.

      exec 10 : $ENV{SKA_SHARE}/arc3/get_iFOT_events.pl
      exec  5 : $ENV{SKA_SHARE}/arc3/get_web_content.pl

Zen of Python no. 2: Explicit is better than implicit!

[jeanconn]

So should this just be

$_->{cmd} = $bin_dir_cmd + join(" ", @cmd_args);

Or does that need different error checking?

[jeanconn]

Should be

$_->{cmd} = $bin_dir_cmd . " " . join(" ", @cmd_args);