Updates for osx compat

[jeanconn]

Updates for osx compat

Closes #263

[jeanconn]

I'd have liked to toss in the use of maude to make this more osx-mobile-ready, but the ACA model code is set to fetch 30 days of AACCCDPT, so for operational use I don't think we want to swap that over to use the full resolution data from the 5min stat it is doing, and it looks like we don't have stat support in fetch just yet.

[taldcroft]

AACCCDPT is sampled at 32.8 sec, so a MAUDE or CXC fetch of 30 days full resolution is no problem. About 80000 samples, which I checked is done in a couple of seconds. So go for it!

[jeanconn]

A MAUDE fetch of 30 days full resolution doesn't immediately seem to be option either:

Exception: MAUDE query failed: cannot query more than 7 days with allow_subset=False at line 28

[taldcroft]

At some level I'd be happier with a more minimal change that just addresses OSX-specific problems, instead of also bundling "don't have an eng-archive" problems. I don't see a problem requiring that at least AACCCDPT is available to run starcheck. (Cutting pitch is fine). Since maude will likely never have some of the ACIS MSIDs, a general model of OSX support probably needs to include a requirement to have the right eng archive data available. That does not seem problematic to me.

[jeanconn]

OK. I read your "So go for it!" feedback as sort-of direction to go this way, but can back it out.

I also didn't see the starcheck code as a real general example? and thought there might be some benefit to the maude strategy as being both more up-to-date and a great fallback plan.

[taldcroft]

What do I need to do for testing? I did:

• Copy JUN1019 directory to a local dir
• Update my ska_sync to include eng_archive files per Update starcheck and acis regress outputs ska_testr#15 (comment)
• conda install -c $ska3conda ska3-perl

This gave me:

(ska3) monkeys$ ./sandbox_starcheck -dir ~/ska/data/mpcrit1/mplogs/2019/JUN1019 -out test
Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fff9841b380 (most recent call first):
Failed to autogenerate /private/var/folders/k1/2ktyqf0x2n7_fzvdnvljq2240000gq/T/starcheck_inline.XXXXXX.wUsWJXGx/config-darwin-2level-5.026002.

 at starcheck/src/lib/Ska/Starcheck/Obsid.pm line 162.
BEGIN failed--compilation aborted at starcheck/src/lib/Ska/Starcheck/Obsid.pm line 162.
Compilation failed in require at ./starcheck/src/starcheck.pl line 33.
BEGIN failed--compilation aborted at ./starcheck/src/starcheck.pl line 33.

[jeanconn]

Did you source into ska?

…

On Mon, Jun 10, 2019, 1:48 PM Tom Aldcroft ***@***.***> wrote: What do I need to do for testing? I did: - Copy JUN1019 directory to a local dir - Update my ska_sync to include eng_archive files per sot/ska_testr#15 (comment) <sot/ska_testr#15 (comment)> - conda install -c $ska3conda ska3-perl This gave me: (ska3) monkeys$ ./sandbox_starcheck -dir ~/ska/data/mpcrit1/mplogs/2019/JUN1019 -out test Fatal Python error: Py_Initialize: unable to load the file system codec ModuleNotFoundError: No module named 'encodings' Current thread 0x00007fff9841b380 (most recent call first): Failed to autogenerate /private/var/folders/k1/2ktyqf0x2n7_fzvdnvljq2240000gq/T/starcheck_inline.XXXXXX.wUsWJXGx/config-darwin-2level-5.026002. at starcheck/src/lib/Ska/Starcheck/Obsid.pm line 162. BEGIN failed--compilation aborted at starcheck/src/lib/Ska/Starcheck/Obsid.pm line 162. Compilation failed in require at ./starcheck/src/starcheck.pl line 33. BEGIN failed--compilation aborted at ./starcheck/src/starcheck.pl line 33. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#302?email_source=notifications&email_token=AAGBHVAA7JQJJ6RG4AMHAFTPZ2HWFA5CNFSM4G7M7UDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXKTORA#issuecomment-500512580>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGBHVBPPELLQXSIKWGEXMTPZ2HWFANCNFSM4G7M7UDA> .

[taldcroft]

OK, success, wow!!

I encountered some difficulties because sourcing ska_envs redefines $SKA to a new location which is not my SKA. So I made a data soft link but then made the mistake of running ska_sync with the new SKA, which blew away the soft link and made a second fresh data repo. So I would say this is a little fiddly right now for standalone Ska, but it does work!

[taldcroft]

It works for me, meaning it ran to completion on JUN1019A and gave reasonable looking results. Did not do any regression test comparison, however.

[jeanconn]

Maybe open a skare3 issue on the problem you had with ska_envs redefining SKA? I thought I had figured out the non-root environment scenarios but maybe missed something.

Starcheck might, in fact, work with just SKA set (without setting LD_LIBRARY_PATH and such from ska_envs.sh) but that wasn't how I was testing.

With regard to regression, I can't see how the changes that ended up in this PR could introduce regressions in flight starcheck on linux. Do you also want to certify osx for load review? I wasn't sure what we needed to test or if "it looks like it works" might be just fine for now.

[taldcroft]

If I use export PYTHONHOME=$CONDA_PREFIX instead of $SKA then both starcheck/src/starcheck and sandbox_starcheck work in a standalone env.

[taldcroft]

About regressions, I was not implying that the linux output could change, I was wondering if the OSX output matches the linux output. That is likely the case, but does need checking for at least one load (by diffing the starcheck.txt output).

I don't foresee certifying use on OSX for flight load review right away, but I will certainly use starcheck on my mac and it would be a problem if it were giving results inconsistent with linux.

[jeanconn]

OK, so was the regression comment an aside, do you want to see regression validation for osx in this PR, or should it be a new issue?

[jeanconn]

I think the real problem is that I replaced SKA_ARCH_OS with SKA in these scripts, but that we've decided that SKA only defines the path of data. For interaction with the Perl I think I do need something that always points to the Python installation (equivalent to old SKA_ROOT?), and that's a little counter to:

sot/skare3#22 (comment)

I think CONDA_PREFIX only works in an env, so would you want to do something like try it first? My thought was that we just want ska_envs.sh to try to do the right thing if possible and perhaps set SKA and SKA_ROOT to reasonable things in the env.

[taldcroft]

For the record, I confirmed that the report .txt file on my Mac for JUN1019A is exactly the same as the output from linux except for the header info up top (including the sybase error on OSX). In both cases I ran ./starcheck_sandbox from the git repo.