Move metapackages to noarch

[taldcroft]

As a follow-on to #19 and #16, plan to move these to noarch. Even if the process for building is relatively quick in time, it still roughly doubles the effort to make an update (requiring logging into a separate build machine and rsyncing and blah blah).

[jeanconn]

I'll plan to move them if it works. IIRC a dependence on an arch package precluded a noarch build (xija?), but will recheck.

[jeanconn]

OK. Looks like it is at least working for ska3-flight if I just say "noarch: python". I'll have to test on the Mac later. I think I haven't looked as much at ska3-core in a while to make sure that all of those are packages that also exist on the Mac and are at the same versions (and if that is actually an issue we might be able to use os selectors in the requirements in the meta.yaml... not sure, haven't tried that).

[jeanconn]

Also, I suppose if we are going to have the convention that we try to just do stuff on one machine, we should add something to the docs about checking to see if we're updating a noarch package.

I had assumed the convention of running the update process and then testing on both supported platforms.

[jeanconn]

Though it looks like on the mac, "conda build --skip-existing" is not working as well as it did on the linux machine. It is supposed to pick up packages on remotes to see what needs to be built, I've got freshly built noarch packages in our ska3-conda remote, and it is still trying to build those packages at those versions, but with different package sha strings. Oh well.

[taldcroft]

About there being differences in available linux vs. Mac packages, I'd be a bit surprised. But I've been surprised before, in that case it seems like we could consider downgrading to the lowest common version.

[jeanconn]

I'll have to read https://conda.io/docs/user-guide/tasks/build-packages/variants.html .

We might also use "--old-build-string" for our application.

[taldcroft]

Though it looks like on the mac, "conda build --skip-existing" is not working as well as it did on the linux machine.

Yck. I wonder if there isn't something else going on. Maybe this is another case of me being surprised, but it really should be doing the same thing on both platforms.

[taldcroft]

I'll have to read https://conda.io/docs/user-guide/tasks/build-packages/variants.html .

I just don't think it needs to be that complicated. Especially for a simple metapackage (which can actually be a regular shining example of a simple noarch package if it installs a version file).

[jeanconn]

"I don't think it needs to be that complicated" doesn't make much sense to me.

I assume before we turn off the build hashes (that were an advancement that I thought I didn't need to dig into much because they were just supposed to help...) that we should have an idea of the content that goes into making them to see if any of it was actually helpful.

The packages themselves by default include "burned in" information about how they were built... for example the content of the .condarc (with our password for our remote, by the way, so keep that in mind if you distribute packages). I don't know how much of that goes into the hash or if we can exclude some information or.... All small subtleties.

And yes, regarding noarch, I had no trouble actually running tests at one point using noarch packages built on one platform on the other platform. So I think the the current noarch packages are probably fine even if right now they appear to get rebuilt on your mac if you build all packages.

[jeanconn]

OK. I think I'm more satisfied that we can just use --old-build-string. While were were developing recipes, this [new build strings] was a good thing to have, as you'd get a new package when you fixed a recipe. Now that we have stable recipes and a better understanding of this, I think the old build strings are fine.

If a recipe needs to be modified while keeping the same package version, one will need to either delete the previous package from the available repo (which may be ska3-conda) or remove access to that repo while rebuilding, so that "--skip-existing" won't find the previous package and skip.

[taldcroft]

Agreed on --old-build-string.

[taldcroft]

Closed by #29.

[jeanconn]

OK, so I'd confirmed that we could make noarch metapackages, but not that the ones I'd made were completely working. So, what would you like to do with system differences? Do you want me to just remove these packages from ska3-core and let them get installed as dependencies as needed? Or go to arch-specific ska3-core? Or some other path?

PackageNotFoundError: Dependencies missing in current osx-64 channels:
  - ska3-flight -> ska3-core ==2018.07.16 -> dbus ==1.10.20
  - ska3-flight -> ska3-core ==2018.07.16 -> gst-plugins-base ==1.8.0
  - ska3-flight -> ska3-core ==2018.07.16 -> gstreamer ==1.8.0
  - ska3-flight -> ska3-core ==2018.07.16 -> libsodium ==1.0.10
  - ska3-flight -> ska3-core ==2018.07.16 -> libxcb ==1.12
  - ska3-flight -> ska3-core ==2018.07.16 -> mpich2 ==1.4.1p1
  - ska3-flight -> ska3-core ==2018.07.16 -> zeromq ==4.1.5

[jeanconn]

It looks like osx has libsodium 1.0.3 and zeromq 4.1.3, so we could probably just downgrade the linux side on those two packages.

[taldcroft]

Could those go into ska3-pinned?

[jeanconn]

I don't really know what the "could those go into ska3-pinned" question means at this point.

[taldcroft]

Skip it, OBE.