Upgrade CI to Ubuntu 20.04 LTS (Focal)

[DreamOfIce]

Why

On one hand Ubuntu18's free LTS cycle is ended, and on the other hand Ubuntu18 does not support nodejs greater than v18.

What's changed.

1. update the Linux Docker image to buildpack-deps:focal
2. some compatibility changes
3. use the new recommended way to install Docker (see here)

See #1173

[sbc100]

I could of comments:

Firstly, we should gets some consensus that we are ok with dropping support for bionic. @juj is often concerned about supported older OSes but I think mostly around macOS and windows.

Secondly, I suggest we update to 20.04 focal which will effect fewer users. This also happens to be the update that we recently did on the emscripten-releases bots which actually build the emsdk binaries, so I think the makes sense to do that same update there. If you are ok with that could you update this PR to use focal instead?

[sbc100]

Looks like we do document the system requirements here: https://github.com/emscripten-core/emsdk#linux , so those should be updated too.

[DreamOfIce]

I could of comments:

Firstly, we should gets some consensus that we are ok with dropping support for bionic. @juj is often concerned about supported older OSes but I think mostly around macOS and windows.

Secondly, I suggest we update to 20.04 focal which will effect fewer users. This also happens to be the update that we recently did on the emscripten-releases bots which actually build the emsdk binaries, so I think the makes sense to do that same update there. If you are ok with that could you update this PR to use focal instead?

Sounds good, I'll give it a try 😄

[DreamOfIce]

All CI tests under focal passed

[juj]

Firstly, we should gets some consensus that we are ok with dropping support for bionic. @juj is often concerned about supported older OSes but I think mostly around macOS and windows.

We do ship Emsdk on Ubuntu 18 and some other Linuxes, though we do build our own Emsdk + Emscripten bundles manually, and don't rely on the automated CI here.

So I am wholly ok with the automated emsdk CI here being upgraded to newer Linux targets, as we will still test to know that our older target is supported. (it does not look like there is any needed change to actively break support for Emsdk/Emscripten for older Linuxes)

[sbc100]

So I am wholly ok with the automated emsdk CI here being upgraded to newer Linux targets, as we will still test to know that our older target is supported. (it does not look like there is any needed change to actively break support for Emsdk/Emscripten for older Linuxes)

The problem is that if we stop testing on older version of linux then dependencies on newer glibc could creep into our binaries without us noticing. If we decide that the binaries are only tested on focal then the chances of them continuing to work on older linuxs over time will be low.

Since you are building your own binaries when you target Ubuntu 18 at least we don't need to worry about that use case, but we should reach on the mailing list to see if others are depending on these binaries running on Ubuntu 18.

We actually currently build the binaries against the debian/stretch sysroot: https://chromium.googlesource.com/emscripten-releases/+/refs/heads/main/src/build.py#65.

So the current situation is a little more confusing that I had thought:

Binaries built against: Debian/Stretch (glibc 2.24)
Binaries built on: Ubuntu/Focal 20.04 (glibc 2.31)
emsdk testing: Ubuntu/Bionic 18.03 (glibc 2.27)
emscripten testing: Ubuntu/Bionic 18.03 (glibc 2.27)
emscripten-releases testing: Ubuntu/Focal 20.04 (glibc 2.31)
Current document requirement: Ubuntu/Xenial 16.04 (glibc 2.23)
Oldest linux version by our users: ???

Actual glibc dependencies in current binaries:

$ readelf -V upstream/bin/clang 
...
Version needs section '.gnu.version_r' contains 8 entries:
 Addr: 0x00000000001111d0  Offset: 0x001111d0  Link: 7 (.dynstr)
...
  0x0180:   Name: GLIBC_2.16  Flags: none  Version: 17

GLIBC_2.16 seems to be the current highest value.

(node v18 requires glibc 2.28: nodesource/distributions#1392)

[DreamOfIce]

So I am wholly ok with the automated emsdk CI here being upgraded to newer Linux targets, as we will still test to know that our older target is supported. (it does not look like there is any needed change to actively break support for Emsdk/Emscripten for older Linuxes)

The problem is that if we stop testing on older version of linux then dependencies on newer glibc could creep into our binaries without us noticing. If we decide that the binaries are only tested on focal then the chances of them continuing to work on older linuxs over time will be low.

Since you are building your own binaries when you target Ubuntu 18 at least we don't need to worry about that use case, but we should reach on the mailing list to see if others are depending on these binaries running on Ubuntu 18.

We actually currently build the binaries against the debian/stretch sysroot: https://chromium.googlesource.com/emscripten-releases/+/refs/heads/main/src/build.py#65.

So the current situation is a little more confusing that I had thought:

Binaries built against: Debian/Stretch (glibc 2.24) Binaries built on: Ubuntu/Focal 20.04 (glibc 2.31) emsdk testing: Ubuntu/Bionic 18.03 (glibc 2.27) emscripten testing: Ubuntu/Bionic 18.03 (glibc 2.27) emscripten-releases testing: Ubuntu/Focal 20.04 (glibc 2.31) Current document requirement: Ubuntu/Xenial 16.04 (glibc 2.23) Oldest linux version by our users: ???

Actual glibc dependencies in current binaries:

$ readelf -V upstream/bin/clang 
...
Version needs section '.gnu.version_r' contains 8 entries:
 Addr: 0x00000000001111d0  Offset: 0x001111d0  Link: 7 (.dynstr)
...
  0x0180:   Name: GLIBC_2.16  Flags: none  Version: 17

GLIBC_2.16 seems to be the current highest value.

(node v18 requires glibc 2.28: nodesource/distributions#1392)

I think it would be better to standardize all the distributions everywhere and change the corresponding minimum requirements in the documentation (e.g. >= Debian10/Ubuntu20.04 or similar)

[sbc100]

I think it would be better to standardize all the distributions everywhere and change the corresponding minimum requirements in the documentation (e.g. >= Debian10/Ubuntu20.04 or similar)

Yes, clearly it would be better to have a more consistent story. The reason I'm holding of landing this change right away (don't get me wrong I think we can land it eventually) is that I'd like to get the story straight first, and I think we should coordinate with all the other places we do testing an building.

[juj]

I'm curious if you have any idea when you guys might drop support for Ubuntu 18?

Mirroring comment #1173 (comment) here as well: we do no longer need Ubuntu 18 support, but currently are at Ubuntu 20. So happy to move forward here.

[sbc100]

@DreamOfIce , would you have time to rebase this change? I think we can move foward with landing it now. If not I can roll it into #1387

[DreamOfIce]

I think I can do this. What should be the target platforms for this upgrade, Ubuntu 20.04/22.04/24.04 and Node.js 18/20LTS?

[sbc100]

I think I can do this. What should be the target platforms for this upgrade, Ubuntu 20.04/22.04/24.04 and Node.js 18/20LTS?

The current target of this PR is good (Focal). I think this PR just need updating to main its good to land.

I'll take care of the node upgrade as a followup in #1387

[DreamOfIce]

Okay, I'll resolve the conflict in these days.
However, as a reminder, there's less than a year left until the end of Ubuntu 20.04 free LTS.

[sbc100]

Okay, I'll resolve the conflict in these days. However, as a reminder, there's less than a year left until the end of Ubuntu 20.04 free LTS.

Those dates are not really the guide we use for the decision on our minimum supported linux version. We are more guided by the users of emsdk. Whether or not canonical supports something is less relevenat. Same with node... we support targeting versions of node that are long long out of LTS support.

[sbc100]

For some reason the circleci jobs are not being kicked off here. Could be a temporary issue with circleci? Or something else?

[sbc100]

I'm not sure whats up with CI here so testing this is a separate PR: #1390. If tests pass there I'll land this PR here.