-
-
Notifications
You must be signed in to change notification settings - Fork 219
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Docker Helper error on macOS Apple Silicon: "qemu: uncaught target signal 11 (Segmentation fault)" #8841
Comments
Hi @sookwalinga - thanks for filing this ticket to let us know you're having an issue. I see there's a forum topic you started with the same subject. We most often engage with the community on the forums, so I'm going to close this ticket and encourage you and others to follow up on the forums. thanks! |
I tested Docker Helper with several releases as below: 4.2.4 - Worked 4.4.0 CouchDB error logs similar to 4.5.1:
4.4.0 nginx container outputs errors too:
4.4.1 - Failed 4.4.1 CouchDB error logs:
To conclude, it looks like the problem was introduced with 4.4.0 (maybe related to CouchDB 3 upgrade?). Additionally, it seems like the |
Noted with thanks, will do that next time |
After doing some experimenting in fixing CHT Core dev env for CouchDB, I think we could automate this for docker helper. Basically, I think we could detect both macOS and Apple Silicon with calls like this: Here's what that command looks like on my Linux desktop:
Here's my my intel mac shows:
And here's what at an M1 MacBook Pro shows:
I'll work a PR up so docker helper will:
|
@dianabarsan - While I think my above idea might work, and unblock Apple Silicon Docker Helper users, I realize you could have docker helper install CHT Core In the end...maybe do the bigger lift and try and get native arm64 builds working on GHA? for at least CouchDB? Hmm - Thoughts? |
I think we should look into getting arm64 builds. This can be a separate build step that we only do for releases if we decide it's too much for branches. |
@dianabarsan - yeah - good call on the bigger fix! We're after Multi-Architecture (aka "Multi-Platform") builds. From my reading you just pass in some extra flags to build for There's some good notes from docker and this article on GHA look helpful. @1yuv - since you have an Apple Silicon laptop - are you interested in researching and improving CHT Docker images we build to be Multi-Platform? I know you've enjoyed hacking on some other CHT Core pieces of code and you came to mind for this as well! Not a big deal if it doesn't work out for you to do the work. |
Thanks @mrjones-plip , I have recently used the development environment directly without docker helper. But I can take a look into all these reference docs and see how we can improve the user experince. |
@1yuv - awesome - great to hear! I suspect if you tried to run the development set up on CHT 4.5 without a custom CouchDB image it would fail because of the reasons above. The goal would be to implement a fix upstream: by updating our GHA build process to be Multi-Platfrom, both Docker Helper and development environment would just work with out any extra steps to build the |
Hi @mrjones-plip , I know other services are working fine at the moment. Do we want to build multi-platform images for other services as well or only for couchdb at the moment? I suspect since everything else is working, we'd go only for couchdb? |
My hope is that the level of effort to do all images is only marginally more than to do one image. Let me know if that's not realistic! My thinking is that ability to natively run all containers will be a huge benefit to developers, both internal and external to Medic, so we should just go for it \o/ |
I will test first with couchdb images. My understanding is image size will be different if we build for multi-platform vs now. Once we've proof of work for couchdb and we know image sizes, we can opt for other. Efforwise, it should not be different. |
I've made the changes and build is successful with multi-platform locally as well as with gihtub actions. However, images built with I tried some other options as well and there are limitations. I am hesitant to make this change as currently we first build, publish, and save images and this has made us easier to have different workflows. I will explore more if there is a way to save first and publish later and not break our current workflow. |
Awesome - thanks for the update! My inclination is to go all in and carefully update our whole build process. While the images are going to be bigger, I think it's a very worthwhile trade off! Let's see what your research next shows as a path forward. |
I'm not too concerned about the size of images.
This sounds like it'll have unintended consequences. Is anyone else worried about pushing more and earlier? |
@garethbowen - can you speak to specific concerns you have? I'm otherwise not at all concerned - assuming we don't publish images publicly that people think are a GA release. @1yuv - this page states that we can run I assume the trick to get this to work for us is to run the test in the same runner that still has the cached build image? I'm not sure how hard/invasive this is to our current pipeline. |
Hi @mrjones-plip, If you take a look at where |
Ah - gotcha - thanks for pointing that out. Sorry I missed that! My take is that this is OK because we're testing the same image we're deploying as it caches the image on
|
Mostly about pushing garbage images that we then end up paying for. The later you push the less likely they are to be garbage. I don't think it'll be significantly different to what we already do. |
I was able to build muli-platform couchdb docker image and it was correctly pushed to intermediate repository (private). However, when we publish the image finally to public ecr, we pull and push. At this time, pull will pull docker image only for linux/amd64 since that's what the runner environment is and it'll push same image to public ecr, hence the arm64 image is not pubished to public ecr. I am thinking about using regctl which seems capable of copying image properly. Other alternative would be to directly publish image to public ecr at the time of testing, and use that. If a test fails, we can delete that image, otherwise leave it as is and there's no need to publish second time. @garethbowen , @dianabarsan, Interested to hear your opinion on this. |
@1yuv What about the "2022 docker built-in solution", the second comment on that SO post you linked?
Better to keep it out of public for as long as possible. And deleting public images might be confusing if someone installs it before it gets deleted. |
feat(#8841): Support multiplatform images
@sookwalinga - thanks for filing this ticket! While the ticket is closed as "fixed" CHT 4.7 is not out yet, so you'll need to take some extra steps to run Docker Helper on Apple Silicon with latest CHT Core. Mainly you need to select a compatible version when prompted, as seen on the final step:
Thanks again! |
Using Docker helper with the latest version of CHT v4.5.1 results in a 502 Bad Gateway error and the couchdb docker container keeps restarting. It was working fine before with v4.2.4.
I ran it using a MacBook Air M1 chip, 8GB RAM
mrjones adds that @andrablaj was able to reproduce, so this is likely a CHT issue and not a user error or configuration problem:
The text was updated successfully, but these errors were encountered: