OpenSearch service of Registry won't start on Linux

[nutjob4life]

🐛 Describe the bug

When running docker compose up to start the services for the Registry on Linux, the elasticsearch service (confusingly named to start OpenSearch) exits with a failed status code.

This occurs on Red Hat Enterprise Linux release 8.7 (Ootpa), Debian GNU/Linux 11 (bullseye), and Linux Mint 20.3.

📜 To Reproduce

$ lsb_release --description
Description:    Linux Mint 20.3
$ docker compose version
Docker Compose version v2.14.1
$ docker compose --profile dev-api up --detach
…
$ sleep 60
$ docker compose --profile dev-api ps | egrep Exited
docker-elasticsearch-1               opensearchproject/opensearch:1.1.0   "./opensearch-docker…"   elasticsearch               About a minute ago   Exited (0) 55 seconds ago   
$ docker compose --profile dev-api logs --tail 4 elasticsearch
docker-elasticsearch-1  | For complete error details, refer to the log at /usr/share/opensearch/logs/docker-cluster.log
docker-elasticsearch-1  | Killing performance analyzer process 35
docker-elasticsearch-1  | OpenSearch exited with code 1
docker-elasticsearch-1  | Performance analyzer exited with code 143
$ echo the root cause seems to be …
echo the root cause seems to be …
$ docker compose --profile dev-api logs elasticsearch | egrep 'Device or resource busy'
docker-elasticsearch-1  | sed: cannot rename /usr/share/opensearch/config/sedERYoCV: Device or resource busy

🕵️ Expected behavior

$ docker compose --profile dev-api ps
NAME                                 COMMAND                  SERVICE                     STATUS              PORTS
docker-elasticsearch-1               "./opensearch-docker…"   elasticsearch               running             0.0.0.0:9200->9200/tcp, 9300/tcp, 0.0.0.0:9600->9600/tcp, 9650/tcp
docker-elasticsearch-init-1          "bash /usr/local/bin…"   elasticsearch-init          exited (1)          
docker-registry-loader-test-init-1   "/usr/local/bin/regi…"   registry-loader-test-init   running             
docker-registry-web-archive-1        "/docker-entrypoint.…"   registry-web-archive        running             0.0.0.0:81->80/tcp

📚 Version of Software Used

main branch of NASA-PDS/registry pulled 2022-12-30 (3f6661c).

🖥 System Info

• Linux of various flavors
• Docker version 20.10.22

---

🦄 Related requirements

⚙️ Engineering Details

[nutjob4life]

For fun, I tried it on Windows 11 Pro with Docker Compose 2.2.3 and it works there just fine 🎉

[nutjob4life]

Been experimenting a bit more. This seems to be a known issue with OpenSearch:

• Fix issue with docker volume-mounted config file opensearch-project/opensearch-build#1130
• [BUG] Docker stack not running correctly when mounting config folder opensearch-project/OpenSearch#1579

Trying 1.1.1 → 1.2.3 all exhibit the same failure ("Device or resource busy"). But 1.2.4 gets past that and fails for a different reason.

[jordanpadams]

ready for review