Smaller image possible?

[thanatos]

There's a comment in the Xenial Dockerfile:

docker-openresty/xenial/Dockerfile

Line 107 in 4e12d1e

# TODO: remove any other apt packages?

I think removing build-essential and make, just prior to the apt-get autoremove should cut the image size by ~100MiB, or ~20%.

[neomantra]

I'm sorry I haven't been responsive on this. I think it does make sense to make all these images as slim as possible. And think that can happen across all the packages. And keep a debian-fat like there is alpine-fat with build packages installed.

But there should be examples in the LuaRocks README section for installing a build environment for C-based Lua rocks. So I don't want to pull this change without some docs (like a sentence and a blockquote of code) and also a line in the CHANGELOG. Thanks.

[neomantra]

My plan is to rework the Dockerfiles using Docker multi-stage builds and provide thin/fat versions of all the packages.

[alyoshark]

It would be great!

[neomantra]

The only images focused on size are the alpine and upstream flavors. If you have a PR to make them smaller, I’ll merge it.

[alyoshark]

I made a POC here:

https://github.com/xch91/openresty-container

[JuryA]

No, openresty containers are big, bloaty and evidently neglected… no offense…

TL;DR

In my professional opinion, containerized environments destined for production deployments should strictly adhere to a lean philosophy, devoid of any extraneous bloatware. A well-curated container should encompass only the indispensable elements for its operational integrity. In this context, that translates to including openresty along with its requisite dependencies. Any additional compiled content can be optionally included or mounted as a volume, particularly since a sidecar proxy obviates even that need. Superfluous binaries not only consume valuable storage but also critically, each extraneous executable introduces a potential attack vector.

Debugging Metadata

Furthermore, it's both prudent and advisable to meticulously strip all binaries of superfluous debugging metadata—symbols, et cetera—either during compilation or post-compile. This exercise serves multiple purposes:

• Conserves storage
• Fortifies security
• Expedites runtime
  It can effortlessly be executed with utilities like strip, making it accessible even for non-technical personnel.

Security Considerations

From a security standpoint, adopting a rootless architecture, enabling read-only root filesystems, and having SELinux or equivalent security mechanisms in place are non-negotiables. These measures drastically reduce the container's attack surface. If additional privileges are ever required, pause to consider whether they are absolutely essential. If unavoidable, grant only the specific permissions necessary—never more. This approach minimizes potential vulnerabilities and aligns with the principle of least privilege.

Shell Access in Production Containers

In a production-grade container, shell access should be deemed unnecessary. With the advent of debugging service containers that can easily be attached to any running instance, the availability of a shell within a container lacks any justifiable merit. This capability is uniformly supported across container orchestration platforms like Docker, Podman, OpenShift, and Kubernetes.

Immutability and Configuration Drift

Additionally, immutability should be a focal point in containerization. Once a container is deployed, its runtime state should remain unchangeable to ensure a consistent environment. This avoids drift in configurations and minimizes vulnerabilities, thereby solidifying security posture. The rise of GitOps further validates this approach, as it promotes the principle of 'Immutable Infrastructure' where code and configuration are version-controlled.

Resource Allocation

Moreover, attention to resource allocation is paramount. Leveraging container orchestration platforms allows for dynamic resource scaling, but it also necessitates stringent resource quotas and limits to prevent resource exhaustion and ensure fair scheduling. This is especially crucial in multi-tenant environments where resource contention can lead to performance degradation or even service outages. And ... Who would be willing to pay for services that waste resources through poor optimization?

---

Now, after reading my commentary, you might be thinking that I'm some sort of armchair expert who's idealizing an Orwellian Utopia—that achieving such an "ideal" in reality is virtually impossible. Yes, you're right... But that doesn't mean we shouldn't strive for it! I would start with eliminating bloatware. For instance, I prepare the foundations of my containers using nix + buildkit-nix and then use them as the base in regular Dockerfiles. In all my security audits, I've never scored worse than low single-digit percentages.

So, while the perfect containerized environment might seem like a pipe dream, making incremental improvements can bring us closer to that ideal. It's all about being vigilant, disciplined, and unafraid to leverage the best tools and practices available.

If you're interested, I can provide some example nix build configurations for automated Docker image builds. This could serve as a practical starting point in achieving some of the security and efficiency gains discussed here. Such an amazing project as openresty deserves a better container!

[neomantra]

No, openresty containers are big, bloaty and evidently neglected… no offense…

No offense taken by me, especially since you provided some great feedback and are offering to contribute. ❤️ Neglect is a complicated word; I will note that I respond to CVE's on this project within hours of notice.

It took a bit longer to reply to this, but I have been thinking about it :) Most everything you write makes sense and is great information for anybody to read -- maybe put it in a gist if you haven't, so it's not just lost in a GH issue.

Personally, I am not going to advance that directly, but I also will not impede progress. I will also continue to make sure that upstream work is pulled and CVE-oriented modifications to build-from-source are incorporated.

---

If you or anyone has specific PR's for the alpine (from-source), or any of the from-upstream packges, I'll review an incorporate. This is a good tracking issue for it.

I personally find Nix interesting and was excited for their recent independent rebuild. Immutable builds, software bill of materials, is all great. I strive hard for the builds of this to be public on Travis, but you can't beat checksums for transparency. However, I have no experience with Nix and I cannot develop that myself.

If you @JuryA want to make a Nix tracking issue and eventually an accompanying PR, I would check it out. We can build some nix- image next to the other flavors and see how it tastes. I'd hope that others concerned netizens can review as well.