docs: external dependency dashboard.

[htuch]

This patch introduces a set of automatically generated tables (based on repository_locations.bzl)
that enumerate the external dependencies that feature on Envoy's data/control planes, test, build, etc.

Version and CPE information is currently included. In the future, we will also have last updated,
distinguish core vs. extensions and populate with external dependency process maturity information.

Part of #10471. This is essentially providing a programmatic variant of #10471 (comment). Future enhancements are
tracked at #12673.

Signed-off-by: Harvey Tuch htuch@google.com

[htuch]

This is quite incomplete, posting as a PR to get some feedback on specific changes before doing a lot of manual work.

[htuch]

CC @yanavlasov @asraa

[moderation]

Should we follow the format of https://github.com/envoyproxy/envoy/blob/master/api/bazel/repository_locations.bzl#L1-L23 where both the version / Git SHA and SHA256 are editable in pairs? Without doing this to bump versions you need to edit the version at the top of the file and then search for the entry below to update the SHA256

[htuch]

Yeah, this seems alright. With this change, are you good with this metadata as a start?

[htuch]

I switched to something a bit cleaner (and will update api/bazel/repository_locations.bzl to this in the future) - using string interpolation to allow this to be a regular dictionary field. PTAL.

[htuch]

Example table so far: https://372226-65214191-gh.circle-artifacts.com/0/generated/docs/intro/arch_overview/security/external_deps.html

[htuch]

This is now ready for review, docs output at https://372762-65214191-gh.circle-artifacts.com/0/generated/docs/intro/arch_overview/security/external_deps.html.

[moderation]

The test output looks good. The mixing of plain English capitalized project name descriptions with lower case Github repo names is a little jarring in code but looks OK in the output and is sorted correctly.

[moderation]

Small thing but I've been wanting to sort USE_CATEGORIES in bazel/repository_locations

USE_CATEGORIES = [
    # This dependency is used in API protos.
    "api",
    # This dependency is used in build process.
    "build",
    # This dependency is used to process xDS requests.
    "controlplane",
    # This dependency is used in processing downstream or upstream requests.
    "dataplane",
    # This dependecy is used for logging, metrics or tracing. It may process unstrusted input.
    "observability",
    # This dependency does not handle untrusted data and is used for various utility purposes.
    "other",
    # This dependency is used for unit tests.
    "test",
]

[mattklein123]

This is awesome! Fine by me to ship and iterate. A few wishes from my view of the rendered page:

1. I think requiring a human description/summary field would be useful. There is risk of this getting out of date so maybe not but something to think about.
2. For GH dependencies, it would be nice to have a link that takes us directly to the tree at the version/SHA we are using.

Nice!

[htuch]

@moderation sure, will alpha sort in the next PR.

@mattklein123 @lizan added your suggestions to #12673

[moderation]

@htuch and @lizan this is a minor thing but there are 3 dependencies that use a modified version string as part of their URL. I've tested the following and it looks like it works. The Bazel documentation is at https://docs.bazel.build/versions/master/skylark/lib/string.html#replace

urls = ["https://github.com/curl/curl/releases/download/curl-{version}.replace('.','_')/curl-{version}.tar.xz"],
urls = ["https://github.com/unicode-org/icu/releases/download/release-{version}/icu4c-{version}.replace('-','_')-src.tgz"],
urls = ["https://github.com/c-ares/c-ares/releases/download/cares-{version}.replace('.','_')/c-ares-{version}.tar.gz"],

Is there any reason to not include this change in a subsequent PR?

[htuch]

@moderation sure, I've added this as a tracked item in #12673