Bundle local images in rustdoc output

text/000-rustdoc-bundle-local-resources.md

@@ -0,0 +1,118 @@
+Rustdoc: Bundle local images
+
+- Feature Name: NONE
+- Start Date: 2023-02-06
+- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
+- Rust Issue: [rust-lang/rust#32104](https://github.com/rust-lang/rust/issues/32104)
+
+# Summary
+[summary]: #summary
+
+This RFC proposes to allow the bundling of local images in rustdoc HTML output. A draft implementation is available as [#107640](https://github.com/rust-lang/rust/pull/107640).
+
+# Motivation
+[motivation]: #motivation
+
+Doc authors want to produce docs that are consistent across local `cargo doc` output, `docs.rs`, and self-hosted docs. They would also like to include images (like logos and diagrams), and scripts (like KaTeX for rendering math symbols). Both doc authors and doc readers would like for those resources to not be subject to link-rot, which means it should be possible to build docs for an old version of a crate and have the images and scripts reliably available. Doc readers would like for `cargo doc` output to be rendered correctly by their browsers even when they are offline.
+
+Right now, there are attributes that can set a logo and a favicon for documentation, but they must to point to an absolute URL, which prevents bundling the logo and favicon in the source repository. Also, while `<script>` tags are allowed in rustdoc, they have a similar problem: if they load script from some URL, that URL needs to be absolute or it won't work consistently across `cargo doc` and `docs.rs`.
+
+# Guide-level explanation
+[guide-level-explanation]: #guide-level-explanation
+
+This RFC proposes to allow rustdoc to include local images in the generated documentation by copying them into the output directory.
+
+This would be done by allowing users to specify the path of a local resource file in doc comments. The resource file would be stored in the `doc.files` folder. The `doc.files` folder will be at the "top level" of the rustdoc output level (at the same level as the `static.files` or the `src` folders).
+
+The only local resources considered will be the ones in the markdown image syntax: `![resource title](path)`, where `<path>` is the path of the resource file relative to the source file.
+
+The path could be any relative or absolute file path. For example, to include an image generated by [`build.rs`](https://doc.rust-lang.org/cargo/reference/build-scripts.html), concatenate a path with the `OUT_DIR` environment variable:
+
+```rust
+/// Using a local image
+#[doc=concat!("![with absolute path](", env!("OUT_DIR"), "/local/image.png)")]
+///
+/// Using a local image ![with relative path](../local/image.png)
+```
+
+Since the local images are all put in the same folder, if the same is imported from different crates, the content won't be duplicated since they have the same name and the same hash.
+
+If the path isn't referring to a file, a warning will be emitted and rustdoc will left the path unchanged in the generated documentation.
+
+For published crates, `docs.rs` builds the contents of the `.crate` package in a sandbox with no internet access. Make sure any resources your docs need are [included](https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields) in the package.
+
+The local resources files are not affected by the `--resource-suffix`.
+
+The impact on `docs.rs` would also be very minimal as the size of a published crate resources is limited to a few megabytes. The only thing needed would be to handle the new `doc.files` folder.
+
+There are two options that will be impacted by this RFC: the favicon and logo that you can respectively set through:
+
+```rust
+#![doc(html_favicon_url = "some_path", html_logo_url = "some_other_path")]
+```
+
+They will follow the same rule as for other images: if this is a local path, the local file will be copied and the paths to it will be rewritten.
+
+To support `#[doc(inline)]` for foreign items using local resources, it will rely on the `-Zrustdoc-map` option.
+
+# Reference-level explanation
+[reference-level-explanation]: #reference-level-explanation
+
+A new rustdoc pass will be added which would go through all documentation to gather local resources into a map.
+
+Then in HTML documentation generation, the local resources pathes will be replaced by their equivalent linking to the output directory instead.
+
+The local resources files will be renamed as follows: `{original filename}-{hash}{extension}`. The `{hash}` information will be computed from the local resource file content.
+
+You can look at what the implementation could look like in [#107640](https://github.com/rust-lang/rust/pull/107640).
+
+When an image is included in an item that gets inlined across a crate, rustdoc will treat it like a cross-crate intra-doc link, using `--extern-html-root-url` so that `docs.rs` can hotlink the image from the crate that holds a copy of the image. This has a few upsides and downsides compared to the approach where the image itself is copied into the crate with the inlined docs.
+
+ * reduces the number of duplicated images that `docs.rs` has to store
+ * doesn't require the source code for the source crate when inlining
+ * only requires storing the hash of the file in the `.rmeta`, not the whole image
+ * requires rustc to look at the doc comments and hash the image(s)
+ * produces broken images when `cargo doc --no-deps` is run locally
+ * requires the URLs generated for these images to be stable across rustdoc versions
+
+Embedding images in rmeta files is probably a bad idea, requiring access to the source code for a dependent crate when building the dependency would work poorly with some third-party build systems, and not supporting images in cross-crate inlined docs would be inconsistent and weird.
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+Allowing local resources in rustdoc output could lead to big output files if users include big resource files. This could lead to slower build times and increase the size of generated documentation (in particular in case of very big local resources!).
+
+Another problem is that people will add images into their published crates, increasing the package size whereas it's only used for documentation.
+
+# Prior art
+[prior-art]: #prior-art
+
+- [sphinx](https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-latex_additional_files)
+- [haddock](https://haskell-haddock.readthedocs.io/en/latest/invoking.html?highlight=image#cmdoption-theme): it's mentioned in this command documentation that local files in the given directory will be copied into the generated output directory.
+- [doxygen](https://doxygen.nl/manual/commands.html#cmdimage): supported through `\image`.
+- [embed-doc-image](https://docs.rs/embed-doc-image/latest/embed_doc_image/): a proc-macro based version which directly embed the content into the generated documented as a base64 string.
+
+Another approach to this feature:
+
+[ePUB packages](https://www.w3.org/publishing/epub3/epub-packages.html#sec-pkg-manifest) use explicitly-declared manifests. It allows to have a fallback chain mechanism (going through resources for an entry until an available resource is found).
+
+# Rationale and alternatives
+[rationale-and-alternatives]: #rationale-and-alternatives
+
+Currently, to provide resources, users need to specify external URLs for resources or inline them (if possible like the `svg` image format) directly into the documentation. It has the advantage to avoid the problem of large output files, but it also requires users to upload their resources to a web server to make them available everywhere.
+
+# Unresolved Questions
+[unresolved-questions]: #unresolved-questions
+
+- Should we put a size limit on the local resources?
+- Should we somehow keep the original local resource filename instead of just using a number instead?
+- Should we use this feature for the logo if it's a local file?
+
+# Possible extensions
+[possible-extensions]: #possible-extensions
+
+This feature could be extended to DOM content using local resources. It would require to add parsing for HTML tags attributes. For example:
+
+```html
+/// <video src="../some-video.mp4">
+```