Auto merge of rust-lang#28869 - alexcrichton:allocator-dox, r=stevekl…

…abnik

This adds a chapter to the nightly section of the book on leveraging and
implementing the `#![allocator]` attribute to write custom allocators as well as
explaining the current situation with allocators.

src/doc/trpl/SUMMARY.md

@@ -68,5 +68,6 @@
     * [Box Syntax and Patterns](box-syntax-and-patterns.md)
     * [Slice Patterns](slice-patterns.md)
     * [Associated Constants](associated-constants.md)
+    * [Custom Allocators](custom-allocators.md)
 * [Glossary](glossary.md)
 * [Bibliography](bibliography.md)

src/doc/trpl/custom-allocators.md

@@ -0,0 +1,170 @@
+% Custom Allocators
+
+Allocating memory isn't always the easiest thing to do, and while Rust generally
+takes care of this by default it often becomes necessary to customize how
+allocation occurs. The compiler and standard library currently allow switching
+out the default global allocator in use at compile time. The design is currently
+spelled out in [RFC 1183][rfc] but this will walk you through how to get your
+own allocator up and running.
+
+[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1183-swap-out-jemalloc.md
+
+# Default Allocator
+
+The compiler currently ships two default allocators: `alloc_system` and
+`alloc_jemalloc` (some targets don't have jemalloc, however). These allocators
+are just normal Rust crates and contain an implementation of the routines to
+allocate and deallocate memory. The standard library is not compiled assuming
+either one, and the compiler will decide which allocator is in use at
+compile-time depending on the type of output artifact being produced.
+
+Binaries generated by the compiler will use `alloc_jemalloc` by default (where
+available). In this situation the compiler "controls the world" in the sense of
+it has power over the final link. Primarily this means that the allocator
+decision can be left up the compiler.
+
+Dynamic and static libraries, however, will use `alloc_system` by default. Here
+Rust is typically a 'guest' in another application or another world where it
+cannot authoritatively decide what allocator is in use. As a result it resorts
+back to the standard APIs (e.g. `malloc` and `free`) for acquiring and releasing
+memory.
+
+# Switching Allocators
+
+Although the compiler's default choices may work most of the time, it's often
+necessary to tweak certain aspects. Overriding the compiler's decision about
+which allocator is in use is done simply by linking to the desired allocator:
+
+```rust,no_run
+#![feature(alloc_system)]
+
+extern crate alloc_system;
+
+fn main() {
+    let a = Box::new(4); // allocates from the system allocator
+    println!("{}", a);
+}
+```
+
+In this example the binary generated will not link to jemalloc by default but
+instead use the system allocator. Conversely to generate a dynamic library which
+uses jemalloc by default one would write:
+
+```rust,ignore
+#![feature(alloc_jemalloc)]
+#![crate_type = "dylib"]
+
+extern crate alloc_jemalloc;
+
+pub fn foo() {
+    let a = Box::new(4); // allocates from jemalloc
+    println!("{}", a);
+}
+# fn main() {}
+```
+
+# Writing a custom allocator
+
+Sometimes even the choices of jemalloc vs the system allocator aren't enough and
+an entirely new custom allocator is required. In this you'll write your own
+crate which implements the allocator API (e.g. the same as `alloc_system` or
+`alloc_jemalloc`). As an example, let's take a look at a simplified and
+annotated version of `alloc_system`
+
+```rust,no_run
+# // only needed for rustdoc --test down below
+# #![feature(lang_items)]
+// The compiler needs to be instructed that this crate is an allocator in order
+// to realize that when this is linked in another allocator like jemalloc should
+// not be linked in
+#![feature(allocator)]
+#![allocator]
+
+// Allocators are not allowed to depend on the standard library which in turn
+// requires an allocator in order to avoid circular dependencies. This crate,
+// however, can use all of libcore.
+#![feature(no_std)]
+#![no_std]
+
+// Let's give a unique name to our custom allocator
+#![crate_name = "my_allocator"]
+#![crate_type = "rlib"]
+
+// Our system allocator will use the in-tree libc crate for FFI bindings. Note
+// that currently the external (crates.io) libc cannot be used because it links
+// to the standard library (e.g. `#![no_std]` isn't stable yet), so that's why
+// this specifically requires the in-tree version.
+#![feature(libc)]
+extern crate libc;
+
+// Listed below are the five allocation functions currently required by custom
+// allocators. Their signatures and symbol names are not currently typechecked
+// by the compiler, but this is a future extension and are required to match
+// what is found below.
+//
+// Note that the standard `malloc` and `realloc` functions do not provide a way
+// to communicate alignment so this implementation would need to be improved
+// with respect to alignment in that aspect.
+
+#[no_mangle]
+pub extern fn __rust_allocate(size: usize, _align: usize) -> *mut u8 {
+    unsafe { libc::malloc(size as libc::size_t) as *mut u8 }
+}
+
+#[no_mangle]
+pub extern fn __rust_deallocate(ptr: *mut u8, _old_size: usize, _align: usize) {
+    unsafe { libc::free(ptr as *mut libc::c_void) }
+}
+
+#[no_mangle]
+pub extern fn __rust_reallocate(ptr: *mut u8, _old_size: usize, size: usize,
+                                _align: usize) -> *mut u8 {
+    unsafe {
+        libc::realloc(ptr as *mut libc::c_void, size as libc::size_t) as *mut u8
+    }
+}
+
+#[no_mangle]
+pub extern fn __rust_reallocate_inplace(_ptr: *mut u8, old_size: usize,
+                                        _size: usize, _align: usize) -> usize {
+    old_size // this api is not supported by libc
+}
+
+#[no_mangle]
+pub extern fn __rust_usable_size(size: usize, _align: usize) -> usize {
+    size
+}
+
+# // just needed to get rustdoc to test this
+# fn main() {}
+# #[lang = "panic_fmt"] fn panic_fmt() {}
+# #[lang = "eh_personality"] fn eh_personality() {}
+# #[lang = "eh_unwind_resume"] extern fn eh_unwind_resume() {}
+```
+
+After we compile this crate, it can be used as follows:
+
+```rust,ignore
+extern crate my_allocator;
+
+fn main() {
+    let a = Box::new(8); // allocates memory via our custom allocator crate
+    println!("{}", a);
+}
+```
+
+# Custom allocator limitations
+
+There are a few restrictions when working with custom allocators which may cause
+compiler errors:
+
+* Any one artifact may only be linked to at most one allocator. Binaries,
+  dylibs, and staticlibs must link to exactly one allocator, and if none have
+  been explicitly chosen the compiler will choose one. On the other than rlibs
+  do not need to link to an allocator (but still can).
+
+* A consumer of an allocator is tagged with `#![needs_allocator]` (e.g. the
+  `liballoc` crate currently) and an `#[allocator]` crate cannot transitively
+  depend on a crate which needs an allocator (e.g. circular dependencies are not
+  allowed). This basically means that allocators must restrict themselves to
+  libcore currently.