core 112.06.00 rc1

[bmillwood]

This one was delayed a while, so it's a big one:

async_extended

• Unwound a recent change to Mailbox where one invocation of receive
  would put aside items, preventing other invocations from noticing
  them.
• Added Delimited.Row.nth_conv_exn, as a counterpart to
  get_conv_exn.
• Fixed File_updates handling of identical mtimes.

async_extra

• In Log, exposed the raw message.

• Changed Rpc creators' connection_state to be a function that takes
  the connection and returns the state.

  This makes it possible for the connection state to actually get
  a handle on the connection itself, which simplifies a number of
  idioms for using RPC. In particular, it makes it easier to respond
  with an RPC back to a client over client's own connection.

• Fixed some nondeterministically failing tests.

• In Log, made logs discard messages when their output list is empty.

  Also, removed redundant tracking of current level.

• Moved Udp.bind_to_interface_exn to Unix module in async_unix.

• Added Versioned_typed_tcp.Repeater.

  Repeater is used in the cases where we want to inspect and possibly
  alter the flow between a client and a server without having to
  change either the client or the server or the protocol between them.

async_kernel

• Added Deferred.Sequence module, analogous to Deferred.List but for
  Core_kernel.Std.Sequence.
• Modernized code style.

async_unix

• Added Writer.behave_nicely_in_pipeline, which makes a program behave
  nicely when used in a shell pipeline where the consumer goes away.

• Modernized code style.

• Removed spurious overrides in Signal: set and signal.

  These overrides are no longer necessary because the functions moved
  from Core.Signal to Core.Signal.Expert.

• Moved async_extra's Udp.bind_to_interface_exn to Unix.

bignum

• Added functions to round from Bignum.t to Bigint.t, and to convert
  Bigint.t into Bignum.t.

bin_prot

• Sped up float and float array operations.

• Removed a use of Obj.magic in code generated by pa_bin_prot for
  polymorphic variants that led to memory unsafety.

  Previously, pa_bin_prot generated this kind of code for
  polymorphic variants:

    match Obj.magic (read_int buf pos) with
    | `A as x -> x
    | `B as x -> x
    | `C -> `C (read_float buf pos)
    | _ -> fail
  

  and this caused the compiler to assume the result is an immediate
  value. To fix this we removed the as x -> x and used the computed
  integer hash.

core

• Renamed Linux_ext.gettid as Unix.gettid, and added OpenBSD support.

  SYS_gettid is not available on OpenBSD, but is used in
  Core_extended. See the mailing list discussion about this here:

  https://groups.google.com/forum/#!topic/ocaml-core/51knlnuJ8MM

  Seems like the OpenBSD alternative is:

  pid_t        getthrid(void);
  

  although it's not defined in any header file, which is a bit unfortunate.

• Added Piecewise_linear.precache, which computes a lookup table that
  speeds up subsequent calls to Piecewise_linear.get.

• Added Time_ns module, representing times as 63-bit integers of
  nanoseconds since the epoch.

• Fixed build of unix_stubs.c on OpenBSD.

• In Daemon, fixed an error message regarding WSTOPPED (fixes Add Thomas' rpc-light #47).

• Added Time.Span.Stable.V2, with sexps that use new suffixes for
  microseconds (us) and nanoseconds (ns).

  Time.Span.of_string supports the new format, but
  Time.Span.to_string doesn't yet produce it -- we plan to change
  that later, after the new of_string has made it out more widely.

• Added Time.Span.to_string_hum, which gives more options for
  rendering time spans.

• Merged the recvmmsg stubs in Bigstring and Iobuf.

  Factored out a shared underlying recvmmsg call that both
  stubs use.

  Restored -pedantic by avoiding a C99 feature (variable-length
  stack arrays).

• Made Date.t abstract, and changed its representation from a 4-word
  record to an immediate int (packing year, month, day).

• In Daemon, changed the permissions of the std{err,out} files
  generated during daemonization from 0o777 to 0o644.

• Moved Thread_safe_queue from core to core_kernel.

  This was done so that Async_kernel can use it, eliminating one of
  Async_kernel's dependencies on Core.

  Thread_safe_queue_unit_tests remains Core, at least for now,
  because it has some dependencies on other stuff in Core.

core_bench

• Solved a problem in which OCaml 4.02 was optimizing away benchmarks,
  making them meaningless.

core_extended

• Sped up String.is_substring by replacing the OCaml implementation
  with a call to libc memmem.

  memmem runs in 20% of the time, incurs minimal GC pressure, is
  portable among UNIXen that we target, AND it's clearer than the ML
  version.

• Made Float_ref support bin_io and sexp.

• Removed gettid, which is now available in Core.Unix.

• Added Fast_int_div module, which speeds up integer division by
  a fixed divisor.

• Moved Sexp.of_sexp_allow_extra_fields to core_kernel.

core_kernel

• Made String_id have Stable_containers.Comparable.

• Changed Gc.disable_compaction to require an allocation_policy.

• Made Option match Invariant.S1.

• Added Sequence.filter, compare, and sexp_of_t.

• Added With_return.with_return_option, abstracting a common pattern
  of with_return.

    val with_return        : ('a return -> 'a  ) -> 'a
    val with_return_option : ('a return -> unit) -> 'a option
  

• Install a handler for uncaught exceptions, using
  Printexc.set_uncaught_exception_handler, new in OCaml 4.02.

• Changed Day_of_week representation to a normal variant.

• Changed Exn.handle_uncaught so that if it is unable to print, it
  still does exit 1.

• Added Sexp.of_sexp_allow_extra_fields, previously in
  Core_extended.Sexp.

• Changed the implementation of Exn.raise_without_backtrace to use
  raise_notrace, new in OCaml 4.02.

• Added Float functions for converting to and from IEEE
  sign/exponent/mantissa.

• Added String.Caseless module, which compares and hashes strings
  ignoring case.

• Reimplemented Type_equal.Id using extensible types (new in OCaml
  4.02), removing a use of Obj.magic.

  Changed Type_equal.Id.same_witness to return option rather than
  Or_error, which allows it to be implemented without allocation.

• Removed a reference to the Unix module. Applications using
  core_kernel should be able to link without unix.cma again.

• Made Char.is_whitespace accept \f and \v as whitespace,
  matching C.

jenga

• Support for user control of stale-artifact deletion, by allowing
  specification of an artifact-determination policy.

• Expose jenga's internal (and better - only quotes when necessary)
  definition of Shell.escape in Api

• Removed Action.shell from the API, superseded by Action.process.

• Changed RPC interface as needed for build manager to switch from
  scraping error messages to RPCs.

• Fixed jenga's per-rule memo table, which mistakenly kept stale values.

• Show what target is being demanded, useful for debugging rules.

• Run user action when persistent format changes.

• When filtering buildable targets by globs, pay attention to the kinds
  allowed by the glob.

  Specifically, if the kinds don't include File` (i.e. only include Directory) then we should not see any buildable_targets` in the filtered list.

ocaml_plugin

• Stopped using the ~exclusive with Reader, because it doesn't work
  on read-only file systems.

  It's not even needed because these files are written atomically.

• Used a generative functor in the generated code, so the user code can
  apply generative functors at toplevel, or unpack first class modules
  that contain type components.

• Fixed bug when mli file references something defined only in
  another ml.

• Made it possible to compile a plugin in one process, and dynload the
  compiled cmxs file without starting async in another process.

  This was done with two new APIs in Ocaml_dynloader.S:

    val compile_ocaml_src_files_into_cmxs_file
      : dynloader
      -> string list
      -> output_file:string
      -> unit Deferred.Or_error.t
  
    val blocking_load_cmxs_file : string -> t Or_error.t
  

• Allowed plugins to optionally have a shebang line.

• Made Ocaml_dynloader.find_dependencies also support files with
  shebang lines.

pa_bench

• Made the code generated by pa_bench for BENCH not use ignore,
  because OCaml 4.02 will remove dead code in some cases, meaning the
  benchmarks are no longer measuring what they should. Instead the ignore
  is deep inside Core_bench, which is likely out of reach of the
  compiler.

  The result of the user functions given to BENCH_FUN and
  BENCH_INDEXED are changed so they don't have to return unit and
  people are encouraged not to use ignore when these functions don't
  return unit (you will get the same warning though, i.e. a warning
  if the result of your function is a function too, thus preventing
  unintended partial applications).

  For example, here are a few benchmarks and their output before the
  fix:

    let x = if Random.bool () then 100 else 1001
    let r = ref 0
    BENCH "ig-1" = 10 / x
    BENCH "ig-2" = ()
    BENCH "ig-3" = phys_equal (10 / x) (Obj.magic 0)
    BENCH "ig-4" = r := (10 / x)
    BENCH "ig-5" = r := x
  
    +----------------+----------+------------+
    | Name           | Time/Run | Percentage |
    +----------------+----------+------------+
    | [misc.ml] ig-1 |   3.92ns |     29.30% |
    | [misc.ml] ig-2 |   3.34ns |     24.95% |
    | [misc.ml] ig-3 |   3.91ns |     29.23% |
    | [misc.ml] ig-4 |  13.37ns |    100.00% |
    | [misc.ml] ig-5 |   3.24ns |     24.20% |
    +----------------+----------+------------+
  

  Many of the the numbers above are much lower than they should be
  because of the implicit ignores inserted by the benchmark caused the
  division to to eliminated by the compiler. After the fix, the same
  benchmarks produced more meaningful numbers:

    +----------------+----------+------------+
    | Name           | Time/Run | Percentage |
    +----------------+----------+------------+
    | [misc.ml] ig-1 |  12.78ns |     94.55% |
    | [misc.ml] ig-2 |   3.23ns |     23.90% |
    | [misc.ml] ig-3 |  13.51ns |     99.94% |
    | [misc.ml] ig-4 |  13.52ns |    100.00% |
    | [misc.ml] ig-5 |   3.30ns |     24.40% |
    +----------------+----------+------------+
  

sexplib

• Improved the implementation of Exn.sexp_of_t, using the unique id in
  exceptions in OCaml 4.02.

  We use the identifier to map exception constructors to converters.

[bmillwood]

@whitequark normally there's a bit more process before we merge these -- see for example #2764

@avsm take a look at this anyway? I think the right thing to do here is just do a final release even if there are problems, and then do 112.06.01 to fix anything that might arise (we already have #3202 to fix)

[whitequark]

@bmillwood Noted.

[samoht]

@whitequark yes usually we try at least to build core manually on OSX before merging.