[Caml-list] [ANN] Core Suite 111.11.00

[Caml-list] [ANN] Core Suite 111.11.00

[Jeremie Dimino]

[-- Attachment #1: Type: text/plain, Size: 10157 bytes --]

I am pleased to announce the 111.11.00 release of the Core suite.

The following packages were upgraded:

- async
- async_extra
- async_kernel
- async_unix
- core
- core_extended
- core_kernel
- ocaml_plugin
- sexplib

Files and documentation for this release are available on our website
and all packages are in opam:

  https://ocaml.janestreet.com/ocaml-core/111.11.00/individual/
  https://ocaml.janestreet.com/ocaml-core/111.11.00/doc/

Here is the changelog for this version:

# 111.11.00

## async

- Updated the sound.ml example

## async_extra

- Made `Log` more fair with respect to other Async jobs, by working on
  fixed-length groups of incoming log messages.

  Previously, `Log` had processed everything available.  The change
  gives other Async jobs more of a chance to run.

## async_kernel

- Added `Clock.run_at_intervals`, which runs a job at regular
  intervals.

## async_unix

- Added `val Scheduler.yield : unit -> unit Deferred.t`, which becomes
  determined after the current cycle finishes.
- Improved the behavior of the scheduler's thread pool when
  `Thread.create` raises.

  With this improvement, when the thread pool is unable to create a
  thread, it presses on with the threads it has rather than raise.
  Subsequent attempts to add work to the thread pool will cause the
  pool to attempt to create the thread, as long as enough time has
  passed since the most recent thread-creation failure.

  Before this change, the thread pool wouldn't handle a
  `Thread.create` exception, and the exception would get raised to
  whatever code happened to be calling the `Thread_pool` function that
  tried to create a thread, e.g. `Thread_pool.add_work`.  This caused
  `In_thread.run` to unexpectedly raise, and in turn
  `In_thread.syscall` to unexpectedly raise, leading to:

  ```
  "Fd.syscall_in_thread bug -- should be impossible"
  ```

  Also, changed `should be impossible` text to `please report`, since
  there may be other lurking rare exceptions that `In_thread.syscall`
  can raise, and we'd like to hear about them.

  We rely on thread-pool-stuck detection to report problems where the
  inability to create threads causes the inability of the thread pool
  to make progress.  A tweak was needed to make that work -- now the
  thread-pool-stuck warning is based on whether the thread pool has
  unstarted work, rather than on whether the thread pool has an
  "available thread".  The latter would no longer work, since it is
  now possible for the thread pool to have unstarted work and to
  appear to have an available thread, i.e. `num_threads <
  max_num_threads`.

## core

- Change some `Bigstring` functions to retry on `EINTR` rather than
  raise.

  The following functions (and their unsafe versions) were affected:

  * `read`
  * `really_read`
  * `really_recv`
  * `really_write`
  * `really_send_no_sigpipe`

  Some other `Bigstring` functions, like `input` and `output`, already
  retried on `EINTR`, so this change has precedent.

  All of the affected stubs raise `Bigstring.IOError` on failure,
  rather than `Unix_error`, which means the normal method for retrying
  on `EINTR` doesn't work.  In particular `Async.Reader` didn't retry
  them, even though it was supposed to.

  Additionally, the documentation for the following functions was
  corrected to say that they raise =Unix_error= rather than =IOError=:

  * `pread_assume_fd_is_nonblocking`
  * `pwrite_assume_fd_is_nonblocking`
- Eliminated global binary-to-decimal tables computed at startup for
  converting `Date` and `Of_day` to string.

  Used an efficient implementation of division by 10, rather than the
  `sprintf` tables in `time_internal.ml`.  This result in much less
  allocation at startup and it is a bit faster:

  * before:

  | Name           | Time/Run | mWd/Run | Percentage |
  |----------------|----------|---------|------------|
  | Date.to_string |  69.39ns |   3.00w |    100.00% |

  - after:

  | Name           | Time/Run | mWd/Run | Percentage |
  |----------------|----------|---------|------------|
  | Date.to_string |  53.38ns |   3.00w |    100.00% |
- Fixed `Time.Zone` tests so that they are deterministic.
- Added `Iobuf.to_string_hum`, which produces a readable, multi-line
  representation of an iobuf, intended for debugging.
- Fixed brittle unit tests of `Command`.

## core_extended

- For `Flang`, added ordering to fields, and added `abs`, `min`, and
  `max` to the language.
- Removed `Loggers` module.

## core_kernel

- Added to `String` functions for substring search and replace, based
  on the KMP algorithm.

  Here are some benchmarks, comparing `Re2` for a fixed pattern,
  Mark's kmp from extended_string, and this implementation ("needle").

  The pattern is the usual `abacabadabacabae...`.  The text looks
  similar, with the pattern occurring at the very end.

  For =Re2= and =Needle= search benchmarks, the pattern is
  preprocessed in advance, outside of the benchmark.

  FWIW: I've also tried searches with pattern size = 32767, but =Re2=
  blows up, saying:

  ```
  re2/dfa.cc:447: DFA out of memory: prog size 32771 mem 2664898
  ```

  | Name                          |        Time/Run |       mWd/Run |
 mjWd/Run | Prom/Run | Percentage |

|-------------------------------|-----------------|---------------|-------------|----------|------------|
  | create_needle_15              |        102.56ns |        21.00w |
      |          |            |
  | re2_compile_15                |      6_261.48ns |               |
3.00w |          |      0.01% |
  | create_needle_1023            |     13_870.48ns |         5.00w |
1_024.01w |          |      0.03% |
  | re2_compile_1023              |    107_533.32ns |               |
3.03w |          |      0.24% |
  | create_needle_8191            |     90_107.02ns |         5.00w |
8_192.01w |          |      0.20% |
  | re2_compile_8191              |  1_059_873.47ns |               |
3.28w |    0.28w |      2.37% |
  | create_needle_524287          |  6_430_623.96ns |         5.00w |
524_288.09w |          |     14.35% |
  | re2_compile_524287            | 44_799_605.83ns |               |
3.77w |    0.77w |    100.00% |
  | needle_search_15_95           |        349.65ns |         4.00w |
      |          |            |
  | re2_search_15_95              |        483.11ns |               |
      |          |            |
  | mshinwell_search_15_95        |      1_151.38ns |       781.01w |
      |          |            |
  | needle_search_15_815          |      2_838.85ns |         4.00w |
      |          |            |
  | re2_search_15_815             |      3_293.06ns |               |
      |          |            |
  | mshinwell_search_15_815       |      8_360.57ns |     5_821.07w |
0.55w |    0.55w |      0.02% |
  | needle_search_15_2415         |      8_395.84ns |         4.00w |
      |          |      0.02% |
  | re2_search_15_2415            |      9_594.14ns |               |
      |          |      0.02% |
  | mshinwell_search_15_2415      |     24_602.09ns |    17_021.16w |
1.62w |    1.62w |      0.05% |
  | needle_search_1023_6143       |     14_825.50ns |         4.00w |
      |          |      0.03% |
  | re2_search_1023_6143          |     40_926.59ns |               |
      |          |      0.09% |
  | mshinwell_search_1023_6143    |     81_930.46ns |    49_149.66w |
1_025.65w |    1.65w |      0.18% |
  | needle_search_1023_52223      |    126_465.96ns |         4.00w |
      |          |      0.28% |
  | re2_search_1023_52223         |    365_359.98ns |               |
      |          |      0.82% |
  | mshinwell_search_1023_52223   |    527_323.73ns |   371_715.39w |
1_033.17w |    9.17w |      1.18% |
  | needle_search_1023_154623     |    377_539.53ns |         4.00w |
      |          |      0.84% |
  | re2_search_1023_154623        |  1_001_251.93ns |               |
      |          |      2.23% |
  | mshinwell_search_1023_154623  |  1_499_835.01ns | 1_088_518.15w |
1_033.19w |    9.19w |      3.35% |
  | needle_search_8191_49151      |    115_223.31ns |         4.00w |
      |          |      0.26% |
  | re2_search_8191_49151         |    559_487.38ns |               |
      |          |      1.25% |
  | mshinwell_search_8191_49151   |    653_981.19ns |   393_219.50w |
8_201.01w |    9.01w |      1.46% |
  | needle_search_8191_417791     |    976_725.24ns |         4.00w |
      |          |      2.18% |
  | re2_search_8191_417791        |  4_713_965.69ns |               |
      |          |     10.52% |
  | mshinwell_search_8191_417791  |  4_224_417.93ns | 2_973_709.32w |
8_202.37w |   10.37w |      9.43% |
  | needle_search_8191_1236991    |  2_912_863.78ns |         4.00w |
      |          |      6.50% |
  | re2_search_8191_1236991       | 14_039_230.59ns |               |
      |          |     31.34% |
  | mshinwell_search_8191_1236991 | 11_997_713.73ns | 8_708_130.87w |
8_202.47w |   10.47w |     26.78% |
- Added to `Set` functions for converting to and from a `Map.t`.

  ```ocaml
  val to_map : ('key, 'cmp) t -> f:('key -> 'data) -> ('key, 'data, 'cmp)
Map.t
  val of_map_keys : ('key, _, 'cmp) Map.t -> ('key, 'cmp) t
  ```

  This required adding some additional type trickery to
  `Core_set_intf` to indicate that the comparator for a given module
  may or may not be fixed.
- Added an optional `iter` parameter to `Container.Make`.

  A direct implementation of `iter` is often more efficient than
  defining `iter` in terms of `fold`, and in these cases, the results
  of `Container.Make` that are defined in terms of `iter` will be more
  efficient also.
- Added `Int.pow` (and for other integer types), for bounds-checked
  integer exponentiation.

## ocaml_plugin

- Added a tag to exceptions coming from the toplevel execution of
  plugins so that we do not confuse them with exceptions coming from
  the library.

  Also, added a function to check a plugin without executing it.  And
  captured the common pattern of checking the compilation of a plugin
  in a `Command.t` offered in the library.

## sexplib

- Added error locations to `Macro`-expansion errors.

-- 
Jeremie Dimino, for the Core team

[-- Attachment #2: Type: text/html, Size: 13285 bytes --]