OCaml 5.0 --------- ### Language features: ### Runtime system: - #11308: Add environment variable to preserve runtime_events after exit If the environment variable OCAML_RUNTIME_EVENTS_PRESERVE exists then the runtime will not remove the runtime events ring buffers at exit. This makes tracing very short running programs more reliable. (Sadiq Jaffer, review by KC Sivaramakrishnan) - #10964: Ring-buffer based runtime tracing (runtime_events) Runtime_events is a very low overhead runtime tracing system designed for continuous monitoring of OCaml applications. (Sadiq Jaffer, review by Anil Madhavapeddy, Enguerrand Decorne, Richard Warburton, Gabriel Scherer, Sabine Schmaltz, Florian Angeletti, Patrick Ferris, Tom Kelly) - #10831: Multicore OCaml (The multicore team, caml-devel and more) * #10723: do not use `-flat-namespace` linking for macOS. (Carlo Cabrera, review by Damien Doligez) * #10863, #10933: Remove support for old, unprefixed C runtime function names such as `alloc`. The new names prefixed with `caml_` must be used instead, such as `caml_alloc`. Consequently, it is no longer needed to define `CAML_NAME_SPACE` to avoid bringing unprefixed names into scope: this is now the default behavior. (Nicolás Ojeda Bär, review by Xavier Leroy) - #10902: Do not register empty code fragments in natdynlink. (David Allsopp, review by Xavier Leroy and Damien Doligez) - #10965: `caml_fatal_error_hook`, GC timing hooks, and `caml_scan_roots_hook` are now atomic variables. Restore GC timing hooks in multicore. (Guillaume Munch-Maccagnoni, review by Enguerrand Decorne, Xavier Leroy, Gabriel Scherer, and KC Sivaramakrishnan) - #11209: Add a public and thread-safe timing hook running at domain termination, after this domain has stopped running any OCaml code: `caml_domain_terminated_hook`. This can be useful for implementing domain-local state in C. (Guillaume Munch-Maccagnoni, review by Xavier Leroy and Gabriel Scherer) - #10875: Add option to allocate stacks with mmap(MAP_STACK) instead of malloc. This is exposed via a configure --enable-mmap-map-stack option, and is enabled by default on OpenBSD where it is mandatory. (Anil Madhavapeddy, review by Gabriel Scherer, Tom Kelly, Michael Hendricks and KC Sivaramakrishnan). - #10950: Do not use mmap to allocate Caml_state. In order to reduce virtual memory usage, we dynamically allocate the domain_state structure. (Enguerrand Decorne, KC Sivaramakrishnan and Tom Kelly, review by Anil Madhavapeddy and Gabriel Scherer) - #11010: Use strerror_r for reentrant error string conversion. (Anil Madhavapeddy and Xavier Leroy, review by same and Edwin Török) - #11002, #11066, #11086: Do not use Begin_roots/End_roots macros in the runtime system. Also fix a missing root registration in the implementation of Unix.write on Windows. (Nicolás Ojeda Bär, Daniel Bünzli and Antonin Décimo, review by Xavier Leroy and David Allsopp) - #11022: Track GC work for all managed bigarray allocations (Stephen Dolan, report by Andrew Hunter, review by Damien Doligez) - #10802: Use 4.12 value macros and helpers in C code (Antonin Décimo, review by Gabriel Scherer) - #11105: Fix handling of fiber stack cache with multiple domains (Jon Ludlam, KC Sivaramakrishnan and Tom Kelly) - #11054: Respect user provided maximum stack space Make sure the stack we initially request is sized accordingly to the user provided settings. tmc/stack_space is also updated by this PR in order to account for this change. (Enguerrand Decorne, report by Jon Ludlam, review by Tom Kelly, KC Sivaramakrishnan and Gabriel Scherer) - #11238: Increase the default limit on the stack size to 128 Mi words, i.e. 1 Gib for 64-bit platforms and 512 Mib for 32-bit platforms. (Xavier Leroy, review by Sébastien Hinderer) * #11295: An ISO C 2011 compliant compiler, including full support for atomic types, is now required to build the OCaml runtime system. (Xavier Leroy, review by David Allsopp and Sébastien Hinderer) - #10915, #11039, #11057, #11095, #11190: Implement quality treatment for asynchronous actions in multicore. Reimplement the old behaviour of `caml_process_pending*` for multicore. (Guillaume Munch-Maccagnoni, review by Sadiq Jaffer and Gabriel Scherer) ### Code generation and optimizations: - #10972: ARM64 multicore support: OCaml & C stack separation; dynamic stack size checks; fiber and effects support. (Tom Kelly and Xavier Leroy, review by KC Sivaramakrishnan, Xavier Leroy Guillaume Munch-Maccagnoni, Eduardo Rafael, Stephen Dolan and Gabriel Scherer) * #10845 Emit frametable size on amd64 BSD (OpenBSD, FreeBSD, NetBSD) systems (emitted for Linux in #8805) (Hannes Mehnert, review by Nicolás Ojeda Bär) ### Standard library: - #10742: Use LXM as the pseudo-random number generator for module Random. Add `Random.State.split` and `Random.split` to "split" a PRNG off another PRNG. (Xavier Leroy, review by Gabriel Scherer and Hugo Heuzard) * #10867: Remove deprecated values: Array.create, Array.make_float, Array.create_matrix, Bytes.uppercase, Bytes.lowercase, Bytes.capitalize, Bytes.uncapitalize, Char.lowercase, Char.uppercase, Filename.temp_dir_name, Int32.format, Int64.format, Nativeint.format, Format.bprintf, Format.kprintf, Format.set_all_formatter_output_functions, Format.get_all_formatter_output_functions, Format.pp_set_all_formatter_output_functions, Format.pp_get_all_formatter_output_functions, Format.pp_open_tag, Format.pp_close_tag, Format.open_tag, Format.close_tag, Format.formatter_tag_functions, Format.pp_set_formatter_tag_functions, Format.pp_get_formatter_tag_functions, Format.set_formatter_tag_functions, Format.get_formatter_tag_functions, Gc (mutability of the fields of type Gc.control), Lazy.lazy_from_fun, Lazy.lazy_from_val, Lazy.lazy_is_val, Obj.set_tag, Obj.truncate, Obj.final_tag, Obj.extension_constructor, Obj.extension_name, Obj.extension_id, Scanf.stdib, Scanf.fscanf, Scanf.kfscanf, Stdlib.( & ), Stdlib.( or ), String.set, String.copy, String.fill, String.unsafe_set, String.unsafe_fill, String.uppercase, String.lowercase, String.capitalize, String.uncapitalize, Thread.kill, Thread.wait_write, Thread.wait_read, the whole ThreadUnix module. (Nicolás Ojeda Bär, review by Damien Doligez) * #10896: Remove Stream, Genlex and Pervasives. Also remove legacy standalone bigarray library (the Bigarray module is now part of the standard library). (Nicolás Ojeda Bär, review by Kate Deplaix and Anil Madhavapeddy) - #10921: Use getentropy, when available, to seed the generator during `Random.self_init` (Michael Hendricks, review by Gabriel Scherer, Xavier Leroy, and Anil Madhavapeddy) * #10924: Add deprecated attribute to Printexc.catch, Printf.kprintf and Unix.SO_ERROR. (Nicolás Ojeda Bär, review by Damien Doligez) - #10986: Add Scanf.sscanf_opt, Scanf.bscanf_opt and Scanf.scanf_opt. (Nicolás Ojeda Bär, review by Florian Angeletti and Gabriel Scherer) * #11157: Rename "hash" in the "Hashtbl.SeededHashedType" signature to "seeded_hash". This allows defining both seeded and unseeded hash functions in the same module. (Nicolás Ojeda Bär, review by Gabriel Scherer and Xavier Leroy) - #8878: Add String.hash and String.seeded_hash. (Tom Kelly, review by Alain Frisch and Nicolás Ojeda Bär) ### Other libraries: * #9071, #9100, #10935: Reimplement `Thread.exit()` as raising the exception `Thread.Exit`, and mark `Thread.exit` as deprecated. The new implementation changes the behavior compared with the OCaml 4 implementation. For example, the new implementation causes `Fun.finally` finalizers to be run and catch-all exception handlers to prevent termination. (Jacques-Henri Jourdan and Xavier Leroy, review by Damien Doligez, Guillaume Munch-Maccagnoni, Gabriel Scherer, and Enguerrand Decorne) - #11034: Dynlink library, add a global lock to make dynlinking thread-safe. (Florian Angeletti, review by Gabriel Scherer) - #11087: deprecate Thread.wait_timed_read, Thread.wait_timed_write, Thread.select, Thread.wait_pid. The same functionality is available in the Unix module. (Nicolás Ojeda Bär, review by Anil Madhavapeddy) ### Tools: * #11004: Litmus tests for checking the implementation of the memory model. (Luc Maranget, review by Gabriel Scherer and Xavier Leroy) - #11024: Handle alerts in ocamldoc. The alert `[@@alert deprecated]` is handled specifically and it's no longer needed to duplicate the deprecated annotation (the alert for the compiler and the tag for the documentation). Every other alerts also appear in the documentation. (Jules Aguillon, review by Florian Angeletti) - #11079: Add the -nobanners option to dumpobj. (Sébastien Hinderer, review by Gabriel Scherer and Vincent Laviron) - #11100: Fix ocamltest to make sure failed tests are not counted as "unexpected error". (Damien Doligez, review by Sébastien Hinderer) - #11245: Merge the common code of ocamlcp and ocamloptp into a single module. (David Allsopp, review by Sébastien Hinderer) ### Manual and documentation: - #11058: runtime/HACKING.adoc tips on debugging the runtime (Gabriel Scherer, review by Enguerrand Decorne and Nicolás Ojeda Bär) - #11093: Add an effect handlers tutorial (KC Sivaramakrishnan, review by François Pottier, Gabriel Scherer, François Bobot and Wiktor Kuchta) - #11192: Better documentation for condition variables. (François Pottier, review by Luc Maranget, Xavier Leroy, and Wiktor Kuchta) ### Compiler user-interface and warnings: - #9140, #11131: New command-line flag -nocwd to not include implicit the current directory to the load path. (Thomas Roglin, review by Gabriel Scherer and Nicolás Ojeda Bär) - #11089: Add 'since ' information to compiler warnings. (André Maroneze, review by Florian Angeletti and Gabriel Scherer) - #10909: Disable warning 59 (assignment to immutable blocks) unless flambda invariant checks are enabled. (Vincent Laviron, review by Gabriel Scherer) - #10981, #11276: Implement a -cmi-file option for ocamlc and ocamlopt. (Sébastien Hinderer, review by Damien Doligez, Daniel Bünzli and Florian Angeletti) * #11049: Stop padding 1-digit compiler minor version numbers. (So for instance OCaml 5.0 rather than 5.00) (Sébastien Hinderer, review by David Allsopp, Florian Angeletti and Xavier Leroy) - #11184: Stop calling ranlib on created / installed libraries (Sébastien Hinderer, review by Xavier Leroy) - #11253: Deprecate `ocaml script` and `ocamlnat` script where `script` has no extension and is an implicit basename. (David Allsopp, review by Florian Angeletti and Sébastien Hinderer) ### Internal/compiler-libs changes: - #10878, #10909: restore flambda after the Multicore merge. (Vincent Laviron, review by Gabriel Scherer and Xavier Leroy) - #10864, #10888: restore afl-fuzz mode for sequential programs. (Jan Midtgaard, review by Xavier Leroy and Gabriel Scherer) - #11008, #11047: rework GC statistics in the Multicore runtime (Gabriel Scherer, review by Enguerrand Decorne) - #11058: basic debugging documentation in runtime/HACKING.adoc (Gabriel Scherer, review by ???) - #11199: Stop installing topdirs.cmi twice. The toplevel now reads topdirs.cmi from +compiler-libs, as the debugger does. (David Allsopp, review by Sébastien Hinderer) ### Build system: * #10893: Remove configuration options --disable-force-safe-string and DEFAULT_STRING=unsafe as well as --enable-force-safe-string and DEFAULT_STRING=safe which are now the default unconditionally. (Kate Deplaix, review by Gabriel Scherer and David Allsopp) - #11092: Build native-code compilers on OpenBSD/aarch64. (Christopher Zimmermann, review by Anil Madhavapeddy) - #11126: Build system: make it possible to choose which ocamldep (and flags) to use when computing depencies for the compiler. Add a -no-slash option to ocamldep to let users override -slash. (Sébastien Hinderer, review by David Allsopp) - #11147: Factorize the stdlib-related compilation flags. Make it possible to control them consistently through the STDLIBFLAGS build variable. Make sure ocamldoc and ocamllex get compiled and linked with debugging informations (-g). (Sébastien Hinderer, review by Gabriel Scherer) - #11149: Make the bootstrap process reproducible on systems with non-big-endian floating point. If the boot/ artefacts are up-to-date, this means that running make bootstrap on any platform should not change the images in boot/ and paves the way for automated testing that the bootstrap is repeatable. (David Allsopp, review by Damien Doligez and Sébastien Hinderer) - #11160: otherlibs: merge win32unix into unix. (Sébastien Hinderer, review by David Allsopp, Nicolás Ojeda Bär, Xavier Leroy, Vincent Laviron and Antonin Décimo) * #11198, #11298: Install the Dynlink, Str and Unix libraries to individual subdirectories of LIBDIR. The compiler, debugger and toplevel automatically add `-I +lib` if required, but display an alert. (David Allsopp, review by Florian Angeletti, Nicolás Ojeda Bär, Valentin Gatien-Baron and Sébastien Hinderer) - #11200: Install ocamlprof's Profiling runtime module to a +profiling, removing it from the default namespace. (David Allsopp, review by Sébastien Hinderer) ### Bug fixes: - #11167: Fix memory leak from signal stack. (Antoni Żewierżejew, review by Gabriel Scherer and Enguerrand Decorne) - #11112: harden -use-runtime against spaces or quotes in the provided path (Gabriel Scherer, report by Brahima Dibassi, review by David Allsopp) - #11068, #11070: Fix typo in function name given in Unix_error exception for Unix.readlink on Windows. (David Allsopp, report by Xia Li-yao) - #10807: Don't duplicate standard handles in the child process spawned by win32unix Unix.create_process if the handles were already inheritable. Fix broken signalling of EOF on standard handles if they were already inheritable. (Antonin Décimo, review by Xavier Leroy and Nicolás Ojeda Bär) - #10868: Fix off-by-1 bug when initializing frame hashtables (Jonah Beckford, review by Tom Kelly, Nicolás Ojeda Bär and KC Sivaramakrishnan) - #11077: Make dumpobj compatible with absence of naked pointer support (Olivier Nicole and Jan Midtgaard, review by Gabriel Scherer) - #11111: fix fork() usage in ocamltest C code. When calling fork() from C code with the Multicore runtime active, one needs to call caml_atfork_hook() on the forked child before it can use the OCaml runtime. (Gabriel Scherer, review by Xavier Leroy, report by Brahima Dibassi) - #10809: Use the WSA_FLAG_NO_HANDLE_INHERIT on Windows when creating sockets with WSASocket if the cloexec (non-inheritable) parameter is true. Fixes a race condition where a child process could inherit the socket and deadlock the parent. (Antonin Décimo, review by Xavier Leroy) - #11204: Fix regression introduced in 4.14.0 that would trigger Warning 17 when calling virtual methods introduced by constraining the self type from within the class definition. (Nicolás Ojeda Bär, review by Leo White) - #11263, #11267: caml/misc.h: check whether `_MSC_VER` is defined before using it to ensure that the headers can always be used in code which turns on -Wundef (or equivalent). (David Allsopp and Nicolás Ojeda Bär, review by Nicolás Ojeda Bär and Sebastien Hinderer) OCaml 4.14.0 ---------------- ### Language features: - #10462: Add attribute to produce a compiler error for polls. (Sadiq Jaffer, review by Mark Shinwell, Stephen Dolan and Guillaume Munch-Maccagnoni) - #10437: Allow explicit binders for type variables. (Stephen Dolan, review by Leo White) - #10441: Remove unnecessary parentheses surrounding immediate objects. Allow 'object ... end # f', 'f object ... end', etc. (Yan Dong, review by Nicolás Ojeda Bär, Florian Angeletti and Gabriel Scherer) - #181, #9760, #10740: opt-in tail-modulo-cons (TMC) transformation let[@tail_mod_cons] rec map f li = ... (Frédéric Bour, Gabriel Scherer, Basile Clément, review by Basile Clément and Pierre Chambart, tested by Konstantin Romanov) ### Runtime system: * #9391, #9424: Fix failed assertion in runtime due to ephemerons *set_* and *blit_* function during Mark phase (François Bobot, reported by Stephen Dolan, reviewed by Damien Doligez) - #10195, #10680: Speed up GC by prefetching during marking (Stephen Dolan, review by Xavier Leroy, Guillaume Munch-Maccagnoni, Jacques-Henri Jourdan, Damien Doligez and Leo White) - #10549: Stack overflow detection and naked pointers checking for ARM64 (Xavier Leroy, review by Stephen Dolan) * #10675: Emit deprecation warnings when old C runtime function names are used. This will break C stub code that uses these old names and treats warnings as errors. The workaround is to use the new names. (Xavier Leroy and David Allsopp, review by Sébastien Hinderer and Damien Doligez) - #10698, #10726: Free the alternate signal stack when the main OCaml code or an OCaml thread stops (Xavier Leroy, review by David Allsopp and Damien Doligez) - #10730, 10731: Fix bug in `Obj.reachable_words` causing a slowdown when called multiple time (Alain Frisch, report by ygrek, review by Xavier Leroy) - #10926: Rename the two internal Windows Unicode functions with `caml_` prefix instead of `win_`. (David Allsopp, review by Kate Deplaix, Damien Doligez and Xavier Leroy) ### Code generation and optimizations: - #10578: Increase the number of integer registers used for parameter passing on PowerPC (16 registers) and on s390x (8 registers). (Xavier Leroy, review by Mark Shinwell) - #10591, #10615: Tune the heuristic for CSE of integer constants so as to avoid excessive CSE on compiler-generated constants and long register allocation times. (Xavier Leroy, report by Edwin Török, review by Nicolás Ojeda Bär) - #10595: Tail calls with up to 64 arguments are guaranteed to be compiled as tail calls. To this end, memory locations in the domain state are used for passing arguments that do not fit in registers. (Xavier Leroy, review by Vincent Laviron) - #10681: Enforce boolean conditions for the native backend (Vincent Laviron, review by Gabriel Scherer) - #10719: Ensure that build_apply respects Lambda.max_arity (Stephen Dolan, review by Xavier Leroy) - #10728: Ensure that functions are evaluated after their arguments (Stephen Dolan, review by Mark Shinwell) - #10732: Ensure right-to-left evaluation of arguments in cmm_helpers (Greta Yorsh, review by Xavier Leroy) ### Standard library: * #10710: Add UTF tools, codecs and validations to the Uchar, Bytes and String modules. (Daniel Bünzli, review by Florian Angeletti, Nicolás Ojeda Bär, Alain Frisch and Gabriel Scherer) * #10622: Annotate `Uchar.t` with immediate attribute (Hongbo Zhang, reivew by Gabriel Scherer and Nicolás Ojeda Bär) * #7812, #10475: `Filename.chop_suffix name suff` now checks that `suff` is actually a suffix of `name` and raises Invalid_argument otherwise. (Xavier Leroy, report by whitequark, review by David Allsopp) * #10482: mark the Stream and Genlex modules as deprecated, in preparation for a future removal. These modules (without deprecation alert) are now provided by the camlp-streams library. (Xavier Leroy, review by Nicolás Ojeda Bär) - #10526: add Random.bits32, Random.bits64, Random.nativebits (Xavier Leroy, review by Gabriel Scherer and François Bobot) * #10568: remove Obj.marshal and Obj.unmarshal (these functions have been deprecated for a while and are superseded by the functions from module Marshal) (François Pottier, review by Gabriel Scherer and Kate Deplaix) - #10545: Add In_channel and Out_channel modules. (Nicolás Ojeda Bär, review by Daniel Bünzli, Simon Cruanes, Gabriel Scherer, Guillaume Munch-Maccagnoni, Alain Frisch and Xavier Leroy) - #10538: add Out_channel.set_buffered and Out_channel.is_buffered to control the buffering mode of output channels. (Nicolás Ojeda Bär, review by John Whitington, Daniel Bünzli, David Allsopp and Xavier Leroy) * #10583, #10998: Add over 40 new functions in Seq. (François Pottier and Simon Cruanes, review by Nicolás Ojeda Bär, Daniel Bünzli, Naëla Courant, Craig Ferguson, Wiktor Kuchta, Xavier Leroy, Guillaume Munch-Maccagnoni, Raphaël Proust, Gabriel Scherer and Thierry Martinez) - #10596, #10978: Add with_open_bin, with_open_text and with_open_gen to In_channel and Out_channel. Also, add In_channel.input_all. (Nicolás Ojeda Bär, review by Daniel Bünzli, Jérémie Dimino, Damien Doligez and Xavier Leroy) - #10658: add detailed information about the current version of OCaml to the Sys module of the standard library. (Sébastien Hinderer, review by Damien Doligez, Gabriel Scherer, David Allsopp, Nicolás Ojeda Bär, Vincent Laviron) - #10642: On Windows, Sys.remove and Unix.unlink now remove symlinks to directories instead of raising EACCES. Introduce caml/winsupport.h to hold more common code between the runtime, lib-sys, and win32unix. (Antonin Décimo, review by David Allsopp and Xavier Leroy) - #10737: add new ephemeron API for forward compatibility with Multicore OCaml. (Damien Doligez, review by Stephen Dolan) - #10786: The implementation of Complex.norm now uses Float.hypot. (Christophe Troestler, review by David Allsopp and Xavier Leroy) ### Other libraries: - #10192: Add support for Unix domain sockets on Windows and use them to emulate Unix.socketpair (only available on Windows 1803+) (Antonin Décimo, review by David Allsopp) - #10469: Add Thread.set_uncaught_exception_handler and Thread.default_uncaught_exception_handler. (Enguerrand Decorne, review by David Allsopp) - #10697: Bindings of dup and dup2 in win32unix now correctly call WSADuplicateSocket on sockets instead of DuplicateHandle. (Antonin Décimo, review by Xavier Leroy and Nicolás Ojeda Bär) * #10926: Ensure all C functions in the Unix library are prefixed with `caml_`. (David Allsopp, review by Kate Deplaix, Damien Doligez and Xavier Leroy) - #10951: Introduce the Thread.Exit exception as an alternative way to terminate threads prematurely. This alternative way will become the standard way in 5.0. (Xavier Leroy, review by Florian Angeletti) ### Tools: - #9701: Release bytecode only after collecting backtrace information for exceptions, same for dynamic loaded code compiled from toplevel on ocamlnat. (Renato Alencar, reported by Krzysztof Leśniak, reviewed by Gabriel Scherer) - #10839: Fix regression of #show when printing class type (Élie Brami, review by Florian Angeletti) - #3959, #7202, #10476: ocaml, in script mode, directive errors (`#use "missing_file";;`) use stderr and exit with an error. (Florian Angeletti, review by Gabriel Scherer) - #10438: add a new toplevel cli argument `-e