New API to turn on manual VDB at runtime

Manual VDB is now enabled by GC_set_manual_vdb_allowed(1) if called
before entering the GC incremental mode.

* alloc.c (GC_allocobj): Expand TRUE_INCREMENTAL macro.
* darwin_stop_world.c [MPROTECT_VDB] (GC_stop_world, GC_start_world):
Use GC_auto_incremental instead of GC_incremental.
* mark.c (alloc_mark_stack): Likewise.
* mark.c [PROC_VDB] (GC_push_all): Likewise.
* mark.c [!NEED_FIXUP_POINTER && THREADS && MPROTECT_VDB]
(GC_push_all_stack): Likewise.
* pthread_support.c [CAN_HANDLE_FORK && GC_DARWIN_THREADS
&& MPROTECT_VDB] (GC_atfork_prepare): Likewise.
* win32_threads.c [MPROTECT_VDB && !CYGWIN32]
(GC_register_my_thread_inner): Likewise.
* win32_threads.c [MPROTECT_VDB] (UNPROTECT_THREAD): Likewise.
* doc/gcdescr.md (Generational Collection and Dirty Bits): Update
documentation for the manual VDB.
* include/gc.h (GC_end_stubborn_change): Update comment.
* mark_rts.c (GC_push_all_stack_partially_eager): Likewise.
* include/gc.h (GC_set_manual_vdb_allowed, GC_get_manual_vdb_allowed):
New public function.
* include/private/gc_priv.h (GC_grungy_pages, GC_dirty_pages): Define
for all VDB modes.
* include/private/gc_priv.h (GC_auto_incremental, GC_manual_vdb):
Define.
* include/private/gc_priv.h [!GC_DISABLE_INCREMENTAL] (GC_dirty): Use
GC_manual_vdb instead of GC_incremental.
* include/private/gcconfig.h (GWW_VDB, MPROTECT_VDB, PCR_VDB): Do not
undefine if MANUAL_VDB.
* mallocx.c (GC_generic_malloc_many): Always allocate a single object
(and call GC_dirty_inner/REACHABLE_AFTER_DIRTY) if GC_manual_vdb.
* misc.c [!CAN_HANDLE_FORK && DARWIN && MPROTECT_VDB && !THREADS
&& !SMALL_CONFIG] (GC_set_handle_fork): Do not ABORT if GC_manual_vdb.
* misc.c [!SMALL_CONFIG] (manual_vdb_allowed): New static variable.
* misc.c [!SMALL_CONFIG] (GC_set_manual_vdb_allowed,
GC_get_manual_vdb_allowed): Implement.
* misc.c [!CHECKSUMS && !SMALL_CONFIG] (GC_init,
GC_enable_incremental): Set GC_manual_vdb and GC_incremental to true
if manual_vdb_allowed; do not call GC_dirty_init if manual_vdb_allowed.
* os_dep.c: Update comment about MANUAL_VDB.
* os_dep.c [MANUAL_VDB] (GC_dirty_init,
async_set_pht_entry_from_index): Remove.
* os_dep.c [!GC_DISABLE_INCREMENTAL] (GC_manual_vdb): Define global
variable.
* os_dep.c [!GC_DISABLE_INCREMENTAL] (GC_dirty_inner): Define
regardless of the VDB mode; add FIXME.
* os_dep.c [!GC_DISABLE_INCREMENTAL] (GC_read_dirty, GC_page_was_dirty,
GC_remove_protection): Implement for the case of GC_manual_vdb is true;
do not depend on MANUAL_VDB.
* tests/disclaim_test.c [TEST_MANUAL_VDB] (main): Call
GC_set_manual_vdb_allowed(1) before GC_INIT.
* tests/staticrootslib.c [TEST_MANUAL_VDB] (libsrl_init): Likewise.
* tests/test_cpp.cc [TEST_MANUAL_VDB] (main): Likewise.
* tests/test.c (INIT_MANUAL_VDB_ALLOWED): New macro.
* tests/test.c (GC_COND_INIT): Invoke INIT_MANUAL_VDB_ALLOWED (before
GC_OPT_INIT).
* tests/test.c [!SMALL_CONFIG] (main): Call GC_get_manual_vdb_allowed.

1 files changed, 7 insertions, 11 deletions

diff --git a/mallocx.c b/mallocx.c
index a318d5b9..9e93c707 100644
--- a/mallocx.c
+++ b/mallocx.c
@@ -311,21 +311,17 @@ GC_API void GC_CALL GC_generic_malloc_many(size_t lb, int k, void **result)
     DCL_LOCK_STATE;
 
     GC_ASSERT(lb != 0 && (lb & (GRANULE_BYTES-1)) == 0);
-    if (!SMALL_OBJ(lb)
-#     ifdef MANUAL_VDB
-        /* Currently a single object is allocated.                      */
-        /* TODO: GC_dirty should be called for each linked object (but  */
-        /* the last one) to support multiple objects allocation.        */
-        || GC_incremental
-#     endif
-       ) {
+    /* Currently a single object is always allocated if manual VDB. */
+    /* TODO: GC_dirty should be called for each linked object (but  */
+    /* the last one) to support multiple objects allocation.        */
+    if (!SMALL_OBJ(lb) || GC_manual_vdb) {
         op = GC_generic_malloc(lb, k);
         if (EXPECT(0 != op, TRUE))
             obj_link(op) = 0;
         *result = op;
-#       ifdef MANUAL_VDB
-          if (GC_is_heap_ptr(result)) {
-            GC_dirty(result);
+#       ifndef GC_DISABLE_INCREMENTAL
+          if (GC_manual_vdb && GC_is_heap_ptr(result)) {
+            GC_dirty_inner(result);
             REACHABLE_AFTER_DIRTY(op);
           }
 #       endif