From 9e5ea67e268be2659cd30ebaed7044d298198ab0 Mon Sep 17 00:00:00 2001 From: Simon Marlow Date: Sat, 23 Apr 2016 21:14:49 +0100 Subject: NUMA support Summary: The aim here is to reduce the number of remote memory accesses on systems with a NUMA memory architecture, typically multi-socket servers. Linux provides a NUMA API for doing two things: * Allocating memory local to a particular node * Binding a thread to a particular node When given the +RTS --numa flag, the runtime will * Determine the number of NUMA nodes (N) by querying the OS * Assign capabilities to nodes, so cap C is on node C%N * Bind worker threads on a capability to the correct node * Keep a separate free lists in the block layer for each node * Allocate the nursery for a capability from node-local memory * Allocate blocks in the GC from node-local memory For example, using nofib/parallel/queens on a 24-core 2-socket machine: ``` $ ./Main 15 +RTS -N24 -s -A64m Total time 173.960s ( 7.467s elapsed) $ ./Main 15 +RTS -N24 -s -A64m --numa Total time 150.836s ( 6.423s elapsed) ``` The biggest win here is expected to be allocating from node-local memory, so that means programs using a large -A value (as here). According to perf, on this program the number of remote memory accesses were reduced by more than 50% by using `--numa`. Test Plan: * validate * There's a new flag --debug-numa= that pretends to do NUMA without actually making the OS calls, which is useful for testing the code on non-NUMA systems. * TODO: I need to add some unit tests Reviewers: erikd, austin, rwbarton, ezyang, bgamari, hvr, niteria Subscribers: thomie Differential Revision: https://phabricator.haskell.org/D2199 --- rts/eventlog/EventLog.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'rts/eventlog') diff --git a/rts/eventlog/EventLog.c b/rts/eventlog/EventLog.c index 9e2ed33d22..7cb69e8d7d 100644 --- a/rts/eventlog/EventLog.c +++ b/rts/eventlog/EventLog.c @@ -273,7 +273,7 @@ initEventLogging(void) */ #ifdef THREADED_RTS // XXX n_capabilities hasn't been initislised yet - n_caps = RtsFlags.ParFlags.nNodes; + n_caps = RtsFlags.ParFlags.nCapabilities; #else n_caps = 1; #endif -- cgit v1.2.1