[scudo] Correct a behavior on the shared TSD registry

Summary:
There is an error in the shared TSD registry logic when looking for a
TSD in the slow path. There is an unlikely event when a TSD's precedence
was 0 after attempting a `tryLock` which indicated that it was grabbed
by another thread in between. We dealt with that case by continuing to
the next iteration, but that meant that the `Index` was not increased
and we ended up trying to lock the same TSD.
This would manifest in heavy contention, and in the end we would still
lock a TSD, but that was a wasted iteration.
So, do not `continue`, just skip the TSD as a potential candidate.

This is in both the standalone & non-standalone versions.

Reviewers: morehouse, eugenis, vitalybuka, hctim

Reviewed By: morehouse

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63783

git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@364345 91177308-0d34-0410-b5e6-96231b3b80d8

2 files changed, 2 insertions, 6 deletions

diff --git a/lib/scudo/scudo_tsd_shared.cpp b/lib/scudo/scudo_tsd_shared.cpp
index 9918a08be..59ad25499 100644
--- a/lib/scudo/scudo_tsd_shared.cpp
+++ b/lib/scudo/scudo_tsd_shared.cpp
@@ -83,9 +83,7 @@ ScudoTSD *getTSDAndLockSlow(ScudoTSD *TSD) {
       }
       const uptr Precedence = TSDs[Index].getPrecedence();
       // A 0 precedence here means another thread just locked this TSD.
-      if (UNLIKELY(Precedence == 0))
-        continue;
-      if (Precedence < LowestPrecedence) {
+      if (Precedence && Precedence < LowestPrecedence) {
         CandidateTSD = &TSDs[Index];
         LowestPrecedence = Precedence;
       }
diff --git a/lib/scudo/standalone/tsd_shared.h b/lib/scudo/standalone/tsd_shared.h
index 126d74355..48747f69f 100644
--- a/lib/scudo/standalone/tsd_shared.h
+++ b/lib/scudo/standalone/tsd_shared.h
@@ -126,9 +126,7 @@ private:
         }
         const uptr Precedence = TSDs[Index].getPrecedence();
         // A 0 precedence here means another thread just locked this TSD.
-        if (UNLIKELY(Precedence == 0))
-          continue;
-        if (Precedence < LowestPrecedence) {
+        if (Precedence && Precedence < LowestPrecedence) {
           CandidateTSD = &TSDs[Index];
           LowestPrecedence = Precedence;
         }