summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJosef Ahmad <josef.ahmad@mongodb.com>2023-05-11 06:39:25 +0000
committerEvergreen Agent <no-reply@evergreen.mongodb.com>2023-05-11 07:18:04 +0000
commitc21700aadbcdbbbd79f40713f89cc37c2174e49e (patch)
tree195d9648d465c279b91a60225c97e465b2112de0
parent0e87bdf088907690685f1a18cdc3faa7c6221f42 (diff)
downloadmongo-c21700aadbcdbbbd79f40713f89cc37c2174e49e.tar.gz
SERVER-68260 Arch guide updates for gracefully handling index build errors
-rw-r--r--src/mongo/db/catalog/README.md38
1 files changed, 35 insertions, 3 deletions
diff --git a/src/mongo/db/catalog/README.md b/src/mongo/db/catalog/README.md
index 2160e8a75d9..ac2454e8f56 100644
--- a/src/mongo/db/catalog/README.md
+++ b/src/mongo/db/catalog/README.md
@@ -1135,6 +1135,16 @@ index build does not need to be the same node that decides to commit it.
See [Index Builds in Replicated Environments - MongoDB
Manual](https://docs.mongodb.com/master/core/index-creation/#index-builds-in-replicated-environments).
+Server 7.1 introduces the following improvements:
+
+* Index builds abort immediately after detecting errors other than duplicate key
+violations. Before 7.1, index builds aborted the index build close to
+completion, potentially long after detection.
+* A secondary member can abort a two-phase index build. Before 7.1, a secondary was forced
+to crash instead. See the [Voting for Abort](#voting-for-abort) section.
+* Index builds are cancelled if there isn't enough storage space available. See the
+ [Disk Space](#disk-space) section.
+
### Commit Quorum
The purpose of `commitQuorm` is to ensure secondaries are ready to commit an index build quickly.
@@ -1158,10 +1168,10 @@ data on a collection and performed the first drain of side-writes. Voting is imp
`voteCommitIndexBuild` command, and is persisted as a write to the replicated
`config.system.indexBuilds` collection.
-While waiting for a commit decision, primaries and secondaries continue recieving and applying new
+While waiting for a commit decision, primaries and secondaries continue receiving and applying new
side writes. When a quorum is reached, the current primary, under a collection X lock, will check
-all index constraints. If there are errors, it will replicate an `abortIndexBuild` oplog entry. If
-the index build is successful, it will replicate a `commitIndexBuild` oplog entry.
+the remaining index constraints. If there are errors, it will replicate an `abortIndexBuild` oplog
+entry. If the index build is successful, it will replicate a `commitIndexBuild` oplog entry.
Secondaries that were not included in the commit quorum and receive a `commitIndexBuild` oplog entry
will block replication until their index build is complete.
@@ -1173,6 +1183,28 @@ server command.
See
[IndexBuildsCoordinator::_waitForNextIndexBuildActionAndCommit](https://github.com/mongodb/mongo/blob/r4.4.0-rc9/src/mongo/db/index_builds_coordinator_mongod.cpp#L632).
+### Voting for Abort
+
+As of 7.1, a secondary can abort a two-phase index build by sending a `voteAbortIndexBuild` signal
+to the primary. In contrast, before 7.1 it was forced to crash. Common causes for aborting the index
+build are a killOp on the index build or running low on storage space.
+The primary, upon receiving a vote to abort the index build from a secondary, will replicate an
+`abortIndexBuild` oplog entry. This will cause all secondaries to gracefully abort the index build,
+even if a specific secondary had already voted to commit the index build.
+
+Note that once a secondary has voted to commit the index build, it cannot retract the vote. In the
+unlikely event that a secondary has voted for commit and for some reason it must abort while waiting
+for the primary to replicate a `commitIndexBuild` oplog entry, the secondary is forced to crash.
+
+### Disk Space
+
+As of 7.1, an index build can abort due to a replica set member running low on disk space. This
+applies both to primary and secondary nodes. Additionally, on a primary the index build won't start
+if the available disk space is low.
+The minimum amount of disk space is controlled by
+[indexBuildMinAvailableDiskSpaceMB](https://github.com/mongodb/mongo/blob/406e69f6f5dee8b698c4e4308de2e9e5cef6c12c/src/mongo/db/storage/two_phase_index_build_knobs.idl#L71)
+which defaults to 500MB.
+
## Resumable Index Builds
On clean shutdown, index builds save their progress in internal idents that will be used for resuming