diff options
| author | Alistair Coles <alistairncoles@gmail.com> | 2021-07-15 18:08:01 +0100 |
|---|---|---|
| committer | Alistair Coles <alistairncoles@gmail.com> | 2021-07-19 21:18:02 +0100 |
| commit | bbaed18e9b681ce9cf26ffa6a5d5292f5cb219b7 (patch) | |
| tree | 5306ebb0ce693c5f1d0f532fb10ac273fdb97f32 /doc | |
| parent | dfd020d11cb9b5aad0027f245e49026b91125749 (diff) | |
| download | swift-bbaed18e9b681ce9cf26ffa6a5d5292f5cb219b7.tar.gz | |
diskfile: don't remove recently written non-durables
DiskFileManager will remove any stale files during
cleanup_ondisk_files(): these include tombstones and nondurable EC
data fragments whose timestamps are older than reclaim_age. It can
usually be safely assumed that a non-durable data fragment older than
reclaim_age is not going to become durable. However, if an agent PUTs
objects with specified older X-Timestamps (for example the reconciler
or container-sync) then there is a window of time during which the
object server has written an old non-durable data file but has not yet
committed it to make it durable.
Previously, if another process (for example the reconstructor) called
cleanup_ondisk_files during this window then the non-durable data file
would be removed. The subsequent attempt to commit the data file would
then result in a traceback due to there no longer being a data file to
rename, and of course the data file is lost.
This patch modifies cleanup_ondisk_files to not remove old, otherwise
stale, non-durable data files that were only written to disk in the
preceding 'commit_window' seconds. 'commit_window' is configurable for
the object server and defaults to 60.0 seconds.
Closes-Bug: #1936508
Related-Change: I0d519ebaaade35249fb7b17bd5f419ffdaa616c0
Change-Id: I5f3318a44af64b77a63713e6ff8d0fd3b6144f13
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/manpages/object-server.conf.5 | 4 | ||||
| -rw-r--r-- | doc/source/config/object_server_config.rst | 12 |
2 files changed, 16 insertions, 0 deletions
diff --git a/doc/manpages/object-server.conf.5 b/doc/manpages/object-server.conf.5 index 3402e025d..7150c6c91 100644 --- a/doc/manpages/object-server.conf.5 +++ b/doc/manpages/object-server.conf.5 @@ -149,6 +149,10 @@ The default is 65536. .IP \fBreclaim_age\fR Time elapsed in seconds before an object can be reclaimed. The default is 604800 seconds. +.IP \fBcommit_window\fR +Time in seconds during which a newly written non-durable data file will not be +reclaimed. The value should be greater than zero and much less than +reclaim_age. The default is 60.0 seconds. .IP \fBnice_priority\fR Modify scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least favorable to the process). diff --git a/doc/source/config/object_server_config.rst b/doc/source/config/object_server_config.rst index 6dac54632..8dc5e3127 100644 --- a/doc/source/config/object_server_config.rst +++ b/doc/source/config/object_server_config.rst @@ -136,6 +136,18 @@ reclaim_age 604800 Time elapsed in seconds before the it will result in dark data. This setting should be consistent across all object services. +commit_window 60 Non-durable data files may also + get reclaimed if they are older + than reclaim_age, but not if the + time they were written to disk + (i.e. mtime) is less than + commit_window seconds ago. A + commit_window greater than zero is + strongly recommended to avoid + unintended reclamation of data + files that were about to become + durable; commit_window should be + much less than reclaim_age. nice_priority None Scheduling priority of server processes. Niceness values range from -20 (most favorable to the process) to 19 (least |