diff options
| author | Alistair Coles <alistairncoles@gmail.com> | 2022-03-23 18:26:50 +0000 |
|---|---|---|
| committer | Alistair Coles <alistairncoles@gmail.com> | 2022-05-04 11:22:50 +0100 |
| commit | 5227cb702b744bbe9aaecfff002604be45f64a8c (patch) | |
| tree | 602bae1a65d9b331849a91fa8563ffc79398aad5 /swift/container | |
| parent | 954032d5d245411e676a5722b7e581c97a0aaaf7 (diff) | |
| download | swift-5227cb702b744bbe9aaecfff002604be45f64a8c.tar.gz | |
Refactor rate-limiting helper into a class
Replaces the ratelimit_sleep helper function with an
EventletRateLimiter class that encapsulates the rate-limiting state
that previously needed to be maintained by the caller of the function.
The ratelimit_sleep function is retained but deprecated, and now
forwards to the EventletRateLimiter class.
The object updater's BucketizedUpdateSkippingLimiter is refactored to
take advantage of the new EventletRateLimiter class.
The rate limiting algorithm is corrected to make the allowed request
rate more uniform: previously pairs of requests would be allowed in
rapid succession before the rate limiter would the sleep for the time
allowance consumed by those two requests; now the rate limiter will
sleep as required after each allowed request. For example, before a
max_rate of 1 per second might result in 2 requests being allowed
followed by a 2 second sleep. That is corrected to be a sleep of 1
second after each request.
Change-Id: Ibcf4dbeb4332dee7e9e233473d4ceaf75a5a85c7
Diffstat (limited to 'swift/container')
| -rw-r--r-- | swift/container/updater.py | 8 |
1 files changed, 3 insertions, 5 deletions
diff --git a/swift/container/updater.py b/swift/container/updater.py index a22bf0b71..fc7b60eaf 100644 --- a/swift/container/updater.py +++ b/swift/container/updater.py @@ -31,7 +31,7 @@ from swift.common.bufferedhttp import http_connect from swift.common.exceptions import ConnectionTimeout, LockTimeout from swift.common.ring import Ring from swift.common.utils import get_logger, config_true_value, \ - dump_recon_cache, majority_size, Timestamp, ratelimit_sleep, \ + dump_recon_cache, majority_size, Timestamp, EventletRateLimiter, \ eventlet_monkey_patch from swift.common.daemon import Daemon from swift.common.http import is_success, HTTP_INTERNAL_SERVER_ERROR @@ -59,10 +59,10 @@ class ContainerUpdater(Daemon): float(conf.get('slowdown', '0.01')) + 0.01) else: containers_per_second = 50 - self.containers_running_time = 0 self.max_containers_per_second = \ float(conf.get('containers_per_second', containers_per_second)) + self.rate_limiter = EventletRateLimiter(self.max_containers_per_second) self.node_timeout = float(conf.get('node_timeout', 3)) self.conn_timeout = float(conf.get('conn_timeout', 0.5)) self.no_changes = 0 @@ -226,9 +226,7 @@ class ContainerUpdater(Daemon): self.logger.exception( "Error processing container %s: %s", dbfile, e) - self.containers_running_time = ratelimit_sleep( - self.containers_running_time, - self.max_containers_per_second) + self.rate_limiter.wait() def process_container(self, dbfile): """ |