summaryrefslogtreecommitdiff
path: root/app/controllers
diff options
context:
space:
mode:
authorBob Van Landuyt <bob@vanlanduyt.co>2017-11-13 16:52:07 +0100
committerBob Van Landuyt <bob@vanlanduyt.co>2017-12-08 09:11:39 +0100
commitf1ae1e39ce6b7578c5697c977bc3b52b119301ab (patch)
tree1d01033287e4e15e505c7b8b3f69ced4e6cf21c8 /app/controllers
parent12d33b883adda7093f0f4b838532871036af3925 (diff)
downloadgitlab-ce-f1ae1e39ce6b7578c5697c977bc3b52b119301ab.tar.gz
Move the circuitbreaker check out in a separate processbvl-circuitbreaker-process
Moving the check out of the general requests, makes sure we don't have any slowdown in the regular requests. To keep the process performing this checks small, the check is still performed inside a unicorn. But that is called from a process running on the same server. Because the checks are now done outside normal request, we can have a simpler failure strategy: The check is now performed in the background every `circuitbreaker_check_interval`. Failures are logged in redis. The failures are reset when the check succeeds. Per check we will try `circuitbreaker_access_retries` times within `circuitbreaker_storage_timeout` seconds. When the number of failures exceeds `circuitbreaker_failure_count_threshold`, we will block access to the storage. After `failure_reset_time` of no checks, we will clear the stored failures. This could happen when the process that performs the checks is not running.
Diffstat (limited to 'app/controllers')
-rw-r--r--app/controllers/admin/health_check_controller.rb2
-rw-r--r--app/controllers/health_controller.rb11
2 files changed, 11 insertions, 2 deletions
diff --git a/app/controllers/admin/health_check_controller.rb b/app/controllers/admin/health_check_controller.rb
index 65a17828feb..61247b280b3 100644
--- a/app/controllers/admin/health_check_controller.rb
+++ b/app/controllers/admin/health_check_controller.rb
@@ -5,7 +5,7 @@ class Admin::HealthCheckController < Admin::ApplicationController
end
def reset_storage_health
- Gitlab::Git::Storage::CircuitBreaker.reset_all!
+ Gitlab::Git::Storage::FailureInfo.reset_all!
redirect_to admin_health_check_path,
notice: _('Git storage health information has been reset')
end
diff --git a/app/controllers/health_controller.rb b/app/controllers/health_controller.rb
index 98c2aaa3526..a931b456a93 100644
--- a/app/controllers/health_controller.rb
+++ b/app/controllers/health_controller.rb
@@ -1,5 +1,5 @@
class HealthController < ActionController::Base
- protect_from_forgery with: :exception
+ protect_from_forgery with: :exception, except: :storage_check
include RequiresWhitelistedMonitoringClient
CHECKS = [
@@ -23,6 +23,15 @@ class HealthController < ActionController::Base
render_check_results(results)
end
+ def storage_check
+ results = Gitlab::Git::Storage::Checker.check_all
+
+ render json: {
+ check_interval: Gitlab::CurrentSettings.current_application_settings.circuitbreaker_check_interval,
+ results: results
+ }
+ end
+
private
def render_check_results(results)