summaryrefslogtreecommitdiff
path: root/config/initializers
diff options
context:
space:
mode:
authorSean McGivern <sean@gitlab.com>2017-10-13 12:22:38 +0100
committerSean McGivern <sean@gitlab.com>2017-10-16 10:31:20 +0100
commitf4e94c6ff7b2b5e96edb56d30737687becc4a1a8 (patch)
treebde6d5e7dc650aceef551b67331f55a600d0b750 /config/initializers
parent5843a43c16e007193f5e26522d1e7368a0bdb2d7 (diff)
downloadgitlab-ce-f4e94c6ff7b2b5e96edb56d30737687becc4a1a8.tar.gz
Make usage ping scheduling more robust39017-gitlabusagepingworker-is-not-running-on-gitlab-com
On GitLab.com, we see that cron jobs sometimes don't run. This appears to be because the process that polls for current cron jobs only has a 60 second validity period, so if (for whatever reason) it misses those 60 seconds, we have to wait until the next execution. For the usage ping, this is particularly problematic, as that's only scheduled to run once a week. Changing it to run for every minute in a 10 minute period should work around this, if the above diagnosis is correct. The job itself obtains an exclusive lease for 24 hours, so rescheduling in quick succession is safe.
Diffstat (limited to 'config/initializers')
-rw-r--r--config/initializers/1_settings.rb12
1 files changed, 7 insertions, 5 deletions
diff --git a/config/initializers/1_settings.rb b/config/initializers/1_settings.rb
index a4b7c1a3919..b790df565c6 100644
--- a/config/initializers/1_settings.rb
+++ b/config/initializers/1_settings.rb
@@ -113,12 +113,14 @@ class Settings < Settingslogic
URI.parse(url_without_path).host
end
- # Random cron time every Sunday to load balance usage pings
- def cron_random_weekly_time
+ # Runs every minute in a random ten-minute period on Sundays, to balance the
+ # load on the server receiving these pings. The usage ping is safe to run
+ # multiple times because of a 24 hour exclusive lock.
+ def cron_for_usage_ping
hour = rand(24)
- minute = rand(60)
+ minute = rand(6)
- "#{minute} #{hour} * * 0"
+ "#{minute}0-#{minute}9 #{hour} * * 0"
end
end
end
@@ -398,7 +400,7 @@ Settings.cron_jobs['stuck_import_jobs_worker'] ||= Settingslogic.new({})
Settings.cron_jobs['stuck_import_jobs_worker']['cron'] ||= '15 * * * *'
Settings.cron_jobs['stuck_import_jobs_worker']['job_class'] = 'StuckImportJobsWorker'
Settings.cron_jobs['gitlab_usage_ping_worker'] ||= Settingslogic.new({})
-Settings.cron_jobs['gitlab_usage_ping_worker']['cron'] ||= Settings.__send__(:cron_random_weekly_time)
+Settings.cron_jobs['gitlab_usage_ping_worker']['cron'] ||= Settings.__send__(:cron_for_usage_ping)
Settings.cron_jobs['gitlab_usage_ping_worker']['job_class'] = 'GitlabUsagePingWorker'
Settings.cron_jobs['schedule_update_user_activity_worker'] ||= Settingslogic.new({})