Make usage ping scheduling more robust39017-gitlabusagepingworker-is-not-running-on-gitlab-com

On GitLab.com, we see that cron jobs sometimes don't run. This appears to be
because the process that polls for current cron jobs only has a 60 second
validity period, so if (for whatever reason) it misses those 60 seconds, we have
to wait until the next execution.

For the usage ping, this is particularly problematic, as that's only scheduled
to run once a week.

Changing it to run for every minute in a 10 minute period should work around
this, if the above diagnosis is correct. The job itself obtains an exclusive
lease for 24 hours, so rescheduling in quick succession is safe.

1 files changed, 7 insertions, 5 deletions

diff --git a/config/initializers/1_settings.rb b/config/initializers/1_settings.rb
index a4b7c1a3919..b790df565c6 100644
--- a/config/initializers/1_settings.rb
+++ b/config/initializers/1_settings.rb
@@ -113,12 +113,14 @@ class Settings < Settingslogic
       URI.parse(url_without_path).host
     end
 
-    # Random cron time every Sunday to load balance usage pings
-    def cron_random_weekly_time
+    # Runs every minute in a random ten-minute period on Sundays, to balance the
+    # load on the server receiving these pings. The usage ping is safe to run
+    # multiple times because of a 24 hour exclusive lock.
+    def cron_for_usage_ping
       hour = rand(24)
-      minute = rand(60)
+      minute = rand(6)
 
-      "#{minute} #{hour} * * 0"
+      "#{minute}0-#{minute}9 #{hour} * * 0"
     end
   end
 end
@@ -398,7 +400,7 @@ Settings.cron_jobs['stuck_import_jobs_worker'] ||= Settingslogic.new({})
 Settings.cron_jobs['stuck_import_jobs_worker']['cron'] ||= '15 * * * *'
 Settings.cron_jobs['stuck_import_jobs_worker']['job_class'] = 'StuckImportJobsWorker'
 Settings.cron_jobs['gitlab_usage_ping_worker'] ||= Settingslogic.new({})
-Settings.cron_jobs['gitlab_usage_ping_worker']['cron'] ||= Settings.__send__(:cron_random_weekly_time)
+Settings.cron_jobs['gitlab_usage_ping_worker']['cron'] ||= Settings.__send__(:cron_for_usage_ping)
 Settings.cron_jobs['gitlab_usage_ping_worker']['job_class'] = 'GitlabUsagePingWorker'
 
 Settings.cron_jobs['schedule_update_user_activity_worker'] ||= Settingslogic.new({})