Update the update guides for background migrations

The introduction of background migrations means some of our requirements
for online upgrades have to be adjusted / clarified.

See https://gitlab.com/gitlab-org/gitlab-ce/issues/35939 for more
information.

2 files changed, 68 insertions, 9 deletions

diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md
index e67db9ff142..f83a60e49e8 100644
--- a/doc/development/background_migrations.md
+++ b/doc/development/background_migrations.md
@@ -7,6 +7,11 @@ storing data in a single JSON column the data is stored in a separate table.
 
 ## When To Use Background Migrations
 
+>**Note:**
+When adding background migrations _you must_ make sure they are announced in the
+monthly release post along with an estimate of how long it will take to complete
+the migrations.
+
 In the vast majority of cases you will want to use a regular Rails migration
 instead. Background migrations should _only_ be used when migrating _data_ in
 tables that have so many rows this process would take hours when performed in a
@@ -91,6 +96,10 @@ BackgroundMigrationWorker.perform_bulk_in(5.minutes, jobs)
 
 ## Cleaning Up
 
+>**Note:**
+Cleaning up any remaining background migrations _must_ be done in either a major
+or minor release, you _must not_ do this in a patch release.
+
 Because background migrations can take a long time you can't immediately clean
 things up after scheduling them. For example, you can't drop a column that's
 used in the migration process as this would cause jobs to fail. This means that
diff --git a/doc/update/README.md b/doc/update/README.md
index 22dbc7c750f..c98e20686e0 100644
--- a/doc/update/README.md
+++ b/doc/update/README.md
@@ -34,17 +34,67 @@ update them are in [a separate document][omnidocker].
 
 ## Upgrading without downtime
 
-Starting with GitLab 9.1.0 it's possible to upgrade to a newer major, minor, or patch version of GitLab
-without having to take your GitLab instance offline. However, for this to work
-there are the following requirements:
-
-1. You can only upgrade 1 minor release at a time. So from 9.1 to 9.2, not to 9.3.
-2. You have to be on the most recent patch release. For example, if 9.1.15 is the last
-   release of 9.1 then you can safely upgrade from that version to any 9.2.x version.
-   However, if you are running 9.1.14 you first need to upgrade to 9.1.15.
+Starting with GitLab 9.1.0 it's possible to upgrade to a newer major, minor, or
+patch version of GitLab without having to take your GitLab instance offline.
+However, for this to work there are the following requirements:
+
+1. You can only upgrade 1 minor release at a time. So from 9.1 to 9.2, not to
+   9.3.
 2. You have to use [post-deployment
    migrations](../development/post_deployment_migrations.md).
-3. You are using PostgreSQL. If you are using MySQL please look at the release post to see if downtime is required.
+3. You are using PostgreSQL. If you are using MySQL please look at the release
+   post to see if downtime is required.
+
+Most of the time you can safely upgrade from a patch release to the next minor
+release if the patch release is not the latest. For example, upgrading from
+9.1.1 to 9.2.0 should be safe even if 9.1.2 has been released. We do recommend
+you check the release posts of any releases between your current and target
+version just in case they include any migrations that may require you to upgrade
+1 release at a time.
+
+Some releases may also include so called "background migrations". These
+migrations are performed in the background by Sidekiq and are often used for
+migrating data. Background migrations are only added in the monthly releases.
+
+Certain major/minor releases may require a set of background migrations to be
+finished. To guarantee this such a release will process any remaining jobs
+before continuing the upgrading procedure. While this won't require downtime
+(if the above conditions are met) we recommend users to keep at least 1 week
+between upgrading major/minor releases, allowing the background migrations to
+finish. The time necessary to complete these migrations can be reduced by
+increasing the number of Sidekiq workers that can process jobs in the
+`background_migration` queue.
+
+As a rule of thumb, any database smaller than 10 GB won't take too much time to
+upgrade; perhaps an hour at most per minor release. Larger databases however may
+require more time, but this is highly dependent on the size of the database and
+the migrations that are being performed.
+
+### Examples
+
+To help explain this, let's look at some examples.
+
+**Example 1:** You are running a large GitLab installation using version 9.4.2,
+which is the latest patch release of 9.4. When GitLab 9.5.0 is released this
+installation can be safely upgraded to 9.5.0 without requiring downtime if the
+requirements mentioned above are met. You can also skip 9.5.0 and upgrade to
+9.5.1 once it's released, but you **can not** upgrade straight to 9.6.0; you
+_have_ to first upgrade to a 9.5.x release.
+
+**Example 2:** You are running a large GitLab installation using version 9.4.2,
+which is the latest patch release of 9.4. GitLab 9.5 includes some background
+migrations, and 10.0 will require these to be completed (processing any
+remaining jobs for you). Skipping 9.5 is not possible without downtime, and due
+to the background migrations would require potentially hours of downtime
+depending on how long it takes for the background migrations to complete. To
+work around this you will have to upgrade to 9.5.x first, then wait at least a
+week before upgrading to 10.0.
+
+**Example 3:** You use MySQL as the database for GitLab. Any upgrade to a new
+major/minor release will require downtime. If a release includes any background
+migrations this could potentially lead to hours of downtime, depending on the
+size of your database. To work around this you will have to use PostgreSQL and
+meet the other online upgrade requirements mentioned above.
 
 ## Upgrading between editions