Merge branch 'gitaly-opt-out' into 'master'

Enable Gitaly by default in GitLab 9.3

Closes gitaly#242

See merge request !11796

8 files changed, 295 insertions, 20 deletions

diff --git a/changelogs/unreleased/gitaly-opt-out.yml b/changelogs/unreleased/gitaly-opt-out.yml
new file mode 100644
index 00000000000..2f89e0bfc9a
--- /dev/null
+++ b/changelogs/unreleased/gitaly-opt-out.yml
@@ -0,0 +1,4 @@
+---
+title: Enable Gitaly by default in installations from source
+merge_request: 11796
+author:
diff --git a/config/gitlab.yml.example b/config/gitlab.yml.example
index a727f7e2fa3..6c1c1f8c041 100644
--- a/config/gitlab.yml.example
+++ b/config/gitlab.yml.example
@@ -449,7 +449,7 @@ production: &base
     # This setting controls whether GitLab uses Gitaly (new component
     # introduced in 9.0). Eventually Gitaly use will become mandatory and
     # this option will disappear.
-    enabled: false
+    enabled: true
 
   #
   # 4. Advanced settings
diff --git a/config/initializers/1_settings.rb b/config/initializers/1_settings.rb
index 4fb4baf631f..45ea2040d23 100644
--- a/config/initializers/1_settings.rb
+++ b/config/initializers/1_settings.rb
@@ -482,7 +482,7 @@ Settings.rack_attack.git_basic_auth['bantime'] ||= 1.hour
 # Gitaly
 #
 Settings['gitaly'] ||= Settingslogic.new({})
-Settings.gitaly['enabled'] ||= false
+Settings.gitaly['enabled'] = true if Settings.gitaly['enabled'].nil?
 
 #
 # Webpack settings
diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md
index 6c6942a7bfe..48929910a9c 100644
--- a/doc/administration/gitaly/index.md
+++ b/doc/administration/gitaly/index.md
@@ -2,7 +2,7 @@
 
 [Gitaly](https://gitlab.com/gitlab-org/gitaly) (introduced in GitLab
 9.0) is a service that provides high-level RPC access to Git
-repositories. As of GitLab 9.1 it is still an optional component with
+repositories. As of GitLab 9.3 it is still an optional component with
 limited scope.
 
 GitLab components that access Git repositories (gitlab-rails,
@@ -35,7 +35,7 @@ gitlab restart`.
 
 ## Configuring GitLab to not use Gitaly
 
-Gitaly is still an optional component in GitLab 9.0. This means you
+Gitaly is still an optional component in GitLab 9.3. This means you
 can choose to not use it.
 
 In Omnibus you can make the following change in
diff --git a/doc/install/installation.md b/doc/install/installation.md
index 0cf7d40507b..af21d99d024 100644
--- a/doc/install/installation.md
+++ b/doc/install/installation.md
@@ -470,10 +470,6 @@ Make GitLab start on boot:
 
 ### Install Gitaly
 
-As of GitLab 9.1 Gitaly is an **optional** component. Its
-configuration is still changing regularly. It is OK to wait
-with setting up Gitaly until you upgrade to GitLab 9.2 or later.
-
     # Fetch Gitaly source with Git and compile with Go
     sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly]" RAILS_ENV=production
 
@@ -491,16 +487,6 @@ Next, make sure gitaly configured:
     cd /home/git/gitaly
     sudo -u git -H editor config.toml
 
-    # Enable Gitaly in the init script
-    echo 'gitaly_enabled=true' | sudo tee -a /etc/default/gitlab
-
-Next, edit `/home/git/gitlab/config/gitlab.yml` and make sure `enabled: true` in
-the `gitaly:` section is uncommented.
-
-    # <- gitlab.yml indentation starts here
-      gitaly:
-        enabled: true
-
 For more information about configuring Gitaly see
 [doc/administration/gitaly](../administration/gitaly).
 
diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md
new file mode 100644
index 00000000000..26049721fd3
--- /dev/null
+++ b/doc/update/9.2-to-9.3.md
@@ -0,0 +1,285 @@
+# From 9.2 to 9.3
+
+Make sure you view this update guide from the tag (version) of GitLab you would
+like to install. In most cases this should be the highest numbered production
+tag (without rc in it). You can select the tag in the version dropdown at the
+top left corner of GitLab (below the menu bar).
+
+If the highest number stable branch is unclear please check the
+[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation
+guide links by version.
+
+### 1. Stop server
+
+```bash
+sudo service gitlab stop
+```
+
+### 2. Backup
+
+```bash
+cd /home/git/gitlab
+
+sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
+```
+
+### 3. Update Ruby
+
+NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be
+sure to upgrade your interpreter if necessary.
+
+You can check which version you are running with `ruby -v`.
+
+Download and compile Ruby:
+
+```bash
+mkdir /tmp/ruby && cd /tmp/ruby
+curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.3.tar.gz
+echo '1014ee699071aa2ddd501907d18cbe15399c997d ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz
+cd ruby-2.3.3
+./configure --disable-install-rdoc
+make
+sudo make install
+```
+
+Install Bundler:
+
+```bash
+sudo gem install bundler --no-ri --no-rdoc
+```
+
+### 4. Update Node
+
+GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and
+it has a minimum requirement of node v4.3.0.
+
+You can check which version you are running with `node -v`. If you are running
+a version older than `v4.3.0` you will need to update to a newer version.  You
+can find instructions to install from community maintained packages or compile
+from source at the nodejs.org website.
+
+<https://nodejs.org/en/download/>
+
+
+Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage
+JavaScript dependencies.
+
+```bash
+curl --location https://yarnpkg.com/install.sh | bash -
+```
+
+More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install).
+
+### 5. Get latest code
+
+```bash
+cd /home/git/gitlab
+
+sudo -u git -H git fetch --all
+sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically
+```
+
+For GitLab Community Edition:
+
+```bash
+cd /home/git/gitlab
+
+sudo -u git -H git checkout 9-3-stable
+```
+
+OR
+
+For GitLab Enterprise Edition:
+
+```bash
+cd /home/git/gitlab
+
+sudo -u git -H git checkout 9-3-stable-ee
+```
+
+### 6. Update gitlab-shell
+
+```bash
+cd /home/git/gitlab-shell
+
+sudo -u git -H git fetch --all --tags
+sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION)
+sudo -u git -H bin/compile
+```
+
+### 7. Update gitlab-workhorse
+
+Install and compile gitlab-workhorse. This requires
+[Go 1.5](https://golang.org/dl) which should already be on your system from
+GitLab 8.1. GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/).
+If you are not using Linux you may have to run `gmake` instead of
+`make` below.
+
+```bash
+cd /home/git/gitlab-workhorse
+
+sudo -u git -H git fetch --all --tags
+sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION)
+sudo -u git -H make
+```
+
+### 8. Update Gitaly
+
+If you have not yet set up Gitaly then follow [Gitaly section of the installation
+guide](../install/installation.md#install-gitaly).
+
+#### Compile Gitaly
+
+```shell
+cd /home/git/gitaly
+sudo -u git -H git fetch --all --tags
+sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION)
+sudo -u git -H make
+```
+
+### 9. Update configuration files
+
+#### New configuration options for `gitlab.yml`
+
+There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`:
+
+```sh
+cd /home/git/gitlab
+
+git diff origin/9-2-stable:config/gitlab.yml.example origin/9-3-stable:config/gitlab.yml.example
+```
+
+#### Nginx configuration
+
+Ensure you're still up-to-date with the latest NGINX configuration changes:
+
+```sh
+cd /home/git/gitlab
+
+# For HTTPS configurations
+git diff origin/9-2-stable:lib/support/nginx/gitlab-ssl origin/9-3-stable:lib/support/nginx/gitlab-ssl
+
+# For HTTP configurations
+git diff origin/9-2-stable:lib/support/nginx/gitlab origin/9-3-stable:lib/support/nginx/gitlab
+```
+
+If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx
+configuration as GitLab application no longer handles setting it.
+
+If you are using Apache instead of NGINX please see the updated [Apache templates].
+Also note that because Apache does not support upstreams behind Unix sockets you
+will need to let gitlab-workhorse listen on a TCP port. You can do this
+via [/etc/default/gitlab].
+
+[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache
+[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-3-stable/lib/support/init.d/gitlab.default.example#L38
+
+#### SMTP configuration
+
+If you're installing from source and use SMTP to deliver mail, you will need to add the following line
+to config/initializers/smtp_settings.rb:
+
+```ruby
+ActionMailer::Base.delivery_method = :smtp
+```
+
+See [smtp_settings.rb.sample] as an example.
+
+[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-2-stable/config/initializers/smtp_settings.rb.sample#L13
+
+#### Init script
+
+There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`:
+
+```sh
+cd /home/git/gitlab
+
+git diff origin/9-2-stable:lib/support/init.d/gitlab.default.example origin/9-3-stable:lib/support/init.d/gitlab.default.example
+```
+
+Ensure you're still up-to-date with the latest init script changes:
+
+```bash
+cd /home/git/gitlab
+
+sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab
+```
+
+For Ubuntu 16.04.1 LTS:
+
+```bash
+sudo systemctl daemon-reload
+```
+
+### 10. Install libs, migrations, etc.
+
+```bash
+cd /home/git/gitlab
+
+# MySQL installations (note: the line below states '--without postgres')
+sudo -u git -H bundle install --without postgres development test --deployment
+
+# PostgreSQL installations (note: the line below states '--without mysql')
+sudo -u git -H bundle install --without mysql development test --deployment
+
+# Optional: clean up old gems
+sudo -u git -H bundle clean
+
+# Run database migrations
+sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production
+
+# Update node dependencies and recompile assets
+sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production
+
+# Clean up cache
+sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production
+```
+
+**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md).
+
+### 11. Start application
+
+```bash
+sudo service gitlab start
+sudo service nginx restart
+```
+
+### 12. Check application status
+
+Check if GitLab and its environment are configured correctly:
+
+```bash
+cd /home/git/gitlab
+
+sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
+```
+
+To make sure you didn't miss anything run a more thorough check:
+
+```bash
+cd /home/git/gitlab
+
+sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
+```
+
+If all items are green, then congratulations, the upgrade is complete!
+
+## Things went south? Revert to previous version (9.2)
+
+### 1. Revert the code to the previous version
+
+Follow the [upgrade guide from 9.1 to 9.2](9.1-to-9.2.md), except for the
+database migration (the backup is already migrated to the previous version).
+
+### 2. Restore from the backup
+
+```bash
+cd /home/git/gitlab
+
+sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production
+```
+
+If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above.
+
+[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-3-stable/config/gitlab.yml.example
+[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-3-stable/lib/support/init.d/gitlab.default.example
diff --git a/lib/support/init.d/gitlab b/lib/support/init.d/gitlab
index 6e351365de0..c5f93336346 100755
--- a/lib/support/init.d/gitlab
+++ b/lib/support/init.d/gitlab
@@ -48,7 +48,7 @@ gitlab_pages_pid_path="$pid_path/gitlab-pages.pid"
 gitlab_pages_options="-pages-domain example.com -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090"
 gitlab_pages_log="$app_root/log/gitlab-pages.log"
 shell_path="/bin/bash"
-gitaly_enabled=false
+gitaly_enabled=true
 gitaly_dir=$(cd $app_root/../gitaly 2> /dev/null && pwd)
 gitaly_pid_path="$pid_path/gitaly.pid"
 gitaly_log="$app_root/log/gitaly.log"
diff --git a/lib/support/init.d/gitlab.default.example b/lib/support/init.d/gitlab.default.example
index b341b5a0309..295c79fccfc 100644
--- a/lib/support/init.d/gitlab.default.example
+++ b/lib/support/init.d/gitlab.default.example
@@ -86,7 +86,7 @@ mail_room_pid_path="$pid_path/mail_room.pid"
 shell_path="/bin/bash"
 
 # This variable controls whether the init script starts/stops Gitaly
-gitaly_enabled=false
+gitaly_enabled=true
 gitaly_dir=$(cd $app_root/../gitaly 2> /dev/null && pwd)
 gitaly_pid_path="$pid_path/gitaly.pid"
 gitaly_log="$app_root/log/gitaly.log"