Move reply by email docs to a new locationdocs/refactor-reply-by-email

[ci skip]

3 files changed, 627 insertions, 1 deletions

diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md
new file mode 100644
index 00000000000..5a9a1582877
--- /dev/null
+++ b/doc/administration/reply_by_email.md
@@ -0,0 +1,302 @@
+# Reply by email
+
+GitLab can be set up to allow users to comment on issues and merge requests by
+replying to notification emails.
+
+## Requirement
+
+Reply by email requires an IMAP-enabled email account. GitLab allows you to use
+three strategies for this feature:
+- using email sub-addressing
+- using a dedicated email address
+- using a catch-all mailbox
+
+### Email sub-addressing
+
+**If your provider or server supports email sub-addressing, we recommend using it.**
+
+[Sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing) is
+a feature where any email to `user+some_arbitrary_tag@example.com` will end up
+in the mailbox for `user@example.com`, and is supported by providers such as
+Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix
+mail server which you can run on-premises.
+
+### Dedicated email address
+
+This solution is really simple to set up: you just have to create an email
+address dedicated to receive your users' replies to GitLab notifications.
+
+### Catch-all mailbox
+
+A [catch-all mailbox](https://en.wikipedia.org/wiki/Catch-all) for a domain will
+"catch all" the emails addressed to the domain that do not exist in the mail
+server.
+
+## How it works?
+
+### 1. GitLab sends a notification email
+
+When GitLab sends a notification and Reply by email is enabled, the `Reply-To`
+header is set to the address defined in your GitLab configuration, with the
+`%{key}` placeholder (if present) replaced by a specific "reply key". In
+addition, this "reply key" is also added to the `References` header.
+
+### 2. You reply to the notification email
+
+When you reply to the notification email, your email client will:
+
+- send the email to the `Reply-To` address it got from the notification email
+- set the `In-Reply-To` header to the value of the `Message-ID` header from the
+  notification email
+- set the `References` header to the value of the `Message-ID` plus the value of
+  the notification email's `References` header.
+
+### 3. GitLab receives your reply to the notification email
+
+When GitLab receives your reply, it will look for the "reply key" in the
+following headers, in this order:
+
+1. the `To` header
+1. the `References` header
+
+If it finds a reply key, it will be able to leave your reply as a comment on
+the entity the notification was about (issue, merge request, commit...).
+
+For more details about the `Message-ID`, `In-Reply-To`, and `References headers`,
+please consult [RFC 5322](https://tools.ietf.org/html/rfc5322#section-3.6.4).
+
+## Set it up
+
+If you want to use Gmail / Google Apps with Reply by email, make sure you have
+[IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018)
+and [allowed less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+
+To set up a basic Postfix mail server with IMAP access on Ubuntu, follow
+[these instructions](./postfix.md).
+
+### Omnibus package installations
+
+1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the
+  feature and fill in the details for your specific IMAP server and email account:
+
+    ```ruby
+    # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com
+    gitlab_rails['incoming_email_enabled'] = true
+
+    # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
+    # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
+    gitlab_rails['incoming_email_address'] = "incoming+%{key}@gitlab.example.com"
+
+    # Email account username
+    # With third party providers, this is usually the full email address.
+    # With self-hosted email servers, this is usually the user part of the email address.
+    gitlab_rails['incoming_email_email'] = "incoming"
+    # Email account password
+    gitlab_rails['incoming_email_password'] = "[REDACTED]"
+
+    # IMAP server host
+    gitlab_rails['incoming_email_host'] = "gitlab.example.com"
+    # IMAP server port
+    gitlab_rails['incoming_email_port'] = 143
+    # Whether the IMAP server uses SSL
+    gitlab_rails['incoming_email_ssl'] = false
+    # Whether the IMAP server uses StartTLS
+    gitlab_rails['incoming_email_start_tls'] = false
+
+    # The mailbox where incoming mail will end up. Usually "inbox".
+    gitlab_rails['incoming_email_mailbox_name'] = "inbox"
+    ```
+
+    ```ruby
+    # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com
+    gitlab_rails['incoming_email_enabled'] = true
+
+    # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
+    # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
+    gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com"
+
+    # Email account username
+    # With third party providers, this is usually the full email address.
+    # With self-hosted email servers, this is usually the user part of the email address.
+    gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com"
+    # Email account password
+    gitlab_rails['incoming_email_password'] = "[REDACTED]"
+
+    # IMAP server host
+    gitlab_rails['incoming_email_host'] = "imap.gmail.com"
+    # IMAP server port
+    gitlab_rails['incoming_email_port'] = 993
+    # Whether the IMAP server uses SSL
+    gitlab_rails['incoming_email_ssl'] = true
+    # Whether the IMAP server uses StartTLS
+    gitlab_rails['incoming_email_start_tls'] = false
+
+    # The mailbox where incoming mail will end up. Usually "inbox".
+    gitlab_rails['incoming_email_mailbox_name'] = "inbox"
+    ```
+
+1. Reconfigure GitLab and restart mailroom for the changes to take effect:
+
+    ```sh
+    sudo gitlab-ctl reconfigure
+    sudo gitlab-ctl restart mailroom
+    ```
+
+1. Verify that everything is configured correctly:
+
+    ```sh
+    sudo gitlab-rake gitlab:incoming_email:check
+    ```
+
+1. Reply by email should now be working.
+
+### Installations from source
+
+1. Go to the GitLab installation directory:
+
+    ```sh
+    cd /home/git/gitlab
+    ```
+
+1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature
+  and fill in the details for your specific IMAP server and email account:
+
+    ```sh
+    sudo editor config/gitlab.yml
+    ```
+
+    ```yaml
+    # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com
+    incoming_email:
+      enabled: true
+
+      # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
+      # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
+      address: "incoming+%{key}@gitlab.example.com"
+
+      # Email account username
+      # With third party providers, this is usually the full email address.
+      # With self-hosted email servers, this is usually the user part of the email address.
+      user: "incoming"
+      # Email account password
+      password: "[REDACTED]"
+
+      # IMAP server host
+      host: "gitlab.example.com"
+      # IMAP server port
+      port: 143
+      # Whether the IMAP server uses SSL
+      ssl: false
+      # Whether the IMAP server uses StartTLS
+      start_tls: false
+
+      # The mailbox where incoming mail will end up. Usually "inbox".
+      mailbox: "inbox"
+    ```
+
+    ```yaml
+    # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com
+    incoming_email:
+      enabled: true
+
+      # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
+      # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
+      address: "gitlab-incoming+%{key}@gmail.com"
+
+      # Email account username
+      # With third party providers, this is usually the full email address.
+      # With self-hosted email servers, this is usually the user part of the email address.
+      user: "gitlab-incoming@gmail.com"
+      # Email account password
+      password: "[REDACTED]"
+
+      # IMAP server host
+      host: "imap.gmail.com"
+      # IMAP server port
+      port: 993
+      # Whether the IMAP server uses SSL
+      ssl: true
+      # Whether the IMAP server uses StartTLS
+      start_tls: false
+
+      # The mailbox where incoming mail will end up. Usually "inbox".
+      mailbox: "inbox"
+    ```
+
+1. Enable `mail_room` in the init script at `/etc/default/gitlab`:
+
+    ```sh
+    sudo mkdir -p /etc/default
+    echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab
+    ```
+
+1. Restart GitLab:
+
+    ```sh
+    sudo service gitlab restart
+    ```
+
+1. Verify that everything is configured correctly:
+
+    ```sh
+    sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production
+    ```
+
+1. Reply by email should now be working.
+
+### Development
+
+1. Go to the GitLab installation directory.
+
+1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account:
+
+    ```yaml
+    # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com
+    incoming_email:
+      enabled: true
+
+      # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to.
+      # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`).
+      address: "gitlab-incoming+%{key}@gmail.com"
+
+      # Email account username
+      # With third party providers, this is usually the full email address.
+      # With self-hosted email servers, this is usually the user part of the email address.
+      user: "gitlab-incoming@gmail.com"
+      # Email account password
+      password: "[REDACTED]"
+
+      # IMAP server host
+      host: "imap.gmail.com"
+      # IMAP server port
+      port: 993
+      # Whether the IMAP server uses SSL
+      ssl: true
+      # Whether the IMAP server uses StartTLS
+      start_tls: false
+
+      # The mailbox where incoming mail will end up. Usually "inbox".
+      mailbox: "inbox"
+    ```
+
+    As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`.
+
+1. Uncomment the `mail_room` line in your `Procfile`:
+
+    ```yaml
+    mail_room: bundle exec mail_room -q -c config/mail_room.yml
+    ```
+
+1. Restart GitLab:
+
+    ```sh
+    bundle exec foreman start
+    ```
+
+1. Verify that everything is configured correctly:
+
+    ```sh
+    bundle exec rake gitlab:incoming_email:check RAILS_ENV=development
+    ```
+
+1. Reply by email should now be working.
diff --git a/doc/administration/reply_by_email_postfix_setup.md b/doc/administration/reply_by_email_postfix_setup.md
new file mode 100644
index 00000000000..22f10489a6c
--- /dev/null
+++ b/doc/administration/reply_by_email_postfix_setup.md
@@ -0,0 +1,324 @@
+# Set up Postfix for Reply by email
+
+This document will take you through the steps of setting up a basic Postfix mail
+server with IMAP authentication on Ubuntu, to be used with [Reply by email].
+
+The instructions make the assumption that you will be using the email address `incoming@gitlab.example.com`, that is, username `incoming` on host `gitlab.example.com`. Don't forget to change it to your actual host when executing the example code snippets.
+
+## Configure your server firewall
+
+1. Open up port 25 on your server so that people can send email into the server over SMTP.
+2. If the mail server is different from the server running GitLab, open up port 143 on your server so that GitLab can read email from the server over IMAP.
+
+## Install packages
+
+1. Install the `postfix` package if it is not installed already:
+
+    ```sh
+    sudo apt-get install postfix
+    ```
+
+    When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`.
+
+1. Install the `mailutils` package.
+
+    ```sh
+    sudo apt-get install mailutils
+    ```
+
+## Create user
+
+1. Create a user for incoming email.
+
+    ```sh
+    sudo useradd -m -s /bin/bash incoming
+    ```
+
+1. Set a password for this user.
+
+    ```sh
+    sudo passwd incoming
+    ```
+
+    Be sure not to forget this, you'll need it later.
+
+## Test the out-of-the-box setup
+
+1. Connect to the local SMTP server:
+
+    ```sh
+    telnet localhost 25
+    ```
+
+    You should see a prompt like this:
+
+    ```sh
+    Trying 127.0.0.1...
+    Connected to localhost.
+    Escape character is '^]'.
+    220 gitlab.example.com ESMTP Postfix (Ubuntu)
+    ```
+
+    If you get a `Connection refused` error instead, verify that `postfix` is running:
+
+    ```sh
+    sudo postfix status
+    ```
+
+    If it is not, start it:
+
+    ```sh
+    sudo postfix start
+    ```
+
+1. Send the new `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+
+    ```
+    ehlo localhost
+    mail from: root@localhost
+    rcpt to: incoming@localhost
+    data
+    Subject: Re: Some issue
+
+    Sounds good!
+    .
+    quit
+    ```
+
+    _**Note:** The `.` is a literal period on its own line._
+
+    _**Note:** If you receive an error after entering `rcpt to: incoming@localhost`
+    then your Postfix `my_network` configuration is not correct. The error will
+    say 'Temporary lookup failure'. See
+    [Configure Postfix to receive email from the Internet](#configure-postfix-to-receive-email-from-the-internet)._
+
+1. Check if the `incoming` user received the email:
+
+    ```sh
+    su - incoming
+    mail
+    ```
+
+    You should see output like this:
+
+    ```
+    "/var/mail/incoming": 1 message 1 unread
+    >U   1 root@localhost                           59/2842  Re: Some issue
+    ```
+
+    Quit the mail app:
+
+    ```sh
+    q
+    ```
+
+1. Log out of the `incoming` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Configure Postfix to use Maildir-style mailboxes
+
+Courier, which we will install later to add IMAP authentication, requires mailboxes to have the Maildir format, rather than mbox.
+
+1. Configure Postfix to use Maildir-style mailboxes:
+
+    ```sh
+    sudo postconf -e "home_mailbox = Maildir/"
+    ```
+
+1. Restart Postfix:
+
+    ```sh
+    sudo /etc/init.d/postfix restart
+    ```
+
+1. Test the new setup:
+
+    1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_.
+    1. Check if the `incoming` user received the email:
+
+        ```sh
+        su - incoming
+        MAIL=/home/incoming/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/incoming/Maildir": 1 message 1 unread
+        >U   1 root@localhost                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+    _**Note:** If `mail` returns an error `Maildir: Is a directory` then your
+    version of `mail` doesn't support Maildir style mailboxes. Install
+    `heirloom-mailx` by running `sudo apt-get install heirloom-mailx`. Then,
+    try the above steps again, substituting `heirloom-mailx` for the `mail`
+    command._
+
+1. Log out of the `incoming` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Install the Courier IMAP server
+
+1. Install the `courier-imap` package:
+
+    ```sh
+    sudo apt-get install courier-imap
+    ```
+
+## Configure Postfix to receive email from the internet
+
+1. Let Postfix know about the domains that it should consider local:
+
+    ```sh
+    sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost"
+    ```
+
+1. Let Postfix know about the IPs that it should consider part of the LAN:
+
+    We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network.
+
+    ```sh
+    sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24"
+    ```
+
+1. Configure Postfix to receive mail on all interfaces, which includes the internet:
+
+    ```sh
+    sudo postconf -e "inet_interfaces = all"
+    ```
+
+1. Configure Postfix to use the `+` delimiter for sub-addressing:
+
+    ```sh
+    sudo postconf -e "recipient_delimiter = +"
+    ```
+
+1. Restart Postfix:
+
+    ```sh
+    sudo service postfix restart
+    ```
+
+## Test the final setup
+
+1. Test SMTP under the new setup:
+
+    1. Connect to the SMTP server:
+
+        ```sh
+        telnet gitlab.example.com 25
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to gitlab.example.com.
+        Escape character is '^]'.
+        220 gitlab.example.com ESMTP Postfix (Ubuntu)
+        ```
+
+        If you get a `Connection refused` error instead, make sure your firewall is setup to allow inbound traffic on port 25.
+
+    1. Send the `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+
+        ```
+        ehlo gitlab.example.com
+        mail from: root@gitlab.example.com
+        rcpt to: incoming@gitlab.example.com
+        data
+        Subject: Re: Some issue
+
+        Sounds good!
+        .
+        quit
+        ```
+
+        (Note: The `.` is a literal period on its own line)
+
+    1. Check if the `incoming` user received the email:
+
+        ```sh
+        su - incoming
+        MAIL=/home/incoming/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/incoming/Maildir": 1 message 1 unread
+        >U   1 root@gitlab.example.com                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+    1. Log out of the `incoming` account and go back to being `root`:
+
+        ```sh
+        logout
+        ```
+
+1. Test IMAP under the new setup:
+
+    1. Connect to the IMAP server:
+
+        ```sh
+        telnet gitlab.example.com 143
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to mail.example.gitlab.com.
+        Escape character is '^]'.
+        - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc.  See COPYING for distribution information.
+        ```
+
+    1. Sign in as the `incoming` user to test IMAP, by entering the following into the IMAP prompt:
+
+        ```
+        a login incoming PASSWORD
+        ```
+
+        Replace PASSWORD with the password you set on the `incoming` user earlier.
+
+        You should see output like this:
+
+        ```
+        a OK LOGIN Ok.
+        ```
+
+    1. Disconnect from the IMAP server:
+
+        ```sh
+        a logout
+        ```
+
+## Done!
+
+If all the tests were successful, Postfix is all set up and ready to receive email! Continue with the [Reply by email](./README.md) guide to configure GitLab.
+
+---
+
+_This document was adapted from https://help.ubuntu.com/community/PostfixBasicSetupHowto, by contributors to the Ubuntu documentation wiki._
+
+[reply by email]: reply_by_email.md
diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md
index 483060395dd..b561c2f82aa 100644
--- a/doc/administration/restart_gitlab.md
+++ b/doc/administration/restart_gitlab.md
@@ -139,7 +139,7 @@ If you are using other init systems, like systemd, you can check the
 
 [omnibus-dl]: https://about.gitlab.com/downloads/ "Download the Omnibus packages"
 [install]: ../install/installation.md "Documentation to install GitLab from source"
-[mailroom]: ../incoming_email/README.md "Used for replying by email in GitLab issues and merge requests"
+[mailroom]: reply_by_email.md "Used for replying by email in GitLab issues and merge requests"
 [chef]: https://www.chef.io/chef/ "Chef official website"
 [src-service]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/init.d/gitlab "GitLab init service file"
 [gl-recipes]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/init "GitLab Recipes repository"