Merge branch 'reply-by-email-docs' into 'master'

Expand Reply by email documentation

TODO:

- [x] Verify this works on dev server
- [ ] Figure out SSL somehow: http://www.postfix.org/TLS_README.html (Moved to #2453)

See merge request !1193

2 files changed, 364 insertions, 31 deletions

diff --git a/doc/reply_by_email/README.md b/doc/reply_by_email/README.md
index 5d36f5121d1..e9187298d79 100644
--- a/doc/reply_by_email/README.md
+++ b/doc/reply_by_email/README.md
@@ -2,11 +2,48 @@
 
 GitLab can be set up to allow users to comment on issues and merge requests by replying to notification emails.
 
-In order to do this, you need access to an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Yahoo! Mail, Outlook.com and iCloud, as well as the [Postfix](http://www.postfix.org/) mail server which you can run on-premises.
+## Get a mailbox
+
+Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises.
+
+If you want to use Gmail with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+
+To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these instructions](./postfix.md).
 
 ## Set it up
 
-In this example, we'll use the Gmail address `gitlab-replies@gmail.com`. If you're actually using Gmail with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+In this example, we'll use the Gmail address `gitlab-replies@gmail.com`.
+
+### Omnibus package installations
+
+1. Find the `reply_by_email` section in `/etc/gitlab/gitlab.rb`, enable the feature, enter the email address including a placeholder for the `reply_key` and fill in the details for your specific IMAP server and email account:
+
+    ```ruby
+    gitlab_rails['reply_by_email_enabled'] = true
+    gitlab_rails['reply_by_email_address'] = "gitlab-replies+%{reply_key}@gmail.com"
+    gitlab_rails['reply_by_email_host'] = "imap.gmail.com" # IMAP server host
+    gitlab_rails['reply_by_email_port'] = 993 # IMAP server port
+    gitlab_rails['reply_by_email_ssl'] = true # Whether the IMAP server uses SSL
+    gitlab_rails['reply_by_email_email'] = "gitlab-replies@gmail.com"  # Email account username. Usually the full email address.
+    gitlab_rails['reply_by_email_password'] = "password" # Email account password
+    gitlab_rails['reply_by_email_mailbox_name'] = "inbox" # The name of the mailbox where incoming mail will end up. Usually "inbox".
+    ```
+
+    As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-replies@gmail.com`.
+
+1. Reconfigure GitLab for the changes to take effect:
+
+    ```sh
+    sudo gitlab-ctl reconfigure
+    ```
+
+1. Verify that everything is configured correctly:
+
+    ```sh
+    sudo gitlab-rake gitlab:reply_by_email:check
+    ```
+
+1. Reply by email should now be working.
 
 ### Installations from source
 
@@ -21,17 +58,17 @@ In this example, we'll use the Gmail address `gitlab-replies@gmail.com`. If you'
     ```sh
     sudo editor config/gitlab.yml
     ```
-    
+
     ```yaml
     reply_by_email:
       enabled: true
       address: "gitlab-replies+%{reply_key}@gmail.com"
     ```
 
-    As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-replies@gmail.com`.
+    As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-replies@gmail.com`.
+
+2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`:
 
-2. Find `config/mail_room.yml.example` and copy it to `config/mail_room.yml`:
-    
     ```sh
     sudo cp config/mail_room.yml.example config/mail_room.yml
     ```
@@ -72,47 +109,33 @@ In this example, we'll use the Gmail address `gitlab-replies@gmail.com`. If you'
           :worker: EmailReceiverWorker
     ```
 
+5. Edit the init script configuration at `/etc/default/gitlab` to enable `mail_room`:
 
-4.  Find `lib/support/init.d/gitlab.default.example` and copy it to `/etc/default/gitlab`:
-    
     ```sh
-    sudo cp lib/support/init.d/gitlab.default.example /etc/default/gitlab
-    ```
-
-5. Edit `/etc/default/gitlab` to enable `mail_room`:
-
-    ```sh
-    sudo editor /etc/default/gitlab
-    ```
-    
-    ```sh
-    mail_room_enabled=true
+    sudo mkdir -p /etc/default
+    echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab
     ```
 
 6. Restart GitLab:
-    
+
     ```sh
     sudo service gitlab restart
     ```
 
-7. Check if everything is configured correctly:
+7. Verify that everything is configured correctly:
 
     ```sh
-    sudo bundle exec rake gitlab:reply_by_email:check RAILS_ENV=production
+    sudo -u git -H bundle exec rake gitlab:reply_by_email:check RAILS_ENV=production
     ```
 
 8. Reply by email should now be working.
 
-### Omnibus package installations
-
-TODO
-
 ### Development
 
 1. Go to the GitLab installation directory.
 
 1. Find the `reply_by_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `reply_key`:
-    
+
     ```yaml
     reply_by_email:
       enabled: true
@@ -121,8 +144,8 @@ TODO
 
     As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-replies@gmail.com`.
 
-2. Find `config/mail_room.yml.example` and copy it to `config/mail_room.yml`:
-    
+2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`:
+
     ```sh
     sudo cp config/mail_room.yml.example config/mail_room.yml
     ```
@@ -166,12 +189,12 @@ TODO
     ```
 
 6. Restart GitLab:
-    
+
     ```sh
     bundle exec foreman start
     ```
 
-7. Check if everything is configured correctly:
+7. Verify that everything is configured correctly:
 
     ```sh
     bundle exec rake gitlab:reply_by_email:check RAILS_ENV=development
diff --git a/doc/reply_by_email/postfix.md b/doc/reply_by_email/postfix.md
new file mode 100644
index 00000000000..b8ab07d9fe1
--- /dev/null
+++ b/doc/reply_by_email/postfix.md
@@ -0,0 +1,310 @@
+# Set up Postfix for Reply by email
+
+This document will take you through the steps of setting up a basic Postfix mail server with IMAP authentication on Ubuntu, to be used with Reply by email.
+
+The instructions make the assumption that you will be using the email address `replies@gitlab.example.com`, that is, username `replies` on host `gitlab.example.com`. Don't forget to change it to your actual host when executing the example code snippets.
+
+## Configure your server firewall
+
+1. Open up port 25 on your server so that people can send email into the server over SMTP.
+2. If the mail server is different from the server running GitLab, open up port 143 on your server so that GitLab can read email from the server over IMAP.
+
+## Install packages
+
+1. Install the `postfix` package if it is not installed already:
+
+    ```sh
+    sudo apt-get install postfix
+    ```
+
+    When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`.
+
+1. Install the `mailutils` package.
+
+    ```sh
+    sudo apt-get install mailutils
+    ```
+
+## Create user
+
+1. Create a user for replies.
+
+    ```sh
+    sudo useradd -m -s /bin/bash replies
+    ```
+
+1. Set a password for this user.
+
+    ```sh
+    sudo passwd replies
+    ```
+
+    Be sure not to forget this, you'll need it later.
+
+## Test the out-of-the-box setup
+
+1. Connect to the local SMTP server:
+    
+    ```sh
+    telnet localhost 25
+    ```
+
+    You should see a prompt like this:
+
+    ```sh
+    Trying 127.0.0.1...
+    Connected to localhost.
+    Escape character is '^]'.
+    220 gitlab.example.com ESMTP Postfix (Ubuntu)
+    ```
+
+    If you get a `Connection refused` error instead, verify that `postfix` is running:
+
+    ```sh
+    sudo postfix status
+    ```
+
+    If it is not, start it:
+
+    ```sh
+    sudo postfix start
+    ```
+
+1. Send the new `replies` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+    
+    ```
+    ehlo localhost
+    mail from: root@localhost
+    rcpt to: replies@localhost
+    data
+    Subject: Re: Some issue
+
+    Sounds good!
+    .
+    quit
+    ```
+
+    (Note: The `.` is a literal period on its own line)
+
+1. Check if the `replies` user received the email:
+    
+    ```sh
+    su - replies
+    mail
+    ```
+
+    You should see output like this:
+
+    ```
+    "/var/mail/replies": 1 message 1 unread
+    >U   1 root@localhost                           59/2842  Re: Some issue
+    ```
+
+    Quit the mail app:
+
+    ```sh
+    q
+    ```
+
+1. Log out of the `replies` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Configure Postfix to use Maildir-style mailboxes
+
+Courier, which we will install later to add IMAP authentication, requires mailboxes to have the Maildir format, rather than mbox.
+
+1. Configure Postfix to use Maildir-style mailboxes:
+    
+    ```sh
+    sudo postconf -e "home_mailbox = Maildir/"
+    ```
+
+1. Restart Postfix:
+    
+    ```sh
+    sudo /etc/init.d/postfix restart
+    ```
+
+1. Test the new setup:
+    
+    1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_.
+    2. Check if the `replies` user received the email:
+    
+        ```sh
+        su - replies
+        MAIL=/home/replies/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/replies/Maildir": 1 message 1 unread
+        >U   1 root@localhost                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+1. Log out of the `replies` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Install the Courier IMAP server
+    
+1. Install the `courier-imap` package:
+
+    ```sh
+    sudo apt-get install courier-imap
+    ```
+
+## Configure Postfix to receive email from the internet
+
+1. Let Postfix know about the domains that it should consider local:
+    
+    ```sh
+    sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost"
+    ```
+
+1. Let Postfix know about the IPs that it should consider part of the LAN:
+    
+    We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network.
+    
+    ```sh
+    sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24"
+    ```
+
+1. Configure Postfix to receive mail on all interfaces, which includes the internet:
+    
+    ```sh
+    sudo postconf -e "inet_interfaces = all"
+    ```
+
+1. Configure Postfix to use the `+` delimiter for sub-addressing:
+    
+    ```sh
+    sudo postconf -e "recipient_delimiter = +"
+    ```
+
+1. Restart Postfix:
+    
+    ```sh
+    sudo service postfix restart
+    ```
+
+## Test the final setup
+
+1. Test SMTP under the new setup:
+    
+    1. Connect to the SMTP server:
+        
+        ```sh
+        telnet gitlab.example.com 25
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to gitlab.example.com.
+        Escape character is '^]'.
+        220 gitlab.example.com ESMTP Postfix (Ubuntu)
+        ```
+
+        If you get a `Connection refused` error instead, make sure your firewall is setup to allow inbound traffic on port 25.
+
+    1. Send the `replies` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+        
+        ```
+        ehlo gitlab.example.com
+        mail from: root@gitlab.example.com
+        rcpt to: replies@gitlab.example.com
+        data
+        Subject: Re: Some issue
+
+        Sounds good!
+        .
+        quit
+        ```
+
+        (Note: The `.` is a literal period on its own line)
+
+    1. Check if the `replies` user received the email:
+    
+        ```sh
+        su - replies
+        MAIL=/home/replies/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/replies/Maildir": 1 message 1 unread
+        >U   1 root@gitlab.example.com                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+    1. Log out of the `replies` account and go back to being `root`:
+
+        ```sh
+        logout
+        ```
+
+1. Test IMAP under the new setup:
+    
+    1. Connect to the IMAP server:
+        
+        ```sh
+        telnet gitlab.example.com 143
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to mail.example.gitlab.com.
+        Escape character is '^]'.
+        - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc.  See COPYING for distribution information.
+        ```
+
+    1. Sign in as the `replies` user to test IMAP, by entering the following into the IMAP prompt:
+
+        ```
+        a login replies PASSWORD
+        ```
+
+        Replace PASSWORD with the password you set on the `replies` user earlier.
+
+        You should see output like this:
+
+        ```
+        a OK LOGIN Ok.
+        ```
+
+    1. Disconnect from the IMAP server:
+
+        ```sh
+        a logout
+        ```
+
+## Done!
+
+If all the tests were successfull, Postfix is all set up and ready to receive email! Continue with the [Reply by email](./README.md) guide to configure GitLab.
+
+---------
+
+_This document was adapted from https://help.ubuntu.com/community/PostfixBasicSetupHowto, by contributors to the Ubuntu documentation wiki._