Rename reply_by_email to incoming_email to prepare for the future.

2 files changed, 513 insertions, 0 deletions

diff --git a/doc/incoming_email/README.md b/doc/incoming_email/README.md
new file mode 100644
index 00000000000..2a94bc07f2e
--- /dev/null
+++ b/doc/incoming_email/README.md
@@ -0,0 +1,203 @@
+# Reply by email
+
+GitLab can be set up to allow users to comment on issues and merge requests by replying to notification emails.
+
+## Get a mailbox
+
+Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises.
+
+If you want to use Gmail with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+
+To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these instructions](./postfix.md).
+
+## Set it up
+
+In this example, we'll use the Gmail address `gitlab-incoming@gmail.com`.
+
+### Omnibus package installations
+
+1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the feature, enter the email address including a placeholder for the `key` and fill in the details for your specific IMAP server and email account:
+
+    ```ruby
+    gitlab_rails['incoming_email_enabled'] = true
+    gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com"
+    gitlab_rails['incoming_email_host'] = "imap.gmail.com" # IMAP server host
+    gitlab_rails['incoming_email_port'] = 993 # IMAP server port
+    gitlab_rails['incoming_email_ssl'] = true # Whether the IMAP server uses SSL
+    gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com"  # Email account username. Usually the full email address.
+    gitlab_rails['incoming_email_password'] = "password" # Email account password
+    gitlab_rails['incoming_email_mailbox_name'] = "inbox" # The name of the mailbox where incoming mail will end up. Usually "inbox".
+    ```
+
+    As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-incoming@gmail.com`.
+
+1. Reconfigure GitLab for the changes to take effect:
+
+    ```sh
+    sudo gitlab-ctl reconfigure
+    ```
+
+1. Verify that everything is configured correctly:
+
+    ```sh
+    sudo gitlab-rake gitlab:incoming_email:check
+    ```
+
+1. Reply by email should now be working.
+
+### Installations from source
+
+1. Go to the GitLab installation directory:
+
+    ```sh
+    cd /home/git/gitlab
+    ```
+
+1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `key`:
+
+    ```sh
+    sudo editor config/gitlab.yml
+    ```
+
+    ```yaml
+    incoming_email:
+      enabled: true
+      address: "gitlab-incoming+%{key}@gmail.com"
+    ```
+
+    As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-incoming@gmail.com`.
+
+2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`:
+
+    ```sh
+    sudo cp config/mail_room.yml.example config/mail_room.yml
+    ```
+
+3. Uncomment the configuration options in `config/mail_room.yml` and fill in the details for your specific IMAP server and email account:
+
+    ```sh
+    sudo editor config/mail_room.yml
+    ```
+
+    ```yaml
+    :mailboxes:
+      -
+        # IMAP server host
+        :host: "imap.gmail.com"
+        # IMAP server port
+        :port: 993
+        # Whether the IMAP server uses SSL
+        :ssl: true
+        # Email account username. Usually the full email address.
+        :email: "gitlab-incoming@gmail.com"
+        # Email account password
+        :password: "[REDACTED]"
+        # The name of the mailbox where incoming mail will end up. Usually "inbox".
+        :name: "inbox"
+        # Always "sidekiq".
+        :delivery_method: sidekiq
+        # Always true.
+        :delete_after_delivery: true
+        :delivery_options:
+          # The URL to the Redis server used by Sidekiq. Should match the URL in config/resque.yml.
+          :redis_url: redis://localhost:6379
+          # Always "resque:gitlab".
+          :namespace: resque:gitlab
+          # Always "incoming_email".
+          :queue: incoming_email
+          # Always "EmailReceiverWorker"
+          :worker: EmailReceiverWorker
+    ```
+
+5. Edit the init script configuration at `/etc/default/gitlab` to enable `mail_room`:
+
+    ```sh
+    sudo mkdir -p /etc/default
+    echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab
+    ```
+
+6. Restart GitLab:
+
+    ```sh
+    sudo service gitlab restart
+    ```
+
+7. Verify that everything is configured correctly:
+
+    ```sh
+    sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production
+    ```
+
+8. Reply by email should now be working.
+
+### Development
+
+1. Go to the GitLab installation directory.
+
+1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `key`:
+
+    ```yaml
+    incoming_email:
+      enabled: true
+      address: "gitlab-incoming+%{key}@gmail.com"
+    ```
+
+    As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`.
+
+2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`:
+
+    ```sh
+    sudo cp config/mail_room.yml.example config/mail_room.yml
+    ```
+
+3. Uncomment the configuration options in `config/mail_room.yml` and fill in the details for your specific IMAP server and email account:
+
+    ```yaml
+    :mailboxes:
+      -
+        # IMAP server host
+        :host: "imap.gmail.com"
+        # IMAP server port
+        :port: 993
+        # Whether the IMAP server uses SSL
+        :ssl: true
+        # Email account username. Usually the full email address.
+        :email: "gitlab-incoming@gmail.com"
+        # Email account password
+        :password: "[REDACTED]"
+        # The name of the mailbox where incoming mail will end up. Usually "inbox".
+        :name: "inbox"
+        # Always "sidekiq".
+        :delivery_method: sidekiq
+        # Always true.
+        :delete_after_delivery: true
+        :delivery_options:
+          # The URL to the Redis server used by Sidekiq. Should match the URL in config/resque.yml.
+          :redis_url: redis://localhost:6379
+          # Always "resque:gitlab".
+          :namespace: resque:gitlab
+          # Always "incoming_email".
+          :queue: incoming_email
+          # Always "EmailReceiverWorker"
+          :worker: EmailReceiverWorker
+    ```
+
+4. Uncomment the `mail_room` line in your `Procfile`:
+
+    ```yaml
+    mail_room: bundle exec mail_room -q -c config/mail_room.yml
+    ```
+
+6. Restart GitLab:
+
+    ```sh
+    bundle exec foreman start
+    ```
+
+7. Verify that everything is configured correctly:
+
+    ```sh
+    bundle exec rake gitlab:incoming_email:check RAILS_ENV=development
+    ```
+
+8. Reply by email should now be working.
diff --git a/doc/incoming_email/postfix.md b/doc/incoming_email/postfix.md
new file mode 100644
index 00000000000..09c5804cb77
--- /dev/null
+++ b/doc/incoming_email/postfix.md
@@ -0,0 +1,310 @@
+# Set up Postfix for Reply by email
+
+This document will take you through the steps of setting up a basic Postfix mail server with IMAP authentication on Ubuntu, to be used with Reply by email.
+
+The instructions make the assumption that you will be using the email address `incoming@gitlab.example.com`, that is, username `incoming` on host `gitlab.example.com`. Don't forget to change it to your actual host when executing the example code snippets.
+
+## Configure your server firewall
+
+1. Open up port 25 on your server so that people can send email into the server over SMTP.
+2. If the mail server is different from the server running GitLab, open up port 143 on your server so that GitLab can read email from the server over IMAP.
+
+## Install packages
+
+1. Install the `postfix` package if it is not installed already:
+
+    ```sh
+    sudo apt-get install postfix
+    ```
+
+    When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`.
+
+1. Install the `mailutils` package.
+
+    ```sh
+    sudo apt-get install mailutils
+    ```
+
+## Create user
+
+1. Create a user for incoming.
+
+    ```sh
+    sudo useradd -m -s /bin/bash incoming
+    ```
+
+1. Set a password for this user.
+
+    ```sh
+    sudo passwd incoming
+    ```
+
+    Be sure not to forget this, you'll need it later.
+
+## Test the out-of-the-box setup
+
+1. Connect to the local SMTP server:
+    
+    ```sh
+    telnet localhost 25
+    ```
+
+    You should see a prompt like this:
+
+    ```sh
+    Trying 127.0.0.1...
+    Connected to localhost.
+    Escape character is '^]'.
+    220 gitlab.example.com ESMTP Postfix (Ubuntu)
+    ```
+
+    If you get a `Connection refused` error instead, verify that `postfix` is running:
+
+    ```sh
+    sudo postfix status
+    ```
+
+    If it is not, start it:
+
+    ```sh
+    sudo postfix start
+    ```
+
+1. Send the new `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+    
+    ```
+    ehlo localhost
+    mail from: root@localhost
+    rcpt to: incoming@localhost
+    data
+    Subject: Re: Some issue
+
+    Sounds good!
+    .
+    quit
+    ```
+
+    (Note: The `.` is a literal period on its own line)
+
+1. Check if the `incoming` user received the email:
+    
+    ```sh
+    su - incoming
+    mail
+    ```
+
+    You should see output like this:
+
+    ```
+    "/var/mail/incoming": 1 message 1 unread
+    >U   1 root@localhost                           59/2842  Re: Some issue
+    ```
+
+    Quit the mail app:
+
+    ```sh
+    q
+    ```
+
+1. Log out of the `incoming` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Configure Postfix to use Maildir-style mailboxes
+
+Courier, which we will install later to add IMAP authentication, requires mailboxes to have the Maildir format, rather than mbox.
+
+1. Configure Postfix to use Maildir-style mailboxes:
+    
+    ```sh
+    sudo postconf -e "home_mailbox = Maildir/"
+    ```
+
+1. Restart Postfix:
+    
+    ```sh
+    sudo /etc/init.d/postfix restart
+    ```
+
+1. Test the new setup:
+    
+    1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_.
+    2. Check if the `incoming` user received the email:
+    
+        ```sh
+        su - incoming
+        MAIL=/home/incoming/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/incoming/Maildir": 1 message 1 unread
+        >U   1 root@localhost                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+1. Log out of the `incoming` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Install the Courier IMAP server
+    
+1. Install the `courier-imap` package:
+
+    ```sh
+    sudo apt-get install courier-imap
+    ```
+
+## Configure Postfix to receive email from the internet
+
+1. Let Postfix know about the domains that it should consider local:
+    
+    ```sh
+    sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost"
+    ```
+
+1. Let Postfix know about the IPs that it should consider part of the LAN:
+    
+    We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network.
+    
+    ```sh
+    sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24"
+    ```
+
+1. Configure Postfix to receive mail on all interfaces, which includes the internet:
+    
+    ```sh
+    sudo postconf -e "inet_interfaces = all"
+    ```
+
+1. Configure Postfix to use the `+` delimiter for sub-addressing:
+    
+    ```sh
+    sudo postconf -e "recipient_delimiter = +"
+    ```
+
+1. Restart Postfix:
+    
+    ```sh
+    sudo service postfix restart
+    ```
+
+## Test the final setup
+
+1. Test SMTP under the new setup:
+    
+    1. Connect to the SMTP server:
+        
+        ```sh
+        telnet gitlab.example.com 25
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to gitlab.example.com.
+        Escape character is '^]'.
+        220 gitlab.example.com ESMTP Postfix (Ubuntu)
+        ```
+
+        If you get a `Connection refused` error instead, make sure your firewall is setup to allow inbound traffic on port 25.
+
+    1. Send the `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+        
+        ```
+        ehlo gitlab.example.com
+        mail from: root@gitlab.example.com
+        rcpt to: incoming@gitlab.example.com
+        data
+        Subject: Re: Some issue
+
+        Sounds good!
+        .
+        quit
+        ```
+
+        (Note: The `.` is a literal period on its own line)
+
+    1. Check if the `incoming` user received the email:
+    
+        ```sh
+        su - incoming
+        MAIL=/home/incoming/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/incoming/Maildir": 1 message 1 unread
+        >U   1 root@gitlab.example.com                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+    1. Log out of the `incoming` account and go back to being `root`:
+
+        ```sh
+        logout
+        ```
+
+1. Test IMAP under the new setup:
+    
+    1. Connect to the IMAP server:
+        
+        ```sh
+        telnet gitlab.example.com 143
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to mail.example.gitlab.com.
+        Escape character is '^]'.
+        - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc.  See COPYING for distribution information.
+        ```
+
+    1. Sign in as the `incoming` user to test IMAP, by entering the following into the IMAP prompt:
+
+        ```
+        a login incoming PASSWORD
+        ```
+
+        Replace PASSWORD with the password you set on the `incoming` user earlier.
+
+        You should see output like this:
+
+        ```
+        a OK LOGIN Ok.
+        ```
+
+    1. Disconnect from the IMAP server:
+
+        ```sh
+        a logout
+        ```
+
+## Done!
+
+If all the tests were successful, Postfix is all set up and ready to receive email! Continue with the [Reply by email](./README.md) guide to configure GitLab.
+
+---------
+
+_This document was adapted from https://help.ubuntu.com/community/PostfixBasicSetupHowto, by contributors to the Ubuntu documentation wiki._