diff options
author | Lin Jen-Shin <godfat@godfat.org> | 2018-01-12 19:43:38 +0800 |
---|---|---|
committer | Lin Jen-Shin <godfat@godfat.org> | 2018-01-12 19:43:38 +0800 |
commit | cfd75101d19db3235b64b05d7a58616db40f22c6 (patch) | |
tree | 6eef1dd8bd6a1ddc8f69ffdf35bf2f9b6771c30c /doc | |
parent | f4bd9c0b5e1eafe6de855d73bfb606909229f382 (diff) | |
parent | f9579df8617add53424f57c0feedfa601a77e923 (diff) | |
download | gitlab-ce-cfd75101d19db3235b64b05d7a58616db40f22c6.tar.gz |
Merge remote-tracking branch 'upstream/master' into 1819-override-ce
* upstream/master: (621 commits)
Add a note about GitLab QA page objects validator to docs
Refactor dispatcher projects blame and blob path
Update export message to mention we can download the file from the UI
Fix Ctrl+Enter keyboard shortcut saving comment/note edit
fix case where tooltip messes up :last-child selector
Add reason to keep postgresql 9.2 for CI
Remove warning noise in ProjectImportOptions
Add changelog entry
Add RedirectRoute factory
Update Ingress extra cost note to be more generic
Fix Rubocop offense
Refactor dispatcher project branches path
Revert "Revert "Fix Route validation for unchanged path""
Document that we need rsync for backing up
Docs: move article "Laravel and Envoy w/ CI/CD"
Recommend against the use of EFS
Adds Rubocop rule for line break around conditionals
Update CHANGELOG.md for 10.1.6
Filter out build traces from logged parameters
Refactored project:n* imports in dispatcher.js
...
Diffstat (limited to 'doc')
-rw-r--r-- | doc/administration/auth/crowd.md | 12 | ||||
-rw-r--r-- | doc/administration/auth/img/crowd_application_authorisation.png | bin | 0 -> 126994 bytes | |||
-rw-r--r-- | doc/administration/high_availability/nfs.md | 4 | ||||
-rw-r--r-- | doc/administration/integration/plantuml.md | 18 | ||||
-rw-r--r-- | doc/administration/operations/fast_ssh_key_lookup.md | 170 | ||||
-rw-r--r-- | doc/administration/operations/img/write_to_authorized_keys_setting.png | bin | 0 -> 94218 bytes | |||
-rw-r--r-- | doc/administration/operations/index.md | 3 | ||||
-rw-r--r-- | doc/administration/operations/speed_up_ssh.md | 1 | ||||
-rw-r--r-- | doc/administration/raketasks/check.md | 46 | ||||
-rw-r--r-- | doc/api/boards.md | 115 | ||||
-rw-r--r-- | doc/api/commits.md | 1 | ||||
-rw-r--r-- | doc/api/issues.md | 39 | ||||
-rw-r--r-- | doc/api/merge_requests.md | 59 | ||||
-rw-r--r-- | doc/api/pages_domains.md | 1 | ||||
-rw-r--r-- | doc/api/pipeline_triggers.md | 5 | ||||
-rw-r--r-- | doc/api/repositories.md | 2 | ||||
-rw-r--r-- | doc/api/runners.md | 40 | ||||
-rw-r--r-- | doc/api/services.md | 3 | ||||
-rw-r--r-- | doc/api/settings.md | 2 | ||||
-rw-r--r-- | doc/api/snippets.md | 17 | ||||
-rw-r--r-- | doc/articles/how_to_install_git/index.md | 67 | ||||
-rw-r--r-- | doc/articles/index.md | 14 | ||||
-rw-r--r-- | doc/articles/laravel_with_gitlab_and_envoy/index.md | 681 | ||||
-rw-r--r-- | doc/articles/openshift_and_gitlab/index.md | 511 | ||||
-rw-r--r-- | doc/ci/README.md | 2 | ||||
-rw-r--r-- | doc/ci/examples/README.md | 1 | ||||
-rw-r--r-- | doc/ci/examples/code_climate.md | 3 | ||||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png) | bin | 4730 -> 4730 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png) | bin | 56091 -> 56091 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg) | bin | 93531 -> 93531 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png) | bin | 339666 -> 339666 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/environment_page.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png) | bin | 185393 -> 185393 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/environments_page.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png) | bin | 134742 -> 134742 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png) | bin | 5785 -> 5785 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png) | bin | 177704 -> 177704 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipeline_page.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png) | bin | 172664 -> 172664 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png) | bin | 119955 -> 119955 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png) | bin | 141393 -> 141393 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png) | bin | 11082 -> 11082 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png) | bin | 21993 -> 21993 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/img/secret_variables_page.png (renamed from doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png) | bin | 233764 -> 233764 bytes | |||
-rw-r--r-- | doc/ci/examples/laravel_with_gitlab_and_envoy/index.md | 684 | ||||
-rw-r--r-- | doc/ci/ssh_keys/README.md | 2 | ||||
-rw-r--r-- | doc/development/README.md | 1 | ||||
-rw-r--r-- | doc/development/architecture.md | 2 | ||||
-rw-r--r-- | doc/development/changelog.md | 2 | ||||
-rw-r--r-- | doc/development/fe_guide/style_guide_js.md | 37 | ||||
-rw-r--r-- | doc/development/fe_guide/style_guide_scss.md | 2 | ||||
-rw-r--r-- | doc/development/gitaly.md | 23 | ||||
-rw-r--r-- | doc/development/testing_guide/end_to_end_tests.md | 80 | ||||
-rw-r--r-- | doc/development/testing_guide/index.md | 8 | ||||
-rw-r--r-- | doc/development/testing_guide/testing_levels.md | 3 | ||||
-rw-r--r-- | doc/development/utilities.md | 92 | ||||
-rw-r--r-- | doc/development/what_requires_downtime.md | 57 | ||||
-rw-r--r-- | doc/development/writing_documentation.md | 4 | ||||
-rw-r--r-- | doc/install/README.md | 2 | ||||
-rw-r--r-- | doc/install/installation.md | 6 | ||||
-rw-r--r-- | doc/install/kubernetes/index.md | 4 | ||||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/add-gitlab-to-project.png (renamed from doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png) | bin | 37386 -> 37386 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/add-to-project.png (renamed from doc/articles/openshift_and_gitlab/img/add-to-project.png) | bin | 21672 -> 21672 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/create-project-ui.png (renamed from doc/articles/openshift_and_gitlab/img/create-project-ui.png) | bin | 22290 -> 22290 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/gitlab-logs.png (renamed from doc/articles/openshift_and_gitlab/img/gitlab-logs.png) | bin | 70858 -> 70858 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/gitlab-overview.png (renamed from doc/articles/openshift_and_gitlab/img/gitlab-overview.png) | bin | 106432 -> 106432 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/gitlab-running.png (renamed from doc/articles/openshift_and_gitlab/img/gitlab-running.png) | bin | 107993 -> 107993 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/gitlab-scale.png (renamed from doc/articles/openshift_and_gitlab/img/gitlab-scale.png) | bin | 36628 -> 36628 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/gitlab-settings.png (renamed from doc/articles/openshift_and_gitlab/img/gitlab-settings.png) | bin | 111366 -> 111366 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/no-resources.png (renamed from doc/articles/openshift_and_gitlab/img/no-resources.png) | bin | 34669 -> 34669 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/openshift-infra-project.png (renamed from doc/articles/openshift_and_gitlab/img/openshift-infra-project.png) | bin | 95725 -> 95725 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/pods-overview.png (renamed from doc/articles/openshift_and_gitlab/img/pods-overview.png) | bin | 106861 -> 106861 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/rc-name.png (renamed from doc/articles/openshift_and_gitlab/img/rc-name.png) | bin | 51390 -> 51390 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/running-pods.png (renamed from doc/articles/openshift_and_gitlab/img/running-pods.png) | bin | 29818 -> 29818 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/storage-volumes.png (renamed from doc/articles/openshift_and_gitlab/img/storage-volumes.png) | bin | 49584 -> 49584 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/img/web-console.png (renamed from doc/articles/openshift_and_gitlab/img/web-console.png) | bin | 34774 -> 34774 bytes | |||
-rw-r--r-- | doc/install/openshift_and_gitlab/index.md | 510 | ||||
-rw-r--r-- | doc/raketasks/backup_restore.md | 21 | ||||
-rw-r--r-- | doc/topics/autodevops/index.md | 15 | ||||
-rw-r--r-- | doc/topics/git/how_to_install_git/index.md | 66 | ||||
-rw-r--r-- | doc/topics/git/index.md | 65 | ||||
-rw-r--r-- | doc/university/high-availability/aws/README.md | 4 | ||||
-rw-r--r-- | doc/university/high-availability/aws/img/reference-arch.png | bin | 0 -> 183997 bytes | |||
-rw-r--r-- | doc/update/10.3-to-10.4.md | 2 | ||||
-rw-r--r-- | doc/user/permissions.md | 19 | ||||
-rw-r--r-- | doc/user/project/clusters/index.md | 17 | ||||
-rw-r--r-- | doc/user/project/integrations/irker.md | 6 | ||||
-rw-r--r-- | doc/user/project/integrations/jira.md | 5 | ||||
-rw-r--r-- | doc/user/project/integrations/kubernetes.md | 5 | ||||
-rw-r--r-- | doc/user/project/integrations/project_services.md | 2 | ||||
-rw-r--r-- | doc/user/project/integrations/redmine.md | 7 | ||||
-rw-r--r-- | doc/user/project/integrations/webhooks.md | 6 | ||||
-rw-r--r-- | doc/user/project/merge_requests/fast_forward_merge.md | 4 | ||||
-rw-r--r-- | doc/user/project/merge_requests/img/ff_merge_mr.png | bin | 21380 -> 0 bytes | |||
-rw-r--r-- | doc/user/project/merge_requests/img/ff_merge_rebase.png | bin | 0 -> 26945 bytes | |||
-rw-r--r-- | doc/user/project/quick_actions.md | 2 | ||||
-rw-r--r-- | doc/user/project/settings/import_export.md | 3 |
94 files changed, 2165 insertions, 1420 deletions
diff --git a/doc/administration/auth/crowd.md b/doc/administration/auth/crowd.md index 2c289c67a6d..6db74958d5a 100644 --- a/doc/administration/auth/crowd.md +++ b/doc/administration/auth/crowd.md @@ -66,3 +66,15 @@ On the sign in page there should now be a Crowd tab in the sign in form. [reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure [restart]: ../restart_gitlab.md#installations-from-source + +## Troubleshooting + +If you see an error message like the one below when you sign in after Crowd authentication is configured, you may want to consult the Crowd administrator for the Crowd log file to know the exact cause: + +``` +could not authorize you from Crowd because invalid credentials +``` + +Please make sure the Crowd users who need to login to GitLab are authorized to [the application](#configure-a-new-crowd-application) in the step of **Authorisation**. This could be verified by try "Authentication test" for Crowd as of 2.11. + +![Example Crowd application authorisation configuration](img/crowd_application_authorisation.png)
\ No newline at end of file diff --git a/doc/administration/auth/img/crowd_application_authorisation.png b/doc/administration/auth/img/crowd_application_authorisation.png Binary files differnew file mode 100644 index 00000000000..70339891b34 --- /dev/null +++ b/doc/administration/auth/img/crowd_application_authorisation.png diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index e09ccaba08c..d8928a7fe4c 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -32,7 +32,9 @@ options: ## AWS Elastic File System -GitLab does not recommend using AWS Elastic File System (EFS). +GitLab strongly recommends against using AWS Elastic File System (EFS). +Our support team will not be able to assist on performance issues related to +file system access. Customers and users have reported that AWS EFS does not perform well for GitLab's use-case. There are several issues that can cause problems. For these reasons diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 93c3642a1f1..65f59b72690 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -58,30 +58,32 @@ our AsciiDoc snippets, wikis and repos using delimited blocks: - **Markdown** - ```plantuml - Bob -> Alice : hello - Alice -> Bob : Go Away - ``` + <pre> + ```plantuml + Bob -> Alice : hello + Alice -> Bob : Go Away + ``` + </pre> - **AsciiDoc** - ``` + <pre> [plantuml, format="png", id="myDiagram", width="200px"] -- Bob->Alice : hello Alice -> Bob : Go Away -- - ``` + </pre> - **reStructuredText** - ``` + <pre> .. plantuml:: :caption: Caption with **bold** and *italic* Bob -> Alice: hello Alice -> Bob: Go Away - ``` + </pre> You can also use the `uml::` directive for compatibility with [sphinxcontrib-plantuml](https://pypi.python.org/pypi/sphinxcontrib-plantuml), but please note that we currently only support the `caption` option. diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md new file mode 100644 index 00000000000..835ed8c8006 --- /dev/null +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -0,0 +1,170 @@ +# Fast lookup of authorized SSH keys in the database + +Regular SSH operations become slow as the number of users grows because OpenSSH +searches for a key to authorize a user via a linear search. In the worst case, +such as when the user is not authorized to access GitLab, OpenSSH will scan the +entire file to search for a key. This can take significant time and disk I/O, +which will delay users attempting to push or pull to a repository. Making +matters worse, if users add or remove keys frequently, the operating system may +not be able to cache the `authorized_keys` file, which causes the disk to be +accessed repeatedly. + +GitLab Shell solves this by providing a way to authorize SSH users via a fast, +indexed lookup in the GitLab database. This page describes how to enable the fast +lookup of authorized SSH keys. + +> **Warning:** OpenSSH version 6.9+ is required because +`AuthorizedKeysCommand` must be able to accept a fingerprint. These +instructions will break installations using older versions of OpenSSH, such as +those included with CentOS 6 as of September 2017. If you want to use this +feature for CentOS 6, follow [the instructions on how to build and install a custom OpenSSH package](#compiling-a-custom-version-of-openssh-for-centos-6) before continuing. + +## Setting up fast lookup via GitLab Shell + +GitLab Shell provides a way to authorize SSH users via a fast, indexed lookup +to the GitLab database. GitLab Shell uses the fingerprint of the SSH key to +check whether the user is authorized to access GitLab. + +Add the following to your `sshd_config` file. This is usuaully located at +`/etc/ssh/sshd_config`, but it will be `/assets/sshd_config` if you're using +Omnibus Docker: + +``` +AuthorizedKeysCommand /opt/embedded/gitlab-shell/bin/gitlab-shell-authorized-keys-check git %u %k +AuthorizedKeysCommandUser git +``` + +Reload OpenSSH: + +```bash +# Debian or Ubuntu installations +sudo service ssh reload + +# CentOS installations +sudo service sshd reload +``` + +Confirm that SSH is working by removing your user's SSH key in the UI, adding a +new one, and attempting to pull a repo. + +> **Warning:** Do not disable writes until SSH is confirmed to be working +perfectly, because the file will quickly become out-of-date. + +In the case of lookup failures (which are not uncommon), the `authorized_keys` +file will still be scanned. So git SSH performance will still be slow for many +users as long as a large file exists. + +You can disable any more writes to the `authorized_keys` file by unchecking +`Write to "authorized_keys" file` in the Application Settings of your GitLab +installation. + +![Write to authorized keys setting](img/write_to_authorized_keys_setting.png) + +Again, confirm that SSH is working by removing your user's SSH key in the UI, +adding a new one, and attempting to pull a repo. + +Then you can backup and delete your `authorized_keys` file for best performance. + +## How to go back to using the `authorized_keys` file + +This is a brief overview. Please refer to the above instructions for more context. + +1. [Rebuild the `authorized_keys` file](../raketasks/maintenance.md#rebuild-authorized_keys-file) +1. Enable writes to the `authorized_keys` file in Application Settings +1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` or from `/assets/sshd_config` if you are using Omnibus Docker. +1. Reload sshd: `sudo service sshd reload` +1. Remove the `/opt/gitlab-shell/authorized_keys` file + +## Compiling a custom version of OpenSSH for CentOS 6 + +Building a custom version of OpenSSH is not necessary for Ubuntu 16.04 users, +since Ubuntu 16.04 ships with OpenSSH 7.2. + +It is also unnecessary for CentOS 7.4 users, as that version ships with +OpenSSH 7.4. If you are using CentOS 7.0 - 7.3, we strongly recommend that you +upgrade to CentOS 7.4 instead of following this procedure. This should be as +simple as running `yum update`. + +CentOS 6 users must build their own OpenSSH package to enable SSH lookups via +the database. The following instructions can be used to build OpenSSH 7.5: + +1. First, download the package and install the required packages: + + ``` + sudo su - + cd /tmp + curl --remote-name https://mirrors.evowise.com/pub/OpenBSD/OpenSSH/portable/openssh-7.5p1.tar.gz + tar xzvf openssh-7.5p1.tar.gz + yum install rpm-build gcc make wget openssl-devel krb5-devel pam-devel libX11-devel xmkmf libXt-devel + ``` + +3. Prepare the build by copying files to the right place: + + ``` + mkdir -p /root/rpmbuild/{SOURCES,SPECS} + cp ./openssh-7.5p1/contrib/redhat/openssh.spec /root/rpmbuild/SPECS/ + cp openssh-7.5p1.tar.gz /root/rpmbuild/SOURCES/ + cd /root/rpmbuild/SPECS + ``` + +3. Next, set the spec settings properly: + + ``` + sed -i -e "s/%define no_gnome_askpass 0/%define no_gnome_askpass 1/g" openssh.spec + sed -i -e "s/%define no_x11_askpass 0/%define no_x11_askpass 1/g" openssh.spec + sed -i -e "s/BuildPreReq/BuildRequires/g" openssh.spec + ``` + +3. Build the RPMs: + + ``` + rpmbuild -bb openssh.spec + ``` + +4. Ensure the RPMs were built: + + ``` + ls -al /root/rpmbuild/RPMS/x86_64/ + ``` + + You should see something as the following: + + ``` + total 1324 + drwxr-xr-x. 2 root root 4096 Jun 20 19:37 . + drwxr-xr-x. 3 root root 19 Jun 20 19:37 .. + -rw-r--r--. 1 root root 470828 Jun 20 19:37 openssh-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 490716 Jun 20 19:37 openssh-clients-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 17020 Jun 20 19:37 openssh-debuginfo-7.5p1-1.x86_64.rpm + -rw-r--r--. 1 root root 367516 Jun 20 19:37 openssh-server-7.5p1-1.x86_64.rpm + ``` + +5. Install the packages. OpenSSH packages will replace `/etc/pam.d/sshd` + with its own version, which may prevent users from logging in, so be sure + that the file is backed up and restored after installation: + + ``` + timestamp=$(date +%s) + cp /etc/pam.d/sshd pam-ssh-conf-$timestamp + rpm -Uvh /root/rpmbuild/RPMS/x86_64/*.rpm + yes | cp pam-ssh-conf-$timestamp /etc/pam.d/sshd + ``` + +6. Verify the installed version. In another window, attempt to login to the server: + + ``` + ssh -v <your-centos-machine> + ``` + + You should see a line that reads: "debug1: Remote protocol version 2.0, remote software version OpenSSH_7.5" + + If not, you may need to restart sshd (e.g. `systemctl restart sshd.service`). + +7. *IMPORTANT!* Open a new SSH session to your server before exiting to make + sure everything is working! If you need to downgrade, simple install the + older package: + + ``` + # Only run this if you run into a problem logging in + yum downgrade openssh-server openssh openssh-clients + ``` diff --git a/doc/administration/operations/img/write_to_authorized_keys_setting.png b/doc/administration/operations/img/write_to_authorized_keys_setting.png Binary files differnew file mode 100644 index 00000000000..232765f1917 --- /dev/null +++ b/doc/administration/operations/img/write_to_authorized_keys_setting.png diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index 320d71a9527..5655b7efec6 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -13,4 +13,5 @@ by GitLab to another file system or another server. that to prioritize important jobs. - [Sidekiq MemoryKiller](sidekiq_memory_killer.md): Configure Sidekiq MemoryKiller to restart Sidekiq. -- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer.
\ No newline at end of file +- [Unicorn](unicorn.md): Understand Unicorn and unicorn-worker-killer. +- [Speed up SSH operations](fast_ssh_key_lookup.md): Authorize SSH users via a fast, indexed lookup to the GitLab database. diff --git a/doc/administration/operations/speed_up_ssh.md b/doc/administration/operations/speed_up_ssh.md new file mode 100644 index 00000000000..89265b3018b --- /dev/null +++ b/doc/administration/operations/speed_up_ssh.md @@ -0,0 +1 @@ +This document was moved to [another location](fast_ssh_key_lookup.md). diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index c8b5434c068..d1ed152b58c 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -28,19 +28,25 @@ exactly which repositories are causing the trouble. ### Check all GitLab repositories +>**Note:** +> +> - `gitlab:repo:check` has been deprecated in favor of `gitlab:git:fsck` +> - [Deprecated][ce-15931] in GitLab 10.4. +> - `gitlab:repo:check` will be removed in the future. [Removal issue][ce-41699] + This task loops through all repositories on the GitLab server and runs the 3 integrity checks described previously. **Omnibus Installation** ``` -sudo gitlab-rake gitlab:repo:check +sudo gitlab-rake gitlab:git:fsck ``` **Source Installation** ```bash -sudo -u git -H bundle exec rake gitlab:repo:check RAILS_ENV=production +sudo -u git -H bundle exec rake gitlab:git:fsck RAILS_ENV=production ``` ### Check repositories for a specific user @@ -70,9 +76,45 @@ Example output: ![gitlab:user:check_repos output](../img/raketasks/check_repos_output.png) +## Uploaded Files Integrity + +The uploads check Rake task will loop through all uploads in the database +and run two checks to determine the integrity of each file: + +1. Check if the file exist on the file system. +1. Check if the checksum of the file on the file system matches the checksum in the database. + +**Omnibus Installation** + +``` +sudo gitlab-rake gitlab:uploads:check +``` + +**Source Installation** + +```bash +sudo -u git -H bundle exec rake gitlab:uploads:check RAILS_ENV=production +``` + +This task also accepts some environment variables which you can use to override +certain values: + +Variable | Type | Description +-------- | ---- | ----------- +`BATCH` | integer | Specifies the size of the batch. Defaults to 200. +`ID_FROM` | integer | Specifies the ID to start from, inclusive of the value. +`ID_TO` | integer | Specifies the ID value to end at, inclusive of the value. + +```bash +sudo gitlab-rake gitlab:uploads:check BATCH=100 ID_FROM=50 ID_TO=250 +``` + ## LDAP Check The LDAP check Rake task will test the bind_dn and password credentials (if configured) and will list a sample of LDAP users. This task is also executed as part of the `gitlab:check` task, but can run independently. See [LDAP Rake Tasks - LDAP Check](ldap.md#check) for details. + +[ce-15931]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15931 +[ce-41699]: https://gitlab.com/gitlab-org/gitlab-ce/issues/41699 diff --git a/doc/api/boards.md b/doc/api/boards.md index 69c47abc806..246de50323e 100644 --- a/doc/api/boards.md +++ b/doc/api/boards.md @@ -15,10 +15,10 @@ GET /projects/:id/boards | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | ```bash -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/:id/boards +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards ``` Example response: @@ -27,6 +27,19 @@ Example response: [ { "id" : 1, + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site" + }, + "milestone": { + "id": 12 + "title": "10.0" + }, "lists" : [ { "id" : 1, @@ -60,6 +73,74 @@ Example response: ] ``` +## Single board + +Get a single board. + +``` +GET /projects/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1 +``` + +Example response: + +```json + { + "id": 1, + "name:": "project issue board", + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site" + }, + "milestone": { + "id": 12 + "title": "10.0" + }, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +``` + ## List board lists Get a list of the board's lists. @@ -71,8 +152,8 @@ GET /projects/:id/boards/:board_id/lists | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists @@ -122,9 +203,9 @@ GET /projects/:id/boards/:board_id/lists/:list_id | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `list_id`| integer | yes | The ID of a board's list | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id`| integer | yes | The ID of a board's list | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1 @@ -154,9 +235,9 @@ POST /projects/:id/boards/:board_id/lists | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `label_id` | integer | yes | The ID of a label | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `label_id` | integer | yes | The ID of a label | ```bash curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists?label_id=5 @@ -186,10 +267,10 @@ PUT /projects/:id/boards/:board_id/lists/:list_id | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `list_id` | integer | yes | The ID of a board's list | -| `position` | integer | yes | The position of the list | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | +| `position` | integer | yes | The position of the list | ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1?position=2 @@ -219,9 +300,9 @@ DELETE /projects/:id/boards/:board_id/lists/:list_id | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `board_id` | integer | yes | The ID of a board | -| `list_id` | integer | yes | The ID of a board's list | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | ```bash curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1 diff --git a/doc/api/commits.md b/doc/api/commits.md index c9b72d4a1dd..63554c63057 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -159,6 +159,7 @@ Parameters: | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | `sha` | string | yes | The commit hash or name of a repository branch or tag | +| `stats` | boolean | no | Include commit stats. Default is true | ```bash curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/5/repository/commits/master diff --git a/doc/api/issues.md b/doc/api/issues.md index d2fefbe68aa..da89db17cd9 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -1124,6 +1124,45 @@ Example response: ``` +## Participants on issues + +``` +GET /projects/:id/issues/:issue_iid/participants +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `issue_iid` | integer | yes | The internal ID of a project's issue | + +```bash +curl --request GET --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/93/participants +``` + +Example response: + +```json +[ + { + "id": 1, + "name": "John Doe1", + "username": "user1", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", + "web_url": "http://localhost/user1" + }, + { + "id": 5, + "name": "John Doe5", + "username": "user5", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/4aea8cf834ed91844a2da4ff7ae6b491?s=80&d=identicon", + "web_url": "http://localhost/user5" + } +] +``` + + ## Comments on issues Comments are done via the [notes](notes.md) resource. diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 4d3592e8f71..22ccc6a46f3 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -308,6 +308,41 @@ Parameters: } ``` +## Get single MR participants + +Get a list of merge request participants. + +``` +GET /projects/:id/merge_requests/:merge_request_iid/participants +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `merge_request_iid` (required) - The internal ID of the merge request + + +```json +[ + { + "id": 1, + "name": "John Doe1", + "username": "user1", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/c922747a93b40d1ea88262bf1aebee62?s=80&d=identicon", + "web_url": "http://localhost/user1" + }, + { + "id": 2, + "name": "John Doe2", + "username": "user2", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/10fc7f102be8de7657fb4d80898bbfe3?s=80&d=identicon", + "web_url": "http://localhost/user2" + }, +] +``` + ## Get single MR commits Get a list of merge request commits. @@ -433,6 +468,30 @@ Parameters: } ``` +## List MR pipelines + +Get a list of merge request pipelines. + +``` +GET /projects/:id/merge_requests/:merge_request_iid/pipelines +``` + +Parameters: + +- `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user +- `merge_request_iid` (required) - The internal ID of the merge request + +```json +[ + { + "id": 77, + "sha": "959e04d7c7a30600c894bd3c0cd0e1ce7f42c11d", + "ref": "master", + "status": "success" + } +] +``` + ## Create MR Creates a new merge request. diff --git a/doc/api/pages_domains.md b/doc/api/pages_domains.md index 50685f335f7..20275b902c6 100644 --- a/doc/api/pages_domains.md +++ b/doc/api/pages_domains.md @@ -21,6 +21,7 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/a { "domain": "ssl.domain.example", "url": "https://ssl.domain.example", + "project_id": 1337, "certificate": { "expired": false, "expiration": "2020-04-12T14:32:00.000Z" diff --git a/doc/api/pipeline_triggers.md b/doc/api/pipeline_triggers.md index 9030ae32d17..e881e61d4ef 100644 --- a/doc/api/pipeline_triggers.md +++ b/doc/api/pipeline_triggers.md @@ -24,7 +24,6 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/ "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -55,7 +54,6 @@ curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/ "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -85,7 +83,6 @@ curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form descri "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -116,7 +113,6 @@ curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form descrip "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", @@ -146,7 +142,6 @@ curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitl "id": 10, "description": "my trigger", "created_at": "2016-01-07T09:53:58.235Z", - "deleted_at": null, "last_used": null, "token": "6d056f63e50fe6f8c5f8f4aa10edb7", "updated_at": "2016-01-07T09:53:58.235Z", diff --git a/doc/api/repositories.md b/doc/api/repositories.md index 03b32577872..5fb25e40ed7 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -113,7 +113,7 @@ GET /projects/:id/repository/archive Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `sha` (optional) - The commit SHA to download defaults to the tip of the default branch +- `sha` (optional) - The commit SHA to download. A tag, branch reference or sha can be used. This defaults to the tip of the default branch if not specified ## Compare branches, tags or commits diff --git a/doc/api/runners.md b/doc/api/runners.md index 015b09a745e..7495c6cdedb 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -30,14 +30,18 @@ Example response: "description": "test-1-20150125", "id": 6, "is_shared": false, - "name": null + "name": null, + "online": true, + "status": "online" }, { "active": true, "description": "test-2-20150125", "id": 8, "is_shared": false, - "name": null + "name": null, + "online": false, + "status": "offline" } ] ``` @@ -69,28 +73,36 @@ Example response: "description": "shared-runner-1", "id": 1, "is_shared": true, - "name": null + "name": null, + "online": true, + "status": "online" }, { "active": true, "description": "shared-runner-2", "id": 3, "is_shared": true, - "name": null + "name": null, + "online": false + "status": "offline" }, { "active": true, "description": "test-1-20150125", "id": 6, "is_shared": false, - "name": null + "name": null, + "online": true + "status": "paused" }, { "active": true, "description": "test-2-20150125", "id": 8, "is_shared": false, - "name": null + "name": null, + "online": false, + "status": "offline" } ] ``` @@ -122,6 +134,8 @@ Example response: "is_shared": false, "contacted_at": "2016-01-25T16:39:48.066Z", "name": null, + "online": true, + "status": "online", "platform": null, "projects": [ { @@ -176,6 +190,8 @@ Example response: "is_shared": false, "contacted_at": "2016-01-25T16:39:48.066Z", "name": null, + "online": true, + "status": "online", "platform": null, "projects": [ { @@ -327,14 +343,18 @@ Example response: "description": "test-2-20150125", "id": 8, "is_shared": false, - "name": null + "name": null, + "online": false, + "status": "offline" }, { "active": true, "description": "development_runner", "id": 5, "is_shared": true, - "name": null + "name": null, + "online": true + "status": "paused" } ] ``` @@ -364,7 +384,9 @@ Example response: "description": "test-2016-02-01", "id": 9, "is_shared": false, - "name": null + "name": null, + "online": true, + "status": "online" } ``` diff --git a/doc/api/services.md b/doc/api/services.md index 7e2afc71f9a..2928ab6cc75 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -562,6 +562,9 @@ DELETE /projects/:id/services/jira Kubernetes / Openshift integration +CAUTION: **Warning:** +Kubernetes service integration has been deprecated in GitLab 10.3. API service endpoints will continue to work as long as the Kubernetes service is active, however if the service is inactive API endpoints will automatically return a `400 Bad Request`. Read [GitLab 10.3 release post](https://about.gitlab.com/2017/12/22/gitlab-10-3-released/#kubernetes-integration-service) for more information. + ### Create/Edit Kubernetes service Set Kubernetes service for a project. diff --git a/doc/api/settings.md b/doc/api/settings.md index 0e4758cda2d..0b5b1f0c134 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -69,7 +69,7 @@ PUT /application/settings | `after_sign_up_text` | string | no | Text shown to the user after signing up | | `akismet_api_key` | string | no | API key for akismet spam protection | | `akismet_enabled` | boolean | no | Enable or disable akismet spam protection | -| `circuitbreaker_access_retries | integer | no | The number of attempts GitLab will make to access a storage. | +| `circuitbreaker_access_retries` | integer | no | The number of attempts GitLab will make to access a storage. | | `circuitbreaker_check_interval` | integer | no | Number of seconds in between storage checks. | | `circuitbreaker_failure_count_threshold` | integer | no | The number of failures of after which GitLab will completely prevent access to the storage. | | `circuitbreaker_failure_reset_time` | integer | no | Time in seconds GitLab will keep storage failure information. When no failures occur during this time, the failure information is reset. | diff --git a/doc/api/snippets.md b/doc/api/snippets.md index fdafbfb5b9e..42b760c107d 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -2,7 +2,7 @@ > [Introduced][ce-6373] in GitLab 8.15. -### Snippet visibility level +## Snippet visibility level Snippets in GitLab can be either private, internal, or public. You can set it with the `visibility` field in the snippet. @@ -84,7 +84,11 @@ Parameters: ``` bash -curl --request POST --data '{"title": "This is a snippet", "content": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/snippets +curl --request POST \ + --data '{"title": "This is a snippet", "content": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' \ + --header 'Content-Type: application/json' \ + --header "PRIVATE-TOKEN: valid_api_token" \ + https://gitlab.example.com/api/v4/snippets ``` Example response: @@ -131,7 +135,11 @@ Parameters: ``` bash -curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data '{"title": "foo", "content": "bar"}' https://gitlab.example.com/api/v4/snippets/1 +curl --request PUT \ + --data '{"title": "foo", "content": "bar"}' \ + --header 'Content-Type: application/json' \ + --header "PRIVATE-TOKEN: valid_api_token" \ + https://gitlab.example.com/api/v4/snippets/1 ``` Example response: @@ -265,4 +273,5 @@ Example response: } ``` -[ce-[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508]: https://gitlab.com/gitlab-org/gitlab-ce/issues/29508 +[ce-6373]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6373 +[ce-29508]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12655 diff --git a/doc/articles/how_to_install_git/index.md b/doc/articles/how_to_install_git/index.md index 37b60501ce2..3e6003a33b7 100644 --- a/doc/articles/how_to_install_git/index.md +++ b/doc/articles/how_to_install_git/index.md @@ -1,66 +1 @@ -# Installing Git - -> **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** user guide || -> **Level:** beginner || -> **Author:** [Sean Packham](https://gitlab.com/SeanPackham) || -> **Publication date:** 2017-05-15 - -To begin contributing to GitLab projects -you will need to install the Git client on your computer. -This article will show you how to install Git on macOS, Ubuntu Linux and Windows. - -## Install Git on macOS using the Homebrew package manager - -Although it is easy to use the version of Git shipped with macOS -or install the latest version of Git on macOS by downloading it from the project website, -we recommend installing it via Homebrew to get access to -an extensive selection of dependancy managed libraries and applications. - -If you are sure you don't need access to any additional development libraries -or don't have approximately 15gb of available disk space for Xcode and Homebrew -use one of the the aforementioned methods. - -### Installing Xcode - -Xcode is needed by Homebrew to build dependencies. -You can install [XCode](https://developer.apple.com/xcode/) -through the macOS App Store. - -### Installing Homebrew - -Once Xcode is installed browse to the [Homebrew website](http://brew.sh/index.html) -for the official Homebrew installation instructions. - -### Installing Git via Homebrew - -With Homebrew installed you are now ready to install Git. -Open a Terminal and enter in the following command: - -```bash -brew install git -``` - -Congratulations you should now have Git installed via Homebrew. -Next read our article on [adding an SSH key to GitLab](../../ssh/README.md). - -## Install Git on Ubuntu Linux - -On Ubuntu and other Linux operating systems -it is recommended to use the built in package manager to install Git. - -Open a Terminal and enter in the following commands -to install the latest Git from the official Git maintained package archives: - -```bash -sudo apt-add-repository ppa:git-core/ppa -sudo apt-get update -sudo apt-get install git -``` - -Congratulations you should now have Git installed via the Ubuntu package manager. -Next read our article on [adding an SSH key to GitLab](../../ssh/README.md). - -## Installing Git on Windows from the Git website - -Browse to the [Git website](https://git-scm.com/) and download and install Git for Windows. -Next read our article on [adding an SSH key to GitLab](../../ssh/README.md). +This document was moved to [another location](../../topics/git/how_to_install_git/index.md). diff --git a/doc/articles/index.md b/doc/articles/index.md index 06675e15d76..8385ef936c6 100644 --- a/doc/articles/index.md +++ b/doc/articles/index.md @@ -17,7 +17,6 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | Article title | Category | Publishing date | | :------------ | :------: | --------------: | | [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Admin guide | 2017-11-24 | -| [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 | | [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | | [Dockerizing GitLab Review Apps](https://about.gitlab.com/2017/07/11/dockerizing-review-apps/) | Concepts | 2017-07-11 | | [Continuous Integration: From Jenkins to GitLab Using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/) | Concepts | 2017-07-27 | @@ -30,18 +29,6 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM | [How to use GitLab CI and MacStadium to build your macOS or iOS projects](https://about.gitlab.com/2017/05/15/how-to-use-macstadium-and-gitlab-ci-to-build-your-macos-or-ios-projects/) | Technical overview | 2017-05-15 | | [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) | Tutorial | 2016-03-10 | -## Git - -Learn how to use [Git with GitLab](../topics/git/index.md): - -| Article title | Category | Publishing date | -| :------------ | :------: | --------------: | -| [Numerous _undo_ possibilities in Git](numerous_undo_possibilities_in_git/index.md) | Tutorial | 2017-08-17 | -| [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) | Concepts | 2017-05-17 | -| [How to install Git](how_to_install_git/index.md) | Tutorial | 2017-05-15 | -| [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) | Tutorial | 2017-01-30 | -| [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) | Technical overview | 2016-12-08 | - ## GitLab Pages Learn how to deploy a static website with [GitLab Pages](../user/project/pages/index.md#getting-started): @@ -74,7 +61,6 @@ upgrade, integrate, migrate to GitLab: | :------------ | :------: | --------------: | | [Video Tutorial: Idea to Production on Google Container Engine (GKE)](https://about.gitlab.com/2017/01/23/video-tutorial-idea-to-production-on-google-container-engine-gke/) | Tutorial | 2017-01-23 | | [How to Setup a GitLab Instance on Microsoft Azure](https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/) | Tutorial | 2016-07-13 | -| [Get started with OpenShift Origin 3 and GitLab](openshift_and_gitlab/index.md) | Tutorial | 2016-06-28 | | [Getting started with GitLab and DigitalOcean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/) | Tutorial | 2016-04-27 | ## Software development diff --git a/doc/articles/laravel_with_gitlab_and_envoy/index.md b/doc/articles/laravel_with_gitlab_and_envoy/index.md index b20bd8c247a..b092cdb0f7a 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/index.md +++ b/doc/articles/laravel_with_gitlab_and_envoy/index.md @@ -1,680 +1 @@ -# Test and deploy Laravel applications with GitLab CI/CD and Envoy - -> **[Article Type](../../development/writing_documentation.md#types-of-technical-articles):** tutorial || -> **Level:** intermediary || -> **Author:** [Mehran Rasulian](https://gitlab.com/mehranrasulian) || -> **Publication date:** 2017-08-31 - -## Introduction - -GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want. - -In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and setup our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../../ci/README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). - -We assume you have a basic experience with Laravel, Linux servers, -and you know how to use GitLab. - -Laravel is a high quality web framework written in PHP. -It has a great community with a [fantastic documentation](https://laravel.com/docs). -Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others. - -We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP. -It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to setup tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). - -## Initialize our Laravel app on GitLab - -We assume [you have installed a new laravel project](https://laravel.com/docs/installation#installation), so let's start with a unit test, and initialize Git for the project. - -### Unit Test - -Every new installation of Laravel (currently 5.4) comes with two type of tests, 'Feature' and 'Unit', placed in the tests directory. -Here's a unit test from `test/Unit/ExampleTest.php`: - -```php -<?php - -namespace Tests\Unit; - -... - -class ExampleTest extends TestCase -{ - public function testBasicTest() - { - $this->assertTrue(true); - } -} -``` - -This test is as simple as asserting that the given value is true. - -Laravel uses `PHPUnit` for tests by default. -If we run `vendor/bin/phpunit` we should see the green output: - -```bash -vendor/bin/phpunit -OK (1 test, 1 assertions) -``` - -This test will be used later for continuously testing our app with GitLab CI/CD. - -### Push to GitLab - -Since we have our app up and running locally, it's time to push the codebase to our remote repository. -Let's create [a new project](../../gitlab-basics/create-project.md) in GitLab named `laravel-sample`. -After that, follow the command line instructions displayed on the project's homepage to initiate the repository on our machine and push the first commit. - - -```bash -cd laravel-sample -git init -git remote add origin git@gitlab.example.com:<USERNAME>/laravel-sample.git -git add . -git commit -m 'Initial Commit' -git push -u origin master -``` - -## Configure the production server - -Before we begin setting up Envoy and GitLab CI/CD, let's quickly make sure the production server is ready for deployment. -We have installed LEMP stack which stands for Linux, Nginx, MySQL and PHP on our Ubuntu 16.04. - -### Create a new user - -Let's now create a new user that will be used to deploy our website and give it -the needed permissions using [Linux ACL](https://serversforhackers.com/video/linux-acls): - -```bash -# Create user deployer -sudo adduser deployer -# Give the read-write-execute permissions to deployer user for directory /var/www -sudo setfacl -R -m u:deployer:rwx /var/www -``` - -If you don't have ACL installed on your Ubuntu server, use this command to install it: - -```bash -sudo apt install acl -``` - -### Add SSH key - -Let's suppose we want to deploy our app to the production server from a private repository on GitLab. First, we need to [generate a new SSH key pair **with no passphrase**](../../ssh/README.md) for the deployer user. - -After that, we need to copy the private key, which will be used to connect to our server as the deployer user with SSH, to be able to automate our deployment process: - -```bash -# As the deployer user on server -# -# Copy the content of public key to authorized_keys -cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys -# Copy the private key text block -cat ~/.ssh/id_rsa -``` - -Now, let's add it to your GitLab project as a [secret variable](../../ci/variables/README.md#secret-variables). -Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. -They can be added per project by navigating to the project's **Settings** > **CI/CD**. - -![secret variables page](img/secret_variables_page.png) - -To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. -We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. - -We also need to add the public key to **Project** > **Settings** > **Repository** as [Deploy Keys](../../ssh/README.md/#deploy-keys), which gives us the ability to access our repository from the server through [SSH protocol](../../gitlab-basics/command-line-commands.md/#start-working-on-your-project). - - -```bash -# As the deployer user on the server -# -# Copy the public key -cat ~/.ssh/id_rsa.pub -``` - -![deploy keys page](img/deploy_keys_page.png) - -To the field **Title**, add any name you want, and paste the public key into the **Key** field. - -Now, let's clone our repository on the server just to make sure the `deployer` user has access to the repository. - -```bash -# As the deployer user on server -# -git clone git@gitlab.example.com:<USERNAME>/laravel-sample.git -``` - ->**Note:** -Answer **yes** if asked `Are you sure you want to continue connecting (yes/no)?`. -It adds GitLab.com to the known hosts. - -### Configuring Nginx - -Now, let's make sure our web server configuration points to the `current/public` rather than `public`. - -Open the default Nginx server block configuration file by typing: - -```bash -sudo nano /etc/nginx/sites-available/default -``` - -The configuration should be like this. - -``` -server { - root /var/www/app/current/public; - server_name example.com; - # Rest of the configuration -} -``` - ->**Note:** -You may replace the app's name in `/var/www/app/current/public` with the folder name of your application. - -## Setting up Envoy - -So we have our Laravel app ready for production. -The next thing is to use Envoy to perform the deploy. - -To use Envoy, we should first install it on our local machine [using the given instructions by Laravel](https://laravel.com/docs/envoy/#introduction). - -### How Envoy works - -The pros of Envoy is that it doesn't require Blade engine, it just uses Blade syntax to define tasks. -To start, we create an `Envoy.blade.php` in the root of our app with a simple task to test Envoy. - - -```php -@servers(['web' => 'remote_username@remote_host']) - -@task('list', [on => 'web']) - ls -l -@endtask -``` - -As you may expect, we have an array within `@servers` directive at the top of the file, which contains a key named `web` with a value of the server's address (e.g. `deployer@192.168.1.1`). -Then within our `@task` directive we define the bash commands that should be run on the server when the task is executed. - -On the local machine use the `run` command to run Envoy tasks. - -```bash -envoy run list -``` - -It should execute the `list` task we defined earlier, which connects to the server and lists directory contents. - -Envoy is not a dependency of Laravel, therefore you can use it for any PHP application. - -### Zero downtime deployment - -Every time we deploy to the production server, Envoy downloads the latest release of our app from GitLab repository and replace it with preview's release. -Envoy does this without any [downtime](https://en.wikipedia.org/wiki/Downtime), -so we don't have to worry during the deployment while someone might be reviewing the site. -Our deployment plan is to clone the latest release from GitLab repository, install the Composer dependencies and finally, activate the new release. - -#### @setup directive - -The first step of our deployment process is to define a set of variables within [@setup](https://laravel.com/docs/envoy/#setup) directive. -You may change the `app` to your application's name: - - -```php -... - -@setup - $repository = 'git@gitlab.example.com:<USERNAME>/laravel-sample.git'; - $releases_dir = '/var/www/app/releases'; - $app_dir = '/var/www/app'; - $release = date('YmdHis'); - $new_release_dir = $releases_dir .'/'. $release; -@endsetup - -... -``` - -- `$repository` is the address of our repository -- `$releases_dir` directory is where we deploy the app -- `$app_dir` is the actual location of the app that is live on the server -- `$release` contains a date, so every time that we deploy a new release of our app, we get a new folder with the current date as name -- `$new_release_dir` is the full path of the new release which is used just to make the tasks cleaner - -#### @story directive - -The [@story](https://laravel.com/docs/envoy/#stories) directive allows us define a list of tasks that can be run as a single task. -Here we have three tasks called `clone_repository`, `run_composer`, `update_symlinks`. These variables are usable to making our task's codes more cleaner: - - -```php -... - -@story('deploy') - clone_repository - run_composer - update_symlinks -@endstory - -... -``` - -Let's create these three tasks one by one. - -#### Clone the repository - -The first task will create the `releases` directory (if it doesn't exist), and then clone the `master` branch of the repository (by default) into the new release directory, given by the `$new_release_dir` variable. -The `releases` directory will hold all our deployments: - -```php -... - -@task('clone_repository') - echo 'Cloning repository' - [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} - git clone --depth 1 {{ $repository }} {{ $new_release_dir }} -@endtask - -... -``` - -While our project grows, its Git history will be very very long over time. -Since we are creating a directory per release, it might not be necessary to have the history of the project downloaded for each release. -The `--depth 1` option is a great solution which saves systems time and disk space as well. - -#### Installing dependencies with Composer - -As you may know, this task just navigates to the new release directory and runs Composer to install the application dependencies: - -```php -... - -@task('run_composer') - echo "Starting deployment ({{ $release }})" - cd {{ $new_release_dir }} - composer install --prefer-dist --no-scripts -q -o -@endtask - -... -``` - -#### Activate new release - -Next thing to do after preparing the requirements of our new release, is to remove the storage directory from it and to create two symbolic links to point the application's `storage` directory and `.env` file to the new release. -Then, we need to create another symbolic link to the new release with the name of `current` placed in the app directory. -The `current` symbolic link always points to the latest release of our app: - -```php -... - -@task('update_symlinks') - echo "Linking storage directory" - rm -rf {{ $new_release_dir }}/storage - ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage - - echo 'Linking .env file' - ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env - - echo 'Linking current release' - ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current -@endtask -``` - -As you see, we use `-nfs` as an option for `ln` command, which says that the `storage`, `.env` and `current` no longer points to the preview's release and will point them to the new release by force (`f` from `-nfs` means force), which is the case when we are doing multiple deployments. - -### Full script - -The script is ready, but make sure to change the `deployer@192.168.1.1` to your server and also change `/var/www/app` with the directory you want to deploy your app. - -At the end, our `Envoy.blade.php` file will look like this: - -```php -@servers(['web' => 'deployer@192.168.1.1']) - -@setup - $repository = 'git@gitlab.example.com:<USERNAME>/laravel-sample.git'; - $releases_dir = '/var/www/app/releases'; - $app_dir = '/var/www/app'; - $release = date('YmdHis'); - $new_release_dir = $releases_dir .'/'. $release; -@endsetup - -@story('deploy') - clone_repository - run_composer - update_symlinks -@endstory - -@task('clone_repository') - echo 'Cloning repository' - [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} - git clone --depth 1 {{ $repository }} {{ $new_release_dir }} -@endtask - -@task('run_composer') - echo "Starting deployment ({{ $release }})" - cd {{ $new_release_dir }} - composer install --prefer-dist --no-scripts -q -o -@endtask - -@task('update_symlinks') - echo "Linking storage directory" - rm -rf {{ $new_release_dir }}/storage - ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage - - echo 'Linking .env file' - ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env - - echo 'Linking current release' - ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current -@endtask -``` - -One more thing we should do before any deployment is to manually copy our application `storage` folder to the `/var/www/app` directory on the server for the first time. -You might want to create another Envoy task to do that for you. -We also create the `.env` file in the same path to setup our production environment variables for Laravel. -These are persistent data and will be shared to every new release. - -Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../ci/environments.md), which will be described [later](#setting-up-gitlab-ci-cd) in this tutorial. - -Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `master` branch. -To keep things simple, we commit directly to `master`, without using [feature-branches](../../workflow/gitlab_flow.md/#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial. -In a real world project, teams may use [Issue Tracker](../../user/project/issues/index.md) and [Merge Requests](../../user/project/merge_requests/index.md) to move their code across branches: - -```bash -git add Envoy.blade.php -git commit -m 'Add Envoy' -git push origin master -``` - -## Continuous Integration with GitLab - -We have our app ready on GitLab, and we also can deploy it manually. -But let's take a step forward to do it automatically with [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) method. -We need to check every commit with a set of automated tests to become aware of issues at the earliest, and then, we can deploy to the target environment if we are happy with the result of the tests. - -[GitLab CI/CD](../../ci/README.md) allows us to use [Docker](https://docker.com/) engine to handle the process of testing and deploying our app. -In the case you're not familiar with Docker, refer to [How to Automate Docker Deployments](http://paislee.io/how-to-automate-docker-deployments/). - -To be able to build, test, and deploy our app with GitLab CI/CD, we need to prepare our work environment. -To do that, we'll use a Docker image which has the minimum requirements that a Laravel app needs to run. -[There are other ways](../../ci/examples/php.md/#test-php-projects-using-the-docker-executor) to do that as well, but they may lead our builds run slowly, which is not what we want when there are faster options to use. - -With Docker images our builds run incredibly faster! - -### Create a Container Image - -Let's create a [Dockerfile](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Dockerfile) in the root directory of our app with the following content: - -```bash -# Set the base image for subsequent instructions -FROM php:7.1 - -# Update packages -RUN apt-get update - -# Install PHP and composer dependencies -RUN apt-get install -qq git curl libmcrypt-dev libjpeg-dev libpng-dev libfreetype6-dev libbz2-dev - -# Clear out the local repository of retrieved package files -RUN apt-get clean - -# Install needed extensions -# Here you can install any other extension that you need during the test and deployment process -RUN docker-php-ext-install mcrypt pdo_mysql zip - -# Install Composer -RUN curl --silent --show-error https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer - -# Install Laravel Envoy -RUN composer global require "laravel/envoy=~1.0" -``` - -We added the [official PHP 7.1 Docker image](https://hub.docker.com/r/_/php/), which consist of a minimum installation of Debian Jessie with PHP pre-installed, and works perfectly for our use case. - -We used `docker-php-ext-install` (provided by the official PHP Docker image) to install the PHP extensions we need. - -#### Setting Up GitLab Container Registry - -Now that we have our `Dockerfile` let's build and push it to our [GitLab Container Registry](../../user/project/container_registry.md). - -> The registry is the place to store and tag images for later use. Developers may want to maintain their own registry for private, company images, or for throw-away images used only in testing. Using GitLab Container Registry means you don't need to set up and administer yet another service or use a public registry. - -On your GitLab project repository navigate to the **Registry** tab. - -![container registry page empty image](img/container_registry_page_empty_image.png) - -You may need to [enable Container Registry](../../user/project/container_registry.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Sharing and permissions**. - -![container registry checkbox](img/container_registry_checkbox.png) - -To start using Container Registry on our machine, we first need to login to the GitLab registry using our GitLab username and password: - -```bash -docker login registry.gitlab.com -``` -Then we can build and push our image to GitLab: - -```bash -docker build -t registry.gitlab.com/<USERNAME>/laravel-sample . - -docker push registry.gitlab.com/<USERNAME>/laravel-sample -``` - ->**Note:** -To run the above commands, we first need to have [Docker](https://docs.docker.com/engine/installation/) installed on our machine. - -Congratulations! You just pushed the first Docker image to the GitLab Registry, and if you refresh the page you should be able to see it: - -![container registry page with image](img/container_registry_page_with_image.jpg) - ->**Note:** -You can also [use GitLab CI/CD](https://about.gitlab.com/2016/05/23/gitlab-container-registry/#use-with-gitlab-ci) to build and push your Docker images, rather than doing that on your machine. - -We'll use this image further down in the `.gitlab-ci.yml` configuration file to handle the process of testing and deploying our app. - -Let's commit the `Dockerfile` file. - -```bash -git add Dockerfile -git commit -m 'Add Dockerfile' -git push origin master -``` - -### Setting up GitLab CI/CD - -In order to build and test our app with GitLab CI/CD, we need a file called `.gitlab-ci.yml` in our repository's root. It is similar to Circle CI and Travis CI, but built-in GitLab. - -Our `.gitlab-ci.yml` file will look like this: - -```yaml -image: registry.gitlab.com/<USERNAME>/laravel-sample:latest - -services: - - mysql:5.7 - -variables: - MYSQL_DATABASE: homestead - MYSQL_ROOT_PASSWORD: secret - DB_HOST: mysql - DB_USERNAME: root - -stages: - - test - - deploy - -unit_test: - stage: test - script: - - cp .env.example .env - - composer install - - php artisan key:generate - - php artisan migrate - - vendor/bin/phpunit - -deploy_production: - stage: deploy - script: - - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - - eval $(ssh-agent -s) - - ssh-add <(echo "$SSH_PRIVATE_KEY") - - mkdir -p ~/.ssh - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - - - ~/.composer/vendor/bin/envoy run deploy - environment: - name: production - url: http://192.168.1.1 - when: manual - only: - - master -``` - -That's a lot to take in, isn't it? Let's run through it step by step. - -#### Image and Services - -[GitLab Runners](../../ci/runners/README.md) run the script defined by `.gitlab-ci.yml`. -The `image` keyword tells the Runners which image to use. -The `services` keyword defines additional images [that are linked to the main image](../../ci/docker/using_docker_images.md/#what-is-a-service). -Here we use the container image we created before as our main image and also use MySQL 5.7 as a service. - -```yaml -image: registry.gitlab.com/<USERNAME>/laravel-sample:latest - -services: - - mysql:5.7 - -... -``` - ->**Note:** -If you wish to test your app with different PHP versions and [database management systems](../../ci/services/README.md), you can define different `image` and `services` keywords for each test job. - -#### Variables - -GitLab CI/CD allows us to use [environment variables](../../ci/yaml/README.md#variables) in our jobs. -We defined MySQL as our database management system, which comes with a superuser root created by default. - -So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`. -Find out more about MySQL variables at the [official MySQL Docker Image](https://hub.docker.com/r/_/mysql/). - -Also set the variables `DB_HOST` to `mysql` and `DB_USERNAME` to `root`, which are Laravel specific variables. -We define `DB_HOST` as `mysql` instead of `127.0.0.1`, as we use MySQL Docker image as a service which [is linked to the main Docker image](../../ci/docker/using_docker_images.md/#how-services-are-linked-to-the-build). - -```yaml -... - -variables: - MYSQL_DATABASE: homestead - MYSQL_ROOT_PASSWORD: secret - DB_HOST: mysql - DB_USERNAME: root - -... -``` - -#### Unit Test as the first job - -We defined the required shell scripts as an array of the [script](../../ci/yaml/README.md#script) variable to be executed when running `unit_test` job. - -These scripts are some Artisan commands to prepare the Laravel, and, at the end of the script, we'll run the tests by `PHPUnit`. - -```yaml -... - -unit_test: - script: - # Install app dependencies - - composer install - # Setup .env - - cp .env.example .env - # Generate an environment key - - php artisan key:generate - # Run migrations - - php artisan migrate - # Run tests - - vendor/bin/phpunit - -... -``` - -#### Deploy to production - -The job `deploy_production` will deploy the app to the production server. -To deploy our app with Envoy, we had to set up the `$SSH_PRIVATE_KEY` variable as an [SSH private key](../../ci/ssh_keys/README.md/#ssh-keys-when-using-the-docker-executor). -If the SSH keys have added successfully, we can run Envoy. - -As mentioned before, GitLab supports [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) methods as well. -The [environment](../../ci/yaml/README.md#environment) keyword tells GitLab that this job deploys to the `production` environment. -The `url` keyword is used to generate a link to our application on the GitLab Environments page. -The `only` keyword tells GitLab CI that the job should be executed only when the pipeline is building the `master` branch. -Lastly, `when: manual` is used to turn the job from running automatically to a manual action. - -```yaml -... - -deploy_production: - script: - # Add the private SSH key to the build environment - - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - - eval $(ssh-agent -s) - - ssh-add <(echo "$SSH_PRIVATE_KEY") - - mkdir -p ~/.ssh - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - - # Run Envoy - - ~/.composer/vendor/bin/envoy run deploy - - environment: - name: production - url: http://192.168.1.1 - when: manual - only: - - master -``` - -You may also want to add another job for [staging environment](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments), to final test your application before deploying to production. - -### Turn on GitLab CI/CD - -We have prepared everything we need to test and deploy our app with GitLab CI/CD. -To do that, commit and push `.gitlab-ci.yml` to the `master` branch. It will trigger a pipeline, which you can watch live under your project's **Pipelines**. - -![pipelines page](img/pipelines_page.png) - -Here we see our **Test** and **Deploy** stages. -The **Test** stage has the `unit_test` build running. -click on it to see the Runner's output. - -![pipeline page](img/pipeline_page.png) - -After our code passed through the pipeline successfully, we can deploy to our production server by clicking the **play** button on the right side. - -![pipelines page deploy button](img/pipelines_page_deploy_button.png) - -Once the deploy pipeline passed successfully, navigate to **Pipelines > Environments**. - -![environments page](img/environments_page.png) - -If something doesn't work as expected, you can roll back to the latest working version of your app. - -![environment page](img/environment_page.png) - -By clicking on the external link icon specified on the right side, GitLab opens the production website. -Our deployment successfully was done and we can see the application is live. - -![laravel welcome page](img/laravel_welcome_page.png) - -In the case that you're interested to know how is the application directory structure on the production server after deployment, here are three directories named `current`, `releases` and `storage`. -As you know, the `current` directory is a symbolic link that points to the latest release. -The `.env` file consists of our Laravel environment variables. - -![production server app directory](img/production_server_app_directory.png) - -If you navigate to the `current` directory, you should see the application's content. -As you see, the `.env` is pointing to the `/var/www/app/.env` file and also `storage` is pointing to the `/var/www/app/storage/` directory. - -![production server current directory](img/production_server_current_directory.png) - -## Conclusion - -We configured GitLab CI to perform automated tests and used the method of [Continuous Delivery](https://continuousdelivery.com/) to deploy to production a Laravel application with Envoy, directly from the codebase. - -Envoy also was a great match to help us deploy the application without writing our custom bash script and doing Linux magics. +This document was moved to [another location](../../ci/examples/laravel_with_gitlab_and_envoy/index.md). diff --git a/doc/articles/openshift_and_gitlab/index.md b/doc/articles/openshift_and_gitlab/index.md index c0bbcfe2a8a..b7594cfef7f 100644 --- a/doc/articles/openshift_and_gitlab/index.md +++ b/doc/articles/openshift_and_gitlab/index.md @@ -1,510 +1 @@ -# Getting started with OpenShift Origin 3 and GitLab - -> **Article [Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || -> **Level:** intermediary || -> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) || -> **Publication date:** 2016-06-28 - -## Introduction - -[OpenShift Origin][openshift] is an open source container application -platform created by [RedHat], based on [kubernetes] and [Docker]. That means -you can host your own PaaS for free and almost with no hassle. - -In this tutorial, we will see how to deploy GitLab in OpenShift using GitLab's -official Docker image while getting familiar with the web interface and CLI -tools that will help us achieve our goal. - ---- - -## Prerequisites - -OpenShift 3 is not yet deployed on RedHat's offered Online platform ([openshift.com]), -so in order to test it, we will use an [all-in-one Virtualbox image][vm] that is -offered by the OpenShift developers and managed by Vagrant. If you haven't done -already, go ahead and install the following components as they are essential to -test OpenShift easily: - -- [VirtualBox] -- [Vagrant] -- [OpenShift Client][oc] (`oc` for short) - -It is also important to mention that for the purposes of this tutorial, the -latest Origin release is used: - -- **oc** `v1.3.0` (must be [installed][oc-gh] locally on your computer) -- **openshift** `v1.3.0` (is pre-installed in the [VM image][vm-new]) -- **kubernetes** `v1.3.0` (is pre-installed in the [VM image][vm-new]) - ->**Note:** -If you intend to deploy GitLab on a production OpenShift cluster, there are some -limitations to bare in mind. Read on the [limitations](#current-limitations) -section for more information and follow the linked links for the relevant -discussions. - -Now that you have all batteries, let's see how easy it is to test OpenShift -on your computer. - -## Getting familiar with OpenShift Origin - -The environment we are about to use is based on CentOS 7 which comes with all -the tools needed pre-installed: Docker, kubernetes, OpenShift, etcd. - -### Test OpenShift using Vagrant - -As of this writing, the all-in-one VM is at version 1.3, and that's -what we will use in this tutorial. - -In short: - -1. Open a terminal and in a new directory run: - ```sh - vagrant init openshift/origin-all-in-one - ``` -1. This will generate a Vagrantfile based on the all-in-one VM image -1. In the same directory where you generated the Vagrantfile - enter: - - ```sh - vagrant up - ``` - -This will download the VirtualBox image and fire up the VM with some preconfigured -values as you can see in the Vagrantfile. As you may have noticed, you need -plenty of RAM (5GB in our example), so make sure you have enough. - -Now that OpenShift is setup, let's see how the web console looks like. - -### Explore the OpenShift web console - -Once Vagrant finishes its thing with the VM, you will be presented with a -message which has some important information. One of them is the IP address -of the deployed OpenShift platform and in particular <https://10.2.2.2:8443/console/>. -Open this link with your browser and accept the self-signed certificate in -order to proceed. - -Let's login as admin with username/password `admin/admin`. This is what the -landing page looks like: - -![openshift web console](img/web-console.png) - -You can see that a number of [projects] are already created for testing purposes. - -If you head over the `openshift-infra` project, a number of services with their -respective pods are there to explore. - -![openshift web console](img/openshift-infra-project.png) - -We are not going to explore the whole interface, but if you want to learn about -the key concepts of OpenShift, read the [core concepts reference][core] in the -official documentation. - -### Explore the OpenShift CLI - -OpenShift Client (`oc`), is a powerful CLI tool that talks to the OpenShift API -and performs pretty much everything you can do from the web UI and much more. - -Assuming you have [installed][oc] it, let's explore some of its main -functionalities. - -Let's first see the version of `oc`: - -```sh -$ oc version - -oc v1.3.0 -kubernetes v1.3.0+52492b4 -``` - -With `oc help` you can see the top level arguments you can run with `oc` and -interact with your cluster, kubernetes, run applications, create projects and -much more. - -Let's login to the all-in-one VM and see how to achieve the same results like -when we visited the web console earlier. The username/password for the -administrator user is `admin/admin`. There is also a test user with username/ -password `user/user`, with limited access. Let's login as admin for the moment: - -```sh -$ oc login https://10.2.2.2:8443 - -Authentication required for https://10.2.2.2:8443 (openshift) -Username: admin -Password: -Login successful. - -You have access to the following projects and can switch between them with 'oc project <projectname>': - - * cockpit - * default (current) - * delete - * openshift - * openshift-infra - * sample - -Using project "default". -``` - -Switch to the `openshift-infra` project with: - -```sh -oc project openshift-infra -``` - -And finally, see its status: - -```sh -oc status -``` - -The last command should spit a bunch of information about the statuses of the -pods and the services, which if you look closely is what we encountered in the -second image when we explored the web console. - -You can always read more about `oc` in the [OpenShift CLI documentation][oc]. - -### Troubleshooting the all-in-one VM - -Using the all-in-one VM gives you the ability to test OpenShift whenever you -want. That means you get to play with it, shutdown the VM, and pick up where -you left off. - -Sometimes though, you may encounter some issues, like OpenShift not running -when booting up the VM. The web UI may not responding or you may see issues -when trying to login with `oc`, like: - -``` -The connection to the server 10.2.2.2:8443 was refused - did you specify the right host or port? -``` - -In that case, the OpenShift service might not be running, so in order to fix it: - -1. SSH into the VM by going to the directory where the Vagrantfile is and then - run: - - ```sh - vagrant ssh - ``` - -1. Run `systemctl` and verify by the output that the `openshift` service is not - running (it will be in red color). If that's the case start the service with: - - ```sh - sudo systemctl start openshift - ``` - -1. Verify the service is up with: - - ```sh - systemctl status openshift -l - ``` - -Now you will be able to login using `oc` (like we did before) and visit the web -console. - -## Deploy GitLab - -Now that you got a taste of what OpenShift looks like, let's deploy GitLab! - -### Create a new project - -First, we will create a new project to host our application. You can do this -either by running the CLI client: - -```bash -$ oc new-project gitlab -``` - -or by using the web interface: - -![Create a new project from the UI](img/create-project-ui.png) - -If you used the command line, `oc` automatically uses the new project and you -can see its status with: - -```sh -$ oc status - -In project gitlab on server https://10.2.2.2:8443 - -You have no services, deployment configs, or build configs. -Run 'oc new-app' to create an application. -``` - -If you visit the web console, you can now see `gitlab` listed in the projects list. - -The next step is to import the OpenShift template for GitLab. - -### Import the template - -The [template][templates] is basically a JSON file which describes a set of -related object definitions to be created together, as well as a set of -parameters for those objects. - -The template for GitLab resides in the Omnibus GitLab repository under the -docker directory. Let's download it locally with `wget`: - -```bash -wget https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/docker/openshift-template.json -``` - -And then let's import it in OpenShift: - -```bash -oc create -f openshift-template.json -n openshift -``` - ->**Note:** -The `-n openshift` namespace flag is a trick to make the template available to all -projects. If you recall from when we created the `gitlab` project, `oc` switched -to it automatically, and that can be verified by the `oc status` command. If -you omit the namespace flag, the application will be available only to the -current project, in our case `gitlab`. The `openshift` namespace is a global -one that the administrators should use if they want the application to be -available to all users. - -We are now ready to finally deploy GitLab! - -### Create a new application - -The next step is to use the template we previously imported. Head over to the -`gitlab` project and hit the **Add to Project** button. - -![Add to project](img/add-to-project.png) - -This will bring you to the catalog where you can find all the pre-defined -applications ready to deploy with the click of a button. Search for `gitlab` -and you will see the previously imported template: - -![Add GitLab to project](img/add-gitlab-to-project.png) - -Select it, and in the following screen you will be presented with the predefined -values used with the GitLab template: - -![GitLab settings](img/gitlab-settings.png) - -Notice at the top that there are three resources to be created with this -template: - -- `gitlab-ce` -- `gitlab-ce-redis` -- `gitlab-ce-postgresql` - -While PostgreSQL and Redis are bundled in Omnibus GitLab, the template is using -separate images as you can see from [this line][line] in the template. - -The predefined values have been calculated for the purposes of testing out -GitLab in the all-in-one VM. You don't need to change anything here, hit -**Create** to start the deployment. - -If you are deploying to production you will want to change the **GitLab instance -hostname** and use greater values for the volume sizes. If you don't provide a -password for PostgreSQL, it will be created automatically. - ->**Note:** -The `gitlab.apps.10.2.2.2.xip.io` hostname that is used by default will -resolve to the host with IP `10.2.2.2` which is the IP our VM uses. It is a -trick to have distinct FQDNs pointing to services that are on our local network. -Read more on how this works in <http://xip.io>. - -Now that we configured this, let's see how to manage and scale GitLab. - -## Manage and scale GitLab - -Setting up GitLab for the first time might take a while depending on your -internet connection and the resources you have attached to the all-in-one VM. -GitLab's docker image is quite big (~500MB), so you'll have to wait until -it's downloaded and configured before you use it. - -### Watch while GitLab gets deployed - -Navigate to the `gitlab` project at **Overview**. You can notice that the -deployment is in progress by the orange color. The Docker images are being -downloaded and soon they will be up and running. - -![GitLab overview](img/gitlab-overview.png) - -Switch to the **Browse > Pods** and you will eventually see all 3 pods in a -running status. Remember the 3 resources that were to be created when we first -created the GitLab app? This is where you can see them in action. - -![Running pods](img/running-pods.png) - -You can see GitLab being reconfigured by taking look at the logs in realtime. -Click on `gitlab-ce-2-j7ioe` (your ID will be different) and go to the **Logs** -tab. - -![GitLab logs](img/gitlab-logs.png) - -At a point you should see a _**gitlab Reconfigured!**_ message in the logs. -Navigate back to the **Overview** and hopefully all pods will be up and running. - -![GitLab running](img/gitlab-running.png) - -Congratulations! You can now navigate to your new shinny GitLab instance by -visiting <http://gitlab.apps.10.2.2.2.xip.io> where you will be asked to -change the root user password. Login using `root` as username and providing the -password you just set, and start using GitLab! - -### Scale GitLab with the push of a button - -If you reach to a point where your GitLab instance could benefit from a boost -of resources, you'd be happy to know that you can scale up with the push of a -button. - -In the **Overview** page just click the up arrow button in the pod where -GitLab is. The change is instant and you can see the number of [replicas] now -running scaled to 2. - -![GitLab scale](img/gitlab-scale.png) - -Upping the GitLab pods is actually like adding new application servers to your -cluster. You can see how that would work if you didn't use GitLab with -OpenShift by following the [HA documentation][ha] for the application servers. - -Bare in mind that you may need more resources (CPU, RAM, disk space) when you -scale up. If a pod is in pending state for too long, you can navigate to -**Browse > Events** and see the reason and message of the state. - -![No resources](img/no-resources.png) - -### Scale GitLab using the `oc` CLI - -Using `oc` is super easy to scale up the replicas of a pod. You may want to -skim through the [basic CLI operations][basic-cli] to get a taste how the CLI -commands are used. Pay extra attention to the object types as we will use some -of them and their abbreviated versions below. - -In order to scale up, we need to find out the name of the replication controller. -Let's see how to do that using the following steps. - -1. Make sure you are in the `gitlab` project: - - ```sh - oc project gitlab - ``` - -1. See what services are used for this project: - - ```sh - oc get svc - ``` - - The output will be similar to: - - ``` - NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE - gitlab-ce 172.30.243.177 <none> 22/TCP,80/TCP 5d - gitlab-ce-postgresql 172.30.116.75 <none> 5432/TCP 5d - gitlab-ce-redis 172.30.105.88 <none> 6379/TCP 5d - ``` - -1. We need to see the replication controllers of the `gitlab-ce` service. - Get a detailed view of the current ones: - - ```sh - oc describe rc gitlab-ce - ``` - - This will return a large detailed list of the current replication controllers. - Search for the name of the GitLab controller, usually `gitlab-ce-1` or if - that failed at some point and you spawned another one, it will be named - `gitlab-ce-2`. - -1. Scale GitLab using the previous information: - - ```sh - oc scale --replicas=2 replicationcontrollers gitlab-ce-2 - ``` - -1. Get the new replicas number to make sure scaling worked: - - ```sh - oc get rc gitlab-ce-2 - ``` - - which will return something like: - - ``` - NAME DESIRED CURRENT AGE - gitlab-ce-2 2 2 5d - ``` - -And that's it! We successfully scaled the replicas to 2 using the CLI. - -As always, you can find the name of the controller using the web console. Just -click on the service you are interested in and you will see the details in the -right sidebar. - -![Replication controller name](img/rc-name.png) - -### Autoscaling GitLab - -In case you were wondering whether there is an option to autoscale a pod based -on the resources of your server, the answer is yes, of course there is. - -We will not expand on this matter, but feel free to read the documentation on -OpenShift's website about [autoscaling]. - -## Current limitations - -As stated in the [all-in-one VM][vm] page: - -> By default, OpenShift will not allow a container to run as root or even a -non-random container assigned userid. Most Docker images in the Dockerhub do not -follow this best practice and instead run as root. - -The all-in-one VM we are using has this security turned off so it will not -bother us. In any case, it is something to keep in mind when deploying GitLab -on a production cluster. - -In order to deploy GitLab on a production cluster, you will need to assign the -GitLab service account to the `anyuid` Security Context. - -1. Edit the Security Context: - ```sh - oc edit scc anyuid - ``` - -1. Add `system:serviceaccount:<project>:gitlab-ce-user` to the `users` section. - If you changed the Application Name from the default the user will - will be `<app-name>-user` instead of `gitlab-ce-user` - -1. Save and exit the editor - -## Conclusion - -By now, you should have an understanding of the basic OpenShift Origin concepts -and a sense of how things work using the web console or the CLI. - -GitLab was hard to install in previous versions of OpenShift, -but now that belongs to the past. Upload a template, create a project, add an -application and you are done. You are ready to login to your new GitLab instance. - -And remember that in this tutorial we just scratched the surface of what Origin -is capable of. As always, you can refer to the detailed -[documentation][openshift-docs] to learn more about deploying your own OpenShift -PaaS and managing your applications with the ease of containers. - -[RedHat]: https://www.redhat.com/en "RedHat website" -[openshift]: https://www.openshift.org "OpenShift Origin website" -[vm]: https://www.openshift.org/vm/ "OpenShift All-in-one VM" -[vm-new]: https://atlas.hashicorp.com/openshift/boxes/origin-all-in-one "Official OpenShift Vagrant box on Atlas" -[template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/docker/openshift-template.json "OpenShift template for GitLab" -[openshift.com]: https://openshift.com "OpenShift Online" -[kubernetes]: http://kubernetes.io/ "Kubernetes website" -[Docker]: https://www.docker.com "Docker website" -[oc]: https://docs.openshift.org/latest/cli_reference/get_started_cli.html "Documentation - oc CLI documentation" -[VirtualBox]: https://www.virtualbox.org/wiki/Downloads "VirtualBox downloads" -[Vagrant]: https://www.vagrantup.com/downloads.html "Vagrant downloads" -[projects]: https://docs.openshift.org/latest/dev_guide/projects.html "Documentation - Projects overview" -[core]: https://docs.openshift.org/latest/architecture/core_concepts/index.html "Documentation - Core concepts of OpenShift Origin" -[templates]: https://docs.openshift.org/latest/architecture/core_concepts/templates.html "Documentation - OpenShift templates" -[old-post]: https://blog.openshift.com/deploy-gitlab-openshift/ "Old post - Deploy GitLab on OpenShift" -[line]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/658c065c8d022ce858dd63eaeeadb0b2ddc8deea/docker/openshift-template.json#L239 "GitLab - OpenShift template" -[oc-gh]: https://github.com/openshift/origin/releases/tag/v1.3.0 "Openshift 1.3.0 release on GitHub" -[ha]: http://docs.gitlab.com/ce/administration/high_availability/gitlab.html "Documentation - GitLab High Availability" -[replicas]: https://docs.openshift.org/latest/architecture/core_concepts/deployments.html#replication-controllers "Documentation - Replication controller" -[autoscaling]: https://docs.openshift.org/latest/dev_guide/pod_autoscaling.html "Documentation - Autoscale" -[basic-cli]: https://docs.openshift.org/latest/cli_reference/basic_cli_operations.html "Documentation - Basic CLI operations" -[openshift-docs]: https://docs.openshift.org "OpenShift documentation" +This document was moved to [another location](../../install/openshift_and_gitlab/index.html). diff --git a/doc/ci/README.md b/doc/ci/README.md index 05d792dea0f..5829aaee9c9 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -120,7 +120,7 @@ Here is an collection of tutorials and guides on setting up your CI pipeline. - [Run PHP Composer & NPM scripts then deploy them to a staging server](examples/deployment/composer-npm-deploy.md) - [Analyze code quality with the Code Climate CLI](examples/code_climate.md) - **Articles** - - [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](../articles/laravel_with_gitlab_and_envoy/index.md) + - [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](examples/laravel_with_gitlab_and_envoy/index.md) - [How to deploy Maven projects to Artifactory with GitLab CI/CD](examples/artifactory_and_gitlab/index.md) - [Automated Debian packaging](https://about.gitlab.com/2016/10/12/automated-debian-package-build-with-gitlab-ci/) - [Spring boot application with GitLab CI and Kubernetes](https://about.gitlab.com/2016/12/14/continuous-delivery-of-a-spring-boot-application-with-gitlab-ci-and-kubernetes/) diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 25a0c5dcff5..d4590d0f495 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -16,6 +16,7 @@ Apart from those, here is an collection of tutorials and guides on setting up yo - [Testing a PHP application](php.md) - [Run PHP Composer & NPM scripts then deploy them to a staging server](deployment/composer-npm-deploy.md) +- [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) ### Ruby diff --git a/doc/ci/examples/code_climate.md b/doc/ci/examples/code_climate.md index 6a5821762cc..f919ed3c797 100644 --- a/doc/ci/examples/code_climate.md +++ b/doc/ci/examples/code_climate.md @@ -16,7 +16,8 @@ codequality: - docker:dind script: - docker pull codeclimate/codeclimate - - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate analyze -f json > codeclimate.json || true + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 init + - docker run --env CODECLIMATE_CODE="$PWD" --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate:0.69.0 analyze -f json > codeclimate.json || true artifacts: paths: [codeclimate.json] ``` diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png Binary files differindex a56c07a0da7..a56c07a0da7 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_checkbox.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png Binary files differindex b1406fed6b8..b1406fed6b8 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_empty_image.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg Binary files differindex d1f0cbc08ab..d1f0cbc08ab 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/container_registry_page_with_image.jpg diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png Binary files differindex 9aae11b8679..9aae11b8679 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/deploy_keys_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environment_page.png Binary files differindex a06b6d417cd..a06b6d417cd 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/environment_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environment_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environments_page.png Binary files differindex d357ecda7d2..d357ecda7d2 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/environments_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/environments_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png Binary files differindex 3bb21fd12b4..3bb21fd12b4 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_welcome_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png Binary files differindex bc188f83fb1..bc188f83fb1 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/laravel_with_gitlab_and_envoy.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipeline_page.png Binary files differindex baf8dec499c..baf8dec499c 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/pipeline_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipeline_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page.png Binary files differindex d96c43bcf16..d96c43bcf16 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png Binary files differindex 997db10189f..997db10189f 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/pipelines_page_deploy_button.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png Binary files differindex 6dbc29fc25c..6dbc29fc25c 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_app_directory.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png Binary files differindex 8a6dcccfa38..8a6dcccfa38 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/production_server_current_directory.png diff --git a/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/secret_variables_page.png Binary files differindex 658c0b5bcac..658c0b5bcac 100644 --- a/doc/articles/laravel_with_gitlab_and_envoy/img/secret_variables_page.png +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/img/secret_variables_page.png diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md new file mode 100644 index 00000000000..e1aff6fdf36 --- /dev/null +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -0,0 +1,684 @@ +--- +redirect_from: 'https://docs.gitlab.com/ee/articles/laravel_with_gitlab_and_envoy/index.html' +--- + +# Test and deploy Laravel applications with GitLab CI/CD and Envoy + +> **[Article Type](../../../development/writing_documentation.md#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Mehran Rasulian](https://gitlab.com/mehranrasulian) || +> **Publication date:** 2017-08-31 + +## Introduction + +GitLab features our applications with Continuous Integration, and it is possible to easily deploy the new code changes to the production server whenever we want. + +In this tutorial, we'll show you how to initialize a [Laravel](http://laravel.com/) application and setup our [Envoy](https://laravel.com/docs/envoy) tasks, then we'll jump into see how to test and deploy it with [GitLab CI/CD](../README.md) via [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/). + +We assume you have a basic experience with Laravel, Linux servers, +and you know how to use GitLab. + +Laravel is a high quality web framework written in PHP. +It has a great community with a [fantastic documentation](https://laravel.com/docs). +Aside from the usual routing, controllers, requests, responses, views, and (blade) templates, out of the box Laravel provides plenty of additional services such as cache, events, localization, authentication and many others. + +We will use [Envoy](https://laravel.com/docs/master/envoy) as an SSH task runner based on PHP. +It uses a clean, minimal [Blade syntax](https://laravel.com/docs/blade) to setup tasks that can run on remote servers, such as, cloning your project from the repository, installing the Composer dependencies, and running [Artisan commands](https://laravel.com/docs/artisan). + +## Initialize our Laravel app on GitLab + +We assume [you have installed a new laravel project](https://laravel.com/docs/installation#installation), so let's start with a unit test, and initialize Git for the project. + +### Unit Test + +Every new installation of Laravel (currently 5.4) comes with two type of tests, 'Feature' and 'Unit', placed in the tests directory. +Here's a unit test from `test/Unit/ExampleTest.php`: + +```php +<?php + +namespace Tests\Unit; + +... + +class ExampleTest extends TestCase +{ + public function testBasicTest() + { + $this->assertTrue(true); + } +} +``` + +This test is as simple as asserting that the given value is true. + +Laravel uses `PHPUnit` for tests by default. +If we run `vendor/bin/phpunit` we should see the green output: + +```bash +vendor/bin/phpunit +OK (1 test, 1 assertions) +``` + +This test will be used later for continuously testing our app with GitLab CI/CD. + +### Push to GitLab + +Since we have our app up and running locally, it's time to push the codebase to our remote repository. +Let's create [a new project](../../../gitlab-basics/create-project.md) in GitLab named `laravel-sample`. +After that, follow the command line instructions displayed on the project's homepage to initiate the repository on our machine and push the first commit. + + +```bash +cd laravel-sample +git init +git remote add origin git@gitlab.example.com:<USERNAME>/laravel-sample.git +git add . +git commit -m 'Initial Commit' +git push -u origin master +``` + +## Configure the production server + +Before we begin setting up Envoy and GitLab CI/CD, let's quickly make sure the production server is ready for deployment. +We have installed LEMP stack which stands for Linux, Nginx, MySQL and PHP on our Ubuntu 16.04. + +### Create a new user + +Let's now create a new user that will be used to deploy our website and give it +the needed permissions using [Linux ACL](https://serversforhackers.com/video/linux-acls): + +```bash +# Create user deployer +sudo adduser deployer +# Give the read-write-execute permissions to deployer user for directory /var/www +sudo setfacl -R -m u:deployer:rwx /var/www +``` + +If you don't have ACL installed on your Ubuntu server, use this command to install it: + +```bash +sudo apt install acl +``` + +### Add SSH key + +Let's suppose we want to deploy our app to the production server from a private repository on GitLab. First, we need to [generate a new SSH key pair **with no passphrase**](../../../ssh/README.md) for the deployer user. + +After that, we need to copy the private key, which will be used to connect to our server as the deployer user with SSH, to be able to automate our deployment process: + +```bash +# As the deployer user on server +# +# Copy the content of public key to authorized_keys +cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys +# Copy the private key text block +cat ~/.ssh/id_rsa +``` + +Now, let's add it to your GitLab project as a [secret variable](../../variables/README.md#secret-variables). +Secret variables are user-defined variables and are stored out of `.gitlab-ci.yml`, for security purposes. +They can be added per project by navigating to the project's **Settings** > **CI/CD**. + +![secret variables page](img/secret_variables_page.png) + +To the field **KEY**, add the name `SSH_PRIVATE_KEY`, and to the **VALUE** field, paste the private key you've copied earlier. +We'll use this variable in the `.gitlab-ci.yml` later, to easily connect to our remote server as the deployer user without entering its password. + +We also need to add the public key to **Project** > **Settings** > **Repository** as [Deploy Keys](../../../ssh/README.md/#deploy-keys), which gives us the ability to access our repository from the server through [SSH protocol](../../../gitlab-basics/command-line-commands.md/#start-working-on-your-project). + + +```bash +# As the deployer user on the server +# +# Copy the public key +cat ~/.ssh/id_rsa.pub +``` + +![deploy keys page](img/deploy_keys_page.png) + +To the field **Title**, add any name you want, and paste the public key into the **Key** field. + +Now, let's clone our repository on the server just to make sure the `deployer` user has access to the repository. + +```bash +# As the deployer user on server +# +git clone git@gitlab.example.com:<USERNAME>/laravel-sample.git +``` + +>**Note:** +Answer **yes** if asked `Are you sure you want to continue connecting (yes/no)?`. +It adds GitLab.com to the known hosts. + +### Configuring Nginx + +Now, let's make sure our web server configuration points to the `current/public` rather than `public`. + +Open the default Nginx server block configuration file by typing: + +```bash +sudo nano /etc/nginx/sites-available/default +``` + +The configuration should be like this. + +``` +server { + root /var/www/app/current/public; + server_name example.com; + # Rest of the configuration +} +``` + +>**Note:** +You may replace the app's name in `/var/www/app/current/public` with the folder name of your application. + +## Setting up Envoy + +So we have our Laravel app ready for production. +The next thing is to use Envoy to perform the deploy. + +To use Envoy, we should first install it on our local machine [using the given instructions by Laravel](https://laravel.com/docs/envoy/#introduction). + +### How Envoy works + +The pros of Envoy is that it doesn't require Blade engine, it just uses Blade syntax to define tasks. +To start, we create an `Envoy.blade.php` in the root of our app with a simple task to test Envoy. + + +```php +@servers(['web' => 'remote_username@remote_host']) + +@task('list', [on => 'web']) + ls -l +@endtask +``` + +As you may expect, we have an array within `@servers` directive at the top of the file, which contains a key named `web` with a value of the server's address (e.g. `deployer@192.168.1.1`). +Then within our `@task` directive we define the bash commands that should be run on the server when the task is executed. + +On the local machine use the `run` command to run Envoy tasks. + +```bash +envoy run list +``` + +It should execute the `list` task we defined earlier, which connects to the server and lists directory contents. + +Envoy is not a dependency of Laravel, therefore you can use it for any PHP application. + +### Zero downtime deployment + +Every time we deploy to the production server, Envoy downloads the latest release of our app from GitLab repository and replace it with preview's release. +Envoy does this without any [downtime](https://en.wikipedia.org/wiki/Downtime), +so we don't have to worry during the deployment while someone might be reviewing the site. +Our deployment plan is to clone the latest release from GitLab repository, install the Composer dependencies and finally, activate the new release. + +#### @setup directive + +The first step of our deployment process is to define a set of variables within [@setup](https://laravel.com/docs/envoy/#setup) directive. +You may change the `app` to your application's name: + + +```php +... + +@setup + $repository = 'git@gitlab.example.com:<USERNAME>/laravel-sample.git'; + $releases_dir = '/var/www/app/releases'; + $app_dir = '/var/www/app'; + $release = date('YmdHis'); + $new_release_dir = $releases_dir .'/'. $release; +@endsetup + +... +``` + +- `$repository` is the address of our repository +- `$releases_dir` directory is where we deploy the app +- `$app_dir` is the actual location of the app that is live on the server +- `$release` contains a date, so every time that we deploy a new release of our app, we get a new folder with the current date as name +- `$new_release_dir` is the full path of the new release which is used just to make the tasks cleaner + +#### @story directive + +The [@story](https://laravel.com/docs/envoy/#stories) directive allows us define a list of tasks that can be run as a single task. +Here we have three tasks called `clone_repository`, `run_composer`, `update_symlinks`. These variables are usable to making our task's codes more cleaner: + + +```php +... + +@story('deploy') + clone_repository + run_composer + update_symlinks +@endstory + +... +``` + +Let's create these three tasks one by one. + +#### Clone the repository + +The first task will create the `releases` directory (if it doesn't exist), and then clone the `master` branch of the repository (by default) into the new release directory, given by the `$new_release_dir` variable. +The `releases` directory will hold all our deployments: + +```php +... + +@task('clone_repository') + echo 'Cloning repository' + [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} + git clone --depth 1 {{ $repository }} {{ $new_release_dir }} +@endtask + +... +``` + +While our project grows, its Git history will be very very long over time. +Since we are creating a directory per release, it might not be necessary to have the history of the project downloaded for each release. +The `--depth 1` option is a great solution which saves systems time and disk space as well. + +#### Installing dependencies with Composer + +As you may know, this task just navigates to the new release directory and runs Composer to install the application dependencies: + +```php +... + +@task('run_composer') + echo "Starting deployment ({{ $release }})" + cd {{ $new_release_dir }} + composer install --prefer-dist --no-scripts -q -o +@endtask + +... +``` + +#### Activate new release + +Next thing to do after preparing the requirements of our new release, is to remove the storage directory from it and to create two symbolic links to point the application's `storage` directory and `.env` file to the new release. +Then, we need to create another symbolic link to the new release with the name of `current` placed in the app directory. +The `current` symbolic link always points to the latest release of our app: + +```php +... + +@task('update_symlinks') + echo "Linking storage directory" + rm -rf {{ $new_release_dir }}/storage + ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage + + echo 'Linking .env file' + ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env + + echo 'Linking current release' + ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current +@endtask +``` + +As you see, we use `-nfs` as an option for `ln` command, which says that the `storage`, `.env` and `current` no longer points to the preview's release and will point them to the new release by force (`f` from `-nfs` means force), which is the case when we are doing multiple deployments. + +### Full script + +The script is ready, but make sure to change the `deployer@192.168.1.1` to your server and also change `/var/www/app` with the directory you want to deploy your app. + +At the end, our `Envoy.blade.php` file will look like this: + +```php +@servers(['web' => 'deployer@192.168.1.1']) + +@setup + $repository = 'git@gitlab.example.com:<USERNAME>/laravel-sample.git'; + $releases_dir = '/var/www/app/releases'; + $app_dir = '/var/www/app'; + $release = date('YmdHis'); + $new_release_dir = $releases_dir .'/'. $release; +@endsetup + +@story('deploy') + clone_repository + run_composer + update_symlinks +@endstory + +@task('clone_repository') + echo 'Cloning repository' + [ -d {{ $releases_dir }} ] || mkdir {{ $releases_dir }} + git clone --depth 1 {{ $repository }} {{ $new_release_dir }} +@endtask + +@task('run_composer') + echo "Starting deployment ({{ $release }})" + cd {{ $new_release_dir }} + composer install --prefer-dist --no-scripts -q -o +@endtask + +@task('update_symlinks') + echo "Linking storage directory" + rm -rf {{ $new_release_dir }}/storage + ln -nfs {{ $app_dir }}/storage {{ $new_release_dir }}/storage + + echo 'Linking .env file' + ln -nfs {{ $app_dir }}/.env {{ $new_release_dir }}/.env + + echo 'Linking current release' + ln -nfs {{ $new_release_dir }} {{ $app_dir }}/current +@endtask +``` + +One more thing we should do before any deployment is to manually copy our application `storage` folder to the `/var/www/app` directory on the server for the first time. +You might want to create another Envoy task to do that for you. +We also create the `.env` file in the same path to setup our production environment variables for Laravel. +These are persistent data and will be shared to every new release. + +Now, we would need to deploy our app by running `envoy run deploy`, but it won't be necessary since GitLab can handle that for us with CI's [environments](../../environments.md), which will be described [later](#setting-up-gitlab-ci-cd) in this tutorial. + +Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `master` branch. +To keep things simple, we commit directly to `master`, without using [feature-branches](../../../workflow/gitlab_flow.md/#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial. +In a real world project, teams may use [Issue Tracker](../../../user/project/issues/index.md) and [Merge Requests](../../../user/project/merge_requests/index.md) to move their code across branches: + +```bash +git add Envoy.blade.php +git commit -m 'Add Envoy' +git push origin master +``` + +## Continuous Integration with GitLab + +We have our app ready on GitLab, and we also can deploy it manually. +But let's take a step forward to do it automatically with [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) method. +We need to check every commit with a set of automated tests to become aware of issues at the earliest, and then, we can deploy to the target environment if we are happy with the result of the tests. + +[GitLab CI/CD](../../README.md) allows us to use [Docker](https://docker.com/) engine to handle the process of testing and deploying our app. +In the case you're not familiar with Docker, refer to [How to Automate Docker Deployments](http://paislee.io/how-to-automate-docker-deployments/). + +To be able to build, test, and deploy our app with GitLab CI/CD, we need to prepare our work environment. +To do that, we'll use a Docker image which has the minimum requirements that a Laravel app needs to run. +[There are other ways](../php.md/#test-php-projects-using-the-docker-executor) to do that as well, but they may lead our builds run slowly, which is not what we want when there are faster options to use. + +With Docker images our builds run incredibly faster! + +### Create a Container Image + +Let's create a [Dockerfile](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Dockerfile) in the root directory of our app with the following content: + +```bash +# Set the base image for subsequent instructions +FROM php:7.1 + +# Update packages +RUN apt-get update + +# Install PHP and composer dependencies +RUN apt-get install -qq git curl libmcrypt-dev libjpeg-dev libpng-dev libfreetype6-dev libbz2-dev + +# Clear out the local repository of retrieved package files +RUN apt-get clean + +# Install needed extensions +# Here you can install any other extension that you need during the test and deployment process +RUN docker-php-ext-install mcrypt pdo_mysql zip + +# Install Composer +RUN curl --silent --show-error https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer + +# Install Laravel Envoy +RUN composer global require "laravel/envoy=~1.0" +``` + +We added the [official PHP 7.1 Docker image](https://hub.docker.com/r/_/php/), which consist of a minimum installation of Debian Jessie with PHP pre-installed, and works perfectly for our use case. + +We used `docker-php-ext-install` (provided by the official PHP Docker image) to install the PHP extensions we need. + +#### Setting Up GitLab Container Registry + +Now that we have our `Dockerfile` let's build and push it to our [GitLab Container Registry](../../../user/project/container_registry.md). + +> The registry is the place to store and tag images for later use. Developers may want to maintain their own registry for private, company images, or for throw-away images used only in testing. Using GitLab Container Registry means you don't need to set up and administer yet another service or use a public registry. + +On your GitLab project repository navigate to the **Registry** tab. + +![container registry page empty image](img/container_registry_page_empty_image.png) + +You may need to [enable Container Registry](../../../user/project/container_registry.md#enable-the-container-registry-for-your-project) to your project to see this tab. You'll find it under your project's **Settings > General > Sharing and permissions**. + +![container registry checkbox](img/container_registry_checkbox.png) + +To start using Container Registry on our machine, we first need to login to the GitLab registry using our GitLab username and password: + +```bash +docker login registry.gitlab.com +``` +Then we can build and push our image to GitLab: + +```bash +docker build -t registry.gitlab.com/<USERNAME>/laravel-sample . + +docker push registry.gitlab.com/<USERNAME>/laravel-sample +``` + +>**Note:** +To run the above commands, we first need to have [Docker](https://docs.docker.com/engine/installation/) installed on our machine. + +Congratulations! You just pushed the first Docker image to the GitLab Registry, and if you refresh the page you should be able to see it: + +![container registry page with image](img/container_registry_page_with_image.jpg) + +>**Note:** +You can also [use GitLab CI/CD](https://about.gitlab.com/2016/05/23/gitlab-container-registry/#use-with-gitlab-ci) to build and push your Docker images, rather than doing that on your machine. + +We'll use this image further down in the `.gitlab-ci.yml` configuration file to handle the process of testing and deploying our app. + +Let's commit the `Dockerfile` file. + +```bash +git add Dockerfile +git commit -m 'Add Dockerfile' +git push origin master +``` + +### Setting up GitLab CI/CD + +In order to build and test our app with GitLab CI/CD, we need a file called `.gitlab-ci.yml` in our repository's root. It is similar to Circle CI and Travis CI, but built-in GitLab. + +Our `.gitlab-ci.yml` file will look like this: + +```yaml +image: registry.gitlab.com/<USERNAME>/laravel-sample:latest + +services: + - mysql:5.7 + +variables: + MYSQL_DATABASE: homestead + MYSQL_ROOT_PASSWORD: secret + DB_HOST: mysql + DB_USERNAME: root + +stages: + - test + - deploy + +unit_test: + stage: test + script: + - cp .env.example .env + - composer install + - php artisan key:generate + - php artisan migrate + - vendor/bin/phpunit + +deploy_production: + stage: deploy + script: + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + - eval $(ssh-agent -s) + - ssh-add <(echo "$SSH_PRIVATE_KEY") + - mkdir -p ~/.ssh + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' + + - ~/.composer/vendor/bin/envoy run deploy + environment: + name: production + url: http://192.168.1.1 + when: manual + only: + - master +``` + +That's a lot to take in, isn't it? Let's run through it step by step. + +#### Image and Services + +[GitLab Runners](../../runners/README.md) run the script defined by `.gitlab-ci.yml`. +The `image` keyword tells the Runners which image to use. +The `services` keyword defines additional images [that are linked to the main image](../../docker/using_docker_images.md/#what-is-a-service). +Here we use the container image we created before as our main image and also use MySQL 5.7 as a service. + +```yaml +image: registry.gitlab.com/<USERNAME>/laravel-sample:latest + +services: + - mysql:5.7 + +... +``` + +>**Note:** +If you wish to test your app with different PHP versions and [database management systems](../../services/README.md), you can define different `image` and `services` keywords for each test job. + +#### Variables + +GitLab CI/CD allows us to use [environment variables](../../yaml/README.md#variables) in our jobs. +We defined MySQL as our database management system, which comes with a superuser root created by default. + +So we should adjust the configuration of MySQL instance by defining `MYSQL_DATABASE` variable as our database name and `MYSQL_ROOT_PASSWORD` variable as the password of `root`. +Find out more about MySQL variables at the [official MySQL Docker Image](https://hub.docker.com/r/_/mysql/). + +Also set the variables `DB_HOST` to `mysql` and `DB_USERNAME` to `root`, which are Laravel specific variables. +We define `DB_HOST` as `mysql` instead of `127.0.0.1`, as we use MySQL Docker image as a service which [is linked to the main Docker image](../../docker/using_docker_images.md/#how-services-are-linked-to-the-build). + +```yaml +... + +variables: + MYSQL_DATABASE: homestead + MYSQL_ROOT_PASSWORD: secret + DB_HOST: mysql + DB_USERNAME: root + +... +``` + +#### Unit Test as the first job + +We defined the required shell scripts as an array of the [script](../../yaml/README.md#script) variable to be executed when running `unit_test` job. + +These scripts are some Artisan commands to prepare the Laravel, and, at the end of the script, we'll run the tests by `PHPUnit`. + +```yaml +... + +unit_test: + script: + # Install app dependencies + - composer install + # Setup .env + - cp .env.example .env + # Generate an environment key + - php artisan key:generate + # Run migrations + - php artisan migrate + # Run tests + - vendor/bin/phpunit + +... +``` + +#### Deploy to production + +The job `deploy_production` will deploy the app to the production server. +To deploy our app with Envoy, we had to set up the `$SSH_PRIVATE_KEY` variable as an [SSH private key](../../ssh_keys/README.md/#ssh-keys-when-using-the-docker-executor). +If the SSH keys have added successfully, we can run Envoy. + +As mentioned before, GitLab supports [Continuous Delivery](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) methods as well. +The [environment](../../yaml/README.md#environment) keyword tells GitLab that this job deploys to the `production` environment. +The `url` keyword is used to generate a link to our application on the GitLab Environments page. +The `only` keyword tells GitLab CI that the job should be executed only when the pipeline is building the `master` branch. +Lastly, `when: manual` is used to turn the job from running automatically to a manual action. + +```yaml +... + +deploy_production: + script: + # Add the private SSH key to the build environment + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + - eval $(ssh-agent -s) + - ssh-add <(echo "$SSH_PRIVATE_KEY") + - mkdir -p ~/.ssh + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' + + # Run Envoy + - ~/.composer/vendor/bin/envoy run deploy + + environment: + name: production + url: http://192.168.1.1 + when: manual + only: + - master +``` + +You may also want to add another job for [staging environment](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments), to final test your application before deploying to production. + +### Turn on GitLab CI/CD + +We have prepared everything we need to test and deploy our app with GitLab CI/CD. +To do that, commit and push `.gitlab-ci.yml` to the `master` branch. It will trigger a pipeline, which you can watch live under your project's **Pipelines**. + +![pipelines page](img/pipelines_page.png) + +Here we see our **Test** and **Deploy** stages. +The **Test** stage has the `unit_test` build running. +click on it to see the Runner's output. + +![pipeline page](img/pipeline_page.png) + +After our code passed through the pipeline successfully, we can deploy to our production server by clicking the **play** button on the right side. + +![pipelines page deploy button](img/pipelines_page_deploy_button.png) + +Once the deploy pipeline passed successfully, navigate to **Pipelines > Environments**. + +![environments page](img/environments_page.png) + +If something doesn't work as expected, you can roll back to the latest working version of your app. + +![environment page](img/environment_page.png) + +By clicking on the external link icon specified on the right side, GitLab opens the production website. +Our deployment successfully was done and we can see the application is live. + +![laravel welcome page](img/laravel_welcome_page.png) + +In the case that you're interested to know how is the application directory structure on the production server after deployment, here are three directories named `current`, `releases` and `storage`. +As you know, the `current` directory is a symbolic link that points to the latest release. +The `.env` file consists of our Laravel environment variables. + +![production server app directory](img/production_server_app_directory.png) + +If you navigate to the `current` directory, you should see the application's content. +As you see, the `.env` is pointing to the `/var/www/app/.env` file and also `storage` is pointing to the `/var/www/app/storage/` directory. + +![production server current directory](img/production_server_current_directory.png) + +## Conclusion + +We configured GitLab CI to perform automated tests and used the method of [Continuous Delivery](https://continuousdelivery.com/) to deploy to production a Laravel application with Envoy, directly from the codebase. + +Envoy also was a great match to help us deploy the application without writing our custom bash script and doing Linux magics. diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index df0e1521150..b8df0bfba20 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -181,7 +181,7 @@ before_script: ## Assuming you created the SSH_KNOWN_HOSTS variable, uncomment the ## following two lines. ## - - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts' + - echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts - chmod 644 ~/.ssh/known_hosts ## diff --git a/doc/development/README.md b/doc/development/README.md index b624aa37c70..12cca9f84b7 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -27,6 +27,7 @@ comments: false ## Backend guides +- [GitLab utilities](utilities.md) - [API styleguide](api_styleguide.md) Use this styleguide if you are contributing to the API. - [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 54029e00507..d1ba7d3dfc3 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -133,8 +133,6 @@ Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [v ### Log locations of the services -Note: `/home/git/` is shorthand for `/home/git`. - gitlabhq (includes Unicorn and Sidekiq logs) - `/home/git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `unicorn.stdout.log`, `githost.log` and `unicorn.stderr.log` normally. diff --git a/doc/development/changelog.md b/doc/development/changelog.md index 48cffc0dd18..18f4177a5e5 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -127,7 +127,7 @@ type: If you're working on the GitLab EE repository, the entry will be added to `changelogs/unreleased-ee/` instead. -#### Arguments +### Arguments | Argument | Shorthand | Purpose | | ----------------- | --------- | ---------------------------------------------------------------------------------------------------------- | diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 1cd66f27492..02773162801 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -101,16 +101,16 @@ followed by any global declarations, then a blank newline prior to any imports o ``` Import statements are following usual naming guidelines, for example object literals use camel case: - + ```javascript // some_object file export default { key: 'value', }; - + // bad import ObjectLiteral from 'some_object'; - + // good import objectLiteral from 'some_object'; ``` @@ -255,6 +255,10 @@ A forEach will cause side effects, it will be mutating the array being iterated. ### Vue.js +#### `eslint-vue-plugin` +We default to [eslint-vue-plugin][eslint-plugin-vue], with the `plugin:vue/recommended`. +Please check this [rules][eslint-plugin-vue-rules] for more documentation. + #### Basic Rules 1. The service has it's own file 1. The store has it's own file @@ -360,6 +364,10 @@ A forEach will cause side effects, it will be mutating the array being iterated. <component bar="bar" /> + + // bad + <component + bar="bar" /> ``` #### Quotes @@ -509,25 +517,7 @@ On those a default key should not be provided. ``` 1. Properties in a Vue Component: - 1. `name` - 1. `props` - 1. `mixins` - 1. `directives` - 1. `data` - 1. `components` - 1. `computedProps` - 1. `methods` - 1. `beforeCreate` - 1. `created` - 1. `beforeMount` - 1. `mounted` - 1. `beforeUpdate` - 1. `updated` - 1. `activated` - 1. `deactivated` - 1. `beforeDestroy` - 1. `destroyed` - + Check [order of properties in components rule][vue-order]. #### Vue and Bootstrap @@ -582,3 +572,6 @@ The goal of this accord is to make sure we are all on the same page. [eslintrc]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.eslintrc [eslint-this]: http://eslint.org/docs/rules/class-methods-use-this [eslint-new]: http://eslint.org/docs/rules/no-new +[eslint-plugin-vue]: https://github.com/vuejs/eslint-plugin-vue +[eslint-plugin-vue-rules]: https://github.com/vuejs/eslint-plugin-vue#bulb-rules +[vue-order]: https://github.com/vuejs/eslint-plugin-vue/blob/master/docs/rules/order-in-components.md diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 77b308c4a43..86a8b4135af 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -216,7 +216,7 @@ If you want a line or set of lines to be ignored by the linter, you can use ```scss // This lint rule is disabled because the class name comes from a gem. // scss-lint:disable SelectorFormat -.ui_charcoal { +.ui_indigo { background-color: #333; } // scss-lint:enable SelectorFormat diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index ca2048c7019..26abf967dcf 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -97,6 +97,29 @@ describe 'Gitaly Request count tests' do end ``` +## Running tests with a locally modified version of Gitaly + +Normally, gitlab-ce/ee tests use a local clone of Gitaly in `tmp/tests/gitaly` +pinned at the version specified in GITALY_SERVER_VERSION. If you want +to run tests locally against a modified version of Gitaly you can +replace `tmp/tests/gitaly` with a symlink. + +```shell +rm -rf tmp/tests/gitaly +ln -s /path/to/gitaly tmp/tests/gitaly +``` + +Make sure you run `make` in your local Gitaly directory before running +tests. Otherwise, Gitaly will fail to boot. + +If you make changes to your local Gitaly in between test runs you need +to manually run `make` again. + +Note that CI tests will not use your locally modified version of +Gitaly. To use a custom Gitaly version in CI you need to update +GITALY_SERVER_VERSION. You can use the format `=revision` to use a +non-tagged commit from https://gitlab.com/gitlab-org/gitaly in CI. + --- [Return to Development documentation](README.md) diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md new file mode 100644 index 00000000000..5b4f6511f04 --- /dev/null +++ b/doc/development/testing_guide/end_to_end_tests.md @@ -0,0 +1,80 @@ +# End-to-End Testing + +## What is End-to-End testing? + +End-to-End testing is a strategy used to check whether your application works +as expected across entire software stack and architecture, including +integration of all microservices and components that are supposed to work +together. + +## How do we test GitLab? + +We use [Omnibus GitLab][omnibus-gitlab] to build GitLab packages and then we +test these packages using [GitLab QA][gitlab-qa] project, which is entirely +black-box, click-driven testing framework. + +### Testing nightly builds + +We run scheduled pipeline each night to test nightly builds created by Omnibus. +You can find these nightly pipelines at [GitLab QA pipelines page][gitlab-qa-pipelines]. + +### Testing code in merge requests + +It is possible to run end-to-end tests (eventually being run within a +[GitLab QA pipeline][gitlab-qa-pipelines]) for a merge request by triggering +the `package-qa` manual action, that should be present in a merge request +widget. + +Manual action that starts end-to-end tests is also available in merge requests +in Omnibus GitLab project. + +Below you can read more about how to use it and how does it work. + +#### How does it work? + +Currently, we are using _multi-project pipeline_-like approach to run QA +pipelines. + +1. Developer triggers a manual action, that can be found in CE and EE merge +requests. This starts a chain of pipelines in multiple projects. + +1. The script being executed triggers a pipeline in GitLab Omnibus and waits +for the resulting status. We call this a _status attribution_. + +1. GitLab packages are being built in Omnibus pipeline. Packages are going to be +pushed to Container Registry. + +1. When packages are ready, and available in the registry, a final step in the +pipeline, that is now running in Omnibus, triggers a new pipeline in the GitLab +QA project. It also waits for a resulting status. + +1. GitLab QA pulls images from the registry, spins-up containers and runs tests +against a test environment that has been just orchestrated by the `gitlab-qa` +tool. + +1. The result of the GitLab QA pipeline is being propagated upstream, through +Omnibus, back to CE / EE merge request. + +#### How do I write tests? + +In order to write new tests, you first need to learn more about GitLab QA +architecture. See the [documentation about it][gitlab-qa-architecture] in +GitLab QA project. + +Once you decided where to put test environment orchestration scenarios and +instance specs, take a look at the [relevant documentation][instance-qa-readme] +and examples in [the `qa/` directory][instance-qa-examples]. + +## Where can I ask for help? + +You can ask question in the `#qa` channel on Slack (GitLab internal) or you can +find an issue you would like to work on in [the issue tracker][gitlab-qa-issues] +and start a new discussion there. + +[omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab +[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa +[gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines +[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md +[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues +[instance-qa-readme]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md +[instance-qa-examples]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 65386f231a0..74d09eb91ff 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -65,6 +65,13 @@ Everything you should know about how to test Rake tasks. --- +## [End-to-end tests](end_to_end_tests.md) + +Everything you should know about how to run end-to-end tests using +[GitLab QA][gitlab-qa] testing framework. + +--- + ## Spinach (feature) tests GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426) @@ -89,3 +96,4 @@ test should be re-implemented using RSpec instead. [Capybara]: https://github.com/teamcapybara/capybara [Karma]: http://karma-runner.github.io/ [Jasmine]: https://jasmine.github.io/ +[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 1cbd4350284..4adf0dc7c7a 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -121,6 +121,9 @@ running feature tests (i.e. using Capybara) against it. The actual test scenarios and steps are [part of GitLab Rails] so that they're always in-sync with the codebase. +Read a separate document about [end-to-end tests](end_to_end_tests.md) to +learn more. + [multiple pieces]: ../architecture.md#components [GitLab Shell]: https://gitlab.com/gitlab-org/gitlab-shell [GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse diff --git a/doc/development/utilities.md b/doc/development/utilities.md new file mode 100644 index 00000000000..951c3ef85ce --- /dev/null +++ b/doc/development/utilities.md @@ -0,0 +1,92 @@ +# GitLab utilities + +We developed a number of utilities to ease development. + +## [`MergeHash`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/utils/merge_hash.rb) + +* Deep merges an array of hashes: + + ``` ruby + Gitlab::Utils::MergeHash.merge( + [{ hello: ["world"] }, + { hello: "Everyone" }, + { hello: { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } }, + "Goodbye", "Hallo"] + ) + ``` + + Gives: + + ``` ruby + [ + { + hello: + [ + "world", + "Everyone", + { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } + ] + }, + "Goodbye" + ] + ``` + +* Extracts all keys and values from a hash into an array: + + ``` ruby + Gitlab::Utils::MergeHash.crush( + { hello: "world", this: { crushes: ["an entire", "hash"] } } + ) + ``` + + Gives: + + ``` ruby + [:hello, "world", :this, :crushes, "an entire", "hash"] + ``` + +## [`StrongMemoize`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/utils/strong_memoize.rb) + +* Memoize the value even if it is `nil` or `false`. + + We often do `@value ||= compute`, however this doesn't work well if + `compute` might eventually give `nil` and we don't want to compute again. + Instead we could use `defined?` to check if the value is set or not. + However it's tedious to write such pattern, and `StrongMemoize` would + help us use such pattern. + + Instead of writing patterns like this: + + ``` ruby + class Find + def result + return @result if defined?(@result) + + @result = search + end + end + ``` + + We could write it like: + + ``` ruby + class Find + include Gitlab::Utils::StrongMemoize + + def result + strong_memoize(:result) do + search + end + end + end + ``` + +* Clear memoization + + ``` ruby + class Find + include Gitlab::Utils::StrongMemoize + end + + Find.new.clear_memoization(:result) + ``` diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md index 05e0a64af18..9d0c62ecc35 100644 --- a/doc/development/what_requires_downtime.md +++ b/doc/development/what_requires_downtime.md @@ -195,6 +195,63 @@ end And that's it, we're done! +## Changing Column Types For Large Tables + +While `change_column_type_concurrently` can be used for changing the type of a +column without downtime it doesn't work very well for large tables. Because all +of the work happens in sequence the migration can take a very long time to +complete, preventing a deployment from proceeding. +`change_column_type_concurrently` can also produce a lot of pressure on the +database due to it rapidly updating many rows in sequence. + +To reduce database pressure you should instead use +`change_column_type_using_background_migration` when migrating a column in a +large table (e.g. `issues`). This method works similar to +`change_column_type_concurrently` but uses background migration to spread the +work / load over a longer time period, without slowing down deployments. + +Usage of this method is fairly simple: + +```ruby +class ExampleMigration < ActiveRecord::Migration + include Gitlab::Database::MigrationHelpers + + disable_ddl_transaction! + + class Issue < ActiveRecord::Base + self.table_name = 'issues' + + include EachBatch + + def self.to_migrate + where('closed_at IS NOT NULL') + end + end + + def up + change_column_type_using_background_migration( + Issue.to_migrate, + :closed_at, + :datetime_with_timezone + ) + end + + def down + change_column_type_using_background_migration( + Issue.to_migrate, + :closed_at, + :datetime + ) + end +end +``` + +This would change the type of `issues.closed_at` to `timestamp with time zone`. + +Keep in mind that the relation passed to +`change_column_type_using_background_migration` _must_ include `EachBatch`, +otherwise it will raise a `TypeError`. + ## Adding Indexes Adding indexes is an expensive process that blocks INSERT and UPDATE queries for diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index 43a79ffcaa5..133ac0234cf 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -15,7 +15,7 @@ request introducing these changes must be accompanied by the documentation (either updating existing ones or creating new ones). This is also valid when changes are introduced to the UI. -The one resposible for writing the first piece of documentation is the developer who +The one responsible for writing the first piece of documentation is the developer who wrote the code. It's the job of the Product Manager to ensure all features are shipped with its docs, whether is a small or big change. At the pace GitLab evolves, this is the only way to keep the docs up-to-date. If you have any questions about it, @@ -31,7 +31,7 @@ Every major feature (regardless if present in GitLab Community or Enterprise edi should present, at the beginning of the document, two main sections: **overview** and **use cases**. Every GitLab EE-only feature should also contain these sections. -**Overview**: at the name suggests, the goal here is to provide an overview of the feature. +**Overview**: as the name suggests, the goal here is to provide an overview of the feature. Describe what is it, what it does, why it is important/cool/nice-to-have, what problem it solves, and what you can do with this feature that you couldn't do before. diff --git a/doc/install/README.md b/doc/install/README.md index 540cb0d3f38..43197351db3 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -27,7 +27,7 @@ the hardware requirements. - [Installing in Kubernetes](kubernetes/index.md) - Install GitLab into a Kubernetes Cluster using our official Helm Chart Repository. -- [Install GitLab on OpenShift](../articles/openshift_and_gitlab/index.md) +- [Install GitLab on OpenShift](openshift_and_gitlab/index.md) - [Install GitLab on DC/OS](https://mesosphere.com/blog/gitlab-dcos/) via [GitLab-Mesosphere integration](https://about.gitlab.com/2016/09/16/announcing-gitlab-and-mesosphere/) - [Install GitLab on Azure](azure/index.md) - [Install GitLab on Google Cloud Platform](google_cloud_platform/index.md) diff --git a/doc/install/installation.md b/doc/install/installation.md index 67b89d608cc..b2acd5e78b5 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -64,7 +64,7 @@ up-to-date and install it. Install the required packages (needed to compile Ruby and native extensions to Ruby gems): - sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate python-docutils pkg-config cmake + sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate rsync python-docutils pkg-config cmake Ubuntu 14.04 (Trusty Tahr) doesn't have the `libre2-dev` package available, but you can [install re2 manually](https://github.com/google/re2/wiki/Install). @@ -299,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-3-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-4-stable gitlab -**Note:** You can change `10-3-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `10-4-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index 0932e1eee3a..cd889e74487 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -9,11 +9,11 @@ should be deployed, upgraded, and configured. ## Chart Overview -* **[GitLab-Omnibus](gitlab_omnibus.md)**: The best way to run GitLab on Kubernetes today, suited for small to medium deployments. The chart is in beta and will be deprecated by the [cloud native GitLab chart](#cloud-native-gitlab-chart). +* **[GitLab-Omnibus](gitlab_omnibus.md)**: The best way to run GitLab on Kubernetes today, suited for small deployments. The chart is in beta and will be deprecated by the [cloud native GitLab chart](#cloud-native-gitlab-chart). * **[Cloud Native GitLab Chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md)**: The next generation GitLab chart, currently in development. Will support large deployments with horizontal scaling of individual GitLab components. * Other Charts * [GitLab Runner Chart](gitlab_runner_chart.md): For deploying just the GitLab Runner. - * [Advanced GitLab Installation](gitlab_chart.md): Deprecated, being replaced by the [cloud native GitLab chart](#cloud-native-gitlab-chart). Provides additional deployment options, but provides less functionality out-of-the-box. + * [Advanced GitLab Installation](gitlab_chart.md): Deprecated, being replaced by the [cloud native GitLab chart](#cloud-native-gitlab-chart). Provides additional deployment options, but provides less functionality out-of-the-box. * [Community Contributed Charts](#community-contributed-charts): Community contributed charts, deprecated by the official GitLab chart. ## GitLab-Omnibus Chart (Recommended) diff --git a/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png b/doc/install/openshift_and_gitlab/img/add-gitlab-to-project.png Binary files differindex fcad4e59ae3..fcad4e59ae3 100644 --- a/doc/articles/openshift_and_gitlab/img/add-gitlab-to-project.png +++ b/doc/install/openshift_and_gitlab/img/add-gitlab-to-project.png diff --git a/doc/articles/openshift_and_gitlab/img/add-to-project.png b/doc/install/openshift_and_gitlab/img/add-to-project.png Binary files differindex bd915a229f6..bd915a229f6 100644 --- a/doc/articles/openshift_and_gitlab/img/add-to-project.png +++ b/doc/install/openshift_and_gitlab/img/add-to-project.png diff --git a/doc/articles/openshift_and_gitlab/img/create-project-ui.png b/doc/install/openshift_and_gitlab/img/create-project-ui.png Binary files differindex e72866f252a..e72866f252a 100644 --- a/doc/articles/openshift_and_gitlab/img/create-project-ui.png +++ b/doc/install/openshift_and_gitlab/img/create-project-ui.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-logs.png b/doc/install/openshift_and_gitlab/img/gitlab-logs.png Binary files differindex 1e24080c7df..1e24080c7df 100644 --- a/doc/articles/openshift_and_gitlab/img/gitlab-logs.png +++ b/doc/install/openshift_and_gitlab/img/gitlab-logs.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-overview.png b/doc/install/openshift_and_gitlab/img/gitlab-overview.png Binary files differindex 3c5df0ea101..3c5df0ea101 100644 --- a/doc/articles/openshift_and_gitlab/img/gitlab-overview.png +++ b/doc/install/openshift_and_gitlab/img/gitlab-overview.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-running.png b/doc/install/openshift_and_gitlab/img/gitlab-running.png Binary files differindex c7db691cb30..c7db691cb30 100644 --- a/doc/articles/openshift_and_gitlab/img/gitlab-running.png +++ b/doc/install/openshift_and_gitlab/img/gitlab-running.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-scale.png b/doc/install/openshift_and_gitlab/img/gitlab-scale.png Binary files differindex 4903c7d7498..4903c7d7498 100644 --- a/doc/articles/openshift_and_gitlab/img/gitlab-scale.png +++ b/doc/install/openshift_and_gitlab/img/gitlab-scale.png diff --git a/doc/articles/openshift_and_gitlab/img/gitlab-settings.png b/doc/install/openshift_and_gitlab/img/gitlab-settings.png Binary files differindex db4360ffef0..db4360ffef0 100644 --- a/doc/articles/openshift_and_gitlab/img/gitlab-settings.png +++ b/doc/install/openshift_and_gitlab/img/gitlab-settings.png diff --git a/doc/articles/openshift_and_gitlab/img/no-resources.png b/doc/install/openshift_and_gitlab/img/no-resources.png Binary files differindex 480fb766468..480fb766468 100644 --- a/doc/articles/openshift_and_gitlab/img/no-resources.png +++ b/doc/install/openshift_and_gitlab/img/no-resources.png diff --git a/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png b/doc/install/openshift_and_gitlab/img/openshift-infra-project.png Binary files differindex 8b9f85aa341..8b9f85aa341 100644 --- a/doc/articles/openshift_and_gitlab/img/openshift-infra-project.png +++ b/doc/install/openshift_and_gitlab/img/openshift-infra-project.png diff --git a/doc/articles/openshift_and_gitlab/img/pods-overview.png b/doc/install/openshift_and_gitlab/img/pods-overview.png Binary files differindex e1cf08bd217..e1cf08bd217 100644 --- a/doc/articles/openshift_and_gitlab/img/pods-overview.png +++ b/doc/install/openshift_and_gitlab/img/pods-overview.png diff --git a/doc/articles/openshift_and_gitlab/img/rc-name.png b/doc/install/openshift_and_gitlab/img/rc-name.png Binary files differindex 889e34adbec..889e34adbec 100644 --- a/doc/articles/openshift_and_gitlab/img/rc-name.png +++ b/doc/install/openshift_and_gitlab/img/rc-name.png diff --git a/doc/articles/openshift_and_gitlab/img/running-pods.png b/doc/install/openshift_and_gitlab/img/running-pods.png Binary files differindex 3fd4e56662f..3fd4e56662f 100644 --- a/doc/articles/openshift_and_gitlab/img/running-pods.png +++ b/doc/install/openshift_and_gitlab/img/running-pods.png diff --git a/doc/articles/openshift_and_gitlab/img/storage-volumes.png b/doc/install/openshift_and_gitlab/img/storage-volumes.png Binary files differindex ae1e5381faa..ae1e5381faa 100644 --- a/doc/articles/openshift_and_gitlab/img/storage-volumes.png +++ b/doc/install/openshift_and_gitlab/img/storage-volumes.png diff --git a/doc/articles/openshift_and_gitlab/img/web-console.png b/doc/install/openshift_and_gitlab/img/web-console.png Binary files differindex aa1425d4f94..aa1425d4f94 100644 --- a/doc/articles/openshift_and_gitlab/img/web-console.png +++ b/doc/install/openshift_and_gitlab/img/web-console.png diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md new file mode 100644 index 00000000000..8fba44aea02 --- /dev/null +++ b/doc/install/openshift_and_gitlab/index.md @@ -0,0 +1,510 @@ +# Getting started with OpenShift Origin 3 and GitLab + +> **[Article Type](../../development/writing_documentation.html#types-of-technical-articles):** tutorial || +> **Level:** intermediary || +> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) || +> **Publication date:** 2016-06-28 + +## Introduction + +[OpenShift Origin][openshift] is an open source container application +platform created by [RedHat], based on [kubernetes] and [Docker]. That means +you can host your own PaaS for free and almost with no hassle. + +In this tutorial, we will see how to deploy GitLab in OpenShift using GitLab's +official Docker image while getting familiar with the web interface and CLI +tools that will help us achieve our goal. + +--- + +## Prerequisites + +OpenShift 3 is not yet deployed on RedHat's offered Online platform ([openshift.com]), +so in order to test it, we will use an [all-in-one Virtualbox image][vm] that is +offered by the OpenShift developers and managed by Vagrant. If you haven't done +already, go ahead and install the following components as they are essential to +test OpenShift easily: + +- [VirtualBox] +- [Vagrant] +- [OpenShift Client][oc] (`oc` for short) + +It is also important to mention that for the purposes of this tutorial, the +latest Origin release is used: + +- **oc** `v1.3.0` (must be [installed][oc-gh] locally on your computer) +- **openshift** `v1.3.0` (is pre-installed in the [VM image][vm-new]) +- **kubernetes** `v1.3.0` (is pre-installed in the [VM image][vm-new]) + +>**Note:** +If you intend to deploy GitLab on a production OpenShift cluster, there are some +limitations to bare in mind. Read on the [limitations](#current-limitations) +section for more information and follow the linked links for the relevant +discussions. + +Now that you have all batteries, let's see how easy it is to test OpenShift +on your computer. + +## Getting familiar with OpenShift Origin + +The environment we are about to use is based on CentOS 7 which comes with all +the tools needed pre-installed: Docker, kubernetes, OpenShift, etcd. + +### Test OpenShift using Vagrant + +As of this writing, the all-in-one VM is at version 1.3, and that's +what we will use in this tutorial. + +In short: + +1. Open a terminal and in a new directory run: + ```sh + vagrant init openshift/origin-all-in-one + ``` +1. This will generate a Vagrantfile based on the all-in-one VM image +1. In the same directory where you generated the Vagrantfile + enter: + + ```sh + vagrant up + ``` + +This will download the VirtualBox image and fire up the VM with some preconfigured +values as you can see in the Vagrantfile. As you may have noticed, you need +plenty of RAM (5GB in our example), so make sure you have enough. + +Now that OpenShift is setup, let's see how the web console looks like. + +### Explore the OpenShift web console + +Once Vagrant finishes its thing with the VM, you will be presented with a +message which has some important information. One of them is the IP address +of the deployed OpenShift platform and in particular <https://10.2.2.2:8443/console/>. +Open this link with your browser and accept the self-signed certificate in +order to proceed. + +Let's login as admin with username/password `admin/admin`. This is what the +landing page looks like: + +![openshift web console](img/web-console.png) + +You can see that a number of [projects] are already created for testing purposes. + +If you head over the `openshift-infra` project, a number of services with their +respective pods are there to explore. + +![openshift web console](img/openshift-infra-project.png) + +We are not going to explore the whole interface, but if you want to learn about +the key concepts of OpenShift, read the [core concepts reference][core] in the +official documentation. + +### Explore the OpenShift CLI + +OpenShift Client (`oc`), is a powerful CLI tool that talks to the OpenShift API +and performs pretty much everything you can do from the web UI and much more. + +Assuming you have [installed][oc] it, let's explore some of its main +functionalities. + +Let's first see the version of `oc`: + +```sh +$ oc version + +oc v1.3.0 +kubernetes v1.3.0+52492b4 +``` + +With `oc help` you can see the top level arguments you can run with `oc` and +interact with your cluster, kubernetes, run applications, create projects and +much more. + +Let's login to the all-in-one VM and see how to achieve the same results like +when we visited the web console earlier. The username/password for the +administrator user is `admin/admin`. There is also a test user with username/ +password `user/user`, with limited access. Let's login as admin for the moment: + +```sh +$ oc login https://10.2.2.2:8443 + +Authentication required for https://10.2.2.2:8443 (openshift) +Username: admin +Password: +Login successful. + +You have access to the following projects and can switch between them with 'oc project <projectname>': + + * cockpit + * default (current) + * delete + * openshift + * openshift-infra + * sample + +Using project "default". +``` + +Switch to the `openshift-infra` project with: + +```sh +oc project openshift-infra +``` + +And finally, see its status: + +```sh +oc status +``` + +The last command should spit a bunch of information about the statuses of the +pods and the services, which if you look closely is what we encountered in the +second image when we explored the web console. + +You can always read more about `oc` in the [OpenShift CLI documentation][oc]. + +### Troubleshooting the all-in-one VM + +Using the all-in-one VM gives you the ability to test OpenShift whenever you +want. That means you get to play with it, shutdown the VM, and pick up where +you left off. + +Sometimes though, you may encounter some issues, like OpenShift not running +when booting up the VM. The web UI may not responding or you may see issues +when trying to login with `oc`, like: + +``` +The connection to the server 10.2.2.2:8443 was refused - did you specify the right host or port? +``` + +In that case, the OpenShift service might not be running, so in order to fix it: + +1. SSH into the VM by going to the directory where the Vagrantfile is and then + run: + + ```sh + vagrant ssh + ``` + +1. Run `systemctl` and verify by the output that the `openshift` service is not + running (it will be in red color). If that's the case start the service with: + + ```sh + sudo systemctl start openshift + ``` + +1. Verify the service is up with: + + ```sh + systemctl status openshift -l + ``` + +Now you will be able to login using `oc` (like we did before) and visit the web +console. + +## Deploy GitLab + +Now that you got a taste of what OpenShift looks like, let's deploy GitLab! + +### Create a new project + +First, we will create a new project to host our application. You can do this +either by running the CLI client: + +```bash +$ oc new-project gitlab +``` + +or by using the web interface: + +![Create a new project from the UI](img/create-project-ui.png) + +If you used the command line, `oc` automatically uses the new project and you +can see its status with: + +```sh +$ oc status + +In project gitlab on server https://10.2.2.2:8443 + +You have no services, deployment configs, or build configs. +Run 'oc new-app' to create an application. +``` + +If you visit the web console, you can now see `gitlab` listed in the projects list. + +The next step is to import the OpenShift template for GitLab. + +### Import the template + +The [template][templates] is basically a JSON file which describes a set of +related object definitions to be created together, as well as a set of +parameters for those objects. + +The template for GitLab resides in the Omnibus GitLab repository under the +docker directory. Let's download it locally with `wget`: + +```bash +wget https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/docker/openshift-template.json +``` + +And then let's import it in OpenShift: + +```bash +oc create -f openshift-template.json -n openshift +``` + +>**Note:** +The `-n openshift` namespace flag is a trick to make the template available to all +projects. If you recall from when we created the `gitlab` project, `oc` switched +to it automatically, and that can be verified by the `oc status` command. If +you omit the namespace flag, the application will be available only to the +current project, in our case `gitlab`. The `openshift` namespace is a global +one that the administrators should use if they want the application to be +available to all users. + +We are now ready to finally deploy GitLab! + +### Create a new application + +The next step is to use the template we previously imported. Head over to the +`gitlab` project and hit the **Add to Project** button. + +![Add to project](img/add-to-project.png) + +This will bring you to the catalog where you can find all the pre-defined +applications ready to deploy with the click of a button. Search for `gitlab` +and you will see the previously imported template: + +![Add GitLab to project](img/add-gitlab-to-project.png) + +Select it, and in the following screen you will be presented with the predefined +values used with the GitLab template: + +![GitLab settings](img/gitlab-settings.png) + +Notice at the top that there are three resources to be created with this +template: + +- `gitlab-ce` +- `gitlab-ce-redis` +- `gitlab-ce-postgresql` + +While PostgreSQL and Redis are bundled in Omnibus GitLab, the template is using +separate images as you can see from [this line][line] in the template. + +The predefined values have been calculated for the purposes of testing out +GitLab in the all-in-one VM. You don't need to change anything here, hit +**Create** to start the deployment. + +If you are deploying to production you will want to change the **GitLab instance +hostname** and use greater values for the volume sizes. If you don't provide a +password for PostgreSQL, it will be created automatically. + +>**Note:** +The `gitlab.apps.10.2.2.2.xip.io` hostname that is used by default will +resolve to the host with IP `10.2.2.2` which is the IP our VM uses. It is a +trick to have distinct FQDNs pointing to services that are on our local network. +Read more on how this works in <http://xip.io>. + +Now that we configured this, let's see how to manage and scale GitLab. + +## Manage and scale GitLab + +Setting up GitLab for the first time might take a while depending on your +internet connection and the resources you have attached to the all-in-one VM. +GitLab's docker image is quite big (~500MB), so you'll have to wait until +it's downloaded and configured before you use it. + +### Watch while GitLab gets deployed + +Navigate to the `gitlab` project at **Overview**. You can notice that the +deployment is in progress by the orange color. The Docker images are being +downloaded and soon they will be up and running. + +![GitLab overview](img/gitlab-overview.png) + +Switch to the **Browse > Pods** and you will eventually see all 3 pods in a +running status. Remember the 3 resources that were to be created when we first +created the GitLab app? This is where you can see them in action. + +![Running pods](img/running-pods.png) + +You can see GitLab being reconfigured by taking look at the logs in realtime. +Click on `gitlab-ce-2-j7ioe` (your ID will be different) and go to the **Logs** +tab. + +![GitLab logs](img/gitlab-logs.png) + +At a point you should see a _**gitlab Reconfigured!**_ message in the logs. +Navigate back to the **Overview** and hopefully all pods will be up and running. + +![GitLab running](img/gitlab-running.png) + +Congratulations! You can now navigate to your new shinny GitLab instance by +visiting <http://gitlab.apps.10.2.2.2.xip.io> where you will be asked to +change the root user password. Login using `root` as username and providing the +password you just set, and start using GitLab! + +### Scale GitLab with the push of a button + +If you reach to a point where your GitLab instance could benefit from a boost +of resources, you'd be happy to know that you can scale up with the push of a +button. + +In the **Overview** page just click the up arrow button in the pod where +GitLab is. The change is instant and you can see the number of [replicas] now +running scaled to 2. + +![GitLab scale](img/gitlab-scale.png) + +Upping the GitLab pods is actually like adding new application servers to your +cluster. You can see how that would work if you didn't use GitLab with +OpenShift by following the [HA documentation][ha] for the application servers. + +Bare in mind that you may need more resources (CPU, RAM, disk space) when you +scale up. If a pod is in pending state for too long, you can navigate to +**Browse > Events** and see the reason and message of the state. + +![No resources](img/no-resources.png) + +### Scale GitLab using the `oc` CLI + +Using `oc` is super easy to scale up the replicas of a pod. You may want to +skim through the [basic CLI operations][basic-cli] to get a taste how the CLI +commands are used. Pay extra attention to the object types as we will use some +of them and their abbreviated versions below. + +In order to scale up, we need to find out the name of the replication controller. +Let's see how to do that using the following steps. + +1. Make sure you are in the `gitlab` project: + + ```sh + oc project gitlab + ``` + +1. See what services are used for this project: + + ```sh + oc get svc + ``` + + The output will be similar to: + + ``` + NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE + gitlab-ce 172.30.243.177 <none> 22/TCP,80/TCP 5d + gitlab-ce-postgresql 172.30.116.75 <none> 5432/TCP 5d + gitlab-ce-redis 172.30.105.88 <none> 6379/TCP 5d + ``` + +1. We need to see the replication controllers of the `gitlab-ce` service. + Get a detailed view of the current ones: + + ```sh + oc describe rc gitlab-ce + ``` + + This will return a large detailed list of the current replication controllers. + Search for the name of the GitLab controller, usually `gitlab-ce-1` or if + that failed at some point and you spawned another one, it will be named + `gitlab-ce-2`. + +1. Scale GitLab using the previous information: + + ```sh + oc scale --replicas=2 replicationcontrollers gitlab-ce-2 + ``` + +1. Get the new replicas number to make sure scaling worked: + + ```sh + oc get rc gitlab-ce-2 + ``` + + which will return something like: + + ``` + NAME DESIRED CURRENT AGE + gitlab-ce-2 2 2 5d + ``` + +And that's it! We successfully scaled the replicas to 2 using the CLI. + +As always, you can find the name of the controller using the web console. Just +click on the service you are interested in and you will see the details in the +right sidebar. + +![Replication controller name](img/rc-name.png) + +### Autoscaling GitLab + +In case you were wondering whether there is an option to autoscale a pod based +on the resources of your server, the answer is yes, of course there is. + +We will not expand on this matter, but feel free to read the documentation on +OpenShift's website about [autoscaling]. + +## Current limitations + +As stated in the [all-in-one VM][vm] page: + +> By default, OpenShift will not allow a container to run as root or even a +non-random container assigned userid. Most Docker images in the Dockerhub do not +follow this best practice and instead run as root. + +The all-in-one VM we are using has this security turned off so it will not +bother us. In any case, it is something to keep in mind when deploying GitLab +on a production cluster. + +In order to deploy GitLab on a production cluster, you will need to assign the +GitLab service account to the `anyuid` Security Context. + +1. Edit the Security Context: + ```sh + oc edit scc anyuid + ``` + +1. Add `system:serviceaccount:<project>:gitlab-ce-user` to the `users` section. + If you changed the Application Name from the default the user will + will be `<app-name>-user` instead of `gitlab-ce-user` + +1. Save and exit the editor + +## Conclusion + +By now, you should have an understanding of the basic OpenShift Origin concepts +and a sense of how things work using the web console or the CLI. + +GitLab was hard to install in previous versions of OpenShift, +but now that belongs to the past. Upload a template, create a project, add an +application and you are done. You are ready to login to your new GitLab instance. + +And remember that in this tutorial we just scratched the surface of what Origin +is capable of. As always, you can refer to the detailed +[documentation][openshift-docs] to learn more about deploying your own OpenShift +PaaS and managing your applications with the ease of containers. + +[RedHat]: https://www.redhat.com/en "RedHat website" +[openshift]: https://www.openshift.org "OpenShift Origin website" +[vm]: https://www.openshift.org/vm/ "OpenShift All-in-one VM" +[vm-new]: https://atlas.hashicorp.com/openshift/boxes/origin-all-in-one "Official OpenShift Vagrant box on Atlas" +[template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/docker/openshift-template.json "OpenShift template for GitLab" +[openshift.com]: https://openshift.com "OpenShift Online" +[kubernetes]: http://kubernetes.io/ "Kubernetes website" +[Docker]: https://www.docker.com "Docker website" +[oc]: https://docs.openshift.org/latest/cli_reference/get_started_cli.html "Documentation - oc CLI documentation" +[VirtualBox]: https://www.virtualbox.org/wiki/Downloads "VirtualBox downloads" +[Vagrant]: https://www.vagrantup.com/downloads.html "Vagrant downloads" +[projects]: https://docs.openshift.org/latest/dev_guide/projects.html "Documentation - Projects overview" +[core]: https://docs.openshift.org/latest/architecture/core_concepts/index.html "Documentation - Core concepts of OpenShift Origin" +[templates]: https://docs.openshift.org/latest/architecture/core_concepts/templates.html "Documentation - OpenShift templates" +[old-post]: https://blog.openshift.com/deploy-gitlab-openshift/ "Old post - Deploy GitLab on OpenShift" +[line]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/658c065c8d022ce858dd63eaeeadb0b2ddc8deea/docker/openshift-template.json#L239 "GitLab - OpenShift template" +[oc-gh]: https://github.com/openshift/origin/releases/tag/v1.3.0 "Openshift 1.3.0 release on GitHub" +[ha]: ../../administration/high_availability/gitlab.html "Documentation - GitLab High Availability" +[replicas]: https://docs.openshift.org/latest/architecture/core_concepts/deployments.html#replication-controllers "Documentation - Replication controller" +[autoscaling]: https://docs.openshift.org/latest/dev_guide/pod_autoscaling.html "Documentation - Autoscale" +[basic-cli]: https://docs.openshift.org/latest/cli_reference/basic_cli_operations.html "Documentation - Basic CLI operations" +[openshift-docs]: https://docs.openshift.org "OpenShift documentation" diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 54c3e20d61d..50bb665216e 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -5,8 +5,8 @@ An application data backup creates an archive file that contains the database, all repositories and all attachments. -You can only restore a backup to **exactly the same version and type (CE/EE)** -of GitLab on which it was created. The best way to migrate your repositories +You can only restore a backup to **exactly the same version and type (CE/EE)** +of GitLab on which it was created. The best way to migrate your repositories from one server to another is through backup restore. ## Backup @@ -14,6 +14,19 @@ from one server to another is through backup restore. GitLab provides a simple command line interface to backup your whole installation, and is flexible enough to fit your needs. +### Requirements + +If you're using GitLab with the Omnibus package, you're all set. If you +installed GitLab from source, make sure the following packages are installed: + +* rsync + +If you're using Ubuntu, you could run: + +``` +sudo apt-get install -y rsync +``` + ### Backup timestamp >**Note:** @@ -431,7 +444,7 @@ The [restore prerequisites section](#restore-prerequisites) includes crucial information. Make sure to read and test the whole restore process at least once before attempting to perform it in a production environment. -You can only restore a backup to **exactly the same version and type (CE/EE)** of +You can only restore a backup to **exactly the same version and type (CE/EE)** of GitLab that you created it on, for example CE 9.1.0. ### Restore prerequisites @@ -511,7 +524,7 @@ sudo service gitlab restart This procedure assumes that: -- You have installed the **exact same version and type (CE/EE)** of GitLab +- You have installed the **exact same version and type (CE/EE)** of GitLab Omnibus with which the backup was created. - You have run `sudo gitlab-ctl reconfigure` at least once. - GitLab is running. If not, start it using `sudo gitlab-ctl start`. diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 339bc2bd4fe..e23c73f46fb 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -20,6 +20,7 @@ project in an easy and automatic way: 1. [Auto Test](#auto-test) 1. [Auto Code Quality](#auto-code-quality) 1. [Auto SAST (Static Application Security Testing)](#auto-sast) +1. [Auto Browser Performance Testing](#auto-browser-performance-testing) 1. [Auto Review Apps](#auto-review-apps) 1. [Auto Deploy](#auto-deploy) 1. [Auto Monitoring](#auto-monitoring) @@ -208,6 +209,20 @@ check out. Any security warnings are also [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html). +### Auto Browser Performance Testing + +> Introduced in [GitLab Enterprise Edition Premium][ee] 10.4. + +Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://hub.docker.com/r/sitespeedio/sitespeed.io/) to measure the performance of a web page. A JSON report is created and uploaded as an artifact, which includes the overall performance score for each page. By default, the root page of Review and Production environments will be tested. If you would like to add additional URL's to test, simply add the paths to a file named `.gitlab-urls.txt` in the root directory, one per line. For example: + +``` +/ +/features +/direction +``` + +In GitLab Enterprise Edition Premium, performance differences between the source and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). + ### Auto Review Apps NOTE: **Note:** diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md new file mode 100644 index 00000000000..cdf61057449 --- /dev/null +++ b/doc/topics/git/how_to_install_git/index.md @@ -0,0 +1,66 @@ +# Installing Git + +> **[Article Type](../../../development/writing_documentation.html#types-of-technical-articles):** user guide || +> **Level:** beginner || +> **Author:** [Sean Packham](https://gitlab.com/SeanPackham) || +> **Publication date:** 2017-05-15 + +To begin contributing to GitLab projects +you will need to install the Git client on your computer. +This article will show you how to install Git on macOS, Ubuntu Linux and Windows. + +## Install Git on macOS using the Homebrew package manager + +Although it is easy to use the version of Git shipped with macOS +or install the latest version of Git on macOS by downloading it from the project website, +we recommend installing it via Homebrew to get access to +an extensive selection of dependancy managed libraries and applications. + +If you are sure you don't need access to any additional development libraries +or don't have approximately 15gb of available disk space for Xcode and Homebrew +use one of the the aforementioned methods. + +### Installing Xcode + +Xcode is needed by Homebrew to build dependencies. +You can install [XCode](https://developer.apple.com/xcode/) +through the macOS App Store. + +### Installing Homebrew + +Once Xcode is installed browse to the [Homebrew website](http://brew.sh/index.html) +for the official Homebrew installation instructions. + +### Installing Git via Homebrew + +With Homebrew installed you are now ready to install Git. +Open a Terminal and enter in the following command: + +```bash +brew install git +``` + +Congratulations you should now have Git installed via Homebrew. +Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). + +## Install Git on Ubuntu Linux + +On Ubuntu and other Linux operating systems +it is recommended to use the built in package manager to install Git. + +Open a Terminal and enter in the following commands +to install the latest Git from the official Git maintained package archives: + +```bash +sudo apt-add-repository ppa:git-core/ppa +sudo apt-get update +sudo apt-get install git +``` + +Congratulations you should now have Git installed via the Ubuntu package manager. +Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). + +## Installing Git on Windows from the Git website + +Browse to the [Git website](https://git-scm.com/) and download and install Git for Windows. +Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 588f4fa369f..f69e2e49f0c 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -14,6 +14,7 @@ We've gathered some resources to help you to get the best from Git with GitLab. ## Getting started - [Git concepts](../../university/training/user_training.md#git-concepts) +- [How to install Git](how_to_install_git/index.md) - [Start using Git on the command line](../../gitlab-basics/start-using-git.md) - [Command Line basic commands](../../gitlab-basics/command-line-commands.md) - [GitLab Git Cheat Sheet (download)](https://gitlab.com/gitlab-com/marketing/raw/master/design/print/git-cheatsheet/print-pdf/git-cheatsheet.pdf) @@ -21,27 +22,39 @@ We've gathered some resources to help you to get the best from Git with GitLab. - [Revert a commit](../../user/project/merge_requests/revert_changes.md#reverting-a-commit) - [Cherry-picking a commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-picking-a-commit) - [Squashing commits](../../workflow/gitlab_flow.md#squashing-commits-with-rebase) -- **Articles:** - - [Numerous _undo_ possibilities in Git](../../articles/numerous_undo_possibilities_in_git/index.md) - - [How to install Git](../../articles/how_to_install_git/index.md) - - [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) - - [Eight Tips to help you work better with Git](https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/) -- **Presentations:** - - [GLU Course: About Version Control](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit?usp=sharing) -- **Third-party resources:** - - What is [Git](https://git-scm.com) - - [Version control](https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control) - - [Getting Started - Git Basics](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) - - [Getting Started - Installing Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) - - [Git on the Server - GitLab](https://git-scm.com/book/en/v2/Git-on-the-Server-GitLab) + +**Third-party references:** + +- [Getting Started - Git website](https://git-scm.com) +- [Getting Started - Version control](https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control) +- [Getting Started - Git Basics](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) +- [Getting Started - Installing Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +- [Git on the Server - GitLab](https://git-scm.com/book/en/v2/Git-on-the-Server-GitLab) + +### Concepts + +- Article (2017-05-17): [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) +- Article (2016-05-11): [The future of SaaS hosted Git repository pricing](https://about.gitlab.com/2016/05/11/git-repository-pricing/) +- GLU Course (Presentation): [About Version Control](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit?usp=sharing) + +## Exploring Git + +- [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) +- [Eight Tips to help you work better with Git](https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/) + +## Troubleshooting Git + +- [Numerous _undo_ possibilities in Git](../../articles/numerous_undo_possibilities_in_git/index.md) +- Learn a few [Git troubleshooting](troubleshooting_git.md) techniques to help you out. ## Branching strategies -- **Articles:** - - [GitLab Flow](https://about.gitlab.com/2014/09/29/gitlab-flow/) -- **Third-party resources:** - - [Git Branching - Branches in a Nutshell](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell) - - [Git Branching - Branching Workflows](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows) +- [GitLab Flow](https://about.gitlab.com/2014/09/29/gitlab-flow/) + +**Third-party references:** + +- [Git Branching - Branches in a Nutshell](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell) +- [Git Branching - Branching Workflows](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows) ## Advanced use @@ -55,17 +68,7 @@ We've gathered some resources to help you to get the best from Git with GitLab. ## Git LFS -- [Git LFS](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) +- [GitLab Git LFS documentation](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) - [Git-Annex to Git-LFS migration guide](https://docs.gitlab.com/ee/workflow/lfs/migrate_from_git_annex_to_git_lfs.html) -- **Articles:** - - [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) - - [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) - -## Troubleshooting - -- Learn a few [Git troubleshooting](troubleshooting_git.md) techniques to help you out. - -## General information - -- **Articles:** - - [The future of SaaS hosted Git repository pricing](https://about.gitlab.com/2016/05/11/git-repository-pricing/) +- Article (2015-08-13): [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 54625996dff..ddc853afded 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -26,6 +26,10 @@ Login to your AWS account through the `My Account` dropdown on Amazon Web Services console from where we can choose all of the services we'll be using to configure our cloud infrastructure. +### Reference Architecture + +![Reference Architecture](img/reference-arch.png) + *** ## Network diff --git a/doc/university/high-availability/aws/img/reference-arch.png b/doc/university/high-availability/aws/img/reference-arch.png Binary files differnew file mode 100644 index 00000000000..271ee5bc614 --- /dev/null +++ b/doc/university/high-availability/aws/img/reference-arch.png diff --git a/doc/update/10.3-to-10.4.md b/doc/update/10.3-to-10.4.md index 850cb3103f4..67b7e634c94 100644 --- a/doc/update/10.3-to-10.4.md +++ b/doc/update/10.3-to-10.4.md @@ -21,6 +21,8 @@ sudo service gitlab stop ### 2. Backup +NOTE: If you installed GitLab from source, make sure `rsync` is installed. + ```bash cd /home/git/gitlab diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 4fa83388d0c..708d07fcec9 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -200,7 +200,7 @@ instance and project. In addition, all admins can use the admin interface under |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | -| Erase job artifacts and trace | | ✓ [^7] | ✓ | ✓ | +| Erase job artifacts and trace | | ✓ [^5] | ✓ | ✓ | | Remove project | | | ✓ | ✓ | | Create project | | | ✓ | ✓ | | Change project configuration | | | ✓ | ✓ | @@ -223,13 +223,13 @@ users: | Run CI job | | ✓ | ✓ | ✓ | | Clone source and LFS from current project | | ✓ | ✓ | ✓ | | Clone source and LFS from public projects | | ✓ | ✓ | ✓ | -| Clone source and LFS from internal projects | | ✓ [^5] | ✓ [^5] | ✓ | -| Clone source and LFS from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | +| Clone source and LFS from internal projects | | ✓ [^6] | ✓ [^6] | ✓ | +| Clone source and LFS from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | | Push source and LFS | | | | | | Pull container images from current project | | ✓ | ✓ | ✓ | | Pull container images from public projects | | ✓ | ✓ | ✓ | -| Pull container images from internal projects| | ✓ [^5] | ✓ [^5] | ✓ | -| Pull container images from private projects | | ✓ [^6] | ✓ [^6] | ✓ [^6] | +| Pull container images from internal projects| | ✓ [^6] | ✓ [^6] | ✓ | +| Pull container images from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | | Push container images to current project | | ✓ | ✓ | ✓ | | Push container images to other projects | | | | | @@ -259,12 +259,13 @@ with the permissions described on the documentation on [auditor users permission Auditor users are available in [GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/) only. -[^1]: On public and internal projects, all users are able to perform this action. +[^1]: On public and internal projects, all users are able to perform this action [^2]: Guest users can only view the confidential issues they created themselves [^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** [^4]: Not allowed for Guest, Reporter, Developer, Master, or Owner -[^5]: Only if user is not external one. -[^6]: Only if user is a member of the project. -[^7]: Only if the build was triggered by the user +[^5]: Only if the job was triggered by the user +[^6]: Only if user is not external one +[^7]: Only if user is a member of the project + [ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 [new-mod]: project/new_ci_build_permissions_model.md diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index e2924c66e70..130f7897b1a 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -2,9 +2,6 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in 10.1. -CAUTION: **Warning:** -The Cluster integration is currently in **Beta**. - With a cluster associated to your project, you can use Review Apps, deploy your applications, run your pipelines, and much more, in an easy way. @@ -25,11 +22,14 @@ prerequisites must be met: be enabled in GitLab at the instance level. If that's not the case, ask your administrator to enable it. - Your associated Google account must have the right privileges to manage - clusters on GKE. That would mean that a - [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) - must be set up. -- You must have Master [permissions] in order to be able to access the **Cluster** - page. + clusters on GKE. That would mean that a [billing + account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) + must be set up and that you have to have permissions to access it. +- You must have Master [permissions] in order to be able to access the + **Cluster** page. +- You must have [Cloud Billing API](https://cloud.google.com/billing/) enabled +- You must have [Resource Manager + API](https://cloud.google.com/resource-manager/) If all of the above requirements are met, you can proceed to add a new GKE cluster. @@ -83,6 +83,7 @@ added directly to your configured cluster. Those applications are needed for | ----------- | :------------: | ----------- | | [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It will be automatically installed as a dependency when you try to install a different app. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | | [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | +| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications | ## Enabling or disabling the Cluster integration diff --git a/doc/user/project/integrations/irker.md b/doc/user/project/integrations/irker.md index c63ea1316fe..ecdd83ce8f0 100644 --- a/doc/user/project/integrations/irker.md +++ b/doc/user/project/integrations/irker.md @@ -47,4 +47,8 @@ Irker accepts channel names of the form `chan` and `#chan`, both for the case, `Aorimn` is treated as a nick and no more as a channel name. Irker can also join password-protected channels. Users need to append -`?key=thesecretpassword` to the chan name. +`?key=thesecretpassword` to the chan name. When using this feature remember to +**not** put the `#` sign in front of the channel name; failing to do so will +result on irker joining a channel literally named `#chan?key=password` henceforth +leaking the channel key through the `/whois` IRC command (depending on IRC server +configuration). This is due to a long standing irker bug. diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 7dc234a9759..f77569e4886 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -173,6 +173,7 @@ where `PROJECT-1` is the issue ID of the JIRA project. - Only commits and merges into the project's default branch (usually **master**) will close an issue in Jira. You can change your projects default branch under [project settings](img/jira_project_settings.png). +- The JIRA issue will not be transitioned if it has a resolution. ### JIRA issue closing example @@ -222,6 +223,10 @@ JIRA issue references and update comments will not work if the GitLab issue trac Make sure the `Transition ID` you set within the JIRA settings matches the one your project needs to close a ticket. +Make sure that the JIRA issue is not already marked as resolved, in other words that +the JIRA issue resolution field is not set. (It should not be struck through in +JIRA lists.) + [services-templates]: services_templates.md [jira-repo-old-docs]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/doc/project_services/jira.md [jira]: https://www.atlassian.com/software/jira diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index e9738b683f9..710cf78e84f 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -1,7 +1,10 @@ --- -last_updated: 2017-09-25 +last_updated: 2017-12-28 --- +CAUTION: **Warning:** +Kubernetes service integration has been deprecated in GitLab 10.3. If the service is active the cluster information still be editable, however we advised to disable and reconfigure the clusters using the new [Clusters](../clusters/index.md) page. If the service is inactive the fields will be uneditable. Read [GitLab 10.3 release post](https://about.gitlab.com/2017/12/22/gitlab-10-3-released/#kubernetes-integration-service) for more information. + # GitLab Kubernetes / OpenShift integration GitLab can be configured to interact with Kubernetes, or other systems using the diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index a0405161495..9496d6f2ce0 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -39,7 +39,7 @@ Click on the service links to see further configuration instructions and details | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | | [JIRA](jira.md) | JIRA issue tracker | | JetBrains TeamCity CI | A continuous integration and build server | -| [Kubernetes](kubernetes.md) | A containerized deployment service | +| [Kubernetes](kubernetes.md) _(Has been deprecated in GitLab 10.3)_ | A containerized deployment service | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | diff --git a/doc/user/project/integrations/redmine.md b/doc/user/project/integrations/redmine.md index f530b6cb649..cc3218fbfd1 100644 --- a/doc/user/project/integrations/redmine.md +++ b/doc/user/project/integrations/redmine.md @@ -10,12 +10,7 @@ in the table below. | `description` | A name for the issue tracker (to differentiate between instances, for example) | | `project_url` | The URL to the project in Redmine which is being linked to this GitLab project | | `issues_url` | The URL to the issue in Redmine project that is linked to this GitLab project. Note that the `issues_url` requires `:id` in the URL. This ID is used by GitLab as a placeholder to replace the issue number. | - | `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project | - - Once you have configured and enabled Redmine: - - the **Issues** link on the GitLab project pages takes you to the appropriate - Redmine issue index - - clicking **New issue** on the project dashboard creates a new Redmine issue + | `new_issue_url` | This is the URL to create a new issue in Redmine for the project linked to this GitLab project. **This is currently not being used and will be removed in a future release.** | As an example, below is a configuration for a project named gitlab-ci. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index eafdd28071d..82175c70e49 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -54,6 +54,12 @@ Below are described the supported events. Triggered when you push to the repository except when pushing tags. +> **Note:** When more than 20 commits are pushed at once, the `commits` web hook + attribute will only contain the first 20 for performance reasons. Loading + detailed commit data is expensive. Note that despite only 20 commits being + present in the `commits` attribute, the `total_commits_count` attribute will + contain the actual total. + **Request header**: ``` diff --git a/doc/user/project/merge_requests/fast_forward_merge.md b/doc/user/project/merge_requests/fast_forward_merge.md index 085170d9f03..3cd91a185e3 100644 --- a/doc/user/project/merge_requests/fast_forward_merge.md +++ b/doc/user/project/merge_requests/fast_forward_merge.md @@ -9,7 +9,7 @@ When the fast-forward merge ([`--ff-only`][ffonly]) setting is enabled, no merge commits will be created and all merges are fast-forwarded, which means that merging is only allowed if the branch could be fast-forwarded. -When a fast-forward merge is not possible, the user must rebase the branch manually. +When a fast-forward merge is not possible, the user is given the option to rebase. ## Use cases @@ -25,7 +25,7 @@ merge commits. In such cases, the fast-forward merge is the perfect candidate. Now, when you visit the merge request page, you will be able to accept it **only if a fast-forward merge is possible**. -![Fast forward merge request](img/ff_merge_mr.png) +![Fast forward merge request](img/ff_merge_rebase.png) If the target branch is ahead of the source branch, you need to rebase the source branch locally before you will be able to do a fast-forward merge. diff --git a/doc/user/project/merge_requests/img/ff_merge_mr.png b/doc/user/project/merge_requests/img/ff_merge_mr.png Binary files differdeleted file mode 100644 index 241cc990343..00000000000 --- a/doc/user/project/merge_requests/img/ff_merge_mr.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/ff_merge_rebase.png b/doc/user/project/merge_requests/img/ff_merge_rebase.png Binary files differnew file mode 100644 index 00000000000..f6139f189ce --- /dev/null +++ b/doc/user/project/merge_requests/img/ff_merge_rebase.png diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index e81e935e37d..442fc978284 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -39,3 +39,5 @@ do. | `/board_move ~column` | Move issue to column on the board | | `/duplicate #issue` | Closes this issue and marks it as a duplicate of another issue | | `/move path/to/project` | Moves issue to another project | +| `/tableflip` | Append the comment with `(╯°□°)╯︵ ┻━┻` | +| `/shrug` | Append the comment with `¯\_(ツ)_/¯` |
\ No newline at end of file diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 1b8a84c9599..b8f865679a2 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -30,7 +30,8 @@ with all their related data and be moved into a new GitLab instance. | GitLab version | Import/Export version | | ---------------- | --------------------- | -| 10.3 to current | 0.2.1 | +| 10.4 to current | 0.2.2 | +| 10.3 | 0.2.1 | | 10.0 | 0.2.0 | | 9.4.0 | 0.1.8 | | 9.2.0 | 0.1.7 | |