diff options
Diffstat (limited to 'doc')
445 files changed, 15246 insertions, 5160 deletions
diff --git a/doc/README.md b/doc/README.md index cf7a828d91e..c30bf328003 100644 --- a/doc/README.md +++ b/doc/README.md @@ -2,37 +2,41 @@ ## User documentation +- [Account Security](user/account/security.md) Securing your account via two-factor authentication, etc. - [API](api/README.md) Automate GitLab via a simple and powerful API. - [CI/CD](ci/README.md) GitLab Continuous Integration (CI) and Continuous Delivery (CD) getting started, `.gitlab-ci.yml` options, and examples. - [GitLab as OAuth2 authentication service provider](integration/oauth_provider.md). It allows you to login to other applications from GitLab. -- [Container Registry](container_registry/README.md) Learn how to use GitLab Container Registry. -- [GitLab Basics](gitlab-basics/README.md) Find step by step how to start working on your commandline and on GitLab. +- [Container Registry](user/project/container_registry.md) Learn how to use GitLab Container Registry. +- [GitLab basics](gitlab-basics/README.md) Find step by step how to start working on your commandline and on GitLab. - [Importing to GitLab](workflow/importing/README.md). - [Importing and exporting projects between instances](user/project/settings/import_export.md). -- [Markdown](markdown/markdown.md) GitLab's advanced formatting system. +- [Markdown](user/markdown.md) GitLab's advanced formatting system. - [Migrating from SVN](workflow/importing/migrating_from_svn.md) Convert a SVN repository to Git and GitLab. -- [Permissions](permissions/permissions.md) Learn what each role in a project (external/guest/reporter/developer/master/owner) can do. +- [Permissions](user/permissions.md) Learn what each role in a project (external/guest/reporter/developer/master/owner) can do. - [Profile Settings](profile/README.md) - [Project Services](project_services/project_services.md) Integrate a project with external services, such as CI and chat. - [Public access](public_access/public_access.md) Learn how you can allow public and internal access to projects. - [SSH](ssh/README.md) Setup your ssh keys and deploy keys for secure access to your projects. - [Webhooks](web_hooks/web_hooks.md) Let GitLab notify you when new code has been pushed to your project. - [Workflow](workflow/README.md) Using GitLab functionality and importing projects from GitHub and SVN. +- [University](university/README.md) Learn Git and GitLab through videos and courses. +- [Git Attributes](user/project/git_attributes.md) Managing Git attributes using a `.gitattributes` file. ## Administrator documentation -- [Access restrictions](administration/access_restrictions.md) Define which Git access protocols can be used to talk to GitLab +- [Access restrictions](user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols) Define which Git access protocols can be used to talk to GitLab - [Authentication/Authorization](administration/auth/README.md) Configure external authentication with LDAP, SAML, CAS and additional Omniauth providers. - [Custom Git hooks](administration/custom_hooks.md) Custom Git hooks (on the filesystem) for when webhooks aren't enough. - [Install](install/README.md) Requirements, directory structures and installation from source. - [Restart GitLab](administration/restart_gitlab.md) Learn how to restart GitLab and its components. - [Integration](integration/README.md) How to integrate with systems such as JIRA, Redmine, Twitter. -- [Issue closing](customization/issue_closing.md) Customize how to close an issue from commit messages. -- [Libravatar](customization/libravatar.md) Use Libravatar for user avatars. +- [Issue closing pattern](administration/issue_closing_pattern.md) Customize how to close an issue from commit messages. +- [Koding](administration/integration/koding.md) Set up Koding to use with GitLab. +- [Libravatar](customization/libravatar.md) Use Libravatar instead of Gravatar for user avatars. - [Log system](administration/logs.md) Log system. - [Environment Variables](administration/environment_variables.md) to configure GitLab. -- [Operations](operations/README.md) Keeping GitLab up and running. +- [Operations](administration/operations.md) Keeping GitLab up and running. - [Raketasks](raketasks/README.md) Backups, maintenance, automatic webhook setup and the importing of projects. - [Repository checks](administration/repository_checks.md) Periodic Git repository checks. - [Repository storages](administration/repository_storages.md) Manage the paths used to store repositories. @@ -40,20 +44,20 @@ - [System hooks](system_hooks/system_hooks.md) Notifications when users, projects and keys are changed. - [Update](update/README.md) Update guides to upgrade your installation. - [Welcome message](customization/welcome_message.md) Add a custom welcome message to the sign-in page. -- [Reply by email](incoming_email/README.md) Allow users to comment on issues and merge requests by replying to notification emails. +- [Reply by email](administration/reply_by_email.md) Allow users to comment on issues and merge requests by replying to notification emails. - [Migrate GitLab CI to CE/EE](migrate_ci_to_ce/README.md) Follow this guide to migrate your existing GitLab CI data to GitLab CE/EE. - [Git LFS configuration](workflow/lfs/lfs_administration.md) - [Housekeeping](administration/housekeeping.md) Keep your Git repository tidy and fast. -- [GitLab Performance Monitoring](monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics. -- [Monitoring uptime](monitoring/health_check.md) Check the server status using the health check endpoint. +- [GitLab Performance Monitoring](administration/monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics. +- [Request Profiling](administration/monitoring/performance/request_profiling.md) Get a detailed profile on slow requests. +- [Monitoring uptime](user/admin_area/monitoring/health_check.md) Check the server status using the health check endpoint. - [Debugging Tips](administration/troubleshooting/debug.md) Tips to debug problems when things go wrong - [Sidekiq Troubleshooting](administration/troubleshooting/sidekiq.md) Debug when Sidekiq appears hung and is not processing jobs. - [High Availability](administration/high_availability/README.md) Configure multiple servers for scaling or high availability. - [Container Registry](administration/container_registry.md) Configure Docker Registry with GitLab. +- [Multiple mountpoints for the repositories storage](administration/repository_storages.md) Define multiple repository storage paths to distribute the storage load. ## Contributor documentation -- [Documentation styleguide](development/doc_styleguide.md) Use this styleguide if you are - contributing to documentation. -- [Development](development/README.md) Explains the architecture and the guidelines for shell commands. +- [Development](development/README.md) All styleguides and explanations how to contribute. - [Legal](legal/README.md) Contributor license agreements. diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index 7186f707ad6..bf7814875bf 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -275,3 +275,9 @@ If you are getting 'Connection Refused' errors when trying to connect to the LDAP server please double-check the LDAP `port` and `method` settings used by GitLab. Common combinations are `method: 'plain'` and `port: 389`, OR `method: 'ssl'` and `port: 636`. + +### Login with valid credentials rejected + +If there is an unexpected error while authenticating the user with the LDAP +backend, the login is rejected and details about the error are logged to +`production.log`. diff --git a/doc/administration/build_artifacts.md b/doc/administration/build_artifacts.md new file mode 100644 index 00000000000..64353f7282b --- /dev/null +++ b/doc/administration/build_artifacts.md @@ -0,0 +1,90 @@ +# Build artifacts administration + +>**Notes:** +>- Introduced in GitLab 8.2 and GitLab Runner 0.7.0. +>- Starting from GitLab 8.4 and GitLab Runner 1.0, the artifacts archive format + changed to `ZIP`. +>- This is the administration documentation. For the user guide see + [user/project/builds/artifacts.md](../user/project/builds/artifacts.md). + +Artifacts is a list of files and directories which are attached to a build +after it completes successfully. This feature is enabled by default in all +GitLab installations. Keep reading if you want to know how to disable it. + +## Disabling build artifacts + +To disable artifacts site-wide, follow the steps below. + +--- + +**In Omnibus installations:** + +1. Edit `/etc/gitlab/gitlab.rb` and add the following line: + + ```ruby + gitlab_rails['artifacts_enabled'] = false + ``` + +1. Save the file and [reconfigure GitLab][] for the changes to take effect. + +--- + +**In installations from source:** + +1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: + + ```yaml + artifacts: + enabled: false + ``` + +1. Save the file and [restart GitLab][] for the changes to take effect. + +## Storing build artifacts + +After a successful build, GitLab Runner uploads an archive containing the build +artifacts to GitLab. + +To change the location where the artifacts are stored, follow the steps below. + +--- + +**In Omnibus installations:** + +_The artifacts are stored by default in +`/var/opt/gitlab/gitlab-rails/shared/artifacts`._ + +1. To change the storage path for example to `/mnt/storage/artifacts`, edit + `/etc/gitlab/gitlab.rb` and add the following line: + + ```ruby + gitlab_rails['artifacts_path'] = "/mnt/storage/artifacts" + ``` + +1. Save the file and [reconfigure GitLab][] for the changes to take effect. + +--- + +**In installations from source:** + +_The artifacts are stored by default in +`/home/git/gitlab/shared/artifacts`._ + +1. To change the storage path for example to `/mnt/storage/artifacts`, edit + `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: + + ```yaml + artifacts: + enabled: true + path: /mnt/storage/artifacts + ``` + +1. Save the file and [restart GitLab][] for the changes to take effect. + +## Set the maximum file size of the artifacts + +Provided the artifacts are enabled, you can change the maximum file size of the +artifacts through the [Admin area settings](../user/admin_area/settings/continuous_integration#maximum-artifacts-size). + +[reconfigure gitlab]: restart_gitlab.md "How to restart GitLab" +[restart gitlab]: restart_gitlab.md "How to restart GitLab" diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md index d5d43303454..d7cfb464f74 100644 --- a/doc/administration/container_registry.md +++ b/doc/administration/container_registry.md @@ -1,43 +1,32 @@ -# GitLab Container Registry Administration +# GitLab Container Registry administration -> **Note:** -This feature was [introduced][ce-4040] in GitLab 8.8. - -With the Docker Container Registry integrated into GitLab, every project can -have its own space to store its Docker images. - -You can read more about Docker Registry at https://docs.docker.com/registry/introduction/. +> [Introduced][ce-4040] in GitLab 8.8. --- -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* +> **Notes:** +- Container Registry manifest `v1` support was added in GitLab 8.9 to support + Docker versions earlier than 1.10. +- This document is about the admin guide. To learn how to use GitLab Container + Registry [user documentation](../user/project/container_registry.md). -- [Enable the Container Registry](#enable-the-container-registry) -- [Container Registry domain configuration](#container-registry-domain-configuration) - - [Configure Container Registry under an existing GitLab domain](#configure-container-registry-under-an-existing-gitlab-domain) - - [Configure Container Registry under its own domain](#configure-container-registry-under-its-own-domain) -- [Disable Container Registry site-wide](#disable-container-registry-site-wide) -- [Disable Container Registry per project](#disable-container-registry-per-project) -- [Disable Container Registry for new projects site-wide](#disable-container-registry-for-new-projects-site-wide) -- [Container Registry storage path](#container-registry-storage-path) -- [Container Registry storage driver](#container-registry-storage-driver) -- [Storage limitations](#storage-limitations) -- [Changelog](#changelog) +With the Container Registry integrated into GitLab, every project can have its +own space to store its Docker images. -<!-- END doctoc generated TOC please keep comment here to allow auto update --> +You can read more about the Container Registry at +https://docs.docker.com/registry/introduction/. ## Enable the Container Registry **Omnibus GitLab installations** All you have to do is configure the domain name under which the Container -Registry will listen to. Read [#container-registry-domain-configuration](#container-registry-domain-configuration) +Registry will listen to. Read +[#container-registry-domain-configuration](#container-registry-domain-configuration) and pick one of the two options that fits your case. >**Note:** -The container Registry works under HTTPS by default. Using HTTP is possible +The container registry works under HTTPS by default. Using HTTP is possible but not recommended and out of the scope of this document. Read the [insecure Registry documentation][docker-insecure] if you want to implement this. @@ -48,7 +37,7 @@ implement this. If you have installed GitLab from source: -1. You will have to [install Docker Registry][registry-deploy] by yourself. +1. You will have to [install Registry][registry-deploy] by yourself. 1. After the installation is complete, you will have to configure the Registry's settings in `gitlab.yml` in order to enable it. 1. Use the sample NGINX configuration file that is found under @@ -81,11 +70,13 @@ where: | `issuer` | This should be the same value as configured in Registry's `issuer`. Read the [token auth configuration documentation][token-config]. | >**Note:** -GitLab does not ship with a Registry init file. Hence, [restarting GitLab][restart gitlab] -will not restart the Registry should you modify its settings. Read the upstream -documentation on how to achieve that. +A Registry init file is not shipped with GitLab if you install it from source. +Hence, [restarting GitLab][restart gitlab] will not restart the Registry should +you modify its settings. Read the upstream documentation on how to achieve that. -The Docker Registry configuration will need `container_registry` as the service and `https://gitlab.example.com/jwt/auth` as the realm: +At the absolute minimum, make sure your [Registry configuration][registry-auth] +has `container_registry` as the service and `https://gitlab.example.com/jwt/auth` +as the realm: ``` auth: @@ -122,6 +113,10 @@ Registry is exposed to the outside world is `4567`, here is what you need to set in `gitlab.rb` or `gitlab.yml` if you are using Omnibus GitLab or installed GitLab from source respectively. +>**Note:** +Be careful to choose a port different than the one that Registry listens to (`5000` by default), +otherwise you will run into conflicts . + --- **Omnibus GitLab installations** @@ -272,12 +267,6 @@ Registry application itself. 1. Save the file and [restart GitLab][] for the changes to take effect. -## Disable Container Registry per project - -If Registry is enabled in your GitLab instance, but you don't need it for your -project, you can disable it from your project's settings. Read the user guide -on how to achieve that. - ## Disable Container Registry for new projects site-wide If the Container Registry is enabled, then it will be available on all new @@ -403,7 +392,8 @@ To configure the storage driver in Omnibus: 's3' => { 'accesskey' => 's3-access-key', 'secretkey' => 's3-secret-key-for-access-key', - 'bucket' => 'your-s3-bucket' + 'bucket' => 'your-s3-bucket', + 'region' => 'your-s3-region' } } ``` @@ -425,12 +415,53 @@ storage: accesskey: 'AKIAKIAKI' secretkey: 'secret123' bucket: 'gitlab-registry-bucket-AKIAKIAKI' + region: 'your-s3-region' cache: blobdescriptor: inmemory delete: enabled: true ``` +## Change the registry's internal port + +> **Note:** +This is not to be confused with the port that GitLab itself uses to expose +the Registry to the world. + +The Registry server listens on localhost at port `5000` by default, +which is the address for which the Registry server should accept connections. +In the examples below we set the Registry's port to `5001`. + +**Omnibus GitLab** + +1. Open `/etc/gitlab/gitlab.rb` and set `registry['registry_http_addr']`: + + ```ruby + registry['registry_http_addr'] = "localhost:5001" + ``` + +1. Save the file and [reconfigure GitLab][] for the changes to take effect. + +--- + +**Installations from source** + +1. Open the configuration file of your Registry server and edit the + [`http:addr`][registry-http-config] value: + + ``` + http + addr: localhost:5001 + ``` + +1. Save the file and restart the Registry server. + +## Disable Container Registry per project + +If Registry is enabled in your GitLab instance, but you don't need it for your +project, you can disable it from your project's settings. Read the user guide +on how to achieve that. + ## Storage limitations Currently, there is no storage limitation, which means a user can upload an @@ -450,6 +481,8 @@ configurable in future releases. [docker-insecure]: https://docs.docker.com/registry/insecure/ [registry-deploy]: https://docs.docker.com/registry/deploying/ [storage-config]: https://docs.docker.com/registry/configuration/#storage +[registry-http-config]: https://docs.docker.com/registry/configuration/#http +[registry-auth]: https://docs.docker.com/registry/configuration/#auth [token-config]: https://docs.docker.com/registry/configuration/#token [8-8-docs]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-8-stable/doc/administration/container_registry.md [registry-ssl]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/registry-ssl diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index e3306c22d3f..0387d730489 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -44,8 +44,7 @@ as appropriate. ## Custom error messages ->**Note:** -This feature was [introduced][5073] in GitLab 8.10. +> [Introduced][5073] in GitLab 8.10. If the commit is declined or an error occurs during the Git hook check, the STDERR or STDOUT message of the hook will be present in GitLab's UI. diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md index 7f53915a4d7..b4a953d1ccc 100644 --- a/doc/administration/environment_variables.md +++ b/doc/administration/environment_variables.md @@ -13,15 +13,17 @@ override certain values. Variable | Type | Description -------- | ---- | ----------- -`GITLAB_ROOT_PASSWORD` | string | Sets the password for the `root` user on installation -`GITLAB_HOST` | string | The full URL of the GitLab server (including `http://` or `https://`) -`RAILS_ENV` | string | The Rails environment; can be one of `production`, `development`, `staging` or `test` -`DATABASE_URL` | string | The database URL; is of the form: `postgresql://localhost/blog_development` -`GITLAB_EMAIL_FROM` | string | The e-mail address used in the "From" field in e-mails sent by GitLab -`GITLAB_EMAIL_DISPLAY_NAME` | string | The name used in the "From" field in e-mails sent by GitLab -`GITLAB_EMAIL_REPLY_TO` | string | The e-mail address used in the "Reply-To" field in e-mails sent by GitLab -`GITLAB_UNICORN_MEMORY_MIN` | integer | The minimum memory threshold (in bytes) for the Unicorn worker killer -`GITLAB_UNICORN_MEMORY_MAX` | integer | The maximum memory threshold (in bytes) for the Unicorn worker killer +`GITLAB_ROOT_PASSWORD` | string | Sets the password for the `root` user on installation +`GITLAB_HOST` | string | The full URL of the GitLab server (including `http://` or `https://`) +`RAILS_ENV` | string | The Rails environment; can be one of `production`, `development`, `staging` or `test` +`DATABASE_URL` | string | The database URL; is of the form: `postgresql://localhost/blog_development` +`GITLAB_EMAIL_FROM` | string | The e-mail address used in the "From" field in e-mails sent by GitLab +`GITLAB_EMAIL_DISPLAY_NAME` | string | The name used in the "From" field in e-mails sent by GitLab +`GITLAB_EMAIL_REPLY_TO` | string | The e-mail address used in the "Reply-To" field in e-mails sent by GitLab +`GITLAB_EMAIL_REPLY_TO` | string | The e-mail address used in the "Reply-To" field in e-mails sent by GitLab +`GITLAB_EMAIL_SUBJECT_SUFFIX` | string | The e-mail subject suffix used in e-mails sent by GitLab +`GITLAB_UNICORN_MEMORY_MIN` | integer | The minimum memory threshold (in bytes) for the Unicorn worker killer +`GITLAB_UNICORN_MEMORY_MAX` | integer | The maximum memory threshold (in bytes) for the Unicorn worker killer ## Complete database variables diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index 8a881ce8863..137fed35a73 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -101,9 +101,9 @@ need some additional configuration. ```ruby gitlab_shell['secret_token'] = 'fbfb19c355066a9afb030992231c4a363357f77345edd0f2e772359e5be59b02538e1fa6cae8f93f7d23355341cea2b93600dab6d6c3edcdced558fc6d739860' - gitlab_rails['secret_token'] = 'b719fe119132c7810908bba18315259ed12888d4f5ee5430c42a776d840a396799b0a5ef0a801348c8a357f07aa72bbd58e25a84b8f247a25c72f539c7a6c5fa' - gitlab_ci['secret_key_base'] = '6e657410d57c71b4fc3ed0d694e7842b1895a8b401d812c17fe61caf95b48a6d703cb53c112bc01ebd197a85da81b18e29682040e99b4f26594772a4a2c98c6d' - gitlab_ci['db_key_base'] = 'bf2e47b68d6cafaef1d767e628b619365becf27571e10f196f98dc85e7771042b9203199d39aff91fcb6837c8ed83f2a912b278da50999bb11a2fbc0fba52964' + gitlab_rails['otp_key_base'] = 'b719fe119132c7810908bba18315259ed12888d4f5ee5430c42a776d840a396799b0a5ef0a801348c8a357f07aa72bbd58e25a84b8f247a25c72f539c7a6c5fa' + gitlab_rails['secret_key_base'] = '6e657410d57c71b4fc3ed0d694e7842b1895a8b401d812c17fe61caf95b48a6d703cb53c112bc01ebd197a85da81b18e29682040e99b4f26594772a4a2c98c6d' + gitlab_rails['db_key_base'] = 'bf2e47b68d6cafaef1d767e628b619365becf27571e10f196f98dc85e7771042b9203199d39aff91fcb6837c8ed83f2a912b278da50999bb11a2fbc0fba52964' ``` 1. Run `touch /etc/gitlab/skip-auto-migrations` to prevent database migrations diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index f6153216f33..bc424330656 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -1,7 +1,12 @@ # Configuring Redis for GitLab HA -You can choose to install and manage Redis yourself, or you can use GitLab -Omnibus packages to help. +You can choose to install and manage Redis yourself, or you can use the one +that comes bundled with GitLab Omnibus packages. + +> **Note:** Redis does not require authentication by default. See + [Redis Security](http://redis.io/topics/security) documentation for more + information. We recommend using a combination of a Redis password and tight + firewall rules to secure your Redis service. ## Configure your own Redis server @@ -9,49 +14,293 @@ If you're hosting GitLab on a cloud provider, you can optionally use a managed service for Redis. For example, AWS offers a managed ElastiCache service that runs Redis. -> **Note:** Redis does not require authentication by default. See - [Redis Security](http://redis.io/topics/security) documentation for more - information. We recommend using a combination of a Redis password and tight - firewall rules to secure your Redis service. +## Configure Redis using Omnibus -## Configure using Omnibus +If you don't want to bother setting up your own Redis server, you can use the +one bundled with Omnibus. In this case, you should disable all services except +Redis. 1. Download/install GitLab Omnibus using **steps 1 and 2** from [GitLab downloads](https://about.gitlab.com/downloads). Do not complete other steps on the download page. 1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration. Be sure to change the `external_url` to match your eventual GitLab front-end - URL. + URL: ```ruby - external_url 'https://gitlab.example.com' + external_url 'https://gitlab.example.com' - # Disable all components except Redis - redis['enable'] = true - bootstrap['enable'] = false - nginx['enable'] = false - unicorn['enable'] = false - sidekiq['enable'] = false - postgresql['enable'] = false - gitlab_workhorse['enable'] = false - mailroom['enable'] = false + # Disable all services except Redis + redis['enable'] = true + bootstrap['enable'] = false + nginx['enable'] = false + unicorn['enable'] = false + sidekiq['enable'] = false + postgresql['enable'] = false + gitlab_workhorse['enable'] = false + mailroom['enable'] = false - # Redis configuration - redis['port'] = 6379 - redis['bind'] = '0.0.0.0' + # Redis configuration + redis['port'] = 6379 + redis['bind'] = '0.0.0.0' - # If you wish to use Redis authentication (recommended) - redis['password'] = 'Redis Password' + # If you wish to use Redis authentication (recommended) + redis['password'] = 'Redis Password' ``` 1. Run `sudo gitlab-ctl reconfigure` to install and configure PostgreSQL. > **Note**: This `reconfigure` step will result in some errors. That's OK - don't be alarmed. + 1. Run `touch /etc/gitlab/skip-auto-migrations` to prevent database migrations from running on upgrade. Only the primary GitLab application server should handle migrations. +## Experimental Redis Sentinel support + +> [Introduced][ce-1877] in GitLab 8.11. + +Since GitLab 8.11, you can configure a list of Redis Sentinel servers that +will monitor a group of Redis servers to provide you with a standard failover +support. + +There is currently one exception to the Sentinel support: `mail_room`, the +component that processes incoming emails. It doesn't support Sentinel yet, but +we hope to integrate a future release that does support it. + +To get a better understanding on how to correctly setup Sentinel, please read +the [Redis Sentinel documentation](http://redis.io/topics/sentinel) first, as +failing to configure it correctly can lead to data loss. + +The configuration consists of three parts: + +- Redis setup +- Sentinel setup +- GitLab setup + +Read carefully how to configure those components below. + +### Redis setup + +You must have at least 2 Redis servers: 1 Master, 1 or more Slaves. +They should be configured the same way and with similar server specs, as +in a failover situation, any Slave can be elected as the new Master by +the Sentinel servers. + +In a minimal setup, the only required change for the slaves in `redis.conf` +is the addition of a `slaveof` line pointing to the initial master. +You can increase the security by defining a `requirepass` configuration in +the master, and `masterauth` in slaves. + +--- + +**Configuring your own Redis server** + +1. Add to the slaves' `redis.conf`: + + ```conf + # IP and port of the master Redis server + slaveof 10.10.10.10 6379 + ``` + +1. Optionally, set up password authentication for increased security. + Add the following to master's `redis.conf`: + + ```conf + # Optional password authentication for increased security + requirepass "<password>" + ``` + +1. Then add this line to all the slave servers' `redis.conf`: + + ```conf + masterauth "<password>" + ``` + +1. Restart the Redis services for the changes to take effect. + +--- + +**Using Redis via Omnibus** + +1. Edit `/etc/gitlab/gitlab.rb` of a master Redis machine (usualy a single machine): + + ```ruby + ## Redis TCP support (will disable UNIX socket transport) + redis['bind'] = '0.0.0.0' # or specify an IP to bind to a single one + redis['port'] = 6379 + + ## Master redis instance + redis['password'] = '<huge password string here>' + ``` + +1. Edit `/etc/gitlab/gitlab.rb` of a slave Redis machine (should be one or more machines): + + ```ruby + ## Redis TCP support (will disable UNIX socket transport) + redis['bind'] = '0.0.0.0' # or specify an IP to bind to a single one + redis['port'] = 6379 + + ## Slave redis instance + redis['master_ip'] = '10.10.10.10' # IP of master Redis server + redis['master_port'] = 6379 # Port of master Redis server + redis['master_password'] = "<huge password string here>" + ``` + +1. Reconfigure the GitLab for the changes to take effect: `sudo gitlab-ctl reconfigure` + +--- + +Now that the Redis servers are all set up, let's configure the Sentinel +servers. + +### Sentinel setup + +We don't provide yet an automated way to setup and run the Sentinel daemon +from Omnibus installation method. You must follow the instructions below and +run it by yourself. + +The support for Sentinel in Ruby has some [caveats](https://github.com/redis/redis-rb/issues/531). +While you can give any name for the `master-group-name` part of the +configuration, as in this example: + +```conf +sentinel monitor <master-group-name> <ip> <port> <quorum> +``` + +,for it to work in Ruby, you have to use the "hostname" of the master Redis +server, otherwise you will get an error message like: +`Redis::CannotConnectError: No sentinels available.`. Read +[Sentinel troubleshooting](#sentinel-troubleshooting) for more information. + +Here is an example configuration file (`sentinel.conf`) for a Sentinel node: + +```conf +port 26379 +sentinel monitor master-redis.example.com 10.10.10.10 6379 1 +sentinel down-after-milliseconds master-redis.example.com 10000 +sentinel config-epoch master-redis.example.com 0 +sentinel leader-epoch master-redis.example.com 0 +``` + +--- + +The final part is to inform the main GitLab application server of the Redis +master and the new sentinels servers. + +### GitLab setup + +You can enable or disable sentinel support at any time in new or existing +installations. From the GitLab application perspective, all it requires is +the correct credentials for the master Redis and for a few Sentinel nodes. + +It doesn't require a list of all Sentinel nodes, as in case of a failure, +the application will need to query only one of them. + +>**Note:** +The following steps should be performed in the [GitLab application server](gitlab.md). + +**For source based installations** + +1. Edit `/home/git/gitlab/config/resque.yml` following the example in + `/home/git/gitlab/config/resque.yml.example`, and uncomment the sentinels + line, changing to the correct server credentials. +1. Restart GitLab for the changes to take effect. + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb` and add/change the following lines: + + ```ruby + gitlab-rails['redis_host'] = "master-redis.example.com" + gitlab-rails['redis_port'] = 6379 + gitlab-rails['redis_password'] = '<huge password string here>' + gitlab-rails['redis_sentinels'] = [ + {'host' => '10.10.10.1', 'port' => 26379}, + {'host' => '10.10.10.2', 'port' => 26379}, + {'host' => '10.10.10.3', 'port' => 26379} + ] + ``` + +1. [Reconfigure] the GitLab for the changes to take effect. + +### Sentinel troubleshooting + +If you get an error like: `Redis::CannotConnectError: No sentinels available.`, +there may be something wrong with your configuration files or it can be related +to [this issue][gh-531] ([pull request][gh-534] that should make things better). + +It's a bit rigid the way you have to config `resque.yml` and `sentinel.conf`, +otherwise `redis-rb` will not work properly. + +The hostname ('my-primary-redis') of the primary Redis server (`sentinel.conf`) +**must** match the one configured in GitLab (`resque.yml` for source installations +or `gitlab-rails['redis_*']` in Omnibus) and it must be valid ex: + +```conf +# sentinel.conf: +sentinel monitor my-primary-redis 10.10.10.10 6379 1 +sentinel down-after-milliseconds my-primary-redis 10000 +sentinel config-epoch my-primary-redis 0 +sentinel leader-epoch my-primary-redis 0 +``` + +```yaml +# resque.yaml +production: + url: redis://my-primary-redis:6378 + sentinels: + - + host: slave1 + port: 26380 # point to sentinel, not to redis port + - + host: slave2 + port: 26381 # point to sentinel, not to redis port +``` + +When in doubt, please read [Redis Sentinel documentation](http://redis.io/topics/sentinel) + +--- + +To make sure your configuration is correct: + +1. SSH into your GitLab application server +1. Enter the Rails console: + + ``` + # For Omnibus installations + sudo gitlab-rails console + + # For source installations + sudo -u git rails console RAILS_ENV=production + ``` + +1. Run in the console: + + ```ruby + redis = Redis.new(Gitlab::Redis.params) + redis.info + ``` + + Keep this screen open and try to simulate a failover below. + +1. To simulate a failover on master Redis, SSH into the Redis server and run: + + ```bash + # port must match your master redis port + redis-cli -h localhost -p 6379 DEBUG sleep 60 + ``` + +1. Then back in the Rails console from the first step, run: + + ``` + redis.info + ``` + + You should see a different port after a few seconds delay + (the failover/reconnect time). + --- Read more on high-availability configuration: @@ -60,3 +309,9 @@ Read more on high-availability configuration: 1. [Configure NFS](nfs.md) 1. [Configure the GitLab application servers](gitlab.md) 1. [Configure the load balancers](load_balancer.md) + +[ce-1877]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1877 +[restart]: ../restart_gitlab.md#installations-from-source +[reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure +[gh-531]: https://github.com/redis/redis-rb/issues/531 +[gh-534]: https://github.com/redis/redis-rb/issues/534 diff --git a/doc/administration/housekeeping.md b/doc/administration/housekeeping.md index a5fa7d358a2..ad1fa98b63b 100644 --- a/doc/administration/housekeeping.md +++ b/doc/administration/housekeeping.md @@ -1,6 +1,6 @@ # Housekeeping -_**Note:** This feature was [introduced][ce-2371] in GitLab 8.4_ +> [Introduced][ce-2371] in GitLab 8.4. --- @@ -12,7 +12,7 @@ revisions (to reduce disk space and increase performance) and removing unreachable objects which may have been created from prior invocations of `git add`. -You can find this option under your **[Project] > Settings**. +You can find this option under your **[Project] > Edit Project**. --- diff --git a/doc/administration/img/access_restrictions.png b/doc/administration/img/access_restrictions.png Binary files differdeleted file mode 100644 index 66fd9491e85..00000000000 --- a/doc/administration/img/access_restrictions.png +++ /dev/null diff --git a/doc/administration/img/housekeeping_settings.png b/doc/administration/img/housekeeping_settings.png Binary files differindex f72ad9a45d5..6ebc6205635 100644 --- a/doc/administration/img/housekeeping_settings.png +++ b/doc/administration/img/housekeeping_settings.png diff --git a/doc/administration/img/repository_storages_admin_ui.png b/doc/administration/img/repository_storages_admin_ui.png Binary files differnew file mode 100644 index 00000000000..599350bc098 --- /dev/null +++ b/doc/administration/img/repository_storages_admin_ui.png diff --git a/doc/administration/img/restricted_url.png b/doc/administration/img/restricted_url.png Binary files differdeleted file mode 100644 index 0a677433dcf..00000000000 --- a/doc/administration/img/restricted_url.png +++ /dev/null diff --git a/doc/administration/integration/koding.md b/doc/administration/integration/koding.md new file mode 100644 index 00000000000..a2c358af095 --- /dev/null +++ b/doc/administration/integration/koding.md @@ -0,0 +1,242 @@ +# Koding & GitLab + +> [Introduced][ce-5909] in GitLab 8.11. + +This document will guide you through installing and configuring Koding with +GitLab. + +First of all, to be able to use Koding and GitLab together you will need public +access to your server. This allows you to use single sign-on from GitLab to +Koding and using vms from cloud providers like AWS. Koding has a registry for +VMs, called Kontrol and it runs on the same server as Koding itself, VMs from +cloud providers register themselves to Kontrol via the agent that we put into +provisioned VMs. This agent is called Klient and it provides Koding to access +and manage the target machine. + +Kontrol and Klient are based on another technology called +[Kite](https://github.com/koding/kite), that we have written at Koding. Which is a +microservice framework that allows you to develop microservices easily. + +## Requirements + +### Hardware + +Minimum requirements are; + + - 2 cores CPU + - 3G RAM + - 10G Storage + +If you plan to use AWS to install Koding it is recommended that you use at +least a `c3.xlarge` instance. + +### Software + + - [Git](https://git-scm.com) + - [Docker](https://www.docker.com) + - [docker-compose](https://www.docker.com/products/docker-compose) + +Koding can run on most of the UNIX based operating systems, since it's shipped +as containerized with Docker support, it can work on any operating system that +supports Docker. + +Required services are: + +- **PostgreSQL** - Kontrol and Service DB provider +- **MongoDB** - Main DB provider the application +- **Redis** - In memory DB used by both application and services +- **RabbitMQ** - Message Queue for both application and services + +which are also provided as a Docker container by Koding. + + +## Getting Started with Development Versions + + +### Koding + +You can run `docker-compose` environment for developing koding by +executing commands in the following snippet. + +```bash +git clone https://github.com/koding/koding.git +cd koding +docker-compose up +``` + +This should start koding on `localhost:8090`. + +By default there is no team exists in Koding DB. You'll need to create a team +called `gitlab` which is the default team name for GitLab integration in the +configuration. To make things in order it's recommended to create the `gitlab` +team first thing after setting up Koding. + + +### GitLab + +To install GitLab to your environment for development purposes it's recommended +to use GitLab Development Kit which you can get it from +[here](https://gitlab.com/gitlab-org/gitlab-development-kit). + +After all those steps, gitlab should be running on `localhost:3000` + + +## Integration + +Integration includes following components; + + - Single Sign On with OAuth from GitLab to Koding + - System Hook integration for handling GitLab events on Koding + (`project_created`, `user_joined` etc.) + - Service endpoints for importing/executing stacks from GitLab to Koding + (`Run/Try on IDE (Koding)` buttons on GitLab Projects, Issues, MRs) + +As it's pointed out before, you will need public access to this machine that +you've installed Koding and GitLab on. Better to use a domain but a static IP +is also fine. + +For IP based installation you can use [xip.io](https://xip.io) service which is +free and provides DNS resolution to IP based requests like following; + + - 127.0.0.1.xip.io -> resolves to 127.0.0.1 + - foo.bar.baz.127.0.0.1.xip.io -> resolves to 127.0.0.1 + - and so on... + +As Koding needs subdomains for team names; `foo.127.0.0.1.xip.io` requests for +a running koding instance on `127.0.0.1` server will be handled as `foo` team +requests. + + +### GitLab Side + +You need to enable Koding integration from Settings under Admin Area. To do +that login with an Admin account and do followings; + + - open [http://127.0.0.1:3000/admin/application_settings](http://127.0.0.1:3000/admin/application_settings) + - scroll to bottom of the page until Koding section + - check `Enable Koding` checkbox + - provide GitLab team page for running Koding instance as `Koding URL`* + +* For `Koding URL` you need to provide the gitlab integration enabled team on +your Koding installation. Team called `gitlab` has integration on Koding out +of the box, so if you didn't change anything your team on Koding should be +`gitlab`. + +So, if your Koding is running on `http://1.2.3.4.xip.io:8090` your URL needs +to be `http://gitlab.1.2.3.4.xip.io:8090`. You need to provide the same host +with your Koding installation here. + + +#### Registering Koding for OAuth integration + +We need `Application ID` and `Secret` to enable login to Koding via GitLab +feature and to do that you need to register running Koding as a new application +to your running GitLab application. Follow +[these](http://docs.gitlab.com/ce/integration/oauth_provider.html) steps to +enable this integration. + +Redirect URI should be `http://gitlab.127.0.0.1:8090/-/oauth/gitlab/callback` +which again you need to _replace `127.0.0.1` with your instance public IP._ + +Take a copy of `Application ID` and `Secret` that is generated by the GitLab +application, we will need those on _Koding Part_ of this guide. + + +#### Registering system hooks to Koding (optional) + +Koding can take actions based on the events generated by GitLab application. +This feature is still in progress and only following events are processed by +Koding at the moment; + + - user_create + - user_destroy + +All system events are handled but not implemented on Koding side. + +To enable this feature you need to provide a `URL` and a `Secret Token` to your +GitLab application. Open your admin area on your GitLab app from +[http://127.0.0.1:3000/admin/hooks](http://127.0.0.1:3000/admin/hooks) +and provide `URL` as `http://gitlab.127.0.0.1:8090/-/api/gitlab` which is the +endpoint to handle GitLab events on Koding side. Provide a `Secret Token` and +keep a copy of it, we will need it on _Koding Part_ of this guide. + +_(replace `127.0.0.1` with your instance public IP)_ + + +### Koding Part + +If you followed the steps in GitLab part we should have followings to enable +Koding part integrations; + + - `Application ID` and `Secret` for OAuth integration + - `Secret Token` for system hook integration + - Public address of running GitLab instance + + +#### Start Koding with GitLab URL + +Now we need to configure Koding with all this information to get things ready. +If it's already running please stop koding first. + +##### From command-line + +Replace followings with the ones you got from GitLab part of this guide; + +```bash +cd koding +docker-compose run \ + --service-ports backend \ + /opt/koding/scripts/bootstrap-container build \ + --host=**YOUR_IP**.xip.io \ + --gitlabHost=**GITLAB_IP** \ + --gitlabPort=**GITLAB_PORT** \ + --gitlabToken=**SECRET_TOKEN** \ + --gitlabAppId=**APPLICATION_ID** \ + --gitlabAppSecret=**SECRET** +``` + +##### By updating configuration + +Alternatively you can update `gitlab` section on +`config/credentials.default.coffee` like following; + +``` +gitlab = + host: '**GITLAB_IP**' + port: '**GITLAB_PORT**' + applicationId: '**APPLICATION_ID**' + applicationSecret: '**SECRET**' + team: 'gitlab' + redirectUri: '' + systemHookToken: '**SECRET_TOKEN**' + hooksEnabled: yes +``` + +and start by only providing the `host`; + +```bash +cd koding +docker-compose run \ + --service-ports backend \ + /opt/koding/scripts/bootstrap-container build \ + --host=**YOUR_IP**.xip.io \ +``` + +#### Enable Single Sign On + +Once you restarted your Koding and logged in with your username and password +you need to activate oauth authentication for your user. To do that + + - Navigate to Dashboard on Koding from; + `http://gitlab.**YOUR_IP**.xip.io:8090/Home/my-account` + - Scroll down to Integrations section + - Click on toggle to turn On integration in GitLab integration section + +This will redirect you to your GitLab instance and will ask your permission ( +if you are not logged in to GitLab at this point you will be redirected after +login) once you accept you will be redirected to your Koding instance. + +From now on you can login by using `SIGN IN WITH GITLAB` button on your Login +screen in your Koding instance. + +[ce-5909]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5909 diff --git a/doc/administration/issue_closing_pattern.md b/doc/administration/issue_closing_pattern.md new file mode 100644 index 00000000000..28e1fd4e12e --- /dev/null +++ b/doc/administration/issue_closing_pattern.md @@ -0,0 +1,49 @@ +# Issue closing pattern + +>**Note:** +This is the administration documentation. +There is a separate [user documentation] on issue closing pattern. + +When a commit or merge request resolves one or more issues, it is possible to +automatically have these issues closed when the commit or merge request lands +in the project's default branch. + +## Change the issue closing pattern + +In order to change the pattern you need to have access to the server that GitLab +is installed on. + +The default pattern can be located in [gitlab.yml.example] under the +"Automatic issue closing" section. + +> **Tip:** +You are advised to use http://rubular.com to test the issue closing pattern. +Because Rubular doesn't understand `%{issue_ref}`, you can replace this by +`#\d+` when testing your patterns, which matches only local issue references like `#123`. + +**For Omnibus installations** + +1. Open `/etc/gitlab/gitlab.rb` with your editor. +1. Change the value of `gitlab_rails['issue_closing_pattern']` to a regular + expression of your liking: + + ```ruby + gitlab_rails['issue_closing_pattern'] = "((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" + ``` +1. [Reconfigure] GitLab for the changes to take effect. + +**For installations from source** + +1. Open `gitlab.yml` with your editor. +1. Change the value of `issue_closing_pattern`: + + ```yaml + issue_closing_pattern: "((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" + ``` + +1. [Restart] GitLab for the changes to take effect. + +[gitlab.yml.example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example +[reconfigure]: restart_gitlab.md#omnibus-gitlab-reconfigure +[restart]: restart_gitlab.md#installations-from-source +[user documentation]: ../user/project/issues/automatic_issue_closing.md diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md new file mode 100644 index 00000000000..771584268d9 --- /dev/null +++ b/doc/administration/monitoring/performance/gitlab_configuration.md @@ -0,0 +1,40 @@ +# GitLab Configuration + +GitLab Performance Monitoring is disabled by default. To enable it and change any of its +settings, navigate to the Admin area in **Settings > Metrics** +(`/admin/application_settings`). + +The minimum required settings you need to set are the InfluxDB host and port. +Make sure _Enable InfluxDB Metrics_ is checked and hit **Save** to save the +changes. + +--- + + + +--- + +Finally, a restart of all GitLab processes is required for the changes to take +effect: + +```bash +# For Omnibus installations +sudo gitlab-ctl restart + +# For installations from source +sudo service gitlab restart +``` + +## Pending Migrations + +When any migrations are pending, the metrics are disabled until the migrations +have been performed. + +--- + +Read more on: + +- [Introduction to GitLab Performance Monitoring](introduction.md) +- [InfluxDB Configuration](influxdb_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) +- [Grafana Install/Configuration](grafana_configuration.md) diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md new file mode 100644 index 00000000000..7947b0fedc4 --- /dev/null +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -0,0 +1,111 @@ +# Grafana Configuration + +[Grafana](http://grafana.org/) is a tool that allows you to visualize time +series metrics through graphs and dashboards. It supports several backend +data stores, including InfluxDB. GitLab writes performance data to InfluxDB +and Grafana will allow you to query InfluxDB to display useful graphs. + +For the easiest installation and configuration, install Grafana on the same +server as InfluxDB. For larger installations, you may want to split out these +services. + +## Installation + +Grafana supplies package repositories (Yum/Apt) for easy installation. +See [Grafana installation documentation](http://docs.grafana.org/installation/) +for detailed steps. + +> **Note**: Before starting Grafana for the first time, set the admin user +and password in `/etc/grafana/grafana.ini`. Otherwise, the default password +will be `admin`. + +## Configuration + +Login as the admin user. Expand the menu by clicking the Grafana logo in the +top left corner. Choose 'Data Sources' from the menu. Then, click 'Add new' +in the top bar. + + + +Fill in the configuration details for the InfluxDB data source. Save and +Test Connection to ensure the configuration is correct. + +- **Name**: InfluxDB +- **Default**: Checked +- **Type**: InfluxDB 0.9.x (Even if you're using InfluxDB 0.10.x) +- **Url**: https://localhost:8086 (Or the remote URL if you've installed InfluxDB +on a separate server) +- **Access**: proxy +- **Database**: gitlab +- **User**: admin (Or the username configured when setting up InfluxDB) +- **Password**: The password configured when you set up InfluxDB + + + +## Apply retention policies and create continuous queries + +If you intend to import the GitLab provided Grafana dashboards, you will need to +set up the right retention policies and continuous queries. The easiest way of +doing this is by using the [influxdb-management](https://gitlab.com/gitlab-org/influxdb-management) +repository. + +To use this repository you must first clone it: + +``` +git clone https://gitlab.com/gitlab-org/influxdb-management.git +cd influxdb-management +``` + +Next you must install the required dependencies: + +``` +gem install bundler +bundle install +``` + +Now you must configure the repository by first copying `.env.example` to `.env` +and then editing the `.env` file to contain the correct InfluxDB settings. Once +configured you can simply run `bundle exec rake` and the InfluxDB database will +be configured for you. + +For more information see the [influxdb-management README](https://gitlab.com/gitlab-org/influxdb-management/blob/master/README.md). + +## Import Dashboards + +You can now import a set of default dashboards that will give you a good +start on displaying useful information. GitLab has published a set of default +[Grafana dashboards][grafana-dashboards] to get you started. Clone the +repository or download a zip/tarball, then follow these steps to import each +JSON file. + +Open the dashboard dropdown menu and click 'Import' + + + +Click 'Choose file' and browse to the location where you downloaded or cloned +the dashboard repository. Pick one of the JSON files to import. + + + +Once the dashboard is imported, be sure to click save icon in the top bar. If +you do not save the dashboard after importing it will be removed when you +navigate away. + + + +Repeat this process for each dashboard you wish to import. + +Alternatively you can automatically import all the dashboards into your Grafana +instance. See the README of the [Grafana dashboards][grafana-dashboards] +repository for more information on this process. + +[grafana-dashboards]: https://gitlab.com/gitlab-org/grafana-dashboards + +--- + +Read more on: + +- [Introduction to GitLab Performance Monitoring](introduction.md) +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Installation/Configuration](influxdb_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) diff --git a/doc/administration/monitoring/performance/img/grafana_dashboard_dropdown.png b/doc/administration/monitoring/performance/img/grafana_dashboard_dropdown.png Binary files differnew file mode 100644 index 00000000000..7e34fad71ce --- /dev/null +++ b/doc/administration/monitoring/performance/img/grafana_dashboard_dropdown.png diff --git a/doc/administration/monitoring/performance/img/grafana_dashboard_import.png b/doc/administration/monitoring/performance/img/grafana_dashboard_import.png Binary files differnew file mode 100644 index 00000000000..f97624365c7 --- /dev/null +++ b/doc/administration/monitoring/performance/img/grafana_dashboard_import.png diff --git a/doc/administration/monitoring/performance/img/grafana_data_source_configuration.png b/doc/administration/monitoring/performance/img/grafana_data_source_configuration.png Binary files differnew file mode 100644 index 00000000000..7d50e4c88c2 --- /dev/null +++ b/doc/administration/monitoring/performance/img/grafana_data_source_configuration.png diff --git a/doc/administration/monitoring/performance/img/grafana_data_source_empty.png b/doc/administration/monitoring/performance/img/grafana_data_source_empty.png Binary files differnew file mode 100644 index 00000000000..aa39a53acae --- /dev/null +++ b/doc/administration/monitoring/performance/img/grafana_data_source_empty.png diff --git a/doc/administration/monitoring/performance/img/grafana_save_icon.png b/doc/administration/monitoring/performance/img/grafana_save_icon.png Binary files differnew file mode 100644 index 00000000000..c740e33cd1c --- /dev/null +++ b/doc/administration/monitoring/performance/img/grafana_save_icon.png diff --git a/doc/administration/monitoring/performance/img/metrics_gitlab_configuration_settings.png b/doc/administration/monitoring/performance/img/metrics_gitlab_configuration_settings.png Binary files differnew file mode 100644 index 00000000000..db396423e30 --- /dev/null +++ b/doc/administration/monitoring/performance/img/metrics_gitlab_configuration_settings.png diff --git a/doc/administration/monitoring/performance/img/request_profile_result.png b/doc/administration/monitoring/performance/img/request_profile_result.png Binary files differnew file mode 100644 index 00000000000..73e2fdcab67 --- /dev/null +++ b/doc/administration/monitoring/performance/img/request_profile_result.png diff --git a/doc/administration/monitoring/performance/img/request_profiling_token.png b/doc/administration/monitoring/performance/img/request_profiling_token.png Binary files differnew file mode 100644 index 00000000000..04d87567816 --- /dev/null +++ b/doc/administration/monitoring/performance/img/request_profiling_token.png diff --git a/doc/administration/monitoring/performance/influxdb_configuration.md b/doc/administration/monitoring/performance/influxdb_configuration.md new file mode 100644 index 00000000000..c30cd2950d8 --- /dev/null +++ b/doc/administration/monitoring/performance/influxdb_configuration.md @@ -0,0 +1,193 @@ +# InfluxDB Configuration + +The default settings provided by [InfluxDB] are not sufficient for a high traffic +GitLab environment. The settings discussed in this document are based on the +settings GitLab uses for GitLab.com, depending on your own needs you may need to +further adjust them. + +If you are intending to run InfluxDB on the same server as GitLab, make sure +you have plenty of RAM since InfluxDB can use quite a bit depending on traffic. + +Unless you are going with a budget setup, it's advised to run it separately. + +## Requirements + +- InfluxDB 0.9.5 or newer +- A fairly modern version of Linux +- At least 4GB of RAM +- At least 10GB of storage for InfluxDB data + +Note that the RAM and storage requirements can differ greatly depending on the +amount of data received/stored. To limit the amount of stored data users can +look into [InfluxDB Retention Policies][influxdb-retention]. + +## Installation + +Installing InfluxDB is out of the scope of this document. Please refer to the +[InfluxDB documentation]. + +## InfluxDB Server Settings + +Since InfluxDB has many settings that users may wish to customize themselves +(e.g. what port to run InfluxDB on), we'll only cover the essentials. + +The configuration file in question is usually located at +`/etc/influxdb/influxdb.conf`. Whenever you make a change in this file, +InfluxDB needs to be restarted. + +### Storage Engine + +InfluxDB comes with different storage engines and as of InfluxDB 0.9.5 a new +storage engine is available, called [TSM Tree]. All users **must** use the new +`tsm1` storage engine as this [will be the default engine][tsm1-commit] in +upcoming InfluxDB releases. + +Make sure you have the following in your configuration file: + +``` +[data] + dir = "/var/lib/influxdb/data" + engine = "tsm1" +``` + +### Admin Panel + +Production environments should have the InfluxDB admin panel **disabled**. This +feature can be disabled by adding the following to your InfluxDB configuration +file: + +``` +[admin] + enabled = false +``` + +### HTTP + +HTTP is required when using the [InfluxDB CLI] or other tools such as Grafana, +thus it should be enabled. When enabling make sure to _also_ enable +authentication: + +``` +[http] + enabled = true + auth-enabled = true +``` + +_**Note:** Before you enable authentication, you might want to [create an +admin user](#create-a-new-admin-user)._ + +### UDP + +GitLab writes data to InfluxDB via UDP and thus this must be enabled. Enabling +UDP can be done using the following settings: + +``` +[[udp]] + enabled = true + bind-address = ":8089" + database = "gitlab" + batch-size = 1000 + batch-pending = 5 + batch-timeout = "1s" + read-buffer = 209715200 +``` + +This does the following: + +1. Enable UDP and bind it to port 8089 for all addresses. +2. Store any data received in the "gitlab" database. +3. Define a batch of points to be 1000 points in size and allow a maximum of + 5 batches _or_ flush them automatically after 1 second. +4. Define a UDP read buffer size of 200 MB. + +One of the most important settings here is the UDP read buffer size as if this +value is set too low, packets will be dropped. You must also make sure the OS +buffer size is set to the same value, the default value is almost never enough. + +To set the OS buffer size to 200 MB, on Linux you can run the following command: + +```bash +sysctl -w net.core.rmem_max=209715200 +``` + +To make this permanent, add the following to `/etc/sysctl.conf` and restart the +server: + +```bash +net.core.rmem_max=209715200 +``` + +It is **very important** to make sure the buffer sizes are large enough to +handle all data sent to InfluxDB as otherwise you _will_ lose data. The above +buffer sizes are based on the traffic for GitLab.com. Depending on the amount of +traffic, users may be able to use a smaller buffer size, but we highly recommend +using _at least_ 100 MB. + +When enabling UDP, users should take care to not expose the port to the public, +as doing so will allow anybody to write data into your InfluxDB database (as +[InfluxDB's UDP protocol][udp] doesn't support authentication). We recommend either +whitelisting the allowed IP addresses/ranges, or setting up a VLAN and only +allowing traffic from members of said VLAN. + +## Create a new admin user + +If you want to [enable authentication](#http), you might want to [create an +admin user][influx-admin]: + +``` +influx -execute "CREATE USER jeff WITH PASSWORD '1234' WITH ALL PRIVILEGES" +``` + +## Create the `gitlab` database + +Once you get InfluxDB up and running, you need to create a database for GitLab. +Make sure you have changed the [storage engine](#storage-engine) to `tsm1` +before creating a database. + +_**Note:** If you [created an admin user](#create-a-new-admin-user) and enabled +[HTTP authentication](#http), remember to append the username (`-username <username>`) +and password (`-password <password>`) you set earlier to the commands below._ + +Run the following command to create a database named `gitlab`: + +```bash +influx -execute 'CREATE DATABASE gitlab' +``` + +The name **must** be `gitlab`, do not use any other name. + +Next, make sure that the database was successfully created: + +```bash +influx -execute 'SHOW DATABASES' +``` + +The output should be similar to: + +``` +name: databases +--------------- +name +_internal +gitlab +``` + +That's it! Now your GitLab instance should send data to InfluxDB. + +--- + +Read more on: + +- [Introduction to GitLab Performance Monitoring](introduction.md) +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) +- [Grafana Install/Configuration](grafana_configuration.md) + +[influxdb-retention]: https://docs.influxdata.com/influxdb/v0.9/query_language/database_management/#retention-policy-management +[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/ +[influxdb cli]: https://docs.influxdata.com/influxdb/v0.9/tools/shell/ +[udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ +[influxdb]: https://influxdata.com/time-series-platform/influxdb/ +[tsm tree]: https://influxdata.com/blog/new-storage-engine-time-structured-merge-tree/ +[tsm1-commit]: https://github.com/influxdata/influxdb/commit/15d723dc77651bac83e09e2b1c94be480966cb0d +[influx-admin]: https://docs.influxdata.com/influxdb/v0.9/administration/authentication_and_authorization/#create-a-new-admin-user diff --git a/doc/administration/monitoring/performance/influxdb_schema.md b/doc/administration/monitoring/performance/influxdb_schema.md new file mode 100644 index 00000000000..eff0e29f58d --- /dev/null +++ b/doc/administration/monitoring/performance/influxdb_schema.md @@ -0,0 +1,97 @@ +# InfluxDB Schema + +The following measurements are currently stored in InfluxDB: + +- `PROCESS_file_descriptors` +- `PROCESS_gc_statistics` +- `PROCESS_memory_usage` +- `PROCESS_method_calls` +- `PROCESS_object_counts` +- `PROCESS_transactions` +- `PROCESS_views` +- `events` + +Here, `PROCESS` is replaced with either `rails` or `sidekiq` depending on the +process type. In all series, any form of duration is stored in milliseconds. + +## PROCESS_file_descriptors + +This measurement contains the number of open file descriptors over time. The +value field `value` contains the number of descriptors. + +## PROCESS_gc_statistics + +This measurement contains Ruby garbage collection statistics such as the amount +of minor/major GC runs (relative to the last sampling interval), the time spent +in garbage collection cycles, and all fields/values returned by `GC.stat`. + +## PROCESS_memory_usage + +This measurement contains the process' memory usage (in bytes) over time. The +value field `value` contains the number of bytes. + +## PROCESS_method_calls + +This measurement contains the methods called during a transaction along with +their duration, and a name of the transaction action that invoked the method (if +available). The method call duration is stored in the value field `duration`, +while the method name is stored in the tag `method`. The tag `action` contains +the full name of the transaction action. Both the `method` and `action` fields +are in the following format: + +``` +ClassName#method_name +``` + +For example, a method called by the `show` method in the `UsersController` class +would have `action` set to `UsersController#show`. + +## PROCESS_object_counts + +This measurement is used to store retained Ruby objects (per class) and the +amount of retained objects. The number of objects is stored in the `count` value +field while the class name is stored in the `type` tag. + +## PROCESS_transactions + +This measurement is used to store basic transaction details such as the time it +took to complete a transaction, how much time was spent in SQL queries, etc. The +following value fields are available: + +| Value | Description | +| ----- | ----------- | +| `duration` | The total duration of the transaction | +| `allocated_memory` | The amount of bytes allocated while the transaction was running. This value is only reliable when using single-threaded application servers | +| `method_duration` | The total time spent in method calls | +| `sql_duration` | The total time spent in SQL queries | +| `view_duration` | The total time spent in views | + +## PROCESS_views + +This measurement is used to store view rendering timings for a transaction. The +following value fields are available: + +| Value | Description | +| ----- | ----------- | +| `duration` | The rendering time of the view | +| `view` | The path of the view, relative to the application's root directory | + +The `action` tag contains the action name of the transaction that rendered the +view. + +## events + +This measurement is used to store generic events such as the number of Git +pushes, Emails sent, etc. Each point in this measurement has a single value +field called `count`. The value of this field is simply set to `1`. Each point +also has at least one tag: `event`. This tag's value is set to the event name. +Depending on the event type additional tags may be available as well. + +--- + +Read more on: + +- [Introduction to GitLab Performance Monitoring](introduction.md) +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Configuration](influxdb_configuration.md) +- [Grafana Install/Configuration](grafana_configuration.md) diff --git a/doc/administration/monitoring/performance/introduction.md b/doc/administration/monitoring/performance/introduction.md new file mode 100644 index 00000000000..79904916b7e --- /dev/null +++ b/doc/administration/monitoring/performance/introduction.md @@ -0,0 +1,65 @@ +# GitLab Performance Monitoring + +GitLab comes with its own application performance measuring system as of GitLab +8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the +Community and Enterprise editions. + +Apart from this introduction, you are advised to read through the following +documents in order to understand and properly configure GitLab Performance Monitoring: + +- [GitLab Configuration](gitlab_configuration.md) +- [InfluxDB Install/Configuration](influxdb_configuration.md) +- [InfluxDB Schema](influxdb_schema.md) +- [Grafana Install/Configuration](grafana_configuration.md) + +## Introduction to GitLab Performance Monitoring + +GitLab Performance Monitoring makes it possible to measure a wide variety of statistics +including (but not limited to): + +- The time it took to complete a transaction (a web request or Sidekiq job). +- The time spent in running SQL queries and rendering HAML views. +- The time spent executing (instrumented) Ruby methods. +- Ruby object allocations, and retained objects in particular. +- System statistics such as the process' memory usage and open file descriptors. +- Ruby garbage collection statistics. + +Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored +data can be visualized using [Grafana][grafana] or any other application that +supports reading data from InfluxDB. Alternatively data can be queried using the +InfluxDB CLI. + +## Metric Types + +Two types of metrics are collected: + +1. Transaction specific metrics. +1. Sampled metrics, collected at a certain interval in a separate thread. + +### Transaction Metrics + +Transaction metrics are metrics that can be associated with a single +transaction. This includes statistics such as the transaction duration, timings +of any executed SQL queries, time spent rendering HAML views, etc. These metrics +are collected for every Rack request and Sidekiq job processed. + +### Sampled Metrics + +Sampled metrics are metrics that can't be associated with a single transaction. +Examples include garbage collection statistics and retained Ruby objects. These +metrics are collected at a regular interval. This interval is made up out of two +parts: + +1. A user defined interval. +1. A randomly generated offset added on top of the interval, the same offset + can't be used twice in a row. + +The actual interval can be anywhere between a half of the defined interval and a +half above the interval. For example, for a user defined interval of 15 seconds +the actual interval can be anywhere between 7.5 and 22.5. The interval is +re-generated for every sampling run instead of being generated once and re-used +for the duration of the process' lifetime. + +[influxdb]: https://influxdata.com/time-series-platform/influxdb/ +[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ +[grafana]: http://grafana.org/ diff --git a/doc/administration/monitoring/performance/request_profiling.md b/doc/administration/monitoring/performance/request_profiling.md new file mode 100644 index 00000000000..c358dfbead2 --- /dev/null +++ b/doc/administration/monitoring/performance/request_profiling.md @@ -0,0 +1,16 @@ +# Request Profiling + +## Procedure +1. Grab the profiling token from `Monitoring > Requests Profiles` admin page +(highlighted in a blue in the image below). + +1. Pass the header `X-Profile-Token: <token>` to the request you want to profile. You can use any of these tools + * [ModHeader](https://chrome.google.com/webstore/detail/modheader/idgpnmonknjnojddfkpgkljpfnnfcklj) Chrome extension + * [Modify Headers](https://addons.mozilla.org/en-US/firefox/addon/modify-headers/) Firefox extension + * `curl --header 'X-Profile-Token: <token>' https://gitlab.example.com/group/project` +1. Once request is finished (which will take a little longer than usual), you can +view the profiling output from `Monitoring > Requests Profiles` admin page. + + +## Cleaning up +Profiling output will be cleared out every day via a Sidekiq worker. diff --git a/doc/administration/operations.md b/doc/administration/operations.md new file mode 100644 index 00000000000..4b582d16b64 --- /dev/null +++ b/doc/administration/operations.md @@ -0,0 +1,6 @@ +# GitLab operations + +- [Sidekiq MemoryKiller](operations/sidekiq_memory_killer.md) +- [Cleaning up Redis sessions](operations/cleaning_up_redis_sessions.md) +- [Understanding Unicorn and unicorn-worker-killer](operations/unicorn.md) +- [Moving repositories to a new location](operations/moving_repositories.md) diff --git a/doc/administration/operations/cleaning_up_redis_sessions.md b/doc/administration/operations/cleaning_up_redis_sessions.md new file mode 100644 index 00000000000..93521e976d5 --- /dev/null +++ b/doc/administration/operations/cleaning_up_redis_sessions.md @@ -0,0 +1,52 @@ +# Cleaning up stale Redis sessions + +Since version 6.2, GitLab stores web user sessions as key-value pairs in Redis. +Prior to GitLab 7.3, user sessions did not automatically expire from Redis. If +you have been running a large GitLab server (thousands of users) since before +GitLab 7.3 we recommend cleaning up stale sessions to compact the Redis +database after you upgrade to GitLab 7.3. You can also perform a cleanup while +still running GitLab 7.2 or older, but in that case new stale sessions will +start building up again after you clean up. + +In GitLab versions prior to 7.3.0, the session keys in Redis are 16-byte +hexadecimal values such as '976aa289e2189b17d7ef525a6702ace9'. Starting with +GitLab 7.3.0, the keys are +prefixed with 'session:gitlab:', so they would look like +'session:gitlab:976aa289e2189b17d7ef525a6702ace9'. Below we describe how to +remove the keys in the old format. + +First we define a shell function with the proper Redis connection details. + +``` +rcli() { + # This example works for Omnibus installations of GitLab 7.3 or newer. For an + # installation from source you will have to change the socket path and the + # path to redis-cli. + sudo /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket "$@" +} + +# test the new shell function; the response should be PONG +rcli ping +``` + +Now we do a search to see if there are any session keys in the old format for +us to clean up. + +``` +# returns the number of old-format session keys in Redis +rcli keys '*' | grep '^[a-f0-9]\{32\}$' | wc -l +``` + +If the number is larger than zero, you can proceed to expire the keys from +Redis. If the number is zero there is nothing to clean up. + +``` +# Tell Redis to expire each matched key after 600 seconds. +rcli keys '*' | grep '^[a-f0-9]\{32\}$' | awk '{ print "expire", $0, 600 }' | rcli +# This will print '(integer) 1' for each key that gets expired. +``` + +Over the next 15 minutes (10 minutes expiry time plus 5 minutes Redis +background save interval) your Redis database will be compacted. If you are +still using GitLab 7.2, users who are not clicking around in GitLab during the +10 minute expiry window will be signed out of GitLab. diff --git a/doc/administration/operations/moving_repositories.md b/doc/administration/operations/moving_repositories.md new file mode 100644 index 00000000000..54adb99386a --- /dev/null +++ b/doc/administration/operations/moving_repositories.md @@ -0,0 +1,180 @@ +# Moving repositories managed by GitLab + +Sometimes you need to move all repositories managed by GitLab to +another filesystem or another server. In this document we will look +at some of the ways you can copy all your repositories from +`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`. + +We will look at three scenarios: the target directory is empty, the +target directory contains an outdated copy of the repositories, and +how to deal with thousands of repositories. + +**Each of the approaches we list can/will overwrite data in the +target directory `/mnt/gitlab/repositories`. Do not mix up the +source and the target.** + +## Target directory is empty: use a tar pipe + +If the target directory `/mnt/gitlab/repositories` is empty the +simplest thing to do is to use a tar pipe. This method has low +overhead and tar is almost always already installed on your system. +However, it is not possible to resume an interrupted tar pipe: if +that happens then all data must be copied again. + +``` +# As the git user +tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ + tar -C /mnt/gitlab/repositories -xf - +``` + +If you want to see progress, replace `-xf` with `-xvf`. + +### Tar pipe to another server + +You can also use a tar pipe to copy data to another server. If your +'git' user has SSH access to the newserver as 'git@newserver', you +can pipe the data through SSH. + +``` +# As the git user +tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ + ssh git@newserver tar -C /mnt/gitlab/repositories -xf - +``` + +If you want to compress the data before it goes over the network +(which will cost you CPU cycles) you can replace `ssh` with `ssh -C`. + +## The target directory contains an outdated copy of the repositories: use rsync + +If the target directory already contains a partial / outdated copy +of the repositories it may be wasteful to copy all the data again +with tar. In this scenario it is better to use rsync. This utility +is either already installed on your system or easily installable +via apt, yum etc. + +``` +# As the 'git' user +rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ + /mnt/gitlab/repositories +``` + +The `/.` in the command above is very important, without it you can +easily get the wrong directory structure in the target directory. +If you want to see progress, replace `-a` with `-av`. + +### Single rsync to another server + +If the 'git' user on your source system has SSH access to the target +server you can send the repositories over the network with rsync. + +``` +# As the 'git' user +rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ + git@newserver:/mnt/gitlab/repositories +``` + +## Thousands of Git repositories: use one rsync per repository + +Every time you start an rsync job it has to inspect all files in +the source directory, all files in the target directory, and then +decide what files to copy or not. If the source or target directory +has many contents this startup phase of rsync can become a burden +for your GitLab server. In cases like this you can make rsync's +life easier by dividing its work in smaller pieces, and sync one +repository at a time. + +In addition to rsync we will use [GNU +Parallel](http://www.gnu.org/software/parallel/). This utility is +not included in GitLab so you need to install it yourself with apt +or yum. Also note that the GitLab scripts we used below were added +in GitLab 8.1. + +** This process does not clean up repositories at the target location that no +longer exist at the source. ** If you start using your GitLab instance with +`/mnt/gitlab/repositories`, you need to run `gitlab-rake gitlab:cleanup:repos` +after switching to the new repository storage directory. + +### Parallel rsync for all repositories known to GitLab + +This will sync repositories with 10 rsync processes at a time. We keep +track of progress so that the transfer can be restarted if necessary. + +First we create a new directory, owned by 'git', to hold transfer +logs. We assume the directory is empty before we start the transfer +procedure, and that we are the only ones writing files in it. + +``` +# Omnibus +sudo mkdir /var/opt/gitlab/transfer-logs +sudo chown git:git /var/opt/gitlab/transfer-logs + +# Source +sudo -u git -H mkdir /home/git/transfer-logs +``` + +We seed the process with a list of the directories we want to copy. + +``` +# Omnibus +sudo -u git sh -c 'gitlab-rake gitlab:list_repos > /var/opt/gitlab/transfer-logs/all-repos-$(date +%s).txt' + +# Source +cd /home/git/gitlab +sudo -u git -H sh -c 'bundle exec rake gitlab:list_repos > /home/git/transfer-logs/all-repos-$(date +%s).txt' +``` + +Now we can start the transfer. The command below is idempotent, and +the number of jobs done by GNU Parallel should converge to zero. If it +does not some repositories listed in all-repos-1234.txt may have been +deleted/renamed before they could be copied. + +``` +# Omnibus +sudo -u git sh -c ' +cat /var/opt/gitlab/transfer-logs/* | sort | uniq -u |\ + /usr/bin/env JOBS=10 \ + /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \ + /var/opt/gitlab/transfer-logs/success-$(date +%s).log \ + /var/opt/gitlab/git-data/repositories \ + /mnt/gitlab/repositories +' + +# Source +cd /home/git/gitlab +sudo -u git -H sh -c ' +cat /home/git/transfer-logs/* | sort | uniq -u |\ + /usr/bin/env JOBS=10 \ + bin/parallel-rsync-repos \ + /home/git/transfer-logs/success-$(date +%s).log \ + /home/git/repositories \ + /mnt/gitlab/repositories +` +``` + +### Parallel rsync only for repositories with recent activity + +Suppose you have already done one sync that started after 2015-10-1 12:00 UTC. +Then you might only want to sync repositories that were changed via GitLab +_after_ that time. You can use the 'SINCE' variable to tell 'rake +gitlab:list_repos' to only print repositories with recent activity. + +``` +# Omnibus +sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ + sudo -u git \ + /usr/bin/env JOBS=10 \ + /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \ + success-$(date +%s).log \ + /var/opt/gitlab/git-data/repositories \ + /mnt/gitlab/repositories + +# Source +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ + sudo -u git -H \ + /usr/bin/env JOBS=10 \ + bin/parallel-rsync-repos \ + success-$(date +%s).log \ + /home/git/repositories \ + /mnt/gitlab/repositories +``` diff --git a/doc/administration/operations/sidekiq_memory_killer.md b/doc/administration/operations/sidekiq_memory_killer.md new file mode 100644 index 00000000000..b5e78348989 --- /dev/null +++ b/doc/administration/operations/sidekiq_memory_killer.md @@ -0,0 +1,40 @@ +# Sidekiq MemoryKiller + +The GitLab Rails application code suffers from memory leaks. For web requests +this problem is made manageable using +[unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer) which +restarts Unicorn worker processes in between requests when needed. The Sidekiq +MemoryKiller applies the same approach to the Sidekiq processes used by GitLab +to process background jobs. + +Unlike unicorn-worker-killer, which is enabled by default for all GitLab +installations since GitLab 6.4, the Sidekiq MemoryKiller is enabled by default +_only_ for Omnibus packages. The reason for this is that the MemoryKiller +relies on Runit to restart Sidekiq after a memory-induced shutdown and GitLab +installations from source do not all use Runit or an equivalent. + +With the default settings, the MemoryKiller will cause a Sidekiq restart no +more often than once every 15 minutes, with the restart causing about one +minute of delay for incoming background jobs. + +## Configuring the MemoryKiller + +The MemoryKiller is controlled using environment variables. + +- `SIDEKIQ_MEMORY_KILLER_MAX_RSS`: if this variable is set, and its value is + greater than 0, then after each Sidekiq job, the MemoryKiller will check the + RSS of the Sidekiq process that executed the job. If the RSS of the Sidekiq + process (expressed in kilobytes) exceeds SIDEKIQ_MEMORY_KILLER_MAX_RSS, a + delayed shutdown is triggered. The default value for Omnibus packages is set + [in the omnibus-gitlab + repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb). +- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults 900 seconds (15 minutes). When + a shutdown is triggered, the Sidekiq process will keep working normally for + another 15 minutes. +- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. When the grace + time has expired, the MemoryKiller tells Sidekiq to stop accepting new jobs. + Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells + Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must + restart Sidekiq. +- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to `SIGKILL`. The name of + the final signal sent to the Sidekiq process when we want it to shut down. diff --git a/doc/administration/operations/unicorn.md b/doc/administration/operations/unicorn.md new file mode 100644 index 00000000000..bad61151bda --- /dev/null +++ b/doc/administration/operations/unicorn.md @@ -0,0 +1,86 @@ +# Understanding Unicorn and unicorn-worker-killer + +## Unicorn + +GitLab uses [Unicorn](http://unicorn.bogomips.org/), a pre-forking Ruby web +server, to handle web requests (web browsers and Git HTTP clients). Unicorn is +a daemon written in Ruby and C that can load and run a Ruby on Rails +application; in our case the Rails application is GitLab Community Edition or +GitLab Enterprise Edition. + +Unicorn has a multi-process architecture to make better use of available CPU +cores (processes can run on different cores) and to have stronger fault +tolerance (most failures stay isolated in only one process and cannot take down +GitLab entirely). On startup, the Unicorn 'master' process loads a clean Ruby +environment with the GitLab application code, and then spawns 'workers' which +inherit this clean initial environment. The 'master' never handles any +requests, that is left to the workers. The operating system network stack +queues incoming requests and distributes them among the workers. + +In a perfect world, the master would spawn its pool of workers once, and then +the workers handle incoming web requests one after another until the end of +time. In reality, worker processes can crash or time out: if the master notices +that a worker takes too long to handle a request it will terminate the worker +process with SIGKILL ('kill -9'). No matter how the worker process ended, the +master process will replace it with a new 'clean' process again. Unicorn is +designed to be able to replace 'crashed' workers without dropping user +requests. + +This is what a Unicorn worker timeout looks like in `unicorn_stderr.log`. The +master process has PID 56227 below. + +``` +[2015-06-05T10:58:08.660325 #56227] ERROR -- : worker=10 PID:53009 timeout (61s > 60s), killing +[2015-06-05T10:58:08.699360 #56227] ERROR -- : reaped #<Process::Status: pid 53009 SIGKILL (signal 9)> worker=10 +[2015-06-05T10:58:08.708141 #62538] INFO -- : worker=10 spawned pid=62538 +[2015-06-05T10:58:08.708824 #62538] INFO -- : worker=10 ready +``` + +### Tunables + +The main tunables for Unicorn are the number of worker processes and the +request timeout after which the Unicorn master terminates a worker process. +See the [omnibus-gitlab Unicorn settings +documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md) +if you want to adjust these settings. + +## unicorn-worker-killer + +GitLab has memory leaks. These memory leaks manifest themselves in long-running +processes, such as Unicorn workers. (The Unicorn master process is not known to +leak memory, probably because it does not handle user requests.) + +To make these memory leaks manageable, GitLab comes with the +[unicorn-worker-killer gem](https://github.com/kzk/unicorn-worker-killer). This +gem [monkey-patches](https://en.wikipedia.org/wiki/Monkey_patch) the Unicorn +workers to do a memory self-check after every 16 requests. If the memory of the +Unicorn worker exceeds a pre-set limit then the worker process exits. The +Unicorn master then automatically replaces the worker process. + +This is a robust way to handle memory leaks: Unicorn is designed to handle +workers that 'crash' so no user requests will be dropped. The +unicorn-worker-killer gem is designed to only terminate a worker process _in +between requests_, so no user requests are affected. + +This is what a Unicorn worker memory restart looks like in unicorn_stderr.log. +You see that worker 4 (PID 125918) is inspecting itself and decides to exit. +The threshold memory value was 254802235 bytes, about 250MB. With GitLab this +threshold is a random value between 200 and 250 MB. The master process (PID +117565) then reaps the worker process and spawns a new 'worker 4' with PID +127549. + +``` +[2015-06-05T12:07:41.828374 #125918] WARN -- : #<Unicorn::HttpServer:0x00000002734770>: worker (pid: 125918) exceeds memory limit (256413696 bytes > 254802235 bytes) +[2015-06-05T12:07:41.828472 #125918] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 125918) alive: 23 sec (trial 1) +[2015-06-05T12:07:42.025916 #117565] INFO -- : reaped #<Process::Status: pid 125918 exit 0> worker=4 +[2015-06-05T12:07:42.034527 #127549] INFO -- : worker=4 spawned pid=127549 +[2015-06-05T12:07:42.035217 #127549] INFO -- : worker=4 ready +``` + +One other thing that stands out in the log snippet above, taken from +GitLab.com, is that 'worker 4' was serving requests for only 23 seconds. This +is a normal value for our current GitLab.com setup and traffic. + +The high frequency of Unicorn memory restarts on some GitLab sites can be a +source of confusion for administrators. Usually they are a [red +herring](https://en.wikipedia.org/wiki/Red_herring). diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index c212059b9d5..39b1883375e 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -1,13 +1,14 @@ # Project import/export >**Note:** - - This feature was [introduced][ce-3050] in GitLab 8.9 - - Importing will not be possible if the import instance version is lower - than that of the exporter. - - For existing installations, the project import option has to be enabled in - application settings (`/admin/application_settings`) under 'Import sources'. - - The exports are stored in a temporary [shared directory][tmp] and are deleted - every 24 hours by a specific worker. +> +> - [Introduced][ce-3050] in GitLab 8.9. +> - Importing will not be possible if the import instance version is lower +> than that of the exporter. +> - For existing installations, the project import option has to be enabled in +> application settings (`/admin/application_settings`) under 'Import sources'. +> - The exports are stored in a temporary [shared directory][tmp] and are deleted +> every 24 hours by a specific worker. The GitLab Import/Export version can be checked by using: diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md new file mode 100644 index 00000000000..5a9a1582877 --- /dev/null +++ b/doc/administration/reply_by_email.md @@ -0,0 +1,302 @@ +# Reply by email + +GitLab can be set up to allow users to comment on issues and merge requests by +replying to notification emails. + +## Requirement + +Reply by email requires an IMAP-enabled email account. GitLab allows you to use +three strategies for this feature: +- using email sub-addressing +- using a dedicated email address +- using a catch-all mailbox + +### Email sub-addressing + +**If your provider or server supports email sub-addressing, we recommend using it.** + +[Sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing) is +a feature where any email to `user+some_arbitrary_tag@example.com` will end up +in the mailbox for `user@example.com`, and is supported by providers such as +Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix +mail server which you can run on-premises. + +### Dedicated email address + +This solution is really simple to set up: you just have to create an email +address dedicated to receive your users' replies to GitLab notifications. + +### Catch-all mailbox + +A [catch-all mailbox](https://en.wikipedia.org/wiki/Catch-all) for a domain will +"catch all" the emails addressed to the domain that do not exist in the mail +server. + +## How it works? + +### 1. GitLab sends a notification email + +When GitLab sends a notification and Reply by email is enabled, the `Reply-To` +header is set to the address defined in your GitLab configuration, with the +`%{key}` placeholder (if present) replaced by a specific "reply key". In +addition, this "reply key" is also added to the `References` header. + +### 2. You reply to the notification email + +When you reply to the notification email, your email client will: + +- send the email to the `Reply-To` address it got from the notification email +- set the `In-Reply-To` header to the value of the `Message-ID` header from the + notification email +- set the `References` header to the value of the `Message-ID` plus the value of + the notification email's `References` header. + +### 3. GitLab receives your reply to the notification email + +When GitLab receives your reply, it will look for the "reply key" in the +following headers, in this order: + +1. the `To` header +1. the `References` header + +If it finds a reply key, it will be able to leave your reply as a comment on +the entity the notification was about (issue, merge request, commit...). + +For more details about the `Message-ID`, `In-Reply-To`, and `References headers`, +please consult [RFC 5322](https://tools.ietf.org/html/rfc5322#section-3.6.4). + +## Set it up + +If you want to use Gmail / Google Apps with Reply by email, make sure you have +[IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) +and [allowed less secure apps to access the account](https://support.google.com/accounts/answer/6010255). + +To set up a basic Postfix mail server with IMAP access on Ubuntu, follow +[these instructions](./postfix.md). + +### Omnibus package installations + +1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the + feature and fill in the details for your specific IMAP server and email account: + + ```ruby + # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com + gitlab_rails['incoming_email_enabled'] = true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). + gitlab_rails['incoming_email_address'] = "incoming+%{key}@gitlab.example.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + gitlab_rails['incoming_email_email'] = "incoming" + # Email account password + gitlab_rails['incoming_email_password'] = "[REDACTED]" + + # IMAP server host + gitlab_rails['incoming_email_host'] = "gitlab.example.com" + # IMAP server port + gitlab_rails['incoming_email_port'] = 143 + # Whether the IMAP server uses SSL + gitlab_rails['incoming_email_ssl'] = false + # Whether the IMAP server uses StartTLS + gitlab_rails['incoming_email_start_tls'] = false + + # The mailbox where incoming mail will end up. Usually "inbox". + gitlab_rails['incoming_email_mailbox_name'] = "inbox" + ``` + + ```ruby + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + gitlab_rails['incoming_email_enabled'] = true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). + gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com" + # Email account password + gitlab_rails['incoming_email_password'] = "[REDACTED]" + + # IMAP server host + gitlab_rails['incoming_email_host'] = "imap.gmail.com" + # IMAP server port + gitlab_rails['incoming_email_port'] = 993 + # Whether the IMAP server uses SSL + gitlab_rails['incoming_email_ssl'] = true + # Whether the IMAP server uses StartTLS + gitlab_rails['incoming_email_start_tls'] = false + + # The mailbox where incoming mail will end up. Usually "inbox". + gitlab_rails['incoming_email_mailbox_name'] = "inbox" + ``` + +1. Reconfigure GitLab and restart mailroom for the changes to take effect: + + ```sh + sudo gitlab-ctl reconfigure + sudo gitlab-ctl restart mailroom + ``` + +1. Verify that everything is configured correctly: + + ```sh + sudo gitlab-rake gitlab:incoming_email:check + ``` + +1. Reply by email should now be working. + +### Installations from source + +1. Go to the GitLab installation directory: + + ```sh + cd /home/git/gitlab + ``` + +1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature + and fill in the details for your specific IMAP server and email account: + + ```sh + sudo editor config/gitlab.yml + ``` + + ```yaml + # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com + incoming_email: + enabled: true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). + address: "incoming+%{key}@gitlab.example.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "incoming" + # Email account password + password: "[REDACTED]" + + # IMAP server host + host: "gitlab.example.com" + # IMAP server port + port: 143 + # Whether the IMAP server uses SSL + ssl: false + # Whether the IMAP server uses StartTLS + start_tls: false + + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" + ``` + + ```yaml + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + incoming_email: + enabled: true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). + address: "gitlab-incoming+%{key}@gmail.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "gitlab-incoming@gmail.com" + # Email account password + password: "[REDACTED]" + + # IMAP server host + host: "imap.gmail.com" + # IMAP server port + port: 993 + # Whether the IMAP server uses SSL + ssl: true + # Whether the IMAP server uses StartTLS + start_tls: false + + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" + ``` + +1. Enable `mail_room` in the init script at `/etc/default/gitlab`: + + ```sh + sudo mkdir -p /etc/default + echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab + ``` + +1. Restart GitLab: + + ```sh + sudo service gitlab restart + ``` + +1. Verify that everything is configured correctly: + + ```sh + sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production + ``` + +1. Reply by email should now be working. + +### Development + +1. Go to the GitLab installation directory. + +1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account: + + ```yaml + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + incoming_email: + enabled: true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). + address: "gitlab-incoming+%{key}@gmail.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "gitlab-incoming@gmail.com" + # Email account password + password: "[REDACTED]" + + # IMAP server host + host: "imap.gmail.com" + # IMAP server port + port: 993 + # Whether the IMAP server uses SSL + ssl: true + # Whether the IMAP server uses StartTLS + start_tls: false + + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" + ``` + + As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. + +1. Uncomment the `mail_room` line in your `Procfile`: + + ```yaml + mail_room: bundle exec mail_room -q -c config/mail_room.yml + ``` + +1. Restart GitLab: + + ```sh + bundle exec foreman start + ``` + +1. Verify that everything is configured correctly: + + ```sh + bundle exec rake gitlab:incoming_email:check RAILS_ENV=development + ``` + +1. Reply by email should now be working. diff --git a/doc/administration/reply_by_email_postfix_setup.md b/doc/administration/reply_by_email_postfix_setup.md new file mode 100644 index 00000000000..22f10489a6c --- /dev/null +++ b/doc/administration/reply_by_email_postfix_setup.md @@ -0,0 +1,324 @@ +# Set up Postfix for Reply by email + +This document will take you through the steps of setting up a basic Postfix mail +server with IMAP authentication on Ubuntu, to be used with [Reply by email]. + +The instructions make the assumption that you will be using the email address `incoming@gitlab.example.com`, that is, username `incoming` on host `gitlab.example.com`. Don't forget to change it to your actual host when executing the example code snippets. + +## Configure your server firewall + +1. Open up port 25 on your server so that people can send email into the server over SMTP. +2. If the mail server is different from the server running GitLab, open up port 143 on your server so that GitLab can read email from the server over IMAP. + +## Install packages + +1. Install the `postfix` package if it is not installed already: + + ```sh + sudo apt-get install postfix + ``` + + When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`. + +1. Install the `mailutils` package. + + ```sh + sudo apt-get install mailutils + ``` + +## Create user + +1. Create a user for incoming email. + + ```sh + sudo useradd -m -s /bin/bash incoming + ``` + +1. Set a password for this user. + + ```sh + sudo passwd incoming + ``` + + Be sure not to forget this, you'll need it later. + +## Test the out-of-the-box setup + +1. Connect to the local SMTP server: + + ```sh + telnet localhost 25 + ``` + + You should see a prompt like this: + + ```sh + Trying 127.0.0.1... + Connected to localhost. + Escape character is '^]'. + 220 gitlab.example.com ESMTP Postfix (Ubuntu) + ``` + + If you get a `Connection refused` error instead, verify that `postfix` is running: + + ```sh + sudo postfix status + ``` + + If it is not, start it: + + ```sh + sudo postfix start + ``` + +1. Send the new `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: + + ``` + ehlo localhost + mail from: root@localhost + rcpt to: incoming@localhost + data + Subject: Re: Some issue + + Sounds good! + . + quit + ``` + + _**Note:** The `.` is a literal period on its own line._ + + _**Note:** If you receive an error after entering `rcpt to: incoming@localhost` + then your Postfix `my_network` configuration is not correct. The error will + say 'Temporary lookup failure'. See + [Configure Postfix to receive email from the Internet](#configure-postfix-to-receive-email-from-the-internet)._ + +1. Check if the `incoming` user received the email: + + ```sh + su - incoming + mail + ``` + + You should see output like this: + + ``` + "/var/mail/incoming": 1 message 1 unread + >U 1 root@localhost 59/2842 Re: Some issue + ``` + + Quit the mail app: + + ```sh + q + ``` + +1. Log out of the `incoming` account and go back to being `root`: + + ```sh + logout + ``` + +## Configure Postfix to use Maildir-style mailboxes + +Courier, which we will install later to add IMAP authentication, requires mailboxes to have the Maildir format, rather than mbox. + +1. Configure Postfix to use Maildir-style mailboxes: + + ```sh + sudo postconf -e "home_mailbox = Maildir/" + ``` + +1. Restart Postfix: + + ```sh + sudo /etc/init.d/postfix restart + ``` + +1. Test the new setup: + + 1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_. + 1. Check if the `incoming` user received the email: + + ```sh + su - incoming + MAIL=/home/incoming/Maildir + mail + ``` + + You should see output like this: + + ``` + "/home/incoming/Maildir": 1 message 1 unread + >U 1 root@localhost 59/2842 Re: Some issue + ``` + + Quit the mail app: + + ```sh + q + ``` + + _**Note:** If `mail` returns an error `Maildir: Is a directory` then your + version of `mail` doesn't support Maildir style mailboxes. Install + `heirloom-mailx` by running `sudo apt-get install heirloom-mailx`. Then, + try the above steps again, substituting `heirloom-mailx` for the `mail` + command._ + +1. Log out of the `incoming` account and go back to being `root`: + + ```sh + logout + ``` + +## Install the Courier IMAP server + +1. Install the `courier-imap` package: + + ```sh + sudo apt-get install courier-imap + ``` + +## Configure Postfix to receive email from the internet + +1. Let Postfix know about the domains that it should consider local: + + ```sh + sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost" + ``` + +1. Let Postfix know about the IPs that it should consider part of the LAN: + + We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network. + + ```sh + sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24" + ``` + +1. Configure Postfix to receive mail on all interfaces, which includes the internet: + + ```sh + sudo postconf -e "inet_interfaces = all" + ``` + +1. Configure Postfix to use the `+` delimiter for sub-addressing: + + ```sh + sudo postconf -e "recipient_delimiter = +" + ``` + +1. Restart Postfix: + + ```sh + sudo service postfix restart + ``` + +## Test the final setup + +1. Test SMTP under the new setup: + + 1. Connect to the SMTP server: + + ```sh + telnet gitlab.example.com 25 + ``` + + You should see a prompt like this: + + ```sh + Trying 123.123.123.123... + Connected to gitlab.example.com. + Escape character is '^]'. + 220 gitlab.example.com ESMTP Postfix (Ubuntu) + ``` + + If you get a `Connection refused` error instead, make sure your firewall is setup to allow inbound traffic on port 25. + + 1. Send the `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: + + ``` + ehlo gitlab.example.com + mail from: root@gitlab.example.com + rcpt to: incoming@gitlab.example.com + data + Subject: Re: Some issue + + Sounds good! + . + quit + ``` + + (Note: The `.` is a literal period on its own line) + + 1. Check if the `incoming` user received the email: + + ```sh + su - incoming + MAIL=/home/incoming/Maildir + mail + ``` + + You should see output like this: + + ``` + "/home/incoming/Maildir": 1 message 1 unread + >U 1 root@gitlab.example.com 59/2842 Re: Some issue + ``` + + Quit the mail app: + + ```sh + q + ``` + + 1. Log out of the `incoming` account and go back to being `root`: + + ```sh + logout + ``` + +1. Test IMAP under the new setup: + + 1. Connect to the IMAP server: + + ```sh + telnet gitlab.example.com 143 + ``` + + You should see a prompt like this: + + ```sh + Trying 123.123.123.123... + Connected to mail.example.gitlab.com. + Escape character is '^]'. + - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information. + ``` + + 1. Sign in as the `incoming` user to test IMAP, by entering the following into the IMAP prompt: + + ``` + a login incoming PASSWORD + ``` + + Replace PASSWORD with the password you set on the `incoming` user earlier. + + You should see output like this: + + ``` + a OK LOGIN Ok. + ``` + + 1. Disconnect from the IMAP server: + + ```sh + a logout + ``` + +## Done! + +If all the tests were successful, Postfix is all set up and ready to receive email! Continue with the [Reply by email](./README.md) guide to configure GitLab. + +--- + +_This document was adapted from https://help.ubuntu.com/community/PostfixBasicSetupHowto, by contributors to the Ubuntu documentation wiki._ + +[reply by email]: reply_by_email.md diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md index 4172b604cec..bc2b1f20ed3 100644 --- a/doc/administration/repository_checks.md +++ b/doc/administration/repository_checks.md @@ -1,8 +1,7 @@ # Repository checks ->**Note:** -This feature was [introduced][ce-3232] in GitLab 8.7. It is OFF by -default because it still causes too many false alarms. +> [Introduced][ce-3232] in GitLab 8.7. It is OFF by default because it still +causes too many false alarms. Git has a built-in mechanism, [git fsck][git-fsck], to verify the integrity of all data committed to a repository. GitLab administrators diff --git a/doc/administration/repository_storages.md b/doc/administration/repository_storages.md index 81bfe173151..55b054fc1a4 100644 --- a/doc/administration/repository_storages.md +++ b/doc/administration/repository_storages.md @@ -1,18 +1,99 @@ # Repository storages -GitLab allows you to define repository storage paths to enable distribution of -storage load between several mount points. - -## For installations from source +> [Introduced][ce-4578] in GitLab 8.10. -Add your repository storage paths in your `gitlab.yml` under repositories -> storages, using key -> value pairs. +GitLab allows you to define multiple repository storage paths to distribute the +storage load between several mount points. >**Notes:** +> - You must have at least one storage path called `default`. -- In order for backups to work correctly the storage path must **not** be a +- The paths are defined in key-value pairs. The key is an arbitrary name you + can pick to name the file path. +- The target directories and any of its subpaths must not be a symlink. + +## Configure GitLab + +>**Warning:** +In order for [backups] to work correctly, the storage path must **not** be a mount point and the GitLab user should have correct permissions for the parent -directory of the path. +directory of the path. In Omnibus GitLab this is taken care of automatically, +but for source installations you should be extra careful. +> +The thing is that for compatibility reasons `gitlab.yml` has a different +structure than Omnibus. In `gitlab.yml` you indicate the path for the +repositories, for example `/home/git/repositories`, while in Omnibus you +indicate `git_data_dirs`, which for the example above would be `/home/git`. +Then, Omnibus will create a `repositories` directory under that path to use with +`gitlab.yml`. +> +This little detail matters because while restoring a backup, the current +contents of `/home/git/repositories` [are moved to][raketask] `/home/git/repositories.old`, +so if `/home/git/repositories` is the mount point, then `mv` would be moving +things between mount points, and bad things could happen. Ideally, +`/home/git` would be the mount point, so then things would be moving within the +same mount point. This is guaranteed with Omnibus installations (because they +don't specify the full repository path but the parent path), but not for source +installations. + +--- + +Now that you've read that big fat warning above, let's edit the configuration +files and add the full paths of the alternative repository storage paths. In +the example below, we add two more mountpoints that are named `nfs` and `cephfs` +respectively. + +**For installations from source** + +1. Edit `gitlab.yml` and add the storage paths: + + ```yaml + repositories: + # Paths where repositories can be stored. Give the canonicalized absolute pathname. + # NOTE: REPOS PATHS MUST NOT CONTAIN ANY SYMLINK!!! + storages: # You must have at least a 'default' storage path. + default: /home/git/repositories + nfs: /mnt/nfs/repositories + cephfs: /mnt/cephfs/repositories + ``` + +1. [Restart GitLab] for the changes to take effect. + +>**Note:** +The [`gitlab_shell: repos_path` entry][repospath] in `gitlab.yml` will be +deprecated and replaced by `repositories: storages` in the future, so if you +are upgrading from a version prior to 8.10, make sure to add the configuration +as described in the step above. After you make the changes and confirm they are +working, you can remove the `repos_path` line. + +--- + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb` by appending the rest of the paths to the + default one: + + ```ruby + git_data_dirs({ + "default" => "/var/opt/gitlab/git-data", + "nfs" => "/mnt/nfs/git-data", + "cephfs" => "/mnt/cephfs/git-data" + }) + ``` + + Note that Omnibus stores the repositories in a `repositories` subdirectory + of the `git-data` directory. + +## Choose where new project repositories will be stored + +Once you set the multiple storage paths, you can choose where new projects will +be stored via the **Application Settings** in the Admin area. -## For omnibus installations + -Follow the instructions at https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/configuration.md#storing-git-data-in-an-alternative-directory +[ce-4578]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4578 +[restart gitlab]: restart_gitlab.md#installations-from-source +[reconfigure gitlab]: restart_gitlab.md#omnibus-gitlab-reconfigure +[backups]: ../raketasks/backup_restore.md +[raketask]: https://gitlab.com/gitlab-org/gitlab-ce/blob/033e5423a2594e08a7ebcd2379bd2331f4c39032/lib/backup/repository.rb#L54-56 +[repospath]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-9-stable/config/gitlab.yml.example#L457 diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md index 483060395dd..b561c2f82aa 100644 --- a/doc/administration/restart_gitlab.md +++ b/doc/administration/restart_gitlab.md @@ -139,7 +139,7 @@ If you are using other init systems, like systemd, you can check the [omnibus-dl]: https://about.gitlab.com/downloads/ "Download the Omnibus packages" [install]: ../install/installation.md "Documentation to install GitLab from source" -[mailroom]: ../incoming_email/README.md "Used for replying by email in GitLab issues and merge requests" +[mailroom]: reply_by_email.md "Used for replying by email in GitLab issues and merge requests" [chef]: https://www.chef.io/chef/ "Chef official website" [src-service]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/init.d/gitlab "GitLab init service file" [gl-recipes]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/init "GitLab Recipes repository" diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index d127d7b85e5..d8dce4388e1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -144,14 +144,14 @@ separate Rails process to debug the issue: 1. Obtain the private token for your user (Profile Settings -> Account). 1. Bring up the GitLab Rails console. For omnibus users, run: - ```` + ``` sudo gitlab-rails console ``` 1. At the Rails console, run: ```ruby - [1] pry(main)> app.get '<URL FROM STEP 1>/private_token?<TOKEN FROM STEP 2>' + [1] pry(main)> app.get '<URL FROM STEP 2>/?private_token=<TOKEN FROM STEP 3>' ``` For example: diff --git a/doc/api/README.md b/doc/api/README.md index d1e6c54c521..3fbe5197a21 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -10,21 +10,31 @@ following locations: - [Award Emoji](award_emoji.md) - [Branches](branches.md) +- [Broadcast Messages](broadcast_messages.md) - [Builds](builds.md) -- [Build triggers](build_triggers.md) +- [Build Triggers](build_triggers.md) - [Build Variables](build_variables.md) - [Commits](commits.md) +- [Deployments](deployments.md) - [Deploy Keys](deploy_keys.md) +- [Gitignores templates](templates/gitignores.md) +- [GitLab CI Config templates](templates/gitlab_ci_ymls.md) - [Groups](groups.md) +- [Group Access Requests](access_requests.md) +- [Group Members](members.md) - [Issues](issues.md) - [Keys](keys.md) - [Labels](labels.md) - [Merge Requests](merge_requests.md) - [Milestones](milestones.md) -- [Open source license templates](licenses.md) +- [Open source license templates](templates/licenses.md) - [Namespaces](namespaces.md) - [Notes](notes.md) (comments) +- [Notification settings](notification_settings.md) +- [Pipelines](pipelines.md) - [Projects](projects.md) including setting Webhooks +- [Project Access Requests](access_requests.md) +- [Project Members](members.md) - [Project Snippets](project_snippets.md) - [Repositories](repositories.md) - [Repository Files](repository_files.md) @@ -35,8 +45,10 @@ following locations: - [Sidekiq metrics](sidekiq_metrics.md) - [System Hooks](system_hooks.md) - [Tags](tags.md) -- [Users](users.md) - [Todos](todos.md) +- [Users](users.md) +- [Validate CI configuration](ci/lint.md) +- [Version](version.md) ### Internal CI API @@ -47,11 +59,12 @@ The following documentation is for the [internal CI API](ci/README.md): ## Authentication -All API requests require authentication via a token. There are three types of tokens -available: private tokens, OAuth 2 tokens, and personal access tokens. +All API requests require authentication via a session cookie or token. There are +three types of tokens available: private tokens, OAuth 2 tokens, and personal +access tokens. -If a token is invalid or omitted, an error message will be returned with -status code `401`: +If authentication information is invalid or omitted, an error message will be +returned with status code `401`: ```json { @@ -74,14 +87,14 @@ You can use an OAuth 2 token to authenticate with the API by passing it either i Example of using the OAuth2 token in the header: ```shell -curl -H "Authorization: Bearer OAUTH-TOKEN" https://gitlab.example.com/api/v3/projects +curl --header "Authorization: Bearer OAUTH-TOKEN" https://gitlab.example.com/api/v3/projects ``` Read more about [GitLab as an OAuth2 client](oauth2.md). ### Personal Access Tokens -> **Note:** This feature was [introduced][ce-3749] in GitLab 8.8 +> [Introduced][ce-3749] in GitLab 8.8. You can create as many personal access tokens as you like from your GitLab profile (`/profile/personal_access_tokens`); perhaps one for each application @@ -90,6 +103,13 @@ that needs access to the GitLab API. Once you have your token, pass it to the API using either the `private_token` parameter or the `PRIVATE-TOKEN` header. + +### Session Cookie + +When signing in to GitLab as an ordinary user, a `_gitlab_session` cookie is +set. The API will use this cookie for authentication if it is present, but using +the API to generate a new session cookie is currently not supported. + ## Basic Usage API requests should be prefixed with `api` and the API version. The API version @@ -154,7 +174,7 @@ be returned with status code `403`: ```json { - "message": "403 Forbidden: Must be admin to use sudo" + "message": "403 Forbidden - Must be admin to use sudo" } ``` @@ -204,7 +224,7 @@ resources you can pass the following parameters: In the example below, we list 50 [namespaces](namespaces.md) per page. ```bash -curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/namespaces?per_page=50 +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/namespaces?per_page=50 ``` ### Pagination Link header @@ -218,7 +238,7 @@ and we request the second page (`page=2`) of [comments](notes.md) of the issue with ID `8` which belongs to the project with ID `8`: ```bash -curl -I -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/8/issues/8/notes?per_page=3&page=2 +curl --head --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/8/issues/8/notes?per_page=3&page=2 ``` The response will then be: @@ -338,6 +358,19 @@ follows: } ``` +## Unknown route + +When you try to access an API URL that does not exist you will receive 404 Not Found. + +``` +HTTP/1.1 404 Not Found +Content-Type: application/json +{ + "error": "404 Not Found" +} +``` + + ## Clients There are many unofficial GitLab API Clients for most of the popular diff --git a/doc/api/access_requests.md b/doc/api/access_requests.md new file mode 100644 index 00000000000..ea308b54d62 --- /dev/null +++ b/doc/api/access_requests.md @@ -0,0 +1,147 @@ +# Group and project access requests + + >**Note:** This feature was introduced in GitLab 8.11 + + **Valid access levels** + + The access levels are defined in the `Gitlab::Access` module. Currently, these levels are recognized: + +``` +10 => Guest access +20 => Reporter access +30 => Developer access +40 => Master access +50 => Owner access # Only valid for groups +``` + +## List access requests for a group or project + +Gets a list of access requests viewable by the authenticated user. + +Returns `200` if the request succeeds. + +``` +GET /groups/:id/access_requests +GET /projects/:id/access_requests +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/access_requests +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/access_requests +``` + +Example response: + +```json +[ + { + "id": 1, + "username": "raymond_smith", + "name": "Raymond Smith", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "requested_at": "2012-10-22T14:13:35Z" + }, + { + "id": 2, + "username": "john_doe", + "name": "John Doe", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "requested_at": "2012-10-22T14:13:35Z" + } +] +``` + +## Request access to a group or project + +Requests access for the authenticated user to a group or project. + +Returns `201` if the request succeeds. + +``` +POST /groups/:id/access_requests +POST /projects/:id/access_requests +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/access_requests +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/access_requests +``` + +Example response: + +```json +{ + "id": 1, + "username": "raymond_smith", + "name": "Raymond Smith", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "requested_at": "2012-10-22T14:13:35Z" +} +``` + +## Approve an access request + +Approves an access request for the given user. + +Returns `201` if the request succeeds. + +``` +PUT /groups/:id/access_requests/:user_id/approve +PUT /projects/:id/access_requests/:user_id/approve +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `user_id` | integer | yes | The user ID of the access requester | +| `access_level` | integer | no | A valid access level (defaults: `30`, developer access level) | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/access_requests/:user_id/approve?access_level=20 +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/access_requests/:user_id/approve?access_level=20 +``` + +Example response: + +```json +{ + "id": 1, + "username": "raymond_smith", + "name": "Raymond Smith", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "access_level": 20 +} +``` + +## Deny an access request + +Denies an access request for the given user. + +Returns `200` if the request succeeds. + +``` +DELETE /groups/:id/access_requests/:user_id +DELETE /projects/:id/access_requests/:user_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `user_id` | integer | yes | The user ID of the access requester | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/access_requests/:user_id +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/access_requests/:user_id +``` diff --git a/doc/api/award_emoji.md b/doc/api/award_emoji.md index b44f8cfd628..06111f4ab67 100644 --- a/doc/api/award_emoji.md +++ b/doc/api/award_emoji.md @@ -1,12 +1,13 @@ # Award Emoji - >**Note:** This feature was introduced in GitLab 8.9 +> [Introduced][ce-4575] in GitLab 8.9, Snippet support in 8.12 + An awarded emoji tells a thousand words, and can be awarded on issues, merge -requests and notes/comments. Issues, merge requests and notes are further called +requests, snippets, and notes/comments. Issues, merge requests, snippets, and notes are further called `awardables`. -## Issues and merge requests +## Issues, merge requests, and snippets ### List an awardable's award emoji @@ -15,6 +16,7 @@ Gets a list of all award emoji ``` GET /projects/:id/issues/:issue_id/award_emoji GET /projects/:id/merge_requests/:merge_request_id/award_emoji +GET /projects/:id/snippets/:snippet_id/award_emoji ``` Parameters: @@ -25,7 +27,7 @@ Parameters: | `awardable_id` | integer | yes | The ID of an awardable | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji ``` Example Response: @@ -41,7 +43,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/root" + "web_url": "http://gitlab.example.com/root" }, "created_at": "2016-06-15T10:09:34.206Z", "updated_at": "2016-06-15T10:09:34.206Z", @@ -57,7 +59,7 @@ Example Response: "id": 26, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/user4" + "web_url": "http://gitlab.example.com/user4" }, "created_at": "2016-06-15T10:09:34.177Z", "updated_at": "2016-06-15T10:09:34.177Z", @@ -67,13 +69,14 @@ Example Response: ] ``` -### Get single issue note +### Get single award emoji -Gets a single award emoji +Gets a single award emoji from an issue, snippet, or merge request. ``` GET /projects/:id/issues/:issue_id/award_emoji/:award_id GET /projects/:id/merge_requests/:merge_request_id/award_emoji/:award_id +GET /projects/:id/snippets/:snippet_id/award_emoji/:award_id ``` Parameters: @@ -85,7 +88,7 @@ Parameters: | `award_id` | integer | yes | The ID of the award emoji | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/1 +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/1 ``` Example Response: @@ -100,7 +103,7 @@ Example Response: "id": 26, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/user4" + "web_url": "http://gitlab.example.com/user4" }, "created_at": "2016-06-15T10:09:34.177Z", "updated_at": "2016-06-15T10:09:34.177Z", @@ -116,6 +119,7 @@ This end point creates an award emoji on the specified resource ``` POST /projects/:id/issues/:issue_id/award_emoji POST /projects/:id/merge_requests/:merge_request_id/award_emoji +POST /projects/:id/snippets/:snippet_id/award_emoji ``` Parameters: @@ -127,7 +131,7 @@ Parameters: | `name` | string | yes | The name of the emoji, without colons | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji?name=blowfish +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji?name=blowfish ``` Example Response: @@ -142,7 +146,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/root" + "web_url": "http://gitlab.example.com/root" }, "created_at": "2016-06-17T17:47:29.266Z", "updated_at": "2016-06-17T17:47:29.266Z", @@ -159,6 +163,7 @@ admins or the author of the award. Status code 200 on success, 401 if unauthoriz ``` DELETE /projects/:id/issues/:issue_id/award_emoji/:award_id DELETE /projects/:id/merge_requests/:merge_request_id/award_emoji/:award_id +DELETE /projects/:id/snippets/:snippet_id/award_emoji/:award_id ``` Parameters: @@ -170,7 +175,7 @@ Parameters: | `award_id` | integer | yes | The ID of a award_emoji | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/344 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/344 ``` Example Response: @@ -185,7 +190,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/root" + "web_url": "http://gitlab.example.com/root" }, "created_at": "2016-06-17T17:47:29.266Z", "updated_at": "2016-06-17T17:47:29.266Z", @@ -197,7 +202,7 @@ Example Response: ## Award Emoji on Notes The endpoints documented above are available for Notes as well. Notes -are a sub-resource of Issues and Merge Requests. The examples below +are a sub-resource of Issues, Merge Requests, or Snippets. The examples below describe working with Award Emoji on notes for an Issue, but can be easily adapted for notes on a Merge Request. @@ -217,7 +222,7 @@ Parameters: ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji ``` Example Response: @@ -233,7 +238,7 @@ Example Response: "id": 26, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/user4" + "web_url": "http://gitlab.example.com/user4" }, "created_at": "2016-06-15T10:09:34.197Z", "updated_at": "2016-06-15T10:09:34.197Z", @@ -259,7 +264,7 @@ Parameters: | `award_id` | integer | yes | The ID of the award emoji | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji/2 +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji/2 ``` Example Response: @@ -274,7 +279,7 @@ Example Response: "id": 26, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/7e65550957227bd38fe2d7fbc6fd2f7b?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/user4" + "web_url": "http://gitlab.example.com/user4" }, "created_at": "2016-06-15T10:09:34.197Z", "updated_at": "2016-06-15T10:09:34.197Z", @@ -299,7 +304,7 @@ Parameters: | `name` | string | yes | The name of the emoji, without colons | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji?name=rocket +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/notes/1/award_emoji?name=rocket ``` Example Response: @@ -314,7 +319,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/root" + "web_url": "http://gitlab.example.com/root" }, "created_at": "2016-06-17T19:59:55.888Z", "updated_at": "2016-06-17T19:59:55.888Z", @@ -342,7 +347,7 @@ Parameters: | `award_id` | integer | yes | The ID of a award_emoji | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/345 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" http://gitlab.example.com/api/v3/projects/1/issues/80/award_emoji/345 ``` Example Response: @@ -357,7 +362,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "http://gitlab.example.com/u/root" + "web_url": "http://gitlab.example.com/root" }, "created_at": "2016-06-17T19:59:55.888Z", "updated_at": "2016-06-17T19:59:55.888Z", @@ -365,3 +370,5 @@ Example Response: "awardable_type": "Note" } ``` + +[ce-4575]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4575 diff --git a/doc/api/boards.md b/doc/api/boards.md new file mode 100644 index 00000000000..28681719f43 --- /dev/null +++ b/doc/api/boards.md @@ -0,0 +1,251 @@ +# Boards + +Every API call to boards must be authenticated. + +If a user is not a member of a project and the project is private, a `GET` +request on that project will result to a `404` status code. + +## Project Board + +Lists Issue Boards in the given project. + +``` +GET /projects/:id/boards +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/boards +``` + +Example response: + +```json +[ + { + "id" : 1, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +] +``` + +## List board lists + +Get a list of the board's lists. +Does not include `backlog` and `done` lists + +``` +GET /projects/:id/boards/:board_id/lists +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/boards/1/lists +``` + +Example response: + +```json +[ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } +] +``` + +## Single board list + +Get a single board list. + +``` +GET /projects/:id/boards/:board_id/lists/:list_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `board_id` | integer | yes | The ID of a board | +| `list_id`| integer | yes | The ID of a board's list | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/boards/1/lists/1 +``` + +Example response: + +```json +{ + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 +} +``` + +## New board list + +Creates a new Issue Board list. + +If the operation is successful, a status code of `200` and the newly-created +list is returned. If an error occurs, an error number and a message explaining +the reason is returned. + +``` +POST /projects/:id/boards/:board_id/lists +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `board_id` | integer | yes | The ID of a board | +| `label_id` | integer | yes | The ID of a label | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/boards/1/lists?label_id=5 +``` + +Example response: + +```json +{ + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 +} +``` + +## Edit board list + +Updates an existing Issue Board list. This call is used to change list position. + +If the operation is successful, a code of `200` and the updated board list is +returned. If an error occurs, an error number and a message explaining the +reason is returned. + +``` +PUT /projects/:id/boards/:board_id/lists/:list_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | +| `position` | integer | yes | The position of the list | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/boards/1/lists/1?position=2 +``` + +Example response: + +```json +{ + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 +} +``` + +## Delete a board list + +Only for admins and project owners. Soft deletes the board list in question. +If the operation is successful, a status code `200` is returned. In case you cannot +destroy this board list, or it is not present, code `404` is given. + +``` +DELETE /projects/:id/boards/:board_id/lists/:list_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `board_id` | integer | yes | The ID of a board | +| `list_id` | integer | yes | The ID of a board's list | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/boards/1/lists/1 +``` +Example response: + +```json +{ + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 +} +``` diff --git a/doc/api/branches.md b/doc/api/branches.md index abc4732c395..0b5f7778fc7 100644 --- a/doc/api/branches.md +++ b/doc/api/branches.md @@ -13,7 +13,7 @@ GET /projects/:id/repository/branches | `id` | integer | yes | The ID of a project | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches ``` Example response: @@ -23,6 +23,8 @@ Example response: { "name": "master", "protected": true, + "developers_can_push": false, + "developers_can_merge": false, "commit": { "author_email": "john@example.com", "author_name": "John Smith", @@ -55,7 +57,7 @@ GET /projects/:id/repository/branches/:branch | `branch` | string | yes | The name of the branch | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches/master +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches/master ``` Example response: @@ -64,6 +66,8 @@ Example response: { "name": "master", "protected": true, + "developers_can_push": false, + "developers_can_merge": false, "commit": { "author_email": "john@example.com", "author_name": "John Smith", @@ -91,13 +95,15 @@ PUT /projects/:id/repository/branches/:branch/protect ``` ```bash -curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches/master/protect +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches/master/protect?developers_can_push=true&developers_can_merge=true ``` | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | integer | yes | The ID of a project | | `branch` | string | yes | The name of the branch | +| `developers_can_push` | boolean | no | Flag if developers can push to the branch | +| `developers_can_merge` | boolean | no | Flag if developers can merge to the branch | Example response: @@ -117,7 +123,9 @@ Example response: ] }, "name": "master", - "protected": true + "protected": true, + "developers_can_push": true, + "developers_can_merge": true } ``` @@ -132,7 +140,7 @@ PUT /projects/:id/repository/branches/:branch/unprotect ``` ```bash -curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches/master/unprotect +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/branches/master/unprotect ``` | Attribute | Type | Required | Description | @@ -158,7 +166,9 @@ Example response: ] }, "name": "master", - "protected": false + "protected": false, + "developers_can_push": false, + "developers_can_merge": false } ``` @@ -175,7 +185,7 @@ POST /projects/:id/repository/branches | `ref` | string | yes | The branch name or commit SHA to create branch from | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/branches?branch_name=newbranch&ref=master" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/branches?branch_name=newbranch&ref=master" ``` Example response: @@ -196,7 +206,9 @@ Example response: ] }, "name": "newbranch", - "protected": false + "protected": false, + "developers_can_push": false, + "developers_can_merge": false } ``` @@ -218,7 +230,7 @@ It returns `200` if it succeeds, `404` if the branch to be deleted does not exis or `400` for other reasons. In case of an error, an explaining message is provided. ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/branches/newbranch" +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/branches/newbranch" ``` Example response: diff --git a/doc/api/broadcast_messages.md b/doc/api/broadcast_messages.md new file mode 100644 index 00000000000..c3a9207a3ae --- /dev/null +++ b/doc/api/broadcast_messages.md @@ -0,0 +1,158 @@ +# Broadcast Messages + +> **Note:** This feature was introduced in GitLab 8.12. + +The broadcast message API is only accessible to administrators. All requests by +guests will respond with `401 Unauthorized`, and all requests by normal users +will respond with `403 Forbidden`. + +## Get all broadcast messages + +``` +GET /broadcast_messages +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/broadcast_messages +``` + +Example response: + +```json +[ + { + "message":"Example broadcast message", + "starts_at":"2016-08-24T23:21:16.078Z", + "ends_at":"2016-08-26T23:21:16.080Z", + "color":"#E75E40", + "font":"#FFFFFF", + "id":1, + "active": false + } +] +``` + +## Get a specific broadcast message + +``` +GET /broadcast_messages/:id +``` + +| Attribute | Type | Required | Description | +| ----------- | -------- | -------- | ------------------------- | +| `id` | integer | yes | Broadcast message ID | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/broadcast_messages/1 +``` + +Example response: + +```json +{ + "message":"Deploy in progress", + "starts_at":"2016-08-24T23:21:16.078Z", + "ends_at":"2016-08-26T23:21:16.080Z", + "color":"#cecece", + "font":"#FFFFFF", + "id":1, + "active":false +} +``` + +## Create a broadcast message + +Responds with `400 Bad request` when the `message` parameter is missing or the +`color` or `font` values are invalid, and `201 Created` when the broadcast +message was successfully created. + +``` +POST /broadcast_messages +``` + +| Attribute | Type | Required | Description | +| ----------- | -------- | -------- | ---------------------------------------------------- | +| `message` | string | yes | Message to display | +| `starts_at` | datetime | no | Starting time (defaults to current time) | +| `ends_at` | datetime | no | Ending time (defaults to one hour from current time) | +| `color` | string | no | Background color hex code | +| `font` | string | no | Foreground color hex code | + +```bash +curl --data "message=Deploy in progress&color=#cecece" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/broadcast_messages +``` + +Example response: + +```json +{ + "message":"Deploy in progress", + "starts_at":"2016-08-26T00:41:35.060Z", + "ends_at":"2016-08-26T01:41:35.060Z", + "color":"#cecece", + "font":"#FFFFFF", + "id":1, + "active": true +} +``` + +## Update a broadcast message + +``` +PUT /broadcast_messages/:id +``` + +| Attribute | Type | Required | Description | +| ----------- | -------- | -------- | ------------------------- | +| `id` | integer | yes | Broadcast message ID | +| `message` | string | no | Message to display | +| `starts_at` | datetime | no | Starting time | +| `ends_at` | datetime | no | Ending time | +| `color` | string | no | Background color hex code | +| `font` | string | no | Foreground color hex code | + +```bash +curl --request PUT --data "message=Update message&color=#000" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/broadcast_messages/1 +``` + +Example response: + +```json +{ + "message":"Update message", + "starts_at":"2016-08-26T00:41:35.060Z", + "ends_at":"2016-08-26T01:41:35.060Z", + "color":"#000", + "font":"#FFFFFF", + "id":1, + "active": true +} +``` + +## Delete a broadcast message + +``` +DELETE /broadcast_messages/:id +``` + +| Attribute | Type | Required | Description | +| ----------- | -------- | -------- | ------------------------- | +| `id` | integer | yes | Broadcast message ID | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/broadcast_messages/1 +``` + +Example response: + +```json +{ + "message":"Update message", + "starts_at":"2016-08-26T00:41:35.060Z", + "ends_at":"2016-08-26T01:41:35.060Z", + "color":"#000", + "font":"#FFFFFF", + "id":1, + "active": true +} +``` diff --git a/doc/api/build_triggers.md b/doc/api/build_triggers.md index 0881a7d7a90..1b7a1840138 100644 --- a/doc/api/build_triggers.md +++ b/doc/api/build_triggers.md @@ -15,7 +15,7 @@ GET /projects/:id/triggers | `id` | integer | yes | The ID of a project | ``` -curl -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers" +curl --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers" ``` ```json @@ -51,7 +51,7 @@ GET /projects/:id/triggers/:token | `token` | string | yes | The `token` of a trigger | ``` -curl -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers/7b9148c158980bbd9bcea92c17522d" +curl --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers/7b9148c158980bbd9bcea92c17522d" ``` ```json @@ -77,7 +77,7 @@ POST /projects/:id/triggers | `id` | integer | yes | The ID of a project | ``` -curl -X POST -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers" +curl --request POST --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers" ``` ```json @@ -104,7 +104,7 @@ DELETE /projects/:id/triggers/:token | `token` | string | yes | The `token` of a trigger | ``` -curl -X DELETE -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers/7b9148c158980bbd9bcea92c17522d" +curl --request DELETE --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/triggers/7b9148c158980bbd9bcea92c17522d" ``` ```json diff --git a/doc/api/build_variables.md b/doc/api/build_variables.md index b96f1bdac8a..a21751a49ea 100644 --- a/doc/api/build_variables.md +++ b/doc/api/build_variables.md @@ -13,7 +13,7 @@ GET /projects/:id/variables | `id` | integer | yes | The ID of a project | ``` -curl -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables" +curl --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables" ``` ```json @@ -43,7 +43,7 @@ GET /projects/:id/variables/:key | `key` | string | yes | The `key` of a variable | ``` -curl -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables/TEST_VARIABLE_1" +curl --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables/TEST_VARIABLE_1" ``` ```json @@ -68,7 +68,7 @@ POST /projects/:id/variables | `value` | string | yes | The `value` of a variable | ``` -curl -X POST -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables" -F "key=NEW_VARIABLE" -F "value=new value" +curl --request POST --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables" --form "key=NEW_VARIABLE" --form "value=new value" ``` ```json @@ -93,7 +93,7 @@ PUT /projects/:id/variables/:key | `value` | string | yes | The `value` of a variable | ``` -curl -X PUT -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables/NEW_VARIABLE" -F "value=updated value" +curl --request PUT --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables/NEW_VARIABLE" --form "value=updated value" ``` ```json @@ -117,7 +117,7 @@ DELETE /projects/:id/variables/:key | `key` | string | yes | The `key` of a variable | ``` -curl -X DELETE -H "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables/VARIABLE_1" +curl --request DELETE --header "PRIVATE_TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/variables/VARIABLE_1" ``` ```json diff --git a/doc/api/builds.md b/doc/api/builds.md index 2adea11247e..e40f198696d 100644 --- a/doc/api/builds.md +++ b/doc/api/builds.md @@ -14,7 +14,7 @@ GET /projects/:id/builds | `scope` | string **or** array of strings | no | The scope of builds to show, one or array of: `pending`, `running`, `failed`, `success`, `canceled`; showing all builds if none provided | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds" ``` Example of response @@ -40,6 +40,12 @@ Example of response "finished_at": "2015-12-24T17:54:27.895Z", "id": 7, "name": "teaspoon", + "pipeline": { + "id": 6, + "ref": "master", + "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", + "status": "pending" + } "ref": "master", "runner": null, "stage": "test", @@ -58,7 +64,7 @@ Example of response "state": "active", "twitter": "", "username": "root", - "web_url": "http://gitlab.dev/u/root", + "web_url": "http://gitlab.dev/root", "website_url": "" } }, @@ -78,6 +84,12 @@ Example of response "finished_at": "2015-12-24T17:54:24.921Z", "id": 6, "name": "spinach:other", + "pipeline": { + "id": 6, + "ref": "master", + "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", + "status": "pending" + } "ref": "master", "runner": null, "stage": "test", @@ -96,7 +108,7 @@ Example of response "state": "active", "twitter": "", "username": "root", - "web_url": "http://gitlab.dev/u/root", + "web_url": "http://gitlab.dev/root", "website_url": "" } } @@ -123,7 +135,7 @@ GET /projects/:id/repository/commits/:sha/builds | `scope` | string **or** array of strings | no | The scope of builds to show, one or array of: `pending`, `running`, `failed`, `success`, `canceled`; showing all builds if none provided | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/repository/commits/0ff3ae198f8601a285adcf5c0fff204ee6fba5fd/builds" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/repository/commits/0ff3ae198f8601a285adcf5c0fff204ee6fba5fd/builds" ``` Example of response @@ -146,6 +158,12 @@ Example of response "finished_at": "2016-01-11T10:14:09.526Z", "id": 69, "name": "rubocop", + "pipeline": { + "id": 6, + "ref": "master", + "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", + "status": "pending" + } "ref": "master", "runner": null, "stage": "test", @@ -170,6 +188,12 @@ Example of response "finished_at": "2015-12-24T17:54:33.913Z", "id": 9, "name": "brakeman", + "pipeline": { + "id": 6, + "ref": "master", + "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", + "status": "pending" + } "ref": "master", "runner": null, "stage": "test", @@ -188,7 +212,7 @@ Example of response "state": "active", "twitter": "", "username": "root", - "web_url": "http://gitlab.dev/u/root", + "web_url": "http://gitlab.dev/root", "website_url": "" } } @@ -209,7 +233,7 @@ GET /projects/:id/builds/:build_id | `build_id` | integer | yes | The ID of a build | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/8" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/8" ``` Example of response @@ -231,6 +255,12 @@ Example of response "finished_at": "2015-12-24T17:54:31.198Z", "id": 8, "name": "rubocop", + "pipeline": { + "id": 6, + "ref": "master", + "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", + "status": "pending" + } "ref": "master", "runner": null, "stage": "test", @@ -249,7 +279,7 @@ Example of response "state": "active", "twitter": "", "username": "root", - "web_url": "http://gitlab.dev/u/root", + "web_url": "http://gitlab.dev/root", "website_url": "" } } @@ -271,7 +301,7 @@ GET /projects/:id/builds/:build_id/artifacts | `build_id` | integer | yes | The ID of a build | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/8/artifacts" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/8/artifacts" ``` Response: @@ -283,6 +313,40 @@ Response: [ce-2893]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2893 +## Download the artifacts file + +> [Introduced][ce-5347] in GitLab 8.10. + +Download the artifacts file from the given reference name and job provided the +build finished successfully. + +``` +GET /projects/:id/builds/artifacts/:ref_name/download?job=name +``` + +Parameters + +| Attribute | Type | Required | Description | +|-------------|---------|----------|-------------------------- | +| `id` | integer | yes | The ID of a project | +| `ref_name` | string | yes | The ref from a repository | +| `job` | string | yes | The name of the job | + +Example request: + +``` +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/artifacts/master/download?job=test" +``` + +Example response: + +| Status | Description | +|-----------|---------------------------------| +| 200 | Serves the artifacts file | +| 404 | Build not found or no artifacts | + +[ce-5347]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5347 + ## Get a trace file Get a trace of a specific build of a project @@ -297,7 +361,7 @@ GET /projects/:id/builds/:build_id/trace | build_id | integer | yes | The ID of a build | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/8/trace" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/8/trace" ``` Response: @@ -321,7 +385,7 @@ POST /projects/:id/builds/:build_id/cancel | `build_id` | integer | yes | The ID of a build | ``` -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/cancel" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/cancel" ``` Example of response @@ -367,7 +431,7 @@ POST /projects/:id/builds/:build_id/retry | `build_id` | integer | yes | The ID of a build | ``` -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/retry" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/retry" ``` Example of response @@ -409,7 +473,7 @@ POST /projects/:id/builds/:build_id/erase Parameters -| Attribute | Type | required | Description | +| Attribute | Type | Required | Description | |-------------|---------|----------|---------------------| | `id` | integer | yes | The ID of a project | | `build_id` | integer | yes | The ID of a build | @@ -417,7 +481,7 @@ Parameters Example of request ``` -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/erase" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/erase" ``` Example of response @@ -459,7 +523,7 @@ POST /projects/:id/builds/:build_id/artifacts/keep Parameters -| Attribute | Type | required | Description | +| Attribute | Type | Required | Description | |-------------|---------|----------|---------------------| | `id` | integer | yes | The ID of a project | | `build_id` | integer | yes | The ID of a build | @@ -467,7 +531,7 @@ Parameters Example request: ``` -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/artifacts/keep" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/artifacts/keep" ``` Example response: @@ -498,3 +562,49 @@ Example response: "user": null } ``` + +## Play a build + +Triggers a manual action to start a build. + +``` +POST /projects/:id/builds/:build_id/play +``` + +| Attribute | Type | Required | Description | +|------------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a project | +| `build_id` | integer | yes | The ID of a build | + +``` +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/builds/1/play" +``` + +Example of response + +```json +{ + "commit": { + "author_email": "admin@example.com", + "author_name": "Administrator", + "created_at": "2015-12-24T16:51:14.000+01:00", + "id": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", + "message": "Test the CI integration.", + "short_id": "0ff3ae19", + "title": "Test the CI integration." + }, + "coverage": null, + "created_at": "2016-01-11T10:13:33.506Z", + "artifacts_file": null, + "finished_at": null, + "id": 69, + "name": "rubocop", + "ref": "master", + "runner": null, + "stage": "test", + "started_at": null, + "status": "started", + "tag": false, + "user": null +} +``` diff --git a/doc/api/ci/builds.md b/doc/api/ci/builds.md index d779463fd8c..b6d79706a84 100644 --- a/doc/api/ci/builds.md +++ b/doc/api/ci/builds.md @@ -35,9 +35,18 @@ POST /ci/api/v1/builds/register ``` -curl -X POST "https://gitlab.example.com/ci/api/v1/builds/register" -F "token=t0k3n" +curl --request POST "https://gitlab.example.com/ci/api/v1/builds/register" --form "token=t0k3n" ``` +**Responses:** + +| Status | Data |Description | +|--------|------|---------------------------------------------------------------------------| +| `201` | yes | When a build is scheduled for a runner | +| `204` | no | When no builds are scheduled for a runner (for GitLab Runner >= `v1.3.0`) | +| `403` | no | When invalid token is used or no token is sent | +| `404` | no | When no builds are scheduled for a runner (for GitLab Runner < `v1.3.0`) **or** when the runner is set to `paused` in GitLab runner's configuration page | + ### Update details of an existing build ``` @@ -52,7 +61,7 @@ PUT /ci/api/v1/builds/:id | `trace` | string | no | The trace of a build | ``` -curl -X PUT "https://gitlab.example.com/ci/api/v1/builds/1234" -F "token=t0k3n" -F "state=running" -F "trace=Running git clone...\n" +curl --request PUT "https://gitlab.example.com/ci/api/v1/builds/1234" --form "token=t0k3n" --form "state=running" --form "trace=Running git clone...\n" ``` ### Incremental build trace update @@ -87,7 +96,7 @@ Headers: | `Content-Range` | string | yes | Bytes range of trace that is sent | ``` -curl -X PATCH "https://gitlab.example.com/ci/api/v1/builds/1234/trace.txt" -H "BUILD-TOKEN=build_t0k3n" -H "Content-Range=0-21" -d "Running git clone...\n" +curl --request PATCH "https://gitlab.example.com/ci/api/v1/builds/1234/trace.txt" --header "BUILD-TOKEN=build_t0k3n" --header "Content-Range=0-21" --data "Running git clone...\n" ``` @@ -104,7 +113,7 @@ POST /ci/api/v1/builds/:id/artifacts | `file` | mixed | yes | Artifacts file | ``` -curl -X POST "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" -F "file=@/path/to/file" +curl --request POST "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" --form "token=build_t0k3n" --form "file=@/path/to/file" ``` ### Download the artifacts file from build @@ -119,7 +128,7 @@ GET /ci/api/v1/builds/:id/artifacts | `token` | string | yes | The build authorization token | ``` -curl "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" +curl "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" --form "token=build_t0k3n" ``` ### Remove the artifacts file from build @@ -134,5 +143,5 @@ DELETE /ci/api/v1/builds/:id/artifacts | `token` | string | yes | The build authorization token | ``` -curl -X DELETE "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" -F "token=build_t0k3n" +curl --request DELETE "https://gitlab.example.com/ci/api/v1/builds/1234/artifacts" --form "token=build_t0k3n" ``` diff --git a/doc/api/ci/lint.md b/doc/api/ci/lint.md new file mode 100644 index 00000000000..0c96b3ee335 --- /dev/null +++ b/doc/api/ci/lint.md @@ -0,0 +1,49 @@ +# Validate the .gitlab-ci.yml + +> [Introduced][ce-5953] in GitLab 8.12. + +Checks if your .gitlab-ci.yml file is valid. + +``` +POST ci/lint +``` + +| Attribute | Type | Required | Description | +| ---------- | ------- | -------- | -------- | +| `content` | string | yes | the .gitlab-ci.yaml content| + +```bash +curl --header "Content-Type: application/json" https://gitlab.example.com/api/v3/ci/lint --data '{"content": "{ \"image\": \"ruby:2.1\", \"services\": [\"postgres\"], \"before_script\": [\"gem install bundler\", \"bundle install\", \"bundle exec rake db:create\"], \"variables\": {\"DB_NAME\": \"postgres\"}, \"types\": [\"test\", \"deploy\", \"notify\"], \"rspec\": { \"script\": \"rake spec\", \"tags\": [\"ruby\", \"postgres\"], \"only\": [\"branches\"]}}"}' +``` + +Be sure to copy paste the exact contents of `.gitlab-ci.yml` as YAML is very picky about indentation and spaces. + +Example responses: + +* Valid content: + + ```json + { + "status": "valid", + "errors": [] + } + ``` + +* Invalid content: + + ```json + { + "status": "invalid", + "errors": [ + "variables config should be a hash of key value pairs" + ] + } + ``` + +* Without the content attribute: + + ```json + { + "error": "content is missing" + } + ``` diff --git a/doc/api/ci/runners.md b/doc/api/ci/runners.md index 96b3c42f773..16028d1f124 100644 --- a/doc/api/ci/runners.md +++ b/doc/api/ci/runners.md @@ -12,7 +12,9 @@ communication channel. For the consumer API see the This API uses two types of authentication: 1. Unique Runner's token, which is the token assigned to the Runner after it - has been registered. + has been registered. This token can be found on the Runner's edit page (go to + **Project > Runners**, select one of the Runners listed under **Runners activated for + this project**). 2. Using Runners' registration token. This is a token that can be found in project's settings. @@ -35,7 +37,7 @@ POST /ci/api/v1/runners/register Example request: ```sh -curl -X POST "https://gitlab.example.com/ci/api/v1/runners/register" -F "token=t0k3n" +curl --request POST "https://gitlab.example.com/ci/api/v1/runners/register" --form "token=t0k3n" ``` ## Delete a Runner @@ -48,10 +50,10 @@ DELETE /ci/api/v1/runners/delete | Attribute | Type | Required | Description | | --------- | ------- | --------- | ----------- | -| `token` | string | yes | Runner's registration token | +| `token` | string | yes | Unique Runner's token | Example request: ```sh -curl -X DELETE "https://gitlab.example.com/ci/api/v1/runners/delete" -F "token=t0k3n" +curl --request DELETE "https://gitlab.example.com/ci/api/v1/runners/delete" --form "token=t0k3n" ``` diff --git a/doc/api/commits.md b/doc/api/commits.md index 57c2e1d9b87..e1ed99d98d3 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -10,13 +10,13 @@ GET /projects/:id/repository/commits | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project | +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user | `ref_name` | string | no | The name of a repository branch or tag or if not given the default branch | | `since` | string | no | Only commits after or in this date will be returned in ISO 8601 format YYYY-MM-DDTHH:MM:SSZ | | `until` | string | no | Only commits before or in this date will be returned in ISO 8601 format YYYY-MM-DDTHH:MM:SSZ | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits" ``` Example response: @@ -46,6 +46,91 @@ Example response: ] ``` +## Create a commit with multiple files and actions + +> [Introduced][ce-6096] in GitLab 8.13. + +Create a commit by posting a JSON payload + +``` +POST /projects/:id/repository/commits +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME | +| `branch_name` | string | yes | The name of a branch | +| `commit_message` | string | yes | Commit message | +| `actions[]` | array | yes | An array of action hashes to commit as a batch. See the next table for what attributes it can take. | +| `author_email` | string | no | Specify the commit author's email address | +| `author_name` | string | no | Specify the commit author's name | + + +| `actions[]` Attribute | Type | Required | Description | +| --------------------- | ---- | -------- | ----------- | +| `action` | string | yes | The action to perform, `create`, `delete`, `move`, `update` | +| `file_path` | string | yes | Full path to the file. Ex. `lib/class.rb` | +| `previous_path` | string | no | Original full path to the file being moved. Ex. `lib/class1.rb` | +| `content` | string | no | File content, required for all except `delete`. Optional for `move` | +| `encoding` | string | no | `text` or `base64`. `text` is default. | + +```bash +PAYLOAD=$(cat << 'JSON' +{ + "branch_name": "master", + "commit_message": "some commit message", + "actions": [ + { + "action": "create", + "file_path": "foo/bar", + "content": "some content" + }, + { + "action": "delete", + "file_path": "foo/bar2", + }, + { + "action": "move", + "file_path": "foo/bar3", + "previous_path": "foo/bar4", + "content": "some content" + }, + { + "action": "update", + "file_path": "foo/bar5", + "content": "new content" + } + ] +} +JSON +) +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data "$PAYLOAD" https://gitlab.example.com/api/v3/projects/1/repository/commits +``` + +Example response: +```json +{ + "id": "ed899a2f4b50b4370feeea94676502b42383c746", + "short_id": "ed899a2f4b5", + "title": "some commit message", + "author_name": "Dmitriy Zaporozhets", + "author_email": "dzaporozhets@sphereconsultinginc.com", + "created_at": "2016-09-20T09:26:24.000-07:00", + "message": "some commit message", + "parent_ids": [ + "ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba" + ], + "committed_date": "2016-09-20T09:26:24.000-07:00", + "authored_date": "2016-09-20T09:26:24.000-07:00", + "stats": { + "additions": 2, + "deletions": 2, + "total": 4 + }, + "status": null +} +``` + ## Get a single commit Get a specific commit identified by the commit hash or name of a branch or tag. @@ -58,11 +143,11 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project | +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user | `sha` | string | yes | The commit hash or name of a repository branch or tag | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits/master +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits/master ``` Example response: @@ -81,6 +166,11 @@ Example response: "parent_ids": [ "ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba" ], + "stats": { + "additions": 15, + "deletions": 10, + "total": 25 + }, "status": "running" } ``` @@ -97,11 +187,11 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project | +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user | `sha` | string | yes | The commit hash or name of a repository branch or tag | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits/master/diff" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits/master/diff" ``` Example response: @@ -133,11 +223,11 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project | +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user | `sha` | string | yes | The commit hash or name of a repository branch or tag | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits/master/comments" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/commits/master/comments" ``` Example response: @@ -182,7 +272,7 @@ POST /projects/:id/repository/commits/:sha/comments | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project | +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user | `sha` | string | yes | The commit SHA or name of a repository branch or tag | | `note` | string | yes | The text of the comment | | `path` | string | no | The file path relative to the repository | @@ -190,7 +280,7 @@ POST /projects/:id/repository/commits/:sha/comments | `line_type` | string | no | The line type. Takes `new` or `old` as arguments | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -F "note=Nice picture man\!" -F "path=dudeism.md" -F "line=11" -F "line_type=new" https://gitlab.example.com/api/v3/projects/17/repository/commits/18f3e63d05582537db6d183d9d557be09e1f90c8/comments +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "note=Nice picture man\!" --form "path=dudeism.md" --form "line=11" --form "line_type=new" https://gitlab.example.com/api/v3/projects/17/repository/commits/18f3e63d05582537db6d183d9d557be09e1f90c8/comments ``` Example response: @@ -198,7 +288,7 @@ Example response: ```json { "author" : { - "web_url" : "https://gitlab.example.com/u/thedude", + "web_url" : "https://gitlab.example.com/thedude", "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png", "username" : "thedude", "state" : "active", @@ -227,15 +317,15 @@ GET /projects/:id/repository/commits/:sha/statuses | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user | `sha` | string | yes | The commit SHA -| `ref_name`| string | no | The name of a repository branch or tag or, if not given, the default branch +| `ref` | string | no | The name of a repository branch or tag or, if not given, the default branch | `stage` | string | no | Filter by [build stage](../ci/yaml/README.md#stages), e.g., `test` | `name` | string | no | Filter by [job name](../ci/yaml/README.md#jobs), e.g., `bundler:audit` | `all` | boolean | no | Return all statuses, not only the latest ones ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/17/repository/commits/18f3e63d05582537db6d183d9d557be09e1f90c8/statuses +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/17/repository/commits/18f3e63d05582537db6d183d9d557be09e1f90c8/statuses ``` Example response: @@ -253,7 +343,7 @@ Example response: "author" : { "username" : "thedude", "state" : "active", - "web_url" : "https://gitlab.example.com/u/thedude", + "web_url" : "https://gitlab.example.com/thedude", "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png", "id" : 28, "name" : "Jeff Lebowski" @@ -280,7 +370,7 @@ Example response: "id" : 28, "name" : "Jeff Lebowski", "username" : "thedude", - "web_url" : "https://gitlab.example.com/u/thedude", + "web_url" : "https://gitlab.example.com/thedude", "state" : "active", "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png" }, @@ -301,7 +391,7 @@ POST /projects/:id/statuses/:sha | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of a project +| `id` | integer/string | yes | The ID of a project or NAMESPACE/PROJECT_NAME owned by the authenticated user | `sha` | string | yes | The commit SHA | `state` | string | yes | The state of the status. Can be one of the following: `pending`, `running`, `success`, `failed`, `canceled` | `ref` | string | no | The `ref` (branch or tag) to which the status refers @@ -310,7 +400,7 @@ POST /projects/:id/statuses/:sha | `description` | string | no | The short description of the status ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/17/statuses/18f3e63d05582537db6d183d9d557be09e1f90c8?state=success" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/17/statuses/18f3e63d05582537db6d183d9d557be09e1f90c8?state=success" ``` Example response: @@ -318,7 +408,7 @@ Example response: ```json { "author" : { - "web_url" : "https://gitlab.example.com/u/thedude", + "web_url" : "https://gitlab.example.com/thedude", "name" : "Jeff Lebowski", "avatar_url" : "https://gitlab.example.com/uploads/user/avatar/28/The-Big-Lebowski-400-400.png", "username" : "thedude", @@ -338,3 +428,5 @@ Example response: "finished_at" : "2016-01-19T09:05:50.365Z" } ``` + +[ce-6096]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6096 "Multi-file commit" diff --git a/doc/api/deploy_key_multiple_projects.md b/doc/api/deploy_key_multiple_projects.md index 3ad836f51b5..73cb4b7ea8c 100644 --- a/doc/api/deploy_key_multiple_projects.md +++ b/doc/api/deploy_key_multiple_projects.md @@ -7,23 +7,23 @@ First, find the ID of the projects you're interested in, by either listing all projects: ``` -curl -H 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' https://gitlab.example.com/api/v3/projects +curl --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' https://gitlab.example.com/api/v3/projects ``` Or finding the ID of a group and then listing all projects in that group: ``` -curl -H 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' https://gitlab.example.com/api/v3/groups +curl --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' https://gitlab.example.com/api/v3/groups # For group 1234: -curl -H 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' https://gitlab.example.com/api/v3/groups/1234 +curl --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' https://gitlab.example.com/api/v3/groups/1234 ``` With those IDs, add the same deploy key to all: ``` for project_id in 321 456 987; do - curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -H "Content-Type: application/json" \ - --data '{"title": "my key", "key": "ssh-rsa AAAA..."}' https://gitlab.example.com/api/v3/projects/${project_id}/keys + curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" \ + --data '{"title": "my key", "key": "ssh-rsa AAAA..."}' https://gitlab.example.com/api/v3/projects/${project_id}/deploy_keys done ``` diff --git a/doc/api/deploy_keys.md b/doc/api/deploy_keys.md index 9da1fe22e61..ca44afbf355 100644 --- a/doc/api/deploy_keys.md +++ b/doc/api/deploy_keys.md @@ -1,11 +1,42 @@ # Deploy Keys -## List deploy keys +## List all deploy keys + +Get a list of all deploy keys across all projects. + +``` +GET /deploy_keys +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/deploy_keys" +``` + +Example response: + +```json +[ + { + "id": 1, + "title": "Public key", + "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=", + "created_at": "2013-10-02T10:12:29Z" + }, + { + "id": 3, + "title": "Another Public key", + "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=", + "created_at": "2013-10-02T11:12:29Z" + } +] +``` + +## List project deploy keys Get a list of a project's deploy keys. ``` -GET /projects/:id/keys +GET /projects/:id/deploy_keys ``` | Attribute | Type | Required | Description | @@ -13,7 +44,7 @@ GET /projects/:id/keys | `id` | integer | yes | The ID of the project | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/keys" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/deploy_keys" ``` Example response: @@ -40,7 +71,7 @@ Example response: Get a single key. ``` -GET /projects/:id/keys/:key_id +GET /projects/:id/deploy_keys/:key_id ``` Parameters: @@ -51,7 +82,7 @@ Parameters: | `key_id` | integer | yes | The ID of the deploy key | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/keys/11" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/deploy_keys/11" ``` Example response: @@ -73,7 +104,7 @@ If the deploy key already exists in another project, it will be joined to curren project only if original one was is accessible by the same user. ``` -POST /projects/:id/keys +POST /projects/:id/deploy_keys ``` | Attribute | Type | Required | Description | @@ -83,7 +114,7 @@ POST /projects/:id/keys | `key` | string | yes | New deploy key | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -H "Content-Type: application/json" --data '{"title": "My deploy key", "key": "ssh-rsa AAAA..."}' "https://gitlab.example.com/api/v3/projects/5/keys/" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"title": "My deploy key", "key": "ssh-rsa AAAA..."}' "https://gitlab.example.com/api/v3/projects/5/deploy_keys/" ``` Example response: @@ -102,7 +133,7 @@ Example response: Delete a deploy key from a project ``` -DELETE /projects/:id/keys/:key_id +DELETE /projects/:id/deploy_keys/:key_id ``` | Attribute | Type | Required | Description | @@ -111,7 +142,7 @@ DELETE /projects/:id/keys/:key_id | `key_id` | integer | yes | The ID of the deploy key | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/keys/13" +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/deploy_keys/13" ``` Example response: @@ -128,3 +159,51 @@ Example response: "id" : 13 } ``` + +## Enable a deploy key + +Enables a deploy key for a project so this can be used. Returns the enabled key, with a status code 201 when successful. + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/deploy_keys/13/enable +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the project | +| `key_id` | integer | yes | The ID of the deploy key | + +Example response: + +```json +{ + "key" : "ssh-rsa AAAA...", + "id" : 12, + "title" : "My deploy key", + "created_at" : "2015-08-29T12:44:31.550Z" +} +``` + +## Disable a deploy key + +Disable a deploy key for a project. Returns the disabled key. + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/deploy_keys/13/disable +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the project | +| `key_id` | integer | yes | The ID of the deploy key | + +Example response: + +```json +{ + "key" : "ssh-rsa AAAA...", + "id" : 12, + "title" : "My deploy key", + "created_at" : "2015-08-29T12:44:31.550Z" +} +``` diff --git a/doc/api/deployments.md b/doc/api/deployments.md new file mode 100644 index 00000000000..3d95c4cde60 --- /dev/null +++ b/doc/api/deployments.md @@ -0,0 +1,218 @@ +# Deployments API + +## List project deployments + +Get a list of deployments in a project. + +``` +GET /projects/:id/deployments +``` + +| Attribute | Type | Required | Description | +|-----------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a project | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/deployments" +``` + +Example of response + +```json +[ + { + "created_at": "2016-08-11T07:36:40.222Z", + "deployable": { + "commit": { + "author_email": "admin@example.com", + "author_name": "Administrator", + "created_at": "2016-08-11T09:36:01.000+02:00", + "id": "99d03678b90d914dbb1b109132516d71a4a03ea8", + "message": "Merge branch 'new-title' into 'master'\r\n\r\nUpdate README\r\n\r\n\r\n\r\nSee merge request !1", + "short_id": "99d03678", + "title": "Merge branch 'new-title' into 'master'\r" + }, + "coverage": null, + "created_at": "2016-08-11T07:36:27.357Z", + "finished_at": "2016-08-11T07:36:39.851Z", + "id": 657, + "name": "deploy", + "ref": "master", + "runner": null, + "stage": "deploy", + "started_at": null, + "status": "success", + "tag": false, + "user": { + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "bio": null, + "created_at": "2016-08-11T07:09:20.351Z", + "id": 1, + "is_admin": true, + "linkedin": "", + "location": null, + "name": "Administrator", + "skype": "", + "state": "active", + "twitter": "", + "username": "root", + "web_url": "http://localhost:3000/root", + "website_url": "" + } + }, + "environment": { + "external_url": "https://about.gitlab.com", + "id": 9, + "name": "production" + }, + "id": 41, + "iid": 1, + "ref": "master", + "sha": "99d03678b90d914dbb1b109132516d71a4a03ea8", + "user": { + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "id": 1, + "name": "Administrator", + "state": "active", + "username": "root", + "web_url": "http://localhost:3000/root" + } + }, + { + "created_at": "2016-08-11T11:32:35.444Z", + "deployable": { + "commit": { + "author_email": "admin@example.com", + "author_name": "Administrator", + "created_at": "2016-08-11T13:28:26.000+02:00", + "id": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "message": "Merge branch 'rename-readme' into 'master'\r\n\r\nRename README\r\n\r\n\r\n\r\nSee merge request !2", + "short_id": "a91957a8", + "title": "Merge branch 'rename-readme' into 'master'\r" + }, + "coverage": null, + "created_at": "2016-08-11T11:32:24.456Z", + "finished_at": "2016-08-11T11:32:35.145Z", + "id": 664, + "name": "deploy", + "ref": "master", + "runner": null, + "stage": "deploy", + "started_at": null, + "status": "success", + "tag": false, + "user": { + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "bio": null, + "created_at": "2016-08-11T07:09:20.351Z", + "id": 1, + "is_admin": true, + "linkedin": "", + "location": null, + "name": "Administrator", + "skype": "", + "state": "active", + "twitter": "", + "username": "root", + "web_url": "http://localhost:3000/root", + "website_url": "" + } + }, + "environment": { + "external_url": "https://about.gitlab.com", + "id": 9, + "name": "production" + }, + "id": 42, + "iid": 2, + "ref": "master", + "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "user": { + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "id": 1, + "name": "Administrator", + "state": "active", + "username": "root", + "web_url": "http://localhost:3000/root" + } + } +] +``` + +## Get a specific deployment + +``` +GET /projects/:id/deployments/:deployment_id +``` + +| Attribute | Type | Required | Description | +|-----------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a project | +| `deployment_id` | integer | yes | The ID of the deployment | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/deployments/1" +``` + +Example of response + +```json +{ + "id": 42, + "iid": 2, + "ref": "master", + "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "created_at": "2016-08-11T11:32:35.444Z", + "user": { + "name": "Administrator", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "environment": { + "id": 9, + "name": "production", + "external_url": "https://about.gitlab.com" + }, + "deployable": { + "id": 664, + "status": "success", + "stage": "deploy", + "name": "deploy", + "ref": "master", + "tag": false, + "coverage": null, + "created_at": "2016-08-11T11:32:24.456Z", + "started_at": null, + "finished_at": "2016-08-11T11:32:35.145Z", + "user": { + "name": "Administrator", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root", + "created_at": "2016-08-11T07:09:20.351Z", + "is_admin": true, + "bio": null, + "location": null, + "skype": "", + "linkedin": "", + "twitter": "", + "website_url": "" + }, + "commit": { + "id": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "short_id": "a91957a8", + "title": "Merge branch 'rename-readme' into 'master'\r", + "author_name": "Administrator", + "author_email": "admin@example.com", + "created_at": "2016-08-11T13:28:26.000+02:00", + "message": "Merge branch 'rename-readme' into 'master'\r\n\r\nRename README\r\n\r\n\r\n\r\nSee merge request !2" + }, + "runner": null + } +} +``` diff --git a/doc/api/enviroments.md b/doc/api/enviroments.md new file mode 100644 index 00000000000..87a5fa67124 --- /dev/null +++ b/doc/api/enviroments.md @@ -0,0 +1,117 @@ +# Environments + +## List environments + +Get all environments for a given project. + +``` +GET /projects/:id/environments +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | integer | yes | The ID of the project | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/1/environments +``` + +Example response: + +```json +[ + { + "id": 1, + "name": "Env1", + "external_url": "https://env1.example.gitlab.com" + } +] +``` + +## Create a new environment + +Creates a new environment with the given name and external_url. + +It returns 201 if the environment was successfully created, 400 for wrong parameters. + +``` +POST /projects/:id/environment +``` + +| Attribute | Type | Required | Description | +| ------------- | ------- | -------- | ---------------------------- | +| `id` | integer | yes | The ID of the project | +| `name` | string | yes | The name of the environment | +| `external_url` | string | no | Place to link to for this environment | + +```bash +curl --data "name=deploy&external_url=https://deploy.example.gitlab.com" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/environments" +``` + +Example response: + +```json +{ + "id": 1, + "name": "deploy", + "external_url": "https://deploy.example.gitlab.com" +} +``` + +## Edit an existing environment + +Updates an existing environment's name and/or external_url. + +It returns 200 if the environment was successfully updated. In case of an error, a status code 400 is returned. + +``` +PUT /projects/:id/environments/:environments_id +``` + +| Attribute | Type | Required | Description | +| --------------- | ------- | --------------------------------- | ------------------------------- | +| `id` | integer | yes | The ID of the project | +| `environment_id` | integer | yes | The ID of the environment | The ID of the environment | +| `name` | string | no | The new name of the environment | +| `external_url` | string | no | The new external_url | + +```bash +curl --request PUT --data "name=staging&external_url=https://staging.example.gitlab.com" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/environment/1" +``` + +Example response: + +```json +{ + "id": 1, + "name": "staging", + "external_url": "https://staging.example.gitlab.com" +} +``` + +## Delete an environment + +It returns 200 if the environment was successfully deleted, and 404 if the environment does not exist. + +``` +DELETE /projects/:id/environments/:environment_id +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | integer | yes | The ID of the project | +| `environment_id` | integer | yes | The ID of the environment | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/environment/1" +``` + +Example response: + +```json +{ + "id": 1, + "name": "deploy", + "external_url": "https://deploy.example.gitlab.com" +} +``` diff --git a/doc/api/groups.md b/doc/api/groups.md index 87480bebfc4..e81d6f9de4b 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -1,514 +1,444 @@ -# Groups
-
-## List groups
-
-Get a list of groups. (As user: my groups, as admin: all groups)
-
-```
-GET /groups
-```
-
-```json
-[
- {
- "id": 1,
- "name": "Foobar Group",
- "path": "foo-bar",
- "description": "An interesting group"
- }
-]
-```
-
-You can search for groups by name or path, see below.
-
-
-## List a group's projects
-
-Get a list of projects in this group.
-
-```
-GET /groups/:id/projects
-```
-
-Parameters:
-
-- `archived` (optional) - if passed, limit by archived status
-- `visibility` (optional) - if passed, limit by visibility `public`, `internal`, `private`
-- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at`
-- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc`
-- `search` (optional) - Return list of authorized projects according to a search criteria
-- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first
-
-```json
-[
- {
- "id": 9,
- "description": "foo",
- "default_branch": "master",
- "tag_list": [],
- "public": false,
- "archived": false,
- "visibility_level": 10,
- "ssh_url_to_repo": "git@gitlab.example.com/html5-boilerplate.git",
- "http_url_to_repo": "http://gitlab.example.com/h5bp/html5-boilerplate.git",
- "web_url": "http://gitlab.example.com/h5bp/html5-boilerplate",
- "name": "Html5 Boilerplate",
- "name_with_namespace": "Experimental / Html5 Boilerplate",
- "path": "html5-boilerplate",
- "path_with_namespace": "h5bp/html5-boilerplate",
- "issues_enabled": true,
- "merge_requests_enabled": true,
- "wiki_enabled": true,
- "builds_enabled": true,
- "snippets_enabled": true,
- "created_at": "2016-04-05T21:40:50.169Z",
- "last_activity_at": "2016-04-06T16:52:08.432Z",
- "shared_runners_enabled": true,
- "creator_id": 1,
- "namespace": {
- "id": 5,
- "name": "Experimental",
- "path": "h5bp",
- "owner_id": null,
- "created_at": "2016-04-05T21:40:49.152Z",
- "updated_at": "2016-04-07T08:07:48.466Z",
- "description": "foo",
- "avatar": {
- "url": null
- },
- "share_with_group_lock": false,
- "visibility_level": 10
- },
- "avatar_url": null,
- "star_count": 1,
- "forks_count": 0,
- "open_issues_count": 3,
- "public_builds": true,
- "shared_with_groups": []
- }
-]
-```
-
-## Details of a group
-
-Get all details of a group.
-
-```
-GET /groups/:id
-```
-
-Parameters:
-
-| Attribute | Type | Required | Description |
-| --------- | ---- | -------- | ----------- |
-| `id` | integer/string | yes | The ID or path of a group |
-
-```bash
-curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/4
-```
-
-Example response:
-
-```json
-{
- "id": 4,
- "name": "Twitter",
- "path": "twitter",
- "description": "Aliquid qui quis dignissimos distinctio ut commodi voluptas est.",
- "visibility_level": 20,
- "avatar_url": null,
- "web_url": "https://gitlab.example.com/groups/twitter",
- "projects": [
- {
- "id": 7,
- "description": "Voluptas veniam qui et beatae voluptas doloremque explicabo facilis.",
- "default_branch": "master",
- "tag_list": [],
- "public": true,
- "archived": false,
- "visibility_level": 20,
- "ssh_url_to_repo": "git@gitlab.example.com:twitter/typeahead-js.git",
- "http_url_to_repo": "https://gitlab.example.com/twitter/typeahead-js.git",
- "web_url": "https://gitlab.example.com/twitter/typeahead-js",
- "name": "Typeahead.Js",
- "name_with_namespace": "Twitter / Typeahead.Js",
- "path": "typeahead-js",
- "path_with_namespace": "twitter/typeahead-js",
- "issues_enabled": true,
- "merge_requests_enabled": true,
- "wiki_enabled": true,
- "builds_enabled": true,
- "snippets_enabled": false,
- "container_registry_enabled": true,
- "created_at": "2016-06-17T07:47:25.578Z",
- "last_activity_at": "2016-06-17T07:47:25.881Z",
- "shared_runners_enabled": true,
- "creator_id": 1,
- "namespace": {
- "id": 4,
- "name": "Twitter",
- "path": "twitter",
- "owner_id": null,
- "created_at": "2016-06-17T07:47:24.216Z",
- "updated_at": "2016-06-17T07:47:24.216Z",
- "description": "Aliquid qui quis dignissimos distinctio ut commodi voluptas est.",
- "avatar": {
- "url": null
- },
- "share_with_group_lock": false,
- "visibility_level": 20
- },
- "avatar_url": null,
- "star_count": 0,
- "forks_count": 0,
- "open_issues_count": 3,
- "public_builds": true,
- "shared_with_groups": []
- },
- {
- "id": 6,
- "description": "Aspernatur omnis repudiandae qui voluptatibus eaque.",
- "default_branch": "master",
- "tag_list": [],
- "public": false,
- "archived": false,
- "visibility_level": 10,
- "ssh_url_to_repo": "git@gitlab.example.com:twitter/flight.git",
- "http_url_to_repo": "https://gitlab.example.com/twitter/flight.git",
- "web_url": "https://gitlab.example.com/twitter/flight",
- "name": "Flight",
- "name_with_namespace": "Twitter / Flight",
- "path": "flight",
- "path_with_namespace": "twitter/flight",
- "issues_enabled": true,
- "merge_requests_enabled": true,
- "wiki_enabled": true,
- "builds_enabled": true,
- "snippets_enabled": false,
- "container_registry_enabled": true,
- "created_at": "2016-06-17T07:47:24.661Z",
- "last_activity_at": "2016-06-17T07:47:24.838Z",
- "shared_runners_enabled": true,
- "creator_id": 1,
- "namespace": {
- "id": 4,
- "name": "Twitter",
- "path": "twitter",
- "owner_id": null,
- "created_at": "2016-06-17T07:47:24.216Z",
- "updated_at": "2016-06-17T07:47:24.216Z",
- "description": "Aliquid qui quis dignissimos distinctio ut commodi voluptas est.",
- "avatar": {
- "url": null
- },
- "share_with_group_lock": false,
- "visibility_level": 20
- },
- "avatar_url": null,
- "star_count": 0,
- "forks_count": 0,
- "open_issues_count": 8,
- "public_builds": true,
- "shared_with_groups": []
- }
- ],
- "shared_projects": [
- {
- "id": 8,
- "description": "Velit eveniet provident fugiat saepe eligendi autem.",
- "default_branch": "master",
- "tag_list": [],
- "public": false,
- "archived": false,
- "visibility_level": 0,
- "ssh_url_to_repo": "git@gitlab.example.com:h5bp/html5-boilerplate.git",
- "http_url_to_repo": "https://gitlab.example.com/h5bp/html5-boilerplate.git",
- "web_url": "https://gitlab.example.com/h5bp/html5-boilerplate",
- "name": "Html5 Boilerplate",
- "name_with_namespace": "H5bp / Html5 Boilerplate",
- "path": "html5-boilerplate",
- "path_with_namespace": "h5bp/html5-boilerplate",
- "issues_enabled": true,
- "merge_requests_enabled": true,
- "wiki_enabled": true,
- "builds_enabled": true,
- "snippets_enabled": false,
- "container_registry_enabled": true,
- "created_at": "2016-06-17T07:47:27.089Z",
- "last_activity_at": "2016-06-17T07:47:27.310Z",
- "shared_runners_enabled": true,
- "creator_id": 1,
- "namespace": {
- "id": 5,
- "name": "H5bp",
- "path": "h5bp",
- "owner_id": null,
- "created_at": "2016-06-17T07:47:26.621Z",
- "updated_at": "2016-06-17T07:47:26.621Z",
- "description": "Id consequatur rem vel qui doloremque saepe.",
- "avatar": {
- "url": null
- },
- "share_with_group_lock": false,
- "visibility_level": 20
- },
- "avatar_url": null,
- "star_count": 0,
- "forks_count": 0,
- "open_issues_count": 4,
- "public_builds": true,
- "shared_with_groups": [
- {
- "group_id": 4,
- "group_name": "Twitter",
- "group_access_level": 30
- },
- {
- "group_id": 3,
- "group_name": "Gitlab Org",
- "group_access_level": 10
- }
- ]
- }
- ]
-}
-```
-
-## New group
-
-Creates a new project group. Available only for users who can create groups.
-
-```
-POST /groups
-```
-
-Parameters:
-
-- `name` (required) - The name of the group
-- `path` (required) - The path of the group
-- `description` (optional) - The group's description
-- `visibility_level` (optional) - The group's visibility. 0 for private, 10 for internal, 20 for public.
-
-## Transfer project to group
-
-Transfer a project to the Group namespace. Available only for admin
-
-```
-POST /groups/:id/projects/:project_id
-```
-
-Parameters:
-
-- `id` (required) - The ID or path of a group
-- `project_id` (required) - The ID of a project
-
-## Update group
-
-Updates the project group. Only available to group owners and administrators.
-
-```
-PUT /groups/:id
-```
-
-| Attribute | Type | Required | Description |
-| --------- | ---- | -------- | ----------- |
-| `id` | integer | yes | The ID of the group |
-| `name` | string | no | The name of the group |
-| `path` | string | no | The path of the group |
-| `description` | string | no | The description of the group |
-| `visibility_level` | integer | no | The visibility level of the group. 0 for private, 10 for internal, 20 for public. |
-
-```bash
-curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/groups/5?name=Experimental"
-
-```
-
-Example response:
-
-```json
-{
- "id": 5,
- "name": "Experimental",
- "path": "h5bp",
- "description": "foo",
- "visibility_level": 10,
- "avatar_url": null,
- "web_url": "http://gitlab.example.com/groups/h5bp",
- "projects": [
- {
- "id": 9,
- "description": "foo",
- "default_branch": "master",
- "tag_list": [],
- "public": false,
- "archived": false,
- "visibility_level": 10,
- "ssh_url_to_repo": "git@gitlab.example.com/html5-boilerplate.git",
- "http_url_to_repo": "http://gitlab.example.com/h5bp/html5-boilerplate.git",
- "web_url": "http://gitlab.example.com/h5bp/html5-boilerplate",
- "name": "Html5 Boilerplate",
- "name_with_namespace": "Experimental / Html5 Boilerplate",
- "path": "html5-boilerplate",
- "path_with_namespace": "h5bp/html5-boilerplate",
- "issues_enabled": true,
- "merge_requests_enabled": true,
- "wiki_enabled": true,
- "builds_enabled": true,
- "snippets_enabled": true,
- "created_at": "2016-04-05T21:40:50.169Z",
- "last_activity_at": "2016-04-06T16:52:08.432Z",
- "shared_runners_enabled": true,
- "creator_id": 1,
- "namespace": {
- "id": 5,
- "name": "Experimental",
- "path": "h5bp",
- "owner_id": null,
- "created_at": "2016-04-05T21:40:49.152Z",
- "updated_at": "2016-04-07T08:07:48.466Z",
- "description": "foo",
- "avatar": {
- "url": null
- },
- "share_with_group_lock": false,
- "visibility_level": 10
- },
- "avatar_url": null,
- "star_count": 1,
- "forks_count": 0,
- "open_issues_count": 3,
- "public_builds": true,
- "shared_with_groups": []
- }
- ]
-}
-```
-
-## Remove group
-
-Removes group with all projects inside.
-
-```
-DELETE /groups/:id
-```
-
-Parameters:
-
-- `id` (required) - The ID or path of a user group
-
-## Search for group
-
-Get all groups that match your string in their name or path.
-
-```
-GET /groups?search=foobar
-```
-
-```json
-[
- {
- "id": 1,
- "name": "Foobar Group",
- "path": "foo-bar",
- "description": "An interesting group"
- }
-]
-```
-
-## Group members
-
-**Group access levels**
-
-The group access levels are defined in the `Gitlab::Access` module. Currently, these levels are recognized:
-
-```
-GUEST = 10
-REPORTER = 20
-DEVELOPER = 30
-MASTER = 40
-OWNER = 50
-```
-
-### List group members
-
-Get a list of group members viewable by the authenticated user.
-
-```
-GET /groups/:id/members
-```
-
-```json
-[
- {
- "id": 1,
- "username": "raymond_smith",
- "name": "Raymond Smith",
- "state": "active",
- "created_at": "2012-10-22T14:13:35Z",
- "access_level": 30
- },
- {
- "id": 2,
- "username": "john_doe",
- "name": "John Doe",
- "state": "active",
- "created_at": "2012-10-22T14:13:35Z",
- "access_level": 30
- }
-]
-```
-
-### Add group member
-
-Adds a user to the list of group members.
-
-```
-POST /groups/:id/members
-```
-
-Parameters:
-
-- `id` (required) - The ID or path of a group
-- `user_id` (required) - The ID of a user to add
-- `access_level` (required) - Project access level
-
-### Edit group team member
-
-Updates a group team member to a specified access level.
-
-```
-PUT /groups/:id/members/:user_id
-```
-
-Parameters:
-
-- `id` (required) - The ID of a group
-- `user_id` (required) - The ID of a group member
-- `access_level` (required) - Project access level
-
-### Remove user team member
-
-Removes user from user team.
-
-```
-DELETE /groups/:id/members/:user_id
-```
-
-Parameters:
-
-- `id` (required) - The ID or path of a user group
-- `user_id` (required) - The ID of a group member
-
-## Namespaces in groups
-
-By default, groups only get 20 namespaces at a time because the API results are paginated.
-
-To get more (up to 100), pass the following as an argument to the API call:
-```
-/groups?per_page=100
-```
-
-And to switch pages add:
-```
-/groups?per_page=100&page=2
-```
+# Groups + +## List groups + +Get a list of groups. (As user: my groups, as admin: all groups) + +``` +GET /groups +``` + +```json +[ + { + "id": 1, + "name": "Foobar Group", + "path": "foo-bar", + "description": "An interesting group" + } +] +``` + +You can search for groups by name or path, see below. + + +## List a group's projects + +Get a list of projects in this group. + +``` +GET /groups/:id/projects +``` + +Parameters: + +- `archived` (optional) - if passed, limit by archived status +- `visibility` (optional) - if passed, limit by visibility `public`, `internal`, `private` +- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` +- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` +- `search` (optional) - Return list of authorized projects according to a search criteria +- `ci_enabled_first` - Return projects ordered by ci_enabled flag. Projects with enabled GitLab CI go first + +```json +[ + { + "id": 9, + "description": "foo", + "default_branch": "master", + "tag_list": [], + "public": false, + "archived": false, + "visibility_level": 10, + "ssh_url_to_repo": "git@gitlab.example.com/html5-boilerplate.git", + "http_url_to_repo": "http://gitlab.example.com/h5bp/html5-boilerplate.git", + "web_url": "http://gitlab.example.com/h5bp/html5-boilerplate", + "name": "Html5 Boilerplate", + "name_with_namespace": "Experimental / Html5 Boilerplate", + "path": "html5-boilerplate", + "path_with_namespace": "h5bp/html5-boilerplate", + "issues_enabled": true, + "merge_requests_enabled": true, + "wiki_enabled": true, + "builds_enabled": true, + "snippets_enabled": true, + "created_at": "2016-04-05T21:40:50.169Z", + "last_activity_at": "2016-04-06T16:52:08.432Z", + "shared_runners_enabled": true, + "creator_id": 1, + "namespace": { + "id": 5, + "name": "Experimental", + "path": "h5bp", + "owner_id": null, + "created_at": "2016-04-05T21:40:49.152Z", + "updated_at": "2016-04-07T08:07:48.466Z", + "description": "foo", + "avatar": { + "url": null + }, + "share_with_group_lock": false, + "visibility_level": 10 + }, + "avatar_url": null, + "star_count": 1, + "forks_count": 0, + "open_issues_count": 3, + "public_builds": true, + "shared_with_groups": [], + "request_access_enabled": false + } +] +``` + +## Details of a group + +Get all details of a group. + +``` +GET /groups/:id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or path of a group | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/4 +``` + +Example response: + +```json +{ + "id": 4, + "name": "Twitter", + "path": "twitter", + "description": "Aliquid qui quis dignissimos distinctio ut commodi voluptas est.", + "visibility_level": 20, + "avatar_url": null, + "web_url": "https://gitlab.example.com/groups/twitter", + "request_access_enabled": false, + "projects": [ + { + "id": 7, + "description": "Voluptas veniam qui et beatae voluptas doloremque explicabo facilis.", + "default_branch": "master", + "tag_list": [], + "public": true, + "archived": false, + "visibility_level": 20, + "ssh_url_to_repo": "git@gitlab.example.com:twitter/typeahead-js.git", + "http_url_to_repo": "https://gitlab.example.com/twitter/typeahead-js.git", + "web_url": "https://gitlab.example.com/twitter/typeahead-js", + "name": "Typeahead.Js", + "name_with_namespace": "Twitter / Typeahead.Js", + "path": "typeahead-js", + "path_with_namespace": "twitter/typeahead-js", + "issues_enabled": true, + "merge_requests_enabled": true, + "wiki_enabled": true, + "builds_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": true, + "created_at": "2016-06-17T07:47:25.578Z", + "last_activity_at": "2016-06-17T07:47:25.881Z", + "shared_runners_enabled": true, + "creator_id": 1, + "namespace": { + "id": 4, + "name": "Twitter", + "path": "twitter", + "owner_id": null, + "created_at": "2016-06-17T07:47:24.216Z", + "updated_at": "2016-06-17T07:47:24.216Z", + "description": "Aliquid qui quis dignissimos distinctio ut commodi voluptas est.", + "avatar": { + "url": null + }, + "share_with_group_lock": false, + "visibility_level": 20 + }, + "avatar_url": null, + "star_count": 0, + "forks_count": 0, + "open_issues_count": 3, + "public_builds": true, + "shared_with_groups": [], + "request_access_enabled": false + }, + { + "id": 6, + "description": "Aspernatur omnis repudiandae qui voluptatibus eaque.", + "default_branch": "master", + "tag_list": [], + "public": false, + "archived": false, + "visibility_level": 10, + "ssh_url_to_repo": "git@gitlab.example.com:twitter/flight.git", + "http_url_to_repo": "https://gitlab.example.com/twitter/flight.git", + "web_url": "https://gitlab.example.com/twitter/flight", + "name": "Flight", + "name_with_namespace": "Twitter / Flight", + "path": "flight", + "path_with_namespace": "twitter/flight", + "issues_enabled": true, + "merge_requests_enabled": true, + "wiki_enabled": true, + "builds_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": true, + "created_at": "2016-06-17T07:47:24.661Z", + "last_activity_at": "2016-06-17T07:47:24.838Z", + "shared_runners_enabled": true, + "creator_id": 1, + "namespace": { + "id": 4, + "name": "Twitter", + "path": "twitter", + "owner_id": null, + "created_at": "2016-06-17T07:47:24.216Z", + "updated_at": "2016-06-17T07:47:24.216Z", + "description": "Aliquid qui quis dignissimos distinctio ut commodi voluptas est.", + "avatar": { + "url": null + }, + "share_with_group_lock": false, + "visibility_level": 20 + }, + "avatar_url": null, + "star_count": 0, + "forks_count": 0, + "open_issues_count": 8, + "public_builds": true, + "shared_with_groups": [], + "request_access_enabled": false + } + ], + "shared_projects": [ + { + "id": 8, + "description": "Velit eveniet provident fugiat saepe eligendi autem.", + "default_branch": "master", + "tag_list": [], + "public": false, + "archived": false, + "visibility_level": 0, + "ssh_url_to_repo": "git@gitlab.example.com:h5bp/html5-boilerplate.git", + "http_url_to_repo": "https://gitlab.example.com/h5bp/html5-boilerplate.git", + "web_url": "https://gitlab.example.com/h5bp/html5-boilerplate", + "name": "Html5 Boilerplate", + "name_with_namespace": "H5bp / Html5 Boilerplate", + "path": "html5-boilerplate", + "path_with_namespace": "h5bp/html5-boilerplate", + "issues_enabled": true, + "merge_requests_enabled": true, + "wiki_enabled": true, + "builds_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": true, + "created_at": "2016-06-17T07:47:27.089Z", + "last_activity_at": "2016-06-17T07:47:27.310Z", + "shared_runners_enabled": true, + "creator_id": 1, + "namespace": { + "id": 5, + "name": "H5bp", + "path": "h5bp", + "owner_id": null, + "created_at": "2016-06-17T07:47:26.621Z", + "updated_at": "2016-06-17T07:47:26.621Z", + "description": "Id consequatur rem vel qui doloremque saepe.", + "avatar": { + "url": null + }, + "share_with_group_lock": false, + "visibility_level": 20 + }, + "avatar_url": null, + "star_count": 0, + "forks_count": 0, + "open_issues_count": 4, + "public_builds": true, + "shared_with_groups": [ + { + "group_id": 4, + "group_name": "Twitter", + "group_access_level": 30 + }, + { + "group_id": 3, + "group_name": "Gitlab Org", + "group_access_level": 10 + } + ] + } + ] +} +``` + +## New group + +Creates a new project group. Available only for users who can create groups. + +``` +POST /groups +``` + +Parameters: + +- `name` (required) - The name of the group +- `path` (required) - The path of the group +- `description` (optional) - The group's description +- `visibility_level` (optional) - The group's visibility. 0 for private, 10 for internal, 20 for public. +- `lfs_enabled` (optional) - Enable/disable Large File Storage (LFS) for the projects in this group +- `request_access_enabled` (optional) - Allow users to request member access. + +## Transfer project to group + +Transfer a project to the Group namespace. Available only for admin + +``` +POST /groups/:id/projects/:project_id +``` + +Parameters: + +- `id` (required) - The ID or path of a group +- `project_id` (required) - The ID of a project + +## Update group + +Updates the project group. Only available to group owners and administrators. + +``` +PUT /groups/:id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the group | +| `name` | string | no | The name of the group | +| `path` | string | no | The path of the group | +| `description` | string | no | The description of the group | +| `visibility_level` | integer | no | The visibility level of the group. 0 for private, 10 for internal, 20 for public. | +| `lfs_enabled` (optional) | boolean | no | Enable/disable Large File Storage (LFS) for the projects in this group | +| `request_access_enabled` | boolean | no | Allow users to request member access. | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/groups/5?name=Experimental" + +``` + +Example response: + +```json +{ + "id": 5, + "name": "Experimental", + "path": "h5bp", + "description": "foo", + "visibility_level": 10, + "avatar_url": null, + "web_url": "http://gitlab.example.com/groups/h5bp", + "request_access_enabled": false, + "projects": [ + { + "id": 9, + "description": "foo", + "default_branch": "master", + "tag_list": [], + "public": false, + "archived": false, + "visibility_level": 10, + "ssh_url_to_repo": "git@gitlab.example.com/html5-boilerplate.git", + "http_url_to_repo": "http://gitlab.example.com/h5bp/html5-boilerplate.git", + "web_url": "http://gitlab.example.com/h5bp/html5-boilerplate", + "name": "Html5 Boilerplate", + "name_with_namespace": "Experimental / Html5 Boilerplate", + "path": "html5-boilerplate", + "path_with_namespace": "h5bp/html5-boilerplate", + "issues_enabled": true, + "merge_requests_enabled": true, + "wiki_enabled": true, + "builds_enabled": true, + "snippets_enabled": true, + "created_at": "2016-04-05T21:40:50.169Z", + "last_activity_at": "2016-04-06T16:52:08.432Z", + "shared_runners_enabled": true, + "creator_id": 1, + "namespace": { + "id": 5, + "name": "Experimental", + "path": "h5bp", + "owner_id": null, + "created_at": "2016-04-05T21:40:49.152Z", + "updated_at": "2016-04-07T08:07:48.466Z", + "description": "foo", + "avatar": { + "url": null + }, + "share_with_group_lock": false, + "visibility_level": 10 + }, + "avatar_url": null, + "star_count": 1, + "forks_count": 0, + "open_issues_count": 3, + "public_builds": true, + "shared_with_groups": [], + "request_access_enabled": false + } + ] +} +``` + +## Remove group + +Removes group with all projects inside. + +``` +DELETE /groups/:id +``` + +Parameters: + +- `id` (required) - The ID or path of a user group + +## Search for group + +Get all groups that match your string in their name or path. + +``` +GET /groups?search=foobar +``` + +```json +[ + { + "id": 1, + "name": "Foobar Group", + "path": "foo-bar", + "description": "An interesting group" + } +] +``` + +## Group members + +Please consult the [Group Members](members.md) documentation. + +## Namespaces in groups + +By default, groups only get 20 namespaces at a time because the API results are paginated. + +To get more (up to 100), pass the following as an argument to the API call: +``` +/groups?per_page=100 +``` + +And to switch pages add: +``` +/groups?per_page=100&page=2 +``` diff --git a/doc/api/issues.md b/doc/api/issues.md index 3ced787b23e..134263d27b4 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -33,7 +33,7 @@ GET /issues?labels=foo,bar&state=opened | `sort` | string | no | Return requests sorted in `asc` or `desc` order. Default is `desc` | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/issues +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/issues ``` Example response: @@ -46,7 +46,7 @@ Example response: "author" : { "state" : "active", "id" : 18, - "web_url" : "https://gitlab.example.com/u/eileen.lowe", + "web_url" : "https://gitlab.example.com/eileen.lowe", "name" : "Alexandra Bashirian", "avatar_url" : null, "username" : "eileen.lowe" @@ -67,7 +67,7 @@ Example response: "state" : "active", "id" : 1, "name" : "Administrator", - "web_url" : "https://gitlab.example.com/u/root", + "web_url" : "https://gitlab.example.com/root", "avatar_url" : null, "username" : "root" }, @@ -78,7 +78,10 @@ Example response: "iid" : 6, "labels" : [], "subscribed" : false, - "user_notes_count": 1 + "user_notes_count": 1, + "due_date": "2016-07-22", + "web_url": "http://example.com/example/example/issues/6", + "confidential": false } ] ``` @@ -109,7 +112,7 @@ GET /groups/:id/issues?milestone=1.0.0&state=opened ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/4/issues +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/4/issues ``` Example response: @@ -131,7 +134,7 @@ Example response: }, "author" : { "state" : "active", - "web_url" : "https://gitlab.example.com/u/root", + "web_url" : "https://gitlab.example.com/root", "avatar_url" : null, "username" : "root", "id" : 1, @@ -142,7 +145,7 @@ Example response: "iid" : 1, "assignee" : { "avatar_url" : null, - "web_url" : "https://gitlab.example.com/u/lennie", + "web_url" : "https://gitlab.example.com/lennie", "state" : "active", "username" : "lennie", "id" : 9, @@ -154,7 +157,10 @@ Example response: "updated_at" : "2016-01-04T15:31:46.176Z", "created_at" : "2016-01-04T15:31:46.176Z", "subscribed" : false, - "user_notes_count": 1 + "user_notes_count": 1, + "due_date": null, + "web_url": "http://example.com/example/example/issues/1", + "confidential": false } ] ``` @@ -187,7 +193,7 @@ GET /projects/:id/issues?iid=42 ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues ``` Example response: @@ -209,7 +215,7 @@ Example response: }, "author" : { "state" : "active", - "web_url" : "https://gitlab.example.com/u/root", + "web_url" : "https://gitlab.example.com/root", "avatar_url" : null, "username" : "root", "id" : 1, @@ -220,7 +226,7 @@ Example response: "iid" : 1, "assignee" : { "avatar_url" : null, - "web_url" : "https://gitlab.example.com/u/lennie", + "web_url" : "https://gitlab.example.com/lennie", "state" : "active", "username" : "lennie", "id" : 9, @@ -232,7 +238,10 @@ Example response: "updated_at" : "2016-01-04T15:31:46.176Z", "created_at" : "2016-01-04T15:31:46.176Z", "subscribed" : false, - "user_notes_count": 1 + "user_notes_count": 1, + "due_date": "2016-07-22", + "web_url": "http://example.com/example/example/issues/1", + "confidential": false } ] ``` @@ -251,7 +260,7 @@ GET /projects/:id/issues/:issue_id | `issue_id`| integer | yes | The ID of a project's issue | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/41 +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/41 ``` Example response: @@ -272,7 +281,7 @@ Example response: }, "author" : { "state" : "active", - "web_url" : "https://gitlab.example.com/u/root", + "web_url" : "https://gitlab.example.com/root", "avatar_url" : null, "username" : "root", "id" : 1, @@ -283,7 +292,7 @@ Example response: "iid" : 1, "assignee" : { "avatar_url" : null, - "web_url" : "https://gitlab.example.com/u/lennie", + "web_url" : "https://gitlab.example.com/lennie", "state" : "active", "username" : "lennie", "id" : 9, @@ -295,7 +304,10 @@ Example response: "updated_at" : "2016-01-04T15:31:46.176Z", "created_at" : "2016-01-04T15:31:46.176Z", "subscribed": false, - "user_notes_count": 1 + "user_notes_count": 1, + "due_date": null, + "web_url": "http://example.com/example/example/issues/1", + "confidential": false } ``` @@ -316,13 +328,15 @@ POST /projects/:id/issues | `id` | integer | yes | The ID of a project | | `title` | string | yes | The title of an issue | | `description` | string | no | The description of an issue | +| `confidential` | boolean | no | Set an issue to be confidential. Default is `false`. | | `assignee_id` | integer | no | The ID of a user to assign issue | | `milestone_id` | integer | no | The ID of a milestone to assign issue | | `labels` | string | no | Comma-separated label names for an issue | -| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` (requires admin or project owner rights) | +| `due_date` | string | no | Date time string in the format YEAR-MONTH-DAY, e.g. `2016-03-11` | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues?title=Issues%20with%20auth&labels=bug +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues?title=Issues%20with%20auth&labels=bug ``` Example response: @@ -343,7 +357,7 @@ Example response: "name" : "Alexandra Bashirian", "avatar_url" : null, "state" : "active", - "web_url" : "https://gitlab.example.com/u/eileen.lowe", + "web_url" : "https://gitlab.example.com/eileen.lowe", "id" : 18, "username" : "eileen.lowe" }, @@ -351,7 +365,10 @@ Example response: "updated_at" : "2016-01-07T12:44:33.959Z", "milestone" : null, "subscribed" : true, - "user_notes_count": 0 + "user_notes_count": 0, + "due_date": null, + "web_url": "http://example.com/example/example/issues/14", + "confidential": false } ``` @@ -374,14 +391,16 @@ PUT /projects/:id/issues/:issue_id | `issue_id` | integer | yes | The ID of a project's issue | | `title` | string | no | The title of an issue | | `description` | string | no | The description of an issue | +| `confidential` | boolean | no | Updates an issue to be confidential | | `assignee_id` | integer | no | The ID of a user to assign the issue to | | `milestone_id` | integer | no | The ID of a milestone to assign the issue to | | `labels` | string | no | Comma-separated label names for an issue | | `state_event` | string | no | The state event of an issue. Set `close` to close the issue and `reopen` to reopen it | -| `updated_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` | +| `updated_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` (requires admin or project owner rights) | +| `due_date` | string | no | Date time string in the format YEAR-MONTH-DAY, e.g. `2016-03-11` | ```bash -curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85?state_event=close +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85?state_event=close ``` Example response: @@ -395,7 +414,7 @@ Example response: "username" : "eileen.lowe", "id" : 18, "state" : "active", - "web_url" : "https://gitlab.example.com/u/eileen.lowe" + "web_url" : "https://gitlab.example.com/eileen.lowe" }, "state" : "closed", "title" : "Issues with auth", @@ -410,7 +429,10 @@ Example response: "assignee" : null, "milestone" : null, "subscribed" : true, - "user_notes_count": 0 + "user_notes_count": 0, + "due_date": "2016-07-22", + "web_url": "http://example.com/example/example/issues/15", + "confidential": false } ``` @@ -430,7 +452,7 @@ DELETE /projects/:id/issues/:issue_id | `issue_id` | integer | yes | The ID of a project's issue | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85 ``` ## Move an issue @@ -455,7 +477,7 @@ POST /projects/:id/issues/:issue_id/move | `to_project_id` | integer | yes | The ID of the new project | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85/move +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues/85/move ``` Example response: @@ -478,7 +500,7 @@ Example response: "id": 12, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/46f6f7dc858ada7be1853f7fb96e81da?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/axel.block" + "web_url": "https://gitlab.example.com/axel.block" }, "author": { "name": "Kris Steuber", @@ -486,8 +508,11 @@ Example response: "id": 10, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/7a190fecbaa68212a4b68aeb6e3acd10?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/solon.cremin" - } + "web_url": "https://gitlab.example.com/solon.cremin" + }, + "due_date": null, + "web_url": "http://example.com/example/example/issues/11", + "confidential": false } ``` @@ -509,7 +534,7 @@ POST /projects/:id/issues/:issue_id/subscription | `issue_id` | integer | yes | The ID of a project's issue | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/subscription +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/subscription ``` Example response: @@ -532,7 +557,7 @@ Example response: "id": 12, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/46f6f7dc858ada7be1853f7fb96e81da?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/axel.block" + "web_url": "https://gitlab.example.com/axel.block" }, "author": { "name": "Kris Steuber", @@ -540,8 +565,11 @@ Example response: "id": 10, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/7a190fecbaa68212a4b68aeb6e3acd10?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/solon.cremin" - } + "web_url": "https://gitlab.example.com/solon.cremin" + }, + "due_date": null, + "web_url": "http://example.com/example/example/issues/11", + "confidential": false } ``` @@ -563,7 +591,7 @@ DELETE /projects/:id/issues/:issue_id/subscription | `issue_id` | integer | yes | The ID of a project's issue | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/subscription +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/subscription ``` Example response: @@ -586,7 +614,7 @@ Example response: "id": 21, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/3e6f06a86cf27fa8b56f3f74f7615987?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/keyon" + "web_url": "https://gitlab.example.com/keyon" }, "author": { "name": "Vivian Hermann", @@ -594,9 +622,12 @@ Example response: "id": 11, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/orville" + "web_url": "https://gitlab.example.com/orville" }, - "subscribed": false + "subscribed": false, + "due_date": null, + "web_url": "http://example.com/example/example/issues/12", + "confidential": false } ``` @@ -617,7 +648,7 @@ POST /projects/:id/issues/:issue_id/todo | `issue_id` | integer | yes | The ID of a project's issue | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/todo +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/93/todo ``` Example response: @@ -638,7 +669,7 @@ Example response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" + "web_url": "https://gitlab.example.com/root" }, "action_name": "marked", "target_type": "Issue", @@ -669,7 +700,7 @@ Example response: "id": 14, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a7fa515d53450023c83d62986d0658a8?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/francisca" + "web_url": "https://gitlab.example.com/francisca" }, "author": { "name": "Maxie Medhurst", @@ -677,12 +708,15 @@ Example response: "id": 12, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" + "web_url": "https://gitlab.example.com/craig_rutherford" }, "subscribed": true, "user_notes_count": 7, "upvotes": 0, - "downvotes": 0 + "downvotes": 0, + "due_date": null, + "web_url": "http://example.com/example/example/issues/110", + "confidential": false }, "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ci/issues/10", "body": "Vel voluptas atque dicta mollitia adipisci qui at.", diff --git a/doc/api/keys.md b/doc/api/keys.md index faa6f212b43..b68f08a007d 100644 --- a/doc/api/keys.md +++ b/doc/api/keys.md @@ -24,7 +24,7 @@ Parameters: "id": 25, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/cfa35b8cd2ec278026357769582fa563?s=40\u0026d=identicon", - "web_url": "http://localhost:3000/u/john_smith", + "web_url": "http://localhost:3000/john_smith", "created_at": "2015-09-03T07:24:01.670Z", "is_admin": false, "bio": null, diff --git a/doc/api/labels.md b/doc/api/labels.md index a181c0f57a2..656232cc940 100644 --- a/doc/api/labels.md +++ b/doc/api/labels.md @@ -13,7 +13,7 @@ GET /projects/:id/labels | `id` | integer | yes | The ID of the project | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/1/labels +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/1/labels ``` Example response: @@ -82,7 +82,7 @@ POST /projects/:id/labels | `description` | string | no | The description of the label | ```bash -curl --data "name=feature&color=#5843AD" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels" +curl --data "name=feature&color=#5843AD" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels" ``` Example response: @@ -113,7 +113,7 @@ DELETE /projects/:id/labels | `name` | string | yes | The name of the label | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels?name=bug" +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels?name=bug" ``` Example response: @@ -148,12 +148,12 @@ PUT /projects/:id/labels | --------------- | ------- | --------------------------------- | ------------------------------- | | `id` | integer | yes | The ID of the project | | `name` | string | yes | The name of the existing label | -| `new_name` | string | yes if `color` if not provided | The new name of the label | +| `new_name` | string | yes if `color` is not provided | The new name of the label | | `color` | string | yes if `new_name` is not provided | The new color of the label in 6-digit hex notation with leading `#` sign | | `description` | string | no | The new description of the label | ```bash -curl -X PUT --data "name=documentation&new_name=docs&color=#8E44AD&description=Documentation" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels" +curl --request PUT --data "name=documentation&new_name=docs&color=#8E44AD&description=Documentation" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels" ``` Example response: @@ -184,7 +184,7 @@ POST /projects/:id/labels/:label_id/subscription | `label_id` | integer or string | yes | The ID or title of a project's label | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/labels/1/subscription +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/labels/1/subscription ``` Example response: @@ -219,7 +219,7 @@ DELETE /projects/:id/labels/:label_id/subscription | `label_id` | integer or string | yes | The ID or title of a project's label | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/labels/1/subscription +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/labels/1/subscription ``` Example response: diff --git a/doc/api/members.md b/doc/api/members.md new file mode 100644 index 00000000000..6535e9a7801 --- /dev/null +++ b/doc/api/members.md @@ -0,0 +1,185 @@ +# Group and project members + +**Valid access levels** + +The access levels are defined in the `Gitlab::Access` module. Currently, these levels are recognized: + +``` +10 => Guest access +20 => Reporter access +30 => Developer access +40 => Master access +50 => Owner access # Only valid for groups +``` + +## List all members of a group or project + +Gets a list of group or project members viewable by the authenticated user. + +Returns `200` if the request succeeds. + +``` +GET /groups/:id/members +GET /projects/:id/members +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `query` | string | no | A query string to search for members | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/members +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/members +``` + +Example response: + +```json +[ + { + "id": 1, + "username": "raymond_smith", + "name": "Raymond Smith", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "access_level": 30 + }, + { + "id": 2, + "username": "john_doe", + "name": "John Doe", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "access_level": 30 + } +] +``` + +## Get a member of a group or project + +Gets a member of a group or project. + +Returns `200` if the request succeeds. + +``` +GET /groups/:id/members/:user_id +GET /projects/:id/members/:user_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `user_id` | integer | yes | The user ID of the member | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/members/:user_id +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/members/:user_id +``` + +Example response: + +```json +{ + "id": 1, + "username": "raymond_smith", + "name": "Raymond Smith", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "access_level": 30, + "expires_at": null +} +``` + +## Add a member to a group or project + +Adds a member to a group or project. + +Returns `201` if the request succeeds. + +``` +POST /groups/:id/members +POST /projects/:id/members +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `user_id` | integer | yes | The user ID of the new member | +| `access_level` | integer | yes | A valid access level | +| `expires_at` | string | no | A date string in the format YEAR-MONTH-DAY | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "user_id=1&access_level=30" https://gitlab.example.com/api/v3/groups/:id/members +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "user_id=1&access_level=30" https://gitlab.example.com/api/v3/projects/:id/members +``` + +Example response: + +```json +{ + "id": 1, + "username": "raymond_smith", + "name": "Raymond Smith", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "access_level": 30 +} +``` + +## Edit a member of a group or project + +Updates a member of a group or project. + +Returns `200` if the request succeeds. + +``` +PUT /groups/:id/members/:user_id +PUT /projects/:id/members/:user_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `user_id` | integer | yes | The user ID of the member | +| `access_level` | integer | yes | A valid access level | +| `expires_at` | string | no | A date string in the format YEAR-MONTH-DAY | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/members/:user_id?access_level=40 +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/members/:user_id?access_level=40 +``` + +Example response: + +```json +{ + "id": 1, + "username": "raymond_smith", + "name": "Raymond Smith", + "state": "active", + "created_at": "2012-10-22T14:13:35Z", + "access_level": 40 +} +``` + +## Remove a member from a group or project + +Removes a user from a group or project. + +Returns `200` if the request succeeds. + +``` +DELETE /groups/:id/members/:user_id +DELETE /projects/:id/members/:user_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `user_id` | integer | yes | The user ID of the member | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/:id/members/:user_id +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/:id/members/:user_id +``` diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 816f09e1007..f4167403c2c 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -68,7 +68,12 @@ Parameters: "merge_when_build_succeeds": true, "merge_status": "can_be_merged", "subscribed" : false, - "user_notes_count": 1 + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, + "user_notes_count": 1, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" } ] ``` @@ -132,7 +137,12 @@ Parameters: "merge_when_build_succeeds": true, "merge_status": "can_be_merged", "subscribed" : true, - "user_notes_count": 1 + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": "9999999999999999999999999999999999999999", + "user_notes_count": 1, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" } ``` @@ -232,7 +242,12 @@ Parameters: "merge_when_build_succeeds": true, "merge_status": "can_be_merged", "subscribed" : true, + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, "user_notes_count": 1, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1", "changes": [ { "old_path": "VERSION", @@ -270,6 +285,7 @@ Parameters: ```json { "id": 1, + "iid": 1, "target_branch": "master", "source_branch": "test1", "project_id": 3, @@ -312,7 +328,12 @@ Parameters: "merge_when_build_succeeds": true, "merge_status": "can_be_merged", "subscribed" : true, - "user_notes_count": 0 + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, + "user_notes_count": 0, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" } ``` @@ -342,6 +363,7 @@ Parameters: ```json { "id": 1, + "iid": 1, "target_branch": "master", "project_id": 3, "title": "test1", @@ -383,7 +405,12 @@ Parameters: "merge_when_build_succeeds": true, "merge_status": "can_be_merged", "subscribed" : true, - "user_notes_count": 1 + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, + "user_notes_count": 1, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" } ``` @@ -406,7 +433,7 @@ DELETE /projects/:id/merge_requests/:merge_request_id | `merge_request_id` | integer | yes | The ID of a project's merge request | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/merge_request/85 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/merge_request/85 ``` ## Accept MR @@ -439,6 +466,7 @@ Parameters: ```json { "id": 1, + "iid": 1, "target_branch": "master", "source_branch": "test1", "project_id": 3, @@ -481,7 +509,12 @@ Parameters: "merge_when_build_succeeds": true, "merge_status": "can_be_merged", "subscribed" : true, - "user_notes_count": 1 + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": "9999999999999999999999999999999999999999", + "user_notes_count": 1, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" } ``` @@ -505,6 +538,7 @@ Parameters: ```json { "id": 1, + "iid": 1, "target_branch": "master", "source_branch": "test1", "project_id": 3, @@ -547,7 +581,12 @@ Parameters: "merge_when_build_succeeds": true, "merge_status": "can_be_merged", "subscribed" : true, - "user_notes_count": 1 + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, + "user_notes_count": 1, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" } ``` @@ -569,7 +608,7 @@ GET /projects/:id/merge_requests/:merge_request_id/closes_issues | `merge_request_id` | integer | yes | The ID of the merge request | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/76/merge_requests/1/closes_issues +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/76/merge_requests/1/closes_issues ``` Example response when the GitLab issue tracker is used: @@ -582,7 +621,7 @@ Example response when the GitLab issue tracker is used: "author" : { "state" : "active", "id" : 18, - "web_url" : "https://gitlab.example.com/u/eileen.lowe", + "web_url" : "https://gitlab.example.com/eileen.lowe", "name" : "Alexandra Bashirian", "avatar_url" : null, "username" : "eileen.lowe" @@ -603,7 +642,7 @@ Example response when the GitLab issue tracker is used: "state" : "active", "id" : 1, "name" : "Administrator", - "web_url" : "https://gitlab.example.com/u/root", + "web_url" : "https://gitlab.example.com/root", "avatar_url" : null, "username" : "root" }, @@ -647,7 +686,7 @@ POST /projects/:id/merge_requests/:merge_request_id/subscription | `merge_request_id` | integer | yes | The ID of the merge request | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/17/subscription +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/17/subscription ``` Example response: @@ -672,7 +711,7 @@ Example response: "id": 19, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/39ce4a2822cc896933ffbd68c1470e55?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/leila" + "web_url": "https://gitlab.example.com/leila" }, "assignee": { "name": "Celine Wehner", @@ -680,7 +719,7 @@ Example response: "id": 16, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/f4cd5605b769dd2ce405a27c6e6f2684?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/carli" + "web_url": "https://gitlab.example.com/carli" }, "source_project_id": 5, "target_project_id": 5, @@ -699,7 +738,9 @@ Example response: }, "merge_when_build_succeeds": false, "merge_status": "cannot_be_merged", - "subscribed": true + "subscribed": true, + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null } ``` @@ -721,7 +762,7 @@ DELETE /projects/:id/merge_requests/:merge_request_id/subscription | `merge_request_id` | integer | yes | The ID of the merge request | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/17/subscription +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/17/subscription ``` Example response: @@ -746,7 +787,7 @@ Example response: "id": 19, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/39ce4a2822cc896933ffbd68c1470e55?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/leila" + "web_url": "https://gitlab.example.com/leila" }, "assignee": { "name": "Celine Wehner", @@ -754,7 +795,7 @@ Example response: "id": 16, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/f4cd5605b769dd2ce405a27c6e6f2684?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/carli" + "web_url": "https://gitlab.example.com/carli" }, "source_project_id": 5, "target_project_id": 5, @@ -773,7 +814,9 @@ Example response: }, "merge_when_build_succeeds": false, "merge_status": "cannot_be_merged", - "subscribed": false + "subscribed": false, + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null } ``` @@ -794,7 +837,7 @@ POST /projects/:id/merge_requests/:merge_request_id/todo | `merge_request_id` | integer | yes | The ID of the merge request | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/27/todo +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/27/todo ``` Example response: @@ -815,7 +858,7 @@ Example response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" + "web_url": "https://gitlab.example.com/root" }, "action_name": "marked", "target_type": "MergeRequest", @@ -838,7 +881,7 @@ Example response: "id": 14, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a7fa515d53450023c83d62986d0658a8?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/francisca" + "web_url": "https://gitlab.example.com/francisca" }, "assignee": { "name": "Dr. Gabrielle Strosin", @@ -846,7 +889,7 @@ Example response: "id": 4, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/733005fcd7e6df12d2d8580171ccb966?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/barrett.krajcik" + "web_url": "https://gitlab.example.com/barrett.krajcik" }, "source_project_id": 3, "target_project_id": 3, @@ -866,7 +909,12 @@ Example response: "merge_when_build_succeeds": false, "merge_status": "unchecked", "subscribed": true, - "user_notes_count": 7 + "sha": "8888888888888888888888888888888888888888", + "merge_commit_sha": null, + "user_notes_count": 7, + "should_remove_source_branch": true, + "force_remove_source_branch": false, + "web_url": "http://example.com/example/example/merge_requests/1" }, "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ci/merge_requests/7", "body": "Et voluptas laudantium minus nihil recusandae ut accusamus earum aut non.", @@ -874,3 +922,112 @@ Example response: "created_at": "2016-07-01T11:14:15.530Z" } ``` + +## Get MR diff versions + +Get a list of merge request diff versions. + +``` +GET /projects/:id/merge_requests/:merge_request_id/versions +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | String | yes | The ID of the project | +| `merge_request_id` | integer | yes | The ID of the merge request | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/1/merge_requests/1/versions +``` + +Example response: + +```json +[{ + "id": 110, + "head_commit_sha": "33e2ee8579fda5bc36accc9c6fbd0b4fefda9e30", + "base_commit_sha": "eeb57dffe83deb686a60a71c16c32f71046868fd", + "start_commit_sha": "eeb57dffe83deb686a60a71c16c32f71046868fd", + "created_at": "2016-07-26T14:44:48.926Z", + "merge_request_id": 105, + "state": "collected", + "real_size": "1" +}, { + "id": 108, + "head_commit_sha": "3eed087b29835c48015768f839d76e5ea8f07a24", + "base_commit_sha": "eeb57dffe83deb686a60a71c16c32f71046868fd", + "start_commit_sha": "eeb57dffe83deb686a60a71c16c32f71046868fd", + "created_at": "2016-07-25T14:21:33.028Z", + "merge_request_id": 105, + "state": "collected", + "real_size": "1" +}] +``` + +## Get a single MR diff version + +Get a single merge request diff version. + +``` +GET /projects/:id/merge_requests/:merge_request_id/versions/:version_id +``` + +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | String | yes | The ID of the project | +| `merge_request_id` | integer | yes | The ID of the merge request | +| `version_id` | integer | yes | The ID of the merge request diff version | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/1/merge_requests/1/versions/1 +``` + +Example response: + +```json +{ + "id": 110, + "head_commit_sha": "33e2ee8579fda5bc36accc9c6fbd0b4fefda9e30", + "base_commit_sha": "eeb57dffe83deb686a60a71c16c32f71046868fd", + "start_commit_sha": "eeb57dffe83deb686a60a71c16c32f71046868fd", + "created_at": "2016-07-26T14:44:48.926Z", + "merge_request_id": 105, + "state": "collected", + "real_size": "1", + "commits": [{ + "id": "33e2ee8579fda5bc36accc9c6fbd0b4fefda9e30", + "short_id": "33e2ee85", + "title": "Change year to 2018", + "author_name": "Administrator", + "author_email": "admin@example.com", + "created_at": "2016-07-26T17:44:29.000+03:00", + "message": "Change year to 2018" + }, { + "id": "aa24655de48b36335556ac8a3cd8bb521f977cbd", + "short_id": "aa24655d", + "title": "Update LICENSE", + "author_name": "Administrator", + "author_email": "admin@example.com", + "created_at": "2016-07-25T17:21:53.000+03:00", + "message": "Update LICENSE" + }, { + "id": "3eed087b29835c48015768f839d76e5ea8f07a24", + "short_id": "3eed087b", + "title": "Add license", + "author_name": "Administrator", + "author_email": "admin@example.com", + "created_at": "2016-07-25T17:21:20.000+03:00", + "message": "Add license" + }], + "diffs": [{ + "old_path": "LICENSE", + "new_path": "LICENSE", + "a_mode": "0", + "b_mode": "100644", + "diff": "--- /dev/null\n+++ b/LICENSE\n@@ -0,0 +1,21 @@\n+The MIT License (MIT)\n+\n+Copyright (c) 2018 Administrator\n+\n+Permission is hereby granted, free of charge, to any person obtaining a copy\n+of this software and associated documentation files (the \"Software\"), to deal\n+in the Software without restriction, including without limitation the rights\n+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n+copies of the Software, and to permit persons to whom the Software is\n+furnished to do so, subject to the following conditions:\n+\n+The above copyright notice and this permission notice shall be included in all\n+copies or substantial portions of the Software.\n+\n+THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n+SOFTWARE.\n", + "new_file": true, + "renamed_file": false, + "deleted_file": false + }] +} +``` diff --git a/doc/api/milestones.md b/doc/api/milestones.md index e4202025f80..ae7d22a4be5 100644 --- a/doc/api/milestones.md +++ b/doc/api/milestones.md @@ -20,7 +20,7 @@ Parameters: | `state` | string | optional | Return only `active` or `closed` milestones` | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/milestones +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/milestones ``` Example Response: diff --git a/doc/api/namespaces.md b/doc/api/namespaces.md index 42d9ce3d391..88cd407d792 100644 --- a/doc/api/namespaces.md +++ b/doc/api/namespaces.md @@ -19,7 +19,7 @@ GET /namespaces Example request: ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/namespaces +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/namespaces ``` Example response: @@ -54,7 +54,7 @@ GET /namespaces?search=foobar Example request: ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/namespaces?search=twitter +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/namespaces?search=twitter ``` Example response: diff --git a/doc/api/notes.md b/doc/api/notes.md index 7aa1c2155bf..58d40eecf3e 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -78,7 +78,8 @@ Parameters: ### Create new issue note -Creates a new note to a single project issue. +Creates a new note to a single project issue. If you create a note where the body +only contains an Award Emoji, you'll receive this object back. ``` POST /projects/:id/issues/:issue_id/notes @@ -124,7 +125,7 @@ Parameters: | `note_id` | integer | yes | The ID of a note | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/11/notes/636 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/11/notes/636 ``` Example Response: @@ -142,7 +143,7 @@ Example Response: "state": "active", "created_at": "2013-09-30T13:46:01Z", "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/pipin" + "web_url": "https://gitlab.example.com/pipin" }, "created_at": "2016-04-05T22:10:44.164Z", "system": false, @@ -204,6 +205,7 @@ Parameters: ### Create new snippet note Creates a new note for a single snippet. Snippet notes are comments users can post to a snippet. +If you create a note where the body only contains an Award Emoji, you'll receive this object back. ``` POST /projects/:id/snippets/:snippet_id/notes @@ -248,7 +250,7 @@ Parameters: | `note_id` | integer | yes | The ID of a note | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/snippets/52/notes/1659 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/snippets/52/notes/1659 ``` Example Response: @@ -266,7 +268,7 @@ Example Response: "state": "active", "created_at": "2013-09-30T13:46:01Z", "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/pipin" + "web_url": "https://gitlab.example.com/pipin" }, "created_at": "2016-04-06T16:51:53.239Z", "system": false, @@ -332,6 +334,8 @@ Parameters: ### Create new merge request note Creates a new note for a single merge request. +If you create a note where the body only contains an Award Emoji, you'll receive +this object back. ``` POST /projects/:id/merge_requests/:merge_request_id/notes @@ -376,7 +380,7 @@ Parameters: | `note_id` | integer | yes | The ID of a note | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/7/notes/1602 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/7/notes/1602 ``` Example Response: @@ -394,7 +398,7 @@ Example Response: "state": "active", "created_at": "2013-09-30T13:46:01Z", "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/pipin" + "web_url": "https://gitlab.example.com/pipin" }, "created_at": "2016-04-05T22:11:59.923Z", "system": false, diff --git a/doc/api/notification_settings.md b/doc/api/notification_settings.md new file mode 100644 index 00000000000..ff6c9e4931c --- /dev/null +++ b/doc/api/notification_settings.md @@ -0,0 +1,169 @@ +# Notification settings + +>**Note:** This feature was [introduced][ce-5632] in GitLab 8.12. + +**Valid notification levels** + +The notification levels are defined in the `NotificationSetting::level` model enumeration. Currently, these levels are recognized: + +``` +disabled +participating +watch +global +mention +custom +``` + +If the `custom` level is used, specific email events can be controlled. Notification email events are defined in the `NotificationSetting::EMAIL_EVENTS` model variable. Currently, these events are recognized: + +``` +new_note +new_issue +reopen_issue +close_issue +reassign_issue +new_merge_request +reopen_merge_request +close_merge_request +reassign_merge_request +merge_merge_request +``` + +## Global notification settings + +Get current notification settings and email address. + +``` +GET /notification_settings +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/notification_settings +``` + +Example response: + +```json +{ + "level": "participating", + "notification_email": "admin@example.com" +} +``` + +## Update global notification settings + +Update current notification settings and email address. + +``` +PUT /notification_settings +``` + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/notification_settings?level=watch +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `level` | string | no | The global notification level | +| `notification_email` | string | no | The email address to send notifications | +| `new_note` | boolean | no | Enable/disable this notification | +| `new_issue` | boolean | no | Enable/disable this notification | +| `reopen_issue` | boolean | no | Enable/disable this notification | +| `close_issue` | boolean | no | Enable/disable this notification | +| `reassign_issue` | boolean | no | Enable/disable this notification | +| `new_merge_request` | boolean | no | Enable/disable this notification | +| `reopen_merge_request` | boolean | no | Enable/disable this notification | +| `close_merge_request` | boolean | no | Enable/disable this notification | +| `reassign_merge_request` | boolean | no | Enable/disable this notification | +| `merge_merge_request` | boolean | no | Enable/disable this notification | + +Example response: + +```json +{ + "level": "watch", + "notification_email": "admin@example.com" +} +``` + +## Group / project level notification settings + +Get current group or project notification settings. + +``` +GET /groups/:id/notification_settings +GET /projects/:id/notification_settings +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/5/notification_settings +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/8/notification_settings +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | + +Example response: + +```json +{ + "level": "global" +} +``` + +## Update group/project level notification settings + +Update current group/project notification settings. + +``` +PUT /groups/:id/notification_settings +PUT /projects/:id/notification_settings +``` + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/5/notification_settings?level=watch +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/8/notification_settings?level=custom&new_note=true +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The group/project ID or path | +| `level` | string | no | The global notification level | +| `new_note` | boolean | no | Enable/disable this notification | +| `new_issue` | boolean | no | Enable/disable this notification | +| `reopen_issue` | boolean | no | Enable/disable this notification | +| `close_issue` | boolean | no | Enable/disable this notification | +| `reassign_issue` | boolean | no | Enable/disable this notification | +| `new_merge_request` | boolean | no | Enable/disable this notification | +| `reopen_merge_request` | boolean | no | Enable/disable this notification | +| `close_merge_request` | boolean | no | Enable/disable this notification | +| `reassign_merge_request` | boolean | no | Enable/disable this notification | +| `merge_merge_request` | boolean | no | Enable/disable this notification | + +Example responses: + +```json +{ + "level": "watch" +} + +{ + "level": "custom", + "events": { + "new_note": true, + "new_issue": false, + "reopen_issue": false, + "close_issue": false, + "reassign_issue": false, + "new_merge_request": false, + "reopen_merge_request": false, + "close_merge_request": false, + "reassign_merge_request": false, + "merge_merge_request": false + } +} +``` + +[ce-5632]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5632 diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md index 31902e145f6..5ef5e3f5744 100644 --- a/doc/api/oauth2.md +++ b/doc/api/oauth2.md @@ -1,51 +1,74 @@ -# GitLab as an OAuth2 client +# GitLab as an OAuth2 provider -This document is about using other OAuth authentication service providers to sign into GitLab. -If you want GitLab to be an OAuth authentication service provider to sign into other services please see the [Oauth2 provider documentation](../integration/oauth_provider.md). +This document covers using the OAuth2 protocol to access GitLab. -OAuth2 is a protocol that enables us to authenticate a user without requiring them to give their password. +If you want GitLab to be an OAuth authentication service provider to sign into other services please see the [Oauth2 provider documentation](../integration/oauth_provider.md). -Before using the OAuth2 you should create an application in user's account. Each application gets a unique App ID and App Secret parameters. You should not share these. +OAuth2 is a protocol that enables us to authenticate a user without requiring them to give their password to a third-party. This functionality is based on [doorkeeper gem](https://github.com/doorkeeper-gem/doorkeeper) ## Web Application Flow -This flow is using for authentication from third-party web sites and is probably used the most. -It basically consists of an exchange of an authorization token for an access token. For more detailed info, check out the [RFC spec here](http://tools.ietf.org/html/rfc6749#section-4.1) +This is the most common type of flow and is used by server-side clients that wish to access GitLab on a user's behalf. + +>**Note:** +This flow **should not** be used for client-side clients as you would leak your `client_secret`. Client-side clients should use the Implicit Grant (which is currently unsupported). -This flow consists from 3 steps. +For more detailed information, check out the [RFC spec](http://tools.ietf.org/html/rfc6749#section-4.1) + +In the following sections you will be introduced to the three steps needed for this flow. ### 1. Registering the client -Create an application in user's account profile. +First, you should create an application (`/profile/applications`) in your user's account. +Each application gets a unique App ID and App Secret parameters. + +>**Note:** +**You should not share/leak your App ID or App Secret.** ### 2. Requesting authorization -To request the authorization token, you should visit the `/oauth/authorize` endpoint. You can do that by visiting manually the URL: +To request the authorization code, you should redirect the user to the `/oauth/authorize` endpoint: + +``` +https://gitlab.example.com/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=code&state=your_unique_state_hash +``` + +This will ask the user to approve the applications access to their account and then redirect back to the `REDIRECT_URI` you provided. + +The redirect will include the GET `code` parameter, for example: ``` -http://localhost:3000/oauth/authorize?client_id=APP_ID&redirect_uri=REDIRECT_URI&response_type=code +http://myapp.com/oauth/redirect?code=1234567890&state=your_unique_state_hash ``` -Where REDIRECT_URI is the URL in your app where users will be sent after authorization. +You should then use the `code` to request an access token. + +>**Important:** +It is highly recommended that you send a `state` value with the request to `/oauth/authorize` and +validate that value is returned and matches in the redirect request. +This is important to prevent [CSRF attacks](http://www.oauthsecurity.com/#user-content-authorization-code-flow), +`state` really should have been a requirement in the standard! ### 3. Requesting the access token -To request the access token, you should use the returned code and exchange it for an access token. To do that you can use any HTTP client. In this case, I used rest-client: +Once you have the authorization code you can request an `access_token` using the code, to do that you can use any HTTP client. In the following example, we are using Ruby's `rest-client`: ``` -parameters = 'client_id=APP_ID&client_secret=APP_SECRET&code=RETURNED_CODE&grant_type=AUTHORIZATION_CODE&redirect_uri=REDIRECT_URI' +parameters = 'client_id=APP_ID&client_secret=APP_SECRET&code=RETURNED_CODE&grant_type=authorization_code&redirect_uri=REDIRECT_URI' RestClient.post 'http://localhost:3000/oauth/token', parameters # The response will be { "access_token": "de6780bc506a0446309bd9362820ba8aed28aa506c71eedbe1c5c4f9dd350e54", - "token_type": "bearer", + "token_type": "bearer", "expires_in": 7200, "refresh_token": "8257e65c97202ed1726cf9571600918f3bffb2544b26e00a61df9897668c33a1" } ``` +>**Note:** +The `redirect_uri` must match the `redirect_uri` used in the original authorization request. You can now make requests to the API with the access token returned. @@ -60,23 +83,26 @@ GET https://localhost:3000/api/v3/user?access_token=OAUTH-TOKEN Or you can put the token to the Authorization header: ``` -curl -H "Authorization: Bearer OAUTH-TOKEN" https://localhost:3000/api/v3/user +curl --header "Authorization: Bearer OAUTH-TOKEN" https://localhost:3000/api/v3/user ``` ## Resource Owner Password Credentials ## Deprecation Notice -1. Starting in GitLab 9.0, the Resource Owner Password Credentials will be *disabled* for users with two-factor authentication turned on. +1. Starting in GitLab 8.11, the Resource Owner Password Credentials has been *disabled* for users with two-factor authentication turned on. 2. These users can access the API using [personal access tokens] instead. --- -In this flow, a token is requested in exchange for the resource owner credentials (username and password). +In this flow, a token is requested in exchange for the resource owner credentials (username and password). The credentials should only be used when there is a high degree of trust between the resource owner and the client (e.g. the client is part of the device operating system or a highly privileged application), and when other authorization grant types are not available (such as an authorization code). +>**Important:** +Never store the users credentials and only use this grant type when your client is deployed to a trusted environment, in 99% of cases [personal access tokens] are a better choice. + Even though this grant type requires direct client access to the resource owner credentials, the resource owner credentials are used for a single request and are exchanged for an access token. This grant type can eliminate the need for the client to store the resource owner credentials for future use, by exchanging the credentials with a long-lived access token or refresh token. @@ -86,7 +112,7 @@ You can do POST request to `/oauth/token` with parameters: { "grant_type" : "password", "username" : "user@example.com", - "password" : "sekret" + "password" : "secret" } ``` @@ -104,8 +130,8 @@ For testing you can use the oauth2 ruby gem: ``` client = OAuth2::Client.new('the_client_id', 'the_client_secret', :site => "http://example.com") -access_token = client.password.get_token('user@example.com', 'sekret') +access_token = client.password.get_token('user@example.com', 'secret') puts access_token.token ``` -[personal access tokens]: ./README.md#personal-access-tokens +[personal access tokens]: ./README.md#personal-access-tokens
\ No newline at end of file diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md new file mode 100644 index 00000000000..a29b3eb6f44 --- /dev/null +++ b/doc/api/pipelines.md @@ -0,0 +1,207 @@ +# Pipelines API + +## List project pipelines + +> [Introduced][ce-5837] in GitLab 8.11 + +``` +GET /projects/:id/pipelines +``` + +| Attribute | Type | Required | Description | +|-----------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a project | + +``` +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/pipelines" +``` + +Example of response + +```json +[ + { + "id": 47, + "status": "pending", + "ref": "new-pipeline", + "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "before_sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "tag": false, + "yaml_errors": null, + "user": { + "name": "Administrator", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2016-08-16T10:23:19.007Z", + "updated_at": "2016-08-16T10:23:19.216Z", + "started_at": null, + "finished_at": null, + "committed_at": null, + "duration": null + }, + { + "id": 48, + "status": "pending", + "ref": "new-pipeline", + "sha": "eb94b618fb5865b26e80fdd8ae531b7a63ad851a", + "before_sha": "eb94b618fb5865b26e80fdd8ae531b7a63ad851a", + "tag": false, + "yaml_errors": null, + "user": { + "name": "Administrator", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2016-08-16T10:23:21.184Z", + "updated_at": "2016-08-16T10:23:21.314Z", + "started_at": null, + "finished_at": null, + "committed_at": null, + "duration": null + } +] +``` + +## Get a single pipeline + +> [Introduced][ce-5837] in GitLab 8.11 + +``` +GET /projects/:id/pipelines/:pipeline_id +``` + +| Attribute | Type | Required | Description | +|------------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a project | +| `pipeline_id` | integer | yes | The ID of a pipeline | + +``` +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/pipeline/46" +``` + +Example of response + +```json +{ + "id": 46, + "status": "success", + "ref": "master", + "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "before_sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "tag": false, + "yaml_errors": null, + "user": { + "name": "Administrator", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2016-08-11T11:28:34.085Z", + "updated_at": "2016-08-11T11:32:35.169Z", + "started_at": null, + "finished_at": "2016-08-11T11:32:35.145Z", + "committed_at": null, + "duration": null +} +``` + +## Retry failed builds in a pipeline + +> [Introduced][ce-5837] in GitLab 8.11 + +``` +POST /projects/:id/pipelines/:pipeline_id/retry +``` + +| Attribute | Type | Required | Description | +|------------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a project | +| `pipeline_id` | integer | yes | The ID of a pipeline | + +``` +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/pipelines/46/retry" +``` + +Response: + +```json +{ + "id": 46, + "status": "pending", + "ref": "master", + "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "before_sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "tag": false, + "yaml_errors": null, + "user": { + "name": "Administrator", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2016-08-11T11:28:34.085Z", + "updated_at": "2016-08-11T11:32:35.169Z", + "started_at": null, + "finished_at": "2016-08-11T11:32:35.145Z", + "committed_at": null, + "duration": null +} +``` + +## Cancel a pipelines builds + +> [Introduced][ce-5837] in GitLab 8.11 + +``` +POST /projects/:id/pipelines/:pipeline_id/cancel +``` + +| Attribute | Type | Required | Description | +|------------|---------|----------|---------------------| +| `id` | integer | yes | The ID of a project | +| `pipeline_id` | integer | yes | The ID of a pipeline | + +``` +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/pipelines/46/cancel" +``` + +Response: + +```json +{ + "id": 46, + "status": "canceled", + "ref": "master", + "sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "before_sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a", + "tag": false, + "yaml_errors": null, + "user": { + "name": "Administrator", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2016-08-11T11:28:34.085Z", + "updated_at": "2016-08-11T11:32:35.169Z", + "started_at": null, + "finished_at": "2016-08-11T11:32:35.145Z", + "committed_at": null, + "duration": null +} +``` + +[ce-5837]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5837 diff --git a/doc/api/project_snippets.md b/doc/api/project_snippets.md index a7acf37b5bc..c6685f54a9d 100644 --- a/doc/api/project_snippets.md +++ b/doc/api/project_snippets.md @@ -53,7 +53,8 @@ Parameters: }, "expires_at": null, "updated_at": "2012-06-28T10:52:04Z", - "created_at": "2012-06-28T10:52:04Z" + "created_at": "2012-06-28T10:52:04Z", + "web_url": "http://example.com/example/example/snippets/1" } ``` diff --git a/doc/api/projects.md b/doc/api/projects.md index dceee7b4ea7..b69db90e70d 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -20,7 +20,7 @@ Constants for project visibility levels are next: ## List projects -Get a list of projects accessible by the authenticated user. +Get a list of projects for which the authenticated user is a member. ``` GET /projects @@ -28,11 +28,150 @@ GET /projects Parameters: -- `archived` (optional) - if passed, limit by archived status -- `visibility` (optional) - if passed, limit by visibility `public`, `internal`, `private` -- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` -- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` -- `search` (optional) - Return list of authorized projects according to a search criteria +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of authorized projects matching the search criteria | +| `simple` | boolean | no | Return only the ID, URL, name, and path of each project | + +```json +[ + { + "id": 4, + "description": null, + "default_branch": "master", + "public": false, + "visibility_level": 0, + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git", + "web_url": "http://example.com/diaspora/diaspora-client", + "tag_list": [ + "example", + "disapora client" + ], + "owner": { + "id": 3, + "name": "Diaspora", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Diaspora Client", + "name_with_namespace": "Diaspora / Diaspora Client", + "path": "diaspora-client", + "path_with_namespace": "diaspora/diaspora-client", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "builds_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "created_at": "2013-09-30T13:46:02Z", + "description": "", + "id": 3, + "name": "Diaspora", + "owner_id": 1, + "path": "diaspora", + "updated_at": "2013-09-30T13:46:02Z" + }, + "archived": false, + "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png", + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_builds": true, + "shared_with_groups": [], + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false + }, + { + "id": 6, + "description": null, + "default_branch": "master", + "public": false, + "visibility_level": 0, + "ssh_url_to_repo": "git@example.com:brightbox/puppet.git", + "http_url_to_repo": "http://example.com/brightbox/puppet.git", + "web_url": "http://example.com/brightbox/puppet", + "tag_list": [ + "example", + "puppet" + ], + "owner": { + "id": 4, + "name": "Brightbox", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Puppet", + "name_with_namespace": "Brightbox / Puppet", + "path": "puppet", + "path_with_namespace": "brightbox/puppet", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "builds_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "container_registry_enabled": false, + "created_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "created_at": "2013-09-30T13:46:02Z", + "description": "", + "id": 4, + "name": "Brightbox", + "owner_id": 1, + "path": "brightbox", + "updated_at": "2013-09-30T13:46:02Z" + }, + "permissions": { + "project_access": { + "access_level": 10, + "notification_level": 3 + }, + "group_access": { + "access_level": 50, + "notification_level": 3 + } + }, + "archived": false, + "avatar_url": null, + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_builds": true, + "shared_with_groups": [], + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false + } +] +``` + +Get a list of projects which the authenticated user can see. + +``` +GET /projects/visible +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of authorized projects matching the search criteria | +| `simple` | boolean | no | Return only the ID, URL, name, and path of each project | ```json [ @@ -159,11 +298,13 @@ GET /projects/owned Parameters: -- `archived` (optional) - if passed, limit by archived status -- `visibility` (optional) - if passed, limit by visibility `public`, `internal`, `private` -- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` -- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` -- `search` (optional) - Return list of authorized projects according to a search criteria +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of authorized projects matching the search criteria | ### List starred projects @@ -175,11 +316,13 @@ GET /projects/starred Parameters: -- `archived` (optional) - if passed, limit by archived status -- `visibility` (optional) - if passed, limit by visibility `public`, `internal`, `private` -- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` -- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` -- `search` (optional) - Return list of authorized projects according to a search criteria +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of authorized projects matching the search criteria | ### List ALL projects @@ -191,11 +334,13 @@ GET /projects/all Parameters: -- `archived` (optional) - if passed, limit by archived status -- `visibility` (optional) - if passed, limit by visibility `public`, `internal`, `private` -- `order_by` (optional) - Return requests ordered by `id`, `name`, `path`, `created_at`, `updated_at` or `last_activity_at` fields. Default is `created_at` -- `sort` (optional) - Return requests sorted in `asc` or `desc` order. Default is `desc` -- `search` (optional) - Return list of authorized projects according to a search criteria +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `archived` | boolean | no | Limit by archived status | +| `visibility` | string | no | Limit by visibility `public`, `internal`, or `private` | +| `order_by` | string | no | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at` | +| `sort` | string | no | Return projects sorted in `asc` or `desc` order. Default is `desc` | +| `search` | string | no | Return list of authorized projects matching the search criteria | ### Get single project @@ -208,7 +353,9 @@ GET /projects/:id Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or NAMESPACE/PROJECT_NAME of the project | ```json { @@ -280,14 +427,16 @@ Parameters: "group_name": "Gitlab Org", "group_access_level": 10 } - ] + ], + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` ### Get project events Get the events for the specified project. -Sorted from newest to latest +Sorted from newest to oldest ``` GET /projects/:id/events @@ -295,7 +444,9 @@ GET /projects/:id/events Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or NAMESPACE/PROJECT_NAME of the project | ```json [ @@ -314,7 +465,7 @@ Parameters: "id": 1, "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" }, "author_username": "root" }, @@ -331,7 +482,7 @@ Parameters: "id": 1, "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" }, "author_username": "john", "data": { @@ -377,7 +528,7 @@ Parameters: "id": 1, "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" }, "author_username": "root" }, @@ -401,7 +552,7 @@ Parameters: "id": 1, "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" }, "created_at": "2015-12-04T10:33:56.698Z", "system": false, @@ -416,7 +567,7 @@ Parameters: "id": 1, "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" }, "author_username": "root" } @@ -433,21 +584,26 @@ POST /projects Parameters: -- `name` (required) - new project name -- `path` (optional) - custom repository name for new project. By default generated based on name -- `namespace_id` (optional) - namespace for the new project (defaults to user) -- `description` (optional) - short project description -- `issues_enabled` (optional) -- `merge_requests_enabled` (optional) -- `builds_enabled` (optional) -- `wiki_enabled` (optional) -- `snippets_enabled` (optional) -- `container_registry_enabled` (optional) -- `shared_runners_enabled` (optional) -- `public` (optional) - if `true` same as setting visibility_level = 20 -- `visibility_level` (optional) -- `import_url` (optional) -- `public_builds` (optional) +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `name` | string | yes | The name of the new project | +| `path` | string | no | Custom repository name for new project. By default generated based on name | +| `namespace_id` | integer | no | Namespace for the new project (defaults to the current user's namespace) | +| `description` | string | no | Short project description | +| `issues_enabled` | boolean | no | Enable issues for this project | +| `merge_requests_enabled` | boolean | no | Enable merge requests for this project | +| `builds_enabled` | boolean | no | Enable builds for this project | +| `wiki_enabled` | boolean | no | Enable wiki for this project | +| `snippets_enabled` | boolean | no | Enable snippets for this project | +| `container_registry_enabled` | boolean | no | Enable container registry for this project | +| `shared_runners_enabled` | boolean | no | Enable shared runners for this project | +| `public` | boolean | no | If `true`, the same as setting `visibility_level` to 20 | +| `visibility_level` | integer | no | See [project visibility level][#project-visibility-level] | +| `import_url` | string | no | URL to import repository from | +| `public_builds` | boolean | no | If `true`, builds can be viewed by non-project-members | +| `only_allow_merge_if_build_succeeds` | boolean | no | Set whether merge requests can only be merged with successful builds | +| `lfs_enabled` | boolean | no | Enable LFS | +| `request_access_enabled` | boolean | no | Allow users to request member access | ### Create project for user @@ -459,20 +615,27 @@ POST /projects/user/:user_id Parameters: -- `user_id` (required) - user_id of owner -- `name` (required) - new project name -- `description` (optional) - short project description -- `issues_enabled` (optional) -- `merge_requests_enabled` (optional) -- `builds_enabled` (optional) -- `wiki_enabled` (optional) -- `snippets_enabled` (optional) -- `container_registry_enabled` (optional) -- `shared_runners_enabled` (optional) -- `public` (optional) - if `true` same as setting visibility_level = 20 -- `visibility_level` (optional) -- `import_url` (optional) -- `public_builds` (optional) +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `user_id` | integer | yes | The user ID of the project owner | +| `name` | string | yes | The name of the new project | +| `path` | string | no | Custom repository name for new project. By default generated based on name | +| `namespace_id` | integer | no | Namespace for the new project (defaults to the current user's namespace) | +| `description` | string | no | Short project description | +| `issues_enabled` | boolean | no | Enable issues for this project | +| `merge_requests_enabled` | boolean | no | Enable merge requests for this project | +| `builds_enabled` | boolean | no | Enable builds for this project | +| `wiki_enabled` | boolean | no | Enable wiki for this project | +| `snippets_enabled` | boolean | no | Enable snippets for this project | +| `container_registry_enabled` | boolean | no | Enable container registry for this project | +| `shared_runners_enabled` | boolean | no | Enable shared runners for this project | +| `public` | boolean | no | If `true`, the same as setting `visibility_level` to 20 | +| `visibility_level` | integer | no | See [project visibility level][#project-visibility-level] | +| `import_url` | string | no | URL to import repository from | +| `public_builds` | boolean | no | If `true`, builds can be viewed by non-project-members | +| `only_allow_merge_if_build_succeeds` | boolean | no | Set whether merge requests can only be merged with successful builds | +| `lfs_enabled` | boolean | no | Enable LFS | +| `request_access_enabled` | boolean | no | Allow users to request member access | ### Edit project @@ -484,28 +647,33 @@ PUT /projects/:id Parameters: -- `id` (required) - The ID of a project -- `name` (optional) - project name -- `path` (optional) - repository name for project -- `description` (optional) - short project description -- `default_branch` (optional) -- `issues_enabled` (optional) -- `merge_requests_enabled` (optional) -- `builds_enabled` (optional) -- `wiki_enabled` (optional) -- `snippets_enabled` (optional) -- `container_registry_enabled` (optional) -- `shared_runners_enabled` (optional) -- `public` (optional) - if `true` same as setting visibility_level = 20 -- `visibility_level` (optional) -- `public_builds` (optional) +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or NAMESPACE/PROJECT_NAME of the project | +| `name` | string | yes | The name of the project | +| `path` | string | no | Custom repository name for the project. By default generated based on name | +| `description` | string | no | Short project description | +| `issues_enabled` | boolean | no | Enable issues for this project | +| `merge_requests_enabled` | boolean | no | Enable merge requests for this project | +| `builds_enabled` | boolean | no | Enable builds for this project | +| `wiki_enabled` | boolean | no | Enable wiki for this project | +| `snippets_enabled` | boolean | no | Enable snippets for this project | +| `container_registry_enabled` | boolean | no | Enable container registry for this project | +| `shared_runners_enabled` | boolean | no | Enable shared runners for this project | +| `public` | boolean | no | If `true`, the same as setting `visibility_level` to 20 | +| `visibility_level` | integer | no | See [project visibility level][#project-visibility-level] | +| `import_url` | string | no | URL to import repository from | +| `public_builds` | boolean | no | If `true`, builds can be viewed by non-project-members | +| `only_allow_merge_if_build_succeeds` | boolean | no | Set whether merge requests can only be merged with successful builds | +| `lfs_enabled` | boolean | no | Enable LFS | +| `request_access_enabled` | boolean | no | Allow users to request member access | On success, method returns 200 with the updated project. If parameters are invalid, 400 is returned. ### Fork project -Forks a project into the user namespace of the authenticated user. +Forks a project into the user namespace of the authenticated user or the one provided. ``` POST /projects/fork/:id @@ -513,7 +681,10 @@ POST /projects/fork/:id Parameters: -- `id` (required) - The ID of the project to be forked +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or NAMESPACE/PROJECT_NAME of the project | +| `namespace` | integer/string | yes | The ID or path of the namespace that the project will be forked to | ### Star a project @@ -524,12 +695,14 @@ Stars a given project. Returns status code `201` and the project on success and POST /projects/:id/star ``` +Parameters: + | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | +| `id` | integer/string | yes | The ID or NAMESPACE/PROJECT_NAME of the project | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/star" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/star" ``` Example response: @@ -577,7 +750,9 @@ Example response: "forks_count": 0, "star_count": 1, "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -592,10 +767,10 @@ DELETE /projects/:id/star | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/star" +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/star" ``` Example response: @@ -643,7 +818,9 @@ Example response: "forks_count": 0, "star_count": 0, "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -662,10 +839,10 @@ POST /projects/:id/archive | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/archive" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/archive" ``` Example response: @@ -729,7 +906,9 @@ Example response: "star_count": 0, "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b", "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -748,10 +927,10 @@ POST /projects/:id/unarchive | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/unarchive" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/unarchive" ``` Example response: @@ -815,7 +994,9 @@ Example response: "star_count": 0, "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b", "public_builds": true, - "shared_with_groups": [] + "shared_with_groups": [], + "only_allow_merge_if_build_succeeds": false, + "request_access_enabled": false } ``` @@ -829,7 +1010,9 @@ DELETE /projects/:id Parameters: -- `id` (required) - The ID of a project +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ## Uploads @@ -843,14 +1026,15 @@ POST /projects/:id/uploads Parameters: -- `id` (required) - The ID of the project -- `file` (required) - The file to be uploaded +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `file` | string | yes | The file to be uploaded | ```json { "alt": "dk", "url": "/uploads/66dbcd21ec5d24ed6ea225176098d52b/dk.png", - "is_image": true, "markdown": "" } ``` @@ -859,95 +1043,9 @@ Parameters: In Markdown contexts, the link is automatically expanded when the format in `markdown` is used. -## Team members - -### List project team members - -Get a list of a project's team members. - -``` -GET /projects/:id/members -``` - -Parameters: - -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `query` (optional) - Query string to search for members - -### Get project team member - -Gets a project team member. - -``` -GET /projects/:id/members/:user_id -``` - -Parameters: - -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `user_id` (required) - The ID of a user - -```json -{ - "id": 1, - "username": "john_smith", - "email": "john@example.com", - "name": "John Smith", - "state": "active", - "created_at": "2012-05-23T08:00:58Z", - "access_level": 40 -} -``` - -### Add project team member - -Adds a user to a project team. This is an idempotent method and can be called multiple times -with the same parameters. Adding team membership to a user that is already a member does not -affect the existing membership. - -``` -POST /projects/:id/members -``` - -Parameters: +## Project members -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `user_id` (required) - The ID of a user to add -- `access_level` (required) - Project access level - -### Edit project team member - -Updates a project team member to a specified access level. - -``` -PUT /projects/:id/members/:user_id -``` - -Parameters: - -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `user_id` (required) - The ID of a team member -- `access_level` (required) - Project access level - -### Remove project team member - -Removes a user from a project team. - -``` -DELETE /projects/:id/members/:user_id -``` - -Parameters: - -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `user_id` (required) - The ID of a team member - -This method removes the project member if the user has the proper access rights to do so. -It returns a status code 403 if the member does not have the proper rights to perform this action. -In all other cases this method is idempotent and revoking team membership for a user who is not -currently a team member is considered success. -Please note that the returned JSON currently differs slightly. Thus you should not -rely on the returned JSON structure. +Please consult the [Project Members](members.md) documentation. ### Share project with group @@ -959,9 +1057,12 @@ POST /projects/:id/share Parameters: -- `id` (required) - The ID of a project -- `group_id` (required) - The ID of a group -- `group_access` (required) - Level of permissions for sharing +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `group_id` | integer | yes | The ID of the group to share with | +| `group_access` | integer | yes | The permissions level to grant the group | +| `expires_at` | string | no | Share expiration date in ISO 8601 format: 2016-09-26 | ## Hooks @@ -978,7 +1079,9 @@ GET /projects/:id/hooks Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ### Get project hook @@ -990,8 +1093,10 @@ GET /projects/:id/hooks/:hook_id Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `hook_id` (required) - The ID of a project hook +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `hook_id` | integer | yes | The ID of a project hook | ```json { @@ -1001,7 +1106,11 @@ Parameters: "push_events": true, "issues_events": true, "merge_requests_events": true, + "tag_push_events": true, "note_events": true, + "build_events": true, + "pipeline_events": true, + "wiki_page_events": true, "enable_ssl_verification": true, "created_at": "2012-10-12T17:04:47Z" } @@ -1017,14 +1126,19 @@ POST /projects/:id/hooks Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `url` (required) - The hook URL -- `push_events` - Trigger hook on push events -- `issues_events` - Trigger hook on issues events -- `merge_requests_events` - Trigger hook on merge_requests events -- `tag_push_events` - Trigger hook on push_tag events -- `note_events` - Trigger hook on note events -- `enable_ssl_verification` - Do SSL verification when triggering the hook +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `url` | string | yes | The hook URL | +| `push_events` | boolean | no | Trigger hook on push events | +| `issues_events` | boolean | no | Trigger hook on issues events | +| `merge_requests_events` | boolean | no | Trigger hook on merge requests events | +| `tag_push_events` | boolean | no | Trigger hook on tag push events | +| `note_events` | boolean | no | Trigger hook on note events | +| `build_events` | boolean | no | Trigger hook on build events | +| `pipeline_events` | boolean | no | Trigger hook on pipeline events | +| `wiki_events` | boolean | no | Trigger hook on wiki events | +| `enable_ssl_verification` | boolean | no | Do SSL verification when triggering the hook | ### Edit project hook @@ -1036,15 +1150,20 @@ PUT /projects/:id/hooks/:hook_id Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `hook_id` (required) - The ID of a project hook -- `url` (required) - The hook URL -- `push_events` - Trigger hook on push events -- `issues_events` - Trigger hook on issues events -- `merge_requests_events` - Trigger hook on merge_requests events -- `tag_push_events` - Trigger hook on push_tag events -- `note_events` - Trigger hook on note events -- `enable_ssl_verification` - Do SSL verification when triggering the hook +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `hook_id` | integer | yes | The ID of the project hook | +| `url` | string | yes | The hook URL | +| `push_events` | boolean | no | Trigger hook on push events | +| `issues_events` | boolean | no | Trigger hook on issues events | +| `merge_requests_events` | boolean | no | Trigger hook on merge requests events | +| `tag_push_events` | boolean | no | Trigger hook on tag push events | +| `note_events` | boolean | no | Trigger hook on note events | +| `build_events` | boolean | no | Trigger hook on build events | +| `pipeline_events` | boolean | no | Trigger hook on pipeline events | +| `wiki_events` | boolean | no | Trigger hook on wiki events | +| `enable_ssl_verification` | boolean | no | Do SSL verification when triggering the hook | ### Delete project hook @@ -1057,14 +1176,18 @@ DELETE /projects/:id/hooks/:hook_id Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `hook_id` (required) - The ID of hook to delete +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `hook_id` | integer | yes | The ID of the project hook | Note the JSON response differs if the hook is available or not. If the project hook is available before it is returned in the JSON response or an empty response is returned. ## Branches +For more information please consult the [Branches](branches.md) documentation. + ### List branches Lists all branches of a project. @@ -1075,7 +1198,9 @@ GET /projects/:id/repository/branches Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ```json [ @@ -1083,56 +1208,46 @@ Parameters: "name": "async", "commit": { "id": "a2b702edecdf41f07b42653eb1abe30ce98b9fca", - "parents": [ - { - "id": "3f94fc7c85061973edc9906ae170cc269b07ca55" - } + "parent_ids": [ + "3f94fc7c85061973edc9906ae170cc269b07ca55" ], - "tree": "c68537c6534a02cc2b176ca1549f4ffa190b58ee", "message": "give Caolan credit where it's due (up top)", - "author": { - "name": "Jeremy Ashkenas", - "email": "jashkenas@example.com" - }, - "committer": { - "name": "Jeremy Ashkenas", - "email": "jashkenas@example.com" - }, + "author_name": "Jeremy Ashkenas", + "author_email": "jashkenas@example.com", "authored_date": "2010-12-08T21:28:50+00:00", + "committer_name": "Jeremy Ashkenas", + "committer_email": "jashkenas@example.com", "committed_date": "2010-12-08T21:28:50+00:00" }, - "protected": false + "protected": false, + "developers_can_push": false, + "developers_can_merge": false }, { "name": "gh-pages", "commit": { "id": "101c10a60019fe870d21868835f65c25d64968fc", - "parents": [ - { - "id": "9c15d2e26945a665131af5d7b6d30a06ba338aaa" - } + "parent_ids": [ + "9c15d2e26945a665131af5d7b6d30a06ba338aaa" ], - "tree": "fb5cc9d45da3014b17a876ad539976a0fb9b352a", "message": "Underscore.js 1.5.2", - "author": { - "name": "Jeremy Ashkenas", - "email": "jashkenas@example.com" - }, - "committer": { - "name": "Jeremy Ashkenas", - "email": "jashkenas@example.com" - }, + "author_name": "Jeremy Ashkenas", + "author_email": "jashkenas@example.com", "authored_date": "2013-09-07T12:58:21+00:00", + "committer_name": "Jeremy Ashkenas", + "committer_email": "jashkenas@example.com", "committed_date": "2013-09-07T12:58:21+00:00" }, - "protected": false + "protected": false, + "developers_can_push": false, + "developers_can_merge": false } ] ``` -### List single branch +### Single branch -Lists a specific branch of a project. +A specific branch of a project. ``` GET /projects/:id/repository/branches/:branch @@ -1140,8 +1255,12 @@ GET /projects/:id/repository/branches/:branch Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `branch` (required) - The name of the branch. +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `branch` | string | yes | The name of the branch | +| `developers_can_push` | boolean | no | Flag if developers can push to the branch | +| `developers_can_merge` | boolean | no | Flag if developers can merge to the branch | ### Protect single branch @@ -1153,8 +1272,10 @@ PUT /projects/:id/repository/branches/:branch/protect Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `branch` (required) - The name of the branch. +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `branch` | string | yes | The name of the branch | ### Unprotect single branch @@ -1166,8 +1287,10 @@ PUT /projects/:id/repository/branches/:branch/unprotect Parameters: -- `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project -- `branch` (required) - The name of the branch. +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `branch` | string | yes | The name of the branch | ## Admin fork relation @@ -1181,8 +1304,10 @@ POST /projects/:id/fork/:forked_from_id Parameters: -- `id` (required) - The ID of the project -- `forked_from_id:` (required) - The ID of the project that was forked from +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | +| `forked_from_id` | ID | yes | The ID of the project that was forked from | ### Delete an existing forked from relationship @@ -1192,7 +1317,9 @@ DELETE /projects/:id/fork Parameter: -- `id` (required) - The ID of the project +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID of the project or NAMESPACE/PROJECT_NAME | ## Search for projects by name @@ -1204,8 +1331,8 @@ GET /projects/search/:query Parameters: -- `query` (required) - A string contained in the project name -- `per_page` (optional) - number of projects to return per page -- `page` (optional) - the page to retrieve -- `order_by` (optional) - Return requests ordered by `id`, `name`, `created_at` or `last_activity_at` fields -- `sort` (optional) - Return requests sorted in `asc` or `desc` order +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `query` | string | yes | A string contained in the project name | +| `order_by` | string | no | Return requests ordered by `id`, `name`, `created_at` or `last_activity_at` fields | +| `sort` | string | no | Return requests sorted in `asc` or `desc` order | diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index 623063f357b..1bc6a24e914 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -12,6 +12,10 @@ Allows you to receive information about file in repository like name, size, cont GET /projects/:id/repository/files ``` +```bash +curl --request GET --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/models/key.rb&ref=master' +``` + Example response: ```json @@ -39,6 +43,10 @@ Parameters: POST /projects/:id/repository/files ``` +```bash +curl --request POST --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20content&commit_message=create%20a%20new%20file' +``` + Example response: ```json @@ -53,6 +61,8 @@ Parameters: - `file_path` (required) - Full path to new file. Ex. lib/class.rb - `branch_name` (required) - The name of branch - `encoding` (optional) - 'text' or 'base64'. Text is default. +- `author_email` (optional) - Specify the commit author's email address +- `author_name` (optional) - Specify the commit author's name - `content` (required) - File content - `commit_message` (required) - Commit message @@ -62,6 +72,10 @@ Parameters: PUT /projects/:id/repository/files ``` +```bash +curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&author_email=author%40example.com&author_name=Firstname%20Lastname&content=some%20other%20content&commit_message=update%20file' +``` + Example response: ```json @@ -76,6 +90,8 @@ Parameters: - `file_path` (required) - Full path to file. Ex. lib/class.rb - `branch_name` (required) - The name of branch - `encoding` (optional) - 'text' or 'base64'. Text is default. +- `author_email` (optional) - Specify the commit author's email address +- `author_name` (optional) - Specify the commit author's name - `content` (required) - New file content - `commit_message` (required) - Commit message @@ -94,6 +110,10 @@ Currently gitlab-shell has a boolean return code, preventing GitLab from specify DELETE /projects/:id/repository/files ``` +```bash +curl --request PUT --header 'PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK' 'https://gitlab.example.com/api/v3/projects/13083/repository/files?file_path=app/project.rb&branch_name=master&author_email=author%40example.com&author_name=Firstname%20Lastname&commit_message=delete%20file' +``` + Example response: ```json @@ -107,4 +127,6 @@ Parameters: - `file_path` (required) - Full path to file. Ex. lib/class.rb - `branch_name` (required) - The name of branch +- `author_email` (optional) - Specify the commit author's email address +- `author_name` (optional) - Specify the commit author's name - `commit_message` (required) - Commit message diff --git a/doc/api/runners.md b/doc/api/runners.md index ddfa298f79d..28610762dca 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -18,7 +18,7 @@ GET /runners?scope=active | `scope` | string | no | The scope of specific runners to show, one of: `active`, `paused`, `online`; showing all runners if none provided | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners" ``` Example response: @@ -57,7 +57,7 @@ GET /runners/all?scope=online | `scope` | string | no | The scope of runners to show, one of: `specific`, `shared`, `active`, `paused`, `online`; showing all runners if none provided | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/all" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/all" ``` Example response: @@ -108,7 +108,7 @@ GET /runners/:id | `id` | integer | yes | The ID of a runner | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/6" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/6" ``` Example response: @@ -158,7 +158,7 @@ PUT /runners/:id | `tag_list` | array | no | The list of tags for a runner; put array of tags, that should be finally assigned to a runner | ``` -curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/6" -F "description=test-1-20150125-test" -F "tag_list=ruby,mysql,tag1,tag2" +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/6" --form "description=test-1-20150125-test" --form "tag_list=ruby,mysql,tag1,tag2" ``` Example response: @@ -207,7 +207,7 @@ DELETE /runners/:id | `id` | integer | yes | The ID of a runner | ``` -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/6" +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/runners/6" ``` Example response: @@ -237,7 +237,7 @@ GET /projects/:id/runners | `id` | integer | yes | The ID of a project | ``` -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners" ``` Example response: @@ -275,7 +275,7 @@ POST /projects/:id/runners | `runner_id` | integer | yes | The ID of a runner | ``` -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners" -F "runner_id=9" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners" --form "runner_id=9" ``` Example response: @@ -306,7 +306,7 @@ DELETE /projects/:id/runners/:runner_id | `runner_id` | integer | yes | The ID of a runner | ``` -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners/9" +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/9/runners/9" ``` Example response: diff --git a/doc/api/services.md b/doc/api/services.md index f821a614047..579fdc0c8c9 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -355,7 +355,7 @@ PUT /projects/:id/services/gemnasium Parameters: -- `api_key` (**required**) - Your personal API KEY on gemnasium.com +- `api_key` (**required**) - Your personal API KEY on gemnasium.com - `token` (**required**) - The project's slug on gemnasium.com ### Delete Gemnasium service @@ -503,6 +503,7 @@ PUT /projects/:id/services/pivotaltracker Parameters: - `token` (**required**) +- `restrict_to_branch` (optional) - Comma-separated list of branches which will be automatically inspected. Leave blank to include all branches. ### Delete PivotalTracker service @@ -661,4 +662,3 @@ Get JetBrains TeamCity CI service settings for a project. ``` GET /projects/:id/services/teamcity ``` - diff --git a/doc/api/session.md b/doc/api/session.md index 066a055702d..f776424023e 100644 --- a/doc/api/session.md +++ b/doc/api/session.md @@ -2,7 +2,7 @@ ## Deprecation Notice -1. Starting in GitLab 9.0, this feature will be *disabled* for users with two-factor authentication turned on. +1. Starting in GitLab 8.11, this feature has been *disabled* for users with two-factor authentication turned on. 2. These users can access the API using [personal access tokens] instead. --- @@ -21,7 +21,7 @@ POST /session | `password` | string | yes | The password of the user | ```bash -curl -X POST "https://gitlab.example.com/api/v3/session?login=john_smith&password=strongpassw0rd" +curl --request POST "https://gitlab.example.com/api/v3/session?login=john_smith&password=strongpassw0rd" ``` Example response: diff --git a/doc/api/settings.md b/doc/api/settings.md index d9b68eaeadf..f7ad3b4cc8e 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -13,7 +13,7 @@ GET /application/settings ``` ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/application/settings +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/application/settings ``` Example response: @@ -33,13 +33,17 @@ Example response: "session_expire_delay" : 10080, "home_page_url" : null, "default_snippet_visibility" : 0, - "restricted_signup_domains" : [], + "domain_whitelist" : [], + "domain_blacklist_enabled" : false, + "domain_blacklist" : [], "created_at" : "2016-01-04T15:44:55.176Z", "default_project_visibility" : 0, "gravatar_enabled" : true, "sign_in_text" : null, "container_registry_token_expire_delay": 5, - "repository_storage": "default" + "repository_storage": "default", + "koding_enabled": false, + "koding_url": null } ``` @@ -63,15 +67,19 @@ PUT /application/settings | `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | | `default_project_visibility` | integer | no | What visibility level new projects receive. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is `0`.| | `default_snippet_visibility` | integer | no | What visibility level new snippets receive. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is `0`.| -| `restricted_signup_domains` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. | +| `domain_whitelist` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. | +| `domain_blacklist_enabled` | boolean | no | Enable/disable the `domain_blacklist` | +| `domain_blacklist` | array of strings | yes (if `domain_blacklist_enabled` is `true`) | People trying to sign-up with emails from this domain will not be allowed to do so. | | `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider | | `after_sign_out_path` | string | no | Where to redirect users after logout | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | | `repository_storage` | string | no | Storage path for new projects. The value should be the name of one of the repository storage paths defined in your gitlab.yml | -| `enabled_git_access_protocol` | string | no | Enabled protocols for Git access. Allowed values are: `ssh`, `http`, and `nil` to allow both protocols. +| `enabled_git_access_protocol` | string | no | Enabled protocols for Git access. Allowed values are: `ssh`, `http`, and `nil` to allow both protocols. | +| `koding_enabled` | boolean | no | Enable Koding integration. Default is `false`. | +| `koding_url` | string | yes (if `koding_enabled` is `true`) | The Koding instance URL for integration. | ```bash -curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/application/settings?signup_enabled=false&default_project_visibility=1 +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/application/settings?signup_enabled=false&default_project_visibility=1 ``` Example response: @@ -93,10 +101,14 @@ Example response: "session_expire_delay": 10080, "default_project_visibility": 1, "default_snippet_visibility": 0, - "restricted_signup_domains": [], + "domain_whitelist": [], + "domain_blacklist_enabled" : false, + "domain_blacklist" : [], "user_oauth_applications": true, "after_sign_out_path": "", "container_registry_token_expire_delay": 5, - "repository_storage": "default" + "repository_storage": "default", + "koding_enabled": false, + "koding_url": null } ``` diff --git a/doc/api/sidekiq_metrics.md b/doc/api/sidekiq_metrics.md index ebd131c94ca..1ae732d40d6 100644 --- a/doc/api/sidekiq_metrics.md +++ b/doc/api/sidekiq_metrics.md @@ -15,7 +15,7 @@ GET /sidekiq/queue_metrics ``` ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/queue_metrics +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/queue_metrics ``` Example response: @@ -40,7 +40,7 @@ GET /sidekiq/process_metrics ``` ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/process_metrics +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/process_metrics ``` Example response: @@ -82,7 +82,7 @@ GET /sidekiq/job_stats ``` ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/job_stats +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/job_stats ``` Example response: @@ -106,7 +106,7 @@ GET /sidekiq/compound_metrics ``` ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/compound_metrics +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/sidekiq/compound_metrics ``` Example response: diff --git a/doc/api/system_hooks.md b/doc/api/system_hooks.md index dc036d7e27f..073e99b7147 100644 --- a/doc/api/system_hooks.md +++ b/doc/api/system_hooks.md @@ -20,7 +20,7 @@ GET /hooks Example request: ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/hooks +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/hooks ``` Example response: @@ -52,7 +52,7 @@ POST /hooks Example request: ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/hooks?url=https://gitlab.example.com/hook" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/hooks?url=https://gitlab.example.com/hook" ``` Example response: @@ -80,7 +80,7 @@ GET /hooks/:id Example request: ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/hooks/2 +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/hooks/2 ``` Example response: @@ -98,11 +98,8 @@ Example response: ## Delete system hook -Deletes a system hook. This is an idempotent API function and returns `200 OK` -even if the hook is not available. - -If the hook is deleted, a JSON object is returned. An error is raised if the -hook is not found. +Deletes a system hook. It returns `200 OK` if the hooks is deleted and +`404 Not Found` if the hook is not found. --- @@ -117,7 +114,7 @@ DELETE /hooks/:id Example request: ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/hooks/2 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/hooks/2 ``` Example response: diff --git a/doc/api/tags.md b/doc/api/tags.md index ac9fac92f4c..54059117456 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -56,7 +56,7 @@ Parameters: | `tag_name` | string | yes | The name of the tag | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/tags/v1.0.0 +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/tags/v1.0.0 ``` Example Response: diff --git a/doc/api/templates/gitignores.md b/doc/api/templates/gitignores.md new file mode 100644 index 00000000000..8235be92b12 --- /dev/null +++ b/doc/api/templates/gitignores.md @@ -0,0 +1,579 @@ +# Gitignores + +## List gitignore templates + +Get all gitignore templates. + +``` +GET /templates/gitignores +``` + +```bash +curl https://gitlab.example.com/api/v3/templates/gitignores +``` + +Example response: + +```json +[ + { + "name": "AppEngine" + }, + { + "name": "Laravel" + }, + { + "name": "Elisp" + }, + { + "name": "SketchUp" + }, + { + "name": "Ada" + }, + { + "name": "Ruby" + }, + { + "name": "Kohana" + }, + { + "name": "Nanoc" + }, + { + "name": "Erlang" + }, + { + "name": "OCaml" + }, + { + "name": "Lithium" + }, + { + "name": "Fortran" + }, + { + "name": "Scala" + }, + { + "name": "Node" + }, + { + "name": "Fancy" + }, + { + "name": "Perl" + }, + { + "name": "Zephir" + }, + { + "name": "WordPress" + }, + { + "name": "Symfony" + }, + { + "name": "FuelPHP" + }, + { + "name": "DM" + }, + { + "name": "Sdcc" + }, + { + "name": "Rust" + }, + { + "name": "C" + }, + { + "name": "Umbraco" + }, + { + "name": "Actionscript" + }, + { + "name": "Android" + }, + { + "name": "Grails" + }, + { + "name": "Composer" + }, + { + "name": "ExpressionEngine" + }, + { + "name": "Gcov" + }, + { + "name": "Qt" + }, + { + "name": "Phalcon" + }, + { + "name": "ArchLinuxPackages" + }, + { + "name": "TeX" + }, + { + "name": "SCons" + }, + { + "name": "Lilypond" + }, + { + "name": "CommonLisp" + }, + { + "name": "Rails" + }, + { + "name": "Mercury" + }, + { + "name": "Magento" + }, + { + "name": "ChefCookbook" + }, + { + "name": "GitBook" + }, + { + "name": "C++" + }, + { + "name": "Eagle" + }, + { + "name": "Go" + }, + { + "name": "OpenCart" + }, + { + "name": "Scheme" + }, + { + "name": "Typo3" + }, + { + "name": "SeamGen" + }, + { + "name": "Swift" + }, + { + "name": "Elm" + }, + { + "name": "Unity" + }, + { + "name": "Agda" + }, + { + "name": "CUDA" + }, + { + "name": "VVVV" + }, + { + "name": "Finale" + }, + { + "name": "LemonStand" + }, + { + "name": "Textpattern" + }, + { + "name": "Julia" + }, + { + "name": "Packer" + }, + { + "name": "Scrivener" + }, + { + "name": "Dart" + }, + { + "name": "Plone" + }, + { + "name": "Jekyll" + }, + { + "name": "Xojo" + }, + { + "name": "LabVIEW" + }, + { + "name": "Autotools" + }, + { + "name": "KiCad" + }, + { + "name": "Prestashop" + }, + { + "name": "ROS" + }, + { + "name": "Smalltalk" + }, + { + "name": "GWT" + }, + { + "name": "OracleForms" + }, + { + "name": "SugarCRM" + }, + { + "name": "Nim" + }, + { + "name": "SymphonyCMS" + }, + { + "name": "Maven" + }, + { + "name": "CFWheels" + }, + { + "name": "Python" + }, + { + "name": "ZendFramework" + }, + { + "name": "CakePHP" + }, + { + "name": "Concrete5" + }, + { + "name": "PlayFramework" + }, + { + "name": "Terraform" + }, + { + "name": "Elixir" + }, + { + "name": "CMake" + }, + { + "name": "Joomla" + }, + { + "name": "Coq" + }, + { + "name": "Delphi" + }, + { + "name": "Haskell" + }, + { + "name": "Yii" + }, + { + "name": "Java" + }, + { + "name": "UnrealEngine" + }, + { + "name": "AppceleratorTitanium" + }, + { + "name": "CraftCMS" + }, + { + "name": "ForceDotCom" + }, + { + "name": "ExtJs" + }, + { + "name": "MetaProgrammingSystem" + }, + { + "name": "D" + }, + { + "name": "Objective-C" + }, + { + "name": "RhodesRhomobile" + }, + { + "name": "R" + }, + { + "name": "EPiServer" + }, + { + "name": "Yeoman" + }, + { + "name": "VisualStudio" + }, + { + "name": "Processing" + }, + { + "name": "Leiningen" + }, + { + "name": "Stella" + }, + { + "name": "Opa" + }, + { + "name": "Drupal" + }, + { + "name": "TurboGears2" + }, + { + "name": "Idris" + }, + { + "name": "Jboss" + }, + { + "name": "CodeIgniter" + }, + { + "name": "Qooxdoo" + }, + { + "name": "Waf" + }, + { + "name": "Sass" + }, + { + "name": "Lua" + }, + { + "name": "Clojure" + }, + { + "name": "IGORPro" + }, + { + "name": "Gradle" + }, + { + "name": "Archives" + }, + { + "name": "SynopsysVCS" + }, + { + "name": "Ninja" + }, + { + "name": "Tags" + }, + { + "name": "OSX" + }, + { + "name": "Dreamweaver" + }, + { + "name": "CodeKit" + }, + { + "name": "NotepadPP" + }, + { + "name": "VisualStudioCode" + }, + { + "name": "Mercurial" + }, + { + "name": "BricxCC" + }, + { + "name": "DartEditor" + }, + { + "name": "Eclipse" + }, + { + "name": "Cloud9" + }, + { + "name": "TortoiseGit" + }, + { + "name": "NetBeans" + }, + { + "name": "GPG" + }, + { + "name": "Espresso" + }, + { + "name": "Redcar" + }, + { + "name": "Xcode" + }, + { + "name": "Matlab" + }, + { + "name": "LyX" + }, + { + "name": "SlickEdit" + }, + { + "name": "Dropbox" + }, + { + "name": "CVS" + }, + { + "name": "Calabash" + }, + { + "name": "JDeveloper" + }, + { + "name": "Vagrant" + }, + { + "name": "IPythonNotebook" + }, + { + "name": "TextMate" + }, + { + "name": "Ensime" + }, + { + "name": "WebMethods" + }, + { + "name": "VirtualEnv" + }, + { + "name": "Emacs" + }, + { + "name": "Momentics" + }, + { + "name": "JetBrains" + }, + { + "name": "SublimeText" + }, + { + "name": "Kate" + }, + { + "name": "ModelSim" + }, + { + "name": "Redis" + }, + { + "name": "KDevelop4" + }, + { + "name": "Bazaar" + }, + { + "name": "Linux" + }, + { + "name": "Windows" + }, + { + "name": "XilinxISE" + }, + { + "name": "Lazarus" + }, + { + "name": "EiffelStudio" + }, + { + "name": "Anjuta" + }, + { + "name": "Vim" + }, + { + "name": "Otto" + }, + { + "name": "MicrosoftOffice" + }, + { + "name": "LibreOffice" + }, + { + "name": "SBT" + }, + { + "name": "MonoDevelop" + }, + { + "name": "SVN" + }, + { + "name": "FlexBuilder" + } +] +``` + +## Single gitignore template + +Get a single gitignore template. + +``` +GET /templates/gitignores/:key +``` + +| Attribute | Type | Required | Description | +| ---------- | ------ | -------- | ----------- | +| `key` | string | yes | The key of the gitignore template | + +```bash +curl https://gitlab.example.com/api/v3/templates/gitignores/Ruby +``` + +Example response: + +```json +{ + "name": "Ruby", + "content": "*.gem\n*.rbc\n/.config\n/coverage/\n/InstalledFiles\n/pkg/\n/spec/reports/\n/spec/examples.txt\n/test/tmp/\n/test/version_tmp/\n/tmp/\n\n# Used by dotenv library to load environment variables.\n# .env\n\n## Specific to RubyMotion:\n.dat*\n.repl_history\nbuild/\n*.bridgesupport\nbuild-iPhoneOS/\nbuild-iPhoneSimulator/\n\n## Specific to RubyMotion (use of CocoaPods):\n#\n# We recommend against adding the Pods directory to your .gitignore. However\n# you should judge for yourself, the pros and cons are mentioned at:\n# https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control\n#\n# vendor/Pods/\n\n## Documentation cache and generated files:\n/.yardoc/\n/_yardoc/\n/doc/\n/rdoc/\n\n## Environment normalization:\n/.bundle/\n/vendor/bundle\n/lib/bundler/man/\n\n# for a library or gem, you might want to ignore these files since the code is\n# intended to run in multiple environments; otherwise, check them in:\n# Gemfile.lock\n# .ruby-version\n# .ruby-gemset\n\n# unless supporting rvm < 1.11.0 or doing something fancy, ignore this:\n.rvmrc\n" +} +``` diff --git a/doc/api/templates/gitlab_ci_ymls.md b/doc/api/templates/gitlab_ci_ymls.md new file mode 100644 index 00000000000..e120016fbe6 --- /dev/null +++ b/doc/api/templates/gitlab_ci_ymls.md @@ -0,0 +1,120 @@ +# GitLab CI YMLs + +## List GitLab CI YML templates + +Get all GitLab CI YML templates. + +``` +GET /templates/gitlab_ci_ymls +``` + +```bash +curl https://gitlab.example.com/api/v3/templates/gitlab_ci_ymls +``` + +Example response: + +```json +[ + { + "name": "C++" + }, + { + "name": "Docker" + }, + { + "name": "Elixir" + }, + { + "name": "LaTeX" + }, + { + "name": "Grails" + }, + { + "name": "Rust" + }, + { + "name": "Nodejs" + }, + { + "name": "Ruby" + }, + { + "name": "Scala" + }, + { + "name": "Maven" + }, + { + "name": "Harp" + }, + { + "name": "Pelican" + }, + { + "name": "Hyde" + }, + { + "name": "Nanoc" + }, + { + "name": "Octopress" + }, + { + "name": "JBake" + }, + { + "name": "HTML" + }, + { + "name": "Hugo" + }, + { + "name": "Metalsmith" + }, + { + "name": "Hexo" + }, + { + "name": "Lektor" + }, + { + "name": "Doxygen" + }, + { + "name": "Brunch" + }, + { + "name": "Jekyll" + }, + { + "name": "Middleman" + } +] +``` + +## Single GitLab CI YML template + +Get a single GitLab CI YML template. + +``` +GET /templates/gitlab_ci_ymls/:key +``` + +| Attribute | Type | Required | Description | +| ---------- | ------ | -------- | ----------- | +| `key` | string | yes | The key of the GitLab CI YML template | + +```bash +curl https://gitlab.example.com/api/v3/templates/gitlab_ci_ymls/Ruby +``` + +Example response: + +```json +{ + "name": "Ruby", + "content": "# This file is a template, and might need editing before it works on your project.\n# Official language image. Look for the different tagged releases at:\n# https://hub.docker.com/r/library/ruby/tags/\nimage: \"ruby:2.3\"\n\n# Pick zero or more services to be used on all builds.\n# Only needed when using a docker container to run your tests in.\n# Check out: http://docs.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-service\nservices:\n - mysql:latest\n - redis:latest\n - postgres:latest\n\nvariables:\n POSTGRES_DB: database_name\n\n# Cache gems in between builds\ncache:\n paths:\n - vendor/ruby\n\n# This is a basic example for a gem or script which doesn't use\n# services such as redis or postgres\nbefore_script:\n - ruby -v # Print out ruby version for debugging\n # Uncomment next line if your rails app needs a JS runtime:\n # - apt-get update -q && apt-get install nodejs -yqq\n - gem install bundler --no-ri --no-rdoc # Bundler is not installed with the image\n - bundle install -j $(nproc) --path vendor # Install dependencies into ./vendor/ruby\n\n# Optional - Delete if not using `rubocop`\nrubocop:\n script:\n - rubocop\n\nrspec:\n script:\n - rspec spec\n\nrails:\n variables:\n DATABASE_URL: \"postgresql://postgres:postgres@postgres:5432/$POSTGRES_DB\"\n script:\n - bundle exec rake db:migrate\n - bundle exec rake db:seed\n - bundle exec rake test\n" +} +``` diff --git a/doc/api/licenses.md b/doc/api/templates/licenses.md index 855b0eab56f..ae7218cf1bd 100644 --- a/doc/api/licenses.md +++ b/doc/api/templates/licenses.md @@ -5,7 +5,7 @@ Get all license templates. ``` -GET /licenses +GET /templates/licenses ``` | Attribute | Type | Required | Description | @@ -13,7 +13,7 @@ GET /licenses | `popular` | boolean | no | If passed, returns only popular licenses | ```bash -curl https://gitlab.example.com/api/v3/licenses?popular=1 +curl https://gitlab.example.com/api/v3/templates/licenses?popular=1 ``` Example response: @@ -102,7 +102,7 @@ Get a single license template. You can pass parameters to replace the license placeholder. ``` -GET /licenses/:key +GET /templates/licenses/:key ``` | Attribute | Type | Required | Description | @@ -116,7 +116,7 @@ If you omit the `fullname` parameter but authenticate your request, the name of the authenticated user will be used to replace the copyright holder placeholder. ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/licenses/mit?project=My+Cool+Project +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/templates/licenses/mit?project=My+Cool+Project ``` Example response: diff --git a/doc/api/todos.md b/doc/api/todos.md index 29e73664410..a5e81801024 100644 --- a/doc/api/todos.md +++ b/doc/api/todos.md @@ -1,6 +1,6 @@ # Todos -**Note:** This feature was [introduced][ce-3188] in GitLab 8.10 +> [Introduced][ce-3188] in GitLab 8.10. ## Get a list of todos @@ -15,14 +15,14 @@ Parameters: | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `action` | string | no | The action to be filtered. Can be `assigned`, `mentioned`, `build_failed`, or `marked`. | +| `action` | string | no | The action to be filtered. Can be `assigned`, `mentioned`, `build_failed`, `marked`, or `approval_required`. | | `author_id` | integer | no | The ID of an author | | `project_id` | integer | no | The ID of a project | | `state` | string | no | The state of the todo. Can be either `pending` or `done` | | `type` | string | no | The type of a todo. Can be either `Issue` or `MergeRequest` | ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/todos +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/todos ``` Example Response: @@ -44,7 +44,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" + "web_url": "https://gitlab.example.com/root" }, "action_name": "marked", "target_type": "MergeRequest", @@ -67,7 +67,7 @@ Example Response: "id": 12, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" + "web_url": "https://gitlab.example.com/craig_rutherford" }, "assignee": { "name": "Administrator", @@ -75,7 +75,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" + "web_url": "https://gitlab.example.com/root" }, "source_project_id": 2, "target_project_id": 2, @@ -117,7 +117,7 @@ Example Response: "id": 12, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" + "web_url": "https://gitlab.example.com/craig_rutherford" }, "action_name": "assigned", "target_type": "MergeRequest", @@ -140,7 +140,7 @@ Example Response: "id": 12, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" + "web_url": "https://gitlab.example.com/craig_rutherford" }, "assignee": { "name": "Administrator", @@ -148,7 +148,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" + "web_url": "https://gitlab.example.com/root" }, "source_project_id": 2, "target_project_id": 2, @@ -194,7 +194,7 @@ Parameters: | `id` | integer | yes | The ID of a todo | ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/todos/130 +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/todos/130 ``` Example Response: @@ -215,7 +215,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" + "web_url": "https://gitlab.example.com/root" }, "action_name": "marked", "target_type": "MergeRequest", @@ -238,7 +238,7 @@ Example Response: "id": 12, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" + "web_url": "https://gitlab.example.com/craig_rutherford" }, "assignee": { "name": "Administrator", @@ -246,7 +246,7 @@ Example Response: "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" + "web_url": "https://gitlab.example.com/root" }, "source_project_id": 2, "target_project_id": 2, @@ -277,168 +277,20 @@ Example Response: ## Mark all todos as done -Marks all pending todos for the current user as done. All todos marked as done -are returned in the response. +Marks all pending todos for the current user as done. It returns the number of marked todos. ``` DELETE /todos ``` ```bash -curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/todos +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/todos ``` Example Response: ```json -[ - { - "id": 102, - "project": { - "id": 2, - "name": "Gitlab Ce", - "name_with_namespace": "Gitlab Org / Gitlab Ce", - "path": "gitlab-ce", - "path_with_namespace": "gitlab-org/gitlab-ce" - }, - "author": { - "name": "Administrator", - "username": "root", - "id": 1, - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" - }, - "action_name": "marked", - "target_type": "MergeRequest", - "target": { - "id": 34, - "iid": 7, - "project_id": 2, - "title": "Dolores in voluptatem tenetur praesentium omnis repellendus voluptatem quaerat.", - "description": "Et ea et omnis illum cupiditate. Dolor aspernatur tenetur ducimus facilis est nihil. Quo esse cupiditate molestiae illo corrupti qui quidem dolor.", - "state": "opened", - "created_at": "2016-06-17T07:49:24.419Z", - "updated_at": "2016-06-17T07:52:43.484Z", - "target_branch": "tutorials_git_tricks", - "source_branch": "DNSBL_docs", - "upvotes": 0, - "downvotes": 0, - "author": { - "name": "Maxie Medhurst", - "username": "craig_rutherford", - "id": 12, - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" - }, - "assignee": { - "name": "Administrator", - "username": "root", - "id": 1, - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" - }, - "source_project_id": 2, - "target_project_id": 2, - "labels": [], - "work_in_progress": false, - "milestone": { - "id": 32, - "iid": 2, - "project_id": 2, - "title": "v1.0", - "description": "Assumenda placeat ea voluptatem voluptate qui.", - "state": "active", - "created_at": "2016-06-17T07:47:34.163Z", - "updated_at": "2016-06-17T07:47:34.163Z", - "due_date": null - }, - "merge_when_build_succeeds": false, - "merge_status": "cannot_be_merged", - "subscribed": true, - "user_notes_count": 7 - }, - "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ce/merge_requests/7", - "body": "Dolores in voluptatem tenetur praesentium omnis repellendus voluptatem quaerat.", - "state": "done", - "created_at": "2016-06-17T07:52:35.225Z" - }, - { - "id": 98, - "project": { - "id": 2, - "name": "Gitlab Ce", - "name_with_namespace": "Gitlab Org / Gitlab Ce", - "path": "gitlab-ce", - "path_with_namespace": "gitlab-org/gitlab-ce" - }, - "author": { - "name": "Maxie Medhurst", - "username": "craig_rutherford", - "id": 12, - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" - }, - "action_name": "assigned", - "target_type": "MergeRequest", - "target": { - "id": 34, - "iid": 7, - "project_id": 2, - "title": "Dolores in voluptatem tenetur praesentium omnis repellendus voluptatem quaerat.", - "description": "Et ea et omnis illum cupiditate. Dolor aspernatur tenetur ducimus facilis est nihil. Quo esse cupiditate molestiae illo corrupti qui quidem dolor.", - "state": "opened", - "created_at": "2016-06-17T07:49:24.419Z", - "updated_at": "2016-06-17T07:52:43.484Z", - "target_branch": "tutorials_git_tricks", - "source_branch": "DNSBL_docs", - "upvotes": 0, - "downvotes": 0, - "author": { - "name": "Maxie Medhurst", - "username": "craig_rutherford", - "id": 12, - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/a0d477b3ea21970ce6ffcbb817b0b435?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/craig_rutherford" - }, - "assignee": { - "name": "Administrator", - "username": "root", - "id": 1, - "state": "active", - "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", - "web_url": "https://gitlab.example.com/u/root" - }, - "source_project_id": 2, - "target_project_id": 2, - "labels": [], - "work_in_progress": false, - "milestone": { - "id": 32, - "iid": 2, - "project_id": 2, - "title": "v1.0", - "description": "Assumenda placeat ea voluptatem voluptate qui.", - "state": "active", - "created_at": "2016-06-17T07:47:34.163Z", - "updated_at": "2016-06-17T07:47:34.163Z", - "due_date": null - }, - "merge_when_build_succeeds": false, - "merge_status": "cannot_be_merged", - "subscribed": true, - "user_notes_count": 7 - }, - "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ce/merge_requests/7", - "body": "Dolores in voluptatem tenetur praesentium omnis repellendus voluptatem quaerat.", - "state": "done", - "created_at": "2016-06-17T07:49:24.624Z" - }, -] +3 ``` [ce-3188]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3188 diff --git a/doc/api/users.md b/doc/api/users.md index 7e848586dbd..a50ba5432fe 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -20,7 +20,7 @@ GET /users "name": "John Smith", "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg", - "web_url": "http://localhost:3000/u/john_smith" + "web_url": "http://localhost:3000/john_smith" }, { "id": 2, @@ -28,7 +28,7 @@ GET /users "name": "Jack Smith", "state": "blocked", "avatar_url": "http://gravatar.com/../e32131cd8.jpeg", - "web_url": "http://localhost:3000/u/jack_smith" + "web_url": "http://localhost:3000/jack_smith" } ] ``` @@ -48,7 +48,7 @@ GET /users "name": "John Smith", "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/index.jpg", - "web_url": "http://localhost:3000/u/john_smith", + "web_url": "http://localhost:3000/john_smith", "created_at": "2012-05-23T08:00:58Z", "is_admin": false, "bio": null, @@ -57,6 +57,7 @@ GET /users "linkedin": "", "twitter": "", "website_url": "", + "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", "theme_id": 1, @@ -80,7 +81,7 @@ GET /users "name": "Jack Smith", "state": "blocked", "avatar_url": "http://localhost:3000/uploads/user/avatar/2/index.jpg", - "web_url": "http://localhost:3000/u/jack_smith", + "web_url": "http://localhost:3000/jack_smith", "created_at": "2012-05-23T08:01:01Z", "is_admin": false, "bio": null, @@ -89,6 +90,7 @@ GET /users "linkedin": "", "twitter": "", "website_url": "", + "organization": "", "last_sign_in_at": null, "confirmed_at": "2012-05-30T16:53:06.148Z", "theme_id": 1, @@ -139,7 +141,7 @@ Parameters: "name": "John Smith", "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg", - "web_url": "http://localhost:3000/u/john_smith", + "web_url": "http://localhost:3000/john_smith", "created_at": "2012-05-23T08:00:58Z", "is_admin": false, "bio": null, @@ -147,7 +149,8 @@ Parameters: "skype": "", "linkedin": "", "twitter": "", - "website_url": "" + "website_url": "", + "organization": "" } ``` @@ -169,7 +172,7 @@ Parameters: "name": "John Smith", "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/index.jpg", - "web_url": "http://localhost:3000/u/john_smith", + "web_url": "http://localhost:3000/john_smith", "created_at": "2012-05-23T08:00:58Z", "is_admin": false, "bio": null, @@ -178,6 +181,7 @@ Parameters: "linkedin": "", "twitter": "", "website_url": "", + "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", "theme_id": 1, @@ -214,6 +218,7 @@ Parameters: - `linkedin` (optional) - LinkedIn - `twitter` (optional) - Twitter account - `website_url` (optional) - Website URL +- `organization` (optional) - Organization name - `projects_limit` (optional) - Number of projects user can create - `extern_uid` (optional) - External UID - `provider` (optional) - External provider name @@ -242,6 +247,7 @@ Parameters: - `linkedin` - LinkedIn - `twitter` - Twitter account - `website_url` - Website URL +- `organization` - Organization name - `projects_limit` - Limit projects each user can create - `extern_uid` - External UID - `provider` - External provider name @@ -287,7 +293,7 @@ GET /user "name": "John Smith", "state": "active", "avatar_url": "http://localhost:3000/uploads/user/avatar/1/index.jpg", - "web_url": "http://localhost:3000/u/john_smith", + "web_url": "http://localhost:3000/john_smith", "created_at": "2012-05-23T08:00:58Z", "is_admin": false, "bio": null, @@ -296,6 +302,7 @@ GET /user "linkedin": "", "twitter": "", "website_url": "", + "organization": "", "last_sign_in_at": "2012-06-01T11:41:01Z", "confirmed_at": "2012-05-23T09:05:22Z", "theme_id": 1, @@ -310,8 +317,7 @@ GET /user "can_create_group": true, "can_create_project": true, "two_factor_enabled": true, - "external": false, - "private_token": "dd34asd13as" + "external": false } ``` @@ -621,3 +627,149 @@ Parameters: Will return `200 OK` on success, `404 User Not Found` is user cannot be found or `403 Forbidden` when trying to unblock a user blocked by LDAP synchronization. + +### Get user contribution events + +Get the contribution events for the specified user, sorted from newest to oldest. + +``` +GET /users/:id/events +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/users/:id/events +``` + +Example response: + +```json +[ + { + "title": null, + "project_id": 15, + "action_name": "closed", + "target_id": 830, + "target_type": "Issue", + "author_id": 1, + "data": null, + "target_title": "Public project search field", + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/root" + }, + "author_username": "root" + }, + { + "title": null, + "project_id": 15, + "action_name": "opened", + "target_id": null, + "target_type": null, + "author_id": 1, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/root" + }, + "author_username": "john", + "data": { + "before": "50d4420237a9de7be1304607147aec22e4a14af7", + "after": "c5feabde2d8cd023215af4d2ceeb7a64839fc428", + "ref": "refs/heads/master", + "user_id": 1, + "user_name": "Dmitriy Zaporozhets", + "repository": { + "name": "gitlabhq", + "url": "git@dev.gitlab.org:gitlab/gitlabhq.git", + "description": "GitLab: self hosted Git management software. \r\nDistributed under the MIT License.", + "homepage": "https://dev.gitlab.org/gitlab/gitlabhq" + }, + "commits": [ + { + "id": "c5feabde2d8cd023215af4d2ceeb7a64839fc428", + "message": "Add simple search to projects in public area", + "timestamp": "2013-05-13T18:18:08+00:00", + "url": "https://dev.gitlab.org/gitlab/gitlabhq/commit/c5feabde2d8cd023215af4d2ceeb7a64839fc428", + "author": { + "name": "Dmitriy Zaporozhets", + "email": "dmitriy.zaporozhets@gmail.com" + } + } + ], + "total_commits_count": 1 + }, + "target_title": null + }, + { + "title": null, + "project_id": 15, + "action_name": "closed", + "target_id": 840, + "target_type": "Issue", + "author_id": 1, + "data": null, + "target_title": "Finish & merge Code search PR", + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/root" + }, + "author_username": "root" + }, + { + "title": null, + "project_id": 15, + "action_name": "commented on", + "target_id": 1312, + "target_type": "Note", + "author_id": 1, + "data": null, + "target_title": null, + "created_at": "2015-12-04T10:33:58.089Z", + "note": { + "id": 1312, + "body": "What an awesome day!", + "attachment": null, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2015-12-04T10:33:56.698Z", + "system": false, + "upvote": false, + "downvote": false, + "noteable_id": 377, + "noteable_type": "Issue" + }, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/root" + }, + "author_username": "root" + } +] +``` diff --git a/doc/api/version.md b/doc/api/version.md new file mode 100644 index 00000000000..287d17cf97f --- /dev/null +++ b/doc/api/version.md @@ -0,0 +1,23 @@ +# Version API + +>**Note:** This feature was introduced in GitLab 8.13 + +Retrieve version information for this GitLab instance. Responds `200 OK` for +authenticated users. + +``` +GET /version +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/version +``` + +Example response: + +```json +{ + "version": "8.13.0-pre", + "revision": "4e963fe" +} +``` diff --git a/doc/ci/README.md b/doc/ci/README.md index a9d407528e8..341bc85a16a 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -14,7 +14,9 @@ - [Use variables in your `.gitlab-ci.yml`](variables/README.md) - [Use SSH keys in your build environment](ssh_keys/README.md) - [Trigger builds through the API](triggers/README.md) -- [Build artifacts](build_artifacts/README.md) -- [User permissions](permissions/README.md) +- [Build artifacts](../user/project/builds/artifacts.md) +- [User permissions](../user/permissions.md#gitlab-ci) +- [Build permissions](../user/permissions.md#build-permissions) - [API](../api/ci/README.md) - [CI services (linked docker containers)](services/README.md) +- [**New CI build permissions model**](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md index 9553bb11e9d..05605f10fb4 100644 --- a/doc/ci/build_artifacts/README.md +++ b/doc/ci/build_artifacts/README.md @@ -1,175 +1,4 @@ -# Introduction to build artifacts +This document was moved to: -Artifacts is a list of files and directories which are attached to a build -after it completes successfully. This feature is enabled by default in all GitLab installations. - -_If you are searching for ways to use artifacts, jump to -[Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._ - -Since GitLab 8.2 and [GitLab Runner] 0.7.0, build artifacts that are created by -GitLab Runner are uploaded to GitLab and are downloadable as a single archive -(`tar.gz`) using the GitLab UI. - -Starting from GitLab 8.4 and GitLab Runner 1.0, the artifacts archive format -changed to `ZIP`, and it is now possible to browse its contents, with the added -ability of downloading the files separately. - -**Note:** -The artifacts browser will be available only for new artifacts that are sent -to GitLab using GitLab Runner version 1.0 and up. It will not be possible to -browse old artifacts already uploaded to GitLab. - -## Disabling build artifacts - -To disable artifacts site-wide, follow the steps below. - ---- - -**In Omnibus installations:** - -1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - - ```ruby - gitlab_rails['artifacts_enabled'] = false - ``` - -1. Save the file and [reconfigure GitLab][] for the changes to take effect. - ---- - -**In installations from source:** - -1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - - ```yaml - artifacts: - enabled: false - ``` - -1. Save the file and [restart GitLab][] for the changes to take effect. - -## Defining artifacts in `.gitlab-ci.yml` - -A simple example of using the artifacts definition in `.gitlab-ci.yml` would be -the following: - -```yaml -pdf: - script: xelatex mycv.tex - artifacts: - paths: - - mycv.pdf -``` - -A job named `pdf` calls the `xelatex` command in order to build a pdf file from -the latex source file `mycv.tex`. We then define the `artifacts` paths which in -turn are defined with the `paths` keyword. All paths to files and directories -are relative to the repository that was cloned during the build. - -For more examples on artifacts, follow the -[separate artifacts yaml documentation](../yaml/README.md#artifacts). - -## Storing build artifacts - -After a successful build, GitLab Runner uploads an archive containing the build -artifacts to GitLab. - -To change the location where the artifacts are stored, follow the steps below. - ---- - -**In Omnibus installations:** - -_The artifacts are stored by default in -`/var/opt/gitlab/gitlab-rails/shared/artifacts`._ - -1. To change the storage path for example to `/mnt/storage/artifacts`, edit - `/etc/gitlab/gitlab.rb` and add the following line: - - ```ruby - gitlab_rails['artifacts_path'] = "/mnt/storage/artifacts" - ``` - -1. Save the file and [reconfigure GitLab][] for the changes to take effect. - ---- - -**In installations from source:** - -_The artifacts are stored by default in -`/home/git/gitlab/shared/artifacts`._ - -1. To change the storage path for example to `/mnt/storage/artifacts`, edit - `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - - ```yaml - artifacts: - enabled: true - path: /mnt/storage/artifacts - ``` - -1. Save the file and [restart GitLab][] for the changes to take effect. - -## Browsing build artifacts - -When GitLab receives an artifacts archive, an archive metadata file is also -generated. This metadata file describes all the entries that are located in the -artifacts archive itself. The metadata file is in a binary format, with -additional GZIP compression. - -GitLab does not extract the artifacts archive in order to save space, memory -and disk I/O. It instead inspects the metadata file which contains all the -relevant information. This is especially important when there is a lot of -artifacts, or an archive is a very large file. - ---- - -After a successful build, if you visit the build's specific page, you can see -that there are two buttons. - -One is for downloading the artifacts archive and the other for browsing its -contents. - - - ---- - -The archive browser shows the name and the actual file size of each file in the -archive. If your artifacts contained directories, then you are also able to -browse inside them. - -Below you can see an image of three different file formats, as well as two -directories. - - - ---- - -## Downloading build artifacts - -If you need to download the whole archive, there are buttons in various places -inside GitLab that make that possible. - -1. While on the builds page, you can see the download icon for each build's - artifacts archive in the right corner - -1. While inside a specific build, you are presented with a download button - along with the one that browses the archive - -1. And finally, when browsing an archive you can see the download button at - the top right corner - ---- - -Note that GitLab does not extract the entire artifacts archive to send just a -single file to the user. - -When clicking on a specific file, [GitLab Workhorse] extracts it from the -archive and the download begins. - -This implementation saves space, memory and disk I/O. - -[gitlab runner]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner "GitLab Runner repository" -[reconfigure gitlab]: ../../administration/restart_gitlab.md "How to restart GitLab documentation" -[restart gitlab]: ../../administration/restart_gitlab.md "How to restart GitLab documentation" -[gitlab workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse "GitLab Workhorse repository" +- [user/project/builds/artifacts.md](../../user/project/builds/artifacts.md) - user guide +- [administration/build_artifacts.md](../../administration/build_artifacts.md) - administrator guide diff --git a/doc/ci/build_artifacts/img/build_artifacts_browser.png b/doc/ci/build_artifacts/img/build_artifacts_browser.png Binary files differdeleted file mode 100644 index 59cf2b8746b..00000000000 --- a/doc/ci/build_artifacts/img/build_artifacts_browser.png +++ /dev/null diff --git a/doc/ci/build_artifacts/img/build_artifacts_browser_button.png b/doc/ci/build_artifacts/img/build_artifacts_browser_button.png Binary files differdeleted file mode 100644 index 7801c2e6fa6..00000000000 --- a/doc/ci/build_artifacts/img/build_artifacts_browser_button.png +++ /dev/null diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index ee924323d96..79bbe8421c6 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -38,7 +38,7 @@ GitLab Runner then executes build scripts as the `gitlab-runner` user. $ sudo gitlab-ci-multi-runner register -n \ --url https://gitlab.com/ci \ --registration-token REGISTRATION_TOKEN \ - --executor shell + --executor shell \ --description "My Runner" ``` diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index a849905ac6b..aba77490915 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -37,7 +37,7 @@ The registered runner will use the `ruby:2.1` docker image and will run two services, `postgres:latest` and `mysql:latest`, both of which will be accessible during the build process. -## What is image +## What is an image The `image` keyword is the name of the docker image that is present in the local Docker Engine (list all images with `docker images`) or any image that @@ -47,7 +47,7 @@ Hub please read the [Docker Fundamentals][] documentation. In short, with `image` we refer to the docker image, which will be used to create a container on which your build will run. -## What is service +## What is a service The `services` keyword defines just another docker image that is run during your build and is linked to the docker image that the `image` keyword defines. @@ -61,7 +61,7 @@ time the project is built. You can see some widely used services examples in the relevant documentation of [CI services examples](../services/README.md). -### How is service linked to the build +### How services are linked to the build To better understand how the container linking works, read [Linking containers together][linking-containers]. @@ -221,7 +221,7 @@ time. *Note: The following commands are run without root privileges. You should be able to run docker with your regular user account.* -First start with creating a file named `build script`: +First start with creating a file named `build_script`: ```bash cat <<EOF > build_script diff --git a/doc/ci/environments.md b/doc/ci/environments.md index d85b8a34ced..e070302fb82 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -14,6 +14,19 @@ Defining environments in a project's `.gitlab-ci.yml` lets developers track Deployments are created when [jobs] deploy versions of code to [environments]. +### Checkout deployments locally + +Since 8.13, a reference in the git repository is saved for each deployment. So +knowing what the state is of your current environments is only a `git fetch` +away. + +In your git config, append the `[remote "<your-remote>"]` block with an extra +fetch line: + +``` +fetch = +refs/environments/*:refs/remotes/origin/environments/* +``` + ## Defining environments You can create and delete environments manually in the web interface, but we diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index c134106bfd0..ffc310ec8c7 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -1,17 +1,21 @@ # CI Examples +A collection of `.gitlab-ci.yml` files is maintained at the [GitLab CI Yml project][gitlab-ci-templates]. +If your favorite programming language or framework are missing we would love your help by sending a merge request +with a `.gitlab-ci.yml`. + +Apart from those, here is an collection of tutorials and guides on setting up your CI pipeline: + - [Testing a PHP application](php.md) - [Test and deploy a Ruby application to Heroku](test-and-deploy-ruby-application-to-heroku.md) - [Test and deploy a Python application to Heroku](test-and-deploy-python-application-to-heroku.md) - [Test a Clojure application](test-clojure-application.md) - [Test a Scala application](test-scala-application.md) +- [Test a Phoenix application](test-phoenix-application.md) - [Using `dpl` as deployment tool](deployment/README.md) -- Help your favorite programming language and GitLab by sending a merge request - with a guide for that language. - -## Outside the documentation - +- [Example project that shows how to use Review Apps](https://gitlab.com/gitlab-examples/review-apps-nginx/) - [Blog post about using GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) -- [Repo's with examples for various languages](https://gitlab.com/groups/gitlab-examples) +- [Repositories with examples for various languages](https://gitlab.com/groups/gitlab-examples) - [The .gitlab-ci.yml file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml) -- [A collection of useful .gitlab-ci.yml templates](https://gitlab.com/gitlab-org/gitlab-ci-yml) + +[gitlab-ci-templates]: https://gitlab.com/gitlab-org/gitlab-ci-yml diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index bfafcc44d66..175e9d79904 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -49,7 +49,7 @@ apt-get update -yqq apt-get install git -yqq # Install phpunit, the tool that we will use for testing -curl -Lo /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar +curl --location --output /usr/local/bin/phpunit https://phar.phpunit.de/phpunit.phar chmod +x /usr/local/bin/phpunit # Install mysql driver @@ -235,7 +235,7 @@ cache: before_script: # Install composer dependencies -- curl -sS https://getcomposer.org/installer | php +- curl --silent --show-error https://getcomposer.org/installer | php - php composer.phar install ... diff --git a/doc/ci/examples/test-phoenix-application.md b/doc/ci/examples/test-phoenix-application.md new file mode 100644 index 00000000000..150698ca04b --- /dev/null +++ b/doc/ci/examples/test-phoenix-application.md @@ -0,0 +1,56 @@ +## Test a Phoenix application + +This example demonstrates the integration of Gitlab CI with Phoenix, Elixir and +Postgres. + +### Add `.gitlab-ci.yml` file to project + +The following `.gitlab-ci.yml` should be added in the root of your +repository to trigger CI: + +```yaml +image: elixir:1.3 + +services: + - postgres:9.6 + +variables: + MIX_ENV: "test" + +before_script: + # Setup phoenix dependencies + - apt-get update + - apt-get install -y postgresql-client + - mix local.hex --force + - mix deps.get --only test + - mix ecto.reset + +test: + script: + - mix test +``` + +The variables will set the Mix environment to "test". The +`before_script` will install `psql`, some Phoenix dependencies, and will also +run your migrations. + +Finally, the test `script` will run your tests. + +### Update the Config Settings + +In `config/test.exs`, update the database hostname: + +```elixir +config :my_app, MyApp.Repo, + hostname: if(System.get_env("CI"), do: "postgres", else: "localhost"), +``` + +### Add the Migrations Folder + +If you do not have any migrations yet, you will need to create an empty +`.gitkeep` file in `priv/repo/migrations`. + +### Sources + +- https://medium.com/@nahtnam/using-phoenix-on-gitlab-ci-5a51eec81142 +- https://davejlong.com/ci-with-phoenix-and-gitlab/ diff --git a/doc/ci/permissions/README.md b/doc/ci/permissions/README.md index d77061c14cd..42eb59f84c8 100644 --- a/doc/ci/permissions/README.md +++ b/doc/ci/permissions/README.md @@ -1,24 +1,3 @@ # Users Permissions -GitLab CI relies on user's role on the GitLab. There are three permissions levels on GitLab CI: admin, master, developer, other. - -Admin user can perform any actions on GitLab CI in scope of instance and project. Also user with admin permission can use admin interface. - - - - -| Action | Guest, Reporter | Developer | Master | Admin | -|---------------------------------------|-----------------|-------------|----------|--------| -| See commits and builds | ✓ | ✓ | ✓ | ✓ | -| Retry or cancel build | | ✓ | ✓ | ✓ | -| Remove project | | | ✓ | ✓ | -| Create project | | | ✓ | ✓ | -| Change project configuration | | | ✓ | ✓ | -| Add specific runners | | | ✓ | ✓ | -| Add shared runners | | | | ✓ | -| See events in the system | | | | ✓ | -| Admin interface | | | | ✓ | - - - - +This document was moved to [user/permissions.md](../../user/permissions.md#gitlab-ci). diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 48a9f994759..729c1dc8c0d 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -5,7 +5,7 @@ Introduced in GitLab 8.8. ## Pipelines -A pipeline is a group of [builds] that get executed in [stages] (batches). All +A pipeline is a group of [builds] that get executed in [stages] \(batches). All of the builds in a stage are executed in parallel (if there are enough concurrent [runners]), and if they all succeed, the pipeline moves on to the next stage. If one of the builds fails, the next stage is not (usually) @@ -31,6 +31,45 @@ project. ## Seeing build status Clicking on a pipeline will show the builds that were run for that pipeline. +Clicking on an individual build will show you its build trace, and allow you to +cancel the build, retry it, or erase the build trace. + +## Badges + +There are build status and test coverage report badges available. + +Go to pipeline settings to see available badges and code you can use to embed +badges in the `README.md` or your website. + +### Build status badge + +You can access a build status badge image using following link: + +``` +http://example.gitlab.com/namespace/project/badges/branch/build.svg +``` + +### Test coverage report badge + +GitLab makes it possible to define the regular expression for coverage report, +that each build log will be matched against. This means that each build in the +pipeline can have the test coverage percentage value defined. + +You can access test coverage badge using following link: + +``` +http://example.gitlab.com/namespace/project/badges/branch/coverage.svg +``` + +If you would like to get the coverage report from the specific job, you can add +a `job=coverage_job_name` parameter to the URL. For example, it is possible to +use following Markdown code to embed the est coverage report into `README.md`: + +```markdown + +``` + +The latest successful pipeline will be used to read the test coverage value. [builds]: #builds [jobs]: yaml/README.md#jobs diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 7fa1a478f34..c40cdd55ea5 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -105,7 +105,8 @@ What is important is that each job is run independently from each other. If you want to check whether your `.gitlab-ci.yml` file is valid, there is a Lint tool under the page `/ci/lint` of your GitLab instance. You can also find -the link under **Settings > CI settings** in your project. +a "CI Lint" button to go to this page under **Pipelines > Pipelines** and +**Pipelines > Builds** in your project. For more information and a complete `.gitlab-ci.yml` syntax, please read [the documentation on .gitlab-ci.yml](../yaml/README.md). @@ -218,22 +219,14 @@ project's settings. For more information read the [Builds emails service documentation](../../project_services/builds_emails.md). -## Builds badge - -You can access a builds badge image using following link: - -``` -http://example.gitlab.com/namespace/project/badges/branch/build.svg -``` - -Awesome! You started using CI in GitLab! - ## Examples Visit the [examples README][examples] to see a list of examples using GitLab CI with various languages. -[runner-install]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tree/master#installation +Awesome! You started using CI in GitLab! + +[runner-install]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tree/master#install-gitlab-runner [blog-ci]: https://about.gitlab.com/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/ [examples]: ../examples/README.md [ci]: https://about.gitlab.com/gitlab-ci/ diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 7c0fb225dac..b858029d25e 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -30,7 +30,8 @@ This is the universal solution which works with any type of executor ## SSH keys when using the Docker executor You will first need to create an SSH key pair. For more information, follow the -instructions to [generate an SSH key](../../ssh/README.md). +instructions to [generate an SSH key](../../ssh/README.md). Do not add a comment +to the SSH key, or the `before_script` will prompt for a passphrase. Then, create a new **Secret Variable** in your project settings on GitLab following **Settings > Variables**. As **Key** add the name `SSH_PRIVATE_KEY` diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 5c316510d0e..84048f1d25f 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -1,6 +1,10 @@ # Triggering Builds through the API -_**Note:** This feature was [introduced][ci-229] in GitLab CE 7.14_ +> [Introduced][ci-229] in GitLab CE 7.14. + +> **Note**: +GitLab 8.12 has a completely redesigned build permissions system. +Read all about the [new model and its implications](../../user/project/new_ci_build_permissions_model.md#build-triggers). Triggers can be used to force a rebuild of a specific branch, tag or commit, with an API call. @@ -77,9 +81,9 @@ See the [Examples](#examples) section below for more details. Using cURL you can trigger a rebuild with minimal effort, for example: ```bash -curl -X POST \ - -F token=TOKEN \ - -F ref=master \ +curl --request POST \ + --form token=TOKEN \ + --form ref=master \ https://gitlab.example.com/api/v3/projects/9/trigger/builds ``` @@ -88,7 +92,7 @@ In this case, the project with ID `9` will get rebuilt on `master` branch. Alternatively, you can pass the `token` and `ref` arguments in the query string: ```bash -curl -X POST \ +curl --request POST \ "https://gitlab.example.com/api/v3/projects/9/trigger/builds?token=TOKEN&ref=master" ``` @@ -103,7 +107,7 @@ need to add in project's A `.gitlab-ci.yml`: build_docs: stage: deploy script: - - "curl -X POST -F token=TOKEN -F ref=master https://gitlab.example.com/api/v3/projects/9/trigger/builds" + - "curl --request POST --form token=TOKEN --form ref=master https://gitlab.example.com/api/v3/projects/9/trigger/builds" only: - tags ``` @@ -158,10 +162,10 @@ You can then trigger a rebuild while you pass the `UPLOAD_TO_S3` variable and the script of the `upload_package` job will run: ```bash -curl -X POST \ - -F token=TOKEN \ - -F ref=master \ - -F "variables[UPLOAD_TO_S3]=true" \ +curl --request POST \ + --form token=TOKEN \ + --form ref=master \ + --form "variables[UPLOAD_TO_S3]=true" \ https://gitlab.example.com/api/v3/projects/9/trigger/builds ``` @@ -172,7 +176,7 @@ in conjunction with cron. The example below triggers a build on the `master` branch of project with ID `9` every night at `00:30`: ```bash -30 0 * * * curl -X POST -F token=TOKEN -F ref=master https://gitlab.example.com/api/v3/projects/9/trigger/builds +30 0 * * * curl --request POST --form token=TOKEN --form ref=master https://gitlab.example.com/api/v3/projects/9/trigger/builds ``` [ci-229]: https://gitlab.com/gitlab-org/gitlab-ci/merge_requests/229 diff --git a/doc/ci/triggers/img/builds_page.png b/doc/ci/triggers/img/builds_page.png Binary files differindex 2dee8ee6107..c2cf4b1852c 100644 --- a/doc/ci/triggers/img/builds_page.png +++ b/doc/ci/triggers/img/builds_page.png diff --git a/doc/ci/triggers/img/trigger_single_build.png b/doc/ci/triggers/img/trigger_single_build.png Binary files differindex baf3fc183d8..fa86f0fee3d 100644 --- a/doc/ci/triggers/img/trigger_single_build.png +++ b/doc/ci/triggers/img/trigger_single_build.png diff --git a/doc/ci/triggers/img/trigger_variables.png b/doc/ci/triggers/img/trigger_variables.png Binary files differindex 908355c33a5..b2fcc65d304 100644 --- a/doc/ci/triggers/img/trigger_variables.png +++ b/doc/ci/triggers/img/trigger_variables.png diff --git a/doc/ci/triggers/img/triggers_page.png b/doc/ci/triggers/img/triggers_page.png Binary files differindex 69cec5cdebf..438f285ae2d 100644 --- a/doc/ci/triggers/img/triggers_page.png +++ b/doc/ci/triggers/img/triggers_page.png diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 137b080a8f7..a4c3a731a20 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -18,25 +18,39 @@ The `API_TOKEN` will take the Secure Variable value: `SECURE`. ### Predefined variables (Environment Variables) -| Variable | Runner | Description | -|-------------------------|-----|--------| -| **CI** | 0.4 | Mark that build is executed in CI environment | -| **GITLAB_CI** | all | Mark that build is executed in GitLab CI environment | -| **CI_SERVER** | all | Mark that build is executed in CI environment | -| **CI_SERVER_NAME** | all | CI server that is used to coordinate builds | -| **CI_SERVER_VERSION** | all | Not yet defined | -| **CI_SERVER_REVISION** | all | Not yet defined | -| **CI_BUILD_REF** | all | The commit revision for which project is built | -| **CI_BUILD_TAG** | 0.5 | The commit tag name. Present only when building tags. | -| **CI_BUILD_NAME** | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | -| **CI_BUILD_STAGE** | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | -| **CI_BUILD_REF_NAME** | all | The branch or tag name for which project is built | -| **CI_BUILD_ID** | all | The unique id of the current build that GitLab CI uses internally | -| **CI_BUILD_REPO** | all | The URL to clone the Git repository | -| **CI_BUILD_TRIGGERED** | 0.5 | The flag to indicate that build was [triggered] | -| **CI_BUILD_TOKEN** | 1.2 | Token used for authenticating with the GitLab Container Registry | -| **CI_PROJECT_ID** | all | The unique id of the current project that GitLab CI uses internally | -| **CI_PROJECT_DIR** | all | The full path where the repository is cloned and where the build is ran | +| Variable | GitLab | Runner | Description | +|-------------------------|--------|--------|-------------| +| **CI** | all | 0.4 | Mark that build is executed in CI environment | +| **GITLAB_CI** | all | all | Mark that build is executed in GitLab CI environment | +| **CI_SERVER** | all | all | Mark that build is executed in CI environment | +| **CI_SERVER_NAME** | all | all | The name of CI server that is used to coordinate builds | +| **CI_SERVER_VERSION** | all | all | GitLab version that is used to schedule builds | +| **CI_SERVER_REVISION** | all | all | GitLab revision that is used to schedule builds | +| **CI_BUILD_ID** | all | all | The unique id of the current build that GitLab CI uses internally | +| **CI_BUILD_REF** | all | all | The commit revision for which project is built | +| **CI_BUILD_TAG** | all | 0.5 | The commit tag name. Present only when building tags. | +| **CI_BUILD_NAME** | all | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | +| **CI_BUILD_STAGE** | all | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | +| **CI_BUILD_REF_NAME** | all | all | The branch or tag name for which project is built | +| **CI_BUILD_REPO** | all | all | The URL to clone the Git repository | +| **CI_BUILD_TRIGGERED** | all | 0.5 | The flag to indicate that build was [triggered] | +| **CI_BUILD_MANUAL** | 8.12 | all | The flag to indicate that build was manually started | +| **CI_BUILD_TOKEN** | all | 1.2 | Token used for authenticating with the GitLab Container Registry | +| **CI_PIPELINE_ID** | 8.10 | 0.5 | The unique id of the current pipeline that GitLab CI uses internally | +| **CI_PROJECT_ID** | all | all | The unique id of the current project that GitLab CI uses internally | +| **CI_PROJECT_NAME** | 8.10 | 0.5 | The project name that is currently being built | +| **CI_PROJECT_NAMESPACE**| 8.10 | 0.5 | The project namespace (username or groupname) that is currently being built | +| **CI_PROJECT_PATH** | 8.10 | 0.5 | The namespace with project name | +| **CI_PROJECT_URL** | 8.10 | 0.5 | The HTTP address to access project | +| **CI_PROJECT_DIR** | all | all | The full path where the repository is cloned and where the build is run | +| **CI_REGISTRY** | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry | +| **CI_REGISTRY_IMAGE** | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | +| **CI_RUNNER_ID** | 8.10 | 0.5 | The unique id of runner being used | +| **CI_RUNNER_DESCRIPTION** | 8.10 | 0.5 | The description of the runner as saved in GitLab | +| **CI_RUNNER_TAGS** | 8.10 | 0.5 | The defined runner tags | +| **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | +| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the build | +| **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the build | **Some of the variables are only available when using runner with at least defined version.** @@ -46,18 +60,31 @@ Example values: export CI_BUILD_ID="50" export CI_BUILD_REF="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_BUILD_REF_NAME="master" -export CI_BUILD_REPO="https://gitlab.com/gitlab-org/gitlab-ce.git" +export CI_BUILD_REPO="https://gitab-ci-token:abcde-1234ABCD5678ef@gitlab.com/gitlab-org/gitlab-ce.git" export CI_BUILD_TAG="1.0.0" export CI_BUILD_NAME="spec:other" export CI_BUILD_STAGE="test" +export CI_BUILD_MANUAL="true" export CI_BUILD_TRIGGERED="true" export CI_BUILD_TOKEN="abcde-1234ABCD5678ef" -export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" +export CI_PIPELINE_ID="1000" export CI_PROJECT_ID="34" +export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" +export CI_PROJECT_NAME="gitlab-ce" +export CI_PROJECT_NAMESPACE="gitlab-org" +export CI_PROJECT_PATH="gitlab-org/gitlab-ce" +export CI_PROJECT_URL="https://gitlab.com/gitlab-org/gitlab-ce" +export CI_REGISTRY="registry.gitlab.com" +export CI_REGISTRY_IMAGE="registry.gitlab.com/gitlab-org/gitlab-ce" +export CI_RUNNER_ID="10" +export CI_RUNNER_DESCRIPTION="my runner" +export CI_RUNNER_TAGS="docker, linux" export CI_SERVER="yes" -export CI_SERVER_NAME="GitLab CI" -export CI_SERVER_REVISION="" -export CI_SERVER_VERSION="" +export CI_SERVER_NAME="GitLab" +export CI_SERVER_REVISION="70606bf" +export CI_SERVER_VERSION="8.9.0" +export GITLAB_USER_ID="42" +export GITLAB_USER_EMAIL="alexzander@sporer.com" ``` ### YAML-defined variables @@ -79,6 +106,39 @@ Variables can be defined at a global level, but also at a job level. More information about Docker integration can be found in [Using Docker Images](../docker/using_docker_images.md). +#### Debug tracing + +> **WARNING:** Enabling debug tracing can have severe security implications. The + output **will** contain the content of all your secure variables and any other + secrets! The output **will** be uploaded to the GitLab server and made visible + in build traces! + +By default, GitLab Runner hides most of the details of what it is doing when +processing a job. This behaviour keeps build traces short, and prevents secrets +from being leaked into the trace unless your script writes them to the screen. + +If a job isn't working as expected, this can make the problem difficult to +investigate; in these cases, you can enable debug tracing in `.gitlab-ci.yml`. +Available on GitLab Runner v1.7+, this feature enables the shell's execution +trace, resulting in a verbose build trace listing all commands that were run, +variables that were set, etc. + +Before enabling this, you should ensure builds are visible to +[team members only](../../../user/permissions.md#project-features). You should +also [erase](../pipelines.md#seeing-build-traces) all generated build traces +before making them visible again. + +To enable debug traces, set the `CI_DEBUG_TRACE` variable to `true`: + +```yaml +job1: + variables: + CI_DEBUG_TRACE: "true" +``` + +The [example project](https://gitlab.com/gitlab-examples/ci-debug-trace) +demonstrates a working configuration, including build trace examples. + ### User-defined variables (Secure Variables) **This feature requires GitLab Runner 0.4.0 or higher** diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index eb81267242e..5c0e1c44e3f 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -6,48 +6,6 @@ GitLab Runner to manage your project's builds. If you want a quick introduction to GitLab CI, follow our [quick start guide](../quick_start/README.md). ---- - -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - -- [.gitlab-ci.yml](#gitlab-ci-yml) - - [image and services](#image-and-services) - - [before_script](#before_script) - - [after_script](#after_script) - - [stages](#stages) - - [types](#types) - - [variables](#variables) - - [cache](#cache) - - [cache:key](#cache-key) -- [Jobs](#jobs) - - [script](#script) - - [stage](#stage) - - [only and except](#only-and-except) - - [job variables](#job-variables) - - [tags](#tags) - - [when](#when) - - [environment](#environment) - - [artifacts](#artifacts) - - [artifacts:name](#artifactsname) - - [artifacts:when](#artifactswhen) - - [artifacts:expire_in](#artifactsexpire_in) - - [dependencies](#dependencies) - - [before_script and after_script](#before_script-and-after_script) -- [Git Strategy](#git-strategy) -- [Shallow cloning](#shallow-cloning) -- [Hidden jobs](#hidden-jobs) -- [Special YAML features](#special-yaml-features) - - [Anchors](#anchors) -- [Validate the .gitlab-ci.yml](#validate-the-gitlab-ciyml) -- [Skipping builds](#skipping-builds) -- [Examples](#examples) - -<!-- END doctoc generated TOC please keep comment here to allow auto update --> - ---- - ## .gitlab-ci.yml From version 7.12, GitLab CI uses a [YAML](https://en.wikipedia.org/wiki/YAML) @@ -132,8 +90,7 @@ builds, including deploy builds. This can be an array or a multi-line string. ### after_script ->**Note:** -Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 (not yet released) +> Introduced in GitLab 8.7 and requires Gitlab Runner v1.2 `after_script` is used to define the command that will be run after for all builds. This has to be an array or a multi-line string. @@ -177,11 +134,10 @@ Alias for [stages](#stages). ### variables ->**Note:** -Introduced in GitLab Runner v0.5.0. +> Introduced in GitLab Runner v0.5.0. GitLab CI allows you to add variables to `.gitlab-ci.yml` that are set in the -build environment. The variables are stored in the git repository and are meant +build environment. The variables are stored in the Git repository and are meant to store non-sensitive project configuration, for example: ```yaml @@ -190,19 +146,25 @@ variables: ``` These variables can be later used in all executed commands and scripts. - The YAML-defined variables are also set to all created service containers, -thus allowing to fine tune them. +thus allowing to fine tune them. Variables can be also defined on a +[job level](#job-variables). -Variables can be also defined on [job level](#job-variables). +Except for the user defined variables, there are also the ones set up by the +Runner itself. One example would be `CI_BUILD_REF_NAME` which has the value of +the branch or tag name for which project is built. Apart from the variables +you can set in `.gitlab-ci.yml`, there are also the so called secret variables +which can be set in GitLab's UI. + +[Learn more about variables.][variables] ### cache ->**Note:** -Introduced in GitLab Runner v0.7.0. +> Introduced in GitLab Runner v0.7.0. `cache` is used to specify a list of files and directories which should be -cached between builds. +cached between builds. You can only use paths that are within the project +workspace. **By default the caching is enabled per-job and per-branch.** @@ -260,8 +222,7 @@ will be always present. For implementation details, please check GitLab Runner. #### cache:key ->**Note:** -Introduced in GitLab Runner v1.0.0. +> Introduced in GitLab Runner v1.0.0. The `key` directive allows you to define the affinity of caching between jobs, allowing to have a single cache for all jobs, @@ -351,7 +312,7 @@ job_name: | except | no | Defines a list of git refs for which build is not created | | tags | no | Defines a list of tags which are used to select Runner | | allow_failure | no | Allow build to fail. Failed build doesn't contribute to commit status | -| when | no | Define when to run build. Can be `on_success`, `on_failure` or `always` | +| when | no | Define when to run build. Can be `on_success`, `on_failure`, `always` or `manual` | | dependencies | no | Define other builds that a build depends on so that you can pass artifacts between them| | artifacts | no | Define list of build artifacts | | cache | no | Define list of files that should be cached between subsequent runs | @@ -377,6 +338,8 @@ job: - bundle exec rspec ``` +Sometimes, `script` commands will need to be wrapped in single or double quotes. For example, commands that contain a colon (`:`) need to be wrapped in quotes so that the YAML parser knows to interpret the whole thing as a string rather than a "key: value" pair. Be careful when using special characters (`:`, `{`, `}`, `[`, `]`, `,`, `&`, `*`, `#`, `?`, `|`, `-`, `<`, `>`, `=`, `!`, `%`, `@`, `` ` ``). + ### stage `stage` allows to group build into different stages. Builds of the same `stage` @@ -473,6 +436,39 @@ job: The specification above, will make sure that `job` is built by a Runner that has both `ruby` AND `postgres` tags defined. +### allow_failure + +`allow_failure` is used when you want to allow a build to fail without impacting +the rest of the CI suite. Failed builds don't contribute to the commit status. + +When enabled and the build fails, the pipeline will be successful/green for all +intents and purposes, but a "CI build passed with warnings" message will be +displayed on the merge request or commit or build page. This is to be used by +builds that are allowed to fail, but where failure indicates some other (manual) +steps should be taken elsewhere. + +In the example below, `job1` and `job2` will run in parallel, but if `job1` +fails, it will not stop the next stage from running, since it's marked with +`allow_failure: true`: + +```yaml +job1: + stage: test + script: + - execute_script_that_will_fail + allow_failure: true + +job2: + stage: test + script: + - execute_script_that_will_succeed + +job3: + stage: deploy + script: + - deploy_to_staging +``` + ### when `when` is used to implement jobs that are run in case of failure or despite the @@ -485,6 +481,8 @@ failure. 1. `on_failure` - execute build only when at least one build from prior stages fails. 1. `always` - execute build regardless of the status of builds from prior stages. +1. `manual` - execute build manually (added in GitLab 8.10). Read about + [manual actions](#manual-actions) below. For example: @@ -516,6 +514,7 @@ deploy_job: stage: deploy script: - make deploy + when: manual cleanup_job: stage: cleanup @@ -526,28 +525,49 @@ cleanup_job: The above script will: -1. Execute `cleanup_build_job` only when `build_job` fails -2. Always execute `cleanup_job` as the last step in pipeline. +1. Execute `cleanup_build_job` only when `build_job` fails. +2. Always execute `cleanup_job` as the last step in pipeline regardless of + success or failure. +3. Allow you to manually execute `deploy_job` from GitLab's UI. + +#### Manual actions + +> Introduced in GitLab 8.10. + +Manual actions are a special type of job that are not executed automatically; +they need to be explicitly started by a user. Manual actions can be started +from pipeline, build, environment, and deployment views. You can execute the +same manual action multiple times. + +An example usage of manual actions is deployment to production. ### environment ->**Note:** -Introduced in GitLab 8.9. +> Introduced in GitLab 8.9. -`environment` is used to define that a job deploys to a specific environment. -This allows easy tracking of all deployments to your environments straight from -GitLab. +> You can read more about environments and find more examples in the +[documentation about environments][environment]. +`environment` is used to define that a job deploys to a specific environment. If `environment` is specified and no environment under that name exists, a new one will be created automatically. -The `environment` name must contain only letters, digits, '-' and '_'. Common -names are `qa`, `staging`, and `production`, but you can use whatever name works -with your workflow. +The `environment` name can contain: ---- +- letters +- digits +- spaces +- `-` +- `_` +- `/` +- `$` +- `{` +- `}` -**Example configurations** +Common names are `qa`, `staging`, and `production`, but you can use whatever +name works with your workflow. + +In its simplest form, the `environment` keyword can be defined like: ``` deploy to production: @@ -556,8 +576,134 @@ deploy to production: environment: production ``` -The `deploy to production` job will be marked as doing deployment to -`production` environment. +In the above example, the `deploy to production` job will be marked as doing a +deployment to the `production` environment. + +#### environment:name + +> Introduced in GitLab 8.11. + +>**Note:** +Before GitLab 8.11, the name of an environment could be defined as a string like +`environment: production`. The recommended way now is to define it under the +`name` keyword. + +Instead of defining the name of the environment right after the `environment` +keyword, it is also possible to define it as a separate value. For that, use +the `name` keyword under `environment`: + +``` +deploy to production: + stage: deploy + script: git push production HEAD:master + environment: + name: production +``` + +#### environment:url + +> Introduced in GitLab 8.11. + +>**Note:** +Before GitLab 8.11, the URL could be added only in GitLab's UI. The +recommended way now is to define it in `.gitlab-ci.yml`. + +This is an optional value that when set, it exposes buttons in various places +in GitLab which when clicked take you to the defined URL. + +In the example below, if the job finishes successfully, it will create buttons +in the merge requests and in the environments/deployments pages which will point +to `https://prod.example.com`. + +``` +deploy to production: + stage: deploy + script: git push production HEAD:master + environment: + name: production + url: https://prod.example.com +``` + +#### environment:on_stop + +> [Introduced][ce-6669] in GitLab 8.13. + +Closing (stoping) environments can be achieved with the `on_stop` keyword defined under +`environment`. It declares a different job that runs in order to close +the environment. + +Read the `environment:action` section for an example. + +#### environment:action + +> [Introduced][ce-6669] in GitLab 8.13. + +The `action` keyword is to be used in conjunction with `on_stop` and is defined +in the job that is called to close the environment. + +Take for instance: + +```yaml +review_app: + stage: deploy + script: make deploy-app + environment: + name: review + on_stop: stop_review_app + +stop_review_app: + stage: deploy + script: make delete-app + when: manual + environment: + name: review + action: stop +``` + +In the above example we set up the `review_app` job to deploy to the `review` +environment, and we also defined a new `stop_review_app` job under `on_stop`. +Once the `review_app` job is successfully finished, it will trigger the +`stop_review_app` job based on what is defined under `when`. In this case we +set it up to `manual` so it will need a [manual action](#manual-actions) via +GitLab's web interface in order to run. + +The `stop_review_app` job is **required** to have the following keywords defined: + +- `when` - [reference](#when) +- `environment:name` +- `environment:action` + +#### dynamic environments + +> [Introduced][ce-6323] in GitLab 8.12 and GitLab Runner 1.6. + +`environment` can also represent a configuration hash with `name` and `url`. +These parameters can use any of the defined [CI variables](#variables) +(including predefined, secure variables and `.gitlab-ci.yml` variables). + +For example: + +``` +deploy as review app: + stage: deploy + script: make deploy + environment: + name: review-apps/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.review.example.com/ +``` + +The `deploy as review app` job will be marked as deployment to dynamically +create the `review-apps/$CI_BUILD_REF_NAME` environment, which `$CI_BUILD_REF_NAME` +is an [environment variable][variables] set by the Runner. If for example the +`deploy as review app` job was run in a branch named `pow`, this environment +should be accessible under `https://pow.review.example.com/`. + +This of course implies that the underlying server which hosts the application +is properly configured. + +The common use case is to create dynamic environments for branches and use them +as Review Apps. You can see a simple example using Review Apps at +https://gitlab.com/gitlab-examples/review-apps-nginx/. ### artifacts @@ -569,8 +715,8 @@ The `deploy to production` job will be marked as doing deployment to > - Build artifacts are only collected for successful builds by default. `artifacts` is used to specify a list of files and directories which should be -attached to the build after success. To pass artifacts between different builds, -see [dependencies](#dependencies). +attached to the build after success. You can only use paths that are within the +project workspace. To pass artifacts between different builds, see [dependencies](#dependencies). Below are some examples. @@ -626,13 +772,13 @@ be available for download in the GitLab UI. #### artifacts:name ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.0. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.0. The `name` directive allows you to define the name of the created artifacts -archive. That way, you can have a unique name of every archive which could be +archive. That way, you can have a unique name for every archive which could be useful when you'd like to download the archive from GitLab. The `artifacts:name` variable can make use of any of the [predefined variables](../variables/README.md). +The default name is `artifacts`, which becomes `artifacts.zip` when downloaded. --- @@ -689,8 +835,7 @@ job: #### artifacts:when ->**Note:** -Introduced in GitLab 8.9 and GitLab Runner v1.3.0. +> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. `artifacts:when` is used to upload artifacts on build failure or despite the failure. @@ -715,8 +860,7 @@ job: #### artifacts:expire_in ->**Note:** -Introduced in GitLab 8.9 and GitLab Runner v1.3.0. +> Introduced in GitLab 8.9 and GitLab Runner v1.3.0. `artifacts:expire_in` is used to delete uploaded artifacts after the specified time. By default, artifacts are stored on GitLab forever. `expire_in` allows you @@ -751,18 +895,18 @@ job: ### dependencies ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. This feature should be used in conjunction with [`artifacts`](#artifacts) and allows you to define the artifacts to pass between different builds. -Note that `artifacts` from previous [stages](#stages) are passed by default. +Note that `artifacts` from all previous [stages](#stages) are passed by default. To use this feature, define `dependencies` in context of the job and pass a list of all previous builds from which the artifacts should be downloaded. You can only define builds from stages that are executed before the current one. An error will be shown if you define builds from the current stage or next ones. +Defining an empty array will skip downloading any artifacts for that job. --- @@ -825,32 +969,48 @@ job: ## Git Strategy ->**Note:** -Introduced in GitLab 8.9 as an experimental feature. May change in future -releases or be removed completely. +> Introduced in GitLab 8.9 as an experimental feature. May change or be removed + completely in future releases. `GIT_STRATEGY=none` requires GitLab Runner + v1.7+. -You can set the `GIT_STRATEGY` used for getting recent application code. `clone` -is slower, but makes sure you have a clean directory before every build. `fetch` -is faster. `GIT_STRATEGY` can be specified in the global `variables` section or -in the `variables` section for individual jobs. If it's not specified, then the -default from project settings will be used. +You can set the `GIT_STRATEGY` used for getting recent application code, either +in the global [`variables`](#variables) section or the [`variables`](#job-variables) +section for individual jobs. If left unspecified, the default from project +settings will be used. + +There are three possible values: `clone`, `fetch`, and `none`. + +`clone` is the slowest option. It clones the repository from scratch for every +job, ensuring that the project workspace is always pristine. ``` variables: GIT_STRATEGY: clone ``` -or +`fetch` is faster as it re-uses the project workspace (falling back to `clone` +if it doesn't exist). `git clean` is used to undo any changes made by the last +job, and `git fetch` is used to retrieve commits made since the last job ran. ``` variables: GIT_STRATEGY: fetch ``` +`none` also re-uses the project workspace, but skips all Git operations +(including GitLab Runner's pre-clone script, if present). It is mostly useful +for jobs that operate exclusively on artifacts (e.g., `deploy`). Git repository +data may be present, but it is certain to be out of date, so you should only +rely on files brought into the project workspace from cache or artifacts. + +``` +variables: + GIT_STRATEGY: none +``` + ## Shallow cloning ->**Note:** -Introduced in GitLab 8.9 as an experimental feature. May change in future +> Introduced in GitLab 8.9 as an experimental feature. May change in future releases or be removed completely. You can specify the depth of fetching and cloning using `GIT_DEPTH`. This allows @@ -878,24 +1038,26 @@ variables: GIT_DEPTH: "3" ``` -## Hidden jobs +## Hidden keys ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. -Jobs that start with a dot (`.`) will be not processed by GitLab CI. You can +Keys that start with a dot (`.`) will be not processed by GitLab CI. You can use this feature to ignore jobs, or use the -[special YAML features](#special-yaml-features) and transform the hidden jobs +[special YAML features](#special-yaml-features) and transform the hidden keys into templates. -In the following example, `.job_name` will be ignored: +In the following example, `.key_name` will be ignored: ```yaml -.job_name: +.key_name: script: - rake spec ``` +Hidden keys can be hashes like normal CI jobs, but you are also allowed to use +different types of structures to leverage special YAML features. + ## Special YAML features It's possible to use special YAML features like anchors (`&`), aliases (`*`) @@ -906,12 +1068,11 @@ Read more about the various [YAML features](https://learnxinyminutes.com/docs/ya ### Anchors ->**Note:** -Introduced in GitLab 8.6 and GitLab Runner v1.1.1. +> Introduced in GitLab 8.6 and GitLab Runner v1.1.1. YAML also has a handy feature called 'anchors', which let you easily duplicate content across your document. Anchors can be used to duplicate/inherit -properties, and is a perfect example to be used with [hidden jobs](#hidden-jobs) +properties, and is a perfect example to be used with [hidden keys](#hidden-keys) to provide templates for your jobs. The following example uses anchors and map merging. It will create two jobs, @@ -919,7 +1080,7 @@ The following example uses anchors and map merging. It will create two jobs, having their own custom `script` defined: ```yaml -.job_template: &job_definition # Hidden job that defines an anchor named 'job_definition' +.job_template: &job_definition # Hidden key that defines an anchor named 'job_definition' image: ruby:2.1 services: - postgres @@ -985,11 +1146,11 @@ directive defined in `.postgres_services` and `.mysql_services` respectively: - ruby test:postgres: - << *job_definition + <<: *job_definition services: *postgres_definition test:mysql: - << *job_definition + <<: *job_definition services: *mysql_definition ``` @@ -1025,7 +1186,14 @@ test:mysql: - ruby ``` -You can see that the hidden jobs are conveniently used as templates. +You can see that the hidden keys are conveniently used as templates. + +## Triggers + +Triggers can be used to force a rebuild of a specific branch, tag or commit, +with an API call. + +[Read more in the triggers documentation.](../triggers/README.md) ## Validate the .gitlab-ci.yml @@ -1043,3 +1211,7 @@ Visit the [examples README][examples] to see a list of examples using GitLab CI with various languages. [examples]: ../examples/README.md +[ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 +[environment]: ../environments.md +[ce-6669]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6669 +[variables]: ../variables/README.md diff --git a/doc/container_registry/README.md b/doc/container_registry/README.md index 1b465434498..fe3e4681ba7 100644 --- a/doc/container_registry/README.md +++ b/doc/container_registry/README.md @@ -1,94 +1 @@ -# GitLab Container Registry - -> **Note:** -This feature was [introduced][ce-4040] in GitLab 8.8. - -> **Note:** -This document is about the user guide. To learn how to enable GitLab Container -Registry across your GitLab instance, visit the -[administrator documentation](../administration/container_registry.md). - -With the Docker Container Registry integrated into GitLab, every project can -have its own space to store its Docker images. - -You can read more about Docker Registry at https://docs.docker.com/registry/introduction/. - ---- - -## Enable the Container Registry for your project - -1. First, ask your system administrator to enable GitLab Container Registry - following the [administration documentation](../administration/container_registry.md). - If you are using GitLab.com, this is enabled by default so you can start using - the Registry immediately. - -1. Go to your project's settings and enable the **Container Registry** feature - on your project. For new projects this might be enabled by default. For - existing projects you will have to explicitly enable it. - -  - -## Build and push images - -After you save your project's settings, you should see a new link in the -sidebar called **Container Registry**. Following this link will get you to -your project's Registry panel where you can see how to login to the Container -Registry using your GitLab credentials. - -For example if the Registry's URL is `registry.example.com`, the you should be -able to login with: - -``` -docker login registry.example.com -``` - -Building and publishing images should be a straightforward process. Just make -sure that you are using the Registry URL with the namespace and project name -that is hosted on GitLab: - -``` -docker build -t registry.example.com/group/project . -docker push registry.example.com/group/project -``` - -## Use images from GitLab Container Registry - -To download and run a container from images hosted in GitLab Container Registry, -use `docker run`: - -``` -docker run [options] registry.example.com/group/project [arguments] -``` - -For more information on running Docker containers, visit the -[Docker documentation][docker-docs]. - -## Control Container Registry from within GitLab - -GitLab offers a simple Container Registry management panel. Go to your project -and click **Container Registry** in the left sidebar. - -This view will show you all tags in your project and will easily allow you to -delete them. - - - -## Build and push images using GitLab CI - -> **Note:** -This feature requires GitLab 8.8 and GitLab Runner 1.2. - -Make sure that your GitLab Runner is configured to allow building docker images. -You have to check the [Using Docker Build documentation](../ci/docker/using_docker_build.md). -Then see the CI documentation on [Using the GitLab Container Registry](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry). - -## Limitations - -In order to use a container image from your private project as an `image:` in -your `.gitlab-ci.yml`, you have to follow the -[Using a private Docker Registry][private-docker] -documentation. This workflow will be simplified in the future. - -[ce-4040]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4040 -[docker-docs]: https://docs.docker.com/engine/userguide/intro/ -[private-docker]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry +This document was moved in [user/project/container_registry](../user/project/container_registry.md). diff --git a/doc/container_registry/img/container_registry.png b/doc/container_registry/img/container_registry.png Binary files differdeleted file mode 100644 index 57d6f9f22c5..00000000000 --- a/doc/container_registry/img/container_registry.png +++ /dev/null diff --git a/doc/container_registry/img/project_feature.png b/doc/container_registry/img/project_feature.png Binary files differdeleted file mode 100644 index a59b4f82b56..00000000000 --- a/doc/container_registry/img/project_feature.png +++ /dev/null diff --git a/doc/container_registry/troubleshooting.md b/doc/container_registry/troubleshooting.md new file mode 100644 index 00000000000..2f8cd37b488 --- /dev/null +++ b/doc/container_registry/troubleshooting.md @@ -0,0 +1 @@ +This document was moved to [user/project/container_registry](../user/project/container_registry.md). diff --git a/doc/customization/issue_closing.md b/doc/customization/issue_closing.md index 4620bb2dcde..31164ccd465 100644 --- a/doc/customization/issue_closing.md +++ b/doc/customization/issue_closing.md @@ -1,39 +1,4 @@ -# Issue closing pattern +This document was split into: -When a commit or merge request resolves one or more issues, it is possible to automatically have these issues closed when the commit or merge request lands in the project's default branch. - -If a commit message or merge request description contains a sentence matching the regular expression below, all issues referenced from -the matched text will be closed. This happens when the commit is pushed to a project's default branch, or when a commit or merge request is merged into there. - -When not specified, the default `issue_closing_pattern` as shown below will be used: - -```bash -((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing))(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) -``` - -Here, `%{issue_ref}` is a complex regular expression defined inside GitLab, that matches a reference to a local issue (`#123`), cross-project issue (`group/project#123`) or a link to an issue (`https://gitlab.example.com/group/project/issues/123`). - -For example: - -``` -git commit -m "Awesome commit message (Fix #20, Fixes #21 and Closes group/otherproject#22). This commit is also related to #17 and fixes #18, #19 and https://gitlab.example.com/group/otherproject/issues/23." -``` - -will close `#18`, `#19`, `#20`, and `#21` in the project this commit is pushed to, as well as `#22` and `#23` in group/otherproject. `#17` won't be closed as it does not match the pattern. It also works with multiline commit messages. - -Tip: you can test this closing pattern at [http://rubular.com][1]. Use this site -to test your own patterns. -Because Rubular doesn't understand `%{issue_ref}`, you can replace this by `#\d+` in testing, which matches only local issue references like `#123`. - -## Change the pattern - -For Omnibus installs you can change the default pattern in `/etc/gitlab/gitlab.rb`: - -``` -issue_closing_pattern: '((?:[Cc]los(?:e[sd]|ing)|[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)' -``` - -For manual installs you can customize the pattern in [gitlab.yml][0] using the `issue_closing_pattern` key. - -[0]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example -[1]: http://rubular.com/r/Xmbexed1OJ +- [administration/issue_closing_pattern.md](../administration/issue_closing_pattern.md). +- [user/project/issues/automatic_issue_closing](../user/project/issues/automatic_issue_closing.md). diff --git a/doc/development/README.md b/doc/development/README.md index c5d5af43864..fb6a8a5b095 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -1,18 +1,45 @@ # Development +## Outside of docs + +- [CONTRIBUTING.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md) main contributing guide +- [PROCESS.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/PROCESS.md) contributing process +- [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) to install a development version + +## Styleguides + +- [Documentation styleguide](doc_styleguide.md) Use this styleguide if you are + contributing to documentation. +- [SQL Migration Style Guide](migration_style_guide.md) for creating safe SQL migrations +- [Testing standards and style guidelines](testing.md) +- [UI guide](ui_guide.md) for building GitLab with existing CSS styles and elements +- [Frontend guidelines](frontend.md) +- [SQL guidelines](sql.md) for working with SQL queries +- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers + +## Process + +- [Code review guidelines](code_review.md) for reviewing code and having code reviewed. +- [Merge request performance guidelines](merge_request_performance_guidelines.md) + for ensuring merge requests do not negatively impact GitLab performance + +## Backend howtos + - [Architecture](architecture.md) of GitLab - [CI setup](ci_setup.md) for testing GitLab -- [Code review guidelines](code_review.md) for reviewing code and having code - reviewed. - [Gotchas](gotchas.md) to avoid - [How to dump production data to staging](db_dump.md) - [Instrumentation](instrumentation.md) -- [Licensing](licensing.md) for ensuring license compliance -- [Migration Style Guide](migration_style_guide.md) for creating safe migrations - [Performance guidelines](performance.md) - [Rake tasks](rake_tasks.md) for development - [Shell commands](shell_commands.md) in the GitLab codebase - [Sidekiq debugging](sidekiq_debugging.md) -- [SQL guidelines](sql.md) for SQL guidelines -- [Testing standards and style guidelines](testing.md) -- [UI guide](ui_guide.md) for building GitLab with existing css styles and elements + +## Databases + +- [What requires downtime?](what_requires_downtime.md) +- [Adding database indexes](adding_database_indexes.md) + +## Compliance + +- [Licensing](licensing.md) for ensuring license compliance diff --git a/doc/development/adding_database_indexes.md b/doc/development/adding_database_indexes.md new file mode 100644 index 00000000000..ea6f14da3b9 --- /dev/null +++ b/doc/development/adding_database_indexes.md @@ -0,0 +1,123 @@ +# Adding Database Indexes + +Indexes can be used to speed up database queries, but when should you add a new +index? Traditionally the answer to this question has been to add an index for +every column used for filtering or joining data. For example, consider the +following query: + +```sql +SELECT * +FROM projects +WHERE user_id = 2; +``` + +Here we are filtering by the `user_id` column and as such a developer may decide +to index this column. + +While in certain cases indexing columns using the above approach may make sense +it can actually have a negative impact. Whenever you write data to a table any +existing indexes need to be updated. The more indexes there are the slower this +can potentially become. Indexes can also take up quite some disk space depending +on the amount of data indexed and the index type. For example, PostgreSQL offers +"GIN" indexes which can be used to index certain data types that can not be +indexed by regular btree indexes. These indexes however generally take up more +data and are slower to update compared to btree indexes. + +Because of all this one should not blindly add a new index for every column used +to filter data by. Instead one should ask themselves the following questions: + +1. Can I write my query in such a way that it re-uses as many existing indexes + as possible? +2. Is the data going to be large enough that using an index will actually be + faster than just iterating over the rows in the table? +3. Is the overhead of maintaining the index worth the reduction in query + timings? + +We'll explore every question in detail below. + +## Re-using Queries + +The first step is to make sure your query re-uses as many existing indexes as +possible. For example, consider the following query: + +```sql +SELECT * +FROM todos +WHERE user_id = 123 +AND state = 'open'; +``` + +Now imagine we already have an index on the `user_id` column but not on the +`state` column. One may think this query will perform badly due to `state` being +unindexed. In reality the query may perform just fine given the index on +`user_id` can filter out enough rows. + +The best way to determine if indexes are re-used is to run your query using +`EXPLAIN ANALYZE`. Depending on any extra tables that may be joined and +other columns being used for filtering you may find an extra index is not going +to make much (if any) difference. On the other hand you may determine that the +index _may_ make a difference. + +In short: + +1. Try to write your query in such a way that it re-uses as many existing + indexes as possible. +2. Run the query using `EXPLAIN ANALYZE` and study the output to find the most + ideal query. + +## Data Size + +A database may decide not to use an index despite it existing in case a regular +sequence scan (= simply iterating over all existing rows) is faster. This is +especially the case for small tables. + +If a table is expected to grow in size and you expect your query has to filter +out a lot of rows you may want to consider adding an index. If the table size is +very small (e.g. only a handful of rows) or any existing indexes filter out +enough rows you may _not_ want to add a new index. + +## Maintenance Overhead + +Indexes have to be updated on every table write. In case of PostgreSQL _all_ +existing indexes will be updated whenever data is written to a table. As a +result of this having many indexes on the same table will slow down writes. + +Because of this one should ask themselves: is the reduction in query performance +worth the overhead of maintaining an extra index? + +If adding an index reduces SELECT timings by 5 milliseconds but increases +INSERT/UPDATE/DELETE timings by 10 milliseconds then the index may not be worth +it. On the other hand, if SELECT timings are reduced but INSERT/UPDATE/DELETE +timings are not affected you may want to add the index after all. + +## Finding Unused Indexes + +To see which indexes are unused you can run the following query: + +```sql +SELECT relname as table_name, indexrelname as index_name, idx_scan, idx_tup_read, idx_tup_fetch, pg_size_pretty(pg_relation_size(indexrelname::regclass)) +FROM pg_stat_all_indexes +WHERE schemaname = 'public' +AND idx_scan = 0 +AND idx_tup_read = 0 +AND idx_tup_fetch = 0 +ORDER BY pg_relation_size(indexrelname::regclass) desc; +``` + +This query outputs a list containing all indexes that are never used and sorts +them by indexes sizes in descending order. This query can be useful to +determine if any previously indexes are useful after all. More information on +the meaning of the various columns can be found at +<https://www.postgresql.org/docs/current/static/monitoring-stats.html>. + +Because the output of this query relies on the actual usage of your database it +may be affected by factors such as (but not limited to): + +* Certain queries never being executed, thus not being able to use certain + indexes. +* Certain tables having little data, resulting in PostgreSQL using sequence + scans instead of index scans. + +In other words, this data is only reliable for a frequently used database with +plenty of data and with as many GitLab features enabled (and being used) as +possible. diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 40ae55ab905..c5c23b5c0b8 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -34,6 +34,10 @@ request is up to one of our merge request "endbosses", denoted on the ## Having your code reviewed +Please keep in mind that code review is a process that can take multiple +iterations, and reviewers may spot things later that they may not have seen the +first time. + - The first reviewer of your code is _you_. Before you perform that first push of your shiny new branch, read through the entire diff. Does it make sense? Did you include something unrelated to the overall purpose of the changes? Did @@ -55,6 +59,7 @@ request is up to one of our merge request "endbosses", denoted on the Understand why the change is necessary (fixes a bug, improves the user experience, refactors the existing code). Then: +- Try to be thorough in your reviews to reduce the number of iterations. - Communicate which ideas you feel strongly about and those you don't. - Identify ways to simplify the code while still solving the problem. - Offer alternative implementations, but assume the author already considered @@ -64,8 +69,10 @@ experience, refactors the existing code). Then: someone else would be confused by it as well. - After a round of line notes, it can be helpful to post a summary note such as "LGTM :thumbsup:", or "Just a couple things to address." -- Avoid accepting a merge request before the build succeeds ("Merge when build - succeeds" is fine). +- Avoid accepting a merge request before the build succeeds. Of course, "Merge + When Build Succeeds" (MWBS) is fine. +- If you set the MR to "Merge When Build Succeeds", you should take over + subsequent revisions for anything that would be spotted after that. ## Credits diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 975bb82c37d..f07d2c9af2d 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -3,12 +3,64 @@ This styleguide recommends best practices to improve documentation and to keep it organized and easy to find. -## Naming +## Location and naming of documents -- When creating a new document and it has more than one word in its name, - make sure to use underscores instead of spaces or dashes (`-`). For example, - a proper naming would be `import_projects_from_github.md`. The same rule - applies to images. +>**Note:** +These guidelines derive from the discussion taken place in issue [#3349][ce-3349]. + +The documentation hierarchy can be vastly improved by providing a better layout +and organization of directories. + +Having a structured document layout, we will be able to have meaningful URLs +like `docs.gitlab.com/user/project/merge_requests.html`. With this pattern, +you can immediately tell that you are navigating a user related documentation +and is about the project and its merge requests. + +The table below shows what kind of documentation goes where. + +| Directory | What belongs here | +| --------- | -------------- | +| `doc/user/` | User related documentation. Anything that can be done within the GitLab UI goes here including `/admin`. | +| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. The admin settings that can be accessed via GitLab's interface go under `doc/user/admin_area/`. | +| `doc/api/` | API related documentation. | +| `doc/development/` | Documentation related to the development of GitLab. Any styleguides should go here. | +| `doc/legal/` | Legal documents about contributing to GitLab. | +| `doc/install/`| Probably the most visited directory, since `installation.md` is there. Ideally this should go under `doc/administration/`, but it's best to leave it as-is in order to avoid confusion (still debated though). | +| `doc/update/` | Same with `doc/install/`. Should be under `administration/`, but this is a well known location, better leave as-is, at least for now. | + +--- + +**General rules:** + +1. The correct naming and location of a new document, is a combination + of the relative URL of the document in question and the GitLab Map design + that is used for UX purposes ([source][graffle], [image][gitlab-map]). +1. When creating a new document and it has more than one word in its name, + make sure to use underscores instead of spaces or dashes (`-`). For example, + a proper naming would be `import_projects_from_github.md`. The same rule + applies to images. +1. There are four main directories, `user`, `administration`, `api` and `development`. +1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, + `profile/`, `dashboard/` and `admin_area/`. + 1. `doc/user/project/` should contain all project related documentation. + 1. `doc/user/group/` should contain all group related documentation. + 1. `doc/user/profile/` should contain all profile related documentation. + Every page you would navigate under `/profile` should have its own document, + i.e. `account.md`, `applications.md`, `emails.md`, etc. + 1. `doc/user/dashboard/` should contain all dashboard related documentation. + 1. `doc/user/admin_area/` should contain all admin related documentation + describing what can be achieved by accessing GitLab's admin interface + (_not to be confused with `doc/administration` where server access is + required_). + 1. Every category under `/admin/application_settings` should have its + own document located at `doc/user/admin_area/settings/`. For example, + the **Visibility and Access Controls** category should have a document + located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. + +--- + +If you are unsure where a document should live, you can ping `@axil` in your +merge request. ## Text @@ -43,10 +95,10 @@ it organized and easy to find. someone in the Merge Request - When introducing a new document, be careful for the headings to be grammatically and syntactically correct. It is advised to mention one or all - of the following GitLab members for a review: `@axil`, `@rspeicher`, - `@dblessing`, `@ashleys`, `@nearlythere`. This is to ensure that no document - with wrong heading is going live without an audit, thus preventing dead links - and redirection issues when corrected + of the following GitLab members for a review: `@axil`, `@rspeicher`, `@marcia`, + `@SeanPackham`. This is to ensure that no document with wrong heading is going + live without an audit, thus preventing dead links and redirection issues when + corrected - Leave exactly one newline after a heading ## Links @@ -103,15 +155,30 @@ Inside the document: - Every piece of documentation that comes with a new feature should declare the GitLab version that feature got introduced. Right below the heading add a - note: `>**Note:** This feature was introduced in GitLab 8.3` + note: + + ``` + > Introduced in GitLab 8.3. + ``` + - If possible every feature should have a link to the MR that introduced it. The above note would be then transformed to: - `>**Note:** This feature was [introduced][ce-1242] in GitLab 8.3`, where - the [link identifier](#links) is named after the repository (CE) and the MR - number -- If the feature is only in GitLab EE, don't forget to mention it, like: - `>**Note:** This feature was introduced in GitLab EE 8.3`. Otherwise, leave - this mention out + + ``` + > [Introduced][ce-1242] in GitLab 8.3. + ``` + + , where the [link identifier](#links) is named after the repository (CE) and + the MR number. + +- If the feature is only in GitLab Enterprise Edition, don't forget to mention + it, like: + + ``` + > Introduced in GitLab Enterprise Edition 8.3. + ``` + + Otherwise, leave this mention out. ## References @@ -170,18 +237,26 @@ For example, if you were to move `doc/workflow/lfs/lfs_administration.md` to ``` 1. Find and replace any occurrences of the old location with the new one. - A quick way to find them is to use `grep`: + A quick way to find them is to use `git grep`. First go to the root directory + where you cloned the `gitlab-ce` repository and then do: ``` - grep -nR "lfs_administration.md" doc/ + git grep -n "workflow/lfs/lfs_administration" + git grep -n "lfs/lfs_administration" ``` - The above command will search in the `doc/` directory for - `lfs_administration.md` recursively and will print the file and the line - where this file is mentioned. Note that we used just the filename - (`lfs_administration.md`) and not the whole the relative path - (`workflow/lfs/lfs_administration.md`). +Things to note: +- Since we also use inline documentation, except for the documentation itself, + the document might also be referenced in the views of GitLab (`app/`) which will + render when visiting `/help`, and sometimes in the testing suite (`spec/`). +- The above `git grep` command will search recursively in the directory you run + it in for `workflow/lfs/lfs_administration` and `lfs/lfs_administration` + and will print the file and the line where this file is mentioned. + You may ask why the two greps. Since we use relative paths to link to + documentation, sometimes it might be useful to search a path deeper. +- The `*.md` extension is not used when a document is linked to GitLab's + built-in help page, that's why we omit it in `git grep`. ## Configuration documentation for source and Omnibus installations @@ -239,11 +314,40 @@ In this case: - different highlighting languages are used for each config in the code block - the [references](#references) guide is used for reconfigure/restart +## Fake tokens + +There may be times where a token is needed to demonstrate an API call using +cURL or a secret variable used in CI. It is strongly advised not to use real +tokens in documentation even if the probability of a token being exploited is +low. + +You can use the following fake tokens as examples. + +| **Token type** | **Token value** | +| --------------------- | --------------------------------- | +| Private user token | `9koXpg98eAheJpvBs5tK` | +| Personal access token | `n671WNGecHugsdEDPsyo` | +| Application ID | `2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6` | +| Application secret | `04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df` | +| Secret CI variable | `Li8j-mLUVA3eZYjPfd_H` | +| Specific Runner token | `yrnZW46BrtBFqM7xDzE7dddd` | +| Shared Runner token | `6Vk7ZsosqQyfreAxXTZr` | +| Trigger token | `be20d8dcc028677c931e04f3871a9b` | +| Webhook secret token | `6XhDroRcYPM5by_h-HLY` | +| Health check token | `Tu7BgjR9qeZTEyRzGG2P` | +| Request profile token | `7VgpS4Ax5utVD2esNstz` | + ## API Here is a list of must-have items. Use them in the exact order that appears on this document. Further explanation is given below. +- Every method must be described using [Grape's DSL](https://github.com/ruby-grape/grape/tree/v0.13.0#describing-methods) + (see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/api/environments.rb + for a good example): + - `desc` for the method summary (you can pass it a block for additional details) + - `params` for the method params (this acts as description **and** validation + of the params) - Every method must have the REST API request. For example: ``` @@ -297,7 +401,7 @@ Below is a set of [cURL][] examples that you can use in the API documentation. Get the details of a group: ```bash -curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/gitlab-org +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/groups/gitlab-org ``` #### cURL example with parameters passed in the URL @@ -305,7 +409,7 @@ curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/ Create a new project under the authenticated user's namespace: ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects?name=foo" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects?name=foo" ``` #### Post data using cURL's --data @@ -315,7 +419,7 @@ cURL's `--data` option. The example below will create a new project `foo` under the authenticated user's namespace. ```bash -curl --data "name=foo" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects" +curl --data "name=foo" --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects" ``` #### Post data using JSON content @@ -324,7 +428,7 @@ curl --data "name=foo" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab. and double quotes. ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -H "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v3/groups +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "Content-Type: application/json" --data '{"path": "my-group", "name": "My group"}' https://gitlab.example.com/api/v3/groups ``` #### Post data using form-data @@ -333,7 +437,7 @@ Instead of using JSON or urlencode you can use multipart/form-data which properly handles data encoding: ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -F "title=ssh-key" -F "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v3/users/25/keys +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form "title=ssh-key" --form "key=ssh-rsa AAAAB3NzaC1yc2EA..." https://gitlab.example.com/api/v3/users/25/keys ``` The above example is run by and administrator and will add an SSH public key @@ -347,7 +451,7 @@ contains spaces in its title. Observe how spaces are escaped using the `%20` ASCII code. ```bash -curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/42/issues?title=Hello%20Dude" +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/42/issues?title=Hello%20Dude" ``` Use `%2F` for slashes (`/`). @@ -359,10 +463,13 @@ restrict the sign-up e-mail domains of a GitLab instance to `*.example.com` and `example.net`, you would do something like this: ```bash -curl -X PUT -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" -d "restricted_signup_domains[]=*.example.com" -d "restricted_signup_domains[]=example.net" https://gitlab.example.com/api/v3/application/settings +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "domain_whitelist[]=*.example.com" --data "domain_whitelist[]=example.net" https://gitlab.example.com/api/v3/application/settings ``` [cURL]: http://curl.haxx.se/ "cURL website" [single spaces]: http://www.slate.com/articles/technology/technology/2011/01/space_invaders.html -[gfm]: http://docs.gitlab.com/ce/markdown/markdown.html#newlines "GitLab flavored markdown documentation" +[gfm]: http://docs.gitlab.com/ce/user/markdown.html#newlines "GitLab flavored markdown documentation" [doc-restart]: ../administration/restart_gitlab.md "GitLab restart documentation" +[ce-3349]: https://gitlab.com/gitlab-org/gitlab-ce/issues/3349 "Documentation restructure" +[graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/d8d39f4a87b90fb9ae89ca12dc565347b4900d5e/production/resources/gitlab-map.graffle +[gitlab-map]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png
\ No newline at end of file diff --git a/doc/development/frontend.md b/doc/development/frontend.md new file mode 100644 index 00000000000..56c8516508e --- /dev/null +++ b/doc/development/frontend.md @@ -0,0 +1,236 @@ +# Frontend Development Guidelines + +This document describes various guidelines to ensure consistency and quality +across GitLab's frontend team. + +## Overview + +GitLab is built on top of [Ruby on Rails][rails] using [Haml][haml] with +[Hamlit][hamlit]. Be wary of [the limitations that come with using +Hamlit][hamlit-limits]. We also use [SCSS][scss] and plain JavaScript with +[ES6 by way of Babel][es6]. + +The asset pipeline is [Sprockets][sprockets], which handles the concatenation, +minification, and compression of our assets. + +[jQuery][jquery] is used throughout the application's JavaScript, with +[Vue.js][vue] for particularly advanced, dynamic elements. + +### Vue + +For more complex frontend features, we recommend using Vue.js. It shares +some ideas with React.js as well as Angular. + +To get started with Vue, read through [their documentation][vue-docs]. + +## Performance + +### Resources + +- [WebPage Test][web-page-test] for testing site loading time and size. +- [Google PageSpeed Insights][pagespeed-insights] grades web pages and provides feedback to improve the page. +- [Profiling with Chrome DevTools][google-devtools-profiling] +- [Browser Diet][browser-diet] is a community-built guide that catalogues practical tips for improving web page performance. + +### Page-specific JavaScript + +Certain pages may require the use of a third party library, such as [d3][d3] for +the User Activity Calendar and [Chart.js][chartjs] for the Graphs pages. These +libraries increase the page size significantly, and impact load times due to +bandwidth bottlenecks and the browser needing to parse more JavaScript. + +In cases where libraries are only used on a few specific pages, we use +"page-specific JavaScript" to prevent the main `application.js` file from +becoming unnecessarily large. + +Steps to split page-specific JavaScript from the main `application.js`: + +1. Create a directory for the specific page(s), e.g. `graphs/`. +1. In that directory, create a `namespace_bundle.js` file, e.g. `graphs_bundle.js`. +1. In `graphs_bundle.js` add the line `//= require_tree .`, this adds all other files in the directory to the bundle. +1. Add any necessary libraries to `app/assets/javascripts/lib/`, all files directly descendant from this directory will be precompiled as separate assets, in this case `chart.js` would be added. +1. Add the new "bundle" file to the list of precompiled assets in +`config/application.rb`. + - For example: `config.assets.precompile << "graphs/graphs_bundle.js"`. +1. Move code reliant on these libraries into the `graphs` directory. +1. In the relevant views, add the scripts to the page with the following: + +```haml +- content_for :page_specific_javascripts do + = page_specific_javascript_tag('lib/chart.js') + = page_specific_javascript_tag('graphs/graphs_bundle.js') +``` + +The above loads `chart.js` and `graphs_bundle.js` for this page only. `chart.js` +is separated from the bundle file so it can be cached separately from the bundle +and reused for other pages that also rely on the library. For an example, see +[this Haml file][page-specific-js-example]. + +### Minimizing page size + +A smaller page size means the page loads faster (especially important on mobile +and poor connections), the page is parsed more quickly by the browser, and less +data is used for users with capped data plans. + +General tips: + +- Don't add new fonts. +- Prefer font formats with better compression, e.g. WOFF2 is better than WOFF, which is better than TTF. +- Compress and minify assets wherever possible (For CSS/JS, Sprockets does this for us). +- If some functionality can reasonably be achieved without adding extra libraries, avoid them. +- Use page-specific JavaScript as described above to dynamically load libraries that are only needed on certain pages. + +## Accessibility + +### Resources + +[Chrome Accessibility Developer Tools][chrome-accessibility-developer-tools] +are useful for testing for potential accessibility problems in GitLab. + +Accessibility best-practices and more in-depth information is available on +[the Audit Rules page][audit-rules] for the Chrome Accessibility Developer Tools. + +## Security + +### Resources + +[Mozilla’s HTTP Observatory CLI][observatory-cli] and the +[Qualys SSL Labs Server Test][qualys-ssl] are good resources for finding +potential problems and ensuring compliance with security best practices. + +<!-- Uncomment these sections when CSP/SRI are implemented. +### Content Security Policy (CSP) + +Content Security Policy is a web standard that intends to mitigate certain +forms of Cross-Site Scripting (XSS) as well as data injection. + +Content Security Policy rules should be taken into consideration when +implementing new features, especially those that may rely on connection with +external services. + +GitLab's CSP is used for the following: + +- Blocking plugins like Flash and Silverlight from running at all on our pages. +- Blocking the use of scripts and stylesheets downloaded from external sources. +- Upgrading `http` requests to `https` when possible. +- Preventing `iframe` elements from loading in most contexts. + +Some exceptions include: + +- Scripts from Google Analytics and Piwik if either is enabled. +- Connecting with GitHub, Bitbucket, GitLab.com, etc. to allow project importing. +- Connecting with Google, Twitter, GitHub, etc. to allow OAuth authentication. + +We use [the Secure Headers gem][secure_headers] to enable Content +Security Policy headers in the GitLab Rails app. + +Some resources on implementing Content Security Policy: + +- [MDN Article on CSP][mdn-csp] +- [GitHub’s CSP Journey on the GitHub Engineering Blog][github-eng-csp] +- The Dropbox Engineering Blog's series on CSP: [1][dropbox-csp-1], [2][dropbox-csp-2], [3][dropbox-csp-3], [4][dropbox-csp-4] + +### Subresource Integrity (SRI) + +Subresource Integrity prevents malicious assets from being provided by a CDN by +guaranteeing that the asset downloaded is identical to the asset the server +is expecting. + +The Rails app generates a unique hash of the asset, which is used as the +asset's `integrity` attribute. The browser generates the hash of the asset +on-load and will reject the asset if the hashes do not match. + +All CSS and JavaScript assets should use Subresource Integrity. For implementation details, +see the documentation for [the Sprockets implementation of SRI][sprockets-sri]. + +Some resources on implementing Subresource Integrity: + +- [MDN Article on SRI][mdn-sri] +- [Subresource Integrity on the GitHub Engineering Blog][github-eng-sri] + +--> + +### Including external resources + +External fonts, CSS, and JavaScript should never be used with the exception of +Google Analytics and Piwik - and only when the instance has enabled it. Assets +should always be hosted and served locally from the GitLab instance. Embedded +resources via `iframes` should never be used except in certain circumstances +such as with ReCaptcha, which cannot be used without an `iframe`. + +### Avoiding inline scripts and styles + +In order to protect users from [XSS vulnerabilities][xss], we will disable inline scripts in the future using Content Security Policy. + +While inline scripts can be useful, they're also a security concern. If +user-supplied content is unintentionally left un-sanitized, malicious users can +inject scripts into the web app. + +Inline styles should be avoided in almost all cases, they should only be used +when no alternatives can be found. This allows reusability of styles as well as +readability. + +## Style guides and linting + +See the relevant style guides for our guidelines and for information on linting: + +- [SCSS][scss-style-guide] + +## Testing + +Feature tests need to be written for all new features. Regression tests +also need to be written for all bug fixes to prevent them from occurring +again in the future. + +See [the Testing Standards and Style Guidelines](testing.md) for more +information. + +## Supported browsers + +For our currently-supported browsers, see our [requirements][requirements]. + +[rails]: http://rubyonrails.org/ +[haml]: http://haml.info/ +[hamlit]: https://github.com/k0kubun/hamlit +[hamlit-limits]: https://github.com/k0kubun/hamlit/blob/master/REFERENCE.md#limitations +[scss]: http://sass-lang.com/ +[es6]: https://babeljs.io/ +[sprockets]: https://github.com/rails/sprockets +[jquery]: https://jquery.com/ +[vue]: http://vuejs.org/ +[vue-docs]: http://vuejs.org/guide/index.html +[web-page-test]: http://www.webpagetest.org/ +[pagespeed-insights]: https://developers.google.com/speed/pagespeed/insights/ +[google-devtools-profiling]: https://developers.google.com/web/tools/chrome-devtools/profile/?hl=en +[browser-diet]: https://browserdiet.com/ +[d3]: https://d3js.org/ +[chartjs]: http://www.chartjs.org/ +[page-specific-js-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/13bb9ed77f405c5f6ee4fdbc964ecf635c9a223f/app/views/projects/graphs/_head.html.haml#L6-8 +[chrome-accessibility-developer-tools]: https://github.com/GoogleChrome/accessibility-developer-tools +[audit-rules]: https://github.com/GoogleChrome/accessibility-developer-tools/wiki/Audit-Rules +[observatory-cli]: https://github.com/mozilla/http-observatory-cli) +[qualys-ssl]: https://www.ssllabs.com/ssltest/analyze.html +[secure_headers]: https://github.com/twitter/secureheaders +[mdn-csp]: https://developer.mozilla.org/en-US/docs/Web/Security/CSP +[github-eng-csp]: http://githubengineering.com/githubs-csp-journey/ +[dropbox-csp-1]: https://blogs.dropbox.com/tech/2015/09/on-csp-reporting-and-filtering/ +[dropbox-csp-2]: https://blogs.dropbox.com/tech/2015/09/unsafe-inline-and-nonce-deployment/ +[dropbox-csp-3]: https://blogs.dropbox.com/tech/2015/09/csp-the-unexpected-eval/ +[dropbox-csp-4]: https://blogs.dropbox.com/tech/2015/09/csp-third-party-integrations-and-privilege-separation/ +[mdn-sri]: https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity +[github-eng-sri]: http://githubengineering.com/subresource-integrity/ +[sprockets-sri]: https://github.com/rails/sprockets-rails#sri-support +[xss]: https://en.wikipedia.org/wiki/Cross-site_scripting +[scss-style-guide]: scss_styleguide.md +[requirements]: ../install/requirements.md#supported-web-browsers + +## Common Errors + +### Rspec (Capybara/Poltergeist) chokes on general JavaScript errors + +If you see very generic JavaScript errors (e.g. `jQuery is undefined`) being thrown in tests, but +can't reproduce them manually, you may have included `ES6`-style JavaScript in files that don't +have the `.js.es6` file extension. Either use ES5-friendly JavaScript or rename the file you're +working in (`git mv <file>.js> <file.js.es6>`). + + diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index 9d7fe7440d2..159d5ce286d 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -41,10 +41,10 @@ Rubocop](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-4-stable/.rubocop.yml#L9 [Exception]: http://stackoverflow.com/q/10048173/223897 -## Don't use inline CoffeeScript in views +## Don't use inline CoffeeScript/JavaScript in views Using the inline `:coffee` or `:coffeescript` Haml filters comes with a -performance overhead. +performance overhead. Using inline JavaScript is not a good way to structure your code and should be avoided. _**Note:** We've [removed these two filters](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/initializers/hamlit.rb) in an initializer._ @@ -52,6 +52,7 @@ in an initializer._ ### Further reading - Pull Request: [Replace CoffeeScript block into JavaScript in Views](https://git.io/vztMu) +- Stack Overflow: [Why you should not write inline JavaScript](http://programmers.stackexchange.com/questions/86589/why-should-i-avoid-inline-scripting) - Stack Overflow: [Performance implications of using :coffescript filter inside HAML templates?](http://stackoverflow.com/a/17571242/223897) ## ID-based CSS selectors need to be a bit more specific diff --git a/doc/development/instrumentation.md b/doc/development/instrumentation.md index c2272ab0a2b..105e2f1242a 100644 --- a/doc/development/instrumentation.md +++ b/doc/development/instrumentation.md @@ -137,3 +137,18 @@ end ``` Here the final value of `sleep_real_time` will be `3`, _not_ `1`. + +## Tracking Custom Events + +Besides instrumenting code GitLab Performance Monitoring also supports tracking +of custom events. This is primarily intended to be used for tracking business +metrics such as the number of Git pushes, repository imports, and so on. + +To track a custom event simply call `Gitlab::Metrics.add_event` passing it an +event name and a custom set of (optional) tags. For example: + +```ruby +Gitlab::Metrics.add_event(:user_login, email: current_user.email) +``` + +Event names should be verbs such as `push_repository` and `remove_branch`. diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 8c8c7486fff..05972b33fdb 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -54,6 +54,7 @@ Libraries with the following licenses are acceptable for use: - [BSD 2-Clause License][BSD-2-Clause]: A permissive (non-copyleft) license as defined by the Open Source Initiative. - [BSD 3-Clause License][BSD-3-Clause] (also known as New BSD or Modified BSD): A permissive (non-copyleft) license as defined by the Open Source Initiative - [ISC License][ISC] (also known as the OpenBSD License): A permissive (non-copyleft) license as defined by the Open Source Initiative. +- [Creative Commons Zero (CC0)][CC0]: A public domain dedication, recommended as a way to disclaim copyright on your work to the maximum extent possible. ## Unacceptable Licenses @@ -85,6 +86,7 @@ Gems which are included only in the "development" or "test" groups by Bundler ar [BSD-2-Clause]: https://opensource.org/licenses/BSD-2-Clause [BSD-3-Clause]: https://opensource.org/licenses/BSD-3-Clause [ISC]: https://opensource.org/licenses/ISC +[CC0]: https://creativecommons.org/publicdomain/zero/1.0/ [GPL]: http://choosealicense.com/licenses/gpl-3.0/ [GPLv2]: http://www.gnu.org/licenses/gpl-2.0.txt [GPLv3]: http://www.gnu.org/licenses/gpl-3.0.txt diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md new file mode 100644 index 00000000000..0363bf8c1d5 --- /dev/null +++ b/doc/development/merge_request_performance_guidelines.md @@ -0,0 +1,171 @@ +# Merge Request Performance Guidelines + +To ensure a merge request does not negatively impact performance of GitLab +_every_ merge request **must** adhere to the guidelines outlined in this +document. There are no exceptions to this rule unless specifically discussed +with and agreed upon by merge request endbosses and performance specialists. + +To measure the impact of a merge request you can use +[Sherlock](profiling.md#sherlock). It's also highly recommended that you read +the following guides: + +* [Performance Guidelines](performance.md) +* [What requires downtime?](what_requires_downtime.md) + +## Impact Analysis + +**Summary:** think about the impact your merge request may have on performance +and those maintaining a GitLab setup. + +Any change submitted can have an impact not only on the application itself but +also those maintaining it and those keeping it up and running (e.g. production +engineers). As a result you should think carefully about the impact of your +merge request on not only the application but also on the people keeping it up +and running. + +Can the queries used potentially take down any critical services and result in +engineers being woken up in the night? Can a malicious user abuse the code to +take down a GitLab instance? Will my changes simply make loading a certain page +slower? Will execution time grow exponentially given enough load or data in the +database? + +These are all questions one should ask themselves before submitting a merge +request. It may sometimes be difficult to assess the impact, in which case you +should ask a performance specialist to review your code. See the "Reviewing" +section below for more information. + +## Performance Review + +**Summary:** ask performance specialists to review your code if you're not sure +about the impact. + +Sometimes it's hard to assess the impact of a merge request. In this case you +should ask one of the merge request (mini) endbosses to review your changes. You +can find a list of these endbosses at <https://about.gitlab.com/team/>. An +endboss in turn can request a performance specialist to review the changes. + +## Query Counts + +**Summary:** a merge request **should not** increase the number of executed SQL +queries unless absolutely necessary. + +The number of queries executed by the code modified or added by a merge request +must not increase unless absolutely necessary. When building features it's +entirely possible you will need some extra queries, but you should try to keep +this at a minimum. + +As an example, say you introduce a feature that updates a number of database +rows with the same value. It may be very tempting (and easy) to write this using +the following pseudo code: + +```ruby +objects_to_update.each do |object| + object.some_field = some_value + object.save +end +``` + +This will end up running one query for every object to update. This code can +easily overload a database given enough rows to update or many instances of this +code running in parallel. This particular problem is known as the +["N+1 query problem"](http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations). + +In this particular case the workaround is fairly easy: + +```ruby +objects_to_update.update_all(some_field: some_value) +``` + +This uses ActiveRecord's `update_all` method to update all rows in a single +query. This in turn makes it much harder for this code to overload a database. + +## Executing Queries in Loops + +**Summary:** SQL queries **must not** be executed in a loop unless absolutely +necessary. + +Executing SQL queries in a loop can result in many queries being executed +depending on the number of iterations in a loop. This may work fine for a +development environment with little data, but in a production environment this +can quickly spiral out of control. + +There are some cases where this may be needed. If this is the case this should +be clearly mentioned in the merge request description. + +## Eager Loading + +**Summary:** always eager load associations when retrieving more than one row. + +When retrieving multiple database records for which you need to use any +associations you **must** eager load these associations. For example, if you're +retrieving a list of blog posts and you want to display their authors you +**must** eager load the author associations. + +In other words, instead of this: + +```ruby +Post.all.each do |post| + puts post.author.name +end +``` + +You should use this: + +```ruby +Post.all.includes(:author).each do |post| + puts post.author.name +end +``` + +## Memory Usage + +**Summary:** merge requests **must not** increase memory usage unless absolutely +necessary. + +A merge request must not increase the memory usage of GitLab by more than the +absolute bare minimum required by the code. This means that if you have to parse +some large document (e.g. an HTML document) it's best to parse it as a stream +whenever possible, instead of loading the entire input into memory. Sometimes +this isn't possible, in that case this should be stated explicitly in the merge +request. + +## Lazy Rendering of UI Elements + +**Summary:** only render UI elements when they're actually needed. + +Certain UI elements may not always be needed. For example, when hovering over a +diff line there's a small icon displayed that can be used to create a new +comment. Instead of always rendering these kind of elements they should only be +rendered when actually needed. This ensures we don't spend time generating +Haml/HTML when it's not going to be used. + +## Instrumenting New Code + +**Summary:** always add instrumentation for new classes, modules, and methods. + +Newly added classes, modules, and methods must be instrumented. This ensures +we can track the performance of this code over time. + +For more information see [Instrumentation](instrumentation.md). This guide +describes how to add instrumentation and where to add it. + +## Use of Caching + +**Summary:** cache data in memory or in Redis when it's needed multiple times in +a transaction or has to be kept around for a certain time period. + +Sometimes certain bits of data have to be re-used in different places during a +transaction. In these cases this data should be cached in memory to remove the +need for running complex operations to fetch the data. You should use Redis if +data should be cached for a certain time period instead of the duration of the +transaction. + +For example, say you process multiple snippets of text containiner username +mentions (e.g. `Hello @alice` and `How are you doing @alice?`). By caching the +user objects for every username we can remove the need for running the same +query for every mention of `@alice`. + +Caching data per transaction can be done using +[RequestStore](https://github.com/steveklabnik/request_store). Caching data in +Redis can be done using [Rails' caching +system](http://guides.rubyonrails.org/caching_with_rails.html). diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index e2ca46504e7..61b0fbc89c9 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -9,9 +9,10 @@ a big burden for most organizations. For this reason it is important that your migrations are written carefully, can be applied online and adhere to the style guide below. Migrations should not require GitLab installations to be taken offline unless -_absolutely_ necessary. If a migration requires downtime this should be -clearly mentioned during the review process as well as being documented in the -monthly release post. +_absolutely_ necessary - see the ["What Requires Downtime?"](what_requires_downtime.md) +page. If a migration requires downtime, this should be clearly mentioned during +the review process, as well as being documented in the monthly release post. For +more information, see the "Downtime Tagging" section below. When writing your migrations, also consider that databases might have stale data or inconsistencies and guard for that. Try to make as little assumptions as possible @@ -20,35 +21,34 @@ about the state of the database. Please don't depend on GitLab specific code since it can change in future versions. If needed copy-paste GitLab code into the migration to make it forward compatible. -## Comments in the migration +## Downtime Tagging -Each migration you write needs to have the two following pieces of information -as comments. +Every migration must specify if it requires downtime or not, and if it should +require downtime it must also specify a reason for this. To do so, add the +following two constants to the migration class' body: -### Online, Offline, errors? +* `DOWNTIME`: a boolean that when set to `true` indicates the migration requires + downtime. +* `DOWNTIME_REASON`: a String containing the reason for the migration requiring + downtime. This constant **must** be set when `DOWNTIME` is set to `true`. -First, you need to provide information on whether the migration can be applied: +For example: -1. online without errors (works on previous version and new one) -2. online with errors on old instances after migrating -3. online with errors on new instances while migrating -4. offline (needs to happen without app servers to prevent db corruption) - -For example: - -``` -# Migration type: online without errors (works on previous version and new one) +```ruby class MyMigration < ActiveRecord::Migration -... -``` + DOWNTIME = true + DOWNTIME_REASON = 'This migration requires downtime because ...' -It is always preferable to have a migration run online. If you expect the migration -to take particularly long (for instance, if it loops through all notes), -this is valuable information to add. + def change + ... + end +end +``` -If you don't provide the information it means that a migration is safe to run online. +It is an error (that is, CI will fail) if the `DOWNTIME` constant is missing +from a migration class. -### Reversibility +## Reversibility Your migration should be reversible. This is very important, as it should be possible to downgrade in case of a vulnerability or bugs. @@ -100,7 +100,7 @@ value of `10` you'd write the following: class MyMigration < ActiveRecord::Migration include Gitlab::Database::MigrationHelpers disable_ddl_transaction! - + def up add_column_with_default(:projects, :foo, :integer, default: 10) end @@ -111,6 +111,28 @@ class MyMigration < ActiveRecord::Migration end ``` + +## Integer column type + +By default, an integer column can hold up to a 4-byte (32-bit) number. That is +a max value of 2,147,483,647. Be aware of this when creating a column that will +hold file sizes in byte units. If you are tracking file size in bytes this +restricts the maximum file size to just over 2GB. + +To allow an integer column to hold up to an 8-byte (64-bit) number, explicitly +set the limit to 8-bytes. This will allow the column to hold a value up to +9,223,372,036,854,775,807. + +Rails migration example: + +``` +add_column_with_default(:projects, :foo, :integer, default: 10, limit: 8) + +# or + +add_column(:projects, :foo, :integer, default: 10, limit: 8) +``` + ## Testing Make sure that your migration works with MySQL and PostgreSQL with data. An empty database does not guarantee that your migration is correct. diff --git a/doc/development/newlines_styleguide.md b/doc/development/newlines_styleguide.md new file mode 100644 index 00000000000..32aac2529a4 --- /dev/null +++ b/doc/development/newlines_styleguide.md @@ -0,0 +1,102 @@ +# Newlines styleguide + +This style guide recommends best practices for newlines in Ruby code. + +## Rule: separate code with newlines only to group together related logic + +```ruby +# bad +def method + issue = Issue.new + + issue.save + + render json: issue +end +``` + +```ruby +# good +def method + issue = Issue.new + issue.save + + render json: issue +end +``` + +## Rule: separate code and block with newlines + +### Newline before block + +```ruby +# bad +def method + issue = Issue.new + if issue.save + render json: issue + end +end +``` + +```ruby +# good +def method + issue = Issue.new + + if issue.save + render json: issue + end +end +``` + +## Newline after block + +```ruby +# bad +def method + if issue.save + issue.send_email + end + render json: issue +end +``` + +```ruby +# good +def method + if issue.save + issue.send_email + end + + render json: issue +end +``` + +### Exception: no need for newline when code block starts or ends right inside another code block + +```ruby +# bad +def method + + if issue + + if issue.valid? + issue.save + end + + end + +end +``` + +```ruby +# good +def method + if issue + if issue.valid? + issue.save + end + end +end +``` diff --git a/doc/development/performance.md b/doc/development/performance.md index fb37b3a889c..c4a964d1da3 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -15,8 +15,8 @@ The process of solving performance problems is roughly as follows: 3. Add your findings based on the measurement period (screenshots of graphs, timings, etc) to the issue mentioned in step 1. 4. Solve the problem. -5. Create a merge request, assign the "performance" label and ping the right - people (e.g. [@yorickpeterse][yorickpeterse] and [@joshfng][joshfng]). +5. Create a merge request, assign the "Performance" label and assign it to + [@yorickpeterse][yorickpeterse] for reviewing. 6. Once a change has been deployed make sure to _again_ measure for at least 24 hours to see if your changes have any impact on the production environment. 7. Repeat until you're done. @@ -34,10 +34,11 @@ graphs/dashboards. ## Tooling -GitLab provides two built-in tools to aid the process of improving performance: +GitLab provides built-in tools to aid the process of improving performance: -* [Sherlock](doc/development/profiling.md#sherlock) -* [GitLab Performance Monitoring](doc/monitoring/performance/monitoring.md) +* [Sherlock](profiling.md#sherlock) +* [GitLab Performance Monitoring](../monitoring/performance/monitoring.md) +* [Request Profiling](../administration/monitoring/performance/request_profiling.md) GitLab employees can use GitLab.com's performance monitoring systems located at <http://performance.gitlab.net>, this requires you to log in using your @@ -253,6 +254,5 @@ impact on runtime performance, and as such, using a constant instead of referencing an object directly may even slow code down. [#15607]: https://gitlab.com/gitlab-org/gitlab-ce/issues/15607 -[yorickpeterse]: https://gitlab.com/u/yorickpeterse -[joshfng]: https://gitlab.com/u/joshfng +[yorickpeterse]: https://gitlab.com/yorickpeterse [anti-pattern]: https://en.wikipedia.org/wiki/Anti-pattern diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 41685c7ee41..a7175f3f87e 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -14,11 +14,33 @@ Note: `db:setup` calls `db:seed` but this does nothing. ## Run tests -This runs all test suites present in GitLab. +In order to run the test you can use the following commands: +- `rake spinach` to run the spinach suite +- `rake spec` to run the rspec suite +- `rake teaspoon` to run the teaspoon test suite +- `rake gitlab:test` to run all the tests -``` -bundle exec rake test -``` +Note: Both `rake spinach` and `rake spec` takes significant time to pass. +Instead of running full test suite locally you can save a lot of time by running +a single test or directory related to your changes. After you submit merge request +CI will run full test suite for you. Green CI status in the merge request means +full test suite is passed. + +Note: You can't run `rspec .` since this will try to run all the `_spec.rb` +files it can find, also the ones in `/tmp` + +To run a single test file you can use: + +- `bundle exec rspec spec/controllers/commit_controller_spec.rb` for a rspec test +- `bundle exec spinach features/project/issues/milestones.feature` for a spinach test + +To run several tests inside one directory: + +- `bundle exec rspec spec/requests/api/` for the rspec tests if you want to test API only +- `bundle exec spinach features/profile/` for the spinach tests if you want to test only profile pages + +If you want to use [Spring](https://github.com/rails/spring) set +`ENABLE_SPRING=1` in your environment. ## Generate searchable docs for source code @@ -53,3 +75,8 @@ Generating a sprite file containing all the Emoji can be done by running: ``` bundle exec rake gemojione:sprite ``` + +If new emoji are added, the spritesheet may change size. To compensate for +such changes, first generate the `emoji.png` spritesheet with the above Rake +task, then check the dimensions of the new spritesheet and update the +`SPRITESHEET_WIDTH` and `SPRITESHEET_HEIGHT` constants accordingly. diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md new file mode 100644 index 00000000000..e3a20f29a09 --- /dev/null +++ b/doc/development/sidekiq_style_guide.md @@ -0,0 +1,38 @@ +# Sidekiq Style Guide + +This document outlines various guidelines that should be followed when adding or +modifying Sidekiq workers. + +## Default Queue + +Use of the "default" queue is not allowed. Every worker should use a queue that +matches the worker's purpose the closest. For example, workers that are to be +executed periodically should use the "cronjob" queue. + +A list of all available queues can be found in `config/sidekiq_queues.yml`. + +## Dedicated Queues + +Most workers should use their own queue. To ease this process a worker can +include the `DedicatedSidekiqQueue` concern as follows: + +```ruby +class ProcessSomethingWorker + include Sidekiq::Worker + include DedicatedSidekiqQueue +end +``` + +This will set the queue name based on the class' name, minus the `Worker` +suffix. In the above example this would lead to the queue being +`process_something`. + +In some cases multiple workers do use the same queue. For example, the various +workers for updating CI pipelines all use the `pipeline` queue. Adding workers +to existing queues should be done with care, as adding more workers can lead to +slow jobs blocking work (even for different jobs) on the shared queue. + +## Tests + +Each Sidekiq worker must be tested using RSpec, just like any other class. These +tests should be placed in `spec/workers`. diff --git a/doc/development/ui_guide.md b/doc/development/ui_guide.md index ce0aaa2fd25..2d1d504202c 100644 --- a/doc/development/ui_guide.md +++ b/doc/development/ui_guide.md @@ -1,43 +1,48 @@ -# UI Guide for building GitLab +# UI Guide for building GitLab ## GitLab UI development kit We created a page inside GitLab where you can check commonly used html and css elements. -When you run GitLab instance locally - just visit http://localhost:3000/help/ui page to see UI examples +When you run GitLab instance locally - just visit http://localhost:3000/help/ui page to see UI examples you can use during GitLab development. ## Design repository -All design files are stored in the [gitlab-design](https://gitlab.com/gitlab-org/gitlab-design) -repository and maintained by GitLab UX designers. +All design files are stored in the [gitlab-design](https://gitlab.com/gitlab-org/gitlab-design) +repository and maintained by GitLab UX designers. ## Navigation -GitLab's layout contains 2 sections: the left sidebar and the content. The left sidebar contains a static navigation menu. -This menu will be visible regardless of what page you visit. The left sidebar also contains the GitLab logo -and the current user's profile picture. The content section contains a header and the content itself. -The header describes the current GitLab page and what navigation is -available to user in this area. Depending on the area (project, group, profile setting) the header name and navigation may change. For example when user visits one of the -project pages the header will contain a project name and navigation for that project. When the user visits a group page it will contain a group name and navigation related to this group. +GitLab's layout contains 2 sections: the left sidebar and the content. The left sidebar contains a static navigation menu. +This menu will be visible regardless of what page you visit. +The content section contains a header and the content itself. The header describes the current GitLab page and what navigation is +available to the user in this area. Depending on the area (project, group, profile setting) the header name and navigation may change. For example, when the user visits one of the +project pages the header will contain the project's name and navigation for that project. When the user visits a group page it will contain the group's name and navigation related to this group. + +You can see a visual representation of the navigation in GitLab in the GitLab Product Map, which is located in the [Design Repository](gitlab-map-graffle) +along with [PDF](gitlab-map-pdf) and [PNG](gitlab-map-png) exports. + ### Adding new tab to header navigation -We try to keep the amount of tabs in the header navigation between 5 and 10 so that it fits on a typical laptop screen. We also try not to confuse the user with too many options. Ideally each -tab should represent separate functionality. Everything related to the issue -tracker should be under the 'Issues' tab while everything related to the wiki should +We try to keep the amount of tabs in the header navigation between 5 and 10 so that it fits on a typical laptop screen. We also try not to confuse the user with too many options. Ideally each +tab should represent separate functionality. Everything related to the issue +tracker should be under the 'Issues' tab while everything related to the wiki should be under 'Wiki' tab and so on and so forth. +When adding a new tab to the header don't use more than 2 words for text in the link. +We want to keep links short and easy to remember and fit all of them in the small screen. -## Mobile screen size +## Mobile screen size -We want GitLab to work well on small mobile screens as well. Size limitations make it is impossible to fit everything on a mobile screen. In this case it is OK to hide -part of the UI for smaller resolutions in favor of a better user experience. +We want GitLab to work well on small mobile screens as well. Size limitations make it is impossible to fit everything on a mobile screen. In this case it is OK to hide +part of the UI for smaller resolutions in favor of a better user experience. However core functionality like browsing files, creating issues, writing comments, should be available on all resolutions. ## Icons -* `trash` icon for button or link that does destructive action like removing +* `trash` icon for button or link that does destructive action like removing information from database or file system * `x` icon for closing/hiding UI element. For example close modal window * `pencil` icon for edit button or link @@ -45,13 +50,58 @@ information from database or file system * `rss` for rss/atom feed * `plus` for link or dropdown that lead to page where you create new object (For example new issue page) +### SVGs + +When exporting SVGs, be sure to follow the following guidelines: + +1. Convert all strokes to outlines. +2. Use pathfinder tools to combine overlapping paths and create compound paths. +3. SVGs that are limited to one color should be exported without a fill color so the color can be set using CSS. +4. Ensure that exported SVGs have been run through an [SVG cleaner](https://github.com/RazrFalcon/SVGCleaner) to remove unused elements and attributes. + +You can open your svg in a text editor to ensure that it is clean. +Incorrect files will look like this: + +```xml +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg width="16px" height="17px" viewBox="0 0 16 17" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> + <!-- Generator: Sketch 3.7.2 (28276) - http://www.bohemiancoding.com/sketch --> + <title>Group</title> + <desc>Created with Sketch.</desc> + <defs></defs> + <g id="Page-1" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"> + <g id="Group" fill="#7E7C7C"> + <path d="M15.1111,1 L0.8891,1 C0.3981,1 0.0001,1.446 0.0001,1.996 L0.0001,15.945 C0.0001,16.495 0.3981,16.941 0.8891,16.941 L15.1111,16.941 C15.6021,16.941 16.0001,16.495 16.0001,15.945 L16.0001,1.996 C16.0001,1.446 15.6021,1 15.1111,1 L15.1111,1 L15.1111,1 Z M14.0001,6.0002 L14.0001,14.949 L2.0001,14.949 L2.0001,6.0002 L14.0001,6.0002 Z M14.0001,4.0002 L14.0001,2.993 L2.0001,2.993 L2.0001,4.0002 L14.0001,4.0002 Z" id="Combined-Shape"></path> + <polygon id="Fill-11" points="3 2.0002 5 2.0002 5 0.0002 3 0.0002"></polygon> + <polygon id="Fill-16" points="11 2.0002 13 2.0002 13 0.0002 11 0.0002"></polygon> + <path d="M5.37709616,11.5511984 L6.92309616,12.7821984 C7.35112915,13.123019 7.97359761,13.0565604 8.32002627,12.6330535 L10.7740263,9.63305349 C11.1237073,9.20557058 11.0606364,8.57555475 10.6331535,8.22587373 C10.2056706,7.87619272 9.57565475,7.93926361 9.22597373,8.36674651 L6.77197373,11.3667465 L8.16890384,11.2176016 L6.62290384,9.98660159 C6.19085236,9.6425813 5.56172188,9.71394467 5.21770159,10.1459962 C4.8736813,10.5780476 4.94504467,11.2071781 5.37709616,11.5511984 L5.37709616,11.5511984 Z" id="Stroke-21"></path> + </g> + </g> +</svg> +``` + +Correct file will look like this: + +```xml +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 17" enable-background="new 0 0 16 17"><path d="m15.1 1h-2.1v-1h-2v1h-6v-1h-2v1h-2.1c-.5 0-.9.5-.9 1v14c0 .6.4 1 .9 1h14.2c.5 0 .9-.4.9-1v-14c0-.5-.4-1-.9-1m-1.1 14h-12v-9h12v9m0-11h-12v-1h12v1"/><path d="m5.4 11.6l1.5 1.2c.4.3 1.1.3 1.4-.1l2.5-3c.3-.4.3-1.1-.1-1.4-.5-.4-1.1-.3-1.5.1l-1.8 2.2-.8-.6c-.4-.3-1.1-.3-1.4.2-.3.4-.3 1 .2 1.4"/></svg> +``` + ## Buttons * Button should contain icon or text. Exceptions should be approved by UX designer. * Use red button for destructive actions (not revertable). For example removing issue. -* Use green or blue button for primary action. Primary button should be only one. -Do not use both green and blue button in one form. -* For all other cases use default white button. -* Text button should have only first word capitalized. So should be "Create issue" instead of "Create Issue" +* Use green or blue button for primary action. Primary button should be only one. +Do not use both green and blue button in one form. +* For all other cases use default white button. +* Text button should have only first word capitalized. So should be "Create issue" instead of "Create Issue" + +## Counts + +* Always use the [`number_with_delimiter`][number_with_delimiter] helper to + display counts in the UI. +[number_with_delimiter]: http://api.rubyonrails.org/classes/ActionView/Helpers/NumberHelper.html#method-i-number_with_delimiter +[gitlab-map-graffle]: https://gitlab.com/gitlab-org/gitlab-design/blob/master/production/resources/gitlab-map.graffle +[gitlab-map-pdf]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.pdf +[gitlab-map-png]: https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png
\ No newline at end of file diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md new file mode 100644 index 00000000000..2574c2c0472 --- /dev/null +++ b/doc/development/what_requires_downtime.md @@ -0,0 +1,161 @@ +# What requires downtime? + +When working with a database certain operations can be performed without taking +GitLab offline, others do require a downtime period. This guide describes +various operations and their impact. + +## Adding Columns + +On PostgreSQL you can safely add a new column to an existing table as long as it +does **not** have a default value. For example, this query would not require +downtime: + +```sql +ALTER TABLE projects ADD COLUMN random_value int; +``` + +Add a column _with_ a default however does require downtime. For example, +consider this query: + +```sql +ALTER TABLE projects ADD COLUMN random_value int DEFAULT 42; +``` + +This requires updating every single row in the `projects` table so that +`random_value` is set to `42` by default. This requires updating all rows and +indexes in a table. This in turn acquires enough locks on the table for it to +effectively block any other queries. + +As of MySQL 5.6 adding a column to a table is still quite an expensive +operation, even when using `ALGORITHM=INPLACE` and `LOCK=NONE`. This means +downtime _may_ be required when modifying large tables as otherwise the +operation could potentially take hours to complete. + +Adding a column with a default value _can_ be done without requiring downtime +when using the migration helper method +`Gitlab::Database::MigrationHelpers#add_column_with_default`. This method works +similar to `add_column` except it updates existing rows in batches without +blocking access to the table being modified. See ["Adding Columns With Default +Values"](migration_style_guide.html#adding-columns-with-default-values) for more +information on how to use this method. + +## Dropping Columns + +On PostgreSQL you can safely remove an existing column without the need for +downtime. When you drop a column in PostgreSQL it's not immediately removed, +instead it is simply disabled. The data is removed on the next vacuum run. + +On MySQL this operation requires downtime. + +While database wise dropping a column may be fine on PostgreSQL this operation +still requires downtime because the application code may still be using the +column that was removed. For example, consider the following migration: + +```ruby +class MyMigration < ActiveRecord::Migration + def change + remove_column :projects, :dummy + end +end +``` + +Now imagine that the GitLab instance is running and actively uses the `dummy` +column. If we were to run the migration this would result in the GitLab instance +producing errors whenever it tries to use the `dummy` column. + +As a result of the above downtime _is_ required when removing a column, even +when using PostgreSQL. + +## Changing Column Constraints + +Generally changing column constraints requires checking all rows in the table to +see if they meet the new constraint, unless a constraint is _removed_. For +example, changing a column that previously allowed NULL values to not allow NULL +values requires the database to verify all existing rows. + +The specific behaviour varies a bit between databases but in general the safest +approach is to assume changing constraints requires downtime. + +## Changing Column Types + +This operation requires downtime. + +## Adding Indexes + +Adding indexes is an expensive process that blocks INSERT and UPDATE queries for +the duration. When using PostgreSQL one can work arounds this by using the +`CONCURRENTLY` option: + +```sql +CREATE INDEX CONCURRENTLY index_name ON projects (column_name); +``` + +Migrations can take advantage of this by using the method +`add_concurrent_index`. For example: + +```ruby +class MyMigration < ActiveRecord::Migration + def change + add_concurrent_index :projects, :column_name + end +end +``` + +When running this on PostgreSQL the `CONCURRENTLY` option mentioned above is +used. On MySQL this method produces a regular `CREATE INDEX` query. + +MySQL doesn't really have a workaround for this. Supposedly it _can_ create +indexes without the need for downtime but only for variable width columns. The +details on this are a bit sketchy. Since it's better to be safe than sorry one +should assume that adding indexes requires downtime on MySQL. + +## Dropping Indexes + +Dropping an index does not require downtime on both PostgreSQL and MySQL. + +## Adding Tables + +This operation is safe as there's no code using the table just yet. + +## Dropping Tables + +This operation requires downtime as application code may still be using the +table. + +## Adding Foreign Keys + +Adding foreign keys acquires an exclusive lock on both the source and target +tables in PostgreSQL. This requires downtime as otherwise the entire application +grinds to a halt for the duration of the operation. + +On MySQL this operation also requires downtime _unless_ foreign key checks are +disabled. Because this means checks aren't enforced this is not ideal, as such +one should assume MySQL also requires downtime. + +## Removing Foreign Keys + +This operation should not require downtime on both PostgreSQL and MySQL. + +## Updating Data + +Updating data should generally be safe. The exception to this is data that's +being migrated from one version to another while the application still produces +data in the old version. + +For example, imagine the application writes the string `'dog'` to a column but +it really is meant to write `'cat'` instead. One might think that the following +migration is all that is needed to solve this problem: + +```ruby +class MyMigration < ActiveRecord::Migration + def up + execute("UPDATE some_table SET column = 'cat' WHERE column = 'dog';") + end +end +``` + +Unfortunately this is not enough. Because the application is still running and +using the old value this may result in the table still containing rows where +`column` is set to `dog`, even after the migration finished. + +In these cases downtime _is_ required, even for rarely updated tables. diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 3aa83975ace..d7e3aa35bdd 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -2,14 +2,14 @@ Step-by-step guides on the basics of working with Git and GitLab. +- [Command line basics](command-line-commands.md) - [Start using Git on the command line](start-using-git.md) - [Create and add your SSH Keys](create-your-ssh-keys.md) -- [Command Line basics](command-line-commands.md) - [Create a project](create-project.md) - [Create a group](create-group.md) - [Create a branch](create-branch.md) - [Fork a project](fork-project.md) - [Add a file](add-file.md) - [Add an image](add-image.md) -- [Create a Merge Request](add-merge-request.md) -- [Create an Issue](create-issue.md) +- [Create an issue](create-issue.md) +- [Create a merge request](add-merge-request.md) diff --git a/doc/gitlab-basics/add-file.md b/doc/gitlab-basics/add-file.md index 57136ac5c39..e9fbcbc23a9 100644 --- a/doc/gitlab-basics/add-file.md +++ b/doc/gitlab-basics/add-file.md @@ -1,31 +1,5 @@ # How to add a file -You can create a file in your [shell](command-line-commands.md) or in GitLab. - -To create a file in GitLab, sign in to GitLab. - -Select a project on the right side of your screen: - - - -It's a good idea to [create a branch](create-branch.md), but it's not necessary. - -Go to the directory where you'd like to add the file and click on the "+" sign next to the name of the project and directory: - - - -Name your file (you can't add spaces, so you can use hyphens or underscores). Don't forget to include the markup language you'd like to use : - - - -Add all the information that you'd like to include in your file: - - - -Add a commit message based on what you just added and then click on "commit changes": - - - -### Note -Besides its regular files, every directory needs a README.md or README.html file which works like an index, telling -what the directory is about. It's the first document you'll find when you open a directory. +You can create a file in your [terminal](command-line-commands.md) and push +to GitLab or you can use the +[web interface](../user/project/repository/web_editor.md#create-a-file). diff --git a/doc/gitlab-basics/add-merge-request.md b/doc/gitlab-basics/add-merge-request.md index 236b4248ea2..bf01fe51dc3 100644 --- a/doc/gitlab-basics/add-merge-request.md +++ b/doc/gitlab-basics/add-merge-request.md @@ -1,42 +1,33 @@ # How to create a merge request -Merge Requests are useful to integrate separate changes that you've made to a project, on different branches. +Merge requests are useful to integrate separate changes that you've made to a +project, on different branches. This is a brief guide on how to create a merge +request. For more information, check the +[merge requests documentation](../user/project/merge_requests.md). -To create a new Merge Request, sign in to GitLab. +--- -Go to the project where you'd like to merge your changes: +1. Before you start, you should have already [created a branch](create-branch.md) + and [pushed your changes](basic-git-commands.md) to GitLab. - +1. You can then go to the project where you'd like to merge your changes and + click on the **Merge requests** tab. -Click on "Merge Requests" on the left side of your screen: +  - +1. Click on **New merge request** on the right side of the screen. -Click on "+ new Merge Request" on the right side of the screen: +  - +1. Select a source branch and click on the **Compare branches and continue** button. -Select a source branch or branch: +  - +1. At a minimum, add a title and a description to your merge request. Optionally, + select a user to review your merge request and to accept or close it. You may + also select a milestone and labels. -Click on the "compare branches" button: +  - - -Add a title and a description to your Merge Request: - - - -Select a user to review your Merge Request and to accept or close it. You may also select milestones and labels (they are optional). Then click on the "submit new Merge Request" button: - - - -Your Merge Request will be ready to be approved and published. - -### Note - -After you created a new branch, you'll immediately find a "create a Merge Request" button at the top of your screen. -You may automatically create a Merge Request from your recently created branch when clicking on this button: - - +1. When ready, click on the **Submit merge request** button. Your merge request + will be ready to be approved and published. diff --git a/doc/gitlab-basics/basicsimages/add_new_merge_request.png b/doc/gitlab-basics/basicsimages/add_new_merge_request.png Binary files differdeleted file mode 100644 index e60992c4c6a..00000000000 --- a/doc/gitlab-basics/basicsimages/add_new_merge_request.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/add_sshkey.png b/doc/gitlab-basics/basicsimages/add_sshkey.png Binary files differdeleted file mode 100644 index 89c86018629..00000000000 --- a/doc/gitlab-basics/basicsimages/add_sshkey.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/branch_info.png b/doc/gitlab-basics/basicsimages/branch_info.png Binary files differdeleted file mode 100644 index 2264f3c5bf2..00000000000 --- a/doc/gitlab-basics/basicsimages/branch_info.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/branch_name.png b/doc/gitlab-basics/basicsimages/branch_name.png Binary files differdeleted file mode 100644 index 75fe8313611..00000000000 --- a/doc/gitlab-basics/basicsimages/branch_name.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/branches.png b/doc/gitlab-basics/basicsimages/branches.png Binary files differdeleted file mode 100644 index 8621bc05776..00000000000 --- a/doc/gitlab-basics/basicsimages/branches.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/button-create-mr.png b/doc/gitlab-basics/basicsimages/button-create-mr.png Binary files differdeleted file mode 100644 index b52ab148839..00000000000 --- a/doc/gitlab-basics/basicsimages/button-create-mr.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/click-on-new-group.png b/doc/gitlab-basics/basicsimages/click-on-new-group.png Binary files differdeleted file mode 100644 index 6450deec6fc..00000000000 --- a/doc/gitlab-basics/basicsimages/click-on-new-group.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/commit_changes.png b/doc/gitlab-basics/basicsimages/commit_changes.png Binary files differdeleted file mode 100644 index a88809c5a3f..00000000000 --- a/doc/gitlab-basics/basicsimages/commit_changes.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/commit_message.png b/doc/gitlab-basics/basicsimages/commit_message.png Binary files differdeleted file mode 100644 index 4abe4517f98..00000000000 --- a/doc/gitlab-basics/basicsimages/commit_message.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/commits.png b/doc/gitlab-basics/basicsimages/commits.png Binary files differdeleted file mode 100644 index 2bfcaf75f01..00000000000 --- a/doc/gitlab-basics/basicsimages/commits.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/compare_branches.png b/doc/gitlab-basics/basicsimages/compare_branches.png Binary files differdeleted file mode 100644 index 8a18453dd05..00000000000 --- a/doc/gitlab-basics/basicsimages/compare_branches.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/create_file.png b/doc/gitlab-basics/basicsimages/create_file.png Binary files differdeleted file mode 100644 index 5ebe1b227dd..00000000000 --- a/doc/gitlab-basics/basicsimages/create_file.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/create_group.png b/doc/gitlab-basics/basicsimages/create_group.png Binary files differdeleted file mode 100644 index 7ecc3baa990..00000000000 --- a/doc/gitlab-basics/basicsimages/create_group.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/edit_file.png b/doc/gitlab-basics/basicsimages/edit_file.png Binary files differdeleted file mode 100644 index 9d3e817d036..00000000000 --- a/doc/gitlab-basics/basicsimages/edit_file.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/file_located.png b/doc/gitlab-basics/basicsimages/file_located.png Binary files differdeleted file mode 100644 index e357cb5c6ab..00000000000 --- a/doc/gitlab-basics/basicsimages/file_located.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/file_name.png b/doc/gitlab-basics/basicsimages/file_name.png Binary files differdeleted file mode 100644 index 01639c77d0d..00000000000 --- a/doc/gitlab-basics/basicsimages/file_name.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/find_file.png b/doc/gitlab-basics/basicsimages/find_file.png Binary files differdeleted file mode 100644 index 6f26d26ae18..00000000000 --- a/doc/gitlab-basics/basicsimages/find_file.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/find_group.png b/doc/gitlab-basics/basicsimages/find_group.png Binary files differdeleted file mode 100644 index 1211510aae9..00000000000 --- a/doc/gitlab-basics/basicsimages/find_group.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/fork.png b/doc/gitlab-basics/basicsimages/fork.png Binary files differdeleted file mode 100644 index 13ff8345616..00000000000 --- a/doc/gitlab-basics/basicsimages/fork.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/group_info.png b/doc/gitlab-basics/basicsimages/group_info.png Binary files differdeleted file mode 100644 index 2507d6c295b..00000000000 --- a/doc/gitlab-basics/basicsimages/group_info.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/groups.png b/doc/gitlab-basics/basicsimages/groups.png Binary files differdeleted file mode 100644 index ef3dca60cc8..00000000000 --- a/doc/gitlab-basics/basicsimages/groups.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/https.png b/doc/gitlab-basics/basicsimages/https.png Binary files differdeleted file mode 100644 index e74dbc13f9a..00000000000 --- a/doc/gitlab-basics/basicsimages/https.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/image_file.png b/doc/gitlab-basics/basicsimages/image_file.png Binary files differdeleted file mode 100644 index 7f304b8e1f2..00000000000 --- a/doc/gitlab-basics/basicsimages/image_file.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/issue_title.png b/doc/gitlab-basics/basicsimages/issue_title.png Binary files differdeleted file mode 100644 index 60a6f7973be..00000000000 --- a/doc/gitlab-basics/basicsimages/issue_title.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/issues.png b/doc/gitlab-basics/basicsimages/issues.png Binary files differdeleted file mode 100644 index 14e9cdb64e1..00000000000 --- a/doc/gitlab-basics/basicsimages/issues.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/key.png b/doc/gitlab-basics/basicsimages/key.png Binary files differdeleted file mode 100644 index 04400173ce8..00000000000 --- a/doc/gitlab-basics/basicsimages/key.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/merge_requests.png b/doc/gitlab-basics/basicsimages/merge_requests.png Binary files differdeleted file mode 100644 index 570164df18b..00000000000 --- a/doc/gitlab-basics/basicsimages/merge_requests.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/new_issue.png b/doc/gitlab-basics/basicsimages/new_issue.png Binary files differdeleted file mode 100644 index 94e7503dd8b..00000000000 --- a/doc/gitlab-basics/basicsimages/new_issue.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/new_merge_request.png b/doc/gitlab-basics/basicsimages/new_merge_request.png Binary files differdeleted file mode 100644 index 842f5ebed74..00000000000 --- a/doc/gitlab-basics/basicsimages/new_merge_request.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/new_project.png b/doc/gitlab-basics/basicsimages/new_project.png Binary files differdeleted file mode 100644 index 421e8bc247b..00000000000 --- a/doc/gitlab-basics/basicsimages/new_project.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/newbranch.png b/doc/gitlab-basics/basicsimages/newbranch.png Binary files differdeleted file mode 100644 index d5fcf33c4ea..00000000000 --- a/doc/gitlab-basics/basicsimages/newbranch.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/paste_sshkey.png b/doc/gitlab-basics/basicsimages/paste_sshkey.png Binary files differdeleted file mode 100644 index 578ebee4440..00000000000 --- a/doc/gitlab-basics/basicsimages/paste_sshkey.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/profile_settings.png b/doc/gitlab-basics/basicsimages/profile_settings.png Binary files differdeleted file mode 100644 index cb3f79f1879..00000000000 --- a/doc/gitlab-basics/basicsimages/profile_settings.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/project_info.png b/doc/gitlab-basics/basicsimages/project_info.png Binary files differdeleted file mode 100644 index e1adb8d48c2..00000000000 --- a/doc/gitlab-basics/basicsimages/project_info.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/select-group.png b/doc/gitlab-basics/basicsimages/select-group.png Binary files differdeleted file mode 100644 index 33b978dd899..00000000000 --- a/doc/gitlab-basics/basicsimages/select-group.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/select-group2.png b/doc/gitlab-basics/basicsimages/select-group2.png Binary files differdeleted file mode 100644 index aee22c638db..00000000000 --- a/doc/gitlab-basics/basicsimages/select-group2.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/select_branch.png b/doc/gitlab-basics/basicsimages/select_branch.png Binary files differdeleted file mode 100644 index f72a3ffb57f..00000000000 --- a/doc/gitlab-basics/basicsimages/select_branch.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/select_project.png b/doc/gitlab-basics/basicsimages/select_project.png Binary files differdeleted file mode 100644 index 3bb832ea8d0..00000000000 --- a/doc/gitlab-basics/basicsimages/select_project.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/settings.png b/doc/gitlab-basics/basicsimages/settings.png Binary files differdeleted file mode 100644 index 78637013d9b..00000000000 --- a/doc/gitlab-basics/basicsimages/settings.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/shh_keys.png b/doc/gitlab-basics/basicsimages/shh_keys.png Binary files differdeleted file mode 100644 index c87f11a9d3d..00000000000 --- a/doc/gitlab-basics/basicsimages/shh_keys.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/submit_new_issue.png b/doc/gitlab-basics/basicsimages/submit_new_issue.png Binary files differdeleted file mode 100644 index 78b854c8903..00000000000 --- a/doc/gitlab-basics/basicsimages/submit_new_issue.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/title_description_mr.png b/doc/gitlab-basics/basicsimages/title_description_mr.png Binary files differdeleted file mode 100644 index c31d61ec336..00000000000 --- a/doc/gitlab-basics/basicsimages/title_description_mr.png +++ /dev/null diff --git a/doc/gitlab-basics/basicsimages/white_space.png b/doc/gitlab-basics/basicsimages/white_space.png Binary files differdeleted file mode 100644 index eaa969bdcf4..00000000000 --- a/doc/gitlab-basics/basicsimages/white_space.png +++ /dev/null diff --git a/doc/gitlab-basics/command-line-commands.md b/doc/gitlab-basics/command-line-commands.md index addd3b6b6eb..3b075ff5fc0 100644 --- a/doc/gitlab-basics/command-line-commands.md +++ b/doc/gitlab-basics/command-line-commands.md @@ -4,18 +4,21 @@ In Git, when you copy a project you say you "clone" it. To work on a git project locally (from your own computer), you will need to clone it. To do this, sign in to GitLab. -When you are on your Dashboard, click on the project that you'd like to clone, which you'll find at the right side of your screen. +When you are on your Dashboard, click on the project that you'd like to clone. +To work in the project, you can copy a link to the Git repository through a SSH +or a HTTPS protocol. SSH is easier to use after it's been +[setup](create-your-ssh-keys.md). While you are at the **Project** tab, select +HTTPS or SSH from the dropdown menu and copy the link using the 'Copy to clipboard' +button (you'll have to paste it on your shell in the next step). - - -To work in the project, you can copy a link to the Git repository through a SSH or a HTTPS protocol. SSH is easier to use after it's been [setup](create-your-ssh-keys.md). When you're in the project, click on the HTTPS or SSH button at the right side of your screen. Then copy the link (you'll have to paste it on your shell in the next step). - - + ## On the command line ### Clone your project + Go to your computer's shell and type the following command: + ``` git clone PASTE HTTPS OR SSH HERE ``` @@ -23,26 +26,31 @@ git clone PASTE HTTPS OR SSH HERE A clone of the project will be created in your computer. ### Go into a project, directory or file to work in it + ``` cd NAME-OF-PROJECT-OR-FILE ``` ### Go back one directory or file + ``` cd ../ ``` ### View what’s in the directory that you are in + ``` ls ``` ### Create a directory + ``` mkdir NAME-OF-YOUR-DIRECTORY ``` ### Create a README.md or file in directory + ``` touch README.md nano README.md @@ -53,27 +61,33 @@ nano README.md ``` ### Remove a file + ``` rm NAME-OF-FILE ``` ### Remove a directory and all of its contents + ``` rm -rf NAME-OF-DIRECTORY ``` ### View history in the command line + ``` history ``` ### Carry out commands for which the account you are using lacks authority + You will be asked for an administrator’s password. + ``` sudo ``` ### Tell where you are + ``` pwd ``` diff --git a/doc/gitlab-basics/create-branch.md b/doc/gitlab-basics/create-branch.md index 7556b0f663e..ad94f0dad29 100644 --- a/doc/gitlab-basics/create-branch.md +++ b/doc/gitlab-basics/create-branch.md @@ -2,38 +2,11 @@ A branch is an independent line of development. -New commits are recorded in the history for the current branch, which results in taking the source from someone’s repository (the place where the history of your work is stored) at certain point in time, and apply your own changes to it in the history of the project. - -To add changes to your GitLab project, you should create a branch. You can do it in your [shell](basic-git-commands.md) or in GitLab. - -To create a new branch in GitLab, sign in and then select a project on the right side of your screen: - - - -Click on "commits" on the menu on the left side of your screen: - - - -Click on the "branches" tab: - - - -Click on the "new branch" button on the right side of the screen: - - - -Fill out the information required: - -1. Add a name for your new branch (you can't add spaces, so you can use hyphens or underscores) - -1. On the "create from" space, add the the name of the branch you want to branch off from - -1. Click on the button "create branch" - - - -### Note: - -You will be able to find and select the name of your branch in the white box next to a project's name: - - +New commits are recorded in the history for the current branch, which results +in taking the source from someone’s repository (the place where the history of +your work is stored) at certain point in time, and apply your own changes to it +in the history of the project. + +To add changes to your GitLab project, you should create a branch. You can do +it in your [terminal](basic-git-commands.md) or by +[using the web interface](../user/project/repository/web_editor.md#create-a-new-branch). diff --git a/doc/gitlab-basics/create-group.md b/doc/gitlab-basics/create-group.md index f80ae62e442..64274ccd5eb 100644 --- a/doc/gitlab-basics/create-group.md +++ b/doc/gitlab-basics/create-group.md @@ -1,43 +1,48 @@ # How to create a group in GitLab -## Create a group - Your projects in GitLab can be organized in 2 different ways: -under your own namespace for single projects, such as ´your-name/project-1'; or under groups. -If you organize your projects under a group, it works like a folder. You can manage your group members' permissions and access to the projects. - -To create a group, follow the instructions below: +under your own namespace for single projects, such as `your-name/project-1` or +under groups. -Sign in to [GitLab.com](https://gitlab.com). +If you organize your projects under a group, it works like a folder. You can +manage your group members' permissions and access to the projects. -When you are on your Dashboard, click on "Groups" on the left menu of your screen: +--- - +To create a group: -Click on "New group" on the top right side of your screen: +1. Expand the left sidebar by clicking the three bars at the upper left corner + and then navigate to **Groups**. - +  -Fill out the information required: +1. Once in your groups dashboard, click on **New group**. -1. Add a group path or group name (you can't add spaces, so you can use hyphens or underscores) +  -1. Add details or a group description +1. Fill out the needed information: -1. You can choose a group avatar if you'd like + 1. Set the "Group path" which will be the namespace under which your projects + will be hosted (path can contain only letters, digits, underscores, dashes + and dots; it cannot start with dashes or end in dot). + 1. Optionally, you can add a description so that others can briefly understand + what this group is about. + 1. Optionally, choose and avatar for your project. + 1. Choose the [visibility level](../public_access/public_access.md). -1. Click on "create group" +1. Finally, click the **Create group** button. - - -## Add a project to a group +## Add a new project to a group There are 2 different ways to add a new project to a group: -* Select a group and then click on "New project" on the right side of your screen. Then you can [create a project](create-project.md) +- Select a group and then click on the **New project** button. + +  - + You can then continue on [creating a project](create-project.md). -* When you are [creating a project](create-project.md), click on "create a group" on the bottom right side of your screen +- While you are [creating a project](create-project.md), select a group namespace + you've already created from the dropdown menu. - +  diff --git a/doc/gitlab-basics/create-issue.md b/doc/gitlab-basics/create-issue.md index 5221d85b661..13e5a738c89 100644 --- a/doc/gitlab-basics/create-issue.md +++ b/doc/gitlab-basics/create-issue.md @@ -1,27 +1,30 @@ # How to create an Issue in GitLab -The Issue Tracker is a good place to add things that need to be improved or solved in a project. +The issue tracker is a good place to add things that need to be improved or +solved in a project. -To create an Issue, sign in to GitLab. +--- -Go to the project where you'd like to create the Issue: +1. Go to the project where you'd like to create the issue and navigate to the + **Issues** tab on top. - +  -Click on "Issues" on the left side of your screen: +1. Click on the **New issue** button on the right side of your screen. - +  -Click on the "+ new issue" button on the right side of your screen: +1. At the very minimum, add a title and a description to your issue. + You may assign it to a user, add a milestone or add labels (all optional). - +  -Add a title and a description to your issue: +1. When ready, click on **Submit issue**. - +--- -You may assign the Issue to a user, add a milestone and add labels (they are all optional). Then click on "submit new issue": - - - -Your Issue will now be added to the Issue Tracker and will be ready to be reviewed. You can comment on it and mention the people involved. You can also link Issues to the Merge Requests where the Issues are solved. To do this, you can use an [Issue closing pattern](http://docs.gitlab.com/ce/customization/issue_closing.html). +Your Issue will now be added to the issue tracker of the project you opened it +at and will be ready to be reviewed. You can comment on it and mention the +people involved. You can also link issues to the merge requests where the issues +are solved. To do this, you can use an +[issue closing pattern](../user/project/issues/automatic_issue_closing.md). diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md index f737dffc024..3f45a631b3a 100644 --- a/doc/gitlab-basics/create-project.md +++ b/doc/gitlab-basics/create-project.md @@ -1,21 +1,24 @@ # How to create a project in GitLab -To create a new project, sign in to GitLab. +There are two ways to create a new project in GitLab. -Go to your Dashboard and click on "new project" on the right side of your screen. +1. While in your dashboard, you can create a new project using the **New project** + green button or you can use the cross icon in the upper right corner next to + your avatar which is always visible. - +  -Fill out the required information: +1. From there you can see several options. -1. Project path or the name of your project (you can't add spaces, so you can use hyphens or underscores) +  -1. Your project's description +1. Fill out the information: -1. Select a [visibility level](https://gitlab.com/help/public_access/public_access) + 1. "Project name" is the name of your project (you can't use spaces, but you + can use hyphens or underscores). + 1. The "Project description" is optional and will be shown in your project's + dashboard so others can briefly understand what your project is about. + 1. Select a [visibility level](../public_access/public_access.md). + 1. You can also [import your existing projects](../workflow/importing/README.md). -1. You can also [import your existing projects](http://docs.gitlab.com/ce/workflow/importing/README.html) - -1. Click on "create project" - -! +1. Finally, click **Create project**. diff --git a/doc/gitlab-basics/create-your-ssh-keys.md b/doc/gitlab-basics/create-your-ssh-keys.md index f31c353f2cf..b6ebe374de3 100644 --- a/doc/gitlab-basics/create-your-ssh-keys.md +++ b/doc/gitlab-basics/create-your-ssh-keys.md @@ -1,33 +1,37 @@ # How to create your SSH Keys -You need to connect your computer to your GitLab account through SSH Keys. They are unique for every computer that you link your GitLab account with. +1. The first thing you need to do is go to your [command line](start-using-git.md) + and follow the [instructions](../ssh/README.md) to generate your SSH key pair. -## Generate your SSH Key +1. Once you do that, login to GitLab with your credentials. +1. On the upper right corner, click on your avatar and go to your **Profile settings**. -Create an account on GitLab. Sign up and check your email for your confirmation link. +  -After you confirm, go to GitLab and sign in to your account. +1. Navigate to the **SSH keys** tab. -## Add your SSH Key +  -On the left side menu, click on "profile settings" and then click on "SSH Keys": +3. Paste your **public** key that you generated in the first step in the 'Key' + box. - +  -Then click on the green button "Add SSH Key": +1. Optionally, give it a descriptive title so that you can recognize it in the + event you add multiple keys. - +  -There, you should paste the SSH Key that your command line will generate for you. Below you'll find the steps to generate it: +1. Finally, click on **Add key** to add it to GitLab. You will be able to see + its fingerprint, its title and creation date. - +  -## To generate an SSH Key on your command line -Go to your [command line](start-using-git.md) and follow the [instructions](../ssh/README.md) to generate it. +>**Note:** +Once you add a key, you cannot edit it, only remove it. In case the paste +didn't work, you will have to remove the offending key and re-add it. -Copy the SSH Key that your command line created and paste it on the "Key" box on the GitLab page. The title will be added automatically. +--- - - -Now, you'll be able to use Git over SSH, instead of Git over HTTP. +Congratulations! You are now ready to use Git over SSH, instead of Git over HTTP! diff --git a/doc/gitlab-basics/fork-project.md b/doc/gitlab-basics/fork-project.md index 5f8b81ea919..6c232fe6086 100644 --- a/doc/gitlab-basics/fork-project.md +++ b/doc/gitlab-basics/fork-project.md @@ -1,19 +1,20 @@ # How to fork a project -A fork is a copy of an original repository that you can put somewhere else -or where you can experiment and apply changes that you can later decide if +A fork is a copy of an original repository that you can put in another namespace +where you can experiment and apply changes that you can later decide if publishing or not, without affecting your original project. It takes just a few steps to fork a project in GitLab. -Sign in to GitLab. +1. Go to a project's dashboard under the **Project** tab and click on the + **Fork** button. -Select a project on the right side of your screen: +  - +1. You will be asked where to fork the repository. Click on the user or group + to where you'd like to add the forked project. -Click on the "fork" button on the right side of your screen: +  - - -Click on the user or group to where you'd like to add the forked project. +1. After a few moments, depending on the repository's size, the forking will + complete. diff --git a/doc/gitlab-basics/img/create_new_group_info.png b/doc/gitlab-basics/img/create_new_group_info.png Binary files differnew file mode 100644 index 00000000000..c8eddfd1bbb --- /dev/null +++ b/doc/gitlab-basics/img/create_new_group_info.png diff --git a/doc/gitlab-basics/img/create_new_group_sidebar.png b/doc/gitlab-basics/img/create_new_group_sidebar.png Binary files differnew file mode 100644 index 00000000000..28017ee02e0 --- /dev/null +++ b/doc/gitlab-basics/img/create_new_group_sidebar.png diff --git a/doc/gitlab-basics/img/create_new_project_button.png b/doc/gitlab-basics/img/create_new_project_button.png Binary files differnew file mode 100644 index 00000000000..e7c794d943f --- /dev/null +++ b/doc/gitlab-basics/img/create_new_project_button.png diff --git a/doc/gitlab-basics/img/create_new_project_from_group.png b/doc/gitlab-basics/img/create_new_project_from_group.png Binary files differnew file mode 100644 index 00000000000..6d41d17f9ca --- /dev/null +++ b/doc/gitlab-basics/img/create_new_project_from_group.png diff --git a/doc/gitlab-basics/img/create_new_project_info.png b/doc/gitlab-basics/img/create_new_project_info.png Binary files differnew file mode 100644 index 00000000000..16d56f0707f --- /dev/null +++ b/doc/gitlab-basics/img/create_new_project_info.png diff --git a/doc/gitlab-basics/img/fork_choose_namespace.png b/doc/gitlab-basics/img/fork_choose_namespace.png Binary files differnew file mode 100644 index 00000000000..82c9c3bd39e --- /dev/null +++ b/doc/gitlab-basics/img/fork_choose_namespace.png diff --git a/doc/gitlab-basics/img/fork_new.png b/doc/gitlab-basics/img/fork_new.png Binary files differnew file mode 100644 index 00000000000..41885223286 --- /dev/null +++ b/doc/gitlab-basics/img/fork_new.png diff --git a/doc/gitlab-basics/img/merge_request_new.png b/doc/gitlab-basics/img/merge_request_new.png Binary files differnew file mode 100644 index 00000000000..0aba5743f01 --- /dev/null +++ b/doc/gitlab-basics/img/merge_request_new.png diff --git a/doc/gitlab-basics/img/merge_request_page.png b/doc/gitlab-basics/img/merge_request_page.png Binary files differnew file mode 100644 index 00000000000..68c3bbf9444 --- /dev/null +++ b/doc/gitlab-basics/img/merge_request_page.png diff --git a/doc/gitlab-basics/img/merge_request_select_branch.png b/doc/gitlab-basics/img/merge_request_select_branch.png Binary files differnew file mode 100644 index 00000000000..516436ff6cc --- /dev/null +++ b/doc/gitlab-basics/img/merge_request_select_branch.png diff --git a/doc/gitlab-basics/img/new_issue_button.png b/doc/gitlab-basics/img/new_issue_button.png Binary files differnew file mode 100644 index 00000000000..46b626bed65 --- /dev/null +++ b/doc/gitlab-basics/img/new_issue_button.png diff --git a/doc/gitlab-basics/img/new_issue_page.png b/doc/gitlab-basics/img/new_issue_page.png Binary files differnew file mode 100644 index 00000000000..843504130b7 --- /dev/null +++ b/doc/gitlab-basics/img/new_issue_page.png diff --git a/doc/gitlab-basics/img/profile_settings.png b/doc/gitlab-basics/img/profile_settings.png Binary files differnew file mode 100644 index 00000000000..f0abd478849 --- /dev/null +++ b/doc/gitlab-basics/img/profile_settings.png diff --git a/doc/gitlab-basics/img/profile_settings_ssh_keys.png b/doc/gitlab-basics/img/profile_settings_ssh_keys.png Binary files differnew file mode 100644 index 00000000000..2c9a42fe10c --- /dev/null +++ b/doc/gitlab-basics/img/profile_settings_ssh_keys.png diff --git a/doc/gitlab-basics/img/profile_settings_ssh_keys_paste_pub.png b/doc/gitlab-basics/img/profile_settings_ssh_keys_paste_pub.png Binary files differnew file mode 100644 index 00000000000..cd7add6937f --- /dev/null +++ b/doc/gitlab-basics/img/profile_settings_ssh_keys_paste_pub.png diff --git a/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png b/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png Binary files differnew file mode 100644 index 00000000000..095beb02be8 --- /dev/null +++ b/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png diff --git a/doc/gitlab-basics/img/profile_settings_ssh_keys_title.png b/doc/gitlab-basics/img/profile_settings_ssh_keys_title.png Binary files differnew file mode 100644 index 00000000000..4b998a7f948 --- /dev/null +++ b/doc/gitlab-basics/img/profile_settings_ssh_keys_title.png diff --git a/doc/gitlab-basics/img/project_clone_url.png b/doc/gitlab-basics/img/project_clone_url.png Binary files differnew file mode 100644 index 00000000000..eed430e1036 --- /dev/null +++ b/doc/gitlab-basics/img/project_clone_url.png diff --git a/doc/gitlab-basics/img/project_navbar.png b/doc/gitlab-basics/img/project_navbar.png Binary files differnew file mode 100644 index 00000000000..97cf3cd9702 --- /dev/null +++ b/doc/gitlab-basics/img/project_navbar.png diff --git a/doc/gitlab-basics/basicsimages/public_file_link.png b/doc/gitlab-basics/img/public_file_link.png Binary files differindex f60df6807f4..f60df6807f4 100644 --- a/doc/gitlab-basics/basicsimages/public_file_link.png +++ b/doc/gitlab-basics/img/public_file_link.png diff --git a/doc/gitlab-basics/img/select_group_dropdown.png b/doc/gitlab-basics/img/select_group_dropdown.png Binary files differnew file mode 100644 index 00000000000..7d8b89c2df9 --- /dev/null +++ b/doc/gitlab-basics/img/select_group_dropdown.png diff --git a/doc/gitlab-basics/start-using-git.md b/doc/gitlab-basics/start-using-git.md index 89ce8bcc3e8..42cd8bb3e48 100644 --- a/doc/gitlab-basics/start-using-git.md +++ b/doc/gitlab-basics/start-using-git.md @@ -1,11 +1,10 @@ # Start using Git on the command line -If you want to start using a Git and GitLab, make sure that you have created an -account on GitLab. +If you want to start using Git and GitLab, make sure that you have created and/or signed into an account on GitLab. ## Open a shell -Depending on your operating system, find the shell of your preference. Here are some suggestions. +Depending on your operating system, you will need to use a shell of your preference. Here are some suggestions: - [Terminal](http://blog.teamtreehouse.com/introduction-to-the-mac-os-x-command-line) on Mac OSX @@ -22,19 +21,19 @@ Type the following command and then press enter: git --version ``` -You should receive a message that will tell you which Git version you have in your computer. If you don’t receive a "Git version" message, it means that you need to [download Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). +You should receive a message that will tell you which Git version you have on your computer. If you don’t receive a "Git version" message, it means that you need to [download Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). If Git doesn't automatically download, there's an option on the website to [download manually](https://git-scm.com/downloads). Then follow the steps on the installation window. -After you finished installing, open a new shell and type "git --version" again to verify that it was correctly installed. +After you are finished installing, open a new shell and type "git --version" again to verify that it was correctly installed. ## Add your Git username and set your email -It is important because every Git commit that you create will use this information. +It is important to configure your Git username and email address as every Git commit will use this information to identify you as the author. On your shell, type the following command to add your username: ``` -git config --global user.name ADD YOUR USERNAME +git config --global user.name "YOUR_USERNAME" ``` Then verify that you have the correct username: @@ -44,7 +43,7 @@ git config --global user.name To set your email address, type the following command: ``` -git config --global user.email ADD YOUR EMAIL +git config --global user.email "your_email_address@example.com" ``` To verify that you entered your email correctly, type: @@ -52,7 +51,7 @@ To verify that you entered your email correctly, type: git config --global user.email ``` -You'll need to do this only once because you are using the "--global" option. It tells Git to always use this information for anything you do on that system. If you want to override this with a different username or email address for specific projects, you can run the command without the "--global" option when you’re in that project. +You'll need to do this only once as you are using the `--global` option. It tells Git to always use this information for anything you do on that system. If you want to override this with a different username or email address for specific projects, you can run the command without the `--global` option when you’re in that project. ## Check your information @@ -76,7 +75,7 @@ git pull REMOTE NAME-OF-BRANCH -u (REMOTE: origin) (NAME-OF-BRANCH: could be "master" or an existing branch) ### Create a branch -Spaces won't be recognized, so you need to use a hyphen or underscore. +Spaces won't be recognized, so you will need to use a hyphen or underscore. ``` git checkout -b NAME-OF-BRANCH ``` @@ -120,3 +119,10 @@ You need to be in the created branch. git checkout NAME-OF-BRANCH git merge master ``` + +### Merge master branch with created branch +You need to be in the master branch. +``` +git checkout master +git merge NAME-OF-BRANCH +``` diff --git a/doc/incoming_email/README.md b/doc/incoming_email/README.md index 5a9a1582877..db0f03f2c98 100644 --- a/doc/incoming_email/README.md +++ b/doc/incoming_email/README.md @@ -1,302 +1 @@ -# Reply by email - -GitLab can be set up to allow users to comment on issues and merge requests by -replying to notification emails. - -## Requirement - -Reply by email requires an IMAP-enabled email account. GitLab allows you to use -three strategies for this feature: -- using email sub-addressing -- using a dedicated email address -- using a catch-all mailbox - -### Email sub-addressing - -**If your provider or server supports email sub-addressing, we recommend using it.** - -[Sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing) is -a feature where any email to `user+some_arbitrary_tag@example.com` will end up -in the mailbox for `user@example.com`, and is supported by providers such as -Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix -mail server which you can run on-premises. - -### Dedicated email address - -This solution is really simple to set up: you just have to create an email -address dedicated to receive your users' replies to GitLab notifications. - -### Catch-all mailbox - -A [catch-all mailbox](https://en.wikipedia.org/wiki/Catch-all) for a domain will -"catch all" the emails addressed to the domain that do not exist in the mail -server. - -## How it works? - -### 1. GitLab sends a notification email - -When GitLab sends a notification and Reply by email is enabled, the `Reply-To` -header is set to the address defined in your GitLab configuration, with the -`%{key}` placeholder (if present) replaced by a specific "reply key". In -addition, this "reply key" is also added to the `References` header. - -### 2. You reply to the notification email - -When you reply to the notification email, your email client will: - -- send the email to the `Reply-To` address it got from the notification email -- set the `In-Reply-To` header to the value of the `Message-ID` header from the - notification email -- set the `References` header to the value of the `Message-ID` plus the value of - the notification email's `References` header. - -### 3. GitLab receives your reply to the notification email - -When GitLab receives your reply, it will look for the "reply key" in the -following headers, in this order: - -1. the `To` header -1. the `References` header - -If it finds a reply key, it will be able to leave your reply as a comment on -the entity the notification was about (issue, merge request, commit...). - -For more details about the `Message-ID`, `In-Reply-To`, and `References headers`, -please consult [RFC 5322](https://tools.ietf.org/html/rfc5322#section-3.6.4). - -## Set it up - -If you want to use Gmail / Google Apps with Reply by email, make sure you have -[IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) -and [allowed less secure apps to access the account](https://support.google.com/accounts/answer/6010255). - -To set up a basic Postfix mail server with IMAP access on Ubuntu, follow -[these instructions](./postfix.md). - -### Omnibus package installations - -1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the - feature and fill in the details for your specific IMAP server and email account: - - ```ruby - # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com - gitlab_rails['incoming_email_enabled'] = true - - # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). - gitlab_rails['incoming_email_address'] = "incoming+%{key}@gitlab.example.com" - - # Email account username - # With third party providers, this is usually the full email address. - # With self-hosted email servers, this is usually the user part of the email address. - gitlab_rails['incoming_email_email'] = "incoming" - # Email account password - gitlab_rails['incoming_email_password'] = "[REDACTED]" - - # IMAP server host - gitlab_rails['incoming_email_host'] = "gitlab.example.com" - # IMAP server port - gitlab_rails['incoming_email_port'] = 143 - # Whether the IMAP server uses SSL - gitlab_rails['incoming_email_ssl'] = false - # Whether the IMAP server uses StartTLS - gitlab_rails['incoming_email_start_tls'] = false - - # The mailbox where incoming mail will end up. Usually "inbox". - gitlab_rails['incoming_email_mailbox_name'] = "inbox" - ``` - - ```ruby - # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com - gitlab_rails['incoming_email_enabled'] = true - - # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). - gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com" - - # Email account username - # With third party providers, this is usually the full email address. - # With self-hosted email servers, this is usually the user part of the email address. - gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com" - # Email account password - gitlab_rails['incoming_email_password'] = "[REDACTED]" - - # IMAP server host - gitlab_rails['incoming_email_host'] = "imap.gmail.com" - # IMAP server port - gitlab_rails['incoming_email_port'] = 993 - # Whether the IMAP server uses SSL - gitlab_rails['incoming_email_ssl'] = true - # Whether the IMAP server uses StartTLS - gitlab_rails['incoming_email_start_tls'] = false - - # The mailbox where incoming mail will end up. Usually "inbox". - gitlab_rails['incoming_email_mailbox_name'] = "inbox" - ``` - -1. Reconfigure GitLab and restart mailroom for the changes to take effect: - - ```sh - sudo gitlab-ctl reconfigure - sudo gitlab-ctl restart mailroom - ``` - -1. Verify that everything is configured correctly: - - ```sh - sudo gitlab-rake gitlab:incoming_email:check - ``` - -1. Reply by email should now be working. - -### Installations from source - -1. Go to the GitLab installation directory: - - ```sh - cd /home/git/gitlab - ``` - -1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature - and fill in the details for your specific IMAP server and email account: - - ```sh - sudo editor config/gitlab.yml - ``` - - ```yaml - # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com - incoming_email: - enabled: true - - # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). - address: "incoming+%{key}@gitlab.example.com" - - # Email account username - # With third party providers, this is usually the full email address. - # With self-hosted email servers, this is usually the user part of the email address. - user: "incoming" - # Email account password - password: "[REDACTED]" - - # IMAP server host - host: "gitlab.example.com" - # IMAP server port - port: 143 - # Whether the IMAP server uses SSL - ssl: false - # Whether the IMAP server uses StartTLS - start_tls: false - - # The mailbox where incoming mail will end up. Usually "inbox". - mailbox: "inbox" - ``` - - ```yaml - # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com - incoming_email: - enabled: true - - # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). - address: "gitlab-incoming+%{key}@gmail.com" - - # Email account username - # With third party providers, this is usually the full email address. - # With self-hosted email servers, this is usually the user part of the email address. - user: "gitlab-incoming@gmail.com" - # Email account password - password: "[REDACTED]" - - # IMAP server host - host: "imap.gmail.com" - # IMAP server port - port: 993 - # Whether the IMAP server uses SSL - ssl: true - # Whether the IMAP server uses StartTLS - start_tls: false - - # The mailbox where incoming mail will end up. Usually "inbox". - mailbox: "inbox" - ``` - -1. Enable `mail_room` in the init script at `/etc/default/gitlab`: - - ```sh - sudo mkdir -p /etc/default - echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab - ``` - -1. Restart GitLab: - - ```sh - sudo service gitlab restart - ``` - -1. Verify that everything is configured correctly: - - ```sh - sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production - ``` - -1. Reply by email should now be working. - -### Development - -1. Go to the GitLab installation directory. - -1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account: - - ```yaml - # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com - incoming_email: - enabled: true - - # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). - address: "gitlab-incoming+%{key}@gmail.com" - - # Email account username - # With third party providers, this is usually the full email address. - # With self-hosted email servers, this is usually the user part of the email address. - user: "gitlab-incoming@gmail.com" - # Email account password - password: "[REDACTED]" - - # IMAP server host - host: "imap.gmail.com" - # IMAP server port - port: 993 - # Whether the IMAP server uses SSL - ssl: true - # Whether the IMAP server uses StartTLS - start_tls: false - - # The mailbox where incoming mail will end up. Usually "inbox". - mailbox: "inbox" - ``` - - As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. - -1. Uncomment the `mail_room` line in your `Procfile`: - - ```yaml - mail_room: bundle exec mail_room -q -c config/mail_room.yml - ``` - -1. Restart GitLab: - - ```sh - bundle exec foreman start - ``` - -1. Verify that everything is configured correctly: - - ```sh - bundle exec rake gitlab:incoming_email:check RAILS_ENV=development - ``` - -1. Reply by email should now be working. +This document was moved to [administration/reply_by_email](../administration/reply_by_email.md). diff --git a/doc/incoming_email/postfix.md b/doc/incoming_email/postfix.md index 787d21f7f8f..90833238ac5 100644 --- a/doc/incoming_email/postfix.md +++ b/doc/incoming_email/postfix.md @@ -1,321 +1 @@ -# Set up Postfix for Reply by email - -This document will take you through the steps of setting up a basic Postfix mail server with IMAP authentication on Ubuntu, to be used with Reply by email. - -The instructions make the assumption that you will be using the email address `incoming@gitlab.example.com`, that is, username `incoming` on host `gitlab.example.com`. Don't forget to change it to your actual host when executing the example code snippets. - -## Configure your server firewall - -1. Open up port 25 on your server so that people can send email into the server over SMTP. -2. If the mail server is different from the server running GitLab, open up port 143 on your server so that GitLab can read email from the server over IMAP. - -## Install packages - -1. Install the `postfix` package if it is not installed already: - - ```sh - sudo apt-get install postfix - ``` - - When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`. - -1. Install the `mailutils` package. - - ```sh - sudo apt-get install mailutils - ``` - -## Create user - -1. Create a user for incoming email. - - ```sh - sudo useradd -m -s /bin/bash incoming - ``` - -1. Set a password for this user. - - ```sh - sudo passwd incoming - ``` - - Be sure not to forget this, you'll need it later. - -## Test the out-of-the-box setup - -1. Connect to the local SMTP server: - - ```sh - telnet localhost 25 - ``` - - You should see a prompt like this: - - ```sh - Trying 127.0.0.1... - Connected to localhost. - Escape character is '^]'. - 220 gitlab.example.com ESMTP Postfix (Ubuntu) - ``` - - If you get a `Connection refused` error instead, verify that `postfix` is running: - - ```sh - sudo postfix status - ``` - - If it is not, start it: - - ```sh - sudo postfix start - ``` - -1. Send the new `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: - - ``` - ehlo localhost - mail from: root@localhost - rcpt to: incoming@localhost - data - Subject: Re: Some issue - - Sounds good! - . - quit - ``` - - _**Note:** The `.` is a literal period on its own line._ - - _**Note:** If you receive an error after entering `rcpt to: incoming@localhost` - then your Postfix `my_network` configuration is not correct. The error will - say 'Temporary lookup failure'. See - [Configure Postfix to receive email from the Internet](#configure-postfix-to-receive-email-from-the-internet)._ - -1. Check if the `incoming` user received the email: - - ```sh - su - incoming - mail - ``` - - You should see output like this: - - ``` - "/var/mail/incoming": 1 message 1 unread - >U 1 root@localhost 59/2842 Re: Some issue - ``` - - Quit the mail app: - - ```sh - q - ``` - -1. Log out of the `incoming` account and go back to being `root`: - - ```sh - logout - ``` - -## Configure Postfix to use Maildir-style mailboxes - -Courier, which we will install later to add IMAP authentication, requires mailboxes to have the Maildir format, rather than mbox. - -1. Configure Postfix to use Maildir-style mailboxes: - - ```sh - sudo postconf -e "home_mailbox = Maildir/" - ``` - -1. Restart Postfix: - - ```sh - sudo /etc/init.d/postfix restart - ``` - -1. Test the new setup: - - 1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_. - 1. Check if the `incoming` user received the email: - - ```sh - su - incoming - MAIL=/home/incoming/Maildir - mail - ``` - - You should see output like this: - - ``` - "/home/incoming/Maildir": 1 message 1 unread - >U 1 root@localhost 59/2842 Re: Some issue - ``` - - Quit the mail app: - - ```sh - q - ``` - - _**Note:** If `mail` returns an error `Maildir: Is a directory` then your - version of `mail` doesn't support Maildir style mailboxes. Install - `heirloom-mailx` by running `sudo apt-get install heirloom-mailx`. Then, - try the above steps again, substituting `heirloom-mailx` for the `mail` - command._ - -1. Log out of the `incoming` account and go back to being `root`: - - ```sh - logout - ``` - -## Install the Courier IMAP server - -1. Install the `courier-imap` package: - - ```sh - sudo apt-get install courier-imap - ``` - -## Configure Postfix to receive email from the internet - -1. Let Postfix know about the domains that it should consider local: - - ```sh - sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost" - ``` - -1. Let Postfix know about the IPs that it should consider part of the LAN: - - We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network. - - ```sh - sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24" - ``` - -1. Configure Postfix to receive mail on all interfaces, which includes the internet: - - ```sh - sudo postconf -e "inet_interfaces = all" - ``` - -1. Configure Postfix to use the `+` delimiter for sub-addressing: - - ```sh - sudo postconf -e "recipient_delimiter = +" - ``` - -1. Restart Postfix: - - ```sh - sudo service postfix restart - ``` - -## Test the final setup - -1. Test SMTP under the new setup: - - 1. Connect to the SMTP server: - - ```sh - telnet gitlab.example.com 25 - ``` - - You should see a prompt like this: - - ```sh - Trying 123.123.123.123... - Connected to gitlab.example.com. - Escape character is '^]'. - 220 gitlab.example.com ESMTP Postfix (Ubuntu) - ``` - - If you get a `Connection refused` error instead, make sure your firewall is setup to allow inbound traffic on port 25. - - 1. Send the `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: - - ``` - ehlo gitlab.example.com - mail from: root@gitlab.example.com - rcpt to: incoming@gitlab.example.com - data - Subject: Re: Some issue - - Sounds good! - . - quit - ``` - - (Note: The `.` is a literal period on its own line) - - 1. Check if the `incoming` user received the email: - - ```sh - su - incoming - MAIL=/home/incoming/Maildir - mail - ``` - - You should see output like this: - - ``` - "/home/incoming/Maildir": 1 message 1 unread - >U 1 root@gitlab.example.com 59/2842 Re: Some issue - ``` - - Quit the mail app: - - ```sh - q - ``` - - 1. Log out of the `incoming` account and go back to being `root`: - - ```sh - logout - ``` - -1. Test IMAP under the new setup: - - 1. Connect to the IMAP server: - - ```sh - telnet gitlab.example.com 143 - ``` - - You should see a prompt like this: - - ```sh - Trying 123.123.123.123... - Connected to mail.example.gitlab.com. - Escape character is '^]'. - - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information. - ``` - - 1. Sign in as the `incoming` user to test IMAP, by entering the following into the IMAP prompt: - - ``` - a login incoming PASSWORD - ``` - - Replace PASSWORD with the password you set on the `incoming` user earlier. - - You should see output like this: - - ``` - a OK LOGIN Ok. - ``` - - 1. Disconnect from the IMAP server: - - ```sh - a logout - ``` - -## Done! - -If all the tests were successful, Postfix is all set up and ready to receive email! Continue with the [Reply by email](./README.md) guide to configure GitLab. - ---------- - -_This document was adapted from https://help.ubuntu.com/community/PostfixBasicSetupHowto, by contributors to the Ubuntu documentation wiki._ +This document was moved to [administration/reply_by_email_postfix_setup](../administration/reply_by_email_postfix_setup.md). diff --git a/doc/install/installation.md b/doc/install/installation.md index 19d083d580d..c9acc9cdfb0 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -89,7 +89,7 @@ Is the system packaged Git too old? Remove it and compile from source. # Download and compile from source cd /tmp - curl -O --progress https://www.kernel.org/pub/software/scm/git/git-2.7.4.tar.gz + curl --remote-name --progress https://www.kernel.org/pub/software/scm/git/git-2.7.4.tar.gz echo '7104c4f5d948a75b499a954524cb281fe30c6649d8abe20982936f75ec1f275b git-2.7.4.tar.gz' | shasum -a256 -c - && tar -xzf git-2.7.4.tar.gz cd git-2.7.4/ ./configure @@ -108,8 +108,7 @@ Then select 'Internet Site' and press enter to confirm the hostname. ## 2. Ruby -_**Note:** The current supported Ruby version is 2.1.x. Ruby 2.2 and 2.3 are -currently not supported._ +**Note:** The current supported Ruby versions are 2.1.x and 2.3.x. 2.3.x is preferred, and support for 2.1.x will be dropped in the future. The use of Ruby version managers such as [RVM], [rbenv] or [chruby] with GitLab in production, frequently leads to hard to diagnose problems. For example, @@ -124,9 +123,9 @@ Remove the old Ruby 1.8 if present: Download Ruby and compile it: mkdir /tmp/ruby && cd /tmp/ruby - curl -O --progress https://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.8.tar.gz - echo 'c7e50159357afd87b13dc5eaf4ac486a70011149 ruby-2.1.8.tar.gz' | shasum -c - && tar xzf ruby-2.1.8.tar.gz - cd ruby-2.1.8 + curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz + echo 'c39b4001f7acb4e334cb60a0f4df72d434bef711 ruby-2.3.1.tar.gz' | shasum -c - && tar xzf ruby-2.3.1.tar.gz + cd ruby-2.3.1 ./configure --disable-install-rdoc make sudo make install @@ -143,7 +142,7 @@ gitlab-workhorse we need a Go compiler. The instructions below assume you use 64-bit Linux. You can find downloads for other platforms at the [Go download page](https://golang.org/dl). - curl -O --progress https://storage.googleapis.com/golang/go1.5.3.linux-amd64.tar.gz + curl --remote-name --progress https://storage.googleapis.com/golang/go1.5.3.linux-amd64.tar.gz echo '43afe0c5017e502630b1aea4d44b8a7f059bf60d7f29dfd58db454d4e4e0ae53 go1.5.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ sudo tar -C /usr/local -xzf go1.5.3.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ @@ -269,9 +268,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-10-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-13-stable gitlab -**Note:** You can change `8-10-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `8-13-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It @@ -332,6 +331,9 @@ sudo usermod -aG redis git # Disable 'git gc --auto' because GitLab already runs 'git gc' when needed sudo -u git -H git config --global gc.auto 0 + # Enable packfile bitmaps + sudo -u git -H git config --global repack.writeBitmaps true + # Configure Redis connection settings sudo -u git -H cp config/resque.yml.example config/resque.yml @@ -379,7 +381,7 @@ sudo usermod -aG redis git GitLab Shell is an SSH access and repository management software developed specially for GitLab. # Run the installation task for gitlab-shell (replace `REDIS_URL` if needed): - sudo -u git -H bundle exec rake gitlab:shell:install REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production + sudo -u git -H bundle exec rake gitlab:shell:install REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production SKIP_STORAGE_VALIDATION=true # By default, the gitlab-shell config is generated from your main GitLab config. # You can review (and modify) the gitlab-shell config as follows: @@ -398,7 +400,7 @@ If you are not using Linux you may have to run `gmake` instead of cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git cd gitlab-workhorse - sudo -u git -H git checkout v0.7.7 + sudo -u git -H git checkout v0.8.5 sudo -u git -H make ### Initialize Database and Activate Advanced Features @@ -561,7 +563,7 @@ Using a self-signed certificate is discouraged but if you must use it follow the ### Enable Reply by email -See the ["Reply by email" documentation](../incoming_email/README.md) for more information on how to set this up. +See the ["Reply by email" documentation](../administration/reply_by_email.md) for more information on how to set this up. ### LDAP Authentication @@ -588,15 +590,17 @@ for the changes to take effect. ### Custom Redis Connection -If you'd like Resque to connect to a Redis server on a non-standard port or on a different host, you can configure its connection string via the `config/resque.yml` file. +If you'd like to connect to a Redis server on a non-standard port or on a different host, you can configure its connection string via the `config/resque.yml` file. # example - production: redis://redis.example.tld:6379 + production: + url: redis://redis.example.tld:6379 If you want to connect the Redis server via socket, then use the "unix:" URL scheme and the path to the Redis socket file in the `config/resque.yml` file. # example - production: unix:/path/to/redis/socket + production: + url: unix:/path/to/redis/socket ### Custom SSH Connection diff --git a/doc/install/requirements.md b/doc/install/requirements.md index a65ac8a5f79..766a7119943 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -32,7 +32,7 @@ Please consider using a virtual machine to run GitLab. ## Ruby versions -GitLab requires Ruby (MRI) 2.1.x and currently does not work with versions 2.2 or 2.3. +GitLab requires Ruby (MRI) 2.3. Support for Ruby versions below 2.3 (2.1, 2.2) will stop with GitLab 8.13. You will have to use the standard MRI implementation of Ruby. We love [JRuby](http://jruby.org/) and [Rubinius](http://rubini.us/) but GitLab @@ -63,30 +63,30 @@ If you have enough RAM memory and a recent CPU the speed of GitLab is mainly lim ### Memory -You need at least 2GB of addressable memory (RAM + swap) to install and use GitLab! +You need at least 4GB of addressable memory (RAM + swap) to install and use GitLab! The operating system and any other running applications will also be using memory -so keep in mind that you need at least 2GB available before running GitLab. With +so keep in mind that you need at least 4GB available before running GitLab. With less memory GitLab will give strange errors during the reconfigure run and 500 errors during usage. -- 512MB RAM + 1.5GB of swap is the absolute minimum but we strongly **advise against** this amount of memory. See the unicorn worker section below for more advice. -- 1GB RAM + 1GB swap supports up to 100 users but it will be very slow -- **2GB RAM** is the **recommended** memory size for all installations and supports up to 100 users -- 4GB RAM supports up to 1,000 users -- 8GB RAM supports up to 2,000 users -- 16GB RAM supports up to 4,000 users -- 32GB RAM supports up to 8,000 users -- 64GB RAM supports up to 16,000 users -- 128GB RAM supports up to 32,000 users +- 1GB RAM + 3GB of swap is the absolute minimum but we strongly **advise against** this amount of memory. See the unicorn worker section below for more advice. +- 2GB RAM + 2GB swap supports up to 100 users but it will be very slow +- **4GB RAM** is the **recommended** memory size for all installations and supports up to 100 users +- 8GB RAM supports up to 1,000 users +- 16GB RAM supports up to 2,000 users +- 32GB RAM supports up to 4,000 users +- 64GB RAM supports up to 8,000 users +- 128GB RAM supports up to 16,000 users +- 256GB RAM supports up to 32,000 users - More users? Run it on [multiple application servers](https://about.gitlab.com/high-availability/) -We recommend having at least 1GB of swap on your server, even if you currently have +We recommend having at least 2GB of swap on your server, even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring if your available memory changes. Notice: The 25 workers of Sidekiq will show up as separate processes in your process overview (such as top or htop) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about many you need of those. -## Gitlab Runner +## GitLab Runner We strongly advise against installing GitLab Runner on the same machine you plan to install GitLab on. Depending on how you decide to configure GitLab Runner and @@ -113,10 +113,8 @@ It's possible to increase the amount of unicorn workers and this will usually he For most instances we recommend using: CPU cores + 1 = unicorn workers. So for a machine with 2 cores, 3 unicorn workers is ideal. -For all machines that have 1GB and up we recommend a minimum of three unicorn workers. -If you have a 512MB machine with a magnetic (non-SSD) swap drive we recommend to configure only one Unicorn worker to prevent excessive swapping. -With one Unicorn worker only git over ssh access will work because the git over HTTP access requires two running workers (one worker to receive the user request and one worker for the authorization check). -If you have a 512MB machine with a SSD drive you can use two Unicorn workers, this will allow HTTP access although it will be slow due to swapping. +For all machines that have 2GB and up we recommend a minimum of three unicorn workers. +If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive swapping. To change the Unicorn workers when you have the Omnibus package please see [the Unicorn settings in the Omnibus GitLab documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md#unicorn-settings). diff --git a/doc/integration/README.md b/doc/integration/README.md index fd330dd7a7d..c2fd299db07 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -11,11 +11,11 @@ See the documentation below for details on how to configure these services. - [OmniAuth](omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, Bitbucket, Facebook, Shibboleth, SAML, Crowd and Azure - [SAML](saml.md) Configure GitLab as a SAML 2.0 Service Provider - [CAS](cas.md) Configure GitLab to sign in using CAS -- [Slack](slack.md) Integrate with the Slack chat service - [OAuth2 provider](oauth_provider.md) OAuth2 application creation - [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages - [reCAPTCHA](recaptcha.md) Configure GitLab to use Google reCAPTCHA for new users - [Akismet](akismet.md) Configure Akismet to stop spam +- [Koding](../administration/integration/koding.md) Configure Koding to use IDE integration GitLab Enterprise Edition contains [advanced Jenkins support][jenkins]. diff --git a/doc/integration/akismet.md b/doc/integration/akismet.md index 5cc09bd536d..a6436b5f926 100644 --- a/doc/integration/akismet.md +++ b/doc/integration/akismet.md @@ -1,9 +1,14 @@ # Akismet +> *Note:* Before 8.11 only issues submitted via the API and for non-project +members were submitted to Akismet. + GitLab leverages [Akismet](http://akismet.com) to protect against spam. Currently -GitLab uses Akismet to prevent users who are not members of a project from -creating spam via the GitLab API. Detected spam will be rejected, and -an entry in the "Spam Log" section in the Admin page will be created. +GitLab uses Akismet to prevent the creation of spam issues on public projects. Issues +created via the WebUI or the API can be submitted to Akismet for review. + +Detected spam will be rejected, and an entry in the "Spam Log" section in the +Admin page will be created. Privacy note: GitLab submits the user's IP and user agent to Akismet. Note that adding a user to a project will disable the Akismet check and prevent this @@ -17,14 +22,37 @@ To use Akismet: 2. Sign-in or create a new account. -3. Click on "Show" to reveal the API key. +3. Click on **Show** to reveal the API key. 4. Go to Applications Settings on Admin Area (`admin/application_settings`) -5. Check the `Enable Akismet` checkbox +5. Check the **Enable Akismet** checkbox 6. Fill in the API key from step 3. 7. Save the configuration.  + + +## Training + +> *Note:* Training the Akismet filter is only available in 8.11 and above. + +As a way to better recognize between spam and ham, you can train the Akismet +filter whenever there is a false positive or false negative. + +When an entry is recognized as spam, it is rejected and added to the Spam Logs. +From here you can review if they are really spam. If one of them is not really +spam, you can use the **Submit as ham** button to tell Akismet that it falsely +recognized an entry as spam. + + + +If an entry that is actually spam was not recognized as such, you will be able +to also submit this to Akismet. The **Submit as spam** button will only appear +to admin users. + + + +Training Akismet will help it to recognize spam more accurately in the future. diff --git a/doc/integration/bitbucket.md b/doc/integration/bitbucket.md index 63432b04432..556d71b8b76 100644 --- a/doc/integration/bitbucket.md +++ b/doc/integration/bitbucket.md @@ -1,111 +1,164 @@ -# Integrate your server with Bitbucket +# Integrate your GitLab server with Bitbucket -Import projects from Bitbucket and login to your GitLab instance with your Bitbucket account. +Import projects from Bitbucket.org and login to your GitLab instance with your +Bitbucket.org account. -To enable the Bitbucket OmniAuth provider you must register your application with Bitbucket. -Bitbucket will generate an application ID and secret key for you to use. +## Overview -1. Sign in to Bitbucket. +You can set up Bitbucket.org as an OAuth provider so that you can use your +credentials to authenticate into GitLab or import your projects from +Bitbucket.org. -1. Navigate to your individual user settings or a team's settings, depending on how you want the application registered. It does not matter if the application is registered as an individual or a team - that is entirely up to you. +- To use Bitbucket.org as an OmniAuth provider, follow the [Bitbucket OmniAuth + provider](#bitbucket-omniauth-provider) section. +- To import projects from Bitbucket, follow both the + [Bitbucket OmniAuth provider](#bitbucket-omniauth-provider) and + [Bitbucket project import](#bitbucket-project-import) sections. -1. Select "OAuth" in the left menu. +## Bitbucket OmniAuth provider -1. Select "Add consumer". +> **Note:** +Make sure to first follow the [Initial OmniAuth configuration][init-oauth] +before proceeding with setting up the Bitbucket integration. -1. Provide the required details. - - Name: This can be anything. Consider something like "\<Organization\>'s GitLab" or "\<Your Name\>'s GitLab" or something else descriptive. - - Application description: Fill this in if you wish. - - URL: The URL to your GitLab installation. 'https://gitlab.company.com' -1. Select "Save". +To enable the Bitbucket OmniAuth provider you must register your application +with Bitbucket.org. Bitbucket will generate an application ID and secret key for +you to use. -1. You should now see a Key and Secret in the list of OAuth customers. - Keep this page open as you continue configuration. +1. Sign in to [Bitbucket.org](https://bitbucket.org). +1. Navigate to your individual user settings (**Bitbucket settings**) or a team's + settings (**Manage team**), depending on how you want the application registered. + It does not matter if the application is registered as an individual or a + team, that is entirely up to you. +1. Select **OAuth** in the left menu under "Access Management". +1. Select **Add consumer**. +1. Provide the required details: -1. On your GitLab server, open the configuration file. + | Item | Description | + | :--- | :---------- | + | **Name** | This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. | + | **Application description** | Fill this in if you wish. | + | **Callback URL** | Leave blank. | + | **URL** | The URL to your GitLab installation, e.g., `https://gitlab.example.com`. | - For omnibus package: + And grant at least the following permissions: - ```sh - sudo editor /etc/gitlab/gitlab.rb + ``` + Account: Email + Repositories: Read, Admin ``` - For installations from source: + >**Note:** + It may seem a little odd to giving GitLab admin permissions to repositories, + but this is needed in order for GitLab to be able to clone the repositories. - ```sh - cd /home/git/gitlab +  + +1. Select **Save**. +1. Select your newly created OAuth consumer and you should now see a Key and + Secret in the list of OAuth customers. Keep this page open as you continue + the configuration. + +  + +1. On your GitLab server, open the configuration file: - sudo -u git -H editor config/gitlab.yml ``` + # For Omnibus packages + sudo editor /etc/gitlab/gitlab.rb -1. See [Initial OmniAuth Configuration](omniauth.md#initial-omniauth-configuration) for initial settings. + # For installations from source + sudo -u git -H editor /home/git/gitlab/config/gitlab.yml + ``` -1. Add the provider configuration: +1. Follow the [Initial OmniAuth Configuration](omniauth.md#initial-omniauth-configuration) + for initial settings. +1. Add the Bitbucket provider configuration: - For omnibus package: + For Omnibus packages: ```ruby - gitlab_rails['omniauth_providers'] = [ - { - "name" => "bitbucket", - "app_id" => "YOUR_KEY", - "app_secret" => "YOUR_APP_SECRET", - "url" => "https://bitbucket.org/" - } - ] + gitlab_rails['omniauth_providers'] = [ + { + "name" => "bitbucket", + "app_id" => "BITBUCKET_APP_KEY", + "app_secret" => "BITBUCKET_APP_SECRET", + "url" => "https://bitbucket.org/" + } + ] ``` - For installation from source: + For installations from source: - ``` - - { name: 'bitbucket', app_id: 'YOUR_KEY', - app_secret: 'YOUR_APP_SECRET' } + ```yaml + - { name: 'bitbucket', + app_id: 'BITBUCKET_APP_KEY', + app_secret: 'BITBUCKET_APP_SECRET' } ``` -1. Change 'YOUR_APP_ID' to the key from the Bitbucket application page from step 7. + --- -1. Change 'YOUR_APP_SECRET' to the secret from the Bitbucket application page from step 7. + Where `BITBUCKET_APP_KEY` is the Key and `BITBUCKET_APP_SECRET` the Secret + from the Bitbucket application page. 1. Save the configuration file. +1. [Reconfigure][] or [restart GitLab][] for the changes to take effect if you + installed GitLab via Omnibus or from source respectively. -1. If you're using the omnibus package, reconfigure GitLab (```gitlab-ctl reconfigure```). - -1. Restart GitLab for the changes to take effect. - -On the sign in page there should now be a Bitbucket icon below the regular sign in form. -Click the icon to begin the authentication process. Bitbucket will ask the user to sign in and authorize the GitLab application. -If everything goes well the user will be returned to GitLab and will be signed in. +On the sign in page there should now be a Bitbucket icon below the regular sign +in form. Click the icon to begin the authentication process. Bitbucket will ask +the user to sign in and authorize the GitLab application. If everything goes +well, the user will be returned to GitLab and will be signed in. ## Bitbucket project import -To allow projects to be imported directly into GitLab, Bitbucket requires two extra setup steps compared to GitHub and GitLab.com. +To allow projects to be imported directly into GitLab, Bitbucket requires two +extra setup steps compared to [GitHub](github.md) and [GitLab.com](gitlab.md). -Bitbucket doesn't allow OAuth applications to clone repositories over HTTPS, and instead requires GitLab to use SSH and identify itself using your GitLab server's SSH key. +Bitbucket doesn't allow OAuth applications to clone repositories over HTTPS, and +instead requires GitLab to use SSH and identify itself using your GitLab +server's SSH key. -### Step 1: Public key +To be able to access repositories on Bitbucket, GitLab will automatically +register your public key with Bitbucket as a deploy key for the repositories to +be imported. Your public key needs to be at `~/.ssh/bitbucket_rsa` which +translates to `/var/opt/gitlab/.ssh/bitbucket_rsa` for Omnibus packages and to +`/home/git/.ssh/bitbucket_rsa.pub` for installations from source. -To be able to access repositories on Bitbucket, GitLab will automatically register your public key with Bitbucket as a deploy key for the repositories to be imported. Your public key needs to be at `~/.ssh/bitbucket_rsa.pub`, which will expand to `/home/git/.ssh/bitbucket_rsa.pub` in most configurations. +--- -If you have that file in place, you're all set and should see the "Import projects from Bitbucket" option enabled. If you don't, do the following: +Below are the steps that will allow GitLab to be able to import your projects +from Bitbucket. -1. Create a new SSH key: +1. Make sure you [have enabled the Bitbucket OAuth support](#bitbucket-omniauth-provider). +1. Create a new SSH key with an **empty passphrase**: ```sh sudo -u git -H ssh-keygen ``` - When asked `Enter file in which to save the key` specify the correct path, eg. `/home/git/.ssh/bitbucket_rsa`. - Make sure to use an **empty passphrase**. + When asked to 'Enter file in which to save the key' enter: + `/var/opt/gitlab/.ssh/bitbucket_rsa` for Omnibus packages or + `/home/git/.ssh/bitbucket_rsa` for installations from source. The name is + important so make sure to get it right. -1. Configure SSH client to use your new key: + > **Warning:** + This key must NOT be associated with ANY existing Bitbucket accounts. If it + is, the import will fail with an `Access denied! Please verify you can add + deploy keys to this repository.` error. - Open the SSH configuration file of the git user. +1. Next, you need to to configure the SSH client to use your new key. Open the + SSH configuration file of the `git` user: - ```sh - sudo editor /home/git/.ssh/config + ``` + # For Omnibus packages + sudo editor /var/opt/gitlab/.ssh/config + + # For installations from source + sudo editor /home/git/.ssh/config ``` - Add a host configuration for `bitbucket.org`. +1. Add a host configuration for `bitbucket.org`: ```sh Host bitbucket.org @@ -113,28 +166,46 @@ If you have that file in place, you're all set and should see the "Import projec User git ``` -### Step 2: Known hosts - -To allow GitLab to connect to Bitbucket over SSH, you need to add 'bitbucket.org' to your GitLab server's known SSH hosts. Take the following steps to do so: - -1. Manually connect to 'bitbucket.org' over SSH, while logged in as the `git` account that GitLab will use: +1. Save the file and exit. +1. Manually connect to `bitbucket.org` over SSH, while logged in as the `git` + user that GitLab will use: ```sh sudo -u git -H ssh bitbucket.org ``` -1. Verify the RSA key fingerprint you'll see in the response matches the one in the [Bitbucket documentation](https://confluence.atlassian.com/display/BITBUCKET/Use+the+SSH+protocol+with+Bitbucket#UsetheSSHprotocolwithBitbucket-KnownhostorBitbucket'spublickeyfingerprints) (the specific IP address doesn't matter): + That step is performed because GitLab needs to connect to Bitbucket over SSH, + in order to add `bitbucket.org` to your GitLab server's known SSH hosts. + +1. Verify the RSA key fingerprint you'll see in the response matches the one + in the [Bitbucket documentation][bitbucket-docs] (the specific IP address + doesn't matter): ```sh - The authenticity of host 'bitbucket.org (207.223.240.182)' can't be established. - RSA key fingerprint is 97:8c:1b:f2:6f:14:6b:5c:3b:ec:aa:46:46:74:7c:40. + The authenticity of host 'bitbucket.org (104.192.143.1)' can't be established. + RSA key fingerprint is SHA256:zzXQOXSRBEiUtuE8AikJYKwbHaxvSc0ojez9YXaGp1A. Are you sure you want to continue connecting (yes/no)? ``` -1. If the fingerprint matches, type `yes` to continue connecting and have 'bitbucket.org' be added to your known hosts. +1. If the fingerprint matches, type `yes` to continue connecting and have + `bitbucket.org` be added to your known SSH hosts. After confirming you should + see a permission denied message. If you see an authentication successful + message you have done something wrong. The key you are using has already been + added to a Bitbucket account and will cause the import script to fail. Ensure + the key you are using CANNOT authenticate with Bitbucket. +1. Restart GitLab to allow it to find the new public key. -1. Your GitLab server is now able to connect to Bitbucket over SSH. +Your GitLab server is now able to connect to Bitbucket over SSH. You should be +able to see the "Import projects from Bitbucket" option on the New Project page +enabled. -1. Restart GitLab to allow it to find the new public key. +## Acknowledgemts + +Special thanks to the writer behind the following article: + +- http://stratus3d.com/blog/2015/09/06/migrating-from-bitbucket-to-local-gitlab-server/ -You should now see the "Import projects from Bitbucket" option on the New Project page enabled. +[init-oauth]: omniauth.md#initial-omniauth-configuration +[bitbucket-docs]: https://confluence.atlassian.com/bitbucket/use-the-ssh-protocol-with-bitbucket-cloud-221449711.html#UsetheSSHprotocolwithBitbucketCloud-KnownhostorBitbucket%27spublickeyfingerprints +[reconfigure]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure +[restart GitLab]: ../administration/restart_gitlab.md#installations-from-source diff --git a/doc/integration/github.md b/doc/integration/github.md index 340c8a55fb3..8a01afd1177 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -16,7 +16,7 @@ GitHub will generate an application ID and secret key for you to use. 1. Select "Register new application". 1. Provide the required details. - - Application name: This can be anything. Consider something like "\<Organization\>'s GitLab" or "\<Your Name\>'s GitLab" or something else descriptive. + - Application name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. - Homepage URL: The URL to your GitLab installation. 'https://gitlab.company.com' - Application description: Fill this in if you wish. - Authorization callback URL is 'http(s)://${YOUR_DOMAIN}' diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index b215cc7c609..6d8f3912ede 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -14,7 +14,7 @@ GitLab.com will generate an application ID and secret key for you to use. 1. Select "New application". 1. Provide the required details. - - Name: This can be anything. Consider something like "\<Organization\>'s GitLab" or "\<Your Name\>'s GitLab" or something else descriptive. + - Name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. - Redirect URI: ``` diff --git a/doc/integration/img/bitbucket_oauth_keys.png b/doc/integration/img/bitbucket_oauth_keys.png Binary files differnew file mode 100644 index 00000000000..3fb2f7524a3 --- /dev/null +++ b/doc/integration/img/bitbucket_oauth_keys.png diff --git a/doc/integration/img/bitbucket_oauth_settings_page.png b/doc/integration/img/bitbucket_oauth_settings_page.png Binary files differnew file mode 100644 index 00000000000..a3047712d8c --- /dev/null +++ b/doc/integration/img/bitbucket_oauth_settings_page.png diff --git a/doc/integration/img/spam_log.png b/doc/integration/img/spam_log.png Binary files differnew file mode 100644 index 00000000000..8d574448690 --- /dev/null +++ b/doc/integration/img/spam_log.png diff --git a/doc/integration/img/submit_issue.png b/doc/integration/img/submit_issue.png Binary files differnew file mode 100644 index 00000000000..5c7896a7eec --- /dev/null +++ b/doc/integration/img/submit_issue.png diff --git a/doc/integration/oauth_provider.md b/doc/integration/oauth_provider.md index 5f8bb57365c..0c53584d201 100644 --- a/doc/integration/oauth_provider.md +++ b/doc/integration/oauth_provider.md @@ -28,7 +28,8 @@ GitLab supports two ways of adding a new OAuth2 application to an instance. You can either add an application as a regular user or add it in the admin area. What this means is that GitLab can actually have instance-wide and a user-wide applications. There is no difference between them except for the different -permission levels they are set (user/admin). +permission levels they are set (user/admin). The default callback URL is +`http://your-gitlab.example.com/users/auth/gitlab/callback` ## Adding an application through the profile diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index 46b260e7033..8a55fce96fe 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -102,8 +102,8 @@ To change these settings: block_auto_created_users: true ``` -Now we can choose one or more of the Supported Providers listed above to continue -the configuration process. +Now we can choose one or more of the [Supported Providers](#supported-providers) +listed above to continue the configuration process. ## Enable OmniAuth for an Existing User diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 8a7205caaa4..f3b2a288776 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -138,7 +138,7 @@ This setting is only available on GitLab 8.7 and above. SAML login includes support for external groups. You can define in the SAML settings which groups, to which your users belong in your IdP, you wish to be -marked as [external](../permissions/permissions.md). +marked as [external](../user/permissions.md). ### Requirements @@ -306,4 +306,4 @@ For this you need take the following into account: validators are optional Make sure that one of the above described scenarios is valid, or the requests will -fail with one of the mentioned errors.
\ No newline at end of file +fail with one of the mentioned errors. diff --git a/doc/integration/slack.md b/doc/integration/slack.md index f6ba80f46d5..8cd151fbf95 100644 --- a/doc/integration/slack.md +++ b/doc/integration/slack.md @@ -1,41 +1 @@ -# Slack integration - -## On Slack - -To enable Slack integration you must create an Incoming WebHooks integration on Slack: - -1. [Sign in to Slack](https://slack.com/signin) - -1. Visit [Incoming WebHooks](https://my.slack.com/services/new/incoming-webhook/) - -1. Choose the channel name you want to send notifications to. - -1. Click **Add Incoming WebHooks Integration** - - Optional step; You can change bot's name and avatar by clicking modifying the bot name or avatar under **Integration Settings**. - -1. Copy the **Webhook URL**, we'll need this later for GitLab. - - -## On GitLab - -After Slack is ready we need to setup GitLab. Here are the steps to achieve this. - -1. Sign in to GitLab - -1. Pick the repository you want. - -1. Navigate to Settings -> Services -> Slack - -1. Pick the triggers you want to activate - -1. Fill in your Slack details - - Webhook: Paste the Webhook URL from the step above - - Username: Fill this in if you want to change the username of the bot - - Channel: Fill this in if you want to change the channel where the messages will be posted - - Mark it as active - -1. Save your settings - -Have fun :) - -*P.S. You can set "branch,pushed,Compare changes" as highlight words on your Slack profile settings, so that you can be aware of new commits when somebody pushes them.* +This document was moved to [project_services/slack.md](../project_services/slack.md). diff --git a/doc/integration/twitter.md b/doc/integration/twitter.md index 4769f26b259..abbea09f22f 100644 --- a/doc/integration/twitter.md +++ b/doc/integration/twitter.md @@ -7,7 +7,7 @@ To enable the Twitter OmniAuth provider you must register your application with 1. Select "Create new app" 1. Fill in the application details. - - Name: This can be anything. Consider something like "\<Organization\>'s GitLab" or "\<Your Name\>'s GitLab" or + - Name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. - Description: Create a description. - Website: The URL to your GitLab installation. 'https://gitlab.example.com' diff --git a/doc/intro/README.md b/doc/intro/README.md index 1850031eb26..1790b2b761f 100644 --- a/doc/intro/README.md +++ b/doc/intro/README.md @@ -22,10 +22,10 @@ Create merge requests and review code. - [Fork a project and contribute to it](../workflow/forking_workflow.md) - [Create a new merge request](../gitlab-basics/add-merge-request.md) -- [Automatically close issues from merge requests](../customization/issue_closing.md) -- [Automatically merge when your builds succeed](../workflow/merge_when_build_succeeds.md) -- [Revert any commit](../workflow/revert_changes.md) -- [Cherry-pick any commit](../workflow/cherry_pick_changes.md) +- [Automatically close issues from merge requests](../user/project/issues/automatic_issue_closing.md) +- [Automatically merge when your builds succeed](../user/project/merge_requests/merge_when_build_succeeds.md) +- [Revert any commit](../user/project/merge_requests/revert_changes.md) +- [Cherry-pick any commit](../user/project/merge_requests/cherry_pick_changes.md) ## Test and Deploy diff --git a/doc/legal/corporate_contributor_license_agreement.md b/doc/legal/corporate_contributor_license_agreement.md index 7b94506c297..7f08188bd65 100644 --- a/doc/legal/corporate_contributor_license_agreement.md +++ b/doc/legal/corporate_contributor_license_agreement.md @@ -6,13 +6,17 @@ You accept and agree to the following terms and conditions for Your present and "You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with GitLab B.V.. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. - "Contribution" shall mean the code, documentation or other original works of authorship expressly identified in Schedule B, as well as any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution." + "Contribution" shall mean the code, documentation or other original works of authorship, including any modifications or additions to an existing work, that is submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution." -2. Grant of Copyright License. Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works. +2. Grant of Copyright License. -3. Grant of Patent License. Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed. +Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works. -4. You represent that You are legally entitled to grant the above license. You represent further that each employee of the Corporation designated on Schedule A below (or in a subsequent written modification to that Schedule) is authorized to submit Contributions on behalf of the Corporation. +3. Grant of Patent License. + +Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed. + +4. You represent that You are legally entitled to grant the above license. You represent further that each of Your employees is authorized to submit Contributions on Your behalf, but excluding employees that are designated in writing by You as "Not authorized to submit Contributions on behalf of [name of Your corporation here]." Such designations of exclusion for unauthorized employees are to be submitted via email to legal@gitlab.com. 5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). @@ -20,6 +24,6 @@ You accept and agree to the following terms and conditions for Your present and 7. Should You wish to submit work that is not Your original creation, You may submit it to GitLab B.V. separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]". -8. It is your responsibility to notify GitLab B.V. when any change is required to the list of designated employees authorized to submit Contributions on behalf of the Corporation, or to the Corporation's Point of Contact with GitLab B.V.. +8. It is Your responsibility to notify GitLab.com when any change is required to the list of designated employees excluded from submitting Contributions on Your behalf per Section 4. Such notification should be sent via email to legal@gitlab.com. This text is licensed under the [Creative Commons Attribution 3.0 License](https://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. diff --git a/doc/markdown/markdown.md b/doc/markdown/markdown.md index fb2dd582754..4ac81ab3ee7 100644 --- a/doc/markdown/markdown.md +++ b/doc/markdown/markdown.md @@ -1,647 +1 @@ -# Markdown - -## Table of Contents - -**[GitLab Flavored Markdown](#gitlab-flavored-markdown-gfm)** - -* [Newlines](#newlines) -* [Multiple underscores in words](#multiple-underscores-in-words) -* [URL auto-linking](#url-auto-linking) -* [Multiline Blockquote](#multiline-blockquote) -* [Code and Syntax Highlighting](#code-and-syntax-highlighting) -* [Inline Diff](#inline-diff) -* [Emoji](#emoji) -* [Special GitLab references](#special-gitlab-references) -* [Task Lists](#task-lists) - -**[Standard Markdown](#standard-markdown)** - -* [Headers](#headers) -* [Emphasis](#emphasis) -* [Lists](#lists) -* [Links](#links) -* [Images](#images) -* [Blockquotes](#blockquotes) -* [Inline HTML](#inline-html) -* [Horizontal Rule](#horizontal-rule) -* [Line Breaks](#line-breaks) -* [Tables](#tables) - -**[References](#references)** - -## GitLab Flavored Markdown (GFM) - -_GitLab uses the [Redcarpet Ruby library][redcarpet] for Markdown processing._ - -GitLab uses "GitLab Flavored Markdown" (GFM). It extends the standard Markdown in a few significant ways to add some useful functionality. It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). - -You can use GFM in - -- comments -- issues -- merge requests -- milestones -- wiki pages - -You can also use other rich text files in GitLab. You might have to install a dependency to do so. Please see the [github-markup gem readme](https://github.com/gitlabhq/markup#markups) for more information. - -## Newlines - -GFM honors the markdown specification in how [paragraphs and line breaks are handled](https://daringfireball.net/projects/markdown/syntax#p). - -A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines. -Line-breaks, or softreturns, are rendered if you end a line with two or more spaces - - Roses are red [followed by two or more spaces] - Violets are blue - - Sugar is sweet - -Roses are red -Violets are blue - -Sugar is sweet - -## Multiple underscores in words - -It is not reasonable to italicize just _part_ of a word, especially when you're dealing with code and names that often appear with multiple underscores. Therefore, GFM ignores multiple underscores in words. - - perform_complicated_task - do_this_and_do_that_and_another_thing - -perform_complicated_task -do_this_and_do_that_and_another_thing - -## URL auto-linking - -GFM will autolink almost any URL you copy and paste into your text. - - * https://www.google.com - * https://google.com/ - * ftp://ftp.us.debian.org/debian/ - * smb://foo/bar/baz - * irc://irc.freenode.net/gitlab - * http://localhost:3000 - -* https://www.google.com -* https://google.com/ -* ftp://ftp.us.debian.org/debian/ -* smb://foo/bar/baz -* irc://irc.freenode.net/gitlab -* http://localhost:3000 - -## Multiline Blockquote - -On top of standard Markdown [blockquotes](#blockquotes), which require prepending `>` to quoted lines, -GFM supports multiline blockquotes fenced by <code>>>></code>. - -```no-highlight ->>> -If you paste a message from somewhere else - -that - -spans - -multiple lines, - -you can quote that without having to manually prepend `>` to every line! ->>> -``` - ->>> -If you paste a message from somewhere else - -that - -spans - -multiple lines, - -you can quote that without having to manually prepend `>` to every line! ->>> - -## Code and Syntax Highlighting - -_GitLab uses the [Rouge Ruby library][rouge] for syntax highlighting. For a -list of supported languages visit the Rouge website._ - -Blocks of code are either fenced by lines with three back-ticks <code>```</code>, or are indented with four spaces. Only the fenced code blocks support syntax highlighting. - -```no-highlight -Inline `code` has `back-ticks around` it. -``` - -Inline `code` has `back-ticks around` it. - -Example: - - ```javascript - var s = "JavaScript syntax highlighting"; - alert(s); - ``` - - ```python - def function(): - #indenting works just fine in the fenced code block - s = "Python syntax highlighting" - print s - ``` - - ```ruby - require 'redcarpet' - markdown = Redcarpet.new("Hello World!") - puts markdown.to_html - ``` - - ``` - No language indicated, so no syntax highlighting. - s = "There is no highlighting for this." - But let's throw in a <b>tag</b>. - ``` - -becomes: - -```javascript -var s = "JavaScript syntax highlighting"; -alert(s); -``` - -```python -def function(): - #indenting works just fine in the fenced code block - s = "Python syntax highlighting" - print s -``` - -```ruby -require 'redcarpet' -markdown = Redcarpet.new("Hello World!") -puts markdown.to_html -``` - -``` -No language indicated, so no syntax highlighting. -s = "There is no highlighting for this." -But let's throw in a <b>tag</b>. -``` - -## Inline Diff - -With inline diffs tags you can display {+ additions +} or [- deletions -]. - -The wrapping tags can be either curly braces or square brackets [+ additions +] or {- deletions -}. - -However the wrapping tags cannot be mixed as such: - -- {+ additions +] -- [+ additions +} -- {- deletions -] -- [- deletions -} - -## Emoji - - Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: - - :zap: You can use emoji anywhere GFM is supported. :v: - - You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. - - If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up on the supported codes. - - Consult the [Emoji Cheat Sheet](http://emoji.codes) for a list of all supported emoji codes. :thumbsup: - -Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: - -:zap: You can use emoji anywhere GFM is supported. :v: - -You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. - -If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up on the supported codes. - -Consult the [Emoji Cheat Sheet](http://emoji.codes) for a list of all supported emoji codes. :thumbsup: - -## Special GitLab References - -GFM recognizes special references. - -You can easily reference e.g. an issue, a commit, a team member or even the whole team within a project. - -GFM will turn that reference into a link so you can navigate between them easily. - -GFM will recognize the following: - -| input | references | -|:-----------------------|:--------------------------- | -| `@user_name` | specific user | -| `@group_name` | specific group | -| `@all` | entire team | -| `#123` | issue | -| `!123` | merge request | -| `$123` | snippet | -| `~123` | label by ID | -| `~bug` | one-word label by name | -| `~"feature request"` | multi-word label by name | -| `%123` | milestone by ID | -| `%v1.23` | one-word milestone by name | -| `%"release candidate"` | multi-word milestone by name | -| `9ba12248` | specific commit | -| `9ba12248...b19a04f5` | commit range comparison | -| `[README](doc/README)` | repository file references | - -GFM also recognizes certain cross-project references: - -| input | references | -|:----------------------------------------|:------------------------| -| `namespace/project#123` | issue | -| `namespace/project!123` | merge request | -| `namespace/project%123` | milestone | -| `namespace/project$123` | snippet | -| `namespace/project@9ba12248` | specific commit | -| `namespace/project@9ba12248...b19a04f5` | commit range comparison | -| `namespace/project~"Some label"` | issues with given label | - -## Task Lists - -You can add task lists to issues, merge requests and comments. To create a task list, add a specially-formatted Markdown list, like so: - -```no-highlight -- [x] Completed task -- [ ] Incomplete task - - [ ] Sub-task 1 - - [x] Sub-task 2 - - [ ] Sub-task 3 -``` - -- [x] Completed task -- [ ] Incomplete task - - [ ] Sub-task 1 - - [x] Sub-task 2 - - [ ] Sub-task 3 - -Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes. - -# Standard Markdown - -## Headers - -```no-highlight -# H1 -## H2 -### H3 -#### H4 -##### H5 -###### H6 - -Alternatively, for H1 and H2, an underline-ish style: - -Alt-H1 -====== - -Alt-H2 ------- -``` - -# H1 -## H2 -### H3 -#### H4 -##### H5 -###### H6 - -Alternatively, for H1 and H2, an underline-ish style: - -Alt-H1 -====== - -Alt-H2 ------- - -### Header IDs and links - -All Markdown-rendered headers automatically get IDs, except in comments. - -On hover a link to those IDs becomes visible to make it easier to copy the link to the header to give it to someone else. - -The IDs are generated from the content of the header according to the following rules: - -1. All text is converted to lowercase -1. All non-word text (e.g., punctuation, HTML) is removed -1. All spaces are converted to hyphens -1. Two or more hyphens in a row are converted to one -1. If a header with the same ID has already been generated, a unique - incrementing number is appended, starting at 1. - -For example: - -``` -# This header has spaces in it -## This header has a :thumbsup: in it -# This header has Unicode in it: 한글 -## This header has spaces in it -### This header has spaces in it -``` - -Would generate the following link IDs: - -1. `this-header-has-spaces-in-it` -1. `this-header-has-a-in-it` -1. `this-header-has-unicode-in-it-한글` -1. `this-header-has-spaces-in-it` -1. `this-header-has-spaces-in-it-1` - -Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. - -## Emphasis - -```no-highlight -Emphasis, aka italics, with *asterisks* or _underscores_. - -Strong emphasis, aka bold, with **asterisks** or __underscores__. - -Combined emphasis with **asterisks and _underscores_**. - -Strikethrough uses two tildes. ~~Scratch this.~~ -``` - -Emphasis, aka italics, with *asterisks* or _underscores_. - -Strong emphasis, aka bold, with **asterisks** or __underscores__. - -Combined emphasis with **asterisks and _underscores_**. - -Strikethrough uses two tildes. ~~Scratch this.~~ - -## Lists - -```no-highlight -1. First ordered list item -2. Another item - * Unordered sub-list. -1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list -4. And another item. - -* Unordered list can use asterisks -- Or minuses -+ Or pluses -``` - -1. First ordered list item -2. Another item - * Unordered sub-list. -1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list -4. And another item. - -* Unordered list can use asterisks -- Or minuses -+ Or pluses - -If a list item contains multiple paragraphs, -each subsequent paragraph should be indented with four spaces. - -```no-highlight -1. First ordered list item - - Second paragraph of first item. -2. Another item -``` - -1. First ordered list item - - Second paragraph of first item. -2. Another item - -If the second paragraph isn't indented with four spaces, -the second list item will be incorrectly labeled as `1`. - -```no-highlight -1. First ordered list item - - Second paragraph of first item. -2. Another item -``` - -1. First ordered list item - - Second paragraph of first item. -2. Another item - -## Links - -There are two ways to create links, inline-style and reference-style. - - [I'm an inline-style link](https://www.google.com) - - [I'm a reference-style link][Arbitrary case-insensitive reference text] - - [I'm a relative reference to a repository file](LICENSE) - - [You can use numbers for reference-style link definitions][1] - - Or leave it empty and use the [link text itself][] - - Some text to show that the reference links can follow later. - - [arbitrary case-insensitive reference text]: https://www.mozilla.org - [1]: http://slashdot.org - [link text itself]: https://www.reddit.com - -[I'm an inline-style link](https://www.google.com) - -[I'm a reference-style link][Arbitrary case-insensitive reference text] - -[I'm a relative reference to a repository file](LICENSE)[^1] - -[You can use numbers for reference-style link definitions][1] - -Or leave it empty and use the [link text itself][] - -Some text to show that the reference links can follow later. - -[arbitrary case-insensitive reference text]: https://www.mozilla.org -[1]: http://slashdot.org -[link text itself]: https://www.reddit.com - -**Note** - -Relative links do not allow referencing project files in a wiki page or wiki page in a project file. The reason for this is that, in GitLab, wiki is always a separate git repository. For example: - -`[I'm a reference-style link](style)` - -will point the link to `wikis/style` when the link is inside of a wiki markdown file. - -## Images - - Here's our logo (hover to see the title text): - - Inline-style: -  - - Reference-style: - ![alt text1][logo] - - [logo]: img/logo.png - -Here's our logo: - -Inline-style: - - - -Reference-style: - -![alt text][logo] - -[logo]: img/logo.png - -## Blockquotes - -```no-highlight -> Blockquotes are very handy in email to emulate reply text. -> This line is part of the same quote. - -Quote break. - -> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. -``` - -> Blockquotes are very handy in email to emulate reply text. -> This line is part of the same quote. - -Quote break. - -> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. - -## Inline HTML - -You can also use raw HTML in your Markdown, and it'll mostly work pretty well. - -See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span` elements. - -```no-highlight -<dl> - <dt>Definition list</dt> - <dd>Is something people use sometimes.</dd> - - <dt>Markdown in HTML</dt> - <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> -</dl> -``` - -<dl> - <dt>Definition list</dt> - <dd>Is something people use sometimes.</dd> - - <dt>Markdown in HTML</dt> - <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> -</dl> - -## Horizontal Rule - -``` -Three or more... - ---- - -Hyphens - -*** - -Asterisks - -___ - -Underscores -``` - -Three or more... - ---- - -Hyphens - -*** - -Asterisks - -___ - -Underscores - -## Line Breaks - -My basic recommendation for learning how line breaks work is to experiment and discover -- hit <Enter> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. "Markdown Toggle" is your friend. - -Here are some things to try out: - -``` -Here's a line for us to start with. - -This line is separated from the one above by two newlines, so it will be a *separate paragraph*. - -This line is also a separate paragraph, but... -This line is only separated by a single newline, so it's a separate line in the *same paragraph*. - -This line is also a separate paragraph, and... -This line is on its own line, because the previous line ends with two -spaces. -``` - -Here's a line for us to start with. - -This line is separated from the one above by two newlines, so it will be a *separate paragraph*. - -This line is also begins a separate paragraph, but... -This line is only separated by a single newline, so it's a separate line in the *same paragraph*. - -This line is also a separate paragraph, and... -This line is on its own line, because the previous line ends with two -spaces. - -## Tables - -Tables aren't part of the core Markdown spec, but they are part of GFM and Markdown Here supports them. - -``` -| header 1 | header 2 | -| -------- | -------- | -| cell 1 | cell 2 | -| cell 3 | cell 4 | -``` - -Code above produces next output: - -| header 1 | header 2 | -| -------- | -------- | -| cell 1 | cell 2 | -| cell 3 | cell 4 | - -**Note** - -The row of dashes between the table header and body must have at least three dashes in each column. - -By including colons in the header row, you can align the text within that column: - -``` -| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | -| :----------- | :------: | ------------: | :----------- | :------: | ------------: | -| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | -| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | -``` - -| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | -| :----------- | :------: | ------------: | :----------- | :------: | ------------: | -| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | -| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | - -## References - -- This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). -- The [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. -- [Dillinger.io](http://dillinger.io) is a handy tool for testing standard markdown. - -[rouge]: http://rouge.jneen.net/ "Rouge website" -[redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" -[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com +This document was moved to [user/markdown.md](../user/markdown.md). diff --git a/doc/monitoring/health_check.md b/doc/monitoring/health_check.md index 0d17799372f..6cf93c33ec2 100644 --- a/doc/monitoring/health_check.md +++ b/doc/monitoring/health_check.md @@ -1,66 +1 @@ -# Health Check - ->**Note:** This feature was [introduced][ce-3888] in GitLab 8.8. - -GitLab provides a health check endpoint for uptime monitoring on the `health_check` web -endpoint. The health check reports on the overall system status based on the status of -the database connection, the state of the database migrations, and the ability to write -and access the cache. This endpoint can be provided to uptime monitoring services like -[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health]. - -## Access Token - -An access token needs to be provided while accessing the health check endpoint. The current -accepted token can be found on the `admin/health_check` page of your GitLab instance. - - - -The access token can be passed as a URL parameter: - -``` -https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN -``` - -or as an HTTP header: - -```bash -curl -H "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json -``` - -## Using the Endpoint - -Once you have the access token, health information can be retrieved as plain text, JSON, -or XML using the `health_check` endpoint: - -- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN` - -You can also ask for the status of specific services: - -- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN` -- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN` - -For example, the JSON output of the following health check: - -```bash -curl -H "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json -``` - -would be like: - -``` -{"healthy":true,"message":"success"} -``` - -## Status - -On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint -will return a valid successful HTTP status code, and a `success` message. Ideally your -uptime monitoring should look for the success message. - -[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888 -[pingdom]: https://www.pingdom.com -[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html -[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring +This document was moved to [user/admin_area/monitoring/health_check](../user/admin_area/monitoring/health_check.md). diff --git a/doc/monitoring/performance/gitlab_configuration.md b/doc/monitoring/performance/gitlab_configuration.md index 771584268d9..a669bb28904 100644 --- a/doc/monitoring/performance/gitlab_configuration.md +++ b/doc/monitoring/performance/gitlab_configuration.md @@ -1,40 +1 @@ -# GitLab Configuration - -GitLab Performance Monitoring is disabled by default. To enable it and change any of its -settings, navigate to the Admin area in **Settings > Metrics** -(`/admin/application_settings`). - -The minimum required settings you need to set are the InfluxDB host and port. -Make sure _Enable InfluxDB Metrics_ is checked and hit **Save** to save the -changes. - ---- - - - ---- - -Finally, a restart of all GitLab processes is required for the changes to take -effect: - -```bash -# For Omnibus installations -sudo gitlab-ctl restart - -# For installations from source -sudo service gitlab restart -``` - -## Pending Migrations - -When any migrations are pending, the metrics are disabled until the migrations -have been performed. - ---- - -Read more on: - -- [Introduction to GitLab Performance Monitoring](introduction.md) -- [InfluxDB Configuration](influxdb_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) -- [Grafana Install/Configuration](grafana_configuration.md) +This document was moved to [administration/monitoring/performance/gitlab_configuration](../administration/monitoring/performance/gitlab_configuration.md). diff --git a/doc/monitoring/performance/grafana_configuration.md b/doc/monitoring/performance/grafana_configuration.md index 7947b0fedc4..0d4be02ff5f 100644 --- a/doc/monitoring/performance/grafana_configuration.md +++ b/doc/monitoring/performance/grafana_configuration.md @@ -1,111 +1 @@ -# Grafana Configuration - -[Grafana](http://grafana.org/) is a tool that allows you to visualize time -series metrics through graphs and dashboards. It supports several backend -data stores, including InfluxDB. GitLab writes performance data to InfluxDB -and Grafana will allow you to query InfluxDB to display useful graphs. - -For the easiest installation and configuration, install Grafana on the same -server as InfluxDB. For larger installations, you may want to split out these -services. - -## Installation - -Grafana supplies package repositories (Yum/Apt) for easy installation. -See [Grafana installation documentation](http://docs.grafana.org/installation/) -for detailed steps. - -> **Note**: Before starting Grafana for the first time, set the admin user -and password in `/etc/grafana/grafana.ini`. Otherwise, the default password -will be `admin`. - -## Configuration - -Login as the admin user. Expand the menu by clicking the Grafana logo in the -top left corner. Choose 'Data Sources' from the menu. Then, click 'Add new' -in the top bar. - - - -Fill in the configuration details for the InfluxDB data source. Save and -Test Connection to ensure the configuration is correct. - -- **Name**: InfluxDB -- **Default**: Checked -- **Type**: InfluxDB 0.9.x (Even if you're using InfluxDB 0.10.x) -- **Url**: https://localhost:8086 (Or the remote URL if you've installed InfluxDB -on a separate server) -- **Access**: proxy -- **Database**: gitlab -- **User**: admin (Or the username configured when setting up InfluxDB) -- **Password**: The password configured when you set up InfluxDB - - - -## Apply retention policies and create continuous queries - -If you intend to import the GitLab provided Grafana dashboards, you will need to -set up the right retention policies and continuous queries. The easiest way of -doing this is by using the [influxdb-management](https://gitlab.com/gitlab-org/influxdb-management) -repository. - -To use this repository you must first clone it: - -``` -git clone https://gitlab.com/gitlab-org/influxdb-management.git -cd influxdb-management -``` - -Next you must install the required dependencies: - -``` -gem install bundler -bundle install -``` - -Now you must configure the repository by first copying `.env.example` to `.env` -and then editing the `.env` file to contain the correct InfluxDB settings. Once -configured you can simply run `bundle exec rake` and the InfluxDB database will -be configured for you. - -For more information see the [influxdb-management README](https://gitlab.com/gitlab-org/influxdb-management/blob/master/README.md). - -## Import Dashboards - -You can now import a set of default dashboards that will give you a good -start on displaying useful information. GitLab has published a set of default -[Grafana dashboards][grafana-dashboards] to get you started. Clone the -repository or download a zip/tarball, then follow these steps to import each -JSON file. - -Open the dashboard dropdown menu and click 'Import' - - - -Click 'Choose file' and browse to the location where you downloaded or cloned -the dashboard repository. Pick one of the JSON files to import. - - - -Once the dashboard is imported, be sure to click save icon in the top bar. If -you do not save the dashboard after importing it will be removed when you -navigate away. - - - -Repeat this process for each dashboard you wish to import. - -Alternatively you can automatically import all the dashboards into your Grafana -instance. See the README of the [Grafana dashboards][grafana-dashboards] -repository for more information on this process. - -[grafana-dashboards]: https://gitlab.com/gitlab-org/grafana-dashboards - ---- - -Read more on: - -- [Introduction to GitLab Performance Monitoring](introduction.md) -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Installation/Configuration](influxdb_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) +This document was moved to [administration/monitoring/performance/grafana_configuration](../../administration/monitoring/performance/grafana_configuration.md). diff --git a/doc/monitoring/performance/influxdb_configuration.md b/doc/monitoring/performance/influxdb_configuration.md index c30cd2950d8..02647de1eb0 100644 --- a/doc/monitoring/performance/influxdb_configuration.md +++ b/doc/monitoring/performance/influxdb_configuration.md @@ -1,193 +1 @@ -# InfluxDB Configuration - -The default settings provided by [InfluxDB] are not sufficient for a high traffic -GitLab environment. The settings discussed in this document are based on the -settings GitLab uses for GitLab.com, depending on your own needs you may need to -further adjust them. - -If you are intending to run InfluxDB on the same server as GitLab, make sure -you have plenty of RAM since InfluxDB can use quite a bit depending on traffic. - -Unless you are going with a budget setup, it's advised to run it separately. - -## Requirements - -- InfluxDB 0.9.5 or newer -- A fairly modern version of Linux -- At least 4GB of RAM -- At least 10GB of storage for InfluxDB data - -Note that the RAM and storage requirements can differ greatly depending on the -amount of data received/stored. To limit the amount of stored data users can -look into [InfluxDB Retention Policies][influxdb-retention]. - -## Installation - -Installing InfluxDB is out of the scope of this document. Please refer to the -[InfluxDB documentation]. - -## InfluxDB Server Settings - -Since InfluxDB has many settings that users may wish to customize themselves -(e.g. what port to run InfluxDB on), we'll only cover the essentials. - -The configuration file in question is usually located at -`/etc/influxdb/influxdb.conf`. Whenever you make a change in this file, -InfluxDB needs to be restarted. - -### Storage Engine - -InfluxDB comes with different storage engines and as of InfluxDB 0.9.5 a new -storage engine is available, called [TSM Tree]. All users **must** use the new -`tsm1` storage engine as this [will be the default engine][tsm1-commit] in -upcoming InfluxDB releases. - -Make sure you have the following in your configuration file: - -``` -[data] - dir = "/var/lib/influxdb/data" - engine = "tsm1" -``` - -### Admin Panel - -Production environments should have the InfluxDB admin panel **disabled**. This -feature can be disabled by adding the following to your InfluxDB configuration -file: - -``` -[admin] - enabled = false -``` - -### HTTP - -HTTP is required when using the [InfluxDB CLI] or other tools such as Grafana, -thus it should be enabled. When enabling make sure to _also_ enable -authentication: - -``` -[http] - enabled = true - auth-enabled = true -``` - -_**Note:** Before you enable authentication, you might want to [create an -admin user](#create-a-new-admin-user)._ - -### UDP - -GitLab writes data to InfluxDB via UDP and thus this must be enabled. Enabling -UDP can be done using the following settings: - -``` -[[udp]] - enabled = true - bind-address = ":8089" - database = "gitlab" - batch-size = 1000 - batch-pending = 5 - batch-timeout = "1s" - read-buffer = 209715200 -``` - -This does the following: - -1. Enable UDP and bind it to port 8089 for all addresses. -2. Store any data received in the "gitlab" database. -3. Define a batch of points to be 1000 points in size and allow a maximum of - 5 batches _or_ flush them automatically after 1 second. -4. Define a UDP read buffer size of 200 MB. - -One of the most important settings here is the UDP read buffer size as if this -value is set too low, packets will be dropped. You must also make sure the OS -buffer size is set to the same value, the default value is almost never enough. - -To set the OS buffer size to 200 MB, on Linux you can run the following command: - -```bash -sysctl -w net.core.rmem_max=209715200 -``` - -To make this permanent, add the following to `/etc/sysctl.conf` and restart the -server: - -```bash -net.core.rmem_max=209715200 -``` - -It is **very important** to make sure the buffer sizes are large enough to -handle all data sent to InfluxDB as otherwise you _will_ lose data. The above -buffer sizes are based on the traffic for GitLab.com. Depending on the amount of -traffic, users may be able to use a smaller buffer size, but we highly recommend -using _at least_ 100 MB. - -When enabling UDP, users should take care to not expose the port to the public, -as doing so will allow anybody to write data into your InfluxDB database (as -[InfluxDB's UDP protocol][udp] doesn't support authentication). We recommend either -whitelisting the allowed IP addresses/ranges, or setting up a VLAN and only -allowing traffic from members of said VLAN. - -## Create a new admin user - -If you want to [enable authentication](#http), you might want to [create an -admin user][influx-admin]: - -``` -influx -execute "CREATE USER jeff WITH PASSWORD '1234' WITH ALL PRIVILEGES" -``` - -## Create the `gitlab` database - -Once you get InfluxDB up and running, you need to create a database for GitLab. -Make sure you have changed the [storage engine](#storage-engine) to `tsm1` -before creating a database. - -_**Note:** If you [created an admin user](#create-a-new-admin-user) and enabled -[HTTP authentication](#http), remember to append the username (`-username <username>`) -and password (`-password <password>`) you set earlier to the commands below._ - -Run the following command to create a database named `gitlab`: - -```bash -influx -execute 'CREATE DATABASE gitlab' -``` - -The name **must** be `gitlab`, do not use any other name. - -Next, make sure that the database was successfully created: - -```bash -influx -execute 'SHOW DATABASES' -``` - -The output should be similar to: - -``` -name: databases ---------------- -name -_internal -gitlab -``` - -That's it! Now your GitLab instance should send data to InfluxDB. - ---- - -Read more on: - -- [Introduction to GitLab Performance Monitoring](introduction.md) -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) -- [Grafana Install/Configuration](grafana_configuration.md) - -[influxdb-retention]: https://docs.influxdata.com/influxdb/v0.9/query_language/database_management/#retention-policy-management -[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/ -[influxdb cli]: https://docs.influxdata.com/influxdb/v0.9/tools/shell/ -[udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ -[influxdb]: https://influxdata.com/time-series-platform/influxdb/ -[tsm tree]: https://influxdata.com/blog/new-storage-engine-time-structured-merge-tree/ -[tsm1-commit]: https://github.com/influxdata/influxdb/commit/15d723dc77651bac83e09e2b1c94be480966cb0d -[influx-admin]: https://docs.influxdata.com/influxdb/v0.9/administration/authentication_and_authorization/#create-a-new-admin-user +This document was moved to [administration/monitoring/performance/influxdb_configuration](../administration/monitoring/performance/influxdb_configuration.md). diff --git a/doc/monitoring/performance/influxdb_schema.md b/doc/monitoring/performance/influxdb_schema.md index 41861860b6d..a989e323e04 100644 --- a/doc/monitoring/performance/influxdb_schema.md +++ b/doc/monitoring/performance/influxdb_schema.md @@ -1,88 +1 @@ -# InfluxDB Schema - -The following measurements are currently stored in InfluxDB: - -- `PROCESS_file_descriptors` -- `PROCESS_gc_statistics` -- `PROCESS_memory_usage` -- `PROCESS_method_calls` -- `PROCESS_object_counts` -- `PROCESS_transactions` -- `PROCESS_views` - -Here, `PROCESS` is replaced with either `rails` or `sidekiq` depending on the -process type. In all series, any form of duration is stored in milliseconds. - -## PROCESS_file_descriptors - -This measurement contains the number of open file descriptors over time. The -value field `value` contains the number of descriptors. - -## PROCESS_gc_statistics - -This measurement contains Ruby garbage collection statistics such as the amount -of minor/major GC runs (relative to the last sampling interval), the time spent -in garbage collection cycles, and all fields/values returned by `GC.stat`. - -## PROCESS_memory_usage - -This measurement contains the process' memory usage (in bytes) over time. The -value field `value` contains the number of bytes. - -## PROCESS_method_calls - -This measurement contains the methods called during a transaction along with -their duration, and a name of the transaction action that invoked the method (if -available). The method call duration is stored in the value field `duration`, -while the method name is stored in the tag `method`. The tag `action` contains -the full name of the transaction action. Both the `method` and `action` fields -are in the following format: - -``` -ClassName#method_name -``` - -For example, a method called by the `show` method in the `UsersController` class -would have `action` set to `UsersController#show`. - -## PROCESS_object_counts - -This measurement is used to store retained Ruby objects (per class) and the -amount of retained objects. The number of objects is stored in the `count` value -field while the class name is stored in the `type` tag. - -## PROCESS_transactions - -This measurement is used to store basic transaction details such as the time it -took to complete a transaction, how much time was spent in SQL queries, etc. The -following value fields are available: - -| Value | Description | -| ----- | ----------- | -| `duration` | The total duration of the transaction | -| `allocated_memory` | The amount of bytes allocated while the transaction was running. This value is only reliable when using single-threaded application servers | -| `method_duration` | The total time spent in method calls | -| `sql_duration` | The total time spent in SQL queries | -| `view_duration` | The total time spent in views | - -## PROCESS_views - -This measurement is used to store view rendering timings for a transaction. The -following value fields are available: - -| Value | Description | -| ----- | ----------- | -| `duration` | The rendering time of the view | -| `view` | The path of the view, relative to the application's root directory | - -The `action` tag contains the action name of the transaction that rendered the -view. - ---- - -Read more on: - -- [Introduction to GitLab Performance Monitoring](introduction.md) -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Configuration](influxdb_configuration.md) -- [Grafana Install/Configuration](grafana_configuration.md) +This document was moved to [administration/monitoring/performance/influxdb_schema](../administration/monitoring/performance/influxdb_schema.md). diff --git a/doc/monitoring/performance/introduction.md b/doc/monitoring/performance/introduction.md index 79904916b7e..ab3f3ac1664 100644 --- a/doc/monitoring/performance/introduction.md +++ b/doc/monitoring/performance/introduction.md @@ -1,65 +1 @@ -# GitLab Performance Monitoring - -GitLab comes with its own application performance measuring system as of GitLab -8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the -Community and Enterprise editions. - -Apart from this introduction, you are advised to read through the following -documents in order to understand and properly configure GitLab Performance Monitoring: - -- [GitLab Configuration](gitlab_configuration.md) -- [InfluxDB Install/Configuration](influxdb_configuration.md) -- [InfluxDB Schema](influxdb_schema.md) -- [Grafana Install/Configuration](grafana_configuration.md) - -## Introduction to GitLab Performance Monitoring - -GitLab Performance Monitoring makes it possible to measure a wide variety of statistics -including (but not limited to): - -- The time it took to complete a transaction (a web request or Sidekiq job). -- The time spent in running SQL queries and rendering HAML views. -- The time spent executing (instrumented) Ruby methods. -- Ruby object allocations, and retained objects in particular. -- System statistics such as the process' memory usage and open file descriptors. -- Ruby garbage collection statistics. - -Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored -data can be visualized using [Grafana][grafana] or any other application that -supports reading data from InfluxDB. Alternatively data can be queried using the -InfluxDB CLI. - -## Metric Types - -Two types of metrics are collected: - -1. Transaction specific metrics. -1. Sampled metrics, collected at a certain interval in a separate thread. - -### Transaction Metrics - -Transaction metrics are metrics that can be associated with a single -transaction. This includes statistics such as the transaction duration, timings -of any executed SQL queries, time spent rendering HAML views, etc. These metrics -are collected for every Rack request and Sidekiq job processed. - -### Sampled Metrics - -Sampled metrics are metrics that can't be associated with a single transaction. -Examples include garbage collection statistics and retained Ruby objects. These -metrics are collected at a regular interval. This interval is made up out of two -parts: - -1. A user defined interval. -1. A randomly generated offset added on top of the interval, the same offset - can't be used twice in a row. - -The actual interval can be anywhere between a half of the defined interval and a -half above the interval. For example, for a user defined interval of 15 seconds -the actual interval can be anywhere between 7.5 and 22.5. The interval is -re-generated for every sampling run instead of being generated once and re-used -for the duration of the process' lifetime. - -[influxdb]: https://influxdata.com/time-series-platform/influxdb/ -[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/ -[grafana]: http://grafana.org/ +This document was moved to [administration/monitoring/performance/introduction](../administration/monitoring/performance/introduction.md). diff --git a/doc/operations/README.md b/doc/operations/README.md index 6a35dab7b6c..58f16aff7bd 100644 --- a/doc/operations/README.md +++ b/doc/operations/README.md @@ -1,5 +1 @@ -# GitLab operations - -- [Sidekiq MemoryKiller](sidekiq_memory_killer.md) -- [Cleaning up Redis sessions](cleaning_up_redis_sessions.md) -- [Understanding Unicorn and unicorn-worker-killer](unicorn.md) +This document was moved to [administration/operations](../administration/operations.md). diff --git a/doc/operations/cleaning_up_redis_sessions.md b/doc/operations/cleaning_up_redis_sessions.md index 93521e976d5..2a1d0a8c8eb 100644 --- a/doc/operations/cleaning_up_redis_sessions.md +++ b/doc/operations/cleaning_up_redis_sessions.md @@ -1,52 +1 @@ -# Cleaning up stale Redis sessions - -Since version 6.2, GitLab stores web user sessions as key-value pairs in Redis. -Prior to GitLab 7.3, user sessions did not automatically expire from Redis. If -you have been running a large GitLab server (thousands of users) since before -GitLab 7.3 we recommend cleaning up stale sessions to compact the Redis -database after you upgrade to GitLab 7.3. You can also perform a cleanup while -still running GitLab 7.2 or older, but in that case new stale sessions will -start building up again after you clean up. - -In GitLab versions prior to 7.3.0, the session keys in Redis are 16-byte -hexadecimal values such as '976aa289e2189b17d7ef525a6702ace9'. Starting with -GitLab 7.3.0, the keys are -prefixed with 'session:gitlab:', so they would look like -'session:gitlab:976aa289e2189b17d7ef525a6702ace9'. Below we describe how to -remove the keys in the old format. - -First we define a shell function with the proper Redis connection details. - -``` -rcli() { - # This example works for Omnibus installations of GitLab 7.3 or newer. For an - # installation from source you will have to change the socket path and the - # path to redis-cli. - sudo /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket "$@" -} - -# test the new shell function; the response should be PONG -rcli ping -``` - -Now we do a search to see if there are any session keys in the old format for -us to clean up. - -``` -# returns the number of old-format session keys in Redis -rcli keys '*' | grep '^[a-f0-9]\{32\}$' | wc -l -``` - -If the number is larger than zero, you can proceed to expire the keys from -Redis. If the number is zero there is nothing to clean up. - -``` -# Tell Redis to expire each matched key after 600 seconds. -rcli keys '*' | grep '^[a-f0-9]\{32\}$' | awk '{ print "expire", $0, 600 }' | rcli -# This will print '(integer) 1' for each key that gets expired. -``` - -Over the next 15 minutes (10 minutes expiry time plus 5 minutes Redis -background save interval) your Redis database will be compacted. If you are -still using GitLab 7.2, users who are not clicking around in GitLab during the -10 minute expiry window will be signed out of GitLab. +This document was moved to [administration/operations/cleaning_up_redis_sessions](../administration/operations/cleaning_up_redis_sessions.md). diff --git a/doc/operations/moving_repositories.md b/doc/operations/moving_repositories.md index 54adb99386a..c54bca324a5 100644 --- a/doc/operations/moving_repositories.md +++ b/doc/operations/moving_repositories.md @@ -1,180 +1 @@ -# Moving repositories managed by GitLab - -Sometimes you need to move all repositories managed by GitLab to -another filesystem or another server. In this document we will look -at some of the ways you can copy all your repositories from -`/var/opt/gitlab/git-data/repositories` to `/mnt/gitlab/repositories`. - -We will look at three scenarios: the target directory is empty, the -target directory contains an outdated copy of the repositories, and -how to deal with thousands of repositories. - -**Each of the approaches we list can/will overwrite data in the -target directory `/mnt/gitlab/repositories`. Do not mix up the -source and the target.** - -## Target directory is empty: use a tar pipe - -If the target directory `/mnt/gitlab/repositories` is empty the -simplest thing to do is to use a tar pipe. This method has low -overhead and tar is almost always already installed on your system. -However, it is not possible to resume an interrupted tar pipe: if -that happens then all data must be copied again. - -``` -# As the git user -tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ - tar -C /mnt/gitlab/repositories -xf - -``` - -If you want to see progress, replace `-xf` with `-xvf`. - -### Tar pipe to another server - -You can also use a tar pipe to copy data to another server. If your -'git' user has SSH access to the newserver as 'git@newserver', you -can pipe the data through SSH. - -``` -# As the git user -tar -C /var/opt/gitlab/git-data/repositories -cf - -- . |\ - ssh git@newserver tar -C /mnt/gitlab/repositories -xf - -``` - -If you want to compress the data before it goes over the network -(which will cost you CPU cycles) you can replace `ssh` with `ssh -C`. - -## The target directory contains an outdated copy of the repositories: use rsync - -If the target directory already contains a partial / outdated copy -of the repositories it may be wasteful to copy all the data again -with tar. In this scenario it is better to use rsync. This utility -is either already installed on your system or easily installable -via apt, yum etc. - -``` -# As the 'git' user -rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ - /mnt/gitlab/repositories -``` - -The `/.` in the command above is very important, without it you can -easily get the wrong directory structure in the target directory. -If you want to see progress, replace `-a` with `-av`. - -### Single rsync to another server - -If the 'git' user on your source system has SSH access to the target -server you can send the repositories over the network with rsync. - -``` -# As the 'git' user -rsync -a --delete /var/opt/gitlab/git-data/repositories/. \ - git@newserver:/mnt/gitlab/repositories -``` - -## Thousands of Git repositories: use one rsync per repository - -Every time you start an rsync job it has to inspect all files in -the source directory, all files in the target directory, and then -decide what files to copy or not. If the source or target directory -has many contents this startup phase of rsync can become a burden -for your GitLab server. In cases like this you can make rsync's -life easier by dividing its work in smaller pieces, and sync one -repository at a time. - -In addition to rsync we will use [GNU -Parallel](http://www.gnu.org/software/parallel/). This utility is -not included in GitLab so you need to install it yourself with apt -or yum. Also note that the GitLab scripts we used below were added -in GitLab 8.1. - -** This process does not clean up repositories at the target location that no -longer exist at the source. ** If you start using your GitLab instance with -`/mnt/gitlab/repositories`, you need to run `gitlab-rake gitlab:cleanup:repos` -after switching to the new repository storage directory. - -### Parallel rsync for all repositories known to GitLab - -This will sync repositories with 10 rsync processes at a time. We keep -track of progress so that the transfer can be restarted if necessary. - -First we create a new directory, owned by 'git', to hold transfer -logs. We assume the directory is empty before we start the transfer -procedure, and that we are the only ones writing files in it. - -``` -# Omnibus -sudo mkdir /var/opt/gitlab/transfer-logs -sudo chown git:git /var/opt/gitlab/transfer-logs - -# Source -sudo -u git -H mkdir /home/git/transfer-logs -``` - -We seed the process with a list of the directories we want to copy. - -``` -# Omnibus -sudo -u git sh -c 'gitlab-rake gitlab:list_repos > /var/opt/gitlab/transfer-logs/all-repos-$(date +%s).txt' - -# Source -cd /home/git/gitlab -sudo -u git -H sh -c 'bundle exec rake gitlab:list_repos > /home/git/transfer-logs/all-repos-$(date +%s).txt' -``` - -Now we can start the transfer. The command below is idempotent, and -the number of jobs done by GNU Parallel should converge to zero. If it -does not some repositories listed in all-repos-1234.txt may have been -deleted/renamed before they could be copied. - -``` -# Omnibus -sudo -u git sh -c ' -cat /var/opt/gitlab/transfer-logs/* | sort | uniq -u |\ - /usr/bin/env JOBS=10 \ - /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \ - /var/opt/gitlab/transfer-logs/success-$(date +%s).log \ - /var/opt/gitlab/git-data/repositories \ - /mnt/gitlab/repositories -' - -# Source -cd /home/git/gitlab -sudo -u git -H sh -c ' -cat /home/git/transfer-logs/* | sort | uniq -u |\ - /usr/bin/env JOBS=10 \ - bin/parallel-rsync-repos \ - /home/git/transfer-logs/success-$(date +%s).log \ - /home/git/repositories \ - /mnt/gitlab/repositories -` -``` - -### Parallel rsync only for repositories with recent activity - -Suppose you have already done one sync that started after 2015-10-1 12:00 UTC. -Then you might only want to sync repositories that were changed via GitLab -_after_ that time. You can use the 'SINCE' variable to tell 'rake -gitlab:list_repos' to only print repositories with recent activity. - -``` -# Omnibus -sudo gitlab-rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ - sudo -u git \ - /usr/bin/env JOBS=10 \ - /opt/gitlab/embedded/service/gitlab-rails/bin/parallel-rsync-repos \ - success-$(date +%s).log \ - /var/opt/gitlab/git-data/repositories \ - /mnt/gitlab/repositories - -# Source -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:list_repos SINCE='2015-10-1 12:00 UTC' |\ - sudo -u git -H \ - /usr/bin/env JOBS=10 \ - bin/parallel-rsync-repos \ - success-$(date +%s).log \ - /home/git/repositories \ - /mnt/gitlab/repositories -``` +This document was moved to [administration/operations/moving_repositories](../administration/operations/moving_repositories.md). diff --git a/doc/operations/sidekiq_memory_killer.md b/doc/operations/sidekiq_memory_killer.md index b5e78348989..cf7c3b2e2ed 100644 --- a/doc/operations/sidekiq_memory_killer.md +++ b/doc/operations/sidekiq_memory_killer.md @@ -1,40 +1 @@ -# Sidekiq MemoryKiller - -The GitLab Rails application code suffers from memory leaks. For web requests -this problem is made manageable using -[unicorn-worker-killer](https://github.com/kzk/unicorn-worker-killer) which -restarts Unicorn worker processes in between requests when needed. The Sidekiq -MemoryKiller applies the same approach to the Sidekiq processes used by GitLab -to process background jobs. - -Unlike unicorn-worker-killer, which is enabled by default for all GitLab -installations since GitLab 6.4, the Sidekiq MemoryKiller is enabled by default -_only_ for Omnibus packages. The reason for this is that the MemoryKiller -relies on Runit to restart Sidekiq after a memory-induced shutdown and GitLab -installations from source do not all use Runit or an equivalent. - -With the default settings, the MemoryKiller will cause a Sidekiq restart no -more often than once every 15 minutes, with the restart causing about one -minute of delay for incoming background jobs. - -## Configuring the MemoryKiller - -The MemoryKiller is controlled using environment variables. - -- `SIDEKIQ_MEMORY_KILLER_MAX_RSS`: if this variable is set, and its value is - greater than 0, then after each Sidekiq job, the MemoryKiller will check the - RSS of the Sidekiq process that executed the job. If the RSS of the Sidekiq - process (expressed in kilobytes) exceeds SIDEKIQ_MEMORY_KILLER_MAX_RSS, a - delayed shutdown is triggered. The default value for Omnibus packages is set - [in the omnibus-gitlab - repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb). -- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults 900 seconds (15 minutes). When - a shutdown is triggered, the Sidekiq process will keep working normally for - another 15 minutes. -- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. When the grace - time has expired, the MemoryKiller tells Sidekiq to stop accepting new jobs. - Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells - Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must - restart Sidekiq. -- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to `SIGKILL`. The name of - the final signal sent to the Sidekiq process when we want it to shut down. +This document was moved to [administration/operations/sidekiq_memory_killer](../administration/operations/sidekiq_memory_killer.md). diff --git a/doc/operations/unicorn.md b/doc/operations/unicorn.md index bad61151bda..fbc9697b755 100644 --- a/doc/operations/unicorn.md +++ b/doc/operations/unicorn.md @@ -1,86 +1 @@ -# Understanding Unicorn and unicorn-worker-killer - -## Unicorn - -GitLab uses [Unicorn](http://unicorn.bogomips.org/), a pre-forking Ruby web -server, to handle web requests (web browsers and Git HTTP clients). Unicorn is -a daemon written in Ruby and C that can load and run a Ruby on Rails -application; in our case the Rails application is GitLab Community Edition or -GitLab Enterprise Edition. - -Unicorn has a multi-process architecture to make better use of available CPU -cores (processes can run on different cores) and to have stronger fault -tolerance (most failures stay isolated in only one process and cannot take down -GitLab entirely). On startup, the Unicorn 'master' process loads a clean Ruby -environment with the GitLab application code, and then spawns 'workers' which -inherit this clean initial environment. The 'master' never handles any -requests, that is left to the workers. The operating system network stack -queues incoming requests and distributes them among the workers. - -In a perfect world, the master would spawn its pool of workers once, and then -the workers handle incoming web requests one after another until the end of -time. In reality, worker processes can crash or time out: if the master notices -that a worker takes too long to handle a request it will terminate the worker -process with SIGKILL ('kill -9'). No matter how the worker process ended, the -master process will replace it with a new 'clean' process again. Unicorn is -designed to be able to replace 'crashed' workers without dropping user -requests. - -This is what a Unicorn worker timeout looks like in `unicorn_stderr.log`. The -master process has PID 56227 below. - -``` -[2015-06-05T10:58:08.660325 #56227] ERROR -- : worker=10 PID:53009 timeout (61s > 60s), killing -[2015-06-05T10:58:08.699360 #56227] ERROR -- : reaped #<Process::Status: pid 53009 SIGKILL (signal 9)> worker=10 -[2015-06-05T10:58:08.708141 #62538] INFO -- : worker=10 spawned pid=62538 -[2015-06-05T10:58:08.708824 #62538] INFO -- : worker=10 ready -``` - -### Tunables - -The main tunables for Unicorn are the number of worker processes and the -request timeout after which the Unicorn master terminates a worker process. -See the [omnibus-gitlab Unicorn settings -documentation](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/unicorn.md) -if you want to adjust these settings. - -## unicorn-worker-killer - -GitLab has memory leaks. These memory leaks manifest themselves in long-running -processes, such as Unicorn workers. (The Unicorn master process is not known to -leak memory, probably because it does not handle user requests.) - -To make these memory leaks manageable, GitLab comes with the -[unicorn-worker-killer gem](https://github.com/kzk/unicorn-worker-killer). This -gem [monkey-patches](https://en.wikipedia.org/wiki/Monkey_patch) the Unicorn -workers to do a memory self-check after every 16 requests. If the memory of the -Unicorn worker exceeds a pre-set limit then the worker process exits. The -Unicorn master then automatically replaces the worker process. - -This is a robust way to handle memory leaks: Unicorn is designed to handle -workers that 'crash' so no user requests will be dropped. The -unicorn-worker-killer gem is designed to only terminate a worker process _in -between requests_, so no user requests are affected. - -This is what a Unicorn worker memory restart looks like in unicorn_stderr.log. -You see that worker 4 (PID 125918) is inspecting itself and decides to exit. -The threshold memory value was 254802235 bytes, about 250MB. With GitLab this -threshold is a random value between 200 and 250 MB. The master process (PID -117565) then reaps the worker process and spawns a new 'worker 4' with PID -127549. - -``` -[2015-06-05T12:07:41.828374 #125918] WARN -- : #<Unicorn::HttpServer:0x00000002734770>: worker (pid: 125918) exceeds memory limit (256413696 bytes > 254802235 bytes) -[2015-06-05T12:07:41.828472 #125918] WARN -- : Unicorn::WorkerKiller send SIGQUIT (pid: 125918) alive: 23 sec (trial 1) -[2015-06-05T12:07:42.025916 #117565] INFO -- : reaped #<Process::Status: pid 125918 exit 0> worker=4 -[2015-06-05T12:07:42.034527 #127549] INFO -- : worker=4 spawned pid=127549 -[2015-06-05T12:07:42.035217 #127549] INFO -- : worker=4 ready -``` - -One other thing that stands out in the log snippet above, taken from -GitLab.com, is that 'worker 4' was serving requests for only 23 seconds. This -is a normal value for our current GitLab.com setup and traffic. - -The high frequency of Unicorn memory restarts on some GitLab sites can be a -source of confusion for administrators. Usually they are a [red -herring](https://en.wikipedia.org/wiki/Red_herring). +This document was moved to [administration/operations/unicorn](../administration/operations/unicorn.md). diff --git a/doc/permissions/permissions.md b/doc/permissions/permissions.md index 44f3f6d3b12..78d67aeec78 100644 --- a/doc/permissions/permissions.md +++ b/doc/permissions/permissions.md @@ -1,104 +1,3 @@ # Permissions -Users have different abilities depending on the access level they have in a particular group or project. - -If a user is both in a project group and in the project itself, the highest permission level is used. - -If a user is a GitLab administrator they receive all permissions. - -On public and internal projects the Guest role is not enforced. -All users will be able to create issues, leave comments, and pull or download the project code. - -To add or import a user, you can follow the [project users and members -documentation](../workflow/add-user/add-user.md). - -## Project - -| Action | Guest | Reporter | Developer | Master | Owner | -|---------------------------------------|---------|------------|-------------|----------|--------| -| Create new issue | ✓ | ✓ | ✓ | ✓ | ✓ | -| Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | -| See a list of builds | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| See a build log | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| Download and browse build artifacts | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| Pull project code | | ✓ | ✓ | ✓ | ✓ | -| Download project | | ✓ | ✓ | ✓ | ✓ | -| Create code snippets | | ✓ | ✓ | ✓ | ✓ | -| Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | -| Manage labels | | ✓ | ✓ | ✓ | ✓ | -| See a commit status | | ✓ | ✓ | ✓ | ✓ | -| See a container registry | | ✓ | ✓ | ✓ | ✓ | -| See environments | | ✓ | ✓ | ✓ | ✓ | -| Manage merge requests | | | ✓ | ✓ | ✓ | -| Create new merge request | | | ✓ | ✓ | ✓ | -| Create new branches | | | ✓ | ✓ | ✓ | -| Push to non-protected branches | | | ✓ | ✓ | ✓ | -| Force push to non-protected branches | | | ✓ | ✓ | ✓ | -| Remove non-protected branches | | | ✓ | ✓ | ✓ | -| Add tags | | | ✓ | ✓ | ✓ | -| Write a wiki | | | ✓ | ✓ | ✓ | -| Cancel and retry builds | | | ✓ | ✓ | ✓ | -| Create or update commit status | | | ✓ | ✓ | ✓ | -| Update a container registry | | | ✓ | ✓ | ✓ | -| Remove a container registry image | | | ✓ | ✓ | ✓ | -| Create new environments | | | ✓ | ✓ | ✓ | -| Create new milestones | | | | ✓ | ✓ | -| Add new team members | | | | ✓ | ✓ | -| Push to protected branches | | | | ✓ | ✓ | -| Enable/disable branch protection | | | | ✓ | ✓ | -| Turn on/off prot. branch push for devs| | | | ✓ | ✓ | -| Rewrite/remove git tags | | | | ✓ | ✓ | -| Edit project | | | | ✓ | ✓ | -| Add deploy keys to project | | | | ✓ | ✓ | -| Configure project hooks | | | | ✓ | ✓ | -| Manage runners | | | | ✓ | ✓ | -| Manage build triggers | | | | ✓ | ✓ | -| Manage variables | | | | ✓ | ✓ | -| Delete environments | | | | ✓ | ✓ | -| Switch visibility level | | | | | ✓ | -| Transfer project to another namespace | | | | | ✓ | -| Remove project | | | | | ✓ | -| Force push to protected branches [^2] | | | | | | -| Remove protected branches [^2] | | | | | | - -[^1]: If **Allow guest to access builds** is enabled in CI settings -[^2]: Not allowed for Guest, Reporter, Developer, Master, or Owner - -## Group - -In order for a group to appear as public and be browsable, it must contain at -least one public project. - -Any user can remove themselves from a group, unless they are the last Owner of the group. - -| Action | Guest | Reporter | Developer | Master | Owner | -|-------------------------|-------|----------|-----------|--------|-------| -| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | -| Edit group | | | | | ✓ | -| Create project in group | | | | ✓ | ✓ | -| Manage group members | | | | | ✓ | -| Remove group | | | | | ✓ | - -## External Users - -In cases where it is desired that a user has access only to some internal or -private projects, there is the option of creating **External Users**. This -feature may be useful when for example a contractor is working on a given -project and should only have access to that project. - -External users can only access projects to which they are explicitly granted -access, thus hiding all other internal or private ones from them. Access can be -granted by adding the user as member to the project or group. - -They will, like usual users, receive a role in the project or group with all -the abilities that are mentioned in the table above. They cannot however create -groups or projects, and they have the same access as logged out users in all -other cases. - -An administrator can flag a user as external [through the API](../api/users.md) -or by checking the checkbox on the admin panel. As an administrator, navigate -to **Admin > Users** to create a new user or edit an existing one. There, you -will find the option to flag the user as external. - -By default new users are not set as external users. This behavior can be changed -by an administrator under **Admin > Application Settings**.
\ No newline at end of file +This document was moved to [user/permissions.md](../user/permissions.md). diff --git a/doc/project_services/img/builds_emails_service.png b/doc/project_services/img/builds_emails_service.png Binary files differindex 88943dc410e..440728795be 100644 --- a/doc/project_services/img/builds_emails_service.png +++ b/doc/project_services/img/builds_emails_service.png diff --git a/doc/project_services/img/slack_configuration.png b/doc/project_services/img/slack_configuration.png Binary files differnew file mode 100644 index 00000000000..b8de8a56db7 --- /dev/null +++ b/doc/project_services/img/slack_configuration.png diff --git a/doc/project_services/project_services.md b/doc/project_services/project_services.md index e15d5db3253..4442b7c1742 100644 --- a/doc/project_services/project_services.md +++ b/doc/project_services/project_services.md @@ -45,7 +45,7 @@ further configuration instructions and details. Contributions are welcome. | PivotalTracker | Project Management Software (Source Commits Endpoint) | | Pushover | Pushover makes it easy to get real-time notifications on your Android device, iPhone, iPad, and Desktop | | [Redmine](redmine.md) | Redmine issue tracker | -| Slack | A team communication tool for the 21st century | +| [Slack](slack.md) | A team communication tool for the 21st century | ## Services Templates diff --git a/doc/project_services/slack.md b/doc/project_services/slack.md new file mode 100644 index 00000000000..3cfe77c9f85 --- /dev/null +++ b/doc/project_services/slack.md @@ -0,0 +1,50 @@ +# Slack Service + +## On Slack + +To enable Slack integration you must create an incoming webhook integration on +Slack: + +1. [Sign in to Slack](https://slack.com/signin) +1. Visit [Incoming WebHooks](https://my.slack.com/services/new/incoming-webhook/) +1. Choose the channel name you want to send notifications to. +1. Click **Add Incoming WebHooks Integration** +1. Copy the **Webhook URL**, we'll need this later for GitLab. + +## On GitLab + +After you set up Slack, it's time to set up GitLab. + +Go to your project's **Settings > Services > Slack** and you will see a +checkbox with the following events that can be triggered: + +- Push +- Issue +- Merge request +- Note +- Tag push +- Build +- Wiki page + +Bellow each of these event checkboxes, you will have an input field to insert +which Slack channel you want to send that event message, with `#general` +being the default. Enter your preferred channel **without** the hash sign (`#`). + +At the end, fill in your Slack details: + +| Field | Description | +| ----- | ----------- | +| **Webhook** | The [incoming webhook URL][slackhook] which you have to setup on Slack. | +| **Username** | Optional username which can be on messages sent to slack. Fill this in if you want to change the username of the bot. | +| **Notify only broken builds** | If you choose to enable the **Build** event and you want to be only notified about failed builds. | + +After you are all done, click **Save changes** for the changes to take effect. + +>**Note:** +You can set "branch,pushed,Compare changes" as highlight words on your Slack +profile settings, so that you can be aware of new commits when somebody pushes +them. + + + +[slackhook]: https://my.slack.com/services/new/incoming-webhook diff --git a/doc/public_access/public_access.md b/doc/public_access/public_access.md index 9a5c5a5c92a..a3921f1b89f 100644 --- a/doc/public_access/public_access.md +++ b/doc/public_access/public_access.md @@ -17,7 +17,7 @@ Public projects can be cloned **without any** authentication. They will also be listed on the public access directory (`/public`). -**Any logged in user** will have [Guest](../permissions/permissions.md) +**Any logged in user** will have [Guest](../user/permissions.md) permissions on the repository. ### Internal projects @@ -27,7 +27,7 @@ Internal projects can be cloned by any logged in user. They will also be listed on the public access directory (`/public`) for logged in users. -Any logged in user will have [Guest](../permissions/permissions.md) permissions +Any logged in user will have [Guest](../user/permissions.md) permissions on the repository. ### How to change project visibility diff --git a/doc/raketasks/backup_hrz.png b/doc/raketasks/backup_hrz.png Binary files differindex 42084717ebe..287587609a1 100644 --- a/doc/raketasks/backup_hrz.png +++ b/doc/raketasks/backup_hrz.png diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index fa976134341..26baffdf792 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -2,33 +2,47 @@  -## Create a backup of the GitLab system - -A backup creates an archive file that contains the database, all repositories and all attachments. -This archive will be saved in backup_path (see `config/gitlab.yml`). -The filename will be `[TIMESTAMP]_gitlab_backup.tar`. This timestamp can be used to restore an specific backup. -You can only restore a backup to exactly the same version of GitLab that you created it -on, for example 7.2.1. The best way to migrate your repositories from one server to +An application data backup creates an archive file that contains the database, +all repositories and all attachments. +This archive will be saved in `backup_path`, which is specified in the +`config/gitlab.yml` file. +The filename will be `[TIMESTAMP]_gitlab_backup.tar`, where `TIMESTAMP` +identifies the time at which each backup was created. + +You can only restore a backup to exactly the same version of GitLab on which it +was created. The best way to migrate your repositories from one server to another is through backup restore. -You need to keep a separate copy of `/etc/gitlab/gitlab-secrets.json` +To restore a backup, you will also need to restore `/etc/gitlab/gitlab-secrets.json` (for omnibus packages) or `/home/git/gitlab/.secret` (for installations -from source). This file contains the database encryption key used -for two-factor authentication. If you restore a GitLab backup without -restoring the database encryption key, users who have two-factor +from source). This file contains the database encryption key and CI secret +variables used for two-factor authentication. If you fail to restore this +encryption key file along with the application data backup, users with two-factor authentication enabled will lose access to your GitLab server. +## Create a backup of the GitLab system + +Use this command if you've installed GitLab with the Omnibus package: ``` -# use this command if you've installed GitLab with the Omnibus package sudo gitlab-rake gitlab:backup:create - -# if you've installed GitLab from source +``` +Use this if you've installed GitLab from source: +``` sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production ``` -Also you can choose what should be backed up by adding environment variable SKIP. Available options: db, -uploads (attachments), repositories, builds(CI build output logs), artifacts (CI build artifacts), lfs (LFS objects). -Use a comma to specify several options at the same time. +You can specify that portions of the application data be skipped using the +environment variable `SKIP`. You can skip: + +- `db` (database) +- `uploads` (attachments) +- `repositories` (Git repositories data) +- `builds` (CI build output logs) +- `artifacts` (CI build artifacts) +- `lfs` (LFS objects) +- `registry` (Container Registry images) + +Separate multiple data types to skip using a comma. For example: ``` sudo gitlab-rake gitlab:backup:create SKIP=db,uploads @@ -68,7 +82,7 @@ Deleting old backups... [SKIPPING] Starting with GitLab 7.4 you can let the backup script upload the '.tar' file it creates. It uses the [Fog library](http://fog.io/) to perform the upload. In the example below we use Amazon S3 for storage. -But Fog also lets you use [other storage providers](http://fog.io/storage/). +Fog also supports [other storage providers](http://fog.io/storage/). For omnibus packages: @@ -78,6 +92,9 @@ gitlab_rails['backup_upload_connection'] = { 'region' => 'eu-west-1', 'aws_access_key_id' => 'AKIAKIAKI', 'aws_secret_access_key' => 'secret123' + # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty + # ie. 'aws_access_key_id' => '', + # 'use_iam_profile' => 'true' } gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' ``` @@ -94,6 +111,9 @@ For installations from source: region: eu-west-1 aws_access_key_id: AKIAKIAKI aws_secret_access_key: 'secret123' + # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty + # ie. aws_access_key_id: '' + # use_iam_profile: 'true' # The remote 'directory' to store your backups. For S3, this would be the bucket name. remote_directory: 'my.s3.bucket' # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional @@ -154,7 +174,7 @@ with the name of your bucket: ### Uploading to locally mounted shares You may also send backups to a mounted share (`NFS` / `CIFS` / `SMB` / etc.) by -using the [`Local`](https://github.com/fog/fog-local#usage) storage provider. +using the Fog [`Local`](https://github.com/fog/fog-local#usage) storage provider. The directory pointed to by the `local_root` key **must** be owned by the `git` user **when mounted** (mounting with the `uid=` of the `git` user for `CIFS` and `SMB`) or the user that you are executing the backup tasks under (for omnibus @@ -221,11 +241,12 @@ of using encryption in the first place! If you use an Omnibus package please see the [instructions in the readme to backup your configuration](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#backup-and-restore-omnibus-gitlab-configuration). If you have a cookbook installation there should be a copy of your configuration in Chef. -If you have an installation from source, please consider backing up your `.secret` file, `gitlab.yml` file, any SSL keys and certificates, and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). +If you installed from source, please consider backing up your `config/secrets.yml` file, `gitlab.yml` file, any SSL keys and certificates, and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). -At the very **minimum** you should backup `/etc/gitlab/gitlab-secrets.json` -(Omnibus) or `/home/git/gitlab/.secret` (source) to preserve your -database encryption key. +At the very **minimum** you should backup `/etc/gitlab/gitlab.rb` and +`/etc/gitlab/gitlab-secrets.json` (Omnibus), or +`/home/git/gitlab/config/secrets.yml` (source) to preserve your database +encryption key. ## Restore a previously created backup @@ -240,11 +261,11 @@ the SQL database it needs to import data into ('gitlabhq_production'). All existing data will be either erased (SQL) or moved to a separate directory (repositories, uploads). -If some or all of your GitLab users are using two-factor authentication -(2FA) then you must also make sure to restore -`/etc/gitlab/gitlab-secrets.json` (Omnibus) or `/home/git/gitlab/.secret` -(installations from source). Note that you need to run `gitlab-ctl -reconfigure` after changing `gitlab-secrets.json`. +If some or all of your GitLab users are using two-factor authentication (2FA) +then you must also make sure to restore `/etc/gitlab/gitlab.rb` and +`/etc/gitlab/gitlab-secrets.json` (Omnibus), or +`/home/git/gitlab/config/secrets.yml` (installations from source). Note that you +need to run `gitlab-ctl reconfigure` after changing `gitlab-secrets.json`. ### Installation from source @@ -382,6 +403,13 @@ backups using all your disk space. To do this add the following lines to gitlab_rails['backup_keep_time'] = 604800 ``` +Note that the `backup_keep_time` configuration option only manages local +files. GitLab does not automatically prune old files stored in a third-party +object storage (e.g. AWS S3) because the user may not have permission to list +and delete files. We recommend that you configure the appropriate retention +policy for your object storage. For example, you can configure [the S3 backup +policy here as described here](http://stackoverflow.com/questions/37553070/gitlab-omnibus-delete-backup-from-amazon-s3). + NOTE: This cron job does not [backup your omnibus-gitlab configuration](#backup-and-restore-omnibus-gitlab-configuration) or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). ## Alternative backup strategies diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md index 8fbcbb983e9..cf891cd90ad 100644 --- a/doc/raketasks/cleanup.md +++ b/doc/raketasks/cleanup.md @@ -2,7 +2,7 @@ ## Remove garbage from filesystem. Important! Data loss! -Remove namespaces(dirs) from `/home/git/repositories` if they don't exist in GitLab database. +Remove namespaces(dirs) from all repository storage paths if they don't exist in GitLab database. ``` # omnibus-gitlab @@ -12,7 +12,7 @@ sudo gitlab-rake gitlab:cleanup:dirs bundle exec rake gitlab:cleanup:dirs RAILS_ENV=production ``` -Rename repositories from `/home/git/repositories` if they don't exist in GitLab database. +Rename repositories from all repository storage paths if they don't exist in GitLab database. The repositories get a `+orphaned+TIMESTAMP` suffix so that they cannot block new repositories from being created. ``` diff --git a/doc/raketasks/maintenance.md b/doc/raketasks/maintenance.md index d9dce2af480..315cb56a089 100644 --- a/doc/raketasks/maintenance.md +++ b/doc/raketasks/maintenance.md @@ -167,3 +167,22 @@ of those assets. Unless you are modifying the JavaScript / CSS code on your production machine after installing the package, there should be no reason to redo rake assets:precompile on the production machine. If you suspect that assets have been corrupted, you should reinstall the omnibus package. + +## Tracking Deployments + +GitLab provides a Rake task that lets you track deployments in GitLab +Performance Monitoring. This Rake task simply stores the current GitLab version +in the GitLab Performance Monitoring database. + +For Omnibus-packages: + +``` +sudo gitlab-rake gitlab:track_deployment +``` + +For installations from source: + +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production +``` diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md index 629d38efc53..044b104f5c2 100644 --- a/doc/raketasks/user_management.md +++ b/doc/raketasks/user_management.md @@ -60,8 +60,8 @@ block_auto_created_users: false ## Disable Two-factor Authentication (2FA) for all users This task will disable 2FA for all users that have it enabled. This can be -useful if GitLab's `.secret` file has been lost and users are unable to login, -for example. +useful if GitLab's `config/secrets.yml` file has been lost and users are unable +to login, for example. ```bash # omnibus-gitlab @@ -70,3 +70,18 @@ sudo gitlab-rake gitlab:two_factor:disable_for_all_users # installation from source bundle exec rake gitlab:two_factor:disable_for_all_users RAILS_ENV=production ``` + +## Clear authentication tokens for all users. Important! Data loss! + +Clear authentication tokens for all users in the GitLab database. This +task is useful if your users' authentication tokens might have been exposed in +any way. All the existing tokens will become invalid, and new tokens are +automatically generated upon sign-in or user modification. + +``` +# omnibus-gitlab +sudo gitlab-rake gitlab:users:clear_all_authentication_tokens + +# installation from source +bundle exec rake gitlab:users:clear_all_authentication_tokens RAILS_ENV=production +``` diff --git a/doc/university/README.md b/doc/university/README.md new file mode 100644 index 00000000000..f5a0dab39fe --- /dev/null +++ b/doc/university/README.md @@ -0,0 +1,216 @@ +# GitLab University + +GitLab University is the best place to learn about **Version Control with Git and GitLab**. + +It doesn't replace, but accompanies our great [Documentation](http://docs.gitlab.com) +and [Blog Articles](https://about.gitlab.com/blog/). + +Would you like to contribute to GitLab University? Then please take a look at our contribution [process](/process) for more information. + +## Gitlab University Curriculum + +The curriculum is composed of GitLab videos, screencasts, presentations, projects and external GitLab content hosted on other services and has been organized into the following sections. + +1. [GitLab Beginner](#beginner) +1. [GitLab Intermediate](#intermediate) +1. [GitLab Advanced](#advanced) +1. [External Articles](#external) +1. [Resources for GitLab Team Members](#team) + +--- + +### 1. <a name="beginner"></a> GitLab Beginner + +#### 1.1. Version Control and Git + +1. [Version Control Systems](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit#slide=id.g72f2e4906_2_29) +1. [Operating Systems and How Git Works](https://drive.google.com/a/gitlab.com/file/d/0B41DBToSSIG_OVYxVFJDOGI3Vzg/view?usp=sharing) +1. [Code School: An Introduction to Git](https://www.codeschool.com/account/courses/try-git) + +#### 1.2. GitLab Basics + +1. [An Overview of GitLab.com - Video](https://www.youtube.com/watch?v=WaiL5DGEMR4) +1. [Why Use Git and GitLab - Slides](https://docs.google.com/a/gitlab.com/presentation/d/1RcZhFmn5VPvoFu6UMxhMOy7lAsToeBZRjLRn0LIdaNc/edit?usp=drive_web) +1. [GitLab Basics - Article](http://doc.gitlab.com/ce/gitlab-basics/README.html) +1. [Git and GitLab Basics - Video](https://www.youtube.com/watch?v=03wb9FvO4Ak&index=5&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e) +1. [Git and GitLab Basics - Online Course](https://courses.platzi.com/classes/git-gitlab/concepto/part-1/part-23370/material/) +1. [Comparison of GitLab Versions](https://about.gitlab.com/features/#compare) + +#### 1.3. Your GitLab Account + +1. [Create a GitLab Account - Online Course](https://courses.platzi.com/classes/git-gitlab/concepto/first-steps/create-an-account-on-gitlab/material/) +1. [Create and Add your SSH key to GitLab - Video](https://www.youtube.com/watch?v=54mxyLo3Mqk) + +#### 1.4. GitLab Projects + +1. [Repositories, Projects and Groups - Video](https://www.youtube.com/watch?v=4TWfh1aKHHw&index=1&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e) +1. [Creating a Project in GitLab - Video](https://www.youtube.com/watch?v=7p0hrpNaJ14) +1. [How to Create Files and Directories](https://about.gitlab.com/2016/02/10/feature-highlight-create-files-and-directories-from-files-page/) +1. [GitLab Todos](https://about.gitlab.com/2016/03/02/gitlab-todos-feature-highlight/) +1. [GitLab's Work in Progress (WIP) Flag](https://about.gitlab.com/2016/01/08/feature-highlight-wip/) + +#### 1.5. Migrating from other Source Control + +1. [Migrating from BitBucket/Stash](http://doc.gitlab.com/ee/workflow/importing/import_projects_from_bitbucket.html) +1. [Migrating from GitHub](http://doc.gitlab.com/ee/workflow/importing/import_projects_from_github.html) +1. [Migrating from SVN](http://doc.gitlab.com/ee/workflow/importing/migrating_from_svn.html) +1. [Migrating from Fogbugz](http://doc.gitlab.com/ee/workflow/importing/import_projects_from_fogbugz.html) + +#### 1.6. GitLab Inc. + +1. [About GitLab](https://about.gitlab.com/about/) +1. [GitLab Direction](https://about.gitlab.com/direction/) +1. [GitLab Master Plan](https://about.gitlab.com/2016/09/13/gitlab-master-plan/) +1. [Making GitLab Great for Everyone - Video](https://www.youtube.com/watch?v=GGC40y4vMx0) - Response to "Dear GitHub" letter +1. [Using Innersourcing to Improve Collaboration](https://about.gitlab.com/2014/09/05/innersourcing-using-the-open-source-workflow-to-improve-collaboration-within-an-organization/) +1. [The Software Development Market and GitLab - Video](https://www.youtube.com/watch?v=sXlhgPK1NTY&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e&index=6) - [Slides](https://docs.google.com/presentation/d/1vCU-NbZWz8NTNK8Vu3y4zGMAHb5DpC8PE5mHtw1PWfI/edit) +1. [The GitLab Book Club](bookclub/index.md) + +#### 1.7 Community and Support + +1. [Getting Help](https://about.gitlab.com/getting-help/) + - Proposing Features and Reporting and Tracking bugs for GitLab + - The GitLab IRC channel, Gitter Chat Room, Community Forum and Mailing List + - Getting Technical Support + - Being part of our Great Community and Contributing to GitLab +1. [Getting Started with the GitLab Development Kit (GDK)](https://about.gitlab.com/2016/06/08/getting-started-with-gitlab-development-kit/) +1. [Contributing Technical Articles to the GitLab Blog](https://about.gitlab.com/2016/01/26/call-for-writers/) +1. [GitLab Training Workshops](https://about.gitlab.com/training) + +#### 1.8 GitLab Training Material + +1. [Git and GitLab Terminology](glossary/README.md) +1. [Git and GitLab Workshop - Slides](https://docs.google.com/presentation/d/1JzTYD8ij9slejV2-TO-NzjCvlvj6mVn9BORePXNJoMI/edit?usp=drive_web) +1. [Git and GitLab Revision](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/university/training/end-user) + +--- + +### 2. <a name="intermediate"></a> GitLab Intermediate + +#### 2.1 GitLab Pages + +1. [Using any Static Site Generator with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) +1. [Securing GitLab Pages with SSL](https://about.gitlab.com/2016/06/24/secure-gitlab-pages-with-startssl/) +1. [GitLab Pages Documentation](http://doc.gitlab.com/ee/pages/README.html) + +#### 2.2. GitLab Issues + +1. [Markdown in GitLab](http://doc.gitlab.com/ce/markdown/markdown.html) +1. [Issues and Merge Requests - Video](https://www.youtube.com/watch?v=raXvuwet78M) +1. [Due Dates and Milestones fro GitLab Issues](https://about.gitlab.com/2016/08/05/feature-highlight-set-dates-for-issues/) +1. [How to Use GitLab Labels](https://about.gitlab.com/2016/08/17/using-gitlab-labels/) +1. [Applying GitLab Labels Automatically](https://about.gitlab.com/2016/08/19/applying-gitlab-labels-automatically/) +1. [GitLab Issue Board - Product Page](https://about.gitlab.com/solutions/issueboard/) +1. [An Overview of GitLab Issue Board](https://about.gitlab.com/2016/08/22/announcing-the-gitlab-issue-board/) +1. [Designing GitLab Issue Board](https://about.gitlab.com/2016/08/31/designing-issue-boards/) +1. [From Idea to Production with GitLab - Video](https://www.youtube.com/watch?v=25pHyknRgEo&index=14&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e) + +#### 2.3. Continuous Integration + +1. [Operating Systems, Servers, VMs, Containers and Unix - Video](https://www.youtube.com/watch?v=V61kL6IC-zY&index=8&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e) +1. [GitLab CI - Product Page](https://about.gitlab.com/gitlab-ci/) +1. [Getting started with GitLab and GitLab CI](https://about.gitlab.com/2015/12/14/getting-started-with-gitlab-and-gitlab-ci/) +1. [GitLab Container Registry](https://about.gitlab.com/2016/05/23/gitlab-container-registry/) +1. [GitLab and Docker - Video](https://www.youtube.com/watch?v=ugOrCcbdHko&index=12&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e) +1. [How we scale GitLab with built in Docker](https://about.gitlab.com/2016/06/21/how-we-scale-gitlab-by-having-docker-built-in/) +1. [Continuous Integration, Delivery, and Deployment with GitLab](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) +1. [Deployments and Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) +1. [Sequential, Parallel or Custom Pipelines](https://about.gitlab.com/2016/07/29/the-basics-of-gitlab-ci/) +1. [Setting up GitLab Runner For Continuous Integration](https://about.gitlab.com/2016/03/01/gitlab-runner-with-docker/) +1. [Setting up GitLab Runner on DigitalOcean](https://about.gitlab.com/2016/04/19/how-to-set-up-gitlab-runner-on-digitalocean/) +1. [Setting up GitLab CI for iOS projects](https://about.gitlab.com/2016/03/10/setting-up-gitlab-ci-for-ios-projects/) +1. [IBM: Continuous Delivery vs Continuous Deployment - Video](https://www.youtube.com/watch?v=igwFj8PPSnw) +1. [Amazon: Transition to Continuous Delivery - Video](https://www.youtube.com/watch?v=esEFaY0FDKc) +1. See **[Integrations](#integrations)** for integrations with other CI services. + +#### 2.4. Workflow + +1. [GitLab Flow - Video](https://youtu.be/enMumwvLAug?list=PLFGfElNsQthZnwMUFi6rqkyUZkI00OxIV) +1. [GitLab Flow vs Forking in GitLab - Video](https://www.youtube.com/watch?v=UGotqAUACZA) +1. [GitLab Flow Overview](https://about.gitlab.com/2014/09/29/gitlab-flow/) +1. [Always Start with an Issue](https://about.gitlab.com/2016/03/03/start-with-an-issue/) +1. [GitLab Flow Documentation](http://doc.gitlab.com/ee/workflow/gitlab_flow.html) + +#### 2.5. GitLab Comparisons + +1. [GitLab Compared to Other Tools](https://about.gitlab.com/comparison/) +1. [Comparing GitLab Terminology](https://about.gitlab.com/2016/01/27/comparing-terms-gitlab-github-bitbucket/) +1. [GitLab Compared to Atlassian (Recording 2016-03-03) ](https://youtu.be/Nbzp1t45ERo) +1. [GitLab Position FAQ](https://about.gitlab.com/handbook/positioning-faq) +1. [Customer review of GitLab with points on why they prefer GitLab](https://www.enovate.co.uk/web-design-blog/2015/11/25/gitlab-review/) + +--- + +### 3. <a name="advanced"></a> GitLab Advanced + +#### 3.1. Dev Ops + +1. [Xebia Labs: Dev Ops Terminology](https://xebialabs.com/glossary/) +1. [Xebia Labs: Periodic Table of DevOps Tools](https://xebialabs.com/periodic-table-of-devops-tools/) +1. [Puppet Labs: State of Dev Ops 2015 - Book](https://puppetlabs.com/sites/default/files/2015-state-of-devops-report.pdf) + +#### 3.2. Installing GitLab with Omnibus + +1. [What is Omnibus - Video](https://www.youtube.com/watch?v=XTmpKudd-Oo) +1. [How to Install GitLab with Omnibus - Video](https://www.youtube.com/watch?v=Q69YaOjqNhg) +1. [Installing GitLab - Online Course](https://courses.platzi.com/classes/git-gitlab/concepto/part-1/part-3/material/) +1. [Using a Non-Packaged PostgreSQL Database](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#using-a-non-packaged-postgresql-database-management-server) +1. [Using a MySQL Database](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#using-a-mysql-database-management-server-enterprise-edition-only) +1. [Installing GitLab on Microsoft Azure](https://about.gitlab.com/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/) +1. [Installing GitLab on Digital Ocean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/) + +#### 3.3. Permissions + +1. [How to Manage Permissions in GitLab EE - Video](https://www.youtube.com/watch?v=DjUoIrkiNuM) + +#### 3.4. Large Files + +1. [Big files in Git (Git LFS, Annex) - Video](https://www.youtube.com/watch?v=DawznUxYDe4) + +#### 3.5. LDAP and Active Directory + +1. [How to Manage LDAP, Active Directory in GitLab - Video](https://www.youtube.com/watch?v=HPMjM-14qa8) + +#### 3.6 Custom Languages + +1. [How to add Syntax Highlighting Support for Custom Langauges to GitLab - Video](how to add support for your favorite language to GitLab) + +#### 3.7. Scalability and High Availability + +1. [Scalability and High Availability - Video](https://www.youtube.com/watch?v=cXRMJJb6sp4&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e&index=2) +1. [High Availability - Video](https://www.youtube.com/watch?v=36KS808u6bE&index=15&list=PLFGfElNsQthbQu_IWlNOxul0TbS_2JH-e) +1. [High Availability Documentation](https://about.gitlab.com/high-availability/) + +#### 3.8 Cycle Analytics + +1. [GitLab Cycle Analytics Overview](https://about.gitlab.com/2016/09/21/cycle-analytics-feature-highlight/) +1. [GitLab Cycle Analytics - Product Page](https://about.gitlab.com/solutions/cycle-analytics/) + +#### 3.9. <a name="integrations"></a> Integrations + +1. [How to Integrate JIRA and Jenkins with GitLab - Video](https://gitlabmeetings.webex.com/gitlabmeetings/ldr.php?RCID=44b548147a67ab4d8a62274047146415) +1. [How to Integrate Jira with GitLab](http://doc.gitlab.com/ee/integration/jira.html) +1. [How to Integrate Jenkins with GitLab](http://doc.gitlab.com/ee/integration/jenkins.html) +1. [How to Integrate Bamboo with GitLab](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/project_services/bamboo.md) +1. [How to Integrate Slack with GitLab](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/integration/slack.md) +1. [How to Integrate Convox with GitLab](https://about.gitlab.com/2016/06/09/continuous-delivery-with-gitlab-and-convox/) +1. [Getting Started with GitLab and Shippable CI](https://about.gitlab.com/2016/05/05/getting-started-gitlab-and-shippable/) + +--- + +## 4. <a name="external"></a> External Articles + +1. [2011 WSJ article by Mark Andreeson - Software is Eating the World](http://www.wsj.com/articles/SB10001424053111903480904576512250915629460) +1. [2014 Blog post by Chris Dixon - Software eats software development](http://cdixon.org/2014/04/13/software-eats-software-development/) +1. [2015 Venture Beat article - Actually, Open Source is Eating the World](http://venturebeat.com/2015/12/06/its-actually-open-source-software-thats-eating-the-world/) + +--- + +## 5. <a name="team"></a> Resources for GitLab Team Members + +*Some content can only be accessed by GitLab team members* + +1. [Support Path](support/README.md) +1. [Sales Path (redirect to sales handbook)](https://about.gitlab.com/handbook/sales-onboarding/) +1. [GitLab architecture for noobs](https://dev.gitlab.org/gitlab/gitlabhq/blob/master/doc/development/architecture.md) +1. [Client Assessment of GitLab versus GitHub](https://docs.google.com/a/gitlab.com/spreadsheets/d/18cRF9Y5I6I7Z_ab6qhBEW55YpEMyU4PitZYjomVHM-M/edit?usp=sharing) diff --git a/doc/university/bookclub/booklist.md b/doc/university/bookclub/booklist.md new file mode 100644 index 00000000000..c4229832e9f --- /dev/null +++ b/doc/university/bookclub/booklist.md @@ -0,0 +1,113 @@ +# Books + +List of books and resources, that may be worth reading. + +## Papers + +1. **The Humble Programmer** + + Edsger W. Dijkstra, 1972 ([paper](http://dl.acm.org/citation.cfm?id=361591)) + +## Programming + +1. **Design Patterns: Elements of Reusable Object-Oriented Software** + + Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides, 1994 ([amazon](http://www.amazon.com/Design-Patterns-Elements-Reusable-Object-Oriented/dp/0201633612)) + +1. **Clean Code: A Handbook of Agile Software Craftsmanship** + + Robert C. "Uncle Bob" Martin, 2008 ([amazon](http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882)) + +1. **Code Complete: A Practical Handbook of Software Construction**, 2nd Edition + + Steve McConnell, 2004 ([amazon](http://www.amazon.com/Code-Complete-Practical-Handbook-Construction/dp/0735619670)) + +1. **The Pragmatic Programmer: From Journeyman to Master** + + Andrew Hunt, David Thomas, 1999 ([amazon](http://www.amazon.com/Pragmatic-Programmer-Journeyman-Master/dp/020161622X)) + +1. **Working Effectively with Legacy Code** + + Michael Feathers, 2004 ([amazon](http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052)) + +1. **Eloquent Ruby** + + Russ Olsen, 2011 ([amazon](http://www.amazon.com/Eloquent-Ruby-Addison-Wesley-Professional/dp/0321584104)) + +1. **Domain-Driven Design: Tackling Complexity in the Heart of Software** + + Eric Evans, 2003 ([amazon](http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215)) + +1. **How to Solve It: A New Aspect of Mathematical Method** + + Polya G. 1957 ([amazon](http://www.amazon.com/How-Solve-Mathematical-Princeton-Science/dp/069116407X)) + +1. **Software Creativity 2.0** + + Robert L. Glass, 2006 ([amazon](http://www.amazon.com/Software-Creativity-2-0-Robert-Glass/dp/0977213315)) + +1. **Object-Oriented Software Construction** + + Bertrand Meyer, 1997 ([amazon](http://www.amazon.com/Object-Oriented-Software-Construction-Book-CD-ROM/dp/0136291554)) + +1. **Refactoring: Improving the Design of Existing Code** + + Martin Fowler, Kent Beck, 1999 ([amazon](http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/dp/0201485672)) + +1. **Test Driven Development: By Example** + + Kent Beck, 2002 ([amazon](http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530)) + +1. **Algorithms in C++: Fundamentals, Data Structure, Sorting, Searching** + + Robert Sedgewick, 1990 ([amazon](http://www.amazon.com/Algorithms-Parts-1-4-Fundamentals-Structure/dp/0201350882)) + +1. **Effective C++** + + Scott Mayers, 1996 ([amazon](http://www.amazon.com/Effective-Specific-Improve-Programs-Designs/dp/0321334876)) + +1. **Extreme Programming Explained: Embrace Change** + + Kent Beck, 1999 ([amazon](http://www.amazon.com/Extreme-Programming-Explained-Embrace-Change/dp/0321278658)) + +1. **The Art of Computer Programming** + + Donald E. Knuth, 1997 ([amazon](http://www.amazon.com/Computer-Programming-Volumes-1-4A-Boxed/dp/0321751043)) + +1. **Writing Efficient Programs** + + Jon Louis Bentley, 1982 ([amazon](http://www.amazon.com/Writing-Efficient-Programs-Prentice-Hall-Software/dp/013970244X)) + +1. **The Mythical Man-Month: Essays on Software Engineering** + + Frederick Phillips Brooks, 1975 ([amazon](http://www.amazon.com/Mythical-Man-Month-Essays-Software-Engineering/dp/0201006502)) + +1. **Peopleware: Productive Projects and Teams** 3rd Edition + + Tom DeMarco, Tim Lister, 2013 ([amazon](http://www.amazon.com/Peopleware-Productive-Projects-Teams-3rd/dp/0321934113)) + +1. **Principles Of Software Engineering Management** + + Tom Gilb, 1988 ([amazon](http://www.amazon.com/Principles-Software-Engineering-Management-Gilb/dp/0201192462)) + +## Other + +1. **Thinking, Fast and Slow** + + Daniel Kahneman, 2013 ([amazon](http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555)) + +1. **The Social Animal** 11th Edition + + Elliot Aronson, 2011 ([amazon](http://www.amazon.com/Social-Animal-Elliot-Aronson/dp/1429233419)) + +1. **Influence: Science and Practice** 5th Edition + + Robert B. Cialdini, 2008 ([amazon](http://www.amazon.com/Influence-Practice-Robert-B-Cialdini/dp/0205609996)) + +1. **Getting to Yes: Negotiating Agreement Without Giving In** + + Roger Fisher, William L. Ury, Bruce Patton, 2011 ([amazon](http://www.amazon.com/Getting-Yes-Negotiating-Agreement-Without/dp/0143118757)) + +1. **How to Win Friends & Influence People** + + Dale Carnegie, 1981 ([amazon](http://www.amazon.com/How-Win-Friends-Influence-People/dp/0671027034)) diff --git a/doc/university/bookclub/index.md b/doc/university/bookclub/index.md new file mode 100644 index 00000000000..022a61f4429 --- /dev/null +++ b/doc/university/bookclub/index.md @@ -0,0 +1,19 @@ +# The GitLab Book Club + +The Book Club is a casual meet-up to read and discuss books we like. +We'll find a time that suits most, if not all. + +See the [book list](booklist.md) for additional recommendations. + +## Currently reading : Books about remote work + +1. **Remote: Office not required** + + David Heinemeier Hansson and Jason Fried, 2013 + ([amazon](http://www.amazon.co.uk/Remote-Required-David-Heinemeier-Hansson/dp/0091954673)) + +1. **The Year Without Pants** + + Scott Berkun, 2013 ([ScottBerkun.com](http://scottberkun.com/yearwithoutpants/)) + +Any other books you'd like to suggest? Edit this page and add them to the queue. diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md new file mode 100644 index 00000000000..cf836667fac --- /dev/null +++ b/doc/university/glossary/README.md @@ -0,0 +1,589 @@ + +## What is the Glossary + +This contains a simplified list and definitions of some of the terms that you will encounter in your day to day activities when working with GitLab. +Please add any terms that you discover that you think would be useful for others. + +### 2FA + +User authentication by combination of 2 different steps during login. This allows for [more security](https://about.gitlab.com/handbook/security/). + +### Access Levels + +Process of selective restriction to create, view, modify or delete a resource based on a set of assigned permissions. See [GitLab's Permission Guidelines](http://doc.gitlab.com/ce/permissions/permissions.html) + +### Active Directory (AD) + +A Microsoft-based [directory service](https://msdn.microsoft.com/en-us/library/bb742424.aspx) for windows domain networks. It uses LDAP technology under the hood. + +### Agile + +Building and [delivering software](http://agilemethodology.org/) in phases/parts rather than trying to build everything at once then delivering to the user/client. The latter is known as the WaterFall model. + +### Application Lifecycle Management (ALM) + +The entire product lifecycle management process for an application, from requirements management, development, and testing until deployment. GitLab has [advantages](https://docs.google.com/presentation/d/1vCU-NbZWz8NTNK8Vu3y4zGMAHb5DpC8PE5mHtw1PWfI/edit#slide=id.g72f2e4906_2_288) over both legacy and modern ALM tools. + +### Artifactory + +A version control [system](https://www.jfrog.com/open-source/#os-arti) for non-text files. + +### Artifacts + +Objects (usually binary and large) created by a build process. These can include use cases, class diagrams, requirements and design documents. + +### Atlassian + +A [company](https://www.atlassian.com) that develops software products for developers and project managers including Bitbucket, Jira, Hipchat, Confluence, Bamboo. + +### Audit Log + +Also called an [audit trail](https://en.wikipedia.org/wiki/Audit_trail), an audit log is a document that records an event in an IT system. + +### Auto Defined User Group + +User groups are a way of centralizing control over important management tasks, particularly access control and password policies. A simple example of such groups are the users and the admins groups. +In most of the cases these groups are auto defined in terms of access, rules of usage, conditions to be part of, etc. + +### Bamboo + +Atlassian's CI tool similar to GitLab CI and Jenkins. + +### Basic Subscription + +Entry level [subscription](https://about.gitlab.com/pricing/) for GitLab EE currently available in packs of 10. + +### Bitbucket + +Atlassian's web hosting service for Git and Mercurial Projects. Read about [migrating](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_bitbucket.html) from BitBucket to a GitLab instance. + +### Branch + +A branch is a parallel version of a repository. This allows you to work on the repository without affecting the "master" branch, and without affecting the current "live" version. When you have made all your changes to your branch you can then merge to the master. When your merge request is accepted your changes will be "live." + +### Branded Login + +Having your own logo on [your GitLab instance login page](https://docs.gitlab.com/ee/customization/branded_login_page.html) instead of the GitLab logo. + +### Build triggers +These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](https://docs.gitlab.com/ce/ci/triggers/README.html) build triggers. + +### CEPH + + A distributed object store and file [system](http://ceph.com/) designed to provide excellent performance, reliability and scalability. + +### ChatOps + +The ability to [initiate an action](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/1412) from chat. ChatBots run in your chat application and give you the ability to do "anything" from chat. + +### Clone + +A [copy](https://git-scm.com/docs/git-clone) of a repository stored on your machine that allows you to use your own editor without being online, but still tracks the changes made remotely. + +### Code Review + +Examination of a progam's code. The main aim is to maintain high quality standards of code that is being shipped. Merge requests [serve as a code review tool](https://about.gitlab.com/2014/09/29/gitlab-flow/) in GitLab. + +### Code Snippet + +A small amount of code, usually selected for the purpose of showing other developers how to do something specific or reproduce a problem. + +### Collaborator + +Person with read and write access to a repository who has been invited by repository owner. + +### Commit + +A [change](https://git-scm.com/docs/git-commit) (revision) to a file that also creates an ID, allowing you to see revision history and the author of the changes. + +### Community + +[Everyone](https://about.gitlab.com/community/) who uses GitLab. + +### Confluence + +Atlassian's product for collaboration on documents and projects. + +### Continuous Delivery + +A [software engineering approach](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which continuous integration, automated testing, and automated deployment capabilities allow software to be developed and deployed rapidly, reliably and repeatedly with minimal human intervention. Still, the deployment to production is defined strategically and triggered manually. + +### Continuous Deployment + +A [software development practice](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which every code change goes through the entire pipeline and is put into production automatically, resulting in many production deployments every day. It does everything that Continuous Delivery does, but the process is fully automated, there's no human intervention at all. + +### Continuous Integration + +A [software development practice](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/) in which you build and test software every time a developer pushes code to the application, and it happens several times a day. + +### Contributor + +Term used for a person contributing to an open source project. + +### Conversational Development (ConvDev) + +A [natural evolution](https://about.gitlab.com/2016/09/14/gitlab-live-event-recap/) of software development that carries a conversation across functional groups throughout the development process, enabling developers to track the full path of development in a cohesive and intuitive way. ConvDev accelerates the development lifecycle by fostering collaboration and knowledge sharing from idea to production. + +### Cycle Time + +The time it takes to move from [idea to production](https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab). + +### Data Centre + +Atlassian product for High Availability. + +### Deploy Keys + +A [SSH key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. + +### Developer + +For us at GitLab, this means a software developer, or someone who makes software. It is also one of the levels of access in our multi-level approval system. + +### DevOps + +The intersection of software engineering, quality assurance, and technology operations. Explore more DevOps topics in the [glossary by XebiaLabs](https://xebialabs.com/glossary/) + +### Diff + +The difference between two commits, or saved changes. This will also be shown visually after the changes. + +#### Directory + +A folder used for storing multiple files. + +### Docker Container Registry + +A [feature](https://docs.gitlab.com/ce/user/project/container_registry.html) of GitLab projects. Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. + +### Dynamic Environment + +### ElasticSearch + +Elasticsearch is a flexible, scalable and powerful search service. When [enabled](https://gitlab.com/help/integration/elasticsearch.md), it helps keep GitLab's search fast when dealing with a huge amount of data. + +### Emacs + +### Fork + +Your [own copy](https://docs.gitlab.com/ce/workflow/forking_workflow.html) of a repository that allows you to make changes to the repository without affecting the original. + +### Gerrit + +A code review [tool](https://www.gerritcodereview.com/) built on top of Git. + +### Git Attributes + +A [git attributes file](https://git-scm.com/docs/gitattributes) is a simple text file that gives attributes to pathnames. + +### Git Hooks + +[Scripts](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) you can use to trigger actions at certain points. + +### GitHost.io + +A single-tenant solution that provides GitLab CE or EE as a managed service. GitLab Inc. is responsible for installing, updating, hosting, and backing up customers' own private and secure GitLab instance. + +### GitHub + +A web-based Git repository hosting service with an enterprise offering. Its main features are: issue tracking, pull request with code review, abundancy of integrations and wiki. It offers free public repos, private repos and enterprise services are paid. Read about [importing a project](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_github.html) from GitHub to GitLab. + +### GitLab CE + +Our free on Premise solution with >100,000 users + +### GitLab CI + +Our own Continuos Integration [feature](https://about.gitlab.com/gitlab-ci/) that is shipped with each instance + +### GitLab EE + +Our premium on premise [solution](https://about.gitlab.com/features/#enterprise) that currently has Basic, Standard and Plus subscription packages with additional features and support. + +### GitLab.com + +Our free SaaS for public and private repositories. + +### GitLab Geo + +Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](https://docs.gitlab.com/ee/gitlab-geo/README.html) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. + +### GitLab Pages +These allow you to [create websites](https://gitlab.com/help/pages/README.md) for your GitLab projects, groups, or user account. + +### Gitolite + +An [access layer](https://git-scm.com/book/en/v1/Git-on-the-Server-Gitolite) that sits on top of Git. Users are granted access to repos via a simple config file. As an admin, you only need the users' public SSH key and a username. + +### Gitorious + +A web-based hosting service for projects using Git. It was acquired by GitLab and we discontinued the service. Read the[Gitorious Acquisition Blog Post](https://about.gitlab.com/2015/03/03/gitlab-acquires-gitorious/). + +### Go + +An open source programming [language](https://golang.org/). + +### GUI/ Git GUI + +A portable [graphical interface](https://git-scm.com/docs/git-gui) to Git that allows users to make changes to their repository by making new commits, amending existing ones, creating branches, performing local merges, and fetching/pushing to remote repositories. + +### High Availability for Disaster Recovery (HADR) + +Sometimes written HA/DR, this usually refers to a strategy for having a failover server in place in case the main server fails. + +### Hip Chat + +Atlassian's real time chat application for teams, Hip Chat is a competitor to Slack, RocketChat and MatterMost. + +### High Availability + +Refers to a [system or component](https://about.gitlab.com/high-availability/) that is continuously operational for a desirably long length of time. Availability can be measured relative to "100% operational" or "never failing." + +### Inner-sourcing + +The [use of](https://about.gitlab.com/2014/09/05/innersourcing-using-the-open-source-workflow-to-improve-collaboration-within-an-organization/) open source development techniques within the corporation. + +### Internet Relay Chat (IRC) + +An [application layer protocol](http://www.irchelp.org/) that facilitates communication in the form of text. + +### Issue Tracker + +A [tool](https://docs.gitlab.com/ee/integration/external-issue-tracker.html) used to manage, organize, and maintain a list of issues, making it easier for an organization to manage. + +### Jenkins + +An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins-ci.org/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. + +### Jira + +Atlassian's [project management software](https://www.atlassian.com/software/jira), i.e. a complex issue tracker. GitLab [can be configured](https://docs.gitlab.com/ee/project_services/jira.html) to interact with JIRA Core either using an on-premise instance or the SaaS solution that Atlassian offers. + +### JUnit + +A testing framework for the Java programming language, [JUnit](http://junit.org/junit4/) has been important in the evolution of test-driven development. + +### Kerberos + +A network authentication [protocol](http://web.mit.edu/kerberos/) that uses secret-key cryptography for security. + +### Kubernetes + +An open source container cluster manager originally designed by Google. It's basically a platform for automating deployment, scaling, and operations of application containers over clusters of hosts. + +### Labels + +An [identifier](https://docs.gitlab.com/ce/user/project/labels.html) to describe a group of one or more specific file revisions. + +### Lightweight Directory Access Protocol (LDAP) + + A directory (electronic address book) with user information (e.g. name, phone_number etc.) + +### LDAP User Authentication + +GitLab [integrates](https://docs.gitlab.com/ce/administration/auth/ldap.html) with LDAP to support user authentication. This enables GitLab to sign in people from an LDAP server (i.e., allowing people whose names are on the electronic user directory server to be able to use their LDAP accounts to login.) + +### LDAP Group Sync + +Allows you to synchronize the members of a GitLab group with one or more LDAP groups. + +### Load Balancer + +A [device](https://en.wikipedia.org/wiki/Load_balancing_(computing)) that distributes network or application traffic across multiple servers. + +### Git Large File Storage (LFS) + +A way [to enable](https://about.gitlab.com/2015/11/23/announcing-git-lfs-support-in-gitlab/) git to handle large binary files by using reference pointers within small text files to point to the large files. Large files such as high resolution images and videos, audio files, and assets can be called from a remote server. + +### Linux + +An operating system like Windows or OS X. It is mostly used by software developers and on servers. + +### Markdown + +A lightweight markup language with plain text formatting syntax designed so that it can be converted to HTML and many other formats using a tool by the same name. Markdown is often used to format readme files, for writing messages in online discussion forums, and to create rich text using a plain text editor. Checkout GitLab's [Markdown guide](https://gitlab.com/help/user/markdown.md). + +### Maria DB + +A community developed fork/variation of MySQL. MySQL is owned by Oracle. + +### Master + +Name of the [default branch](https://git-scm.com/book/en/v1/Git-Branching-What-a-Branch-Is) in every git repository. + +### Mattermost + +An open source, self-hosted messaging alternative to Slack. View GitLab's Mattermost [feature](https://gitlab.com/gitlab-org/gitlab-mattermost). + +### Mercurial + +A free distributed version control system similar to and a competitor with Git. + +### Merge + +Takes changes from one branch, and [applies them](https://git-scm.com/docs/git-merge) into another branch. + +### Merge Conflict + +[Arises](https://about.gitlab.com/2016/09/06/resolving-merge-conflicts-from-the-gitlab-ui/) when a merge can't be performed cleanly between two versions of the same file. + +### Meteor + +A [platform](https://www.meteor.com) for building javascript apps. + +### Milestones + +Allow you to [organize issues](https://docs.gitlab.com/ce/workflow/milestones.html) and merge requests in GitLab into a cohesive group, optionally setting a due date. A common use is keeping track of an upcoming software version. Milestones are created per-project. + +### Mirror Repositories + +A project that is setup to automatically have its branches, tags, and commits [updated from an upstream repository](https://docs.gitlab.com/ee/workflow/repository_mirroring.html). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. + +### MIT License + +A type of software license. It lets people do anything with your code with proper attribution and without warranty. It is the most common license for open source applications written in Ruby on Rails. GitLab CE is issued under this [license](https://docs.gitlab.com/ce/development/licensing.html). This means you can download the code, modify it as you want, and even build a new commercial product using the underlying code and it's not illegal. The only condition is that there is no form of warranty provided by GitLab so whatever happens when you use the code is your own problem. + +### Mondo Rescue + +A free disaster recovery [software](https://help.ubuntu.com/community/MondoMindi). + +### MySQL + +A relational [database](http://www.mysql.com/) owned by Oracle. Currently only supported if you are using EE. + +### Namespace + +A set of symbols that are used to organize objects of various kinds so that these objects may be referred to by name. Examples of namespaces in action include file systems that assign names to files; programming languages that organize their variables and subroutines in namespaces; and computer networks and distributed systems that assign names to resources, such as computers, printers, websites, (remote) files, etc. + +### Nginx + +A web [server](https://www.nginx.com/resources/wiki/) (pronounced "engine x"). It can act as a reverse proxy server for HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as a load balancer and an HTTP cache. + +### OAuth + +An open standard for authorization, commonly used as a way for internet users to log into third party websites using their Microsoft, Google, Facebook or Twitter accounts without exposing their password. GitLab [is](https://docs.gitlab.com/ce/integration/oauth_provider.html) an OAuth2 authentication service provider. + +### Omnibus Packages + +A way to [package different services and tools](https://docs.gitlab.com/omnibus/) required to run GitLab, so that most developers can install it without laborious configuration. + +### On Premise + +On your own server. In GitLab, this [refers](https://about.gitlab.com/2015/02/12/why-ship-on-premises-in-the-saas-era/) to the ability to download GitLab EE/GitLab CE and host it on your own server rather than using GitLab.com, which is hosted by GitLab Inc's servers. + +### Open Core + +GitLab's [business model](https://about.gitlab.com/2016/07/20/gitlab-is-open-core-github-is-closed-source/). Coined by Andrew Lampitt in 2008, the [open core model](https://en.wikipedia.org/wiki/Open_core) primarily involves offering a "core" or feature-limited version of a software product as free and open-source software, while offering "commercial" versions or add-ons as proprietary software. + +### Open Source Software + +Software for which the original source code is freely [available](https://opensource.org/docs/osd) and may be redistributed and modified. GitLab prioritizes open source [stewardship](https://about.gitlab.com/2016/01/11/being-a-good-open-source-steward/). + +### Owner + +The most powerful person on a GitLab project. They have the permissions of all the other users plus the additional permission of being able to destroy (i.e. delete) the project. + +### Platform as a Service (PaaS) + +Typically referred to in regards to application development, PaaS is a model in which a cloud provider delivers hardware and software tools to its users as a service. + +### Perforce + +The company that produces Helix. A commercial, proprietary, centralised VCS well known for its ability to version files of any size and type. They OEM a re-branded version of GitLab called "GitSwarm" that is tightly integrated with their "GitFusion" product, which in turn represents a portion of a Helix repository (called a depot) as a git repo. + +### Phabricator + +A suite of web-based software development collaboration tools, including the Differential code review tool, the Diffusion repository browser, the Herald change monitoring tool, the Maniphest bug tracker and the Phriction wiki. Phabricator integrates with Git, Mercurial, and Subversion. + +### Piwik Analytics + +An open source analytics software to help you analyze web traffic. It is similar to Google Analytics, except that the latter is not open source and information is stored by Google. In Piwik, the information is stored on your own server and hence is fully private. + +### Plus Subscription + +GitLab Premium EE [subscription](https://about.gitlab.com/pricing/) that includes training and dedicated Account Management and Service Engineer and complete support package. + +### PostgreSQL + +An [object-relational](https://en.wikipedia.org/wiki/PostgreSQL) database. Touted as the most advanced open source database, it is one of two database management systems [supported by](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/database.md) GitLab, the other being MySQL. + +### Protected Branches + +A [feature](https://docs.gitlab.com/ce/user/project/protected_branches.html) that protects branches from unauthorized pushes, force pushing or deletion. + +### Pull + +Git command to [synchronize](https://git-scm.com/docs/git-pull) the local repository with the remote repository, by fetching all remote changes and merging them into the local repository. + +### Puppet + +A popular DevOps [automation tool](https://puppet.com/product/how-puppet-works). + +### Push + +Git [command](https://git-scm.com/docs/git-push) to send commits from the local repository to the remote repository. Read about [advanced push rules](https://gitlab.com/help/pages/README.md) in GitLab. + +### RE Read Only + +Permissions to see a file and its contents, but not change it. + +### Rebase + +In addition to the merge, the [rebase](https://git-scm.com/book/en/v2/Git-Branching-Rebasing) is a main way to integrate changes from one branch into another. + +### (Git) Repository + +A directory where Git [has been initiatlized](https://git-scm.com/book/en/v2/Git-Basics-Getting-a-Git-Repository) to start version controlling your files. The history of your work is stored here. A remote repository is not on your machine, but usually online (like on GitLab.com, for instance). The main remote repository is usually called "Origin." + +### Requirements management + +Gives your distributed teams a single shared repository to collaborate and share requirements, understand their relationship to tests, and evaluate linked defects. It includes multiple, preconfigured requirement types. + +### Revision Control + +Also known as version control or source control, this is the management of changes to documents, computer programs, large web sites, and other collections of information. Changes are usually identified by a number or letter code, termed the "revision number," "revision level," or simply "revision." + +### RocketChat + +An open source chat application for teams, RocketChat is very similar to Slack but it is also open-source. + +### Route Table + +A route table contains rules (called routes) that determine where network traffic is directed. Each [subnet in a VPC](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html) must be associated with a route table. + +### Runners + +Actual build machines/containers that [run and execute tests](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner) you have specified to be run on GitLab CI. + +### Sidekiq + +The background job processor GitLab [uses](https://docs.gitlab.com/ce/administration/troubleshooting/sidekiq.html) to asynchronously run tasks. + +### Software as a service (SaaS) + +Software that is hosted centrally and accessed on-demand (i.e. whenever you want to). This applies to GitLab.com. + +### Software Configuration Management (SCM) + +This term is often used by people when they mean "Version Control." + +## Scrum + +An Agile [framework](https://www.scrum.org/Resources/What-is-Scrum) designed to typically help complete complex software projects. It's made up of several parts: product requirements backlog, sprint planning, sprint (development), sprint review, and retrospec (analyzing the sprint). The goal is to end up with potentially shippable products. + +### Scrum Board + +The board used to track the status and progress of each of the sprint backlog items. + +### Shell + +Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git]() and make changes to GitLab projects in your shell. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. + +### Single-tenant + +The tenant purchases their own copy of the software and the software can be customized to meet the specific and needs of that customer. [GitHost.io](https://about.gitlab.com/handbook/positioning-faq/) is our provider of single-tenant 'managed cloud' GitLab instances. + +### Slack + +Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](https://docs.gitlab.com/ce/project_services/slack.html) to trigger push, issue, and merge request events among others. + +### Slave Servers + +Also known as secondary servers, these help to spread the load over multiple machines. They also provide backups when the master/primary server crashes. + +### Source Code + +Program code as typed by a computer programmer (i.e. it has not yet been compiled/translated by the computer to machine language). + +### SSH Key + +A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) + +### Single Sign On (SSO) + +An authentication process that allows you enter one username and password to access multiple applications. + +### Staging Area + +[Staging occurs](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) before the commit process in git. The staging area is a file, generally contained in your Git directory, that stores information about what will go into your next commit. It’s sometimes referred to as the “index."" + +### Standard Subscription + +Our mid range EE subscription that includes 24/7 support and support for High Availability [Standard Subscription](https://about.gitlab.com/pricing/). + +### Stash + +Atlassian's Git on-premise solution. Think of it as Atlassian's GitLab EE, now known as BitBucket Server. + +### Static Site Generators (SSGs) + +A [software](https://wiki.python.org/moin/StaticSiteGenerator) that takes some text and templates as input and produces html files on the output. + +### Subversion + +Non-proprietary, centralized version control system. + +### Sudo + +A program that allows you to perform superuser/administrator actions on Unix Operating Systems (e.g., Linux, OS X.) It actually stands for 'superuser do.' + +### Subversion (SVN) + +An open source version control system. Read about [migrating from SVN](https://docs.gitlab.com/ce/workflow/importing/migrating_from_svn.html) to GitLab using SubGit. + +### Tag + +[Represents](https://docs.gitlab.com/ce/api/tags.html) a version of a particular branch at a moment in time. + +### Tool Stack + +The set of tools used in a process to achieve a common outcome (e.g. set of tools used in Application Lifecycle Management). + +### Trac + +An open source project management and bug tracking web [application](https://trac.edgewall.org/). + +### Untracked files + +New files that Git has not [been told](https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository) to track previously. + +### User + +Anyone interacting with the software. + +### Version Control Software (VCS) + +Version control is a system that records changes to a file or set of files over time so that you can recall specific versions later. VCS [has evolved](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit#slide=id.gd69537a19_0_32) from local version control systems, to centralized version control systems, to the present distributed version control systems like Git, Mercurial, Bazaar, and Darcs. + +### Virtual Private Cloud (VPC) + +An on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [setup High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). + +### Virtual private server (VPS) + +A [virtual machine](https://en.wikipedia.org/wiki/Virtual_private_server) sold as a service by an Internet hosting service. A VPS runs its own copy of an operating system, and customers have superuser-level access to that operating system instance, so they can install almost any software that runs on that OS. + +### VM Instance + +In object-oriented programming, an [instance](http://stackoverflow.com/questions/20461907/what-is-meaning-of-instance-in-programming) is a specific realization of any object. An object may be varied in a number of ways. Each realized variation of that object is an instance. Therefore, a VM instance is an instance of a virtual machine, which is an emulation of a computer system. + +### Waterfall + +A [model](http://www.umsl.edu/~hugheyd/is6840/waterfall.html) of building software that involves collecting all requirements from the customer, then building and refining all the requirements and finally delivering the complete software to the customer that meets all the requirements they specified. + +### Webhooks + +A way for for an app to [provide](https://docs.gitlab.com/ce/web_hooks/web_hooks.html) other applications with real-time information (e.g., send a message to a slack channel when a commit is pushed.) Read about setting up [custom git hooks](https://gitlab.com/help/administration/custom_hooks.md) for when webhooks are insufficient. + +### Wiki + +A [website/system](http://www.wiki.com/) that allows for collaborative editing of its content by the users. In programming, wikis usually contain documentation of how to use the software. + +### Working Tree + +[Consists of files](http://stackoverflow.com/questions/3689838/difference-between-head-working-tree-index-in-git) that you are currently working on. + +### YAML + +A human-readable data serialization [language](http://www.yaml.org/about.html) that takes concepts from programming languages such as C, Perl, and Python, and ideas from XML and the data format of electronic mail. + diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md new file mode 100644 index 00000000000..088f1cd7290 --- /dev/null +++ b/doc/university/high-availability/aws/README.md @@ -0,0 +1,387 @@ + +# High Availability on AWS + +GitLab on AWS can leverage many of the services that are already +configurable with High Availability. These services have a lot of +flexibility and are able to adopt to most companies, best of all is the +ability to automate both vertical and horizontal scaling. + +In this article we'll go through a basic HA setup where we'll start by +configuring our Virtual Private Cloud and subnets to later integrate +services such as RDS for our database server and ElastiCache as a Redis +cluster to finally manage them within an auto scaling group with custom +scaling policies. + +*** + +## Where to Start + +Login to your AWS account through the `My Account` dropdown on +`https://aws.amazon.com` or through the URI assigned to your team such as +`https://myteam.signin.aws.amazon.com/console/`. You'll start on the +Amazon Web Services console from where we can choose all of the services +we'll be using to configure our cloud infrastructure. + +*** + +## Network + +We'll start by creating a VPC for our GitLab cloud infrastructure, then +we can create subnets to have public and private instances in at least +two AZs. Public subnets will require a Route Table keep an associated +Internet Gateway. + +### VPC + +Start by looking for the VPC option on the web console. Now create a new +VPC. We can use `10.0.0.0/16` for the CIDR block and leave tenancy as +default if we don't require dedicated hardware. + + + +If you're setting up the Elastic File System service then select the VPC +and from the Actions dropdown choose Edit DNS Hostnames and select Yes. + +### Subnet + +Now let's create some subnets in different Availability Zones. Make sure +that each subnet is associated the the VPC we just created, that it has +a distinct VPC and lastly that CIDR blocks don't overlap. This will also +allow us to enable multi AZ for redundancy. + +We will create private and public subnets to match load balancers and +RDS instances as well. + + + +The subnets are listed with their name, AZ and CIDR block: + +* gitlab-public-10.0.0.0 - us-west-2a - 10.0.0.0 +* gitlab-private-10.0.1.0 - us-west-2a - 10.0.1.0 +* gitlab-public-10.0.2.0 - us-west-2b - 10.0.2.0 +* gitlab-private-10.0.3.0 - us-west-2b - 10.0.3.0 + +### Route Table + +Up to now all our subnets are private. We need to create a Route Table +to associate an Internet Gateway. On the same VPC dashboard choose +Route Tables on the left column and give it a name and associate it to +our newly created VPC. + + + + +### Internet Gateway + +Now still on the same dashboard head over to Internet Gateways and +create a new one. After its created pres on the `Attach to VPC` button and +select our VPC. + + + +### Configure Subnets + +Go back to the Router Tables screen and select the newly created one, +press the Routes tab on the bottom section and edit it. We need to add a +new target which will be our Internet Gateway and have it receive +traffic from any destination. + + + +Before leaving this screen select the next tab to the rgiht which is +Subnet Associations and add our public subnets. If you followed our +naming convention they should be easy to find. + +*** + +## Database with RDS + +For our database server we will use Amazon RDS which offers Multi AZ +for redundancy. Lets start by creating a subnet group and then we'll +create the actual RDS instance. + +### Subnet Group + +From the RDS dashboard select Subnet Groups. Lets select our VPC from +the VPC ID dropdown and at the bottom we can add our private subnets. + + + +### RDS + +Select the RDS service from the Database section and create a new +PostgreSQL instance. After choosing between a Production or +Development instance we'll start with the actual configuration. On the +image bellow we have the settings for this article but note the +following two options which are of particular interest for HA: + +1. Multi-AZ-Deployment is recommended as redundancy. Read more at +[High Availability (Multi-AZ)](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.html) +1. While we chose a General Purpose (SSD) for this article a Provisioned +IOPS (SSD) is best suited for HA. Read more about it at +[Storage for Amazon RDS](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html) + + + +The rest of the setting on this page request a DB identifier, username +and a master password. We've chosen to use `gitlab-ha`, `gitlab` and a +very secure password respectively. Keep these in hand for later. + + + +Make sure to choose our gitlab VPC, our subnet group, not have it public, +and to leave it to create a new security group. The only additional +change which will be helpful is the database name for which we can use +`gitlabhq_production`. + +*** + +## ElastiCache + +EC is an in-memory hosted caching solution. Redis maintains its own +persistance and is used for certain types of application. + +Let's choose the ElastiCache service in the Database section from our +AWS console. Now lets create a cache subnet group which will be very +similar to the RDS subnet group. Make sure to select our VPC and its +private subnets. + + + +Now press the Launch a Cache Cluster and choose Redis for our +DB engine. You'll be able to configure details such as replication, +Multi AZ and node types. The second section will allow us to choose our +subnet and security group and + + + + + +*** + +## Elastic File System + +This new AWS offering allows us to create a file system accessible by
+EC2 instances within a VPC. Choose our VPC and the subnets will be +
automatically configured assuming we don't need to set explicit IPs. +The
next section allows us to add tags and choose between General +Purpose or
Max I/O which is a good option when being accessed by a +large number of
EC2 instances. + +
 + +To actually mount and install the NFS client we'll use the User Data +section when adding our Launch Configuration. + +*** + +## Initiate AMI + +We are going to launch an EC2 instance and bake an image so that we can +later use it for auto scaling. We'll also take this opportunity to add an +extension to our RDS through this temporary EC2 instance. + +### EC2 Instance + +Look for the EC2 option and choose to create an instance. We'll need at +least a t2.medium type and for this article we'll choose an Ubuntu 14.04 +HVM 64-bit. In the Configure Instance section choose our GitLab VPC and +a public subnet. I'd choose at least 10GB of storage. + +In the security group we'll create a new one considering that we need to +SSH into the instance and also try it out through http. So let's add the +http traffic from anywhere and name it something such as +`gitlab-ec2-security-group`. + +While we wait for it to launch we can allocate an Elastic IP and +associate it with our new EC2 instance. + +### RDS and Redis Security Group + +After the instance is being created we will navigate to our EC2 security +groups and add a small change for our EC2 instances to be able to +connect to RDS. First copy the security group name we just defined, +namely `gitlab-ec2-security-group`, and edit select the RDS security +group and edit the inbound rules. Choose the rule type to be PostgreSQL +and paste the name under source. + + + +Similar to the above we'll jump to the `gitlab-ec2-security-group` group +and add a custom TCP rule for port 6379 accessible within itself. + +### Install GitLab + +To connect through SSH you will need to have the `pem` file which you +chose available and with the correct permissions such as `400`. + +After accessing your server don't forget to update and upgrade your +packages. + + sudo apt-get update && sudo apt-get upgrade -y + +Then follow installation instructions from +[GitLab](https://about.gitlab.com/downloads-ee/#ubuntu1404), but before +running reconfigure we need to make sure all our services are tied down +so just leave the reconfigure command until after we edit our gitlab.rb +file. + + +### Extension for PostgreSQL + +Connect to your new RDS instance to verify access and to install +a required extension. We can find the host or endpoint by selecting the +instance and we just created and after the details drop down we'll find +it labeled as 'Endpoint'; do remember not to include the colon and port +number. + + sudo /opt/gitlab/embedded/bin/psql -U gitlab -h <rds-endpoint> -d gitlabhq_production + psql (9.4.7) + Type "help" for help. + + gitlab=# CREATE EXTENSION pg_trgm; + gitlab=# \q + +### Configure GitLab + +While connected to your server edit the `gitlab.rb` file at `/etc/gitlab/gitlab.rb` +find the `external_url 'http://gitlab.example.com'` option and change it +to the domain you will be using or the public IP address of the current +instance to test the configuration. + +For a more detailed description about configuring GitLab read [Configuring GitLab for HA](http://docs.gitlab.com/ee/administration/high_availability/gitlab.html) + +Now look for the GitLab database settings and uncomment as necessary. In +our current case we'll specify the adapter, encoding, host, db name, +username, and password. + + gitlab_rails['db_adapter'] = "postgresql" + gitlab_rails['db_encoding'] = "unicode" + gitlab_rails['db_database'] = "gitlabhq_production" + gitlab_rails['db_username'] = "gitlab" + gitlab_rails['db_password'] = "mypassword" + gitlab_rails['db_host'] = "<rds-endpoint>" + +Next we only need to configure the Redis section by adding the host and +uncommenting the port. + + + +The last configuration step is to [change the default file locations ](http://docs.gitlab.com/ee/administration/high_availability/nfs.html) +to make the EFS integration easier to manage. + + gitlab_rails['redis_host'] = "<redis-endpoint>" + gitlab_rails['redis_port'] = 6379 + +Finally run reconfigure, you might find it useful to run a check and +a service status to make sure everything has been setup correctly. + + sudo gitlab-ctl reconfigure + sudo gitlab-rake gitlab:check + sudo gitlab-ctl status + +If everything looks good copy the Elastic IP over to your browser and +test the instance manually. + +### AMI + +After you finish testing your EC2 instance go back to its dashboard and +while the instance is selected press on the Actions dropdown to choose +Image -> Create an Image. Give it a name and description and confirm. + +*** + +## Load Balancer + +On the same dashboard look for Load Balancer on the left column and press +the Create button. Choose a classic Load Balancer, our gitlab VPC, not +internal and make sure its listening for HTTP and HTTPS on port 80. + +Here is a tricky part though, when adding subnets we need to associate +public subnets instead of the private ones where our instances will +actually live. + +On the secruity group section let's create a new one named +`gitlab-loadbalancer-sec-group` and allow both HTTP ad HTTPS traffic +from anywhere. + +The Load Balancer Health will allow us to indicate where to ping and what +makes up a healthy or unhealthy instance. + +We won't add the instance on the next session because we'll destroy it +momentarily as we'll be using the image we where creating. We will keep +the Enable Cross-Zone and Enable Connection Draining active. + +After we finish creating the Load Balancer we can re visit our Security +Groups to improve access only through the ELB and any other requirement +you might have. + +*** + +## Auto Scaling Group + +Our AMI should be done by now so we can start working on our Auto +Scaling Group. + +This option is also available through the EC2 dashboard on the left +sidebar. Press on the create button. Select the new image on My AMIs and +give it a `t2.medium` size. To be able to use Elastic File System we need +to add a script to mount EFS automatically at launch. We'll do this at +the Advanced Details section where we have a [User Data](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html) +text area that allows us to add a lot of custom configurations which +allows you to add a custom script for when launching an instance. Let's +add the following script to the User Data section: + + + #cloud-config + package_upgrade: true + packages: + - nfs-common + runcmd: + - mkdir -p /gitlab-data + - chown ec2-user:ec2-user /gitlab-data + - echo "$(curl --silent http://169.254.169.254/latest/meta-data/placement/availability-zone).file-system-id.aws-region.amazonaws.com:/ /gitlab-data nfs defaults,vers=4.1 0 0" >> /etc/fstab + - mount -a -t nfs + - sudo gitlab-ctl reconfigure + +On the security group section we can chosse our existing +`gitlab-ec2-security-group` group which has already been tested. + +After this is launched we are able to start creating our Auto Scaling +Group. Start by giving it a name and assinging it our VPC and private +subnets. We also want to always start with two instances and if you +scroll down to Advanced Details we can choose to receive traffic from ELBs. +Lets enable that option and select our ELB. We also want to use the ELB's +health check. + + + +### Policies + +This is the really great part of Auto Scaling, we get to choose when AWS +launches new instances and when it removes them. For this group we'll +scale between 2 and 4 instances where one instance will be added if CPU +utilization is greater than 60% and one instance is removed if it falls +to less than 45%. Here are the complete policies: + + + +You'll notice that after we save this AWS starts launching our two +instances in different AZs and without a public IP which is exactly what +we where aiming for. + +*** + +## Final Thoughts + +After you're done with the policies section have some fun trying to break +instances. You should be able to see how the Auto Scaling Group and the +EC2 screen start bringing them up again. + +High Availability is a very big area, we went mostly through scaling and +some redundancy options but it might also imply Geographic replication. +There is a lot of ground yet to cover so have a read through these other +resources and feel free to open an issue to request additional material. + + * [GitLab High Availability](http://docs.gitlab.com/ce/administration/high_availability/README.html#sts=High Availability) + * [GitLab Geo](http://docs.gitlab.com/ee/gitlab-geo/README.html) diff --git a/doc/university/high-availability/aws/img/auto-scaling-det.png b/doc/university/high-availability/aws/img/auto-scaling-det.png Binary files differnew file mode 100644 index 00000000000..e9b65529495 --- /dev/null +++ b/doc/university/high-availability/aws/img/auto-scaling-det.png diff --git a/doc/university/high-availability/aws/img/db-subnet-group.png b/doc/university/high-availability/aws/img/db-subnet-group.png Binary files differnew file mode 100644 index 00000000000..0768aa73c45 --- /dev/null +++ b/doc/university/high-availability/aws/img/db-subnet-group.png diff --git a/doc/university/high-availability/aws/img/ec-subnet.png b/doc/university/high-availability/aws/img/ec-subnet.png Binary files differnew file mode 100644 index 00000000000..f41d78b271d --- /dev/null +++ b/doc/university/high-availability/aws/img/ec-subnet.png diff --git a/doc/university/high-availability/aws/img/elastic-file-system.png b/doc/university/high-availability/aws/img/elastic-file-system.png Binary files differnew file mode 100644 index 00000000000..7de866d1e89 --- /dev/null +++ b/doc/university/high-availability/aws/img/elastic-file-system.png diff --git a/doc/university/high-availability/aws/img/ig-rt.png b/doc/university/high-availability/aws/img/ig-rt.png Binary files differnew file mode 100644 index 00000000000..93bb0c2ae02 --- /dev/null +++ b/doc/university/high-availability/aws/img/ig-rt.png diff --git a/doc/university/high-availability/aws/img/ig.png b/doc/university/high-availability/aws/img/ig.png Binary files differnew file mode 100644 index 00000000000..cc50456370f --- /dev/null +++ b/doc/university/high-availability/aws/img/ig.png diff --git a/doc/university/high-availability/aws/img/instance_specs.png b/doc/university/high-availability/aws/img/instance_specs.png Binary files differnew file mode 100644 index 00000000000..ef31dc41dae --- /dev/null +++ b/doc/university/high-availability/aws/img/instance_specs.png diff --git a/doc/university/high-availability/aws/img/new_vpc.png b/doc/university/high-availability/aws/img/new_vpc.png Binary files differnew file mode 100644 index 00000000000..4aac6af7c7a --- /dev/null +++ b/doc/university/high-availability/aws/img/new_vpc.png diff --git a/doc/university/high-availability/aws/img/policies.png b/doc/university/high-availability/aws/img/policies.png Binary files differnew file mode 100644 index 00000000000..8c58117e4fa --- /dev/null +++ b/doc/university/high-availability/aws/img/policies.png diff --git a/doc/university/high-availability/aws/img/rds-net-opt.png b/doc/university/high-availability/aws/img/rds-net-opt.png Binary files differnew file mode 100644 index 00000000000..bc204de2474 --- /dev/null +++ b/doc/university/high-availability/aws/img/rds-net-opt.png diff --git a/doc/university/high-availability/aws/img/rds-sec-group.png b/doc/university/high-availability/aws/img/rds-sec-group.png Binary files differnew file mode 100644 index 00000000000..8864dc3e463 --- /dev/null +++ b/doc/university/high-availability/aws/img/rds-sec-group.png diff --git a/doc/university/high-availability/aws/img/redis-cluster-det.png b/doc/university/high-availability/aws/img/redis-cluster-det.png Binary files differnew file mode 100644 index 00000000000..9e9a81283c5 --- /dev/null +++ b/doc/university/high-availability/aws/img/redis-cluster-det.png diff --git a/doc/university/high-availability/aws/img/redis-net.png b/doc/university/high-availability/aws/img/redis-net.png Binary files differnew file mode 100644 index 00000000000..037bd6d6897 --- /dev/null +++ b/doc/university/high-availability/aws/img/redis-net.png diff --git a/doc/university/high-availability/aws/img/route_table.png b/doc/university/high-availability/aws/img/route_table.png Binary files differnew file mode 100644 index 00000000000..1dea322474d --- /dev/null +++ b/doc/university/high-availability/aws/img/route_table.png diff --git a/doc/university/high-availability/aws/img/subnet.png b/doc/university/high-availability/aws/img/subnet.png Binary files differnew file mode 100644 index 00000000000..dbc71201992 --- /dev/null +++ b/doc/university/high-availability/aws/img/subnet.png diff --git a/doc/university/process/README.md b/doc/university/process/README.md new file mode 100644 index 00000000000..7ff53c2cc3f --- /dev/null +++ b/doc/university/process/README.md @@ -0,0 +1,30 @@ +--- +title: University | Process +--- + +## Suggesting improvements + +If you would like to teach a class or participate or help in any way please +submit a merge request and assign it to [Job](https://gitlab.com/u/JobV). + +If you have suggestions for additional courses you would like to see, +please submit a merge request to add an upcoming class, assign to +[Chad](https://gitlab.com/u/chadmalchow) and /cc [Job](https://gitlab.com/u/JobV). + +## Adding classes + +1. All training materials of any kind should be added to [GitLab CE](https://gitlab.com/gitlab-org/gitlab-ce/) + to ensure they are available to a broad audience (don't use any other repo or + storage for training materials). +1. Don't make materials that are needlessly specific to one group of people, try + to keep the wording broad and inclusive (don't make things for only GitLab Inc. + people, only interns, only customers, etc.). +1. To allow people to contribute all content should be in git. +1. The content can go in a subdirectory under `/doc/university/`. +1. To make, view or modify the slides of the classes use [Deckset](http://www.decksetapp.com/) + or [RevealJS](http://lab.hakim.se/reveal-js/). Do not use Powerpoint or Google + Slides since this prevents everyone from contributing. +1. Please upload any video recordings to our Youtube channel. We prefer them to + be public, if needed they can be unlisted but if so they should be linked from + this page. +1. Please create a merge request and assign to [SeanPackham](https://gitlab.com/u/SeanPackham). diff --git a/doc/university/support/README.md b/doc/university/support/README.md new file mode 100644 index 00000000000..da991e56370 --- /dev/null +++ b/doc/university/support/README.md @@ -0,0 +1,188 @@ + +## Support Boot Camp + +**Goal:** Prepare new Service Engineers at GitLab + +For each stage there are learning goals and content to support the learning of the engineer. +The goal of this boot camp is to have every Service Engineer prepared to help our customers +with whatever needs they might have and to also assist our awesome community with their +questions. + +Always start with the [University Overview](../README.md) and then work +your way here for more advanced and specific training. Once you feel comfortable +with the topics of the current stage, move to the next. + +### Stage 1 + +Follow the topics on the [University Overview](../README.md), concentrate on it +during your first Stage, but also: + +- Perform the [first steps](https://about.gitlab.com/handbook/support/onboarding/#first-steps) of + the on-boarding process for new Service Engineers + +#### Goals + +Aim to have a good overview of the Product and main features, Git and the Company + +### Stage 2 + +Continue to look over remaining portions of the [University Overview](../README.md) and continue on to these topics: + +#### Set up your development machine + +Get your development machine ready to familiarize yourself with the codebase, the components, and to be prepared to reproduce issues that our users encounter + +- Install the [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) + - [Setup OpenLDAP as part of this](https://gitlab.com/gitlab-org/gitlab-development-kit#openldap) + +#### Become comfortable with the Installation processes that we support + +It's important to understand how to install GitLab in the same way that our users do. Try installing different versions and upgrading and downgrading between them. Installation from source will give you a greater understanding of the components that we employ and how everything fits together. + +Sometimes we need to upgrade customers from old versions of GitLab to latest, so it's good to get some experience of doing that now. + +- [Installation Methods](https://about.gitlab.com/installation/): + - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/) + - [Docker](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/docker) + - [Source](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md) +- Get yourself a Digital Ocean droplet, where you can install and maintain your own instance of GitLab + - Ask in #infrastructure about this + - Populate with some test data + - Keep this up-to-date as patch and version releases become available, just like our customers would +- Try out the following installation path + - [Install GitLab 4.2 from source](https://gitlab.com/gitlab-org/gitlab-ce/blob/d67117b5a185cfb15a1d7e749588ff981ffbf779/doc/install/installation.md) + - External MySQL database + - External NGINX + - Create some test data + - Populated Repos + - Users + - Groups + - Projects + - [Backup using our Backup rake task](http://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) + - [Upgrade to 5.0 source using our Upgrade documentation](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/4.2-to-5.0.md) + - [Upgrade to 5.1 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.0-to-5.1.md) + - [Upgrade to 6.0 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.1-to-6.0.md) + - [Upgrade to 7.14 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/6.x-or-7.x-to-7.14.md) + - [Backup using our Backup rake task](http://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) + - [Perform the MySQL to PostgreSQL migration to convert your backup](http://docs.gitlab.com/ce/update/mysql_to_postgresql.html#converting-a-gitlab-backup-file-from-mysql-to-postgres) + - [Upgrade to Omnibus 7.14](http://doc.gitlab.com/omnibus/update/README.html#upgrading-from-a-non-omnibus-installation-to-an-omnibus-installation) + - [Restore backup using our Restore rake task](http://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-a-previously-created-backup) + - [Upgrade to latest EE](https://about.gitlab.com/downloads-ee) + - (GitLab inc. only) Acquire and apply a license for the Enterprise Edition product, ask in #support +- Perform a downgrade from [EE to CE](http://doc.gitlab.com/ee/downgrade_ee_to_ce/README.html) + +#### Start to learn about some of the integrations that we support + +Our integrations add great value to GitLab. User questions often relate to integrating GitLab with existing external services and the configuration involved + +- Learn about our Integrations (specially, not only): + - [LDAP](http://doc.gitlab.com/ee/integration/ldap.html) + - [JIRA](http://doc.gitlab.com/ee/project_services/jira.html) + - [Jenkins](http://doc.gitlab.com/ee/integration/jenkins.html) + - [SAML](http://doc.gitlab.com/ce/integration/saml.html) + +#### Goals + +- Aim to be comfortable with installation of the GitLab product and configuration of some of the major integrations +- Aim to have an installation available for reproducing customer reports + +### Stage 3 + +#### Understand the gathering of diagnostics for GitLab instances + +- Learn about the GitLab checks that are available + - [Environment Information and maintenance checks](http://docs.gitlab.com/ce/raketasks/maintenance.html) + - [GitLab check](http://docs.gitlab.com/ce/raketasks/check.html) + - Omnibus commands + - [Status](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#get-service-status) + - [Starting and stopping services](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#starting-and-stopping) + - [Starting a rails console](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#invoking-rake-tasks) + +#### Learn about the Support process + +Zendesk is our Support Centre and our main communication line with our Customers. We communicate with customers through several other channels too + +- Familiarize yourself with ZenDesk + - [UI Overview](https://support.zendesk.com/hc/en-us/articles/203661806-Introduction-to-the-Zendesk-agent-interface) + - [Updating Tickets](https://support.zendesk.com/hc/en-us/articles/212530318-Updating-and-solving-tickets) + - [Working w/ Tickets](https://support.zendesk.com/hc/en-us/articles/203690856-Working-with-tickets) *Read: avoiding agent collision.* +- Dive into our ZenDesk support process by reading how to [handle tickets](https://about.gitlab.com/handbook/support/onboarding/#handling-tickets) +- Start getting real world experience by handling real tickets, all the while gaining further experience with the Product. + - First, learn about our [Support Channels](https://about.gitlab.com/handbook/support/#support-channels) + - Ask other Service Engineers for help, when necessary, and to review your responses + - Start with [StackOverflow](https://about.gitlab.com/handbook/support/#stack-overflowa-namestack-overflowa) and the [GitLab forum](https://about.gitlab.com/handbook/support/#foruma-namegitlab-foruma) + - Here you will find a large variety of queries mainly from our Users who are self hosting GitLab CE + - Understand the questions that are asked and dig in to try to find a solution + - [Proceed on to the GitLab.com Support Forum](https://about.gitlab.com/handbook/support/#gitlabcom-support-trackera-namesupp-foruma) + - Here you will find queries regarding our own GitLab.com + - Helping Users here will give you an understanding of our Admin interface and other tools + - [Proceed on to the Twitter tickets in Zendesk](https://about.gitlab.com/handbook/support/#twitter) + - Here you will gain a great insight into our userbase + - Learn from any complaints and problems and feed them back to the team + - Tweets can range from help needed with GitLab installations, the API and just general queries + - [Proceed on to Regular email Support tickets](https://about.gitlab.com/handbook/support/#regular-zendesk-tickets-a-nameregulara) + - Here you will find tickets from our GitLab EE Customers and GitLab CE Users + - Tickets here are extremely varied and often very technical + - You should be prepared for these tickets, given the knowledge gained from previous tiers and your training +- Check out your colleagues' responses + - Hop on to the #support-live-feed channel in Slack and see the tickets as they come in and are updated + - Read through old tickets that your colleagues have worked on +- Start arranging to pair on calls with other Service Engineers. Aim to cover a few of each type of call + - [Learn about Cisco WebEx](https://about.gitlab.com/handbook/support/onboarding/#webexa-namewebexa) + - Training calls + - Information gathering calls + - It's good to find out how new and prospective customers are going to be using the product and how they will set up their infrastructure + - Diagnosis calls + - When email isn't enough we may need to hop on a call and do some debugging along side the customer + - These paired calls are a great learning experience + - Upgrade calls + - Emergency calls + +#### Learn about the Escalation process for tickets + +Some tickets need specific knowledge or a deep understanding of a particular component and will need to be escalated to a Senior Service Engineer or Developer + +- Read about [Escalation](https://about.gitlab.com/handbook/support/onboarding/#create-issuesa-namecreate-issuea) +- Find the macros in Zendesk for ticket escalations +- Take a look at the [GitLab.com Team page](https://about.gitlab.com/team/) to find the resident experts in their fields + +#### Learn about raising issues and fielding feature proposals + +- Understand what's in the pipeline and proposed features at GitLab: [Direction Page](https://about.gitlab.com/direction/) +- Practice searching issues and filtering using [labels](https://gitlab.com/gitlab-org/gitlab-ce/labels) to find existing feature proposals and bugs +- If raising a new issue always provide a relevant label and a link to the relevant ticket in Zendesk +- Add [customer labels](https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name%5B%5D=customer) for those issues relevant to our subscribers +- Take a look at the [existing issue templates](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#issue-tracker) to see what is expected +- Raise issues for bugs in a manner that would make the issue easily reproducible. A Developer or a contributor may work on your issue + +#### Goals + +- Aim to have a good understanding of the problems that customers are facing +- Aim to have gained experience in scheduling and participating in calls with customers +- Aim to have a good understanding of ticket flow through Zendesk and how to interat with our various channels + +### Stage 4 + +#### Advanced GitLab topics + +Move on to understanding some of GitLab's more advanced features. You can make use of GitLab.com to understand the features from an end-user perspective and then use your own instance to understand setup and configuration of the feature from an Administrative perspective + +- Set up and try [Git Annex](http://doc.gitlab.com/ee/workflow/git_annex.html) +- Set up and try [Git LFS](http://doc.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html) +- Get to know the [GitLab API](http://doc.gitlab.com/ee/api/README.html), its capabilities and shortcomings +- Learn how to [migrate from SVN to Git](http://doc.gitlab.com/ee/workflow/importing/migrating_from_svn.html) +- Set up [GitLab CI](http://doc.gitlab.com/ee/ci/quick_start/README.html) +- Create your first [GitLab Page](http://doc.gitlab.com/ee/pages/administration.html) +- Get to know the GitLab Codebase by reading through the source code: + - Find the differences between the [EE codebase](https://gitlab.com/gitlab-org/gitlab-ce) + and the [CE codebase](https://gitlab.com/gitlab-org/gitlab-ce) +- Ask as many questions as you can think of on the `#support` chat channel + +#### Get initiated for on-call duty + +- Read over the [public run-books to understand common tasks](https://gitlab.com/gitlab-com/runbooks) +- Create an issue on the internal Organization tracker to schedule time with the DevOps / Production team, so that you learn how to handle GitLab.com going down. Once you are trained for this, you are ready to be added to the on-call rotation. + +#### Goals + +- Aim to become a fully-fledged Service Engineer! diff --git a/doc/university/training/end-user/README.md b/doc/university/training/end-user/README.md new file mode 100644 index 00000000000..03c62a81b10 --- /dev/null +++ b/doc/university/training/end-user/README.md @@ -0,0 +1,420 @@ + +# Training + +This training material is the markdown used to generate training slides +which can be found at [End User Slides](https://gitlab-org.gitlab.io/end-user-training-slides/#/) +through it's [RevealJS](https://gitlab.com/gitlab-org/end-user-training-slides) +project. + +--- + +## Git Intro + +--- + +### What is a Version Control System (VCS) + +- Records changes to a file +- Maintains history of changes +- Disaster Recovery +- Types of VCS: Local, Centralized and Distributed + +--- + +### Short Story of Git + +- 1991-2002: The Linux kernel was being maintaned by sharing archived files + and patches. +- 2002: The Linux kernel project began using a DVCS called BitKeeper +- 2005: BitKeeper revoked the free-of-charge status and Git was created + +--- + +### What is Git + +- Distributed Version Control System +- Great branching model that adapts well to most workflows +- Fast and reliable +- Keeps a complete history +- Disaster recovery friendly +- Open Source + +--- + +### Getting Help + +- Use the tools at your disposal when you get stuck. + - Use `git help <command>` command + - Use Google (i.e. StackOverflow, Google groups) + - Read documentation at https://git-scm.com + +--- + +## Git Setup +Workshop Time! + +--- + +### Setup + +- Windows: Install 'Git for Windows' + - https://git-for-windows.github.io +- Mac: Type `git` in the Terminal application. + - If it's not installed, it will prompt you to install it. +- Linux + - Debian: `sudo apt-get install git-all` + - Red Hat `sudo yum install git-all` + +--- + +### Configure + +- One-time configuration of the Git client: + +```bash +git config --global user.name "Your Name" +git config --global user.email you@example.com +``` + +- If you don't use the global flag you can setup a different author for + each project +- Check settings with: + +```bash +git config --global --list +``` +- You might want or be required to use an SSH key. + - Instructions: [SSH](http://doc.gitlab.com/ce/ssh/README.html) + +--- + +### Workspace + +- Choose a directory on you machine easy to access +- Create a workspace or development directory +- This is where we'll be working and adding content + +--- + +```bash +mkdir ~/development +cd ~/development + +-or- + +mkdir ~/workspace +cd ~/workspace +``` + +--- + +## Git Basics + +--- + +### Git Workflow + +- Untracked files + - New files that Git has not been told to track previously. +- Working area (Workspace) + - Files that have been modified but are not committed. +- Staging area (Index) + - Modified files that have been marked to go in the next commit. +- Upstream + - Hosted repository on a shared server + +--- + +### GitLab + +- GitLab is an application to code, test and deploy. +- Provides repository management with access controls, code reviews, + issue tracking, Merge Requests, and other features. +- The hosted version of GitLab is gitlab.com + +--- + +### New Project + +- Sign in into your gitlab.com account +- Create a project +- Choose to import from 'Any Repo by URL' and use https://gitlab.com/gitlab-org/training-examples.git +- On your machine clone the `training-examples` project + +--- + +### Git and GitLab basics + +1. Edit `edit_this_file.rb` in `training-examples` +2. See it listed as a changed file (working area) +3. View the differences +4. Stage the file +5. Commit +6. Push the commit to the remote +7. View the git log + +--- + +```shell +# Edit `edit_this_file.rb` +git status +git diff +git add <file> +git commit -m 'My change' +git push origin master +git log +``` + +--- + +### Feature Branching + +1. Create a new feature branch called `squash_some_bugs` +2. Edit `bugs.rb` and remove all the bugs. +3. Commit +4. Push + +--- + +```shell +git checkout -b squash_some_bugs +# Edit `bugs.rb` +git status +git add bugs.rb +git commit -m 'Fix some buggy code' +git push origin squash_some_bugs +``` + +--- + +## Merge Request + +--- + +### Merge requests + +- When you want feedback create a merge request +- Target is the ‘default’ branch (usually master) +- Assign or mention the person you would like to review +- Add `WIP` to the title if it's a work in progress +- When accepting, always delete the branch +- Anyone can comment, not just the assignee +- Push corrections to the same branch + + +--- + +### Merge request example + +- Create your first merge request + - Use the blue button in the activity feed + - View the diff (changes) and leave a comment + - Push a new commit to the same branch + - Review the changes again and notice the update + +--- + +### Feedback and Collaboration + +- Merge requests are a time for feedback and collaboration +- Giving feedback is hard +- Be as kind as possible +- Receiving feedback is hard +- Be as receptive as possible +- Feedback is about the best code, not the person. You are not your code +- Feedback and Collaboration + +--- + +### Feedback and Collaboration + +- Review the Thoughtbot code-review guide for suggestions to follow when reviewing merge requests:[Thoughtbot](https://github.com/thoughtbot/guides/tree/master/code-review) +- See GitLab merge requests for examples: [Merge Requests](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests) + +--- + +## Merge Conflicts + +--- + +### Merge Conflicts +* Happen often +* Learning to fix conflicts is hard +* Practice makes perfect +* Force push after fixing conflicts. Be careful! + +--- + +### Example Plan +1. Checkout a new branch and edit conflicts.rb. Add 'Line4' and 'Line5'. +2. Commit and push +3. Checkout master and edit conflicts.rb. Add 'Line6' and 'Line7' below 'Line3'. +4. Commit and push to master +5. Create a merge request and watch it fail +6. Rebase our new branch with master +7. Fix conflicts on the conflicts.rb file. +8. Stage the file and continue rebasing +9. Force push the changes +10. Finally continue with the Merge Request + +--- + +### Example 1/2 + + git checkout -b conflicts_branch + + # vi conflicts.rb + # Add 'Line4' and 'Line5' + + git commit -am "add line4 and line5" + git push origin conflicts_branch + + git checkout master + + # vi conflicts.rb + # Add 'Line6' and 'Line7' + git commit -am "add line6 and line7" + git push origin master + +--- + +### Example 2/2 + +Create a merge request on the GitLab web UI. You'll see a conflict warning. + + git checkout conflicts_branch + git fetch + git rebase master + + # Fix conflicts by editing the files. + + git add conflicts.rb + # No need to commit this file + + git rebase --continue + + # Remember that we have rewritten our commit history so we + # need to force push so that our remote branch is restructured + git push origin conflicts_branch -f + +--- + +### Notes + +* When to use `git merge` and when to use `git rebase` +* Rebase when updating your branch with master +* Merge when bringing changes from feature to master +* Reference: https://www.atlassian.com/git/tutorials/merging-vs-rebasing/ + +--- + +## Revert and Unstage + +--- + +### Unstage + +To remove files from stage use reset HEAD. Where HEAD is the last commit of the current branch: + + git reset HEAD <file> + +This will unstage the file but maintain the modifications. To revert the file back to the state it was in before the changes we can use: + + git checkout -- <file> + +To remove a file from disk and repo use 'git rm' and to rm a dir use the '-r' flag: + + git rm '*.txt' + git rm -r <dirname> + +If we want to remove a file from the repository but keep it on disk, say we forgot to add it to our .gitignore file then use `--cache`: + + git rm <filename> --cache + +--- + +### Undo Commits + +Undo last commit putting everything back into the staging area: + + git reset --soft HEAD^ + +Add files and change message with: + + git commit --amend -m "New Message" + +Undo last and remove changes + + git reset --hard HEAD^ + +Same as last one but for two commits back: + + git reset --hard HEAD^^ + +Don't reset after pushing + +--- + +### Reset Workflow + +1. Edit file again 'edit_this_file.rb' +2. Check status +3. Add and commit with wrong message +4. Check log +5. Amend commit +6. Check log +7. Soft reset +8. Check log +9. Pull for updates +10. Push changes + +---- + + # Change file edit_this_file.rb + git status + git commit -am "kjkfjkg" + git log + git commit --amend -m "New comment added" + git log + git reset --soft HEAD^ + git log + git pull origin master + git push origin master + +--- + +### Note + +git revert vs git reset +Reset removes the commit while revert removes the changes but leaves the commit +Revert is safer considering we can revert a revert + + + # Changed file + git commit -am "bug introduced" + git revert HEAD + # New commit created reverting changes + # Now we want to re apply the reverted commit + git log # take hash from the revert commit + git revert <rev commit hash> + # reverted commit is back (new commit created again) + +--- + +## Questions + +--- + +## Instructor Notes + +--- + +### Version Control + - Local VCS was used with a filesystem or a simple db. + - Centralized VCS such as Subversion includes collaboration but + still is prone to data loss as the main server is the single point of + failure. + - Distributed VCS enables the team to have a complete copy of the project + and work with little dependency to the main server. In case of a main + server failing the project can be recovered by any of the latest copies + from the team diff --git a/doc/update/4.0-to-4.1.md b/doc/update/4.0-to-4.1.md index c163bfd348d..c66c6dd0fd8 100644 --- a/doc/update/4.0-to-4.1.md +++ b/doc/update/4.0-to-4.1.md @@ -42,7 +42,7 @@ sudo -u gitlab -H bundle exec rake db:migrate RAILS_ENV=production sudo mv /etc/init.d/gitlab /etc/init.d/gitlab.old # get new one using sidekiq -sudo curl -L --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlab-recipes/4-1-stable/init.d/gitlab +sudo curl --location --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlab-recipes/4-1-stable/init.d/gitlab sudo chmod +x /etc/init.d/gitlab ``` diff --git a/doc/update/4.2-to-5.0.md b/doc/update/4.2-to-5.0.md index ee6de51c923..7654f4a0131 100644 --- a/doc/update/4.2-to-5.0.md +++ b/doc/update/4.2-to-5.0.md @@ -126,7 +126,7 @@ sudo chmod -R u+rwX /home/git/gitlab/tmp/pids ```bash # init.d sudo rm /etc/init.d/gitlab -sudo curl -L --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlab-recipes/5-0-stable/init.d/gitlab +sudo curl --location --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlab-recipes/5-0-stable/init.d/gitlab sudo chmod +x /etc/init.d/gitlab # unicorn diff --git a/doc/update/5.0-to-5.1.md b/doc/update/5.0-to-5.1.md index f0fddcf83af..c19a819ab5a 100644 --- a/doc/update/5.0-to-5.1.md +++ b/doc/update/5.0-to-5.1.md @@ -63,7 +63,7 @@ sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production ```bash # init.d sudo rm /etc/init.d/gitlab -sudo curl -L --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlab-recipes/5-1-stable/init.d/gitlab +sudo curl --location --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlab-recipes/5-1-stable/init.d/gitlab sudo chmod +x /etc/init.d/gitlab ``` diff --git a/doc/update/5.2-to-5.3.md b/doc/update/5.2-to-5.3.md index c5254f6fb0c..fe8990b6843 100644 --- a/doc/update/5.2-to-5.3.md +++ b/doc/update/5.2-to-5.3.md @@ -67,7 +67,7 @@ sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production ```bash sudo rm /etc/init.d/gitlab -sudo curl -L --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlabhq/5-3-stable/lib/support/init.d/gitlab +sudo curl --location --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlabhq/5-3-stable/lib/support/init.d/gitlab sudo chmod +x /etc/init.d/gitlab ``` diff --git a/doc/update/5.3-to-5.4.md b/doc/update/5.3-to-5.4.md index c4a6146dcda..5f82ad7d444 100644 --- a/doc/update/5.3-to-5.4.md +++ b/doc/update/5.3-to-5.4.md @@ -71,7 +71,7 @@ sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production ```bash sudo rm /etc/init.d/gitlab -sudo curl -L --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlabhq/5-4-stable/lib/support/init.d/gitlab +sudo curl --location --output /etc/init.d/gitlab https://raw.github.com/gitlabhq/gitlabhq/5-4-stable/lib/support/init.d/gitlab sudo chmod +x /etc/init.d/gitlab ``` diff --git a/doc/update/6.9-to-7.0.md b/doc/update/6.9-to-7.0.md index 236430b5951..5352fd52f93 100644 --- a/doc/update/6.9-to-7.0.md +++ b/doc/update/6.9-to-7.0.md @@ -33,7 +33,7 @@ Download and compile Ruby: ```bash mkdir /tmp/ruby && cd /tmp/ruby -curl -L --progress ftp://ftp.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz | tar xz +curl --location --progress ftp://ftp.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz | tar xz cd ruby-2.1.2 ./configure --disable-install-rdoc make diff --git a/doc/update/7.0-to-7.1.md b/doc/update/7.0-to-7.1.md index a4e9be9946e..71f39c44077 100644 --- a/doc/update/7.0-to-7.1.md +++ b/doc/update/7.0-to-7.1.md @@ -33,7 +33,7 @@ Download and compile Ruby: ```bash mkdir /tmp/ruby && cd /tmp/ruby -curl -L --progress ftp://ftp.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz | tar xz +curl --location --progress ftp://ftp.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz | tar xz cd ruby-2.1.2 ./configure --disable-install-rdoc make diff --git a/doc/update/7.14-to-8.0.md b/doc/update/7.14-to-8.0.md index 305017b7048..117e2afaaa0 100644 --- a/doc/update/7.14-to-8.0.md +++ b/doc/update/7.14-to-8.0.md @@ -71,7 +71,7 @@ sudo -u git -H git checkout v2.6.5 First we download Go 1.5 and install it into `/usr/local/go`: ```bash -curl -O --progress https://storage.googleapis.com/golang/go1.5.linux-amd64.tar.gz +curl --remote-name --progress https://storage.googleapis.com/golang/go1.5.linux-amd64.tar.gz echo '5817fa4b2252afdb02e11e8b9dc1d9173ef3bd5a go1.5.linux-amd64.tar.gz' | shasum -c - && \ sudo tar -C /usr/local -xzf go1.5.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md index d57c0d0674d..bfb83cf79b1 100644 --- a/doc/update/8.0-to-8.1.md +++ b/doc/update/8.0-to-8.1.md @@ -99,6 +99,10 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS # Update init.d script sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab ``` + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 7. Update configuration files diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md index 46dfa2232b4..7f36ce00e96 100644 --- a/doc/update/8.1-to-8.2.md +++ b/doc/update/8.1-to-8.2.md @@ -116,6 +116,10 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS # Update init.d script sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab ``` + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 7. Update configuration files diff --git a/doc/update/8.10-to-8.11.md b/doc/update/8.10-to-8.11.md new file mode 100644 index 00000000000..119c5f475e4 --- /dev/null +++ b/doc/update/8.10-to-8.11.md @@ -0,0 +1,197 @@ +# From 8.10 to 8.11 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +We will continue supporting Ruby < 2.3 for the time being but we recommend you +upgrade to Ruby 2.3 if you're running a source installation, as this is the same +version that ships with our Omnibus package. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz +echo 'c39b4001f7acb4e334cb60a0f4df72d434bef711 ruby-2.3.1.tar.gz' | shasum --check - && tar xzf ruby-2.3.1.tar.gz +cd ruby-2.3.1 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-11-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-11-stable-ee +``` + +### 5. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v3.4.0 +``` + +### 6. Update gitlab-workhorse + +Install and compile gitlab-workhorse. This requires +[Go 1.5](https://golang.org/dl) which should already be on your system from +GitLab 8.1. + +```bash +cd /home/git/gitlab-workhorse +sudo -u git -H git fetch --all +sudo -u git -H git checkout v0.7.11 +sudo -u git -H make +``` + +### 7. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production + +``` + +### 8. Update configuration files + +#### New configuration options for `gitlab.yml` + +There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +git diff origin/8-10-stable:config/gitlab.yml.example origin/8-11-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +# For HTTPS configurations +git diff origin/8-10-stable:lib/support/nginx/gitlab-ssl origin/8-11-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-10-stable:lib/support/nginx/gitlab origin/8-11-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-11-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-11-stable/config/initializers/smtp_settings.rb.sample#L13? + +#### Init script + +Ensure you're still up-to-date with the latest init script changes: + + sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload + +### 9. Start application + + sudo service gitlab start + sudo service nginx restart + +### 10. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.10) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 8.9 to 8.10](8.9-to-8.10.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. diff --git a/doc/update/8.11-to-8.12.md b/doc/update/8.11-to-8.12.md new file mode 100644 index 00000000000..cddfa7e3e01 --- /dev/null +++ b/doc/update/8.11-to-8.12.md @@ -0,0 +1,205 @@ +# From 8.11 to 8.12 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +We will continue supporting Ruby < 2.3 for the time being but we recommend you +upgrade to Ruby 2.3 if you're running a source installation, as this is the same +version that ships with our Omnibus package. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz +echo 'c39b4001f7acb4e334cb60a0f4df72d434bef711 ruby-2.3.1.tar.gz' | shasum --check - && tar xzf ruby-2.3.1.tar.gz +cd ruby-2.3.1 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-12-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-12-stable-ee +``` + +### 5. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v3.6.1 +``` + +### 6. Update gitlab-workhorse + +Install and compile gitlab-workhorse. This requires +[Go 1.5](https://golang.org/dl) which should already be on your system from +GitLab 8.1. + +```bash +cd /home/git/gitlab-workhorse +sudo -u git -H git fetch --all +sudo -u git -H git checkout v0.8.2 +sudo -u git -H make +``` + +### 7. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production +``` + +### 8. Update configuration files + +#### New configuration options for `gitlab.yml` + +There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +git diff origin/8-11-stable:config/gitlab.yml.example origin/8-12-stable:config/gitlab.yml.example +``` + +#### Git configuration + +Configure Git to generate packfile bitmaps (introduced in Git 2.0) on +the GitLab server during `git gc`. + +```sh +sudo -u git -H git config --global repack.writeBitmaps true +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +# For HTTPS configurations +git diff origin/8-11-stable:lib/support/nginx/gitlab-ssl origin/8-12-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-11-stable:lib/support/nginx/gitlab origin/8-12-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-12-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-12-stable/config/initializers/smtp_settings.rb.sample#L13? + +#### Init script + +Ensure you're still up-to-date with the latest init script changes: + + sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload + +### 9. Start application + + sudo service gitlab start + sudo service nginx restart + +### 10. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.11) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 8.10 to 8.11](8.10-to-8.11.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. diff --git a/doc/update/8.12-to-8.13.md b/doc/update/8.12-to-8.13.md new file mode 100644 index 00000000000..8940d14559b --- /dev/null +++ b/doc/update/8.12-to-8.13.md @@ -0,0 +1,205 @@ +# From 8.12 to 8.13 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +We will continue supporting Ruby < 2.3 for the time being but we recommend you +upgrade to Ruby 2.3 if you're running a source installation, as this is the same +version that ships with our Omnibus package. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz +echo 'c39b4001f7acb4e334cb60a0f4df72d434bef711 ruby-2.3.1.tar.gz' | shasum --check - && tar xzf ruby-2.3.1.tar.gz +cd ruby-2.3.1 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-13-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-13-stable-ee +``` + +### 5. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v3.6.3 +``` + +### 6. Update gitlab-workhorse + +Install and compile gitlab-workhorse. This requires +[Go 1.5](https://golang.org/dl) which should already be on your system from +GitLab 8.1. + +```bash +cd /home/git/gitlab-workhorse +sudo -u git -H git fetch --all +sudo -u git -H git checkout v0.8.5 +sudo -u git -H make +``` + +### 7. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production +``` + +### 8. Update configuration files + +#### New configuration options for `gitlab.yml` + +There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +git diff origin/8-12-stable:config/gitlab.yml.example origin/8-13-stable:config/gitlab.yml.example +``` + +#### Git configuration + +Configure Git to generate packfile bitmaps (introduced in Git 2.0) on +the GitLab server during `git gc`. + +```sh +sudo -u git -H git config --global repack.writeBitmaps true +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +# For HTTPS configurations +git diff origin/8-12-stable:lib/support/nginx/gitlab-ssl origin/8-13-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-12-stable:lib/support/nginx/gitlab origin/8-13-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +Ensure you're still up-to-date with the latest init script changes: + + sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload + +### 9. Start application + + sudo service gitlab start + sudo service nginx restart + +### 10. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.12) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 8.11 to 8.12](8.11-to-8.12.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index 9f5c6c4dc84..dd3fdafd8d1 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -158,6 +158,10 @@ it where the 'public' directory of GitLab is. cd /home/git/gitlab sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab ``` + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 8. Use Redis v2.8.0+ diff --git a/doc/update/8.3-to-8.4.md b/doc/update/8.3-to-8.4.md index 9f6517d9487..e62d894609a 100644 --- a/doc/update/8.3-to-8.4.md +++ b/doc/update/8.3-to-8.4.md @@ -98,6 +98,10 @@ We updated the init script for GitLab in order to set a specific PATH for gitlab cd /home/git/gitlab sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab ``` + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 8. Start application diff --git a/doc/update/8.4-to-8.5.md b/doc/update/8.4-to-8.5.md index 0cb137a03cc..678cc69d773 100644 --- a/doc/update/8.4-to-8.5.md +++ b/doc/update/8.4-to-8.5.md @@ -119,6 +119,10 @@ via [/etc/default/gitlab]. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 8. Start application diff --git a/doc/update/8.5-to-8.6.md b/doc/update/8.5-to-8.6.md index 6267f14eba4..a76346516b9 100644 --- a/doc/update/8.5-to-8.6.md +++ b/doc/update/8.5-to-8.6.md @@ -138,6 +138,10 @@ via [/etc/default/gitlab]. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 9. Start application diff --git a/doc/update/8.6-to-8.7.md b/doc/update/8.6-to-8.7.md index cb66ef920bb..05ef4e61759 100644 --- a/doc/update/8.6-to-8.7.md +++ b/doc/update/8.6-to-8.7.md @@ -127,6 +127,10 @@ via [/etc/default/gitlab]. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 8. Start application diff --git a/doc/update/8.7-to-8.8.md b/doc/update/8.7-to-8.8.md index 32906650f6f..8ce434e5f78 100644 --- a/doc/update/8.7-to-8.8.md +++ b/doc/update/8.7-to-8.8.md @@ -127,6 +127,10 @@ via [/etc/default/gitlab]. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 8. Start application diff --git a/doc/update/8.8-to-8.9.md b/doc/update/8.8-to-8.9.md index f078a2bece5..aa077316bbe 100644 --- a/doc/update/8.8-to-8.9.md +++ b/doc/update/8.8-to-8.9.md @@ -156,6 +156,10 @@ See [smtp_settings.rb.sample] as an example. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 9. Start application diff --git a/doc/update/8.9-to-8.10.md b/doc/update/8.9-to-8.10.md index 84065a84e50..bb2c79fbb84 100644 --- a/doc/update/8.9-to-8.10.md +++ b/doc/update/8.9-to-8.10.md @@ -46,7 +46,7 @@ sudo -u git -H git checkout 8-10-stable-ee ```bash cd /home/git/gitlab-shell sudo -u git -H git fetch --all --tags -sudo -u git -H git checkout v3.2.0 +sudo -u git -H git checkout v3.2.1 ``` ### 5. Update gitlab-workhorse @@ -58,7 +58,7 @@ GitLab 8.1. ```bash cd /home/git/gitlab-workhorse sudo -u git -H git fetch --all -sudo -u git -H git checkout v0.7.7 +sudo -u git -H git checkout v0.7.8 sudo -u git -H make ``` @@ -156,6 +156,10 @@ See [smtp_settings.rb.sample] as an example. Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +For Ubuntu 16.04.1 LTS: + + sudo systemctl daemon-reload ### 9. Start application diff --git a/doc/user/account/security.md b/doc/user/account/security.md new file mode 100644 index 00000000000..816094bf8d2 --- /dev/null +++ b/doc/user/account/security.md @@ -0,0 +1,3 @@ +# Account Security + +- [Two-Factor Authentication](two_factor_authentication.md) diff --git a/doc/user/account/two_factor_authentication.md b/doc/user/account/two_factor_authentication.md new file mode 100644 index 00000000000..881358ed94d --- /dev/null +++ b/doc/user/account/two_factor_authentication.md @@ -0,0 +1,68 @@ +# Two-Factor Authentication + +## Recovery options + +If you lose your code generation device (such as your mobile phone) and you need +to disable two-factor authentication on your account, you have several options. + +### Use a saved recovery code + +When you enabled two-factor authentication for your account, a series of +recovery codes were generated. If you saved those codes somewhere safe, you +may use one to sign in. + +First, enter your username/email and password on the GitLab sign in page. When +prompted for a two-factor code, enter one of the recovery codes you saved +previously. + +> **Note:** Once a particular recovery code has been used, it cannot be used again. + You may still use the other saved recovery codes at a later time. + +### Generate new recovery codes using SSH + +It's not uncommon for users to forget to save the recovery codes when enabling +two-factor authentication. If you have an SSH key added to your GitLab account, +you can generate a new set of recovery codes using SSH. + +Run `ssh git@gitlab.example.com 2fa_recovery_codes`. You will be prompted to +confirm that you wish to generate new codes. If you choose to continue, any +previously saved codes will be invalidated. + +```bash +$ ssh git@gitlab.example.com 2fa_recovery_codes +Are you sure you want to generate new two-factor recovery codes? +Any existing recovery codes you saved will be invalidated. (yes/no) +yes + +Your two-factor authentication recovery codes are: + +119135e5a3ebce8e +11f6v2a498810dcd +3924c7ab2089c902 +e79a3398bfe4f224 +34bd7b74adbc8861 +f061691d5107df1a +169bf32a18e63e7f +b510e7422e81c947 +20dbed24c5e74663 +df9d3b9403b9c9f0 + +During sign in, use one of the codes above when prompted for +your two-factor code. Then, visit your Profile Settings and add +a new device so you do not lose access to your account again. +``` + +Next, go to the GitLab sign in page and enter your username/email and password. +When prompted for a two-factor code, enter one of the recovery codes obtained +from the command line output. + +> **Note:** After signing in, you should immediately visit your **Profile Settings + -> Account** to set up two-factor authentication with a new device. + +### Ask a GitLab administrator to disable two-factor on your account + +If the above two methods are not possible, you may ask a GitLab global +administrator to disable two-factor authentication for your account. Please +be aware that this will temporarily leave your account in a less secure state. +You should sign in and re-enable two-factor authentication as soon as possible +after the administrator disables it. diff --git a/doc/user/admin_area/img/admin_labels.png b/doc/user/admin_area/img/admin_labels.png Binary files differnew file mode 100644 index 00000000000..1ee33a534ab --- /dev/null +++ b/doc/user/admin_area/img/admin_labels.png diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md new file mode 100644 index 00000000000..9e2a89ebdf6 --- /dev/null +++ b/doc/user/admin_area/labels.md @@ -0,0 +1,9 @@ +# Labels + +## Default Labels + +### Define your own default Label Set + +Labels that are created within the Labels view on the Admin Dashboard will be automatically added to each new project. + + diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md new file mode 100644 index 00000000000..eac57bc3de4 --- /dev/null +++ b/doc/user/admin_area/monitoring/health_check.md @@ -0,0 +1,66 @@ +# Health Check + +> [Introduced][ce-3888] in GitLab 8.8. + +GitLab provides a health check endpoint for uptime monitoring on the `health_check` web +endpoint. The health check reports on the overall system status based on the status of +the database connection, the state of the database migrations, and the ability to write +and access the cache. This endpoint can be provided to uptime monitoring services like +[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health]. + +## Access Token + +An access token needs to be provided while accessing the health check endpoint. The current +accepted token can be found on the `admin/health_check` page of your GitLab instance. + + + +The access token can be passed as a URL parameter: + +``` +https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN +``` + +or as an HTTP header: + +```bash +curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json +``` + +## Using the Endpoint + +Once you have the access token, health information can be retrieved as plain text, JSON, +or XML using the `health_check` endpoint: + +- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN` +- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN` +- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN` + +You can also ask for the status of specific services: + +- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN` +- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN` +- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN` + +For example, the JSON output of the following health check: + +```bash +curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json +``` + +would be like: + +``` +{"healthy":true,"message":"success"} +``` + +## Status + +On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint +will return a valid successful HTTP status code, and a `success` message. Ideally your +uptime monitoring should look for the success message. + +[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888 +[pingdom]: https://www.pingdom.com +[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html +[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring diff --git a/doc/monitoring/img/health_check_token.png b/doc/user/admin_area/monitoring/img/health_check_token.png Binary files differindex 2d7c82a65a8..2d7c82a65a8 100644 --- a/doc/monitoring/img/health_check_token.png +++ b/doc/user/admin_area/monitoring/img/health_check_token.png diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md new file mode 100644 index 00000000000..34e2e557f89 --- /dev/null +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -0,0 +1,20 @@ +# Continuous integration Admin settings + +## Maximum artifacts size + +The maximum size of the [build artifacts][art-yml] can be set in the Admin area +of your GitLab instance. The value is in MB and the default is 100MB. Note that +this setting is set for each build. + +1. Go to **Admin area > Settings** (`/admin/application_settings`). + +  + +1. Change the value of the maximum artifacts size (in MB): + +  + +1. Hit **Save** for the changes to take effect. + + +[art-yml]: ../../../administration/build_artifacts.md diff --git a/doc/user/admin_area/settings/img/access_restrictions.png b/doc/user/admin_area/settings/img/access_restrictions.png Binary files differnew file mode 100644 index 00000000000..8eea84320d7 --- /dev/null +++ b/doc/user/admin_area/settings/img/access_restrictions.png diff --git a/doc/user/admin_area/settings/img/admin_area_maximum_artifacts_size.png b/doc/user/admin_area/settings/img/admin_area_maximum_artifacts_size.png Binary files differnew file mode 100644 index 00000000000..53f7e76033e --- /dev/null +++ b/doc/user/admin_area/settings/img/admin_area_maximum_artifacts_size.png diff --git a/doc/user/admin_area/settings/img/admin_area_settings_button.png b/doc/user/admin_area/settings/img/admin_area_settings_button.png Binary files differnew file mode 100644 index 00000000000..509708b627f --- /dev/null +++ b/doc/user/admin_area/settings/img/admin_area_settings_button.png diff --git a/doc/user/admin_area/settings/img/domain_blacklist.png b/doc/user/admin_area/settings/img/domain_blacklist.png Binary files differnew file mode 100644 index 00000000000..bd87b73cf9e --- /dev/null +++ b/doc/user/admin_area/settings/img/domain_blacklist.png diff --git a/doc/user/admin_area/settings/img/restricted_url.png b/doc/user/admin_area/settings/img/restricted_url.png Binary files differnew file mode 100644 index 00000000000..8b00a18320b --- /dev/null +++ b/doc/user/admin_area/settings/img/restricted_url.png diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md new file mode 100644 index 00000000000..4b540473a6e --- /dev/null +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -0,0 +1,22 @@ +# Sign-up restrictions + +## Blacklist email domains + +> [Introduced][ce-5259] in GitLab 8.10. + +With this feature enabled, you can block email addresses of a specific domain +from creating an account on your GitLab server. This is particularly useful to +prevent spam. Disposable email addresses are usually used by malicious users to +create dummy accounts and spam issues. + +This feature can be activated via the **Application Settings** in the Admin area, +and you have the option of entering the list manually, or uploading a file with +the list. + +The blacklist accepts wildcards, so you can use `*.test.com` to block every +`test.com` subdomain, or `*.io` to block all domains ending in `.io`. Domains +should be separated by a whitespace, semicolon, comma, or a new line. + + + +[ce-5259]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5259 diff --git a/doc/administration/access_restrictions.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index 51d7996effd..633f16a617c 100644 --- a/doc/administration/access_restrictions.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -1,6 +1,8 @@ -# Access Restrictions +# Visibility and access controls -> **Note:** This feature is only available on versions 8.10 and above. +## Enabled Git access protocols + +> [Introduced][ce-4696] in GitLab 8.10. With GitLab's Access restrictions you can choose which Git access protocols you want your users to use to communicate with GitLab. This feature can be enabled @@ -15,8 +17,6 @@ to choose between:  -## Enabled Protocol - When both SSH and HTTP(S) are enabled, GitLab will behave as usual, it will give your users the option to choose which protocol they would like to use. @@ -35,4 +35,6 @@ not selected. > **Note:** Please keep in mind that disabling an access protocol does not actually block access to the server itself. The ports used for the protocol, be it SSH or HTTP, will still be accessible. What GitLab does is restrict access on the - application level.
\ No newline at end of file + application level. + +[ce-4696]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4696 diff --git a/doc/markdown/img/logo.png b/doc/user/img/markdown_logo.png Binary files differindex 05c8b0d0ccf..05c8b0d0ccf 100644 --- a/doc/markdown/img/logo.png +++ b/doc/user/img/markdown_logo.png diff --git a/doc/user/img/markdown_video.mp4 b/doc/user/img/markdown_video.mp4 Binary files differnew file mode 100644 index 00000000000..1fc478842f5 --- /dev/null +++ b/doc/user/img/markdown_video.mp4 diff --git a/doc/user/markdown.md b/doc/user/markdown.md new file mode 100644 index 00000000000..56e5b802a52 --- /dev/null +++ b/doc/user/markdown.md @@ -0,0 +1,796 @@ +# Markdown + +## Table of Contents + +**[GitLab Flavored Markdown](#gitlab-flavored-markdown-gfm)** + +* [Newlines](#newlines) +* [Multiple underscores in words](#multiple-underscores-in-words) +* [URL auto-linking](#url-auto-linking) +* [Multiline Blockquote](#multiline-blockquote) +* [Code and Syntax Highlighting](#code-and-syntax-highlighting) +* [Inline Diff](#inline-diff) +* [Emoji](#emoji) +* [Special GitLab references](#special-gitlab-references) +* [Task Lists](#task-lists) +* [Videos](#videos) + +**[Standard Markdown](#standard-markdown)** + +* [Headers](#headers) +* [Emphasis](#emphasis) +* [Lists](#lists) +* [Links](#links) +* [Images](#images) +* [Blockquotes](#blockquotes) +* [Inline HTML](#inline-html) +* [Horizontal Rule](#horizontal-rule) +* [Line Breaks](#line-breaks) +* [Tables](#tables) +* [Footnotes](#footnotes) + +**[Wiki-Specific Markdown](#wiki-specific-markdown)** + +* [Wiki - Direct page link](#wiki-direct-page-link) +* [Wiki - Direct file link](#wiki-direct-file-link) +* [Wiki - Hierarchical link](#wiki-hierarchical-link) +* [Wiki - Root link](#wiki-root-link) + +**[References](#references)** + +## GitLab Flavored Markdown (GFM) + +> **Note:** +Not all of the GitLab-specific extensions to Markdown that are described in +this document currently work on our documentation website. +> +For the best result, we encourage you to check this document out as rendered +by GitLab: [markdown.md] + +_GitLab uses the [Redcarpet Ruby library][redcarpet] for Markdown processing._ + +GitLab uses "GitLab Flavored Markdown" (GFM). It extends the standard Markdown in a few significant ways to add some useful functionality. It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). + +You can use GFM in the following areas: + +- comments +- issues +- merge requests +- milestones +- snippets (the snippet must be named with a `.md` extension) +- wiki pages +- markdown documents inside the repository + +You can also use other rich text files in GitLab. You might have to install a +dependency to do so. Please see the [github-markup gem readme](https://github.com/gitlabhq/markup#markups) for more information. + +## Newlines + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#newlines + +GFM honors the markdown specification in how [paragraphs and line breaks are handled](https://daringfireball.net/projects/markdown/syntax#p). + +A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines. +Line-breaks, or softreturns, are rendered if you end a line with two or more spaces: + + Roses are red [followed by two or more spaces] + Violets are blue + + Sugar is sweet + +Roses are red +Violets are blue + +Sugar is sweet + +## Multiple underscores in words + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiple-underscores-in-words + +It is not reasonable to italicize just _part_ of a word, especially when you're dealing with code and names that often appear with multiple underscores. Therefore, GFM ignores multiple underscores in words: + + perform_complicated_task + + do_this_and_do_that_and_another_thing + +perform_complicated_task + +do_this_and_do_that_and_another_thing + +## URL auto-linking + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#url-auto-linking + +GFM will autolink almost any URL you copy and paste into your text: + + * https://www.google.com + * https://google.com/ + * ftp://ftp.us.debian.org/debian/ + * smb://foo/bar/baz + * irc://irc.freenode.net/gitlab + * http://localhost:3000 + +* https://www.google.com +* https://google.com/ +* ftp://ftp.us.debian.org/debian/ +* smb://foo/bar/baz +* irc://irc.freenode.net/gitlab +* http://localhost:3000 + +## Multiline Blockquote + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiline-blockquote + +On top of standard Markdown [blockquotes](#blockquotes), which require prepending `>` to quoted lines, +GFM supports multiline blockquotes fenced by <code>>>></code>: + +```no-highlight +>>> +If you paste a message from somewhere else + +that + +spans + +multiple lines, + +you can quote that without having to manually prepend `>` to every line! +>>> +``` + +>>> +If you paste a message from somewhere else + +that + +spans + +multiple lines, + +you can quote that without having to manually prepend `>` to every line! +>>> + +## Code and Syntax Highlighting + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting + +_GitLab uses the [Rouge Ruby library][rouge] for syntax highlighting. For a +list of supported languages visit the Rouge website._ + +Blocks of code are either fenced by lines with three back-ticks <code>```</code>, +or are indented with four spaces. Only the fenced code blocks support syntax +highlighting: + +```no-highlight +Inline `code` has `back-ticks around` it. +``` + +Inline `code` has `back-ticks around` it. + +Example: + + ```javascript + var s = "JavaScript syntax highlighting"; + alert(s); + ``` + + ```python + def function(): + #indenting works just fine in the fenced code block + s = "Python syntax highlighting" + print s + ``` + + ```ruby + require 'redcarpet' + markdown = Redcarpet.new("Hello World!") + puts markdown.to_html + ``` + + ``` + No language indicated, so no syntax highlighting. + s = "There is no highlighting for this." + But let's throw in a <b>tag</b>. + ``` + +becomes: + +```javascript +var s = "JavaScript syntax highlighting"; +alert(s); +``` + +```python +def function(): + #indenting works just fine in the fenced code block + s = "Python syntax highlighting" + print s +``` + +```ruby +require 'redcarpet' +markdown = Redcarpet.new("Hello World!") +puts markdown.to_html +``` + +``` +No language indicated, so no syntax highlighting. +s = "There is no highlighting for this." +But let's throw in a <b>tag</b>. +``` + +## Inline Diff + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-diff + +With inline diffs tags you can display {+ additions +} or [- deletions -]. + +The wrapping tags can be either curly braces or square brackets [+ additions +] or {- deletions -}. + +However the wrapping tags cannot be mixed as such: + +- {+ additions +] +- [+ additions +} +- {- deletions -] +- [- deletions -} + +## Emoji + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji + + Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: + + :zap: You can use emoji anywhere GFM is supported. :v: + + You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. + + If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up on the supported codes. + + Consult the [Emoji Cheat Sheet](http://emoji.codes) for a list of all supported emoji codes. :thumbsup: + +Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: + +:zap: You can use emoji anywhere GFM is supported. :v: + +You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. + +If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up on the supported codes. + +Consult the [Emoji Cheat Sheet](http://emoji.codes) for a list of all supported emoji codes. :thumbsup: + +## Special GitLab References + +GFM recognizes special references. + +You can easily reference e.g. an issue, a commit, a team member or even the whole team within a project. + +GFM will turn that reference into a link so you can navigate between them easily. + +GFM will recognize the following: + +| input | references | +|:-----------------------|:--------------------------- | +| `@user_name` | specific user | +| `@group_name` | specific group | +| `@all` | entire team | +| `#123` | issue | +| `!123` | merge request | +| `$123` | snippet | +| `~123` | label by ID | +| `~bug` | one-word label by name | +| `~"feature request"` | multi-word label by name | +| `%123` | milestone by ID | +| `%v1.23` | one-word milestone by name | +| `%"release candidate"` | multi-word milestone by name | +| `9ba12248` | specific commit | +| `9ba12248...b19a04f5` | commit range comparison | +| `[README](doc/README)` | repository file references | + +GFM also recognizes certain cross-project references: + +| input | references | +|:----------------------------------------|:------------------------| +| `namespace/project#123` | issue | +| `namespace/project!123` | merge request | +| `namespace/project%123` | milestone | +| `namespace/project$123` | snippet | +| `namespace/project@9ba12248` | specific commit | +| `namespace/project@9ba12248...b19a04f5` | commit range comparison | +| `namespace/project~"Some label"` | issues with given label | + +## Task Lists + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#task-lists + +You can add task lists to issues, merge requests and comments. To create a task list, add a specially-formatted Markdown list, like so: + +```no-highlight +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 +``` + +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 + +Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes. + +## Videos + +> If this is not rendered correctly, see +https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#videos + +Image tags with a video extension are automatically converted to a video player. + +The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`. + + Here's a sample video: + +  + +Here's a sample video: + + + +# Standard Markdown + +## Headers + +```no-highlight +# H1 +## H2 +### H3 +#### H4 +##### H5 +###### H6 + +Alternatively, for H1 and H2, an underline-ish style: + +Alt-H1 +====== + +Alt-H2 +------ +``` + +# H1 +## H2 +### H3 +#### H4 +##### H5 +###### H6 + +Alternatively, for H1 and H2, an underline-ish style: + +Alt-H1 +====== + +Alt-H2 +------ + +### Header IDs and links + +All Markdown-rendered headers automatically get IDs, except in comments. + +On hover a link to those IDs becomes visible to make it easier to copy the link to the header to give it to someone else. + +The IDs are generated from the content of the header according to the following rules: + +1. All text is converted to lowercase +1. All non-word text (e.g., punctuation, HTML) is removed +1. All spaces are converted to hyphens +1. Two or more hyphens in a row are converted to one +1. If a header with the same ID has already been generated, a unique + incrementing number is appended, starting at 1. + +For example: + +``` +# This header has spaces in it +## This header has a :thumbsup: in it +# This header has Unicode in it: 한글 +## This header has spaces in it +### This header has spaces in it +``` + +Would generate the following link IDs: + +1. `this-header-has-spaces-in-it` +1. `this-header-has-a-in-it` +1. `this-header-has-unicode-in-it-한글` +1. `this-header-has-spaces-in-it` +1. `this-header-has-spaces-in-it-1` + +Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. + +## Emphasis + +```no-highlight +Emphasis, aka italics, with *asterisks* or _underscores_. + +Strong emphasis, aka bold, with **asterisks** or __underscores__. + +Combined emphasis with **asterisks and _underscores_**. + +Strikethrough uses two tildes. ~~Scratch this.~~ +``` + +Emphasis, aka italics, with *asterisks* or _underscores_. + +Strong emphasis, aka bold, with **asterisks** or __underscores__. + +Combined emphasis with **asterisks and _underscores_**. + +Strikethrough uses two tildes. ~~Scratch this.~~ + +## Lists + +```no-highlight +1. First ordered list item +2. Another item + * Unordered sub-list. +1. Actual numbers don't matter, just that it's a number + 1. Ordered sub-list +4. And another item. + +* Unordered list can use asterisks +- Or minuses ++ Or pluses +``` + +1. First ordered list item +2. Another item + * Unordered sub-list. +1. Actual numbers don't matter, just that it's a number + 1. Ordered sub-list +4. And another item. + +* Unordered list can use asterisks +- Or minuses ++ Or pluses + +If a list item contains multiple paragraphs, +each subsequent paragraph should be indented with four spaces. + +```no-highlight +1. First ordered list item + + Second paragraph of first item. +2. Another item +``` + +1. First ordered list item + + Second paragraph of first item. +2. Another item + +If the second paragraph isn't indented with four spaces, +the second list item will be incorrectly labeled as `1`. + +```no-highlight +1. First ordered list item + + Second paragraph of first item. +2. Another item +``` + +1. First ordered list item + + Second paragraph of first item. +2. Another item + +## Links + +There are two ways to create links, inline-style and reference-style. + + [I'm an inline-style link](https://www.google.com) + + [I'm a reference-style link][Arbitrary case-insensitive reference text] + + [I'm a relative reference to a repository file](LICENSE) + + [You can use numbers for reference-style link definitions][1] + + Or leave it empty and use the [link text itself][] + + Some text to show that the reference links can follow later. + + [arbitrary case-insensitive reference text]: https://www.mozilla.org + [1]: http://slashdot.org + [link text itself]: https://www.reddit.com + +[I'm an inline-style link](https://www.google.com) + +[I'm a reference-style link][Arbitrary case-insensitive reference text] + +[I'm a relative reference to a repository file](LICENSE)[^1] + +[You can use numbers for reference-style link definitions][1] + +Or leave it empty and use the [link text itself][] + +Some text to show that the reference links can follow later. + +[arbitrary case-insensitive reference text]: https://www.mozilla.org +[1]: http://slashdot.org +[link text itself]: https://www.reddit.com + +**Note** + +Relative links do not allow referencing project files in a wiki page or wiki page in a project file. The reason for this is that, in GitLab, wiki is always a separate git repository. For example: + +`[I'm a reference-style link](style)` + +will point the link to `wikis/style` when the link is inside of a wiki markdown file. + +## Images + + Here's our logo (hover to see the title text): + + Inline-style: +  + + Reference-style: + ![alt text1][logo] + + [logo]: img/markdown_logo.png + +Here's our logo: + +Inline-style: + + + +Reference-style: + +![alt text][logo] + +[logo]: img/markdown_logo.png + +## Blockquotes + +```no-highlight +> Blockquotes are very handy in email to emulate reply text. +> This line is part of the same quote. + +Quote break. + +> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. +``` + +> Blockquotes are very handy in email to emulate reply text. +> This line is part of the same quote. + +Quote break. + +> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. + +## Inline HTML + +You can also use raw HTML in your Markdown, and it'll mostly work pretty well. + +See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span` elements. + +```no-highlight +<dl> + <dt>Definition list</dt> + <dd>Is something people use sometimes.</dd> + + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> +</dl> +``` + +<dl> + <dt>Definition list</dt> + <dd>Is something people use sometimes.</dd> + + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> +</dl> + +## Horizontal Rule + +``` +Three or more... + +--- + +Hyphens + +*** + +Asterisks + +___ + +Underscores +``` + +Three or more... + +--- + +Hyphens + +*** + +Asterisks + +___ + +Underscores + +## Line Breaks + +My basic recommendation for learning how line breaks work is to experiment and discover -- hit <Enter> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. "Markdown Toggle" is your friend. + +Here are some things to try out: + +``` +Here's a line for us to start with. + +This line is separated from the one above by two newlines, so it will be a *separate paragraph*. + +This line is also a separate paragraph, but... +This line is only separated by a single newline, so it's a separate line in the *same paragraph*. + +This line is also a separate paragraph, and... +This line is on its own line, because the previous line ends with two +spaces. +``` + +Here's a line for us to start with. + +This line is separated from the one above by two newlines, so it will be a *separate paragraph*. + +This line is also begins a separate paragraph, but... +This line is only separated by a single newline, so it's a separate line in the *same paragraph*. + +This line is also a separate paragraph, and... +This line is on its own line, because the previous line ends with two +spaces. + +## Tables + +Tables aren't part of the core Markdown spec, but they are part of GFM and Markdown Here supports them. + +``` +| header 1 | header 2 | +| -------- | -------- | +| cell 1 | cell 2 | +| cell 3 | cell 4 | +``` + +Code above produces next output: + +| header 1 | header 2 | +| -------- | -------- | +| cell 1 | cell 2 | +| cell 3 | cell 4 | + +**Note** + +The row of dashes between the table header and body must have at least three dashes in each column. + +By including colons in the header row, you can align the text within that column: + +``` +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :----------- | :------: | ------------: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +``` + +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :----------- | :------: | ------------: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | + +## Footnotes + +You can add footnotes to your text as follows.[^1] +[^1]: This is my awesome footnote. + +``` +You can add footnotes to your text as follows.[^1] +[^1]: This is my awesome footnote. +``` + +## Wiki-specific Markdown + +The following examples show how links inside wikis behave. + +### Wiki - Direct page link + +A link which just includes the slug for a page will point to that page, +_at the base level of the wiki_. + +This snippet would link to a `documentation` page at the root of your wiki: + +```markdown +[Link to Documentation](documentation) +``` + +### Wiki - Direct file link + +Links with a file extension point to that file, _relative to the current page_. + +If this snippet was placed on a page at `<your_wiki>/documentation/related`, +it would link to `<your_wiki>/documentation/file.md`: + +```markdown +[Link to File](file.md) +``` + +### Wiki - Hierarchical link + +A link can be constructed relative to the current wiki page using `./<page>`, +`../<page>`, etc. + +- If this snippet was placed on a page at `<your_wiki>/documentation/main`, + it would link to `<your_wiki>/documentation/related`: + + ```markdown + [Link to Related Page](./related) + ``` + +- If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, + it would link to `<your_wiki>/documentation/main`: + + ```markdown + [Link to Related Page](../main) + ``` + +- If this snippet was placed on a page at `<your_wiki>/documentation/main`, + it would link to `<your_wiki>/documentation/related.md`: + + ```markdown + [Link to Related Page](./related.md) + ``` + +- If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, + it would link to `<your_wiki>/documentation/main.md`: + + ```markdown + [Link to Related Page](../main.md) + ``` + +### Wiki - Root link + +A link starting with a `/` is relative to the wiki root. + +- This snippet links to `<wiki_root>/documentation`: + + ```markdown + [Link to Related Page](/documentation) + ``` + +- This snippet links to `<wiki_root>/miscellaneous.md`: + + ```markdown + [Link to Related Page](/miscellaneous.md) + ``` +## References + +- This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). +- The [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. +- [Dillinger.io](http://dillinger.io) is a handy tool for testing standard markdown. + +[markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md +[rouge]: http://rouge.jneen.net/ "Rouge website" +[redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" +[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com diff --git a/doc/user/permissions.md b/doc/user/permissions.md new file mode 100644 index 00000000000..d6216a8dd50 --- /dev/null +++ b/doc/user/permissions.md @@ -0,0 +1,172 @@ +# Permissions + +Users have different abilities depending on the access level they have in a +particular group or project. If a user is both in a group's project and the +project itself, the highest permission level is used. + +On public and internal projects the Guest role is not enforced. All users will +be able to create issues, leave comments, and pull or download the project code. + +GitLab administrators receive all permissions. + +To add or import a user, you can follow the [project users and members +documentation](../workflow/add-user/add-user.md). + +## Project + +The following table depicts the various user permission levels in a project. + +| Action | Guest | Reporter | Developer | Master | Owner | +|---------------------------------------|---------|------------|-------------|----------|--------| +| Create new issue | ✓ | ✓ | ✓ | ✓ | ✓ | +| Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | +| See a list of builds | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| See a build log | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| Download and browse build artifacts | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| View wiki pages | ✓ | ✓ | ✓ | ✓ | ✓ | +| Pull project code | | ✓ | ✓ | ✓ | ✓ | +| Download project | | ✓ | ✓ | ✓ | ✓ | +| Create code snippets | | ✓ | ✓ | ✓ | ✓ | +| Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | +| Manage labels | | ✓ | ✓ | ✓ | ✓ | +| See a commit status | | ✓ | ✓ | ✓ | ✓ | +| See a container registry | | ✓ | ✓ | ✓ | ✓ | +| See environments | | ✓ | ✓ | ✓ | ✓ | +| See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | +| Manage/Accept merge requests | | | ✓ | ✓ | ✓ | +| Create new merge request | | | ✓ | ✓ | ✓ | +| Create new branches | | | ✓ | ✓ | ✓ | +| Push to non-protected branches | | | ✓ | ✓ | ✓ | +| Force push to non-protected branches | | | ✓ | ✓ | ✓ | +| Remove non-protected branches | | | ✓ | ✓ | ✓ | +| Add tags | | | ✓ | ✓ | ✓ | +| Write a wiki | | | ✓ | ✓ | ✓ | +| Cancel and retry builds | | | ✓ | ✓ | ✓ | +| Create or update commit status | | | ✓ | ✓ | ✓ | +| Update a container registry | | | ✓ | ✓ | ✓ | +| Remove a container registry image | | | ✓ | ✓ | ✓ | +| Create new environments | | | ✓ | ✓ | ✓ | +| Create new milestones | | | | ✓ | ✓ | +| Add new team members | | | | ✓ | ✓ | +| Push to protected branches | | | | ✓ | ✓ | +| Enable/disable branch protection | | | | ✓ | ✓ | +| Turn on/off protected branch push for devs| | | | ✓ | ✓ | +| Rewrite/remove Git tags | | | | ✓ | ✓ | +| Edit project | | | | ✓ | ✓ | +| Add deploy keys to project | | | | ✓ | ✓ | +| Configure project hooks | | | | ✓ | ✓ | +| Manage runners | | | | ✓ | ✓ | +| Manage build triggers | | | | ✓ | ✓ | +| Manage variables | | | | ✓ | ✓ | +| Delete environments | | | | ✓ | ✓ | +| Switch visibility level | | | | | ✓ | +| Transfer project to another namespace | | | | | ✓ | +| Remove project | | | | | ✓ | +| Force push to protected branches [^2] | | | | | | +| Remove protected branches [^2] | | | | | | + +[^1]: If **Public pipelines** is enabled in **Project Settings > CI/CD Pipelines** +[^2]: Not allowed for Guest, Reporter, Developer, Master, or Owner + +## Group + +Any user can remove themselves from a group, unless they are the last Owner of +the group. The following table depicts the various user permission levels in a +group. + +| Action | Guest | Reporter | Developer | Master | Owner | +|-------------------------|-------|----------|-----------|--------|-------| +| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | +| Edit group | | | | | ✓ | +| Create project in group | | | | ✓ | ✓ | +| Manage group members | | | | | ✓ | +| Remove group | | | | | ✓ | + +## External Users + +In cases where it is desired that a user has access only to some internal or +private projects, there is the option of creating **External Users**. This +feature may be useful when for example a contractor is working on a given +project and should only have access to that project. + +External users can only access projects to which they are explicitly granted +access, thus hiding all other internal or private ones from them. Access can be +granted by adding the user as member to the project or group. + +They will, like usual users, receive a role in the project or group with all +the abilities that are mentioned in the table above. They cannot however create +groups or projects, and they have the same access as logged out users in all +other cases. + +An administrator can flag a user as external [through the API](../api/users.md) +or by checking the checkbox on the admin panel. As an administrator, navigate +to **Admin > Users** to create a new user or edit an existing one. There, you +will find the option to flag the user as external. + +By default new users are not set as external users. This behavior can be changed +by an administrator under **Admin > Application Settings**. + +## Project features + +Project features like wiki and issues can be hidden from users depending on +which visibility level you select on project settings. + +- Disabled: disabled for everyone +- Only team members: only team members will see even if your project is public or internal +- Everyone with access: everyone can see depending on your project visibility level + +## GitLab CI + +GitLab CI permissions rely on the role the user has in GitLab. There are four +permission levels it total: + +- admin +- master +- developer +- guest/reporter + +The admin user can perform any action on GitLab CI in scope of the GitLab +instance and project. In addition, all admins can use the admin interface under +`/admin/runners`. + +| Action | Guest, Reporter | Developer | Master | Admin | +|---------------------------------------|-----------------|-------------|----------|--------| +| See commits and builds | ✓ | ✓ | ✓ | ✓ | +| Retry or cancel build | | ✓ | ✓ | ✓ | +| Remove project | | | ✓ | ✓ | +| Create project | | | ✓ | ✓ | +| Change project configuration | | | ✓ | ✓ | +| Add specific runners | | | ✓ | ✓ | +| Add shared runners | | | | ✓ | +| See events in the system | | | | ✓ | +| Admin interface | | | | ✓ | + +### Build permissions + +> Changed in GitLab 8.12. + +GitLab 8.12 has a completely redesigned build permissions system. +Read all about the [new model and its implications][new-mod]. + +This table shows granted privileges for builds triggered by specific types of +users: + +| Action | Guest, Reporter | Developer | Master | Admin | +|---------------------------------------------|-----------------|-------------|----------|--------| +| Run CI build | | ✓ | ✓ | ✓ | +| Clone source and LFS from current project | | ✓ | ✓ | ✓ | +| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | +| Clone source and LFS from internal projects | | ✓ [^3] | ✓ [^3] | ✓ | +| Clone source and LFS from private projects | | ✓ [^4] | ✓ [^4] | ✓ [^4] | +| Push source and LFS | | | | | +| Pull container images from current project | | ✓ | ✓ | ✓ | +| Pull container images from public projects | | ✓ | ✓ | ✓ | +| Pull container images from internal projects| | ✓ [^3] | ✓ [^3] | ✓ | +| Pull container images from private projects | | ✓ [^4] | ✓ [^4] | ✓ [^4] | +| Push container images to current project | | ✓ | ✓ | ✓ | +| Push container images to other projects | | | | | + +[^3]: Only if user is not external one. +[^4]: Only if user is a member of the project. +[ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 +[new-mod]: project/new_ci_build_permissions_model.md diff --git a/doc/user/project/builds/artifacts.md b/doc/user/project/builds/artifacts.md new file mode 100644 index 00000000000..88f1863dddb --- /dev/null +++ b/doc/user/project/builds/artifacts.md @@ -0,0 +1,136 @@ +# Introduction to build artifacts + +>**Notes:** +>- Since GitLab 8.2 and GitLab Runner 0.7.0, build artifacts that are created by + GitLab Runner are uploaded to GitLab and are downloadable as a single archive + (`tar.gz`) using the GitLab UI. +>- Starting from GitLab 8.4 and GitLab Runner 1.0, the artifacts archive format + changed to `ZIP`, and it is now possible to browse its contents, with the added + ability of downloading the files separately. +>- The artifacts browser will be available only for new artifacts that are sent + to GitLab using GitLab Runner version 1.0 and up. It will not be possible to + browse old artifacts already uploaded to GitLab. +>- This is the user documentation. For the administration guide see + [administration/build_artifacts.md](../../../administration/build_artifacts.md). + +Artifacts is a list of files and directories which are attached to a build +after it completes successfully. This feature is enabled by default in all GitLab installations. + +## Defining artifacts in `.gitlab-ci.yml` + +A simple example of using the artifacts definition in `.gitlab-ci.yml` would be +the following: + +```yaml +pdf: + script: xelatex mycv.tex + artifacts: + paths: + - mycv.pdf +``` + +A job named `pdf` calls the `xelatex` command in order to build a pdf file from +the latex source file `mycv.tex`. We then define the `artifacts` paths which in +turn are defined with the `paths` keyword. All paths to files and directories +are relative to the repository that was cloned during the build. + +For more examples on artifacts, follow the artifacts reference in +[`.gitlab-ci.yml` documentation](../../../ci/yaml/README.md#artifacts). + +## Browsing build artifacts + +When GitLab receives an artifacts archive, an archive metadata file is also +generated. This metadata file describes all the entries that are located in the +artifacts archive itself. The metadata file is in a binary format, with +additional GZIP compression. + +GitLab does not extract the artifacts archive in order to save space, memory +and disk I/O. It instead inspects the metadata file which contains all the +relevant information. This is especially important when there is a lot of +artifacts, or an archive is a very large file. + +--- + +After a build finishes, if you visit the build's specific page, you can see +that there are two buttons. One is for downloading the artifacts archive and +the other for browsing its contents. + + + +--- + +The archive browser shows the name and the actual file size of each file in the +archive. If your artifacts contained directories, then you are also able to +browse inside them. + +Below you can see how browsing looks like. In this case we have browsed inside +the archive and at this point there is one directory and one HTML file. + + + +--- + +## Downloading build artifacts + +>**Note:** +GitLab does not extract the entire artifacts archive to send just a single file +to the user. When clicking on a specific file, [GitLab Workhorse] extracts it +from the archive and the download begins. This implementation saves space, +memory and disk I/O. + +If you need to download the whole archive, there are buttons in various places +inside GitLab that make that possible. + +1. While on the pipelines page, you can see the download icon for each build's + artifacts archive in the right corner: + +  + +1. While on the builds page, you can see the download icon for each build's + artifacts archive in the right corner: + +  + +1. While inside a specific build, you are presented with a download button + along with the one that browses the archive: + +  + +1. And finally, when browsing an archive you can see the download button at + the top right corner: + +  + +## Downloading the latest build artifacts + +It is possible to download the latest artifacts of a build via a well known URL +so you can use it for scripting purposes. + +The structure of the URL is the following: + +``` +https://example.com/<namespace>/<project>/builds/artifacts/<ref>/download?job=<job_name> +``` + +For example, to download the latest artifacts of the job named `rspec 6 20` of +the `master` branch of the `gitlab-ce` project that belongs to the `gitlab-org` +namespace, the URL would be: + +``` +https://gitlab.com/gitlab-org/gitlab-ce/builds/artifacts/master/download?job=rspec+6+20 +``` + +The latest builds are also exposed in the UI in various places. Specifically, +look for the download button in: + +- the main project's page +- the branches page +- the tags page + +If the latest build has failed to upload the artifacts, you can see that +information in the UI. + + + + +[gitlab workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse "GitLab Workhorse repository" diff --git a/doc/user/project/builds/img/build_artifacts_browser.png b/doc/user/project/builds/img/build_artifacts_browser.png Binary files differnew file mode 100644 index 00000000000..d95e2800c0f --- /dev/null +++ b/doc/user/project/builds/img/build_artifacts_browser.png diff --git a/doc/user/project/builds/img/build_artifacts_browser_button.png b/doc/user/project/builds/img/build_artifacts_browser_button.png Binary files differnew file mode 100644 index 00000000000..463540634e3 --- /dev/null +++ b/doc/user/project/builds/img/build_artifacts_browser_button.png diff --git a/doc/user/project/builds/img/build_artifacts_builds_page.png b/doc/user/project/builds/img/build_artifacts_builds_page.png Binary files differnew file mode 100644 index 00000000000..db78386ba7b --- /dev/null +++ b/doc/user/project/builds/img/build_artifacts_builds_page.png diff --git a/doc/user/project/builds/img/build_artifacts_pipelines_page.png b/doc/user/project/builds/img/build_artifacts_pipelines_page.png Binary files differnew file mode 100644 index 00000000000..6c2d1a4bdc7 --- /dev/null +++ b/doc/user/project/builds/img/build_artifacts_pipelines_page.png diff --git a/doc/user/project/builds/img/build_latest_artifacts_browser.png b/doc/user/project/builds/img/build_latest_artifacts_browser.png Binary files differnew file mode 100644 index 00000000000..d8e9071958c --- /dev/null +++ b/doc/user/project/builds/img/build_latest_artifacts_browser.png diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md new file mode 100644 index 00000000000..b205fea2c40 --- /dev/null +++ b/doc/user/project/container_registry.md @@ -0,0 +1,253 @@ +# GitLab Container Registry + +> [Introduced][ce-4040] in GitLab 8.8. + +--- + +> **Note** +Docker Registry manifest `v1` support was added in GitLab 8.9 to support Docker +versions earlier than 1.10. +> +This document is about the user guide. To learn how to enable GitLab Container +Registry across your GitLab instance, visit the +[administrator documentation](../../administration/container_registry.md). + +With the Docker Container Registry integrated into GitLab, every project can +have its own space to store its Docker images. + +You can read more about Docker Registry at https://docs.docker.com/registry/introduction/. + +--- + +## Enable the Container Registry for your project + +1. First, ask your system administrator to enable GitLab Container Registry + following the [administration documentation](../../administration/container_registry.md). + If you are using GitLab.com, this is enabled by default so you can start using + the Registry immediately. + +1. Go to your project's settings and enable the **Container Registry** feature + on your project. For new projects this might be enabled by default. For + existing projects (prior GitLab 8.8), you will have to explicitly enable it. + +  + +1. Hit **Save changes** for the changes to take effect. You should now be able + to see the **Registry** link in the project menu. + +  + +## Build and push images + +If you visit the **Registry** link under your project's menu, you can see the +explicit instructions to login to the Container Registry using your GitLab +credentials. + +For example if the Registry's URL is `registry.example.com`, the you should be +able to login with: + +``` +docker login registry.example.com +``` + +Building and publishing images should be a straightforward process. Just make +sure that you are using the Registry URL with the namespace and project name +that is hosted on GitLab: + +``` +docker build -t registry.example.com/group/project . +docker push registry.example.com/group/project +``` + +Your image will be named after the following scheme: + +``` +<registry URL>/<namespace>/<project> +``` + +As such, the name of the image is unique, but you can differentiate the images +using tags. + +## Use images from GitLab Container Registry + +To download and run a container from images hosted in GitLab Container Registry, +use `docker run`: + +``` +docker run [options] registry.example.com/group/project [arguments] +``` + +For more information on running Docker containers, visit the +[Docker documentation][docker-docs]. + +## Control Container Registry from within GitLab + +GitLab offers a simple Container Registry management panel. Go to your project +and click **Registry** in the project menu. + +This view will show you all tags in your project and will easily allow you to +delete them. + + + +## Build and push images using GitLab CI + +> **Note:** +This feature requires GitLab 8.8 and GitLab Runner 1.2. + +Make sure that your GitLab Runner is configured to allow building Docker images by +following the [Using Docker Build](../ci/docker/using_docker_build.md) +and [Using the GitLab Container Registry documentation](../ci/docker/using_docker_build.md#using-the-gitlab-container-registry). + +## Limitations + +In order to use a container image from your private project as an `image:` in +your `.gitlab-ci.yml`, you have to follow the +[Using a private Docker Registry][private-docker] +documentation. This workflow will be simplified in the future. + +## Troubleshooting the GitLab Container Registry + +### Basic Troubleshooting + +1. Check to make sure that the system clock on your Docker client and GitLab server have + been synchronized (e.g. via NTP). + +2. If you are using an S3-backed Registry, double check that the IAM + permissions and the S3 credentials (including region) are correct. See [the + sample IAM policy](https://docs.docker.com/registry/storage-drivers/s3/) + for more details. + +3. Check the Registry logs (e.g. `/var/log/gitlab/registry/current`) and the GitLab production logs + for errors (e.g. `/var/log/gitlab/gitlab-rails/production.log`). You may be able to find clues + there. + +### Advanced Troubleshooting + +>**NOTE:** The following section is only recommended for experts. + +Sometimes it's not obvious what is wrong, and you may need to dive deeper into +the communication between the Docker client and the Registry to find out +what's wrong. We will use a concrete example in the past to illustrate how to +diagnose a problem with the S3 setup. + +#### Unexpected 403 error during push + +A user attempted to enable an S3-backed Registry. The `docker login` step went +fine. However, when pushing an image, the output showed: + +``` +The push refers to a repository [s3-testing.myregistry.com:4567/root/docker-test] +dc5e59c14160: Pushing [==================================================>] 14.85 kB +03c20c1a019a: Pushing [==================================================>] 2.048 kB +a08f14ef632e: Pushing [==================================================>] 2.048 kB +228950524c88: Pushing 2.048 kB +6a8ecde4cc03: Pushing [==> ] 9.901 MB/205.7 MB +5f70bf18a086: Pushing 1.024 kB +737f40e80b7f: Waiting +82b57dbc5385: Waiting +19429b698a22: Waiting +9436069b92a3: Waiting +error parsing HTTP 403 response body: unexpected end of JSON input: "" +``` + +This error is ambiguous, as it's not clear whether the 403 is coming from the +GitLab Rails application, the Docker Registry, or something else. In this +case, since we know that since the login succeeded, we probably need to look +at the communication between the client and the Registry. + +The REST API between the Docker client and Registry is [described +here](https://docs.docker.com/registry/spec/api/). Normally, one would just +use Wireshark or tcpdump to capture the traffic and see where things went +wrong. However, since all communication between Docker clients and servers +are done over HTTPS, it's a bit difficult to decrypt the traffic quickly even +if you know the private key. What can we do instead? + +One way would be to disable HTTPS by setting up an [insecure +Registry](https://docs.docker.com/registry/insecure/). This could introduce a +security hole and is only recommended for local testing. If you have a +production system and can't or don't want to do this, there is another way: +use mitmproxy, which stands for Man-in-the-Middle Proxy. + +#### mitmproxy + +[mitmproxy](https://mitmproxy.org/) allows you to place a proxy between your +client and server to inspect all traffic. One wrinkle is that your system +needs to trust the mitmproxy SSL certificates for this to work. + +The following installation instructions assume you are running Ubuntu: + +1. Install mitmproxy (see http://docs.mitmproxy.org/en/stable/install.html) +1. Run `mitmproxy --port 9000` to generate its certificates. + Enter <kbd>CTRL</kbd>-<kbd>C</kbd> to quit. +1. Install the certificate from `~/.mitmproxy` to your system: + + ```sh + sudo cp ~/.mitmproxy/mitmproxy-ca-cert.pem /usr/local/share/ca-certificates/mitmproxy-ca-cert.crt + sudo update-ca-certificates + ``` + +If successful, the output should indicate that a certificate was added: + +```sh +Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done. +Running hooks in /etc/ca-certificates/update.d....done. +``` + +To verify that the certificates are properly installed, run: + +```sh +mitmproxy --port 9000 +``` + +This will run mitmproxy on port `9000`. In another window, run: + +```sh +curl --proxy http://localhost:9000 https://httpbin.org/status/200 +``` + +If everything is setup correctly, you will see information on the mitmproxy window and +no errors from the curl commands. + +#### Running the Docker daemon with a proxy + +For Docker to connect through a proxy, you must start the Docker daemon with the +proper environment variables. The easiest way is to shutdown Docker (e.g. `sudo initctl stop docker`) +and then run Docker by hand. As root, run: + +```sh +export HTTP_PROXY="http://localhost:9000" +export HTTPS_PROXY="https://localhost:9000" +docker daemon --debug +``` + +This will launch the Docker daemon and proxy all connections through mitmproxy. + +#### Running the Docker client + +Now that we have mitmproxy and Docker running, we can attempt to login and push +a container image. You may need to run as root to do this. For example: + +```sh +docker login s3-testing.myregistry.com:4567 +docker push s3-testing.myregistry.com:4567/root/docker-test +``` + +In the example above, we see the following trace on the mitmproxy window: + + + +The above image shows: + +* The initial PUT requests went through fine with a 201 status code. +* The 201 redirected the client to the S3 bucket. +* The HEAD request to the AWS bucket reported a 403 Unauthorized. + +What does this mean? This strongly suggests that the S3 user does not have the right +[permissions to perform a HEAD request](http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectHEAD.html). +The solution: check the [IAM permissions again](https://docs.docker.com/registry/storage-drivers/s3/). +Once the right permissions were set, the error will go away. + +[ce-4040]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4040 +[docker-docs]: https://docs.docker.com/engine/userguide/intro/ +[private-docker]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry diff --git a/doc/user/project/cycle_analytics.md b/doc/user/project/cycle_analytics.md new file mode 100644 index 00000000000..1892ccabb70 --- /dev/null +++ b/doc/user/project/cycle_analytics.md @@ -0,0 +1,170 @@ +# Cycle Analytics + +> [Introduced][ce-5986] in GitLab 8.12. +> +> **Note:** +There are more changes coming to Cycle Analytics, you can follow the following +issue to track the changes to this feature: [#20975][ce-20975]. + +Cycle Analytics measures the time it takes to go from an [idea to production] for +each project you have. This is achieved by not only indicating the total time it +takes to reach at that point, but the total time is broken down into the +multiple stages an idea has to pass through to be shipped. + +Cycle Analytics is that it is tightly coupled with the [GitLab flow] and +calculates a separate median for each stage. + +## Overview + +You can find the Cycle Analytics page under your project's **Pipelines > Cycle +Analytics** tab. + + + +You can see that there are seven stages in total: + +- **Issue** (Tracker) + - Median time from issue creation until given a milestone or list label + (first assignment, any milestone, milestone date or assignee is not required) +- **Plan** (Board) + - Median time from giving an issue a milestone or label until pushing the + first commit to the branch +- **Code** (IDE) + - Median time from the first commit to the branch until the merge request is + created +- **Test** (CI) + - Median total test time for all commits/merges +- **Review** (Merge Request/MR) + - Median time from merge request creation until the merge request is merged + (closed merge requests won't be taken into account) +- **Staging** (Continuous Deployment) + - Median time from when the merge request got merged until the deploy to + production (production is last stage/environment) +- **Production** (Total) + - Sum of all the above stages' times excluding the Test (CI) time. To clarify, + it's not so much that CI time is "excluded", but rather CI time is already + counted in the review stage since CI is done automatically. Most of the + other stages are purely sequential, but **Test** is not. + +## How the data is measured + +Cycle Analytics records cycle time and data based on the project issues with the +exception of the staging and production stages, where only data deployed to +production are measured. + +Specifically, if your CI is not set up and you have not defined a `production` +[environment], then you will not have any data for those stages. + +Below you can see in more detail what the various stages of Cycle Analytics mean. + +| **Stage** | **Description** | +| --------- | --------------- | +| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label will be tracked only if it already has an [Issue Board list][board] created for it. | +| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (e.g., `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. | +| Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern] to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the issue closing pattern is not present in the merge request description, the MR is not considered to the measurement time of the stage. | +| Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. `master` is not excluded. It does not attempt to track time for any particular stages. | +| Review | Measures the median time taken to review the merge request, between its creation and until it's merged. | +| Staging | Measures the median time between merging the merge request until the very first deployment to production. It's tracked by the [environment] set to `production` (case-sensitive, `Production` won't work) in your GitLab CI configuration. If there isn't a `production` environment, this is not tracked. | +| Production| The sum of all time (medians) taken to run the entire process, from issue creation to deploying the code to production. | + +--- + +Here's a little explanation of how this works behind the scenes: + +1. Issues and merge requests are grouped together in pairs, such that for each + `<issue, merge request>` pair, the merge request has the [issue closing pattern] + for the corresponding issue. All other issues and merge requests are **not** + considered. +1. Then the <issue, merge request> pairs are filtered out by last XX days (specified + by the UI - default is 90 days). So it prohibits these pairs from being considered. +1. For the remaining `<issue, merge request>` pairs, we check the information that + we need for the stages, like issue creation date, merge request merge time, + etc. + +To sum up, anything that doesn't follow the [GitLab flow] won't be tracked at all. +So, if a merge request doesn't close an issue or an issue is not labeled with a +label present in the Issue Board or assigned a milestone or a project has no +`production` environment (for staging and production stages), the Cycle Analytics +dashboard won't present any data at all. + +## Example workflow + +Below is a simple fictional workflow of a single cycle that happens in a +single day passing through all seven stages. Note that if a stage does not have +a start/stop mark, it is not measured and hence not calculated in the median +time. It is assumed that milestones are created and CI for testing and setting +environments is configured. + +1. Issue is created at 09:00 (start of **Issue** stage). +1. Issue is added to a milestone at 11:00 (stop of **Issue** stage / start of + **Plan** stage). +1. Start working on the issue, create a branch locally and make one commit at + 12:00. +1. Make a second commit to the branch which mentions the issue number at 12.30 + (stop of **Plan** stage / start of **Code** stage). +1. Push branch and create a merge request that contains the [issue closing pattern] + in its description at 14:00 (stop of **Code** stage / start of **Test** and + **Review** stages). +1. The CI starts running your scripts defined in [`.gitlab-ci.yml`][yml] and + takes 5min (stop of **Test** stage). +1. Review merge request, ensure that everything is OK and merge the merge + request at 19:00. (stop of **Review** stage / start of **Staging** stage). +1. Now that the merge request is merged, a deployment to the `production` + environment starts and finishes at 19:30 (stop of **Staging** stage). +1. The cycle completes and the sum of the median times of the previous stages + is recorded to the **Production** stage. That is the time between creating an + issue and deploying its relevant merge request to production. + +From the above example you can conclude the time it took each stage to complete +as long as their total time: + +- **Issue**: 2h (11:00 - 09:00) +- **Plan**: 1h (12:00 - 11:00) +- **Code**: 2h (14:00 - 12:00) +- **Test**: 5min +- **Review**: 5h (19:00 - 14:00) +- **Staging**: 30min (19:30 - 19:00) +- **Production**: Since this stage measures the sum of median time off all + previous stages, we cannot calculate it if we don't know the status of the + stages before. In case this is the very first cycle that is run in the project, + then the **Production** time is 10h 30min (19:30 - 09:00) + +A few notes: + +- In the above example we demonstrated that it doesn't matter if your first + commit doesn't mention the issue number, you can do this later in any commit + of the branch you are working on. +- You can see that the **Test** stage is not calculated to the overall time of + the cycle since it is included in the **Review** process (every MR should be + tested). +- The example above was just **one cycle** of the seven stages. Add multiple + cycles, calculate their median time and the result is what the dashboard of + Cycle Analytics is showing. + +## Permissions + +The current permissions on the Cycle Analytics dashboard are: + +- Public projects - anyone can access +- Private/internal projects - any member (guest level and above) can access + +You can [read more about permissions][permissions] in general. + +## More resources + +Learn more about Cycle Analytics in the following resources: + +- [Cycle Analytics feature page](https://about.gitlab.com/solutions/cycle-analytics/) +- [Cycle Analytics feature preview](https://about.gitlab.com/2016/09/16/feature-preview-introducing-cycle-analytics/) +- [Cycle Analytics feature highlight](https://about.gitlab.com/2016/09/21/cycle-analytics-feature-highlight/) + + +[board]: issue_board.md#creating-a-new-list +[ce-5986]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5986 +[ce-20975]: https://gitlab.com/gitlab-org/gitlab-ce/issues/20975 +[environment]: ../../ci/yaml/README.md#environment +[GitLab flow]: ../../workflow/gitlab_flow.md +[idea to production]: https://about.gitlab.com/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab +[issue closing pattern]: issues/automatic_issue_closing.md +[permissions]: ../permissions.md +[yml]: ../../ci/yaml/README.md diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md new file mode 100644 index 00000000000..ea7496af089 --- /dev/null +++ b/doc/user/project/description_templates.md @@ -0,0 +1,42 @@ +# Description templates + +>[Introduced][ce-4981] in GitLab 8.11. + +Description templates allow you to define context-specific templates for issue +and merge request description fields for your project. + +## Overview + +By using the description templates, users that create a new issue or merge +request can select a description template to help them communicate with other +contributors effectively. + +Every GitLab project can define its own set of description templates as they +are added to the root directory of a GitLab project's repository. + +Description templates must be written in [Markdown](../markdown.md) and stored +in your project's repository under a directory named `.gitlab`. Only the +templates of the default branch will be taken into account. + +## Creating issue templates + +Create a new Markdown (`.md`) file inside the `.gitlab/issue_templates/` +directory in your repository. Commit and push to your default branch. + +## Creating merge request templates + +Similarly to issue templates, create a new Markdown (`.md`) file inside the +`.gitlab/merge_request_templates/` directory in your repository. Commit and +push to your default branch. + +## Using the templates + +Let's take for example that you've created the file `.gitlab/issue_templates/Bug.md`. +This will enable the `Bug` dropdown option when creating or editing issues. When +`Bug` is selected, the content from the `Bug.md` template file will be copied +to the issue description field. The 'Reset template' button will discard any +changes you made after picking the template and return it to its initial status. + + + +[ce-4981]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4981 diff --git a/doc/user/project/git_attributes.md b/doc/user/project/git_attributes.md new file mode 100644 index 00000000000..21ef94e61f7 --- /dev/null +++ b/doc/user/project/git_attributes.md @@ -0,0 +1,22 @@ +# Git Attributes + +GitLab supports defining custom [Git attributes][gitattributes] such as what +files to treat as binary, and what language to use for syntax highlighting +diffs. + +To define these attributes, create a file called `.gitattributes` in the root +directory of your repository and push it to the default branch of your project. + +## Encoding Requirements + +The `.gitattributes` file _must_ be encoded in UTF-8 and _must not_ contain a +Byte Order Mark. If a different encoding is used, the file's contents will be +ignored. + +## Syntax Highlighting + +The `.gitattributes` file can be used to define which language to use when +syntax highlighting files and diffs. See ["Syntax +Highlighting"](highlighting.md) for more information. + +[gitattributes]: https://git-scm.com/docs/gitattributes diff --git a/doc/user/project/img/container_registry_enable.png b/doc/user/project/img/container_registry_enable.png Binary files differnew file mode 100644 index 00000000000..6fffa2a91d8 --- /dev/null +++ b/doc/user/project/img/container_registry_enable.png diff --git a/doc/user/project/img/container_registry_panel.png b/doc/user/project/img/container_registry_panel.png Binary files differnew file mode 100644 index 00000000000..60fd76192b7 --- /dev/null +++ b/doc/user/project/img/container_registry_panel.png diff --git a/doc/user/project/img/container_registry_tab.png b/doc/user/project/img/container_registry_tab.png Binary files differnew file mode 100644 index 00000000000..36b883aaa97 --- /dev/null +++ b/doc/user/project/img/container_registry_tab.png diff --git a/doc/user/project/img/cycle_analytics_landing_page.png b/doc/user/project/img/cycle_analytics_landing_page.png Binary files differnew file mode 100644 index 00000000000..b212134d5ed --- /dev/null +++ b/doc/user/project/img/cycle_analytics_landing_page.png diff --git a/doc/user/project/img/description_templates.png b/doc/user/project/img/description_templates.png Binary files differnew file mode 100644 index 00000000000..c41cc77a94c --- /dev/null +++ b/doc/user/project/img/description_templates.png diff --git a/doc/user/project/img/issue_board.png b/doc/user/project/img/issue_board.png Binary files differnew file mode 100644 index 00000000000..63c269f6dbc --- /dev/null +++ b/doc/user/project/img/issue_board.png diff --git a/doc/user/project/img/issue_board_add_list.png b/doc/user/project/img/issue_board_add_list.png Binary files differnew file mode 100644 index 00000000000..2b8c10eaa0a --- /dev/null +++ b/doc/user/project/img/issue_board_add_list.png diff --git a/doc/user/project/img/issue_board_search_backlog.png b/doc/user/project/img/issue_board_search_backlog.png Binary files differnew file mode 100644 index 00000000000..112ea171539 --- /dev/null +++ b/doc/user/project/img/issue_board_search_backlog.png diff --git a/doc/user/project/img/issue_board_system_notes.png b/doc/user/project/img/issue_board_system_notes.png Binary files differnew file mode 100644 index 00000000000..b69ef034954 --- /dev/null +++ b/doc/user/project/img/issue_board_system_notes.png diff --git a/doc/user/project/img/issue_board_welcome_message.png b/doc/user/project/img/issue_board_welcome_message.png Binary files differnew file mode 100644 index 00000000000..b757faeb230 --- /dev/null +++ b/doc/user/project/img/issue_board_welcome_message.png diff --git a/doc/user/project/img/koding_build-in-progress.png b/doc/user/project/img/koding_build-in-progress.png Binary files differnew file mode 100644 index 00000000000..f8cc81834c4 --- /dev/null +++ b/doc/user/project/img/koding_build-in-progress.png diff --git a/doc/user/project/img/koding_build-logs.png b/doc/user/project/img/koding_build-logs.png Binary files differnew file mode 100644 index 00000000000..a04cd5aff99 --- /dev/null +++ b/doc/user/project/img/koding_build-logs.png diff --git a/doc/user/project/img/koding_build-success.png b/doc/user/project/img/koding_build-success.png Binary files differnew file mode 100644 index 00000000000..2a0dd296480 --- /dev/null +++ b/doc/user/project/img/koding_build-success.png diff --git a/doc/user/project/img/koding_commit-koding.yml.png b/doc/user/project/img/koding_commit-koding.yml.png Binary files differnew file mode 100644 index 00000000000..3e133c50327 --- /dev/null +++ b/doc/user/project/img/koding_commit-koding.yml.png diff --git a/doc/user/project/img/koding_different-stack-on-mr-try.png b/doc/user/project/img/koding_different-stack-on-mr-try.png Binary files differnew file mode 100644 index 00000000000..fd25e32f648 --- /dev/null +++ b/doc/user/project/img/koding_different-stack-on-mr-try.png diff --git a/doc/user/project/img/koding_edit-on-ide.png b/doc/user/project/img/koding_edit-on-ide.png Binary files differnew file mode 100644 index 00000000000..fd5aaff75f5 --- /dev/null +++ b/doc/user/project/img/koding_edit-on-ide.png diff --git a/doc/user/project/img/koding_enable-koding.png b/doc/user/project/img/koding_enable-koding.png Binary files differnew file mode 100644 index 00000000000..c0ae0ee9918 --- /dev/null +++ b/doc/user/project/img/koding_enable-koding.png diff --git a/doc/user/project/img/koding_landing.png b/doc/user/project/img/koding_landing.png Binary files differnew file mode 100644 index 00000000000..7c629d9b05e --- /dev/null +++ b/doc/user/project/img/koding_landing.png diff --git a/doc/user/project/img/koding_open-gitlab-from-koding.png b/doc/user/project/img/koding_open-gitlab-from-koding.png Binary files differnew file mode 100644 index 00000000000..c958cf8f224 --- /dev/null +++ b/doc/user/project/img/koding_open-gitlab-from-koding.png diff --git a/doc/user/project/img/koding_run-in-ide.png b/doc/user/project/img/koding_run-in-ide.png Binary files differnew file mode 100644 index 00000000000..f91ee0f74cc --- /dev/null +++ b/doc/user/project/img/koding_run-in-ide.png diff --git a/doc/user/project/img/koding_run-mr-in-ide.png b/doc/user/project/img/koding_run-mr-in-ide.png Binary files differnew file mode 100644 index 00000000000..502817a2a46 --- /dev/null +++ b/doc/user/project/img/koding_run-mr-in-ide.png diff --git a/doc/user/project/img/koding_set-up-ide.png b/doc/user/project/img/koding_set-up-ide.png Binary files differnew file mode 100644 index 00000000000..7f408c980b5 --- /dev/null +++ b/doc/user/project/img/koding_set-up-ide.png diff --git a/doc/user/project/img/koding_stack-import.png b/doc/user/project/img/koding_stack-import.png Binary files differnew file mode 100644 index 00000000000..2a4e3c87fc8 --- /dev/null +++ b/doc/user/project/img/koding_stack-import.png diff --git a/doc/user/project/img/koding_start-build.png b/doc/user/project/img/koding_start-build.png Binary files differnew file mode 100644 index 00000000000..52159440f62 --- /dev/null +++ b/doc/user/project/img/koding_start-build.png diff --git a/doc/user/project/img/mitmproxy-docker.png b/doc/user/project/img/mitmproxy-docker.png Binary files differnew file mode 100644 index 00000000000..4e3e37b413d --- /dev/null +++ b/doc/user/project/img/mitmproxy-docker.png diff --git a/doc/user/project/img/project_settings_list.png b/doc/user/project/img/project_settings_list.png Binary files differnew file mode 100644 index 00000000000..57ca2ac5f9e --- /dev/null +++ b/doc/user/project/img/project_settings_list.png diff --git a/doc/user/project/img/protected_branches_choose_branch.png b/doc/user/project/img/protected_branches_choose_branch.png Binary files differnew file mode 100644 index 00000000000..26328143717 --- /dev/null +++ b/doc/user/project/img/protected_branches_choose_branch.png diff --git a/doc/user/project/img/protected_branches_devs_can_push.png b/doc/user/project/img/protected_branches_devs_can_push.png Binary files differnew file mode 100644 index 00000000000..812cc8767b7 --- /dev/null +++ b/doc/user/project/img/protected_branches_devs_can_push.png diff --git a/doc/user/project/img/protected_branches_error_ui.png b/doc/user/project/img/protected_branches_error_ui.png Binary files differnew file mode 100644 index 00000000000..cc61df7ca97 --- /dev/null +++ b/doc/user/project/img/protected_branches_error_ui.png diff --git a/doc/user/project/img/protected_branches_list.png b/doc/user/project/img/protected_branches_list.png Binary files differnew file mode 100644 index 00000000000..f33f1b2bdb6 --- /dev/null +++ b/doc/user/project/img/protected_branches_list.png diff --git a/doc/user/project/img/protected_branches_matches.png b/doc/user/project/img/protected_branches_matches.png Binary files differnew file mode 100644 index 00000000000..30ce53f704e --- /dev/null +++ b/doc/user/project/img/protected_branches_matches.png diff --git a/doc/user/project/img/protected_branches_page.png b/doc/user/project/img/protected_branches_page.png Binary files differnew file mode 100644 index 00000000000..1585dde5b29 --- /dev/null +++ b/doc/user/project/img/protected_branches_page.png diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md new file mode 100644 index 00000000000..4a6c0d88241 --- /dev/null +++ b/doc/user/project/issue_board.md @@ -0,0 +1,188 @@ +# Issue board + +> [Introduced][ce-5554] in GitLab 8.11. + +The GitLab Issue Board is a software project management tool used to plan, +organize, and visualize a workflow for a feature or product release. +It can be seen like a light version of a [Kanban] or a [Scrum] board. + +Other interesting links: + +- [GitLab Issue Board landing page on about.gitlab.com][landing] +- [YouTube video introduction to Issue Boards][youtube] + +## Overview + +The Issue Board builds on GitLab's existing issue tracking functionality and +leverages the power of [labels] by utilizing them as lists of the scrum board. + +With the Issue Board you can have a different view of your issues while also +maintaining the same filtering and sorting abilities you see across the +issue tracker. + +Below is a table of the definitions used for GitLab's Issue Board. + +| What we call it | What it means | +| -------------- | ------------- | +| **Issue Board** | It represents a different view for your issues. It can have multiple lists with each list consisting of issues represented by cards. | +| **List** | Each label that exists in the issue tracker can have its own dedicated list. Every list is named after the label it is based on and is represented by a column which contains all the issues associated with that label. You can think of a list like the results you get when you filter the issues by a label in your issue tracker. | +| **Card** | Every card represents an issue and it is shown under the list for which it has a label. The information you can see on a card consists of the issue number, the issue title, the assignee and the labels associated with it. You can drag cards around from one list to another. Issues inside lists are [ordered by priority](labels.md#prioritize-labels). | + +There are three types of lists, the ones you create based on your labels, and +two default: + +- **Backlog** (default): shows all issues that do not fall in one of the other lists. Always appears on the very left. +- **Done** (default): shows all closed issues. Always appears on the very right. +Label list: a list based on a label. It shows all issues with that label. +- Label list: a list based on a label. It shows all opened issues with that label. + + + +--- + +In short, here's a list of actions you can take in an Issue Board: + +- [Create a new list](#creating-a-new-list). +- [Delete an existing list](#deleting-a-list). +- Drag issues between lists. +- Drag and reorder the lists themselves. +- Change issue labels on-the-fly while dragging issues between lists. +- Close an issue if you drag it to the **Done** list. +- Create a new list from a non-existing label by [creating the label on-the-fly](#creating-a-new-list) + within the Issue Board. +- [Filter issues](#filtering-issues) that appear across your Issue Board. + +If you are not able to perform one or more of the things above, make sure you +have the right [permissions](#permissions). + +## First time using the Issue Board + +The first time you navigate to your Issue Board, you will be presented with the +two default lists (**Backlog** and **Done**) and a welcoming message that gives +you two options. You can either create a predefined set of labels and create +their corresponding lists to the Issue Board or opt-out and use your own lists. + + + +If you choose to use and create the predefined lists, they will appear as empty +because the labels associated to them will not exist up until that moment, +which means the system has no way of populating them automatically. That's of +course if the predefined labels don't already exist. If any of them does exist, +the list will be created and filled with the issues that have that label. + +## Creating a new list + +Create a new list by clicking on the **Create new list** button at the upper +right corner of the Issue Board. + + + +Simply choose the label to create the list from. The new list will be inserted +at the end of the lists, before **Done**. Moving and reordering lists is as +easy as dragging them around. + +To create a list for a label that doesn't yet exist, simply create the label by +choosing **Create new label**. The label will be created on-the-fly and it will +be immediately added to the dropdown. You can now choose it to create a list. + +## Deleting a list + +To delete a list from the Issue Board use the small trash icon that is present +in the list's heading. A confirmation dialog will appear for you to confirm. + +Deleting a list doesn't have any effect in issues and labels, it's just the +list view that is removed. You can always add it back later if you need. + +## Searching issues in the Backlog list + +The very first time you start using the Issue Board, it is very likely your +issue tracker is already populated with labels and issues. In that case, +**Backlog** will have all the issues that don't belong to another list, and +**Done** will have all the closed ones. + +For performance and visibility reasons, each list shows the first 20 issues +by default. If you have more than 20, you have to start scrolling down for the +next 20 issues to appear. This can be cumbersome if your issue tracker hosts +hundreds of issues, and for that reason it is easier to search for issues to +move from **Backlog** to another list. + +Start typing in the search bar under the **Backlog** list and the relevant +issues will appear. + + + +## Filtering issues + +You should be able to use the filters on top of your Issue Board to show only +the results you want. This is similar to the filtering used in the issue tracker +since the metadata from the issues and labels are re-used in the Issue Board. + +You can filter by author, assignee, milestone and label. + +## Creating workflows + +By reordering your lists, you can create workflows. As lists in Issue Boards are +based on labels, it works out of the box with your existing issues. So if you've +already labeled things with 'Backend' and 'Frontend', the issue will appear in +the lists as you create them. In addition, this means you can easily move +something between lists by changing a label. + +A typical workflow of using the Issue Board would be: + +1. You have [created][create-labels] and [prioritized][label-priority] labels + so that you can easily categorize your issues. +1. You have a bunch of issues (ideally labeled). +1. You visit the Issue Board and start [creating lists](#creating-a-new-list) to + create a workflow. +1. You move issues around in lists so that your team knows who should be working + on what issue. +1. When the work by one team is done, the issue can be dragged to the next list + so someone else can pick up. +1. When the issue is finally resolved, the issue is moved to the **Done** list + and gets automatically closed. + +For instance you can create a list based on the label of 'Frontend' and one for +'Backend'. A designer can start working on an issue by dragging it from +**Backlog** to 'Frontend'. That way, everyone knows that this issue is now being +worked on by the designers. Then, once they're done, all they have to do is +drag it over to the next list, 'Backend', where a backend developer can +eventually pick it up. Once they’re done, they move it to **Done**, to close the +issue. + +This process can be seen clearly when visiting an issue since with every move +to another list the label changes and a system not is recorded. + + + +## Permissions + +[Developers and up](../permissions.md) can use all the functionality of the +Issue Board, that is create/delete lists and drag issues around. + +## Tips + +A few things to remember: + +- The label that corresponds to a list is hidden for issues under that list. +- Moving an issue between lists removes the label from the list it came from + and adds the label from the list it goes to. +- When moving a card to **Done**, the label of the list it came from is removed + and the issue gets closed. +- An issue can exist in multiple lists if it has more than one label. +- Lists are populated with issues automatically if the issues are labeled. +- Clicking on the issue title inside a card will take you to that issue. +- Clicking on a label inside a card will quickly filter the entire Issue Board + and show only the issues from all lists that have that label. +- Issues inside lists are [ordered by priority][label-priority]. +- For performance and visibility reasons, each list shows the first 20 issues + by default. If you have more than 20 issues start scrolling down and the next + 20 will appear. + +[ce-5554]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5554 +[labels]: ./labels.md +[scrum]: https://en.wikipedia.org/wiki/Scrum_(software_development) +[kanban]: https://en.wikipedia.org/wiki/Kanban_(development) +[create-labels]: ./labels.md#create-new-labels +[label-priority]: ./labels.md#prioritize-labels +[landing]: https://about.gitlab.com/solutions/issueboard +[youtube]: https://www.youtube.com/watch?v=UWsJ8tkHAa8 diff --git a/doc/user/project/issues/automatic_issue_closing.md b/doc/user/project/issues/automatic_issue_closing.md new file mode 100644 index 00000000000..d6f3a7d5555 --- /dev/null +++ b/doc/user/project/issues/automatic_issue_closing.md @@ -0,0 +1,55 @@ +# Automatic issue closing + +>**Note:** +This is the user docs. In order to change the default issue closing pattern, +follow the steps in the [administration docs]. + +When a commit or merge request resolves one or more issues, it is possible to +automatically have these issues closed when the commit or merge request lands +in the project's default branch. + +If a commit message or merge request description contains a sentence matching +a certain regular expression, all issues referenced from the matched text will +be closed. This happens when the commit is pushed to a project's **default** +branch, or when a commit or merge request is merged into it. + +## Default closing pattern value + +When not specified, the default issue closing pattern as shown below will be +used: + +```bash +((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing))(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) +``` + +Note that `%{issue_ref}` is a complex regular expression defined inside GitLab's +source code that can match a reference to 1) a local issue (`#123`), +2) a cross-project issue (`group/project#123`) or 3) a link to an issue +(`https://gitlab.example.com/group/project/issues/123`). + +--- + +This translates to the following keywords: + +- Close, Closes, Closed, Closing, close, closes, closed, closing +- Fix, Fixes, Fixed, Fixing, fix, fixes, fixed, fixing +- Resolve, Resolves, Resolved, Resolving, resolve, resolves, resolved, resolving + +--- + +For example the following commit message: + +``` +Awesome commit message + +Fix #20, Fixes #21 and Closes group/otherproject#22. +This commit is also related to #17 and fixes #18, #19 +and https://gitlab.example.com/group/otherproject/issues/23. +``` + +will close `#18`, `#19`, `#20`, and `#21` in the project this commit is pushed +to, as well as `#22` and `#23` in group/otherproject. `#17` won't be closed as +it does not match the pattern. It works with multi-line commit messages as well +as one-liners when used with `git commit -m`. + +[administration docs]: ../../../administration/issue_closing_pattern.md diff --git a/doc/user/project/koding.md b/doc/user/project/koding.md new file mode 100644 index 00000000000..c56a1efe3c2 --- /dev/null +++ b/doc/user/project/koding.md @@ -0,0 +1,128 @@ +# Koding & GitLab + +> [Introduced][ce-5909] in GitLab 8.11. + +This document will guide you through using Koding integration on GitLab in +detail. For configuring and installing please follow the +[administrator guide](../../administration/integration/koding.md). + +You can use Koding integration to run and develop your projects on GitLab. This +will allow you and the users to test your project without leaving the browser. +Koding handles projects as stacks which are basic recipes to define your +environment for your project. With this integration you can automatically +create a proper stack template for your projects. Currently auto-generated +stack templates are designed to work with AWS which requires a valid AWS +credential to be able to use these stacks. You can find more information about +stacks and the other providers that you can use on Koding following the +[Koding documentation][koding-docs]. + +## Enable Integration + +You can enable Koding integration by providing the running Koding instance URL +in Application Settings under **Admin area > Settings** (`/admin/application_settings`). + + + +Once enabled you will see `Koding` link on your sidebar which leads you to +Koding Landing page. + + + +You can navigate to running Koding instance from here. For more information and +details about configuring the integration, please follow the +[administrator guide](../../administration/integration/koding.md). + +## Set up Koding on Projects + +Once it's enabled, you will see some integration buttons on Project pages, +Merge Requests etc. To get started working on a specific project you first need +to create a `.koding.yml` file under your project root. You can easily do that +by using `Set Up Koding` button which will be visible on every project's +landing page; + + + +Once you click this will open a New File page on GitLab with auto-generated +`.koding.yml` content based on your server and repository configuration. + + + + +## Run a project on Koding + +If there is `.koding.yml` exists in your project root, you will see +`Run in IDE (Koding)` button in your project landing page. You can initiate the +process from here. + + + +This will open Koding defined in the settings in a new window and will start +importing the project's stack file. + + + +You should see the details of your repository imported into your Koding +instance. Once it's completed it will lead you to the Stack Editor and from +there you can start using your new stack integrated with your project on your +GitLab instance. For details about what's next you can follow +[this guide](https://www.koding.com/docs/creating-an-aws-stack) from step 8. + +Once stack initialized you will see the `README.md` content from your project +in `Stack Build` wizard, this wizard will let you build the stack and import +your project into it. **Once it's completed it will automatically open the +related vm instead of importing from scratch**. + + + +This will take time depending on the required environment. + + + +It usually takes ~4 min. to make it ready with a `t2.nano` instance on given +AWS region. (`t2.nano` is default vm type on auto-generated stack template +which can be manually changed). + + + +You can check out the `Build Logs` from this success modal as well. + + + +You can now `Start Coding`! + + + +## Try a Merge Request on IDE + +It's also possible to try a change on IDE before merging it. This flow only +enabled if the target project has `.koding.yml` in it's target branch. You +should see the alternative version of `Run in IDE (Koding)` button in merge +request pages as well; + + + +This will again take you to Koding with proper arguments passed, which will +allow Koding to modify the stack template provided by target branch. You can +see the difference; + + + +The flow for the branch stack is also same with the regular project flow. + +## Open GitLab from Koding + +Since stacks generated with import flow defined in previous steps, they have +information about the repository they are belonging to. By using this +information you can access to related GitLab page from stacks on your sidebar +on Koding. + + + +## Other links + +- [YouTube video on GitLab + Koding workflow][youtube] +- [Koding documentation][koding-docs] + +[ce-5909]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5909 +[youtube]: https://youtu.be/3wei5yv_Ye8 +[koding-docs]: https://www.koding.com/docs diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 4258185b7d0..cf1d9cbe69c 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -1,8 +1,8 @@ # Labels Labels provide an easy way to categorize the issues or merge requests based on -descriptive titles like `bug`, `documentation` or any other text you feel like -it. They can have different colors, a description, and are visible throughout +descriptive titles like `bug`, `documentation` or any other text you feel like. +They can have different colors, a description, and are visible throughout the issue tracker or inside each issue individually. With labels, you can navigate the issue tracker and filter any bloated @@ -22,34 +22,47 @@ created yet.  +Creating a new label from scratch is as easy as pressing the **New label** +button. From there on you can choose the name, give it an optional description, +a color and you are set. + +When you are ready press the **Create label** button to create the new label. + + + --- -You can skip that and create a new label or click that link and GitLab will -generate a set of predefined labels for you. There 8 default generated labels +## Default Labels + +It's possible to populate the labels for your project from a set of predefined labels. + +### Generate GitLab's predefined label set + + + +Click the link to 'Generate a default set of labels' and GitLab will +generate a set of predefined labels for you. There are 8 default generated labels in total and you can see them in the screenshot below.  --- -You can see that from the labels page you can have an overview of the number of -issues and merge requests assigned to each label. - -Creating a new label from scratch is as easy as pressing the **New label** -button. From there on you can choose the name, give it an optional description, -a color and you are set. +## Labels Overview -When you are ready press the **Create label** button to create the new label. + - +You can see that from the labels page you can have an overview of the number of +issues and merge requests assigned to each label. ## Prioritize labels >**Notes:** - - This feature was introduced in GitLab 8.9. - - Priority sorting is based on the highest priority label only. This might - change in the future, follow the discussion in - https://gitlab.com/gitlab-org/gitlab-ce/issues/18554. +> +> - Introduced in GitLab 8.9. +> - Priority sorting is based on the highest priority label only. This might +> change in the future, follow the discussion in +> https://gitlab.com/gitlab-org/gitlab-ce/issues/18554. Prioritized labels are like any other label, but sorted by priority. This allows you to sort issues and merge requests by priority. @@ -87,8 +100,7 @@ important. ## Create a new label right from the issue tracker ->**Note:** -This feature was introduced in GitLab 8.6. +> Introduced in GitLab 8.6. There are times when you are already in the issue tracker searching for a label, only to realize it doesn't exist. Instead of going to the **Labels** diff --git a/doc/user/project/merge_requests.md b/doc/user/project/merge_requests.md new file mode 100644 index 00000000000..5af9a5d049c --- /dev/null +++ b/doc/user/project/merge_requests.md @@ -0,0 +1,169 @@ +# Merge Requests + +Merge requests allow you to exchange changes you made to source code and +collaborate with other people on the same project. + +## Authorization for merge requests + +There are two main ways to have a merge request flow with GitLab: + +1. Working with [protected branches][] in a single repository +1. Working with forks of an authoritative project + +[Learn more about the authorization for merge requests.](merge_requests/authorization_for_merge_requests.md) + +## Cherry-pick changes + +Cherry-pick any commit in the UI by simply clicking the **Cherry-pick** button +in a merged merge requests or a commit. + +[Learn more about cherry-picking changes.](merge_requests/cherry_pick_changes.md) + +## Merge when build succeeds + +When reviewing a merge request that looks ready to merge but still has one or +more CI builds running, you can set it to be merged automatically when all +builds succeed. This way, you don't have to wait for the builds to finish and +remember to merge the request manually. + +[Learn more about merging when build succeeds.](merge_requests/merge_when_build_succeeds.md) + +## Resolve discussion comments in merge requests reviews + +Keep track of the progress during a code review with resolving comments. +Resolving comments prevents you from forgetting to address feedback and lets +you hide discussions that are no longer relevant. + +[Read more about resolving discussion comments in merge requests reviews.](merge_requests/merge_request_discussion_resolution.md) + +## Resolve conflicts + +When a merge request has conflicts, GitLab may provide the option to resolve +those conflicts in the GitLab UI. + +[Learn more about resolving merge conflicts in the UI.](merge_requests/resolve_conflicts.md) + +## Revert changes + +GitLab implements Git's powerful feature to revert any commit with introducing +a **Revert** button in merge requests and commit details. + +[Learn more about reverting changes in the UI](merge_requests/revert_changes.md) + +## Merge requests versions + +Every time you push to a branch that is tied to a merge request, a new version +of merge request diff is created. When you visit a merge request that contains +more than one pushes, you can select and compare the versions of those merge +request diffs. + +[Read more about the merge requests versions.](merge_requests/versions.md) + +## Work In Progress merge requests + +To prevent merge requests from accidentally being accepted before they're +completely ready, GitLab blocks the "Accept" button for merge requests that +have been marked as a **Work In Progress**. + +[Learn more about settings a merge request as "Work In Progress".](merge_requests/work_in_progress_merge_requests.md) + +## Ignore whitespace changes in Merge Request diff view + +If you click the **Hide whitespace changes** button, you can see the diff +without whitespace changes (if there are any). This is also working when on a +specific commit page. + + + +>**Tip:** +You can append `?w=1` while on the diffs page of a merge request to ignore any +whitespace changes. + +## Tips + +Here are some tips that will help you be more efficient with merge requests in +the command line. + +> **Note:** +This section might move in its own document in the future. + +### Checkout merge requests locally + +A merge request contains all the history from a repository, plus the additional +commits added to the branch associated with the merge request. Here's a few +tricks to checkout a merge request locally. + +Please note that you can checkout a merge request locally even if the source +project is a fork (even a private fork) of the target project. + +#### Checkout locally by adding a git alias + +Add the following alias to your `~/.gitconfig`: + +``` +[alias] + mr = !sh -c 'git fetch $1 merge-requests/$2/head:mr-$1-$2 && git checkout mr-$1-$2' - +``` + +Now you can check out a particular merge request from any repository and any +remote. For example, to check out the merge request with ID 5 as shown in GitLab +from the `upstream` remote, do: + +``` +git mr upstream 5 +``` + +This will fetch the merge request into a local `mr-upstream-5` branch and check +it out. + +#### Checkout locally by modifying `.git/config` for a given repository + +Locate the section for your GitLab remote in the `.git/config` file. It looks +like this: + +``` +[remote "origin"] + url = https://gitlab.com/gitlab-org/gitlab-ce.git + fetch = +refs/heads/*:refs/remotes/origin/* +``` + +You can open the file with: + +``` +git config -e +``` + +Now add the following line to the above section: + +``` +fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/* +``` + +In the end, it should look like this: + +``` +[remote "origin"] + url = https://gitlab.com/gitlab-org/gitlab-ce.git + fetch = +refs/heads/*:refs/remotes/origin/* + fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/* +``` + +Now you can fetch all the merge requests: + +``` +git fetch origin + +... +From https://gitlab.com/gitlab-org/gitlab-ce.git + * [new ref] refs/merge-requests/1/head -> origin/merge-requests/1 + * [new ref] refs/merge-requests/2/head -> origin/merge-requests/2 +... +``` + +And to check out a particular merge request: + +``` +git checkout origin/merge-requests/1 +``` + +[protected branches]: protected_branches.md diff --git a/doc/user/project/merge_requests/authorization_for_merge_requests.md b/doc/user/project/merge_requests/authorization_for_merge_requests.md new file mode 100644 index 00000000000..59b3fe7242c --- /dev/null +++ b/doc/user/project/merge_requests/authorization_for_merge_requests.md @@ -0,0 +1,56 @@ +# Authorization for Merge requests + +There are two main ways to have a merge request flow with GitLab: + +1. Working with [protected branches] in a single repository. +1. Working with forks of an authoritative project. + +## Protected branch flow + +With the protected branch flow everybody works within the same GitLab project. + +The project maintainers get Master access and the regular developers get +Developer access. + +The maintainers mark the authoritative branches as 'Protected'. + +The developers push feature branches to the project and create merge requests +to have their feature branches reviewed and merged into one of the protected +branches. + +By default, only users with Master access can merge changes into a protected +branch. + +**Advantages** + +- Fewer projects means less clutter. +- Developers need to consider only one remote repository. + +**Disadvantages** + +- Manual setup of protected branch required for each new project + +## Forking workflow + +With the forking workflow the maintainers get Master access and the regular +developers get Reporter access to the authoritative repository, which prohibits +them from pushing any changes to it. + +Developers create forks of the authoritative project and push their feature +branches to their own forks. + +To get their changes into master they need to create a merge request across +forks. + +**Advantages** + +- In an appropriately configured GitLab group, new projects automatically get + the required access restrictions for regular developers: fewer manual steps + to configure authorization for new projects. + +**Disadvantages** + +- The project need to keep their forks up to date, which requires more advanced + Git skills (managing multiple remotes). + +[protected branches]: ../protected_branches.md diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md new file mode 100644 index 00000000000..64b94d81024 --- /dev/null +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -0,0 +1,52 @@ +# Cherry-pick changes + +> [Introduced][ce-3514] in GitLab 8.7. + +--- + +GitLab implements Git's powerful feature to [cherry-pick any commit][git-cherry-pick] +with introducing a **Cherry-pick** button in Merge Requests and commit details. + +## Cherry-picking a Merge Request + +After the Merge Request has been merged, a **Cherry-pick** button will be available +to cherry-pick the changes introduced by that Merge Request: + + + +--- + +You can cherry-pick the changes directly into the selected branch or you can opt to +create a new Merge Request with the cherry-pick changes: + + + +## Cherry-picking a Commit + +You can cherry-pick a Commit from the Commit details page: + + + +--- + +Similar to cherry-picking a Merge Request, you can opt to cherry-pick the changes +directly into the target branch or create a new Merge Request to cherry-pick the +changes: + + + +--- + +Please note that when cherry-picking merge commits, the mainline will always be the +first parent. If you want to use a different mainline then you need to do that +from the command line. + +Here is a quick example to cherry-pick a merge commit using the second parent as the +mainline: + +```bash +git cherry-pick -m 2 7a39eb0 +``` + +[ce-3514]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3514 "Cherry-pick button Merge Request" +[git-cherry-pick]: https://git-scm.com/docs/git-cherry-pick "Git cherry-pick documentation" diff --git a/doc/workflow/img/cherry_pick_changes_commit.png b/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png Binary files differindex 7fb68cc9e9b..7fb68cc9e9b 100644 --- a/doc/workflow/img/cherry_pick_changes_commit.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png diff --git a/doc/workflow/img/cherry_pick_changes_commit_modal.png b/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png Binary files differindex 5267e04562f..5267e04562f 100644 --- a/doc/workflow/img/cherry_pick_changes_commit_modal.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png diff --git a/doc/workflow/img/cherry_pick_changes_mr.png b/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png Binary files differindex 975fb13e463..975fb13e463 100644 --- a/doc/workflow/img/cherry_pick_changes_mr.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png diff --git a/doc/workflow/img/cherry_pick_changes_mr_modal.png b/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png Binary files differindex 6c003bacbe3..6c003bacbe3 100644 --- a/doc/workflow/img/cherry_pick_changes_mr_modal.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png diff --git a/doc/workflow/merge_requests/commit_compare.png b/doc/user/project/merge_requests/img/commit_compare.png Binary files differindex 0e4a2b23c04..0e4a2b23c04 100644 --- a/doc/workflow/merge_requests/commit_compare.png +++ b/doc/user/project/merge_requests/img/commit_compare.png diff --git a/doc/user/project/merge_requests/img/conflict_section.png b/doc/user/project/merge_requests/img/conflict_section.png Binary files differnew file mode 100644 index 00000000000..842e50b14b2 --- /dev/null +++ b/doc/user/project/merge_requests/img/conflict_section.png diff --git a/doc/user/project/merge_requests/img/discussion_view.png b/doc/user/project/merge_requests/img/discussion_view.png Binary files differnew file mode 100644 index 00000000000..83bb60acce2 --- /dev/null +++ b/doc/user/project/merge_requests/img/discussion_view.png diff --git a/doc/user/project/merge_requests/img/discussions_resolved.png b/doc/user/project/merge_requests/img/discussions_resolved.png Binary files differnew file mode 100644 index 00000000000..85428129ac8 --- /dev/null +++ b/doc/user/project/merge_requests/img/discussions_resolved.png diff --git a/doc/user/project/merge_requests/img/merge_request_diff.png b/doc/user/project/merge_requests/img/merge_request_diff.png Binary files differnew file mode 100644 index 00000000000..06ee4908edc --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_request_diff.png diff --git a/doc/user/project/merge_requests/img/merge_request_widget.png b/doc/user/project/merge_requests/img/merge_request_widget.png Binary files differnew file mode 100644 index 00000000000..ffb96b17b07 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_request_widget.png diff --git a/doc/workflow/merge_when_build_succeeds/enable.png b/doc/user/project/merge_requests/img/merge_when_build_succeeds_enable.png Binary files differindex b86e6d7b3fd..b86e6d7b3fd 100644 --- a/doc/workflow/merge_when_build_succeeds/enable.png +++ b/doc/user/project/merge_requests/img/merge_when_build_succeeds_enable.png diff --git a/doc/user/project/merge_requests/img/merge_when_build_succeeds_only_if_succeeds_msg.png b/doc/user/project/merge_requests/img/merge_when_build_succeeds_only_if_succeeds_msg.png Binary files differnew file mode 100644 index 00000000000..6b9756b7418 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_when_build_succeeds_only_if_succeeds_msg.png diff --git a/doc/workflow/merge_requests/only_allow_merge_if_build_succeeds.png b/doc/user/project/merge_requests/img/merge_when_build_succeeds_only_if_succeeds_settings.png Binary files differindex 18bebf5fe92..18bebf5fe92 100644 --- a/doc/workflow/merge_requests/only_allow_merge_if_build_succeeds.png +++ b/doc/user/project/merge_requests/img/merge_when_build_succeeds_only_if_succeeds_settings.png diff --git a/doc/workflow/merge_when_build_succeeds/status.png b/doc/user/project/merge_requests/img/merge_when_build_succeeds_status.png Binary files differindex f3ea61d8147..f3ea61d8147 100644 --- a/doc/workflow/merge_when_build_succeeds/status.png +++ b/doc/user/project/merge_requests/img/merge_when_build_succeeds_status.png diff --git a/doc/user/project/merge_requests/img/resolve_comment_button.png b/doc/user/project/merge_requests/img/resolve_comment_button.png Binary files differnew file mode 100644 index 00000000000..2c4ab2f5d53 --- /dev/null +++ b/doc/user/project/merge_requests/img/resolve_comment_button.png diff --git a/doc/user/project/merge_requests/img/resolve_discussion_button.png b/doc/user/project/merge_requests/img/resolve_discussion_button.png Binary files differnew file mode 100644 index 00000000000..73f265bb101 --- /dev/null +++ b/doc/user/project/merge_requests/img/resolve_discussion_button.png diff --git a/doc/workflow/img/revert_changes_commit.png b/doc/user/project/merge_requests/img/revert_changes_commit.png Binary files differindex e7194fc3504..e7194fc3504 100644 --- a/doc/workflow/img/revert_changes_commit.png +++ b/doc/user/project/merge_requests/img/revert_changes_commit.png diff --git a/doc/workflow/img/revert_changes_commit_modal.png b/doc/user/project/merge_requests/img/revert_changes_commit_modal.png Binary files differindex c660ec7eaec..c660ec7eaec 100644 --- a/doc/workflow/img/revert_changes_commit_modal.png +++ b/doc/user/project/merge_requests/img/revert_changes_commit_modal.png diff --git a/doc/workflow/img/revert_changes_mr.png b/doc/user/project/merge_requests/img/revert_changes_mr.png Binary files differindex 3002f0ac1c5..3002f0ac1c5 100644 --- a/doc/workflow/img/revert_changes_mr.png +++ b/doc/user/project/merge_requests/img/revert_changes_mr.png diff --git a/doc/workflow/img/revert_changes_mr_modal.png b/doc/user/project/merge_requests/img/revert_changes_mr_modal.png Binary files differindex c6aaeecc8a6..c6aaeecc8a6 100644 --- a/doc/workflow/img/revert_changes_mr_modal.png +++ b/doc/user/project/merge_requests/img/revert_changes_mr_modal.png diff --git a/doc/user/project/merge_requests/img/versions.png b/doc/user/project/merge_requests/img/versions.png Binary files differnew file mode 100644 index 00000000000..6c86f2c68ac --- /dev/null +++ b/doc/user/project/merge_requests/img/versions.png diff --git a/doc/user/project/merge_requests/img/versions_compare.png b/doc/user/project/merge_requests/img/versions_compare.png Binary files differnew file mode 100644 index 00000000000..890cae7768c --- /dev/null +++ b/doc/user/project/merge_requests/img/versions_compare.png diff --git a/doc/user/project/merge_requests/img/versions_dropdown.png b/doc/user/project/merge_requests/img/versions_dropdown.png Binary files differnew file mode 100644 index 00000000000..9bab9304e14 --- /dev/null +++ b/doc/user/project/merge_requests/img/versions_dropdown.png diff --git a/doc/user/project/merge_requests/img/versions_system_note.png b/doc/user/project/merge_requests/img/versions_system_note.png Binary files differnew file mode 100644 index 00000000000..7c9d7715745 --- /dev/null +++ b/doc/user/project/merge_requests/img/versions_system_note.png diff --git a/doc/workflow/wip_merge_requests/blocked_accept_button.png b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png Binary files differindex 89c458aa8d9..89c458aa8d9 100644 --- a/doc/workflow/wip_merge_requests/blocked_accept_button.png +++ b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png diff --git a/doc/workflow/wip_merge_requests/mark_as_wip.png b/doc/user/project/merge_requests/img/wip_mark_as_wip.png Binary files differindex 9c37354a653..9c37354a653 100644 --- a/doc/workflow/wip_merge_requests/mark_as_wip.png +++ b/doc/user/project/merge_requests/img/wip_mark_as_wip.png diff --git a/doc/workflow/wip_merge_requests/unmark_as_wip.png b/doc/user/project/merge_requests/img/wip_unmark_as_wip.png Binary files differindex 31f7326beb0..31f7326beb0 100644 --- a/doc/workflow/wip_merge_requests/unmark_as_wip.png +++ b/doc/user/project/merge_requests/img/wip_unmark_as_wip.png diff --git a/doc/user/project/merge_requests/merge_request_discussion_resolution.md b/doc/user/project/merge_requests/merge_request_discussion_resolution.md new file mode 100644 index 00000000000..2559f5f5250 --- /dev/null +++ b/doc/user/project/merge_requests/merge_request_discussion_resolution.md @@ -0,0 +1,40 @@ +# Merge Request discussion resolution + +> [Introduced][ce-5022] in GitLab 8.11. + +Discussion resolution helps keep track of progress during code review. +Resolving comments prevents you from forgetting to address feedback and lets you +hide discussions that are no longer relevant. + +!["A discussion between two people on a piece of code"][discussion-view] + +Comments and discussions can be resolved by anyone with at least Developer +access to the project, as well as by the author of the merge request. + +## Marking a comment or discussion as resolved + +You can mark a discussion as resolved by clicking the "Resolve discussion" +button at the bottom of the discussion. + +!["Resolve discussion" button][resolve-discussion-button] + +Alternatively, you can mark each comment as resolved individually. + +!["Resolve comment" button][resolve-comment-button] + +## Jumping between unresolved discussions + +When a merge request has a large number of comments it can be difficult to track +what remains unresolved. You can jump between unresolved discussions with the +Jump button next to the Reply field on a discussion. + +You can also jump to the first unresolved discussion from the button next to the +resolved discussions tracker. + +!["3/4 discussions resolved"][discussions-resolved] + +[ce-5022]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5022 +[resolve-discussion-button]: img/resolve_discussion_button.png +[resolve-comment-button]: img/resolve_comment_button.png +[discussion-view]: img/discussion_view.png +[discussions-resolved]: img/discussions_resolved.png diff --git a/doc/user/project/merge_requests/merge_when_build_succeeds.md b/doc/user/project/merge_requests/merge_when_build_succeeds.md new file mode 100644 index 00000000000..c138061fd40 --- /dev/null +++ b/doc/user/project/merge_requests/merge_when_build_succeeds.md @@ -0,0 +1,46 @@ +# Merge When Build Succeeds + +When reviewing a merge request that looks ready to merge but still has one or +more CI builds running, you can set it to be merged automatically when the +builds pipeline succeed. This way, you don't have to wait for the builds to +finish and remember to merge the request manually. + + + +When you hit the "Merge When Build Succeeds" button, the status of the merge +request will be updated to represent the impending merge. If you cannot wait +for the pipeline to succeed and want to merge immediately, this option is +available in the dropdown menu on the right of the main button. + +Both team developers and the author of the merge request have the option to +cancel the automatic merge if they find a reason why it shouldn't be merged +after all. + + + +When the pipeline succeeds, the merge request will automatically be merged. +When the pipeline fails, the author gets a chance to retry any failed builds, +or to push new commits to fix the failure. + +When the builds are retried and succeed on the second try, the merge request +will automatically be merged after all. When the merge request is updated with +new commits, the automatic merge is automatically canceled to allow the new +changes to be reviewed. + +## Only allow merge requests to be merged if the build succeeds + +> **Note:** +You need to have builds configured to enable this feature. + +You can prevent merge requests from being merged if their build did not succeed. + +Navigate to your project's settings page, select the +**Only allow merge requests to be merged if the build succeeds** check box and +hit **Save** for the changes to take effect. + + + +From now on, every time the pipelinefails you will not be able to merge the +merge request from the UI, until you make all relevant builds pass. + + diff --git a/doc/user/project/merge_requests/resolve_conflicts.md b/doc/user/project/merge_requests/resolve_conflicts.md new file mode 100644 index 00000000000..4d7225bd820 --- /dev/null +++ b/doc/user/project/merge_requests/resolve_conflicts.md @@ -0,0 +1,42 @@ +# Merge conflict resolution + +> [Introduced][ce-5479] in GitLab 8.11. + +When a merge request has conflicts, GitLab may provide the option to resolve +those conflicts in the GitLab UI. (See +[conflicts available for resolution](#conflicts-available-for-resolution) for +more information on when this is available.) If this is an option, you will see +a **resolve these conflicts** link in the merge request widget: + + + +Clicking this will show a list of files with conflicts, with conflict sections +highlighted: + + + +Once all conflicts have been marked as using 'ours' or 'theirs', the conflict +can be resolved. This will perform a merge of the target branch of the merge +request into the source branch, resolving the conflicts using the options +chosen. If the source branch is `feature` and the target branch is `master`, +this is similar to performing `git checkout feature; git merge master` locally. + +## Conflicts available for resolution + +GitLab allows resolving conflicts in a file where all of the below are true: + +- The file is text, not binary +- The file is in a UTF-8 compatible encoding +- The file does not already contain conflict markers +- The file, with conflict markers added, is not over 200 KB in size +- The file exists under the same path in both branches + +If any file with conflicts in that merge request does not meet all of these +criteria, the conflicts for that merge request cannot be resolved in the UI. + +Additionally, GitLab does not detect conflicts in renames away from a path. For +example, this will not create a conflict: on branch `a`, doing `git mv file1 +file2`; on branch `b`, doing `git mv file1 file3`. Instead, both files will be +present in the branch after the merge request is merged. + +[ce-5479]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5479 diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md new file mode 100644 index 00000000000..5ead9f4177f --- /dev/null +++ b/doc/user/project/merge_requests/revert_changes.md @@ -0,0 +1,64 @@ +# Reverting changes + +> [Introduced][ce-1990] in GitLab 8.5. + +--- + +GitLab implements Git's powerful feature to [revert any commit][git-revert] +with introducing a **Revert** button in Merge Requests and commit details. + +## Reverting a Merge Request + +_**Note:** The **Revert** button will only be available for Merge Requests +created since GitLab 8.5. However, you can still revert a Merge Request +by reverting the merge commit from the list of Commits page._ + +After the Merge Request has been merged, a **Revert** button will be available +to revert the changes introduced by that Merge Request: + + + +--- + +You can revert the changes directly into the selected branch or you can opt to +create a new Merge Request with the revert changes: + + + +--- + +After the Merge Request has been reverted, the **Revert** button will not be +available anymore. + +## Reverting a Commit + +You can revert a Commit from the Commit details page: + + + +--- + +Similar to reverting a Merge Request, you can opt to revert the changes +directly into the target branch or create a new Merge Request to revert the +changes: + + + +--- + +After the Commit has been reverted, the **Revert** button will not be available +anymore. + +Please note that when reverting merge commits, the mainline will always be the +first parent. If you want to use a different mainline then you need to do that +from the command line. + +Here is a quick example to revert a merge commit using the second parent as the +mainline: + +```bash +git revert -m 2 7a39eb0 +``` + +[ce-1990]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1990 "Revert button Merge Request" +[git-revert]: https://git-scm.com/docs/git-revert "Git revert documentation" diff --git a/doc/user/project/merge_requests/versions.md b/doc/user/project/merge_requests/versions.md new file mode 100644 index 00000000000..77eab7ba5e3 --- /dev/null +++ b/doc/user/project/merge_requests/versions.md @@ -0,0 +1,42 @@ +# Merge requests versions + +> Will be [introduced][ce-5467] in GitLab 8.12. + +Every time you push to a branch that is tied to a merge request, a new version +of merge request diff is created. When you visit a merge request that contains +more than one pushes, you can select and compare the versions of those merge +request diffs. + + + +--- + +By default, the latest version of changes is shown. However, you +can select an older one from version dropdown. + + + +--- + +You can also compare the merge request version with an older one to see what has +changed since then. + + + +--- + +Every time you push new changes to the branch, a link to compare the last +changes appears as a system note. + + + +--- + +>**Notes:** +- Comments are disabled while viewing outdated merge versions or comparing to + versions other than base. +- Merge request versions are based on push not on commit. So, if you pushed 5 + commits in a single push, it will be a single option in the dropdown. If you + pushed 5 times, that will count for 5 options. + +[ce-5467]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5467 diff --git a/doc/user/project/merge_requests/work_in_progress_merge_requests.md b/doc/user/project/merge_requests/work_in_progress_merge_requests.md new file mode 100644 index 00000000000..546c8bdc5e5 --- /dev/null +++ b/doc/user/project/merge_requests/work_in_progress_merge_requests.md @@ -0,0 +1,17 @@ +# "Work In Progress" Merge Requests + +To prevent merge requests from accidentally being accepted before they're +completely ready, GitLab blocks the "Accept" button for merge requests that +have been marked a **Work In Progress**. + + + +To mark a merge request a Work In Progress, simply start its title with `[WIP]` +or `WIP:`. + + + +To allow a Work In Progress merge request to be accepted again when it's ready, +simply remove the `WIP` prefix. + + diff --git a/doc/user/project/new_ci_build_permissions_model.md b/doc/user/project/new_ci_build_permissions_model.md new file mode 100644 index 00000000000..8827b501901 --- /dev/null +++ b/doc/user/project/new_ci_build_permissions_model.md @@ -0,0 +1,311 @@ +# New CI build permissions model + +> Introduced in GitLab 8.12. + +GitLab 8.12 has a completely redesigned [build permissions] system. You can find +all discussion and all our concerns when choosing the current approach in issue +[#18994](https://gitlab.com/gitlab-org/gitlab-ce/issues/18994). + +--- + +Builds permissions should be tightly integrated with the permissions of a user +who is triggering a build. + +The reasons to do it like that are: + +- We already have a permissions system in place: group and project membership + of users. +- We already fully know who is triggering a build (using `git push`, using the + web UI, executing triggers). +- We already know what user is allowed to do. +- We use the user permissions for builds that are triggered by the user. +- It opens a lot of possibilities to further enforce user permissions, like + allowing only specific users to access runners or use secure variables and + environments. +- It is simple and convenient that your build can access everything that you + as a user have access to. +- Short living unique tokens are now used, granting access for time of the build + and maximizing security. + +With the new behavior, any build that is triggered by the user, is also marked +with their permissions. When a user does a `git push` or changes files through +the web UI, a new pipeline will be usually created. This pipeline will be marked +as created be the pusher (local push or via the UI) and any build created in this +pipeline will have the permissions of the pusher. + +This allows us to make it really easy to evaluate the access for all projects +that have Git submodules or are using container images that the pusher would +have access too. **The permission is granted only for time that build is running. +The access is revoked after the build is finished.** + +## Types of users + +It is important to note that we have a few types of users: + +- **Administrators**: CI builds created by Administrators will not have access + to all GitLab projects, but only to projects and container images of projects + that the administrator is a member of.That means that if a project is either + public or internal users have access anyway, but if a project is private, the + Administrator will have to be a member of it in order to have access to it + via another project's build. + +- **External users**: CI builds created by [external users][ext] will have + access only to projects to which user has at least reporter access. This + rules out accessing all internal projects by default, + +This allows us to make the CI and permission system more trustworthy. +Let's consider the following scenario: + +1. You are an employee of a company. Your company has a number of internal tools + hosted in private repositories and you have multiple CI builds that make use + of these repositories. + +2. You invite a new [external user][ext]. CI builds created by that user do not + have access to internal repositories, because the user also doesn't have the + access from within GitLab. You as an employee have to grant explicit access + for this user. This allows us to prevent from accidental data leakage. + +## Build token + +A unique build token is generated for each build and it allows the user to +access all projects that would be normally accessible to the user creating that +build. + +We try to make sure that this token doesn't leak by: + +1. Securing all API endpoints to not expose the build token. +1. Masking the build token from build logs. +1. Allowing to use the build token **only** when build is running. + +However, this brings a question about the Runners security. To make sure that +this token doesn't leak, you should also make sure that you configure +your Runners in the most possible secure way, by avoiding the following: + +1. Any usage of Docker's `privileged` mode is risky if the machines are re-used. +1. Using the `shell` executor since builds run on the same machine. + +By using an insecure GitLab Runner configuration, you allow the rogue developers +to steal the tokens of other builds. + +## Build triggers + +[Build triggers][triggers] do not support the new permission model. +They continue to use the old authentication mechanism where the CI build +can access only its own sources. We plan to remove that limitation in one of +the upcoming releases. + +## Before GitLab 8.12 + +In versions before GitLab 8.12, all CI builds would use the CI Runner's token +to checkout project sources. + +The project's Runner's token was a token that you could find under the +project's **Settings > CI/CD Pipelines** and was limited to access only that +project. +It could be used for registering new specific Runners assigned to the project +and to checkout project sources. +It could also be used with the GitLab Container Registry for that project, +allowing pulling and pushing Docker images from within the CI build. + +--- + +GitLab would create a special checkout URL like: + +``` +https://gitlab-ci-token:<project-runners-token>/gitlab.com/gitlab-org/gitlab-ce.git +``` + +And then the users could also use it in their CI builds all Docker related +commands to interact with GitLab Container Registry. For example: + +``` +docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com +``` + +Using single token had multiple security implications: + +- The token would be readable to anyone who had developer access to a project + that could run CI builds, allowing the developer to register any specific + Runner for that project. +- The token would allow to access only the project's sources, forbidding from + accessing any other projects. +- The token was not expiring and was multi-purpose: used for checking out sources, + for registering specific runners and for accessing a project's container + registry with read-write permissions. + +All the above led to a new permission model for builds that was introduced +with GitLab 8.12. + +## Making use of the new CI build permissions model + +With the new build permissions model, there is now an easy way to access all +dependent source code in a project. That way, we can: + +1. Access a project's Git submodules +1. Access private container images +1. Access project's and submodule LFS objects + +Below you can see the prerequisites needed to make use of the new permissions +model and how that works with Git submodules and private Docker images hosted on +the container registry. + +### Prerequisites to use the new permissions model + +With the new permissions model in place, there may be times that your build will +fail. This is most likely because your project tries to access other project's +sources, and you don't have the appropriate permissions. In the build log look +for information about 403 or forbidden access messages. + +In short here's what you need to do should you encounter any issues. + +As an administrator: + +- **500 errors**: You will need to update [GitLab Workhorse][workhorse] to at + least 0.8.2. This is done automatically for Omnibus installations, you need to + [check manually][update-docs] for installations from source. +- **500 errors**: Check if you have another web proxy sitting in front of NGINX (HAProxy, + Apache, etc.). It might be a good idea to let GitLab use the internal NGINX + web server and not disable it completely. See [this comment][comment] for an + example. +- **403 errors**: You need to make sure that your installation has [HTTP(S) + cloning enabled][https]. HTTP(S) support is now a **requirement** by GitLab CI + to clone all sources. + +As a user: + +- Make sure you are a member of the group or project you're trying to have + access to. As an Administrator, you can verify that by impersonating the user + and retry the failing build in order to verify that everything is correct. + +### Git submodules + +> +It often happens that while working on one project, you need to use another +project from within it; perhaps it’s a library that a third party developed or +you’re developing a project separately and are using it in multiple parent +projects. +A common issue arises in these scenarios: you want to be able to treat the two +projects as separate yet still be able to use one from within the other. +> +_Excerpt from the [Git website][git-scm] about submodules._ + +If dealing with submodules, your project will probably have a file named +`.gitmodules`. And this is how it usually looks like: + +``` +[submodule "tools"] + path = tools + url = git@gitlab.com/group/tools.git +``` + +> **Note:** +If you are **not** using GitLab 8.12 or higher, you would need to work your way +around this issue in order to access the sources of `gitlab.com/group/tools` +(e.g., use [SSH keys](../ssh_keys/README.md)). +> +With GitLab 8.12 onward, your permissions are used to evaluate what a CI build +can access. More information about how this system works can be found in the +[Build permissions model](../../user/permissions.md#builds-permissions). + +To make use of the new changes, you have to update your `.gitmodules` file to +use a relative URL. + +Let's consider the following example: + +1. Your project is located at `https://gitlab.com/secret-group/my-project`. +1. To checkout your sources you usually use an SSH address like + `git@gitlab.com:secret-group/my-project.git`. +1. Your project depends on `https://gitlab.com/group/tools`. +1. You have the `.gitmodules` file with above content. + +Since Git allows the usage of relative URLs for your `.gitmodules` configuration, +this easily allows you to use HTTP for cloning all your CI builds and SSH +for all your local checkouts. + +For example, if you change the `url` of your `tools` dependency, from +`git@gitlab.com/group/tools.git` to `../../group/tools.git`, this will instruct +Git to automatically deduce the URL that should be used when cloning sources. +Whether you use HTTP or SSH, Git will use that same channel and it will allow +to make all your CI builds use HTTPS (because GitLab CI uses HTTPS for cloning +your sources), and all your local clones will continue using SSH. + +Given the above explanation, your `.gitmodules` file should eventually look +like this: + +``` +[submodule "tools"] + path = tools + url = ../../group/tools.git +``` + +However, you have to explicitly tell GitLab CI to clone your submodules as this +is not done automatically. You can achieve that by adding a `before_script` +section to your `.gitlab-ci.yml`: + +``` +before_script: + - git submodule update --init --recursive + +test: + script: + - run-my-tests +``` + +This will make GitLab CI initialize (fetch) and update (checkout) all your +submodules recursively. + +In case your environment or your Docker image doesn't have Git installed, +you have to either ask your Administrator or install the missing dependency +yourself: + +``` +# Debian / Ubuntu +before_script: + - apt-get update -y + - apt-get install -y git-core + - git submodule update --init --recursive + +# CentOS / RedHat +before_script: + - yum install git + - git submodule update --init --recursive + +# Alpine +before_script: + - apk add -U git + - git submodule update --init --recursive +``` + +### Container Registry + +With the update permission model we also extended the support for accessing +Container Registries for private projects. + +> **Note:** +As GitLab Runner 1.6 doesn't yet incorporate the introduced changes for +permissions, this makes the `image:` directive to not work with private projects +automatically. The manual configuration by an Administrator is required to use +private images. We plan to remove that limitation in one of the upcoming releases. + +Your builds can access all container images that you would normally have access +to. The only implication is that you can push to the Container Registry of the +project for which the build is triggered. + +This is how an example usage can look like: + +``` +test: + script: + - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY + - docker pull $CI_REGISTRY/group/other-project:latest + - docker run $CI_REGISTRY/group/other-project:latest +``` + +[build permissions]: ../permissions.md#builds-permissions +[comment]: https://gitlab.com/gitlab-org/gitlab-ce/issues/22484#note_16648302 +[ext]: ../permissions.md#external-users +[git-scm]: https://git-scm.com/book/en/v2/Git-Tools-Submodules +[https]: ../admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols +[triggers]: ../../ci/triggers/README.md +[update-docs]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update +[workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md new file mode 100644 index 00000000000..f7a686d2ccf --- /dev/null +++ b/doc/user/project/protected_branches.md @@ -0,0 +1,116 @@ +# Protected Branches + +[Permissions](../permissions.md) in GitLab are fundamentally defined around the +idea of having read or write permission to the repository and branches. To +prevent people from messing with history or pushing code without review, we've +created protected branches. + +## Overview + +By default, a protected branch does four simple things: + +- it prevents its creation, if not already created, from everybody except users + with Master permission +- it prevents pushes from everybody except users with Master permission +- it prevents **anyone** from force pushing to the branch +- it prevents **anyone** from deleting the branch + +See the [Changelog](#changelog) section for changes over time. + +> +>Additional functionality for GitLab Enterprise Edition: +> +>- Restrict push and merge access to [certain users][ee-restrict] + +## Configuring protected branches + +To protect a branch, you need to have at least Master permission level. Note +that the `master` branch is protected by default. + +1. Navigate to the main page of the project. +1. In the upper right corner, click the settings wheel and select **Protected branches**. + +  + +1. From the **Branch** dropdown menu, select the branch you want to protect and + click **Protect**. In the screenshot below, we chose the `develop` branch. + +  + +1. Once done, the protected branch will appear in the "Protected branches" list. + +  + +## Using the Allowed to merge and Allowed to push settings + +> [Introduced][ce-5081] in GitLab 8.11. + +Since GitLab 8.11, we added another layer of branch protection which provides +more granular management of protected branches. The "Developers can push" +option was replaced by an "Allowed to push" setting which can be set to +allow/prohibit Masters and/or Developers to push to a protected branch. + +Using the "Allowed to push" and "Allowed to merge" settings, you can control +the actions that different roles can perform with the protected branch. +For example, you could set "Allowed to push" to "No one", and "Allowed to merge" +to "Developers + Masters", to require _everyone_ to submit a merge request for +changes going into the protected branch. This is compatible with workflows like +the [GitLab workflow](../../workflow/gitlab_flow.md). + +However, there are workflows where that is not needed, and only protecting from +force pushes and branch removal is useful. For those workflows, you can allow +everyone with write access to push to a protected branch by setting +"Allowed to push" to "Developers + Masters". + +You can set the "Allowed to push" and "Allowed to merge" options while creating +a protected branch or afterwards by selecting the option you want from the +dropdown list in the "Already protected" area. + + + +If you don't choose any of those options while creating a protected branch, +they are set to "Masters" by default. + +## Wildcard protected branches + +> [Introduced][ce-4665] in GitLab 8.10. + +You can specify a wildcard protected branch, which will protect all branches +matching the wildcard. For example: + +| Wildcard Protected Branch | Matching Branches | +|---------------------------+--------------------------------------------------------| +| `*-stable` | `production-stable`, `staging-stable` | +| `production/*` | `production/app-server`, `production/load-balancer` | +| `*gitlab*` | `gitlab`, `gitlab/staging`, `master/gitlab/production` | + +Protected branch settings (like "Developers can push") apply to all matching +branches. + +Two different wildcards can potentially match the same branch. For example, +`*-stable` and `production-*` would both match a `production-stable` branch. +In that case, if _any_ of these protected branches have a setting like +"Allowed to push", then `production-stable` will also inherit this setting. + +If you click on a protected branch's name, you will be presented with a list of +all matching branches: + + + +## Changelog + +**8.11** + +- Allow creating protected branches that can't be pushed to [gitlab-org/gitlab-ce!5081][ce-5081] + +**8.10** + +- Allow developers to merge into a protected branch without having push access [gitlab-org/gitlab-ce!4892][ce-4892] +- Allow specifying protected branches using wildcards [gitlab-org/gitlab-ce!4665][ce-4665] + +--- + +[ce-4665]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4665 "Allow specifying protected branches using wildcards" +[ce-4892]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/4892 "Allow developers to merge into a protected branch without having push access" +[ce-5081]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5081 "Allow creating protected branches that can't be pushed to" +[ee-restrict]: http://docs.gitlab.com/ee/user/project/protected_branches.html#restricting-push-and-merge-access-to-certain-users diff --git a/doc/workflow/img/web_editor_new_branch_dropdown.png b/doc/user/project/repository/img/web_editor_new_branch_dropdown.png Binary files differindex a8e635d2faf..a8e635d2faf 100644 --- a/doc/workflow/img/web_editor_new_branch_dropdown.png +++ b/doc/user/project/repository/img/web_editor_new_branch_dropdown.png diff --git a/doc/user/project/repository/img/web_editor_new_branch_from_issue.png b/doc/user/project/repository/img/web_editor_new_branch_from_issue.png Binary files differnew file mode 100644 index 00000000000..b0a63ddf0ab --- /dev/null +++ b/doc/user/project/repository/img/web_editor_new_branch_from_issue.png diff --git a/doc/workflow/img/web_editor_new_branch_page.png b/doc/user/project/repository/img/web_editor_new_branch_page.png Binary files differindex 7f36b7faf63..7f36b7faf63 100644 --- a/doc/workflow/img/web_editor_new_branch_page.png +++ b/doc/user/project/repository/img/web_editor_new_branch_page.png diff --git a/doc/workflow/img/web_editor_new_directory_dialog.png b/doc/user/project/repository/img/web_editor_new_directory_dialog.png Binary files differindex d16e3c67116..d16e3c67116 100644 --- a/doc/workflow/img/web_editor_new_directory_dialog.png +++ b/doc/user/project/repository/img/web_editor_new_directory_dialog.png diff --git a/doc/workflow/img/web_editor_new_directory_dropdown.png b/doc/user/project/repository/img/web_editor_new_directory_dropdown.png Binary files differindex c8d77b16ee8..c8d77b16ee8 100644 --- a/doc/workflow/img/web_editor_new_directory_dropdown.png +++ b/doc/user/project/repository/img/web_editor_new_directory_dropdown.png diff --git a/doc/workflow/img/web_editor_new_file_dropdown.png b/doc/user/project/repository/img/web_editor_new_file_dropdown.png Binary files differindex 3fcb91c9b93..3fcb91c9b93 100644 --- a/doc/workflow/img/web_editor_new_file_dropdown.png +++ b/doc/user/project/repository/img/web_editor_new_file_dropdown.png diff --git a/doc/workflow/img/web_editor_new_file_editor.png b/doc/user/project/repository/img/web_editor_new_file_editor.png Binary files differindex 21c340b9288..21c340b9288 100644 --- a/doc/workflow/img/web_editor_new_file_editor.png +++ b/doc/user/project/repository/img/web_editor_new_file_editor.png diff --git a/doc/workflow/img/web_editor_new_push_widget.png b/doc/user/project/repository/img/web_editor_new_push_widget.png Binary files differindex c7738a4c930..c7738a4c930 100644 --- a/doc/workflow/img/web_editor_new_push_widget.png +++ b/doc/user/project/repository/img/web_editor_new_push_widget.png diff --git a/doc/workflow/img/web_editor_new_tag_dropdown.png b/doc/user/project/repository/img/web_editor_new_tag_dropdown.png Binary files differindex ac7415009b3..ac7415009b3 100644 --- a/doc/workflow/img/web_editor_new_tag_dropdown.png +++ b/doc/user/project/repository/img/web_editor_new_tag_dropdown.png diff --git a/doc/workflow/img/web_editor_new_tag_page.png b/doc/user/project/repository/img/web_editor_new_tag_page.png Binary files differindex 231e1a13fc0..231e1a13fc0 100644 --- a/doc/workflow/img/web_editor_new_tag_page.png +++ b/doc/user/project/repository/img/web_editor_new_tag_page.png diff --git a/doc/workflow/img/web_editor_start_new_merge_request.png b/doc/user/project/repository/img/web_editor_start_new_merge_request.png Binary files differindex 2755501dfd1..2755501dfd1 100644 --- a/doc/workflow/img/web_editor_start_new_merge_request.png +++ b/doc/user/project/repository/img/web_editor_start_new_merge_request.png diff --git a/doc/user/project/repository/img/web_editor_template_dropdown_buttons.png b/doc/user/project/repository/img/web_editor_template_dropdown_buttons.png Binary files differnew file mode 100644 index 00000000000..4efc51cc423 --- /dev/null +++ b/doc/user/project/repository/img/web_editor_template_dropdown_buttons.png diff --git a/doc/user/project/repository/img/web_editor_template_dropdown_first_file.png b/doc/user/project/repository/img/web_editor_template_dropdown_first_file.png Binary files differnew file mode 100644 index 00000000000..67190c58823 --- /dev/null +++ b/doc/user/project/repository/img/web_editor_template_dropdown_first_file.png diff --git a/doc/user/project/repository/img/web_editor_template_dropdown_mit_license.png b/doc/user/project/repository/img/web_editor_template_dropdown_mit_license.png Binary files differnew file mode 100644 index 00000000000..47719113805 --- /dev/null +++ b/doc/user/project/repository/img/web_editor_template_dropdown_mit_license.png diff --git a/doc/workflow/img/web_editor_upload_file_dialog.png b/doc/user/project/repository/img/web_editor_upload_file_dialog.png Binary files differindex 9d6d8250bbe..9d6d8250bbe 100644 --- a/doc/workflow/img/web_editor_upload_file_dialog.png +++ b/doc/user/project/repository/img/web_editor_upload_file_dialog.png diff --git a/doc/workflow/img/web_editor_upload_file_dropdown.png b/doc/user/project/repository/img/web_editor_upload_file_dropdown.png Binary files differindex 6b5205b05ec..6b5205b05ec 100644 --- a/doc/workflow/img/web_editor_upload_file_dropdown.png +++ b/doc/user/project/repository/img/web_editor_upload_file_dropdown.png diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md new file mode 100644 index 00000000000..675e89e4247 --- /dev/null +++ b/doc/user/project/repository/web_editor.md @@ -0,0 +1,175 @@ +# GitLab Web Editor + +Sometimes it's easier to make quick changes directly from the GitLab interface +than to clone the project and use the Git command line tool. In this feature +highlight we look at how you can create a new file, directory, branch or +tag from the file browser. All of these actions are available from a single +dropdown menu. + +## Create a file + +From a project's files page, click the '+' button to the right of the branch selector. +Choose **New file** from the dropdown. + + + +--- + +Enter a file name in the **File name** box. Then, add file content in the editor +area. Add a descriptive commit message and choose a branch. The branch field +will default to the branch you were viewing in the file browser. If you enter +a new branch name, a checkbox will appear allowing you to start a new merge +request after you commit the changes. + +When you are satisfied with your new file, click **Commit Changes** at the bottom. + + + +### Template dropdowns + +When starting a new project, there are some common files which the new project +might need too. Therefore a message will be displayed by GitLab to make this +easy for you. + + + +When clicking on either `LICENSE` or `.gitignore`, a dropdown will be displayed +to provide you with a template which might be suitable for your project. + + + +The license, changelog, contribution guide, or `.gitlab-ci.yml` file could also +be added through a button on the project page. In the example below the license +has already been created, which creates a link to the license itself. + + + +>**Note:** +The **Set up CI** button will not appear on an empty repository. You have to at +least add a file in order for the button to show up. + +## Upload a file + +The ability to create a file is great when the content is text. However, this +doesn't work well for binary data such as images, PDFs or other file types. In +this case you need to upload a file. + +From a project's files page, click the '+' button to the right of the branch +selector. Choose **Upload file** from the dropdown. + + + +--- + +Once the upload dialog pops up there are two ways to upload your file. Either +drag and drop a file on the pop up or use the **click to upload** link. A file +preview will appear once you have selected a file to upload. + +Enter a commit message, choose a branch, and click **Upload file** when you are +ready. + + + +## Create a directory + +To keep files in the repository organized it is often helpful to create a new +directory. + +From a project's files page, click the '+' button to the right of the branch selector. +Choose **New directory** from the dropdown. + + + +--- + +In the new directory dialog enter a directory name, a commit message and choose +the target branch. Click **Create directory** to finish. + + + +## Create a new branch + +There are multiple ways to create a branch from GitLab's web interface. + +### Create a new branch from an issue + +> [Introduced][ce-2808] in GitLab 8.6. + +In case your development workflow dictates to have an issue for every merge +request, you can quickly create a branch right on the issue page which will be +tied with the issue itself. You can see a **New branch** button after the issue +description, unless there is already a branch with the same name or a referenced +merge request. + + + +Once you click it, a new branch will be created that diverges from the default +branch of your project, by default `master`. The branch name will be based on +the title of the issue and as suffix it will have its ID. Thus, the example +screenshot above will yield a branch named +`2-et-cum-et-sed-expedita-repellat-consequatur-ut-assumenda-numquam-rerum`. + +After the branch is created, you can edit files in the repository to fix +the issue. When a merge request is created based on the newly created branch, +the description field will automatically display the [issue closing pattern] +`Closes #ID`, where `ID` the ID of the issue. This will close the issue once the +merge request is merged. + +### Create a new branch from a project's dashboard + +If you want to make changes to several files before creating a new merge +request, you can create a new branch up front. From a project's files page, +choose **New branch** from the dropdown. + + + +--- + +Enter a new **Branch name**. Optionally, change the **Create from** field +to choose which branch, tag or commit SHA this new branch will originate from. +This field will autocomplete if you start typing an existing branch or tag. +Click **Create branch** and you will be returned to the file browser on this new +branch. + + + +--- + +You can now make changes to any files, as needed. When you're ready to merge +the changes back to master you can use the widget at the top of the screen. +This widget only appears for a period of time after you create the branch or +modify files. + + + +## Create a new tag + +Tags are useful for marking major milestones such as production releases, +release candidates, and more. You can create a tag from a branch or a commit +SHA. From a project's files page, choose **New tag** from the dropdown. + + + +--- + +Give the tag a name such as `v1.0.0`. Choose the branch or SHA from which you +would like to create this new tag. You can optionally add a message and +release notes. The release notes section supports markdown format and you can +also upload an attachment. Click **Create tag** and you will be taken to the tag +list page. + + + +## Tips + +When creating or uploading a new file, or creating a new directory, you can +trigger a new merge request rather than committing directly to master. Enter +a new branch name in the **Target branch** field. You will notice a checkbox +appear that is labeled **Start a new merge request with these changes**. After +you commit the changes you will be taken to a new merge request form. + + + + +[ce-2808]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2808 +[issue closing pattern]: ../user/project/issues/automatic_issue_closing.md diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 38e9786123d..dfc762fe1d3 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -1,22 +1,38 @@ # Project import/export >**Notes:** - - This feature was [introduced][ce-3050] in GitLab 8.9 - - Importing will not be possible if the import instance version is lower - than that of the exporter. - - For existing installations, the project import option has to be enabled in - application settings (`/admin/application_settings`) under 'Import sources'. - Ask your administrator if you don't see the **GitLab export** button when - creating a new project. - - You can find some useful raketasks if you are an administrator in the - [import_export](../../../administration/raketasks/project_import_export.md) - raketask. - - The exports are stored in a temporary [shared directory][tmp] and are deleted - every 24 hours by a specific worker. +> +> - [Introduced][ce-3050] in GitLab 8.9. +> - Importing will not be possible if the import instance version differs from +> that of the exporter. +> - For existing installations, the project import option has to be enabled in +> application settings (`/admin/application_settings`) under 'Import sources'. +> Ask your administrator if you don't see the **GitLab export** button when +> creating a new project. +> - You can find some useful raketasks if you are an administrator in the +> [import_export](../../../administration/raketasks/project_import_export.md) +> raketask. +> - The exports are stored in a temporary [shared directory][tmp] and are deleted +> every 24 hours by a specific worker. Existing projects running on any GitLab instance or GitLab.com can be exported with all their related data and be moved into a new GitLab instance. +## Version history + +| GitLab version | Import/Export version | +| -------- | -------- | +| 8.13.0 to current | 0.1.5 | +| 8.12.0 | 0.1.4 | +| 8.10.3 | 0.1.3 | +| 8.10.0 | 0.1.2 | +| 8.9.5 | 0.1.1 | +| 8.9.0 | 0.1.0 | + + > The table reflects what GitLab version we updated the Import/Export version at. + > For instance, 8.10.3 and 8.11 will have the same Import/Export version (0.1.3) + > and the exports between them will be compatible. + ## Exported contents The following items will be exported: diff --git a/doc/user/project/slash_commands.md b/doc/user/project/slash_commands.md new file mode 100644 index 00000000000..5f6a6c6503e --- /dev/null +++ b/doc/user/project/slash_commands.md @@ -0,0 +1,31 @@ +# GitLab slash commands + +Slash commands are textual shortcuts for common actions on issues or merge +requests that are usually done by clicking buttons or dropdowns in GitLab's UI. +You can enter these commands while creating a new issue or merge request, and +in comments. Each command should be on a separate line in order to be properly +detected and executed. The commands are removed from the issue, merge request or +comment body before it is saved and will not be visible to anyone else. + +Below is a list of all of the available commands and descriptions about what they +do. + +| Command | Action | +|:---------------------------|:-------------| +| `/close` | Close the issue or merge request | +| `/reopen` | Reopen the issue or merge request | +| `/title <New title>` | Change title | +| `/assign @username` | Assign | +| `/unassign` | Remove assignee | +| `/milestone %milestone` | Set milestone | +| `/remove_milestone` | Remove milestone | +| `/label ~foo ~"bar baz"` | Add label(s) | +| `/unlabel ~foo ~"bar baz"` | Remove all or specific label(s) | +| `/relabel ~foo ~"bar baz"` | Replace all label(s) | +| `/todo` | Add a todo | +| `/done` | Mark todo as done | +| `/subscribe` | Subscribe | +| `/unsubscribe` | Unsubscribe | +| <code>/due <in 2 days | this Friday | December 31st></code> | Set due date | +| `/remove_due_date` | Remove due date | +| `/wip` | Toggle the Work In Progress status | diff --git a/doc/web_hooks/web_hooks.md b/doc/web_hooks/web_hooks.md index 8559b67af04..33c1a79d59c 100644 --- a/doc/web_hooks/web_hooks.md +++ b/doc/web_hooks/web_hooks.md @@ -26,6 +26,10 @@ GitLab webhooks keep in mind the following things: you are writing a low-level hook this is important to remember. - GitLab ignores the HTTP status code returned by your endpoint. +## Secret Token + +If you specify a secret token, it will be sent with the hook request in the `X-Gitlab-Token` HTTP header. Your webhook endpoint can check that to verify that the request is legitimate. + ## SSL Verification By default, the SSL certificate of the webhook endpoint is verified based on @@ -750,6 +754,174 @@ X-Gitlab-Event: Wiki Page Hook } ``` +## Pipeline events + +Triggered on status change of Pipeline. + +**Request Header**: + +``` +X-Gitlab-Event: Pipeline Hook +``` + +**Request Body**: + +```json +{ + "object_kind": "pipeline", + "object_attributes":{ + "id": 31, + "ref": "master", + "tag": false, + "sha": "bcbb5ec396a2c0f828686f14fac9b80b780504f2", + "before_sha": "bcbb5ec396a2c0f828686f14fac9b80b780504f2", + "status": "success", + "stages":[ + "build", + "test", + "deploy" + ], + "created_at": "2016-08-12 15:23:28 UTC", + "finished_at": "2016-08-12 15:26:29 UTC", + "duration": 63 + }, + "user":{ + "name": "Administrator", + "username": "root", + "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" + }, + "project":{ + "name": "Gitlab Test", + "description": "Atque in sunt eos similique dolores voluptatem.", + "web_url": "http://192.168.64.1:3005/gitlab-org/gitlab-test", + "avatar_url": null, + "git_ssh_url": "git@192.168.64.1:gitlab-org/gitlab-test.git", + "git_http_url": "http://192.168.64.1:3005/gitlab-org/gitlab-test.git", + "namespace": "Gitlab Org", + "visibility_level": 20, + "path_with_namespace": "gitlab-org/gitlab-test", + "default_branch": "master" + }, + "commit":{ + "id": "bcbb5ec396a2c0f828686f14fac9b80b780504f2", + "message": "test\n", + "timestamp": "2016-08-12T17:23:21+02:00", + "url": "http://example.com/gitlab-org/gitlab-test/commit/bcbb5ec396a2c0f828686f14fac9b80b780504f2", + "author":{ + "name": "User", + "email": "user@gitlab.com" + } + }, + "builds":[ + { + "id": 380, + "stage": "deploy", + "name": "production", + "status": "skipped", + "created_at": "2016-08-12 15:23:28 UTC", + "started_at": null, + "finished_at": null, + "when": "manual", + "manual": true, + "user":{ + "name": "Administrator", + "username": "root", + "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" + }, + "runner": null, + "artifacts_file":{ + "filename": null, + "size": null + } + }, + { + "id": 377, + "stage": "test", + "name": "test-image", + "status": "success", + "created_at": "2016-08-12 15:23:28 UTC", + "started_at": "2016-08-12 15:26:12 UTC", + "finished_at": null, + "when": "on_success", + "manual": false, + "user":{ + "name": "Administrator", + "username": "root", + "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" + }, + "runner": null, + "artifacts_file":{ + "filename": null, + "size": null + } + }, + { + "id": 378, + "stage": "test", + "name": "test-build", + "status": "success", + "created_at": "2016-08-12 15:23:28 UTC", + "started_at": "2016-08-12 15:26:12 UTC", + "finished_at": "2016-08-12 15:26:29 UTC", + "when": "on_success", + "manual": false, + "user":{ + "name": "Administrator", + "username": "root", + "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" + }, + "runner": null, + "artifacts_file":{ + "filename": null, + "size": null + } + }, + { + "id": 376, + "stage": "build", + "name": "build-image", + "status": "success", + "created_at": "2016-08-12 15:23:28 UTC", + "started_at": "2016-08-12 15:24:56 UTC", + "finished_at": "2016-08-12 15:25:26 UTC", + "when": "on_success", + "manual": false, + "user":{ + "name": "Administrator", + "username": "root", + "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" + }, + "runner": null, + "artifacts_file":{ + "filename": null, + "size": null + } + }, + { + "id": 379, + "stage": "deploy", + "name": "staging", + "status": "created", + "created_at": "2016-08-12 15:23:28 UTC", + "started_at": null, + "finished_at": null, + "when": "on_success", + "manual": false, + "user":{ + "name": "Administrator", + "username": "root", + "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" + }, + "runner": null, + "artifacts_file":{ + "filename": null, + "size": null + } + } + ] +} +``` + #### Example webhook receiver If you want to see GitLab's webhooks in action for testing purposes you can use diff --git a/doc/workflow/README.md b/doc/workflow/README.md index ddb2f7281b1..2d9bfbc0629 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -1,10 +1,13 @@ # Workflow -- [Authorization for merge requests](authorization_for_merge_requests.md) +- [Automatic issue closing](../user/project/issues/automatic_issue_closing.md) - [Change your time zone](timezone.md) +- [Cycle Analytics](../user/project/cycle_analytics.md) +- [Description templates](../user/project/description_templates.md) - [Feature branch workflow](workflow.md) - [GitLab Flow](gitlab_flow.md) - [Groups](groups.md) +- [Issue Board](../user/project/issue_board.md) - [Keyboard shortcuts](shortcuts.md) - [File finder](file_finder.md) - [Labels](../user/project/labels.md) @@ -12,17 +15,22 @@ - [Project Features](project_features.md) - [Project forking workflow](forking_workflow.md) - [Project users](add-user/add-user.md) -- [Protected branches](protected_branches.md) +- [Protected branches](../user/project/protected_branches.md) +- [Slash commands](../user/project/slash_commands.md) - [Sharing a project with a group](share_with_group.md) - [Share projects with other groups](share_projects_with_other_groups.md) -- [Web Editor](web_editor.md) +- [Web Editor](../user/project/repository/web_editor.md) - [Releases](releases.md) - [Milestones](milestones.md) -- [Merge Requests](merge_requests.md) -- [Revert changes](revert_changes.md) -- [Cherry-pick changes](cherry_pick_changes.md) -- ["Work In Progress" Merge Requests](wip_merge_requests.md) -- [Merge When Build Succeeds](merge_when_build_succeeds.md) +- [Merge Requests](../user/project/merge_requests.md) + - [Authorization for merge requests](../user/project/merge_requests/authorization_for_merge_requests.md) + - [Cherry-pick changes](../user/project/merge_requests/cherry_pick_changes.md) + - [Merge when build succeeds](../user/project/merge_requests/merge_when_build_succeeds.md) + - [Resolve discussion comments in merge requests reviews](../user/project/merge_requests/merge_request_discussion_resolution.md) + - [Resolve merge conflicts in the UI](../user/project/merge_requests/resolve_conflicts.md) + - [Revert changes in the UI](../user/project/merge_requests/revert_changes.md) + - [Merge requests versions](../user/project/merge_requests/versions.md) + - ["Work In Progress" merge requests](../user/project/merge_requests/work_in_progress_merge_requests.md) - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) - [Importing from SVN, GitHub, BitBucket, etc](importing/README.md) - [Todos](todos.md) diff --git a/doc/workflow/add-user/add-user.md b/doc/workflow/add-user/add-user.md index 4b551130255..e541111d7b3 100644 --- a/doc/workflow/add-user/add-user.md +++ b/doc/workflow/add-user/add-user.md @@ -23,7 +23,7 @@ want to add. --- -Select the user and the [permission level](../../permissions/permissions.md) +Select the user and the [permission level](../../user/permissions.md) that you'd like to give the user. Note that you can select more than one user.  @@ -90,6 +90,9 @@ GitLab account using the same e-mail address the invitation was sent to. ## Request access to a project +As a project owner you can enable or disable non members to request access to +your project. Go to the project settings and click on **Allow users to request access**. + As a user, you can request to be a member of a project. Go to the project you'd like to be a member of, and click the **Request Access** button on the right side of your screen. diff --git a/doc/workflow/authorization_for_merge_requests.md b/doc/workflow/authorization_for_merge_requests.md index d1d6d94ec11..7bf80a3ad0d 100644 --- a/doc/workflow/authorization_for_merge_requests.md +++ b/doc/workflow/authorization_for_merge_requests.md @@ -1,40 +1 @@ -# Authorization for Merge requests - -There are two main ways to have a merge request flow with GitLab: working with protected branches in a single repository, or working with forks of an authoritative project. - -## Protected branch flow - -With the protected branch flow everybody works within the same GitLab project. - -The project maintainers get Master access and the regular developers get Developer access. - -The maintainers mark the authoritative branches as 'Protected'. - -The developers push feature branches to the project and create merge requests to have their feature branches reviewed and merged into one of the protected branches. - -Only users with Master access can merge changes into a protected branch. - -### Advantages - -- fewer projects means less clutter -- developers need to consider only one remote repository - -### Disadvantages - -- manual setup of protected branch required for each new project - -## Forking workflow - -With the forking workflow the maintainers get Master access and the regular developers get Reporter access to the authoritative repository, which prohibits them from pushing any changes to it. - -Developers create forks of the authoritative project and push their feature branches to their own forks. - -To get their changes into master they need to create a merge request across forks. - -### Advantages - -- in an appropriately configured GitLab group, new projects automatically get the required access restrictions for regular developers: fewer manual steps to configure authorization for new projects - -### Disadvantages - -- the project need to keep their forks up to date, which requires more advanced Git skills (managing multiple remotes) +This document was moved to [user/project/merge_requests/authorization_for_merge_requests](../user/project/merge_requests/authorization_for_merge_requests.md) diff --git a/doc/workflow/award_emoji.md b/doc/workflow/award_emoji.md index e6f8b792707..1df0698afd0 100644 --- a/doc/workflow/award_emoji.md +++ b/doc/workflow/award_emoji.md @@ -1,7 +1,7 @@ # Award emoji >**Note:** -This feature was [introduced][1825] in GitLab 8.2. +[Introduced][1825] in GitLab 8.2. When you're collaborating online, you get fewer opportunities for high-fives and thumbs-ups. Emoji can be awarded to issues and merge requests, making @@ -16,7 +16,7 @@ award emoji. ## Sort issues and merge requests on vote count >**Note:** -This feature was [introduced][2871] in GitLab 8.5. +[Introduced][2871] in GitLab 8.5. You can quickly sort issues and merge requests by the number of votes they have received. The sort options can be found in the dropdown menu as "Most @@ -45,7 +45,7 @@ downvotes. ## Award emoji for comments >**Note:** -This feature was [introduced][4291] in GitLab 8.9. +[Introduced][4291] in GitLab 8.9. Award emoji can also be applied to individual comments when you want to celebrate an accomplishment or agree with an opinion. diff --git a/doc/workflow/cherry_pick_changes.md b/doc/workflow/cherry_pick_changes.md index 4a499009842..663ffd3f746 100644 --- a/doc/workflow/cherry_pick_changes.md +++ b/doc/workflow/cherry_pick_changes.md @@ -1,53 +1 @@ -# Cherry-pick changes - ->**Note:** -This feature was [introduced][ce-3514] in GitLab 8.7. - ---- - -GitLab implements Git's powerful feature to [cherry-pick any commit][git-cherry-pick] -with introducing a **Cherry-pick** button in Merge Requests and commit details. - -## Cherry-picking a Merge Request - -After the Merge Request has been merged, a **Cherry-pick** button will be available -to cherry-pick the changes introduced by that Merge Request: - - - ---- - -You can cherry-pick the changes directly into the selected branch or you can opt to -create a new Merge Request with the cherry-pick changes: - - - -## Cherry-picking a Commit - -You can cherry-pick a Commit from the Commit details page: - - - ---- - -Similar to cherry-picking a Merge Request, you can opt to cherry-pick the changes -directly into the target branch or create a new Merge Request to cherry-pick the -changes: - - - ---- - -Please note that when cherry-picking merge commits, the mainline will always be the -first parent. If you want to use a different mainline then you need to do that -from the command line. - -Here is a quick example to cherry-pick a merge commit using the second parent as the -mainline: - -```bash -git cherry-pick -m 2 7a39eb0 -``` - -[ce-3514]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3514 "Cherry-pick button Merge Request" -[git-cherry-pick]: https://git-scm.com/docs/git-cherry-pick "Git cherry-pick documentation" +This document was moved to [user/project/merge_requests/cherry_pick_changes](../user/project/merge_requests/cherry_pick_changes.md). diff --git a/doc/workflow/file_finder.md b/doc/workflow/file_finder.md index b69ae663272..8d87b030c83 100644 --- a/doc/workflow/file_finder.md +++ b/doc/workflow/file_finder.md @@ -1,6 +1,6 @@ # File finder -_**Note:** This feature was [introduced][gh-9889] in GitLab 8.4._ +> [Introduced][gh-9889] in GitLab 8.4. --- diff --git a/doc/workflow/forking_workflow.md b/doc/workflow/forking_workflow.md index 217a4a4012f..733d079bd4a 100644 --- a/doc/workflow/forking_workflow.md +++ b/doc/workflow/forking_workflow.md @@ -38,7 +38,7 @@ Forking a project is in most cases a two-step process. --- After the forking is done, you can start working on the newly created -repository. There, you will have full [Owner](../permissions/permissions.md) +repository. There, you will have full [Owner](../user/permissions.md) access, so you can set it up as you please. ## Merging upstream diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 2b2f140f8bf..2215f37b81a 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -89,7 +89,7 @@ In this case the master branch is deployed on staging. When someone wants to dep And going live with code happens by merging the pre-production branch into the production branch. This workflow where commits only flow downstream ensures that everything has been tested on all environments. If you need to cherry-pick a commit with a hotfix it is common to develop it on a feature branch and merge it into master with a merge request, do not delete the feature branch. -If master is good to go (it should be if you a practicing [continuous delivery](http://martinfowler.com/bliki/ContinuousDelivery.html)) you then merge it to the other branches. +If master is good to go (it should be if you are practicing [continuous delivery](http://martinfowler.com/bliki/ContinuousDelivery.html)) you then merge it to the other branches. If this is not possible because more manual testing is required you can send merge requests from the feature branch to the downstream branches. An 'extreme' version of environment branches are setting up an environment for each feature branch as done by [Teatro](https://teatro.io/). @@ -115,7 +115,7 @@ In this flow it is not common to have a production branch (or git flow master br Merge or pull requests are created in a git management application and ask an assigned person to merge two branches. Tools such as GitHub and Bitbucket choose the name pull request since the first manual action would be to pull the feature branch. -Tools such as GitLab and Gitorious choose the name merge request since that is the final action that is requested of the assignee. +Tools such as GitLab and others choose the name merge request since that is the final action that is requested of the assignee. In this article we'll refer to them as merge requests. If you work on a feature branch for more than a few hours it is good to share the intermediate result with the rest of the team. @@ -228,7 +228,7 @@ We'll discuss the three reasons to merge in master: leveraging code, merge confl If you need to leverage some code that was introduced in master after you created the feature branch you can sometimes solve this by just cherry-picking a commit. If your feature branch has a merge conflict, creating a merge commit is a normal way of solving this. You can prevent some merge conflicts by using [gitattributes](http://git-scm.com/docs/gitattributes) for files that can be in a random order. -For example in GitLab our changelog file is specified in .gitattributes as `CHANGELOG merge=union` so that there are fewer merge conflicts in it. +For example in GitLab our changelog file is specified in .gitattributes as `CHANGELOG.md merge=union` so that there are fewer merge conflicts in it. The last reason for creating merge commits is having long lived branches that you want to keep up to date with the latest state of the project. Martin Fowler, in [his article about feature branches](http://martinfowler.com/bliki/FeatureBranch.html) talks about this Continuous Integration (CI). At GitLab we are guilty of confusing CI with branch testing. Quoting Martin Fowler: "I've heard people say they are doing CI because they are running builds, perhaps using a CI server, on every branch with every commit. diff --git a/doc/workflow/groups.md b/doc/workflow/groups.md index 9b50286b179..a693cc3d0fd 100644 --- a/doc/workflow/groups.md +++ b/doc/workflow/groups.md @@ -53,6 +53,9 @@ If necessary, you can increase the access level of an individual user for a spec ## Requesting access to a group +As a group owner you can enable or disable non members to request access to +your group. Go to the group settings and click on **Allow users to request access**. + As a user, you can request to be a member of a group. Go to the group you'd like to be a member of, and click the **Request Access** button on the right side of your screen. diff --git a/doc/workflow/importing/img/import_projects_from_github_importer.png b/doc/workflow/importing/img/import_projects_from_github_importer.png Binary files differindex b6ed8dd692a..eadd33c695f 100644 --- a/doc/workflow/importing/img/import_projects_from_github_importer.png +++ b/doc/workflow/importing/img/import_projects_from_github_importer.png diff --git a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png b/doc/workflow/importing/img/import_projects_from_github_new_project_page.png Binary files differindex c8f35a50f48..6e91c430a33 100644 --- a/doc/workflow/importing/img/import_projects_from_github_new_project_page.png +++ b/doc/workflow/importing/img/import_projects_from_github_new_project_page.png diff --git a/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png Binary files differnew file mode 100644 index 00000000000..c11863ab10c --- /dev/null +++ b/doc/workflow/importing/img/import_projects_from_github_select_auth_method.png diff --git a/doc/workflow/importing/import_projects_from_github.md b/doc/workflow/importing/import_projects_from_github.md index a2b2a4b88f9..c36dfdb78ec 100644 --- a/doc/workflow/importing/import_projects_from_github.md +++ b/doc/workflow/importing/import_projects_from_github.md @@ -1,56 +1,118 @@ # Import your project from GitHub to GitLab
+Import your projects from GitHub to GitLab with minimal effort.
+
+## Overview
+
>**Note:**
-In order to enable the GitHub import setting, you may also want to
-enable the [GitHub integration][gh-import] in your GitLab instance. This
-configuration is optional, you will be able import your GitHub repositories
-with a Personal Access Token.
+If you are an administrator you can enable the [GitHub integration][gh-import]
+in your GitLab instance sitewide. This configuration is optional, users will be
+able import their GitHub repositories with a [personal access token][gh-token].
+
+- At its current state, GitHub importer can import:
+ - the repository description (GitLab 7.7+)
+ - the Git repository data (GitLab 7.7+)
+ - the issues (GitLab 7.7+)
+ - the pull requests (GitLab 8.4+)
+ - the wiki pages (GitLab 8.4+)
+ - the milestones (GitLab 8.7+)
+ - the labels (GitLab 8.7+)
+ - the release note descriptions (GitLab 8.12+)
+- References to pull requests and issues are preserved (GitLab 8.7+)
+- Repository public access is retained. If a repository is private in GitHub
+ it will be created as private in GitLab as well.
+
+## How it works
+
+When issues/pull requests are being imported, the GitHub importer tries to find
+the GitHub author/assignee in GitLab's database using the GitHub ID. For this
+to work, the GitHub author/assignee should have signed in beforehand in GitLab
+and [**associated their GitHub account**][social sign-in]. If the user is not
+found in GitLab's database, the project creator (most of the times the current
+user that started the import process) is set as the author, but a reference on
+the issue about the original GitHub author is kept.
+
+The importer will create any new namespaces (groups) if they don't exist or in
+the case the namespace is taken, the repository will be imported under the user's
+namespace that started the import process.
+
+## Importing your GitHub repositories
+
+The importer page is visible when you create a new project.
+
+
-At its current state, GitHub importer can import:
+Click on the **GitHub** link and the import authorization process will start.
+There are two ways to authorize access to your GitHub repositories:
-- the repository description (introduced in GitLab 7.7)
-- the git repository data (introduced in GitLab 7.7)
-- the issues (introduced in GitLab 7.7)
-- the pull requests (introduced in GitLab 8.4)
-- the wiki pages (introduced in GitLab 8.4)
-- the milestones (introduced in GitLab 8.7)
-- the labels (introduced in GitLab 8.7)
+1. [Using the GitHub integration][gh-integration] (if it's enabled by your
+ GitLab administrator). This is the preferred way as it's possible to
+ preserve the GitHub authors/assignees. Read more in the [How it works](#how-it-works)
+ section.
+1. [Using a personal access token][gh-token] provided by GitHub.
-With GitLab 8.7+, references to pull requests and issues are preserved.
+
-It is not yet possible to import your cross-repository pull requests (those from
-forks). We are working on improving this in the near future.
+### Authorize access to your repositories using the GitHub integration
-The importer page is visible when you [create a new project][new-project].
-Click on the **GitHub** link and, if you are logged in via the GitHub
-integration, you will be redirected to GitHub for permission to access your
-projects. After accepting, you'll be automatically redirected to the importer.
+If the [GitHub integration][gh-import] is enabled by your GitLab administrator,
+you can use it instead of the personal access token.
+
+1. First you may want to connect your GitHub account to GitLab in order for
+ the username mapping to be correct. Follow the [social sign-in] documentation
+ on how to do so.
+1. Once you connect GitHub, click the **List your GitHub repositories** button
+ and you will be redirected to GitHub for permission to access your projects.
+1. After accepting, you'll be automatically redirected to the importer.
+
+You can now go on and [select which repositories to import](#select-which-repositories-to-import).
+
+### Authorize access to your repositories using a personal access token
+
+>**Note:**
+For a proper author/assignee mapping for issues and pull requests, the
+[GitHub integration][gh-integration] should be used instead of the
+[personal access token][gh-token]. If the GitHub integration is enabled by your
+GitLab administrator, it should be the preferred method to import your repositories.
+Read more in the [How it works](#how-it-works) section.
If you are not using the GitHub integration, you can still perform a one-off
-authorization with GitHub to access your projects.
+authorization with GitHub to grant GitLab access your repositories:
-Alternatively, you can also enter a GitHub Personal Access Token. Once you enter
-your token, you'll be taken to the importer.
+1. Go to <https://github.com/settings/tokens/new>.
+1. Enter a token description.
+1. Check the `repo` scope.
+1. Click **Generate token**.
+1. Copy the token hash.
+1. Go back to GitLab and provide the token to the GitHub importer.
+1. Hit the **List your GitHub repositories** button and wait while GitLab reads
+ your repositories' information. Once done, you'll be taken to the importer
+ page to select the repositories to import.
-
+### Select which repositories to import
----
+After you've authorized access to your GitHub repositories, you will be
+redirected to the GitHub importer page.
+
+From there, you can see the import statuses of your GitHub repositories.
+
+- Those that are being imported will show a _started_ status,
+- those already successfully imported will be green with a _done_ status,
+- whereas those that are not yet imported will have an **Import** button on the
+ right side of the table.
-While at the GitHub importer page, you can see the import statuses of your
-GitHub projects. Those that are being imported will show a _started_ status,
-those already imported will be green, whereas those that are not yet imported
-have an **Import** button on the right side of the table. If you want, you can
-import all your GitHub projects in one go by hitting **Import all projects**
-in the upper left corner.
+If you want, you can import all your GitHub projects in one go by hitting
+**Import all projects** in the upper left corner.
---
-The importer will create any new namespaces if they don't exist or in the
-case the namespace is taken, the project will be imported on the user's
-namespace.
+You can also choose a different name for the project and a different namespace,
+if you have the privileges to do so.
[gh-import]: ../../integration/github.md "GitHub integration"
-[ee-gh]: http://docs.gitlab.com/ee/integration/github.html "GitHub integration for GitLab EE"
[new-project]: ../../gitlab-basics/create-project.md "How to create a new project in GitLab"
+[gh-integration]: #authorize-access-to-your-repositories-using-the-github-integration
+[gh-token]: #authorize-access-to-your-repositories-using-a-personal-access-token
+[social sign-in]: ../../profile/account/social_sign_in.md
diff --git a/doc/workflow/importing/migrating_from_svn.md b/doc/workflow/importing/migrating_from_svn.md index 4828bb5dce6..423b095e69e 100644 --- a/doc/workflow/importing/migrating_from_svn.md +++ b/doc/workflow/importing/migrating_from_svn.md @@ -4,6 +4,112 @@ Subversion (SVN) is a central version control system (VCS) while Git is a distributed version control system. There are some major differences between the two, for more information consult your favorite search engine. +## Overview + +There are two approaches to SVN to Git migration: + +1. [Git/SVN Mirror](#smooth-migration-with-a-gitsvn-mirror-using-subgit) which: + - Makes the GitLab repository to mirror the SVN project. + - Git and SVN repositories are kept in sync; you can use either one. + - Smoothens the migration process and allows to manage migration risks. + +1. [Cut over migration](#cut-over-migration-with-svn2git) which: + - Translates and imports the existing data and history from SVN to Git. + - Is a fire and forget approach, good for smaller teams. + +## Smooth migration with a Git/SVN mirror using SubGit + +[SubGit](https://subgit.com) is a tool for a smooth, stress-free SVN to Git +migration. It creates a writable Git mirror of a local or remote Subversion +repository and that way you can use both Subversion and Git as long as you like. +It requires access to your GitLab server as it talks with the Git repositories +directly in a filesystem level. + +### SubGit prerequisites + +1. Install Oracle JRE 1.8 or newer. On Debian-based Linux distributions you can + follow [this article](http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html). +1. Download SubGit from https://subgit.com/download/. +1. Unpack the downloaded SubGit zip archive to the `/opt` directory. The `subgit` + command will be available at `/opt/subgit-VERSION/bin/subgit`. + +### SubGit configuration + +The first step to mirror you SVN repository in GitLab is to create a new empty +project which will be used as a mirror. For Omnibus installations the path to +the repository will be located at +`/var/opt/gitlab/git-data/repositories/USER/REPO.git` by default. For +installations from source, the default repository directory will be +`/home/git/repositories/USER/REPO.git`. For convenience, assign this path to a +variable: + +``` +GIT_REPO_PATH=/var/opt/gitlab/git-data/repositories/USER/REPOS.git +``` + +SubGit will keep this repository in sync with a remote SVN project. For +convenience, assign your remote SVN project URL to a variable: + +``` +SVN_PROJECT_URL=http://svn.company.com/repos/project +``` + +Next you need to run SubGit to set up a Git/SVN mirror. Make sure the following +`subgit` command is ran on behalf of the same user that keeps ownership of +GitLab Git repositories (by default `git`): + +``` +subgit configure --layout auto $SVN_PROJECT_URL $GIT_REPO_PATH +``` + +Adjust authors and branches mappings, if necessary. Open with your favorite +text editor: + +``` +edit $GIT_REPO_PATH/subgit/authors.txt +edit $GIT_REPO_PATH/subgit/config +``` + +For more information regarding the SubGit configuration options, refer to +[SubGit's documentation](https://subgit.com/documentation.html) website. + +### Initial translation + +Now that SubGit has configured the Git/SVN repos, run `subgit` to perform the +initial translation of existing SVN revisions into the Git repository: + +``` +subgit install $GIT_REPOS_PATH +``` + +After the initial translation is completed, the Git repository and the SVN +project will be kept in sync by `subgit` - new Git commits will be translated to +SVN revisions and new SVN revisions will be translated to Git commits. Mirror +works transparently and does not require any special commands. + +If you would prefer to perform one-time cut over migration with `subgit`, use +the `import` command instead of `install`: + +``` +subgit import $GIT_REPO_PATH +``` + +### SubGit licensing + +Running SubGit in a mirror mode requires a +[registration](https://subgit.com/pricing.html). Registration is free for open +source, academic and startup projects. + +We're currently working on deeper GitLab/SubGit integration. You may track our +progress at [this issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/990). + +### SubGit support + +For any questions related to SVN to GitLab migration with SubGit, you can +contact the SubGit team directly at [support@subgit.com](mailto:support@subgit.com). + +## Cut over migration with svn2git + If you are currently using an SVN repository, you can migrate the repository to Git and GitLab. We recommend a hard cut over - run the migration command once and then have all developers start using the new GitLab repository immediately. @@ -75,5 +181,3 @@ git push --tags origin ## Contribute to this guide We welcome all contributions that would expand this guide with instructions on how to migrate from SVN and other version control systems. - - diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md index 9dc1e9b47e3..b3c73e947f0 100644 --- a/doc/workflow/lfs/lfs_administration.md +++ b/doc/workflow/lfs/lfs_administration.md @@ -45,5 +45,5 @@ In `config/gitlab.yml`: * Currently, storing GitLab Git LFS objects on a non-local storage (like S3 buckets) is not supported * Currently, removing LFS objects from GitLab Git LFS storage is not supported -* LFS authentications via SSH is not supported for the time being -* Only compatible with the GitLFS client versions 1.1.0 or 1.0.2. +* LFS authentications via SSH was added with GitLab 8.12 +* Only compatible with the GitLFS client versions 1.1.0 and up, or 1.0.2. diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md index 9fe065fa680..1a4f213a792 100644 --- a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -35,6 +35,10 @@ Documentation for GitLab instance administrators is under [LFS administration do credentials store is recommended * Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have to add the URL to Git config manually (see #troubleshooting) + +>**Note**: With 8.12 GitLab added LFS support to SSH. The Git LFS communication + still goes over HTTP, but now the SSH client passes the correct credentials + to the Git LFS client, so no action is required by the user. ## Using Git LFS @@ -132,6 +136,10 @@ git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs" ### Credentials are always required when pushing an object +>**Note**: With 8.12 GitLab added LFS support to SSH. The Git LFS communication + still goes over HTTP, but now the SSH client passes the correct credentials + to the Git LFS client, so no action is required by the user. + Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing the LFS object on every push for every object, user HTTPS credentials are required. diff --git a/doc/workflow/merge_requests.md b/doc/workflow/merge_requests.md index d2ec56e6504..a68bb8b27ca 100644 --- a/doc/workflow/merge_requests.md +++ b/doc/workflow/merge_requests.md @@ -1,63 +1 @@ -# Merge Requests - -Merge requests allow you to exchange changes you made to source code - -## Only allow merge requests to be merged if the build succeeds - -You can prevent merge requests from being merged if their build did not succeed -in the project settings page. - - - -Navigate to project settings page and select the `Only allow merge requests to be merged if the build succeeds` check box. - -Please note that you need to have builds configured to enable this feature. - -## Checkout merge requests locally - -Locate the section for your GitLab remote in the `.git/config` file. It looks like this: - -``` -[remote "origin"] - url = https://gitlab.com/gitlab-org/gitlab-ce.git - fetch = +refs/heads/*:refs/remotes/origin/* -``` - -Now add the line `fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*` to this section. - -It should look like this: - -``` -[remote "origin"] - url = https://gitlab.com/gitlab-org/gitlab-ce.git - fetch = +refs/heads/*:refs/remotes/origin/* - fetch = +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/* -``` - -Now you can fetch all the merge requests requests: - -``` -$ git fetch origin -From https://gitlab.com/gitlab-org/gitlab-ce.git - * [new ref] refs/merge-requests/1/head -> origin/merge-requests/1 - * [new ref] refs/merge-requests/2/head -> origin/merge-requests/2 -... -``` - -To check out a particular merge request: - -``` -$ git checkout origin/merge-requests/1 -``` - -## Ignore whitespace changes in Merge Request diff view - - - -If you click the "Hide whitespace changes" button, you can see the diff without whitespace changes. - - - -It is also working on commits compare view. - - +This document was moved to [user/project/merge_requests](../user/project/merge_requests.md). diff --git a/doc/workflow/merge_requests/merge_request_diff.png b/doc/workflow/merge_requests/merge_request_diff.png Binary files differdeleted file mode 100644 index 3ebbfb75ea3..00000000000 --- a/doc/workflow/merge_requests/merge_request_diff.png +++ /dev/null diff --git a/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png b/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png Binary files differdeleted file mode 100644 index a0db535019c..00000000000 --- a/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png +++ /dev/null diff --git a/doc/workflow/merge_when_build_succeeds.md b/doc/workflow/merge_when_build_succeeds.md index 75e1fdff2b2..95afd12ebdb 100644 --- a/doc/workflow/merge_when_build_succeeds.md +++ b/doc/workflow/merge_when_build_succeeds.md @@ -1,15 +1 @@ -# Merge When Build Succeeds - -When reviewing a merge request that looks ready to merge but still has one or more CI builds running, you can set it to be merged automatically when all builds succeed. This way, you don't have to wait for the builds to finish and remember to merge the request manually. - - - -When you hit the "Merge When Build Succeeds" button, the status of the merge request will be updated to represent the impending merge. If you cannot wait for the build to succeed and want to merge immediately, this option is available in the dropdown menu on the right of the main button. - -Both team developers and the author of the merge request have the option to cancel the automatic merge if they find a reason why it shouldn't be merged after all. - - - -When the build succeeds, the merge request will automatically be merged. When the build fails, the author gets a chance to retry any failed builds, or to push new commits to fix the failure. - -When the builds are retried and succeed on the second try, the merge request will automatically be merged after all. When the merge request is updated with new commits, the automatic merge is automatically canceled to allow the new changes to be reviewed. +This document was moved to [user/project/merge_requests/merge_when_build_succeeds](../user/project/merge_requests/merge_when_build_succeeds.md). diff --git a/doc/workflow/notifications.md b/doc/workflow/notifications.md index b4a9c2f3d3e..1b49a5c385f 100644 --- a/doc/workflow/notifications.md +++ b/doc/workflow/notifications.md @@ -67,7 +67,7 @@ In all of the below cases, the notification will be sent to: - Participants: - the author and assignee of the issue/merge request - authors of comments on the issue/merge request - - anyone mentioned by `@username` in the issue/merge request description + - anyone mentioned by `@username` in the issue/merge request title or description - anyone mentioned by `@username` in any of the comments on the issue/merge request ...with notification level "Participating" or higher @@ -89,6 +89,11 @@ In all of the below cases, the notification will be sent to: | Merge merge request | | | New comment | The above, plus anyone mentioned by `@username` in the comment, with notification level "Mention" or higher | + +In addition, if the title or description of an Issue or Merge Request is +changed, notifications will be sent to any **new** mentions by `@username` as +if they had been mentioned in the original text. + You won't receive notifications for Issues, Merge Requests or Milestones created by yourself. You will only receive automatic notifications when somebody else comments or adds changes to the ones that you've created or diff --git a/doc/workflow/project_features.md b/doc/workflow/project_features.md index a523b3facbe..f19e7df8c9a 100644 --- a/doc/workflow/project_features.md +++ b/doc/workflow/project_features.md @@ -32,4 +32,12 @@ Snippets are little bits of code or text. This is a nice place to put code or text that is used semi-regularly within the project, but does not belong in source control. -For example, a specific config file that is used by > the team that is only valid for the people that work on the code. +For example, a specific config file that is used by the team that is only valid for the people that work on the code. + +## Git LFS + +>**Note:** Project-specific LFS setting was added on 8.12 and is available only to admins. + +Git Large File Storage allows you to easily manage large binary files with Git. +With this setting admins can better control which projects are allowed to use +LFS. diff --git a/doc/workflow/protected_branches.md b/doc/workflow/protected_branches.md index 67adfc2f43a..aa48b8f750e 100644 --- a/doc/workflow/protected_branches.md +++ b/doc/workflow/protected_branches.md @@ -1,55 +1 @@ -# Protected Branches - -Permissions in GitLab are fundamentally defined around the idea of having read or write permission to the repository and branches. - -To prevent people from messing with history or pushing code without review, we've created protected branches. - -A protected branch does three simple things: - -* it prevents pushes from everybody except users with Master permission -* it prevents anyone from force pushing to the branch -* it prevents anyone from deleting the branch - -You can make any branch a protected branch. GitLab makes the master branch a protected branch by default. - -To protect a branch, user needs to have at least a Master permission level, see [permissions document](../permissions/permissions.md). - - - -Navigate to project settings page and select `protected branches`. From the `Branch` dropdown menu select the branch you want to protect. - -Some workflows, like [GitLab workflow](gitlab_flow.md), require all users with write access to submit a Merge request in order to get the code into a protected branch. - -Since Masters and Owners can already push to protected branches, that means Developers cannot push to protected branch and need to submit a Merge request. - -However, there are workflows where that is not needed and only protecting from force pushes and branch removal is useful. - -For those workflows, you can allow everyone with write access to push to a protected branch by selecting `Developers can push` check box. - -On already protected branches you can also allow developers to push to the repository by selecting the `Developers can push` check box. - - - -## Wildcard Protected Branches - ->**Note:** -This feature was added in GitLab 8.10. - -1. You can specify a wildcard protected branch, which will protect all branches matching the wildcard. For example: - - | Wildcard Protected Branch | Matching Branches | - |---------------------------+--------------------------------------------------------| - | `*-stable` | `production-stable`, `staging-stable` | - | `production/*` | `production/app-server`, `production/load-balancer` | - | `*gitlab*` | `gitlab`, `gitlab/staging`, `master/gitlab/production` | - -1. Protected branch settings (like "Developers Can Push") apply to all matching branches. - -1. Two different wildcards can potentially match the same branch. For example, `*-stable` and `production-*` would both match a `production-stable` branch. - >**Note:** - If _any_ of these protected branches have "Developers Can Push" set to true, then `production-stable` has it set to true. - -1. If you click on a protected branch's name, you will be presented with a list of all matching branches: - -  - +This document is moved to [user/project/protected_branches.md](../user/project/protected_branches.md) diff --git a/doc/workflow/protected_branches/protected_branches1.png b/doc/workflow/protected_branches/protected_branches1.png Binary files differdeleted file mode 100644 index c00443803de..00000000000 --- a/doc/workflow/protected_branches/protected_branches1.png +++ /dev/null diff --git a/doc/workflow/protected_branches/protected_branches2.png b/doc/workflow/protected_branches/protected_branches2.png Binary files differdeleted file mode 100644 index a4f664d3b21..00000000000 --- a/doc/workflow/protected_branches/protected_branches2.png +++ /dev/null diff --git a/doc/workflow/protected_branches/protected_branches3.png b/doc/workflow/protected_branches/protected_branches3.png Binary files differdeleted file mode 100644 index 2a50cb174bb..00000000000 --- a/doc/workflow/protected_branches/protected_branches3.png +++ /dev/null diff --git a/doc/workflow/revert_changes.md b/doc/workflow/revert_changes.md index 399366b0cdc..cf1292253fc 100644 --- a/doc/workflow/revert_changes.md +++ b/doc/workflow/revert_changes.md @@ -1,64 +1 @@ -# Reverting changes - -_**Note:** This feature was [introduced][ce-1990] in GitLab 8.5._ - ---- - -GitLab implements Git's powerful feature to [revert any commit][git-revert] -with introducing a **Revert** button in Merge Requests and commit details. - -## Reverting a Merge Request - -_**Note:** The **Revert** button will only be available for Merge Requests -created since GitLab 8.5. However, you can still revert a Merge Request -by reverting the merge commit from the list of Commits page._ - -After the Merge Request has been merged, a **Revert** button will be available -to revert the changes introduced by that Merge Request: - - - ---- - -You can revert the changes directly into the selected branch or you can opt to -create a new Merge Request with the revert changes: - - - ---- - -After the Merge Request has been reverted, the **Revert** button will not be -available anymore. - -## Reverting a Commit - -You can revert a Commit from the Commit details page: - - - ---- - -Similar to reverting a Merge Request, you can opt to revert the changes -directly into the target branch or create a new Merge Request to revert the -changes: - - - ---- - -After the Commit has been reverted, the **Revert** button will not be available -anymore. - -Please note that when reverting merge commits, the mainline will always be the -first parent. If you want to use a different mainline then you need to do that -from the command line. - -Here is a quick example to revert a merge commit using the second parent as the -mainline: - -```bash -git revert -m 2 7a39eb0 -``` - -[ce-1990]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1990 "Revert button Merge Request" -[git-revert]: https://git-scm.com/docs/git-revert "Git revert documentation" +This document was moved to [user/project/merge_requests/revert_changes](../user/project/merge_requests/revert_changes.md). diff --git a/doc/workflow/share_projects_with_other_groups.md b/doc/workflow/share_projects_with_other_groups.md index 4c59f59c587..8e50cb03e63 100644 --- a/doc/workflow/share_projects_with_other_groups.md +++ b/doc/workflow/share_projects_with_other_groups.md @@ -1,22 +1,24 @@ # Share Projects with other Groups -In GitLab Enterprise Edition you can share projects with other groups. -This makes it possible to add a group of users to a project with a single action. +You can share projects with other groups. This makes it possible to add a group of users +to a project with a single action. ## Groups as collections of users -In GitLab Community Edition groups are used primarily to [create collections of projects](groups.md). -In GitLab Enterprise Edition you can also take advantage of the fact that groups define collections of _users_, namely the group members. +Groups are used primarily to [create collections of projects](groups.md), but you can also +take advantage of the fact that groups define collections of _users_, namely the group +members. ## Sharing a project with a group of users -The primary mechanism to give a group of users, say 'Engineering', access to a project, say 'Project Acme', in GitLab is to make the 'Engineering' group the owner of 'Project Acme'. -But what if 'Project Acme' already belongs to another group, say 'Open Source'? -This is where the (Enterprise Edition only) group sharing feature can be of use. +The primary mechanism to give a group of users, say 'Engineering', access to a project, +say 'Project Acme', in GitLab is to make the 'Engineering' group the owner of 'Project +Acme'. But what if 'Project Acme' already belongs to another group, say 'Open Source'? +This is where the group sharing feature can be of use. To share 'Project Acme' with the 'Engineering' group, go to the project settings page for 'Project Acme' and use the left navigation menu to go to the 'Groups' section. - + Now you can add the 'Engineering' group with the maximum access level of your choice. After sharing 'Project Acme' with 'Engineering', the project is listed on the group dashboard. diff --git a/doc/workflow/shortcuts.md b/doc/workflow/shortcuts.md index ffcb832cdd7..36516883ef6 100644 --- a/doc/workflow/shortcuts.md +++ b/doc/workflow/shortcuts.md @@ -2,4 +2,75 @@ You can see GitLab's keyboard shortcuts by using 'shift + ?' -
\ No newline at end of file +## Global Shortcuts + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>s</kbd> | Focus search | +| <kbd>?</kbd> | Show/hide this dialog | +| <kbd>⌘</kbd> + <kbd>shift</kbd> + <kbd>p</kbd> | Toggle markdown preview | +| <kbd>↑</kbd> | Edit last comment (when focused on an empty textarea) | + +## Project Files Browsing + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>↑</kbd> | Move selection up | +| <kbd>↓</kbd> | Move selection down | +| <kbd>enter</kbd> | Open selection | + +## Finding Project File + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>↑</kbd> | Move selection up | +| <kbd>↓</kbd> | Move selection down | +| <kbd>enter</kbd> | Open selection | +| <kbd>esc</kbd> | Go back | + +## Global Dashboard + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>g</kbd> + <kbd>a</kbd> | Go to the activity feed | +| <kbd>g</kbd> + <kbd>p</kbd> | Go to projects | +| <kbd>g</kbd> + <kbd>i</kbd> | Go to issues | +| <kbd>g</kbd> + <kbd>m</kbd> | Go to merge requests | + +## Project + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>g</kbd> + <kbd>p</kbd> | Go to the project's home page | +| <kbd>g</kbd> + <kbd>e</kbd> | Go to the project's activity feed | +| <kbd>g</kbd> + <kbd>f</kbd> | Go to files | +| <kbd>g</kbd> + <kbd>c</kbd> | Go to commits | +| <kbd>g</kbd> + <kbd>b</kbd> | Go to builds | +| <kbd>g</kbd> + <kbd>n</kbd> | Go to network graph | +| <kbd>g</kbd> + <kbd>g</kbd> | Go to graphs | +| <kbd>g</kbd> + <kbd>i</kbd> | Go to issues | +| <kbd>g</kbd> + <kbd>m</kbd> | Go to merge requests | +| <kbd>g</kbd> + <kbd>s</kbd> | Go to snippets | +| <kbd>t</kbd> | Go to finding file | +| <kbd>i</kbd> | New issue | + +## Network Graph + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>←</kbd> or <kbd>h</kbd> | Scroll left | +| <kbd>→</kbd> or <kbd>l</kbd> | Scroll right | +| <kbd>↑</kbd> or <kbd>k</kbd> | Scroll up | +| <kbd>↓</kbd> or <kbd>j</kbd> | Scroll down | +| <kbd>shift</kbd> + <kbd>↑</kbd> or <kbd>shift</kbd> + <kbd>k</kbd> | Scroll to top | +| <kbd>shift</kbd> + <kbd>↓</kbd> or <kbd>shift</kbd> + <kbd>j</kbd> | Scroll to bottom | + +## Issues and Merge Requests + +| Keyboard Shortcut | Description | +| ----------------- | ----------- | +| <kbd>a</kbd> | Change assignee | +| <kbd>m</kbd> | Change milestone | +| <kbd>r</kbd> | Reply (quoting selected text) | +| <kbd>e</kbd> | Edit issue/merge request | +| <kbd>l</kbd> | Change label |
\ No newline at end of file diff --git a/doc/workflow/shortcuts.png b/doc/workflow/shortcuts.png Binary files differdeleted file mode 100644 index a9b1c4b4dcc..00000000000 --- a/doc/workflow/shortcuts.png +++ /dev/null diff --git a/doc/workflow/todos.md b/doc/workflow/todos.md index 9524ffd5420..a50ba305deb 100644 --- a/doc/workflow/todos.md +++ b/doc/workflow/todos.md @@ -1,6 +1,6 @@ # GitLab Todos ->**Note:** This feature was [introduced][ce-2817] in GitLab 8.5. +> [Introduced][ce-2817] in GitLab 8.5. When you log into GitLab, you normally want to see where you should spend your time and take some action, or what you need to keep an eye on. All without the diff --git a/doc/workflow/web_editor.md b/doc/workflow/web_editor.md index 1832567a34c..595c7da155b 100644 --- a/doc/workflow/web_editor.md +++ b/doc/workflow/web_editor.md @@ -1,152 +1 @@ -# GitLab Web Editor - -Sometimes it's easier to make quick changes directly from the GitLab interface -than to clone the project and use the Git command line tool. In this feature -highlight we look at how you can create a new file, directory, branch or -tag from the file browser. All of these actions are available from a single -dropdown menu. - -## Create a file - -From a project's files page, click the '+' button to the right of the branch selector. -Choose **New file** from the dropdown. - - - ---- - -Enter a file name in the **File name** box. Then, add file content in the editor -area. Add a descriptive commit message and choose a branch. The branch field -will default to the branch you were viewing in the file browser. If you enter -a new branch name, a checkbox will appear allowing you to start a new merge -request after you commit the changes. - -When you are satisfied with your new file, click **Commit Changes** at the bottom. - - - -## Upload a file - -The ability to create a file is great when the content is text. However, this -doesn't work well for binary data such as images, PDFs or other file types. In -this case you need to upload a file. - -From a project's files page, click the '+' button to the right of the branch -selector. Choose **Upload file** from the dropdown. - - - ---- - -Once the upload dialog pops up there are two ways to upload your file. Either -drag and drop a file on the pop up or use the **click to upload** link. A file -preview will appear once you have selected a file to upload. - -Enter a commit message, choose a branch, and click **Upload file** when you are -ready. - - - -## Create a directory - -To keep files in the repository organized it is often helpful to create a new -directory. - -From a project's files page, click the '+' button to the right of the branch selector. -Choose **New directory** from the dropdown. - - - ---- - -In the new directory dialog enter a directory name, a commit message and choose -the target branch. Click **Create directory** to finish. - - - -## Create a new branch - -There are multiple ways to create a branch from GitLab's web interface. - -### Create a new branch from an issue - ->**Note:** -This feature was [introduced][ce-2808] in GitLab 8.6. - -In case your development workflow dictates to have an issue for every merge -request, you can quickly create a branch right on the issue page which will be -tied with the issue itself. You can see a **New Branch** button after the issue -description, unless there is already a branch with the same name or a referenced -merge request. - - - -Once you click it, a new branch will be created that diverges from the default -branch of your project, by default `master`. The branch name will be based on -the title of the issue and as suffix it will have its ID. Thus, the example -screenshot above will yield a branch named -`2-et-cum-et-sed-expedita-repellat-consequatur-ut-assumenda-numquam-rerum`. - -After the branch is created, you can edit files in the repository to fix -the issue. When a merge request is created based on the newly created branch, -the description field will automatically display the [issue closing pattern] -`Closes #ID`, where `ID` the ID of the issue. This will close the issue once the -merge request is merged. - -### Create a new branch from a project's dashboard - -If you want to make changes to several files before creating a new merge -request, you can create a new branch up front. From a project's files page, -choose **New branch** from the dropdown. - - - ---- - -Enter a new **Branch name**. Optionally, change the **Create from** field -to choose which branch, tag or commit SHA this new branch will originate from. -This field will autocomplete if you start typing an existing branch or tag. -Click **Create branch** and you will be returned to the file browser on this new -branch. - - - ---- - -You can now make changes to any files, as needed. When you're ready to merge -the changes back to master you can use the widget at the top of the screen. -This widget only appears for a period of time after you create the branch or -modify files. - - - -## Create a new tag - -Tags are useful for marking major milestones such as production releases, -release candidates, and more. You can create a tag from a branch or a commit -SHA. From a project's files page, choose **New tag** from the dropdown. - - - ---- - -Give the tag a name such as `v1.0.0`. Choose the branch or SHA from which you -would like to create this new tag. You can optionally add a message and -release notes. The release notes section supports markdown format and you can -also upload an attachment. Click **Create tag** and you will be taken to the tag -list page. - - - -## Tips - -When creating or uploading a new file, or creating a new directory, you can -trigger a new merge request rather than committing directly to master. Enter -a new branch name in the **Target branch** field. You will notice a checkbox -appear that is labeled **Start a new merge request with these changes**. After -you commit the changes you will be taken to a new merge request form. - - - -[ce-2808]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2808 -[issue closing pattern]: ../customization/issue_closing.md +This document was moved to [user/project/repository/web_editor](../user/project/repository/web_editor.md). diff --git a/doc/workflow/wip_merge_requests.md b/doc/workflow/wip_merge_requests.md index 46035a5e6b6..abb8002f442 100644 --- a/doc/workflow/wip_merge_requests.md +++ b/doc/workflow/wip_merge_requests.md @@ -1,13 +1 @@ -# "Work In Progress" Merge Requests - -To prevent merge requests from accidentally being accepted before they're completely ready, GitLab blocks the "Accept" button for merge requests that have been marked a **Work In Progress**. - - - -To mark a merge request a Work In Progress, simply start its title with `[WIP]` or `WIP:`. - - - -To allow a Work In Progress merge request to be accepted again when it's ready, simply remove the `WIP` prefix. - - +This document was moved to [user/project/merge_requests/work_in_progress_merge_requests](../user/project/merge_requests/work_in_progress_merge_requests.md). |
