diff options
Diffstat (limited to 'doc')
375 files changed, 5669 insertions, 2117 deletions
diff --git a/doc/README.md b/doc/README.md index c6500a37aa9..7c33a708dc7 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,5 +1,6 @@ --- toc: false +comments: false --- # GitLab Documentation @@ -85,7 +86,7 @@ Manage your [repositories](user/project/repository/index.md) from the UI (user i ### Issues and Merge Requests (MRs) -- [Discussions](user/discussions/index.md) Threads, comments, and resolvable discussions in issues, commits, and merge requests. +- [Discussions](user/discussions/index.md): Threads, comments, and resolvable discussions in issues, commits, and merge requests. - [Issues](user/project/issues/index.md) - [Project issue Board](user/project/issue_board.md) - [Issues and merge requests templates](user/project/description_templates.md): Create templates for submitting new issues and merge requests. @@ -155,7 +156,7 @@ have access to GitLab administration tools and settings. - [Git LFS configuration](workflow/lfs/lfs_administration.md): Learn how to use LFS under GitLab. - [GitLab Pages configuration](administration/pages/index.md): Configure GitLab Pages. - [High Availability](administration/high_availability/README.md): Configure multiple servers for scaling or high availability. -- [User cohorts](user/admin_area/user_cohorts.md) View user activity over time. +- [User cohorts](user/admin_area/user_cohorts.md): View user activity over time. - [Web terminals](administration/integration/terminal.md): Provide terminal access to environments from within GitLab. - GitLab CI - [CI admin settings](user/admin_area/settings/continuous_integration.md): Define max artifacts size and expiration time. diff --git a/doc/administration/auth/README.md b/doc/administration/auth/README.md index 13bd501e397..ee9b9a9466a 100644 --- a/doc/administration/auth/README.md +++ b/doc/administration/auth/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Authentication and Authorization GitLab integrates with the following external authentication and authorization diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index ad904908472..ad903aef896 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -287,11 +287,11 @@ LDAP email address, and then sign into GitLab via their LDAP credentials. There are two encryption methods, `simple_tls` and `start_tls`. -For either encryption method, if setting `validate_certificates: false`, TLS +For either encryption method, if setting `verify_certificates: false`, TLS encryption is established with the LDAP server before any LDAP-protocol data is exchanged but no validation of the LDAP server's SSL certificate is performed. ->**Note**: Before GitLab 9.5, `validate_certificates: false` is the default if +>**Note**: Before GitLab 9.5, `verify_certificates: false` is the default if unspecified. ## Limitations diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 40099dcc967..e3b10119090 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -32,6 +32,14 @@ prometheus_listen_addr = "localhost:9236" Changes to `/home/git/gitaly/config.toml` are applied when you run `service gitlab restart`. +## Client-side GRPC logs + +Gitaly uses the [gRPC](https://grpc.io/) RPC framework. The Ruby gRPC +client has its own log file which may contain useful information when +you are seeing Gitaly errors. You can control the log level of the +gRPC client with the `GRPC_LOG_LEVEL` environment variable. The +default level is `WARN`. + ## Running Gitaly on its own server > This is an optional way to deploy Gitaly which can benefit GitLab diff --git a/doc/administration/img/circuitbreaker_config.png b/doc/administration/img/circuitbreaker_config.png Binary files differnew file mode 100644 index 00000000000..e811d173634 --- /dev/null +++ b/doc/administration/img/circuitbreaker_config.png diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 652ca9cf454..93c3642a1f1 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -56,29 +56,34 @@ that, login with an Admin account and do following: With PlantUML integration enabled and configured, we can start adding diagrams to our AsciiDoc snippets, wikis and repos using delimited blocks: -``` -[plantuml, format="png", id="myDiagram", width="200px"] --- -Bob->Alice : hello -Alice -> Bob : Go Away --- -``` +- **Markdown** + + ```plantuml + Bob -> Alice : hello + Alice -> Bob : Go Away + ``` -And in Markdown using fenced code blocks: +- **AsciiDoc** - ```plantuml - Bob -> Alice : hello + ``` + [plantuml, format="png", id="myDiagram", width="200px"] + -- + Bob->Alice : hello Alice -> Bob : Go Away + -- ``` -And in reStructuredText using a directive: +- **reStructuredText** -``` -.. plantuml:: + ``` + .. plantuml:: + :caption: Caption with **bold** and *italic* - Bob -> Alice: hello - Alice -> Bob: Go Away -``` + Bob -> Alice: hello + Alice -> Bob: Go Away + ``` + + You can also use the `uml::` directive for compatibility with [sphinxcontrib-plantuml](https://pypi.python.org/pypi/sphinxcontrib-plantuml), but please note that we currently only support the `caption` option. The above blocks will be converted to an HTML img tag with source pointing to the PlantUML instance. If the PlantUML server is correctly configured, this should diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 3587696225c..86b436d89dd 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -142,9 +142,9 @@ and [projects APIs](../api/projects.md). ## Implementation details When GitLab receives an artifacts archive, an archive metadata file is also -generated. This metadata file describes all the entries that are located in the -artifacts archive itself. The metadata file is in a binary format, with -additional GZIP compression. +generated by [GitLab Workhorse]. This metadata file describes all the entries +that are located in the artifacts archive itself. +The metadata file is in a binary format, with additional GZIP compression. GitLab does not extract the artifacts archive in order to save space, memory and disk I/O. It instead inspects the metadata file which contains all the diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 76e071dc673..c9ed2d84ccb 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -18,8 +18,7 @@ other than production, the corresponding logfile is shown here.) It contains a structured log for Rails controller requests received from GitLab, thanks to [Lograge](https://github.com/roidrage/lograge/). Note that -requests from the API [are not yet logged to this -file](https://gitlab.com/gitlab-org/gitlab-ce/issues/36189). +requests from the API are logged to a separate file in `api_json.log`. Each line contains a JSON line that can be ingested by Elasticsearch, Splunk, etc. For example: @@ -73,6 +72,27 @@ In this example we can see that server processed an HTTP request with URL 19:34:53 +0200. Also we can see that request was processed by `Projects::TreeController`. +## `api_json.log` + +Introduced in GitLab 10.0, this file lives in +`/var/log/gitlab/gitlab-rails/api_json.log` for Omnibus GitLab packages or in +`/home/git/gitlab/log/api_json.log` for installations from source. + +It helps you see requests made directly to the API. For example: + +```json +{"time":"2017-10-10T12:30:11.579Z","severity":"INFO","duration":16.84,"db":1.57,"view":15.27,"status":200,"method":"POST","path":"/api/v4/internal/allowed","params":{"action":"git-upload-pack","changes":"_any","gl_repository":null,"project":"root/foobar.git","protocol":"ssh","env":"{}","key_id":"[FILTERED]","secret_token":"[FILTERED]"},"host":"127.0.0.1","ip":"127.0.0.1","ua":"Ruby"} +``` + +This entry above shows an access to an internal endpoint to check whether an +associated SSH key can download the project in question via a `git fetch` or +`git clone`. In this example, we see: + +1. `method`: The HTTP method used to make the request +1. `path`: The relative path of the query +1. `params`: Key-value pairs passed in a query string or HTTP body. Sensitive parameters (e.g. passwords, tokens, etc.) are filtered out. +1. `ua`: The User-Agent of the requester + ## `application.log` This file lives in `/var/log/gitlab/gitlab-rails/application.log` for diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 68efe0aae5c..b9464945cea 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -28,6 +28,12 @@ will be allowed to display the Performance Bar. Make sure _Enable the Performance Bar_ is checked and hit **Save** to save the changes. +Once the Performance Bar is enabled, you will need to press the [<kbd>p</kbd> + +<kbd>b</kbd> keyboard shortcut](../../../workflow/shortcuts.md) to actually +display it. + +You can toggle the Bar using the same shortcut. + ---  diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 6baae20d16a..11d5e077a36 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -20,7 +20,7 @@ it, the client IP needs to be [included in a whitelist][whitelist]. Currently the embedded Prometheus server is not automatically configured to collect metrics from this endpoint. We recommend setting up another Prometheus server, because the embedded server configuration is overwritten once every -[reconfigure of GitLab][reconfigure]. In the future this will not be required. +[reconfigure of GitLab][reconfigure]. In the future this will not be required. ## Metrics available @@ -45,6 +45,8 @@ In this experimental phase, only a few metrics are available: | redis_ping_success | Gauge | 9.4 | Whether or not the last redis ping succeeded | | redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | | user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | +| filesystem_circuitbreaker_latency_seconds | Histogram | 9.5 | Latency of the stat check the circuitbreaker uses to probe a shard | +| filesystem_circuitbreaker | Gauge | 9.5 | Wether or not the circuit for a certain shard is broken or not | ## Metrics shared directory diff --git a/doc/administration/operations/sidekiq_memory_killer.md b/doc/administration/operations/sidekiq_memory_killer.md index b5e78348989..cbffd883774 100644 --- a/doc/administration/operations/sidekiq_memory_killer.md +++ b/doc/administration/operations/sidekiq_memory_killer.md @@ -28,7 +28,7 @@ The MemoryKiller is controlled using environment variables. delayed shutdown is triggered. The default value for Omnibus packages is set [in the omnibus-gitlab repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb). -- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults 900 seconds (15 minutes). When +- `SIDEKIQ_MEMORY_KILLER_GRACE_TIME`: defaults to 900 seconds (15 minutes). When a shutdown is triggered, the Sidekiq process will keep working normally for another 15 minutes. - `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_WAIT`: defaults to 30 seconds. When the grace @@ -36,5 +36,3 @@ The MemoryKiller is controlled using environment variables. Existing jobs get 30 seconds to finish. After that, the MemoryKiller tells Sidekiq to shut down, and an external supervision mechanism (e.g. Runit) must restart Sidekiq. -- `SIDEKIQ_MEMORY_KILLER_SHUTDOWN_SIGNAL`: defaults to `SIGKILL`. The name of - the final signal sent to the Sidekiq process when we want it to shut down. diff --git a/doc/administration/raketasks/github_import.md b/doc/administration/raketasks/github_import.md index 04c70c3644e..6b8ad1b039b 100644 --- a/doc/administration/raketasks/github_import.md +++ b/doc/administration/raketasks/github_import.md @@ -7,6 +7,7 @@ > projects. You can get it from: https://github.com/settings/tokens > - You also need to pass an username as the second argument to the rake task > which will become the owner of the project. +> - You can also resume an import with the same command. To import a project from the list of your GitHub projects available: diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md new file mode 100644 index 00000000000..bac8fa4bd9d --- /dev/null +++ b/doc/administration/raketasks/storage.md @@ -0,0 +1,107 @@ +# Repository Storage Rake Tasks + +This is a collection of rake tasks you can use to help you list and migrate +existing projects from Legacy storage to the new Hashed storage type. + +You can read more about the storage types [here][storage-types]. + +## List projects on Legacy storage + +To have a simple summary of projects using **Legacy** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:legacy_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:legacy_projects + +``` + +------ + +To list projects using **Legacy** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:list_legacy_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_legacy_projects + +``` + +## List projects on Hashed storage + +To have a simple summary of projects using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:hashed_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:hashed_projects + +``` + +------ + +To list projects using **Hashed** storage: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:list_hashed_projects +``` + +**Source Installation** + +```bash +rake gitlab:storage:list_hashed_projects + +``` + +## Migrate existing projects to Hashed storage + +Before migrating your existing projects, you should +[enable hashed storage][storage-migration] for the new projects as well. + +This task will schedule all your existing projects to be migrated to the +**Hashed** storage type: + +**Omnibus Installation** + +```bash +gitlab-rake gitlab:storage:migrate_to_hashed +``` + +**Source Installation** + +```bash +rake gitlab:storage:migrate_to_hashed + +``` + +You can monitor the progress in the _Admin > Monitoring > Background jobs_ screen. +There is a specific Queue you can watch to see how long it will take to finish: **project_migrate_hashed_storage** + +After it reaches zero, you can confirm every project has been migrated by running the commands above. +If you find it necessary, you can run this migration script again to schedule missing projects. + +Any error or warning will be logged in the sidekiq log file. + + +[storage-types]: ../repository_storage_types.md +[storage-migration]: ../repository_storage_types.md#how-to-migrate-to-hashed-storage diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 624a908b3a3..96f436fa7c3 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -105,61 +105,40 @@ When GitLab detects access to the repositories storage fails repeatedly, it can gracefully prevent attempts to access the storage. This might be useful when the repositories are stored somewhere on the network. -The configuration could look as follows: +This can be configured from the admin interface: -**For Omnibus installations** - -1. Edit `/etc/gitlab/gitlab.rb`: + - ```ruby - git_data_dirs({ - "default" => { - "path" => "/mnt/nfs-01/git-data", - "failure_count_threshold" => 10, - "failure_wait_time" => 30, - "failure_reset_time" => 1800, - "storage_timeout" => 5 - } - }) - ``` - -1. Save the file and [reconfigure GitLab][reconfigure-gitlab] for the changes to take effect. - ---- - -**For installations from source** +**Number of access attempts**: The number of attempts GitLab will make to access a +storage when probing a shard. -1. Edit `config/gitlab.yml`: +**Number of failures before backing off**: The number of failures after which +GitLab will start temporarily disabling access to a storage shard on a host. - ```yaml - repositories: - storages: # You must have at least a `default` storage path. - default: - path: /home/git/repositories/ - failure_count_threshold: 10 # number of failures before stopping attempts - failure_wait_time: 30 # Seconds after last access failure before trying again - failure_reset_time: 1800 # Time in seconds to expire failures - storage_timeout: 5 # Time in seconds to wait before aborting a storage access attempt - ``` - -1. Save the file and [restart GitLab][restart-gitlab] for the changes to take effect. - - -**`failure_count_threshold`:** The number of failures of after which GitLab will +**Maximum git storage failures:** The number of failures of after which GitLab will completely prevent access to the storage. The number of failures can be reset in the admin interface: `https://gitlab.example.com/admin/health_check` or using the [api](../api/repository_storage_health.md) to allow access to the storage again. -**`failure_wait_time`:** When access to a storage fails. GitLab will prevent -access to the storage for the time specified here. This allows the filesystem to -recover without. +**Seconds to wait after a storage failure:** When access to a storage fails. GitLab +will prevent access to the storage for the time specified here. This allows the +filesystem to recover. -**`failure_reset_time`:** The time in seconds GitLab will keep failure -information. When no failures occur during this time, information about the +**Seconds before reseting failure information:** The time in seconds GitLab will +keep failure information. When no failures occur during this time, information about the mount is reset. -**`storage_timeout`:** The time in seconds GitLab will try to access storage. -After this time a timeout error will be raised. +**Seconds to wait for a storage access attempt:** The time in seconds GitLab will +try to access storage. After this time a timeout error will be raised. + +To enable the circuitbreaker for repository storage you can flip the feature flag from a rails console: + +``` +Feature.enable('git_storage_circuit_breaker') +``` + +Alternatively it can be enabled by setting `true` in the `GIT_STORAGE_CIRCUIT_BREAKER` environment variable. +This approach would be used when enabling the circuit breaker on a single host. When storage failures occur, this will be visible in the admin interface like this: diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md new file mode 100644 index 00000000000..bc9b6253f1a --- /dev/null +++ b/doc/administration/repository_storage_types.md @@ -0,0 +1,92 @@ +# Repository Storage Types + +> [Introduced][ce-28283] in GitLab 10.0. + +## Legacy Storage + +Legacy Storage is the storage behavior prior to version 10.0. For historical reasons, GitLab replicated the same +mapping structure from the projects URLs: + + * Project's repository: `#{namespace}/#{project_name}.git` + * Project's wiki: `#{namespace}/#{project_name}.wiki.git` + +This structure made simple to migrate from existing solutions to GitLab and easy for Administrators to find where the +repository is stored. + +On the other hand this has some drawbacks: + +Storage location will concentrate huge amount of top-level namespaces. The impact can be reduced by the introduction of [multiple storage paths][storage-paths]. + +Because Backups are a snapshot of the same URL mapping, if you try to recover a very old backup, you need to verify +if any project has taken the place of an old removed project sharing the same URL. This means that `mygroup/myproject` +from your backup may not be the same original project that is today in the same URL. + +Any change in the URL will need to be reflected on disk (when groups / users or projects are renamed). This can add a lot +of load in big installations, and can be even worst if they are using any type of network based filesystem. + +Last, for GitLab Geo, this storage type means we have to synchronize the disk state, replicate renames in the correct +order or we may end-up with wrong repository or missing data temporarily. + +This pattern also exists in other objects stored in GitLab, like issue Attachments, GitLab Pages artifacts, +Docker Containers for the integrated Registry, etc. + +## Hashed Storage + +Hashed Storage is the new storage behavior we are rolling out with 10.0. It's not enabled by default yet, but we +encourage everyone to try-it and take the time to fix any script you may have that depends on the old behavior. + +Instead of coupling project URL and the folder structure where the repository will be stored on disk, we are coupling +a hash, based on the project's ID. + +This makes the folder structure immutable, and therefore eliminates any requirement to synchronize state from URLs to +disk structure. This means that renaming a group, user or project will cost only the database transaction, and will take +effect immediately. + +The hash also helps to spread the repositories more evenly on the disk, so the top-level directory will contain less +folders than the total amount of top-level namespaces. + +Hash format is based on hexadecimal representation of SHA256: `SHA256(project.id)`. +Top-level folder uses first 2 characters, followed by another folder with the next 2 characters. They are both stored in +a special folder `@hashed`, to co-exist with existing Legacy projects: + +```ruby +# Project's repository: +"@hashed/#{hash[0..1]}/#{hash[2..3]}/#{hash}.git" + +# Wiki's repository: +"@hashed/#{hash[0..1]}/#{hash[2..3]}/#{hash}.wiki.git" +``` + +This new format also makes possible to restore backups with confidence, as when restoring a repository from the backup, +you will never mistakenly restore a repository in the wrong project (considering the backup is made after the migration). + +### How to migrate to Hashed Storage + +In GitLab, go to **Admin > Settings**, find the **Repository Storage** section and select +"_Create new projects using hashed storage paths_". + +To migrate your existing projects to the new storage type, check the specific [rake tasks]. + +[ce-28283]: https://gitlab.com/gitlab-org/gitlab-ce/issues/28283 +[rake tasks]: raketasks/storage.md#migrate-existing-projects-to-hashed-storage +[storage-paths]: repository_storage_types.md + +### Hashed Storage coverage + +We are incrementally moving every storable object in GitLab to the Hashed Storage pattern. You can check the current +coverage status below. + +Note that things stored in an S3 compatible endpoint will not have the downsides mentioned earlier, if they are not +prefixed with `#{namespace}/#{project_name}`, which is true for CI Cache and LFS Objects. + +| Storable Object | Legacy Storage | Hashed Storage | S3 Compatible | GitLab Version | +| ----------------| -------------- | -------------- | ------------- | -------------- | +| Repository | Yes | Yes | - | 10.0 | +| Attachments | Yes | Yes | - | 10.2 | +| Avatars | Yes | No | - | - | +| Pages | Yes | No | - | - | +| Docker Registry | Yes | No | - | - | +| CI Build Logs | No | No | - | - | +| CI Artifacts | No | No | - | - | +| CI Cache | No | No | Yes | - | +| LFS Objects | Yes | No | Yes (EEP) | - | diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index 6f1356ddf8f..be538ea250a 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -141,7 +141,7 @@ separate Rails process to debug the issue: 1. Log in to your GitLab account. 1. Copy the URL that is causing problems (e.g. https://gitlab.com/ABC). -1. Obtain the private token for your user (Profile Settings -> Account). +1. Create a Personal Access Token for your user (Profile Settings -> Access Tokens). 1. Bring up the GitLab Rails console. For omnibus users, run: ``` diff --git a/doc/api/README.md b/doc/api/README.md index 6cbea29bda6..f226716c3b5 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -14,6 +14,7 @@ following locations: - [Project-level Variables](project_level_variables.md) - [Group-level Variables](group_level_variables.md) - [Commits](commits.md) +- [Custom Attributes](custom_attributes.md) - [Deployments](deployments.md) - [Deploy Keys](deploy_keys.md) - [Environments](environments.md) @@ -36,6 +37,7 @@ following locations: - [Notes](notes.md) (comments) - [Notification settings](notification_settings.md) - [Open source license templates](templates/licenses.md) +- [Pages Domains](pages_domains.md) - [Pipelines](pipelines.md) - [Pipeline Triggers](pipeline_triggers.md) - [Pipeline Schedules](pipeline_schedules.md) @@ -48,7 +50,6 @@ following locations: - [Repository Files](repository_files.md) - [Runners](runners.md) - [Services](services.md) -- [Session](session.md) - [Settings](settings.md) - [Sidekiq metrics](sidekiq_metrics.md) - [System Hooks](system_hooks.md) @@ -62,7 +63,21 @@ following locations: ## Road to GraphQL -We have changed our plans to move to GraphQL. After reviewing the GraphQL license, anything related to the Facebook BSD plus patent license will not be allowed at GitLab. +Going forward, we will start on moving to +[GraphQL](http://graphql.org/learn/best-practices/) and deprecate the use of +controller-specific endpoints. GraphQL has a number of benefits: + +1. We avoid having to maintain two different APIs. +2. Callers of the API can request only what they need. +3. It is versioned by default. + +It will co-exist with the current v4 REST API. If we have a v5 API, this should +be a compatibility layer on top of GraphQL. + +Although there were some patenting and licensing concerns with GraphQL, these +have been resolved to our satisfaction by the relicensing of the reference +implementations under MIT, and the use of the OWF license for the GraphQL +specification. ## Basic usage @@ -70,27 +85,10 @@ API requests should be prefixed with `api` and the API version. The API version is defined in [`lib/api.rb`][lib-api-url]. For example, the root of the v4 API is at `/api/v4`. -For endpoints that require [authentication](#authentication), you need to pass -a `private_token` parameter via query string or header. If passed as a header, -the header name must be `PRIVATE-TOKEN` (uppercase and with a dash instead of -an underscore). - -Example of a valid API request: - -``` -GET /projects?private_token=9koXpg98eAheJpvBs5tK -``` - -Example of a valid API request using cURL and authentication via header: - -```shell -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects" -``` - -Example of a valid API request using cURL and authentication via a query string: +Example of a valid API request using cURL: ```shell -curl "https://gitlab.example.com/api/v4/projects?private_token=9koXpg98eAheJpvBs5tK" +curl "https://gitlab.example.com/api/v4/projects" ``` The API uses JSON to serialize data. You don't need to specify `.json` at the @@ -98,15 +96,20 @@ end of an API URL. ## Authentication -Most API requests require authentication via a session cookie or token. For +Most API requests require authentication, or will only return public data when +authentication is not provided. For those cases where it is not required, this will be mentioned in the documentation for each individual endpoint. For example, the [`/projects/:id` endpoint](projects.md). -There are three types of access tokens available: +There are three ways to authenticate with the GitLab API: 1. [OAuth2 tokens](#oauth2-tokens) -1. [Private tokens](#private-tokens) 1. [Personal access tokens](#personal-access-tokens) +1. [Session cookie](#session-cookie) + +For admins who want to authenticate with the API as a specific user, or who want to build applications or scripts that do so, two options are available: +1. [Impersonation tokens](#impersonation-tokens) +2. [Sudo](#sudo) If authentication information is invalid or omitted, an error message will be returned with status code `401`: @@ -117,74 +120,84 @@ returned with status code `401`: } ``` -### Session cookie +### OAuth2 tokens -When signing in to GitLab as an ordinary user, a `_gitlab_session` cookie is -set. The API will use this cookie for authentication if it is present, but using -the API to generate a new session cookie is currently not supported. +You can use an [OAuth2 token](oauth2.md) to authenticate with the API by passing it in either the +`access_token` parameter or the `Authorization` header. -### OAuth2 tokens +Example of using the OAuth2 token in a parameter: -You can use an OAuth 2 token to authenticate with the API by passing it either in the -`access_token` parameter or in the `Authorization` header. +```shell +curl https://gitlab.example.com/api/v4/projects?access_token=OAUTH-TOKEN +``` -Example of using the OAuth2 token in the header: +Example of using the OAuth2 token in a header: ```shell curl --header "Authorization: Bearer OAUTH-TOKEN" https://gitlab.example.com/api/v4/projects ``` -Read more about [GitLab as an OAuth2 client](oauth2.md). +Read more about [GitLab as an OAuth2 provider](oauth2.md). -### Private tokens +### Personal access tokens -Private tokens provide full access to the GitLab API. Anyone with access to -them can interact with GitLab as if they were you. You can find or reset your -private token in your account page (`/profile/account`). +You can use a [personal access token][pat] to authenticate with the API by passing it in either the +`private_token` parameter or the `Private-Token` header. -For examples of usage, [read the basic usage section](#basic-usage). +Example of using the personal access token in a parameter: -### Personal access tokens +```shell +curl https://gitlab.example.com/api/v4/projects?private_token=9koXpg98eAheJpvBs5tK +``` -Instead of using your private token which grants full access to your account, -personal access tokens could be a better fit because of their granular -permissions. +Example of using the personal access token in a header: -Once you have your token, pass it to the API using either the `private_token` -parameter or the `PRIVATE-TOKEN` header. For examples of usage, -[read the basic usage section](#basic-usage). +```shell +curl --header "Private-Token: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects +``` + +Read more about [personal access tokens][pat]. + +### Session cookie + +When signing in to the main GitLab application, a `_gitlab_session` cookie is +set. The API will use this cookie for authentication if it is present, but using +the API to generate a new session cookie is currently not supported. -[Read more about personal access tokens.][pat] +The primary user of this authentication method is the web frontend of GitLab itself, +which can use the API as the authenticated user to get a list of their projects, +for example, without needing to explicitly pass an access token. ### Impersonation tokens > [Introduced][ce-9099] in GitLab 9.0. Needs admin permissions. Impersonation tokens are a type of [personal access token][pat] -that can only be created by an admin for a specific user. +that can only be created by an admin for a specific user. They are a great fit +if you want to build applications or scripts that authenticate with the API as a specific user. -They are a better alternative to using the user's password/private token -or using the [Sudo](#sudo) feature which also requires the admin's password -or private token, since the password/token can change over time. Impersonation -tokens are a great fit if you want to build applications or tools which -authenticate with the API as a specific user. +They are an alternative to directly using the user's password or one of their +personal access tokens, and to using the [Sudo](#sudo) feature, since the user's (or admin's, in the case of Sudo) +password/token may not be known or may change over time. For more information, refer to the [users API](users.md#retrieve-user-impersonation-tokens) docs. -For examples of usage, [read the basic usage section](#basic-usage). +Impersonation tokens are used exactly like regular personal access tokens, and can be passed in either the +`private_token` parameter or the `Private-Token` header. ### Sudo > Needs admin permissions. All API requests support performing an API call as if you were another user, -provided your private token is from an administrator account. You need to pass -the `sudo` parameter either via query string or a header with an ID/username of +provided you are authenticated as an administrator with an OAuth or Personal Access Token that has the `sudo` scope. + +You need to pass the `sudo` parameter either via query string or a header with an ID/username of the user you want to perform the operation as. If passed as a header, the -header name must be `SUDO` (uppercase). +header name must be `Sudo`. -If a non administrative `private_token` is provided, then an error message will +If a non administrative access token is provided, an error message will be returned with status code `403`: ```json @@ -193,12 +206,23 @@ be returned with status code `403`: } ``` +If an access token without the `sudo` scope is provided, an error message will +be returned with status code `403`: + +```json +{ + "error": "insufficient_scope", + "error_description": "The request requires higher privileges than provided by the access token.", + "scope": "sudo" +} +``` + If the sudo user ID or username cannot be found, an error message will be returned with status code `404`: ```json { - "message": "404 Not Found: No user id or username for: <id/username>" + "message": "404 User with ID or username '123' Not Found" } ``` @@ -212,7 +236,7 @@ GET /projects?private_token=9koXpg98eAheJpvBs5tK&sudo=username ``` ```shell -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "SUDO: username" "https://gitlab.example.com/api/v4/projects" +curl --header "Private-Token: 9koXpg98eAheJpvBs5tK" --header "Sudo: username" "https://gitlab.example.com/api/v4/projects" ``` Example of a valid API call and a request using cURL with sudo request, @@ -223,7 +247,7 @@ GET /projects?private_token=9koXpg98eAheJpvBs5tK&sudo=23 ``` ```shell -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --header "SUDO: 23" "https://gitlab.example.com/api/v4/projects" +curl --header "Private-Token: 9koXpg98eAheJpvBs5tK" --header "Sudo: 23" "https://gitlab.example.com/api/v4/projects" ``` ## Status codes @@ -440,6 +464,23 @@ Content-Type: application/json } ``` +## Encoding `+` in ISO 8601 dates + +If you need to include a `+` in a query parameter, you may need to use `%2B` instead due +a [W3 recommendation](http://www.w3.org/Addressing/URL/4_URI_Recommentations.html) that +causes a `+` to be interpreted as a space. For example, in an ISO 8601 date, you may want to pass +a time in Mountain Standard Time, such as: + +``` +2017-10-17T23:11:13.000+05:30 +``` + +The correct encoding for the query parameter would be: + +``` +2017-10-17T23:11:13.000%2B05:30 +``` + ## Clients There are many unofficial GitLab API Clients for most of the popular diff --git a/doc/api/commits.md b/doc/api/commits.md index 2a78553782f..5a4a8d888b3 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -181,6 +181,12 @@ Example response: "parent_ids": [ "ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba" ], + "last_pipeline" : { + "id": 8, + "ref": "master", + "sha": "2dc6aa325a317eda67812f05600bdf0fcdc70ab0" + "status": "created" + } "stats": { "additions": 15, "deletions": 10, diff --git a/doc/api/custom_attributes.md b/doc/api/custom_attributes.md new file mode 100644 index 00000000000..8b26f7093ab --- /dev/null +++ b/doc/api/custom_attributes.md @@ -0,0 +1,105 @@ +# Custom Attributes API + +Every API call to custom attributes must be authenticated as administrator. + +## List custom attributes + +Get all custom attributes on a user. + +``` +GET /users/:id/custom_attributes +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes +``` + +Example response: + +```json +[ + { + "key": "location", + "value": "Antarctica" + }, + { + "key": "role", + "value": "Developer" + } +] +``` + +## Single custom attribute + +Get a single custom attribute on a user. + +``` +GET /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` + +Example response: + +```json +{ + "key": "location", + "value": "Antarctica" +} +``` + +## Set custom attribute + +Set a custom attribute on a user. The attribute will be updated if it already exists, +or newly created otherwise. + +``` +PUT /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | +| `value` | string | yes | The value of the custom attribute | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --data "value=Greenland" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` + +Example response: + +```json +{ + "key": "location", + "value": "Greenland" +} +``` + +## Delete custom attribute + +Delete a custom attribute on a user. + +``` +DELETE /users/:id/custom_attributes/:key +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a user | +| `key` | string | yes | The key of the custom attribute | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/users/42/custom_attributes/location +``` diff --git a/doc/api/groups.md b/doc/api/groups.md index c2daa8bc029..99d200c9c93 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -40,6 +40,38 @@ GET /groups ] ``` +When adding the parameter `statistics=true` and the authenticated user is an admin, additional group statistics are returned. + +``` +GET /groups?statistics=true +``` + +```json +[ + { + "id": 1, + "name": "Foobar Group", + "path": "foo-bar", + "description": "An interesting group", + "visibility": "public", + "lfs_enabled": true, + "avatar_url": "http://localhost:3000/uploads/group/avatar/1/foo.jpg", + "web_url": "http://localhost:3000/groups/foo-bar", + "request_access_enabled": false, + "full_name": "Foobar Group", + "full_path": "foo-bar", + "parent_id": null, + "statistics": { + "storage_size" : 212, + "repository_size" : 33, + "lfs_objects_size" : 123, + "job_artifacts_size" : 57 + + } + } +] +``` + You can search for groups by name or path, see below. ## List a group's projects diff --git a/doc/api/issues.md b/doc/api/issues.md index cd2cfe8e430..ec8ff3cd3f3 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -110,7 +110,8 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, - "confidential": false + "confidential": false, + "discussion_locked": false } ] ``` @@ -216,7 +217,8 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, - "confidential": false + "confidential": false, + "discussion_locked": false } ] ``` @@ -323,7 +325,8 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, - "confidential": false + "confidential": false, + "discussion_locked": false } ] ``` @@ -407,6 +410,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -482,6 +486,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -515,6 +520,8 @@ PUT /projects/:id/issues/:issue_iid | `state_event` | string | no | The state event of an issue. Set `close` to close the issue and `reopen` to reopen it | | `updated_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` (requires admin or project owner rights) | | `due_date` | string | no | Date time string in the format YEAR-MONTH-DAY, e.g. `2016-03-11` | +| `discussion_locked` | boolean | no | Flag indicating if the issue's discussion is locked. If the discussion is locked only project members can add or edit comments. | + ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/4/issues/85?state_event=close @@ -558,6 +565,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -657,6 +665,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -735,6 +744,7 @@ Example response: "human_total_time_spent": null }, "confidential": false, + "discussion_locked": false, "_links": { "self": "http://example.com/api/v4/projects/1/issues/2", "notes": "http://example.com/api/v4/projects/1/issues/2/notes", @@ -765,6 +775,44 @@ POST /projects/:id/issues/:issue_iid/unsubscribe curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/issues/93/unsubscribe ``` +Example response: + +```json +{ + "id": 93, + "iid": 12, + "project_id": 5, + "title": "Incidunt et rerum ea expedita iure quibusdam.", + "description": "Et cumque architecto sed aut ipsam.", + "state": "opened", + "created_at": "2016-04-05T21:41:45.217Z", + "updated_at": "2016-04-07T13:02:37.905Z", + "labels": [], + "milestone": null, + "assignee": { + "name": "Edwardo Grady", + "username": "keyon", + "id": 21, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/3e6f06a86cf27fa8b56f3f74f7615987?s=80&d=identicon", + "web_url": "https://gitlab.example.com/keyon" + }, + "author": { + "name": "Vivian Hermann", + "username": "orville", + "id": 11, + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", + "web_url": "https://gitlab.example.com/orville" + }, + "subscribed": false, + "due_date": null, + "web_url": "http://example.com/example/example/issues/12", + "confidential": false, + "discussion_locked": false +} +``` + ## Create a todo Manually creates a todo for the current user on an issue. If @@ -857,7 +905,8 @@ Example response: "downvotes": 0, "due_date": null, "web_url": "http://example.com/example/example/issues/110", - "confidential": false + "confidential": false, + "discussion_locked": false }, "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ci/issues/10", "body": "Vel voluptas atque dicta mollitia adipisci qui at.", diff --git a/doc/api/jobs.md b/doc/api/jobs.md index d60c7c12881..e7060e154f4 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -336,7 +336,7 @@ Parameters | Attribute | Type | Required | Description | |-------------|---------|----------|-------------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `ref_name` | string | yes | The ref from a repository | +| `ref_name` | string | yes | The ref from a repository (can only be branch or tag name, not HEAD or SHA) | | `job` | string | yes | The name of the job | Example request: diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index bff8a2d3e4d..50a971102fb 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -121,6 +121,15 @@ GET /projects/:id/merge_requests?labels=bug,reproduced GET /projects/:id/merge_requests?my_reaction_emoji=star ``` +`project_id` represents the ID of the project where the MR resides. +`project_id` will always equal `target_project_id`. + +In the case of a merge request from the same project, +`source_project_id`, `target_project_id` and `project_id` +will be the same. In the case of a merge request from a fork, +`target_project_id` and `project_id` will be the same and +`source_project_id` will be the fork project's ID. + Parameters: | Attribute | Type | Required | Description | @@ -192,6 +201,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -267,6 +277,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -378,6 +389,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -471,6 +483,7 @@ POST /projects/:id/merge_requests "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -500,6 +513,7 @@ PUT /projects/:id/merge_requests/:merge_request_iid | `labels` | string | no | Labels for MR as a comma-separated list | | `milestone_id` | integer | no | The ID of a milestone | | `remove_source_branch` | boolean | no | Flag indicating if a merge request should remove the source branch when merging | +| `discussion_locked` | boolean | no | Flag indicating if the merge request's discussion is locked. If the discussion is locked only project members can add, edit or resolve comments. | Must include at least one non-required attribute from above. @@ -554,6 +568,7 @@ Must include at least one non-required attribute from above. "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -658,6 +673,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -734,6 +750,7 @@ Parameters: "should_remove_source_branch": true, "force_remove_source_branch": false, "web_url": "http://example.com/example/example/merge_requests/1", + "discussion_locked": false, "time_stats": { "time_estimate": 0, "total_time_spent": 0, @@ -1028,7 +1045,8 @@ Example response: "id": 14, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/a7fa515d53450023c83d62986d0658a8?s=80&d=identicon", - "web_url": "https://gitlab.example.com/francisca" + "web_url": "https://gitlab.example.com/francisca", + "discussion_locked": false }, "assignee": { "name": "Dr. Gabrielle Strosin", diff --git a/doc/api/pages_domains.md b/doc/api/pages_domains.md new file mode 100644 index 00000000000..51962595e33 --- /dev/null +++ b/doc/api/pages_domains.md @@ -0,0 +1,170 @@ +# Pages domains API + +Endpoints for connecting custom domain(s) and TLS certificates in [GitLab Pages](https://about.gitlab.com/features/pages/). + +The GitLab Pages feature must be enabled to use these endpoints. Find out more about [administering](../administration/pages/index.md) and [using](../user/project/pages/index.md) the feature. + +## List pages domains + +Get a list of project pages domains. The user must have permissions to view pages domains. + +```http +GET /projects/:id/pages/domains +``` + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/pages/domains +``` + +```json +[ + { + "domain": "www.domain.example", + "url": "http://www.domain.example" + }, + { + "domain": "ssl.domain.example", + "url": "https://ssl.domain.example", + "certificate": { + "subject": "/O=Example, Inc./OU=Example Origin CA/CN=Example Origin Certificate", + "expired": false, + "certificate": "-----BEGIN CERTIFICATE-----\n … \n-----END CERTIFICATE-----", + "certificate_text": "Certificate:\n … \n" + } + } +] +``` + +## Single pages domain + +Get a single project pages domain. The user must have permissions to view pages domains. + +```http +GET /projects/:id/pages/domains/:domain +``` + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `domain` | string | yes | The domain | + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/pages/domains/www.domain.example +``` + +```json +{ + "domain": "www.domain.example", + "url": "http://www.domain.example" +} +``` + +```bash +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/pages/domains/ssl.domain.example +``` + +```json +{ + "domain": "ssl.domain.example", + "url": "https://ssl.domain.example", + "certificate": { + "subject": "/O=Example, Inc./OU=Example Origin CA/CN=Example Origin Certificate", + "expired": false, + "certificate": "-----BEGIN CERTIFICATE-----\n … \n-----END CERTIFICATE-----", + "certificate_text": "Certificate:\n … \n" + } +} +``` + +## Create new pages domain + +Creates a new pages domain. The user must have permissions to create new pages domains. + +```http +POST /projects/:id/pages/domains +``` + +| Attribute | Type | Required | Description | +| ------------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `domain` | string | yes | The domain | +| `certificate` | file/string | no | The certificate in PEM format with intermediates following in most specific to least specific order.| +| `key` | file/string | no | The certificate key in PEM format. | + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form="domain=ssl.domain.example" --form="certificate=@/path/to/cert.pem" --form="key=@/path/to/key.pem" https://gitlab.example.com/api/v4/projects/5/pages/domains +``` + +```bash +curl --request POST --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form="domain=ssl.domain.example" --form="certificate=$CERT_PEM" --form="key=$KEY_PEM" https://gitlab.example.com/api/v4/projects/5/pages/domains +``` + +```json +{ + "domain": "ssl.domain.example", + "url": "https://ssl.domain.example", + "certificate": { + "subject": "/O=Example, Inc./OU=Example Origin CA/CN=Example Origin Certificate", + "expired": false, + "certificate": "-----BEGIN CERTIFICATE-----\n … \n-----END CERTIFICATE-----", + "certificate_text": "Certificate:\n … \n" + } +} +``` + +## Update pages domain + +Updates an existing project pages domain. The user must have permissions to change an existing pages domains. + +```http +PUT /projects/:id/pages/domains/:domain +``` + +| Attribute | Type | Required | Description | +| ------------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `domain` | string | yes | The domain | +| `certificate` | file/string | no | The certificate in PEM format with intermediates following in most specific to least specific order.| +| `key` | file/string | no | The certificate key in PEM format. | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form="certificate=@/path/to/cert.pem" --form="key=@/path/to/key.pem" https://gitlab.example.com/api/v4/projects/5/pages/domains/ssl.domain.example +``` + +```bash +curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" --form="certificate=$CERT_PEM" --form="key=$KEY_PEM" https://gitlab.example.com/api/v4/projects/5/pages/domains/ssl.domain.example +``` + +```json +{ + "domain": "ssl.domain.example", + "url": "https://ssl.domain.example", + "certificate": { + "subject": "/O=Example, Inc./OU=Example Origin CA/CN=Example Origin Certificate", + "expired": false, + "certificate": "-----BEGIN CERTIFICATE-----\n … \n-----END CERTIFICATE-----", + "certificate_text": "Certificate:\n … \n" + } +} +``` + +## Delete pages domain + +Deletes an existing project pages domain. + +```http +DELETE /projects/:id/pages/domains/:domain +``` + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ---------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `domain` | string | yes | The domain | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/5/pages/domains/ssl.domain.example +``` diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index 890945cfc7e..a6631cab8c3 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -57,7 +57,7 @@ GET /projects/:id/pipelines/:pipeline_id | `pipeline_id` | integer | yes | The ID of a pipeline | ``` -curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/pipeline/46" +curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v4/projects/1/pipelines/46" ``` Example of response diff --git a/doc/api/repositories.md b/doc/api/repositories.md index bccef924375..594babc74be 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -85,7 +85,7 @@ GET /projects/:id/repository/blobs/:sha Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user -- `sha` (required) - The commit or branch name +- `sha` (required) - The blob SHA ## Raw blob content diff --git a/doc/api/services.md b/doc/api/services.md index 49b87a4228c..e642ec964de 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -478,8 +478,8 @@ PUT /projects/:id/services/jira | --------- | ---- | -------- | ----------- | | `url` | string | yes | The URL to the JIRA project which is being linked to this GitLab project, e.g., `https://jira.example.com`. | | `project_key` | string | yes | The short identifier for your JIRA project, all uppercase, e.g., `PROJ`. | -| `username` | string | no | The username of the user created to be used with GitLab/JIRA. | -| `password` | string | no | The password of the user created to be used with GitLab/JIRA. | +| `username` | string | yes | The username of the user created to be used with GitLab/JIRA. | +| `password` | string | yes | The password of the user created to be used with GitLab/JIRA. | | `jira_issue_transition_id` | integer | no | The ID of a transition that moves issues to a closed state. You can find this number under the JIRA workflow administration (**Administration > Issues > Workflows**) by selecting **View** under **Operations** of the desired workflow of your project. The ID of each state can be found inside the parenthesis of each transition name under the **Transitions (id)** column ([see screenshot][trans]). By default, this ID is set to `2`. | ### Delete JIRA service @@ -582,6 +582,40 @@ Delete Mattermost slash command service for a project. DELETE /projects/:id/services/mattermost-slash-commands ``` +## Packagist + +Update your project on Packagist, the main Composer repository, when commits or tags are pushed to GitLab. + +### Create/Edit Packagist service + +Set Packagist service for a project. + +``` +PUT /projects/:id/services/packagist +``` + +Parameters: + +- `username` (**required**) +- `token` (**required**) +- `server` (optional) + +### Delete Packagist service + +Delete Packagist service for a project. + +``` +DELETE /projects/:id/services/packagist +``` + +### Get Packagist service settings + +Get Packagist service settings for a project. + +``` +GET /projects/:id/services/packagist +``` + ## Pipeline-Emails Get emails for GitLab CI pipelines. diff --git a/doc/api/session.md b/doc/api/session.md deleted file mode 100644 index b97e26f34a2..00000000000 --- a/doc/api/session.md +++ /dev/null @@ -1,55 +0,0 @@ -# Session API - ->**Deprecation notice:** -Starting in GitLab 8.11, this feature has been **disabled** for users with -[two-factor authentication][2fa] turned on. These users can access the API -using [personal access tokens] instead. - -You can login with both GitLab and LDAP credentials in order to obtain the -private token. - -``` -POST /session -``` - -| Attribute | Type | Required | Description | -| ---------- | ------- | -------- | -------- | -| `login` | string | yes | The username of the user| -| `email` | string | yes if login is not provided | The email of the user | -| `password` | string | yes | The password of the user | - -```bash -curl --request POST "https://gitlab.example.com/api/v4/session?login=john_smith&password=strongpassw0rd" -``` - -Example response: - -```json -{ - "name": "John Smith", - "username": "john_smith", - "id": 32, - "state": "active", - "avatar_url": null, - "created_at": "2015-01-29T21:07:19.440Z", - "is_admin": true, - "bio": null, - "skype": "", - "linkedin": "", - "twitter": "", - "website_url": "", - "email": "john@example.com", - "theme_id": 1, - "color_scheme_id": 1, - "projects_limit": 10, - "current_sign_in_at": "2015-07-07T07:10:58.392Z", - "identities": [], - "can_create_group": true, - "can_create_project": true, - "two_factor_enabled": false, - "private_token": "9koXpg98eAheJpvBs5tK" -} -``` - -[2fa]: ../user/profile/account/two_factor_authentication.md -[personal access tokens]: ../user/profile/personal_access_tokens.md diff --git a/doc/api/settings.md b/doc/api/settings.md index b78f1252108..4e24e4bbfc3 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -64,38 +64,95 @@ PUT /application/settings | Attribute | Type | Required | Description | | --------- | ---- | :------: | ----------- | -| `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | -| `signup_enabled` | boolean | no | Enable registration. Default is `true`. | -| `password_authentication_enabled` | boolean | no | Enable authentication via a GitLab account password. Default is `true`. | -| `gravatar_enabled` | boolean | no | Enable Gravatar | -| `sign_in_text` | string | no | Text on login page | -| `home_page_url` | string | no | Redirect to this URL when not logged in | -| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `2`. | -| `restricted_visibility_levels` | array of strings | no | Selected levels cannot be used by non-admin users for projects or snippets. Can take `private`, `internal` and `public` as a parameter. Default is null which means there is no restriction. | -| `max_attachment_size` | integer | no | Limit attachment size in MB | -| `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | -| `default_project_visibility` | string | no | What visibility level new projects receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`.| -| `default_snippet_visibility` | string | no | What visibility level new snippets receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`.| -| `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`.| -| `domain_whitelist` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. | -| `domain_blacklist_enabled` | boolean | no | Enable/disable the `domain_blacklist` | -| `domain_blacklist` | array of strings | yes (if `domain_blacklist_enabled` is `true`) | People trying to sign-up with emails from this domain will not be allowed to do so. | -| `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider | -| `after_sign_out_path` | string | no | Where to redirect users after logout | -| `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | -| `repository_storages` | array of strings | no | A list of names of enabled storage paths, taken from `gitlab.yml`. New projects will be created in one of these stores, chosen at random. | -| `enabled_git_access_protocol` | string | no | Enabled protocols for Git access. Allowed values are: `ssh`, `http`, and `nil` to allow both protocols. | -| `koding_enabled` | boolean | no | Enable Koding integration. Default is `false`. | -| `koding_url` | string | yes (if `koding_enabled` is `true`) | The Koding instance URL for integration. | -| `disabled_oauth_sign_in_sources` | Array of strings | no | Disabled OAuth sign-in sources | -| `plantuml_enabled` | boolean | no | Enable PlantUML integration. Default is `false`. | -| `plantuml_url` | string | yes (if `plantuml_enabled` is `true`) | The PlantUML instance URL for integration. | -| `terminal_max_session_time` | integer | no | Maximum time for web terminal websocket connection (in seconds). Set to 0 for unlimited time. | -| `polling_interval_multiplier` | decimal | no | Interval multiplier used by endpoints that perform polling. Set to 0 to disable polling. | -| `rsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded RSA key. Default is `0` (no restriction). `-1` disables RSA keys. -| `dsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded DSA key. Default is `0` (no restriction). `-1` disables DSA keys. -| `ecdsa_key_restriction` | integer | no | The minimum allowed curve size (in bits) of an uploaded ECDSA key. Default is `0` (no restriction). `-1` disables ECDSA keys. -| `ed25519_key_restriction` | integer | no | The minimum allowed curve size (in bits) of an uploaded ED25519 key. Default is `0` (no restriction). `-1` disables ED25519 keys. +| `admin_notification_email` | string | no | Abuse reports will be sent to this address if it is set. Abuse reports are always available in the admin area. | +| `after_sign_out_path` | string | no | Where to redirect users after logout | +| `after_sign_up_text` | string | no | Text shown to the user after signing up | +| `akismet_api_key` | string | no | API key for akismet spam protection | +| `akismet_enabled` | boolean | no | Enable or disable akismet spam protection | +| `circuitbreaker_access_retries | integer | no | The number of attempts GitLab will make to access a storage. | +| `circuitbreaker_backoff_threshold | integer | no | The number of failures after which GitLab will start temporarily disabling access to a storage shard on a host. | +| `circuitbreaker_failure_count_threshold` | integer | no | The number of failures of after which GitLab will completely prevent access to the storage. | +| `circuitbreaker_failure_reset_time` | integer | no | Time in seconds GitLab will keep storage failure information. When no failures occur during this time, the failure information is reset. | +| `circuitbreaker_failure_wait_time` | integer | no | Time in seconds GitLab will block access to a failing storage to allow it to recover. | +| `circuitbreaker_storage_timeout` | integer | no | Seconds to wait for a storage access attempt | +| `clientside_sentry_dsn` | string | no | Required if `clientside_sentry_dsn` is enabled | +| `clientside_sentry_enabled` | boolean | no | Enable Sentry error reporting for the client side | +| `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes | +| `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts | +| `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `2`. | +| `default_group_visibility` | string | no | What visibility level new groups receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | +| `default_project_visibility` | string | no | What visibility level new projects receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | +| `default_projects_limit` | integer | no | Project limit per user. Default is `100000` | +| `default_snippet_visibility` | string | no | What visibility level new snippets receive. Can take `private`, `internal` and `public` as a parameter. Default is `private`. | +| `disabled_oauth_sign_in_sources` | Array of strings | no | Disabled OAuth sign-in sources | +| `domain_blacklist_enabled` | boolean | no | Enable/disable the `domain_blacklist` | +| `domain_blacklist` | array of strings | yes (if `domain_blacklist_enabled` is `true`) | People trying to sign-up with emails from this domain will not be allowed to do so. | +| `domain_whitelist` | array of strings | no | Force people to use only corporate emails for sign-up. Default is null, meaning there is no restriction. | +| `dsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded DSA key. Default is `0` (no restriction). `-1` disables DSA keys. | +| `ecdsa_key_restriction` | integer | no | The minimum allowed curve size (in bits) of an uploaded ECDSA key. Default is `0` (no restriction). `-1` disables ECDSA keys. | +| `ed25519_key_restriction` | integer | no | The minimum allowed curve size (in bits) of an uploaded ED25519 key. Default is `0` (no restriction). `-1` disables ED25519 keys. | +| `email_author_in_body` | boolean | no | Some email servers do not support overriding the email sender name. Enable this option to include the name of the author of the issue, merge request or comment in the email body instead. | +| `enabled_git_access_protocol` | string | no | Enabled protocols for Git access. Allowed values are: `ssh`, `http`, and `nil` to allow both protocols. | +| `gravatar_enabled` | boolean | no | Enable Gravatar | +| `help_page_hide_commercial_content` | boolean | no | Hide marketing-related entries from help | +| `help_page_support_url` | string | no | Alternate support URL for help page | +| `home_page_url` | string | no | Redirect to this URL when not logged in | +| `housekeeping_bitmaps_enabled` | boolean | no | Enable Git pack file bitmap creation | +| `housekeeping_enabled` | boolean | no | Enable or disable git housekeeping | +| `housekeeping_full_repack_period` | integer | no | Number of Git pushes after which an incremental 'git repack' is run. | +| `housekeeping_gc_period` | integer | no | Number of Git pushes after which 'git gc' is run. | +| `housekeeping_incremental_repack_period` | integer | no | Number of Git pushes after which an incremental 'git repack' is run. | +| `html_emails_enabled` | boolean | no | Enable HTML emails | +| `import_sources` | Array of strings | no | Sources to allow project import from, possible values: "github bitbucket gitlab google_code fogbugz git gitlab_project | +| `koding_enabled` | boolean | no | Enable Koding integration. Default is `false`. | +| `koding_url` | string | yes (if `koding_enabled` is `true`) | The Koding instance URL for integration. | +| `max_artifacts_size` | integer | no | Maximum artifacts size in MB | +| `max_attachment_size` | integer | no | Limit attachment size in MB | +| `max_pages_size` | integer | no | Maximum size of pages repositories in MB | +| `metrics_enabled` | boolean | no | Enable influxDB metrics | +| `metrics_host` | string | yes (if `metrics_enabled` is `true`) | InfluxDB host | +| `metrics_method_call_threshold` | integer | yes (if `metrics_enabled` is `true`) | A method call is only tracked when it takes longer than the given amount of milliseconds | +| `metrics_packet_size` | integer | yes (if `metrics_enabled` is `true`) | The amount of datapoints to send in a single UDP packet. | +| `metrics_pool_size` | integer | yes (if `metrics_enabled` is `true`) | The amount of InfluxDB connections to keep open | +| `metrics_port` | integer | no | The UDP port to use for connecting to InfluxDB | +| `metrics_sample_interval` | integer | yes (if `metrics_enabled` is `true`) | The sampling interval in seconds. | +| `metrics_timeout` | integer | yes (if `metrics_enabled` is `true`) | The amount of seconds after which InfluxDB will time out. | +| `password_authentication_enabled` | boolean | no | Enable authentication via a GitLab account password. Default is `true`. | +| `performance_bar_allowed_group_id` | string | no | The group that is allowed to enable the performance bar | +| `performance_bar_enabled` | boolean | no | Allow enabling the performance bar | +| `plantuml_enabled` | boolean | no | Enable PlantUML integration. Default is `false`. | +| `plantuml_url` | string | yes (if `plantuml_enabled` is `true`) | The PlantUML instance URL for integration. | +| `polling_interval_multiplier` | decimal | no | Interval multiplier used by endpoints that perform polling. Set to 0 to disable polling. | +| `project_export_enabled` | boolean | no | Enable project export | +| `prometheus_metrics_enabled` | boolean | no | Enable prometheus metrics | +| `recaptcha_enabled` | boolean | no | Enable recaptcha | +| `recaptcha_private_key` | string | yes (if `recaptcha_enabled` is true) | Private key for recaptcha | +| `recaptcha_site_key` | string | yes (if `recaptcha_enabled` is true) | Site key for recaptcha | +| `repository_checks_enabled` | boolean | no | GitLab will periodically run 'git fsck' in all project and wiki repositories to look for silent disk corruption issues. | +| `repository_storages` | array of strings | no | A list of names of enabled storage paths, taken from `gitlab.yml`. New projects will be created in one of these stores, chosen at random. | +| `require_two_factor_authentication` | boolean | no | Require all users to setup Two-factor authentication | +| `restricted_visibility_levels` | array of strings | no | Selected levels cannot be used by non-admin users for projects or snippets. Can take `private`, `internal` and `public` as a parameter. Default is null which means there is no restriction. | +| `rsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded RSA key. Default is `0` (no restriction). `-1` disables RSA keys. | +| `send_user_confirmation_email` | boolean | no | Send confirmation email on sign-up | +| `sentry_dsn` | string | yes (if `sentry_enabled` is true) | Sentry Data Source Name | +| `sentry_enabled` | boolean | no | Sentry is an error reporting and logging tool which is currently not shipped with GitLab, get it here: https://getsentry.com | +| `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | +| `shared_runners_enabled` | true | no | Enable shared runners for new projects | +| `shared_runners_text` | string | no | Shared runners text | +| `sidekiq_throttling_enabled` | boolean | no | Enable Sidekiq Job Throttling | +| `sidekiq_throttling_factor` | decimal | yes (if `sidekiq_throttling_enabled` is true) | The factor by which the queues should be throttled. A value between 0.0 and 1.0, exclusive. | +| `sidekiq_throttling_queues` | array of strings | yes (if `sidekiq_throttling_enabled` is true) | Choose which queues you wish to throttle | +| `sign_in_text` | string | no | Text on login page | +| `signup_enabled` | boolean | no | Enable registration. Default is `true`. | +| `terminal_max_session_time` | integer | no | Maximum time for web terminal websocket connection (in seconds). Set to 0 for unlimited time. | +| `two_factor_grace_period` | integer | no | Amount of time (in hours) that users are allowed to skip forced configuration of two-factor authentication | +| `unique_ips_limit_enabled` | boolean | no | Limit sign in from multiple ips | +| `unique_ips_limit_per_user` | integer | yes (if `unique_ips_limit_enabled` is true) | Maximum number of ips per user | +| `unique_ips_limit_time_window` | integer | yes (if `unique_ips_limit_enabled` is true) | How many seconds an IP will be counted towards the limit | +| `usage_ping_enabled` | boolean | no | Every week GitLab will report license usage back to GitLab, Inc. | +| `user_default_external` | boolean | no | Newly registered users will by default be external | +| `user_oauth_applications` | boolean | no | Allow users to register any application to use GitLab as an OAuth provider | +| `version_check_enabled` | boolean | no | Let GitLab inform you when an update is available. | ```bash curl --request PUT --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/application/settings?signup_enabled=false&default_project_visibility=internal diff --git a/doc/api/users.md b/doc/api/users.md index 6d5db16b36a..aa711090af1 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -154,6 +154,12 @@ You can search users by creation date time range with: GET /users?created_before=2001-01-02T00:00:00.060Z&created_after=1999-01-02T00:00:00.060 ``` +You can filter by [custom attributes](custom_attributes.md) with: + +``` +GET /users?custom_attributes[key]=value&custom_attributes[other_key]=other_value +``` + ## Single user Get a single user. @@ -404,8 +410,7 @@ GET /user "can_create_group": true, "can_create_project": true, "two_factor_enabled": true, - "external": false, - "private_token": "dd34asd13as" + "external": false } ``` diff --git a/doc/ci/README.md b/doc/ci/README.md index 5cfd82de381..12404eddbe2 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Continuous Integration (GitLab CI)  @@ -42,7 +46,7 @@ digging into specific reference guides. - **The permissions model** - Learn about the access levels a user can have for performing certain CI actions - [User permissions](../user/permissions.md#gitlab-ci) - - [Jobs permissions](../user/permissions.md#jobs-permissions) + - [Job permissions](../user/permissions.md#job-permissions) ## Auto DevOps diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md index 99669a9272a..b0e01d74f7e 100644 --- a/doc/ci/docker/README.md +++ b/doc/ci/docker/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Docker integration - [Using Docker Images](using_docker_images.md) diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index f28c9791bee..0a2419b7ed2 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -31,12 +31,12 @@ There are three methods to enable the use of `docker build` and `docker run` dur The simplest approach is to install GitLab Runner in `shell` execution mode. GitLab Runner then executes job scripts as the `gitlab-runner` user. -1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation). +1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/#installation). 1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: ```bash - sudo gitlab-ci-multi-runner register -n \ + sudo gitlab-runner register -n \ --url https://gitlab.com/ \ --registration-token REGISTRATION_TOKEN \ --executor shell \ @@ -93,7 +93,7 @@ In order to do that, follow the steps: mode: ```bash - sudo gitlab-ci-multi-runner register -n \ + sudo gitlab-runner register -n \ --url https://gitlab.com/ \ --registration-token REGISTRATION_TOKEN \ --executor docker \ @@ -178,7 +178,7 @@ In order to do that, follow the steps: 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: ```bash - sudo gitlab-ci-multi-runner register -n \ + sudo gitlab-runner register -n \ --url https://gitlab.com/ \ --registration-token REGISTRATION_TOKEN \ --executor docker \ @@ -250,6 +250,8 @@ By default, when using `docker:dind`, Docker uses the `vfs` storage driver which copies the filesystem on every run. This is a very disk-intensive operation which can be avoided if a different driver is used, for example `overlay2`. +### Requirements + 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: @@ -271,14 +273,27 @@ which can be avoided if a different driver is used, for example `overlay2`. overlay ``` -1. Use the driver by defining a variable at the top of your `.gitlab-ci.yml`: +### Use driver per project - ``` - variables: - DOCKER_DRIVER: overlay2 - ``` - -> **Note:** +You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`: + +``` +variables: + DOCKER_DRIVER: overlay2 +``` + +### Use driver for every project + +To enable the driver for every project, you can set the environment variable for every build by adding `environment` in the `[[runners]]` section of `config.toml`: + +```toml +environment = ["DOCKER_DRIVER=overlay2"] +``` + +If you're running multiple Runners you will have to modify all configuration files. + +> **Notes:** +- More information about the Runner configuration is available in the [Runner documentation](https://docs.gitlab.com/runner/configuration/). - For more information about using OverlayFS with Docker, you can read [Use the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index fa823ea4721..ecb8f15c851 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -327,10 +327,6 @@ means, that when starting the container without additional options, it will run the database's process, while Runner expects that the image will have no entrypoint or at least will start with a shell as its entrypoint. -Previously we would need to create our own image based on the -`super/sql:experimental` image, set the entrypoint to a shell, and then use -it in job's configuration, e.g.: - Before the new extended Docker configuration options, you would need to create your own image based on the `super/sql:experimental` image, set the entrypoint to a shell and then use it in job's configuration, like: @@ -505,8 +501,8 @@ First start with creating a file named `build_script`: ```bash cat <<EOF > build_script -git clone https://gitlab.com/gitlab-org/gitlab-ci-multi-runner.git /builds/gitlab-org/gitlab-ci-multi-runner -cd /builds/gitlab-org/gitlab-ci-multi-runner +git clone https://gitlab.com/gitlab-org/gitlab-runner.git /builds/gitlab-org/gitlab-runner +cd /builds/gitlab-org/gitlab-runner make EOF ``` diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 796a025b951..7aa7de97c43 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -1,51 +1,46 @@ -## Enable or disable GitLab CI +# How to enable or disable GitLab CI/CD -_To effectively use GitLab CI, you need a valid [`.gitlab-ci.yml`](yaml/README.md) +To effectively use GitLab CI/CD, you need a valid [`.gitlab-ci.yml`](yaml/README.md) file present at the root directory of your project and a [runner](runners/README.md) properly set up. You can read our -[quick start guide](quick_start/README.md) to get you started._ +[quick start guide](quick_start/README.md) to get you started. -If you are using an external CI server like Jenkins or Drone CI, it is advised -to disable GitLab CI in order to not have any conflicts with the commits status +If you are using an external CI/CD server like Jenkins or Drone CI, it is advised +to disable GitLab CI/CD in order to not have any conflicts with the commits status API. --- -GitLab CI is exposed via the `/pipelines` and `/builds` pages of a project. -Disabling GitLab CI in a project does not delete any previous jobs. -In fact, the `/pipelines` and `/builds` pages can still be accessed, although +GitLab CI/CD is exposed via the `/pipelines` and `/jobs` pages of a project. +Disabling GitLab CI/CD in a project does not delete any previous jobs. +In fact, the `/pipelines` and `/jobs` pages can still be accessed, although it's hidden from the left sidebar menu. -GitLab CI is enabled by default on new installations and can be disabled either +GitLab CI/CD is enabled by default on new installations and can be disabled either individually under each project's settings, or site-wide by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. -### Per-project user setting +## Per-project user setting -The setting to enable or disable GitLab CI can be found with the name **Pipelines** -under the **Sharing & Permissions** area of a project's settings along with -**Merge Requests**. Choose one of **Disabled**, **Only team members** and -**Everyone with access** and hit **Save changes** for the settings to take effect. +The setting to enable or disable GitLab CI/CD can be found under your project's +**Settings > General > Permissions**. Choose one of "Disabled", "Only team members" +or "Everyone with access" and hit **Save changes** for the settings to take effect. - + ---- - -### Site-wide administrator setting +## Site-wide admin setting -You can disable GitLab CI site-wide, by modifying the settings in `gitlab.yml` +You can disable GitLab CI/CD site-wide, by modifying the settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations respectively. Two things to note: -1. Disabling GitLab CI, will affect only newly-created projects. Projects that +1. Disabling GitLab CI/CD, will affect only newly-created projects. Projects that had it enabled prior to this modification, will work as before. -1. Even if you disable GitLab CI, users will still be able to enable it in the +1. Even if you disable GitLab CI/CD, users will still be able to enable it in the project's settings. ---- - For installations from source, open `gitlab.yml` with your editor and set `builds` to `false`: diff --git a/doc/ci/environments.md b/doc/ci/environments.md index acd5682841a..c03e16b1b38 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -26,7 +26,7 @@ so every environment can have one or more deployments. GitLab keeps track of your deployments, so you always know what is currently being deployed on your servers. If you have a deployment service such as [Kubernetes][kubernetes-service] enabled for your project, you can use it to assist with your deployments, and -can even access a web terminal for your environment from within GitLab! +can even access a [web terminal](#web-terminals) for your environment from within GitLab! To better understand how environments and deployments work, let's consider an example. We assume that you have already created a project in GitLab and set up @@ -119,7 +119,7 @@ where you can find information of the last deployment status of an environment. Here's how the Environments page looks so far. - + There's a bunch of information there, specifically you can see: @@ -229,7 +229,7 @@ You can find it in the pipeline, job, environment, and deployment views. | Pipelines | Single pipeline | Environments | Deployments | jobs | | --------- | ----------------| ------------ | ----------- | -------| -|  |  |  |  |  | +|  |  |  |  |  | Clicking on the play button in either of these places will trigger the `deploy_prod` job, and the deployment will be recorded under a new @@ -402,7 +402,7 @@ places within GitLab. | In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button | | -------------------- | ------------ | ----------- | -|  |  |  | +|  |  |  | If a merge request is eventually merged to the default branch (in our case `master`) and that branch also deploys to an environment (in our case `staging` @@ -574,7 +574,7 @@ Once configured, GitLab will attempt to retrieve [supported performance metrics] environment which has had a successful deployment. If monitoring data was successfully retrieved, a Monitoring button will appear for each environment. - + Clicking on the Monitoring button will display a new page, showing up to the last 8 hours of performance data. It may take a minute or two for data to appear @@ -593,10 +593,11 @@ Web terminals were added in GitLab 8.15 and are only available to project masters and owners. If you deploy to your environments with the help of a deployment service (e.g., -the [Kubernetes service][kubernetes-service], GitLab can open +the [Kubernetes service][kubernetes-service]), GitLab can open a terminal session to your environment! This is a very powerful feature that allows you to debug issues without leaving the comfort of your web browser. To -enable it, just follow the instructions given in the service documentation. +enable it, just follow the instructions given in the service integration +documentation. Once enabled, your environments will gain a "terminal" button: diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index f094546c3bd..d05b4db953a 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab CI Examples A collection of `.gitlab-ci.yml` files is maintained at the [GitLab CI Yml project][gitlab-ci-templates]. diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index b9f0485290e..bed379b0254 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -1,4 +1,4 @@ -## Running Composer and NPM scripts with deployment via SCP +# Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD This guide covers the building dependencies of a PHP project while compiling assets via an NPM script. @@ -39,13 +39,13 @@ In this particular case, the `npm deploy` script is a Gulp script that does the All these operations will put all files into a `build` folder, which is ready to be deployed to a live server. -### How to transfer files to a live server? +## How to transfer files to a live server You have multiple options: rsync, scp, sftp and so on. For now, we will use scp. To make this work, you need to add a GitLab Secret Variable (accessible on _gitlab.example/your-project-name/variables_). That variable will be called `STAGING_PRIVATE_KEY` and it's the **private** ssh key of your server. -#### Security tip +### Security tip Create a user that has access **only** to the folder that needs to be updated! @@ -69,7 +69,7 @@ In order, this means that: And this is basically all you need in the `before_script` section. -## How to deploy things? +## How to deploy things As we stated above, we need to deploy the `build` folder from the docker image to our server. To do so, we create a new job: @@ -88,7 +88,7 @@ stage_deploy: - ssh -p22 server_user@server_host "rm -rf htdocs/wp-content/themes/_old" ``` -### What's going on here? +Here's the breakdown: 1. `only:dev` means that this build will run only when something is pushed to the `dev` branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want) 2. `ssh-add ...` we will add that private key you added on the web UI to the docker container @@ -99,7 +99,7 @@ stage_deploy: What's the deal with the artifacts? We just tell GitLab CI to keep the `build` directory (later on, you can download that as needed). -#### Why we do it this way? +### Why we do it this way If you're using this only for stage server, you could do this in two steps: @@ -112,7 +112,7 @@ The problem is that there will be a small period of time when you won't have the So we use so many steps because we want to make sure that at any given time we have a functional app in place. -## Where to go next? +## Where to go next Since this was a WordPress project, I gave real life code snippets. Some ideas you can pursuit: diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index f2dd12b67d3..6768a2e012f 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -267,10 +267,10 @@ terminal execute: ```bash # Check using docker executor -gitlab-ci-multi-runner exec docker test:app +gitlab-runner exec docker test:app # Check using shell executor -gitlab-ci-multi-runner exec shell test:app +gitlab-runner exec shell test:app ``` ## Example project diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 73aebaf6d7f..a6ed1c54e16 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -1,10 +1,13 @@ -## Test and Deploy a python application +# Test and Deploy a python application with GitLab CI/CD + This example will guide you how to run tests in your Python application and deploy it automatically as Heroku application. -You can checkout the example [source](https://gitlab.com/ayufan/python-getting-started) and check [CI status](https://gitlab.com/ayufan/python-getting-started/builds?scope=all). +You can checkout the [example source](https://gitlab.com/ayufan/python-getting-started). + +## Configure project -### Configure project This is what the `.gitlab-ci.yml` file looks like for this project: + ```yaml test: script: @@ -41,23 +44,27 @@ This project has three jobs: 2. `staging` - used to automatically deploy staging environment every push to `master` branch 3. `production` - used to automatically deploy production environmnet for every created tag -### Store API keys +## Store API keys + You'll need to create two variables in `Project > Variables`: 1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, 2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). -### Create Heroku application +## Create Heroku application + For each of your environments, you'll need to create a new Heroku application. You can do this through the [Dashboard](https://dashboard.heroku.com/). -### Create runner +## Create Runner + First install [Docker Engine](https://docs.docker.com/installation/). -To build this project you also need to have [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner). +To build this project you also need to have [GitLab Runner](https://docs.gitlab.com/runner). You can use public runners available on `gitlab.com`, but you can register your own: + ``` -gitlab-ci-multi-runner register \ +gitlab-runner register \ --non-interactive \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index 6fa64a67e82..10fd2616fab 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -1,10 +1,13 @@ -## Test and Deploy a ruby application +# Test and Deploy a ruby application with GitLab CI/CD + This example will guide you how to run tests in your Ruby on Rails application and deploy it automatically as Heroku application. You can checkout the example [source](https://gitlab.com/ayufan/ruby-getting-started) and check [CI status](https://gitlab.com/ayufan/ruby-getting-started/builds?scope=all). -### Configure project +## Configure the project + This is what the `.gitlab-ci.yml` file looks like for this project: + ```yaml test: script: @@ -36,23 +39,28 @@ This project has three jobs: 2. `staging` - used to automatically deploy staging environment every push to `master` branch 3. `production` - used to automatically deploy production environment for every created tag -### Store API keys -You'll need to create two variables in `Project > Variables`: +## Store API keys + +You'll need to create two variables in your project's **Settings > CI/CD > Variables**: + 1. `HEROKU_STAGING_API_KEY` - Heroku API key used to deploy staging app, 2. `HEROKU_PRODUCTION_API_KEY` - Heroku API key used to deploy production app. Find your Heroku API key in [Manage Account](https://dashboard.heroku.com/account). -### Create Heroku application +## Create Heroku application + For each of your environments, you'll need to create a new Heroku application. You can do this through the [Dashboard](https://dashboard.heroku.com/). -### Create runner +## Create Runner + First install [Docker Engine](https://docs.docker.com/installation/). To build this project you also need to have [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner). You can use public runners available on `gitlab.com`, but you can register your own: + ``` -gitlab-ci-multi-runner register \ +gitlab-runner register \ --non-interactive \ --url "https://gitlab.com/" \ --registration-token "PROJECT_REGISTRATION_TOKEN" \ @@ -62,6 +70,6 @@ gitlab-ci-multi-runner register \ --docker-postgres latest ``` -With the command above, you create a runner that uses [ruby:2.2](https://hub.docker.com/r/_/ruby/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database. +With the command above, you create a Runner that uses [ruby:2.2](https://hub.docker.com/r/_/ruby/) image and uses [postgres](https://hub.docker.com/r/_/postgres/) database. To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password. diff --git a/doc/ci/examples/test-clojure-application.md b/doc/ci/examples/test-clojure-application.md index 56b746ce025..3b1026d174f 100644 --- a/doc/ci/examples/test-clojure-application.md +++ b/doc/ci/examples/test-clojure-application.md @@ -1,10 +1,10 @@ -## Test a Clojure application +# Test a Clojure application with GitLab CI/CD This example will guide you how to run tests in your Clojure application. You can checkout the example [source](https://gitlab.com/dzaporozhets/clojure-web-application) and check [CI status](https://gitlab.com/dzaporozhets/clojure-web-application/builds?scope=all). -### Configure project +## Configure the project This is what the `.gitlab-ci.yml` file looks like for this project: @@ -23,13 +23,13 @@ before_script: - lein deps - lein migratus migrate -test: - script: +test: + script: - lein test ``` -In before script we install JRE and [Leiningen](http://leiningen.org/). -Sample project uses [migratus](https://github.com/yogthos/migratus) library to manage database migrations. +In before script we install JRE and [Leiningen](http://leiningen.org/). +Sample project uses [migratus](https://github.com/yogthos/migratus) library to manage database migrations. So we added database migration as last step of `before_script` section You can use public runners available on `gitlab.com` for testing your application with such configuration. diff --git a/doc/ci/examples/test-phoenix-application.md b/doc/ci/examples/test-phoenix-application.md index 150698ca04b..f6c81b076bc 100644 --- a/doc/ci/examples/test-phoenix-application.md +++ b/doc/ci/examples/test-phoenix-application.md @@ -1,9 +1,9 @@ -## Test a Phoenix application +# Test a Phoenix application with GitLab CI/CD This example demonstrates the integration of Gitlab CI with Phoenix, Elixir and Postgres. -### Add `.gitlab-ci.yml` file to project +## Add `.gitlab-ci.yml` to project The following `.gitlab-ci.yml` should be added in the root of your repository to trigger CI: @@ -36,7 +36,7 @@ run your migrations. Finally, the test `script` will run your tests. -### Update the Config Settings +## Update the Config Settings In `config/test.exs`, update the database hostname: @@ -45,12 +45,12 @@ config :my_app, MyApp.Repo, hostname: if(System.get_env("CI"), do: "postgres", else: "localhost"), ``` -### Add the Migrations Folder +## Add the Migrations Folder If you do not have any migrations yet, you will need to create an empty `.gitkeep` file in `priv/repo/migrations`. -### Sources +## Sources - https://medium.com/@nahtnam/using-phoenix-on-gitlab-ci-5a51eec81142 - https://davejlong.com/ci-with-phoenix-and-gitlab/ diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 36c6e153d95..c83d3f6f248 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -61,7 +61,7 @@ correctly with your CI jobs: 1. First, make sure you have used [relative URLs](#configuring-the-gitmodules-file) for the submodules located in the same GitLab server. -1. Next, if you are using `gitlab-ci-multi-runner` v1.10+, you can set the +1. Next, if you are using `gitlab-runner` v1.10+, you can set the `GIT_SUBMODULE_STRATEGY` variable to either `normal` or `recursive` to tell the runner to fetch your submodules before the job: ```yaml @@ -71,7 +71,7 @@ correctly with your CI jobs: See the [`.gitlab-ci.yml` reference](yaml/README.md#git-submodule-strategy) for more details about `GIT_SUBMODULE_STRATEGY`. -1. If you are using an older version of `gitlab-ci-multi-runner`, then use +1. If you are using an older version of `gitlab-runner`, then use `git submodule sync/update` in `before_script`: ```yaml diff --git a/doc/ci/img/builds_tab.png b/doc/ci/img/builds_tab.png Binary files differdeleted file mode 100644 index 2d7eec8a949..00000000000 --- a/doc/ci/img/builds_tab.png +++ /dev/null diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differindex 7ded0c97b72..436fed5f465 100644 --- a/doc/ci/img/deployments_view.png +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environments_available.png b/doc/ci/img/environments_available.png Binary files differnew file mode 100644 index 00000000000..2991a309655 --- /dev/null +++ b/doc/ci/img/environments_available.png diff --git a/doc/ci/img/environments_available_staging.png b/doc/ci/img/environments_available_staging.png Binary files differdeleted file mode 100644 index 5c031ad0d9d..00000000000 --- a/doc/ci/img/environments_available_staging.png +++ /dev/null diff --git a/doc/ci/img/environments_dynamic_groups.png b/doc/ci/img/environments_dynamic_groups.png Binary files differindex 0f42b368c5b..45124b3d8d8 100644 --- a/doc/ci/img/environments_dynamic_groups.png +++ b/doc/ci/img/environments_dynamic_groups.png diff --git a/doc/ci/img/environments_link_url.png b/doc/ci/img/environments_link_url.png Binary files differdeleted file mode 100644 index 44010f6aa6f..00000000000 --- a/doc/ci/img/environments_link_url.png +++ /dev/null diff --git a/doc/ci/img/environments_link_url_deployments.png b/doc/ci/img/environments_link_url_deployments.png Binary files differdeleted file mode 100644 index 4f90143527a..00000000000 --- a/doc/ci/img/environments_link_url_deployments.png +++ /dev/null diff --git a/doc/ci/img/environments_link_url_mr.png b/doc/ci/img/environments_link_url_mr.png Binary files differindex 64f134e0b0d..7ce46063062 100644 --- a/doc/ci/img/environments_link_url_mr.png +++ b/doc/ci/img/environments_link_url_mr.png diff --git a/doc/ci/img/environments_manual_action_builds.png b/doc/ci/img/environments_manual_action_builds.png Binary files differdeleted file mode 100644 index e7cf63a1031..00000000000 --- a/doc/ci/img/environments_manual_action_builds.png +++ /dev/null diff --git a/doc/ci/img/environments_manual_action_deployments.png b/doc/ci/img/environments_manual_action_deployments.png Binary files differindex 2b3f6f3edad..93beaa0de54 100644 --- a/doc/ci/img/environments_manual_action_deployments.png +++ b/doc/ci/img/environments_manual_action_deployments.png diff --git a/doc/ci/img/environments_manual_action_environments.png b/doc/ci/img/environments_manual_action_environments.png Binary files differindex e0c07604e7f..9490be63f14 100644 --- a/doc/ci/img/environments_manual_action_environments.png +++ b/doc/ci/img/environments_manual_action_environments.png diff --git a/doc/ci/img/environments_manual_action_jobs.png b/doc/ci/img/environments_manual_action_jobs.png Binary files differnew file mode 100644 index 00000000000..9ae223cf77f --- /dev/null +++ b/doc/ci/img/environments_manual_action_jobs.png diff --git a/doc/ci/img/environments_manual_action_pipelines.png b/doc/ci/img/environments_manual_action_pipelines.png Binary files differindex 82bbae88027..129e44f6fb0 100644 --- a/doc/ci/img/environments_manual_action_pipelines.png +++ b/doc/ci/img/environments_manual_action_pipelines.png diff --git a/doc/ci/img/environments_manual_action_single_pipeline.png b/doc/ci/img/environments_manual_action_single_pipeline.png Binary files differindex 36337cb1870..1eeb4379eb7 100644 --- a/doc/ci/img/environments_manual_action_single_pipeline.png +++ b/doc/ci/img/environments_manual_action_single_pipeline.png diff --git a/doc/ci/img/environments_mr_review_app.png b/doc/ci/img/environments_mr_review_app.png Binary files differindex 7bff84362a3..4bb643d708f 100644 --- a/doc/ci/img/environments_mr_review_app.png +++ b/doc/ci/img/environments_mr_review_app.png diff --git a/doc/ci/img/environments_terminal_button_on_index.png b/doc/ci/img/environments_terminal_button_on_index.png Binary files differindex 6f05b2aa343..061bb7c3c87 100644 --- a/doc/ci/img/environments_terminal_button_on_index.png +++ b/doc/ci/img/environments_terminal_button_on_index.png diff --git a/doc/ci/img/environments_terminal_button_on_show.png b/doc/ci/img/environments_terminal_button_on_show.png Binary files differindex 9469fab99ab..4d24304bc93 100644 --- a/doc/ci/img/environments_terminal_button_on_show.png +++ b/doc/ci/img/environments_terminal_button_on_show.png diff --git a/doc/ci/img/environments_view.png b/doc/ci/img/environments_view.png Binary files differdeleted file mode 100644 index 821352188ef..00000000000 --- a/doc/ci/img/environments_view.png +++ /dev/null diff --git a/doc/ci/img/permissions_settings.png b/doc/ci/img/permissions_settings.png Binary files differdeleted file mode 100644 index 1454c75fd24..00000000000 --- a/doc/ci/img/permissions_settings.png +++ /dev/null diff --git a/doc/ci/img/prometheus_environment_detail_with_metrics.png b/doc/ci/img/prometheus_environment_detail_with_metrics.png Binary files differdeleted file mode 100644 index 214b10624a9..00000000000 --- a/doc/ci/img/prometheus_environment_detail_with_metrics.png +++ /dev/null diff --git a/doc/ci/permissions/README.md b/doc/ci/permissions/README.md index 42eb59f84c8..80d8e46f29c 100644 --- a/doc/ci/permissions/README.md +++ b/doc/ci/permissions/README.md @@ -1,3 +1 @@ -# Users Permissions - This document was moved to [user/permissions.md](../../user/permissions.md#gitlab-ci). diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 2d56b2540ef..f621bf07251 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,4 +1,4 @@ -# Getting started with GitLab CI +# Getting started with GitLab CI/CD >**Note:** Starting from version 8.0, GitLab [Continuous Integration][ci] (CI) is fully integrated into GitLab itself and is [enabled] by default on all diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index 8b51d112a2c..df66810a838 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,4 +1,4 @@ -# Runners +# Configuring GitLab Runners In GitLab CI, Runners run the code defined in [`.gitlab-ci.yml`](../yaml/README.md). They are isolated (virtual) machines that pick up jobs through the coordinator diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 4b79461d55c..d94b472b768 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,4 +1,8 @@ -## GitLab CI Services +--- +comments: false +--- + +# GitLab CI Services GitLab CI uses the `services` keyword to define what docker containers should be linked with your base image. Below is a list of examples you may use. diff --git a/doc/ci/services/docker-services.md b/doc/ci/services/docker-services.md index df36ebaf7d4..787c5e462e4 100644 --- a/doc/ci/services/docker-services.md +++ b/doc/ci/services/docker-services.md @@ -1,5 +1,9 @@ -## GitLab CI Services +--- +comments: false +--- -+ [Using MySQL](mysql.md) -+ [Using PostgreSQL](postgres.md) -+ [Using Redis](redis.md) +# GitLab CI Services + +- [Using MySQL](mysql.md) +- [Using PostgreSQL](postgres.md) +- [Using Redis](redis.md) diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index ebcb92b5db1..a9e6bda9916 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -1,4 +1,4 @@ -# Variables +# GitLab CI/CD Variables When receiving a job from GitLab CI, the [Runner] prepares the build environment. It starts by setting a list of **predefined variables** (environment variables) @@ -43,6 +43,7 @@ future GitLab releases.** | **CI_COMMIT_TAG** | 9.0 | 0.5 | The commit tag name. Present only when building tags. | | **CI_CONFIG_PATH** | 9.4 | 0.5 | The path to CI config file. Defaults to `.gitlab-ci.yml` | | **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled | +| **CI_DISPOSABLE_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. | | **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this job | | **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. | | **CI_ENVIRONMENT_URL** | 9.3 | all | The URL of the environment for this job | @@ -65,6 +66,7 @@ future GitLab releases.** | **CI_PROJECT_PATH** | 8.10 | 0.5 | The namespace with project name | | **CI_PROJECT_PATH_SLUG** | 9.3 | all | `$CI_PROJECT_PATH` lowercased and with everything except `0-9` and `a-z` replaced with `-`. Use in URLs and domain names. | | **CI_PROJECT_URL** | 8.10 | 0.5 | The HTTP address to access project | +| **CI_PROJECT_VISIBILITY** | 10.3 | all | The project visibility (internal, private, public) | | **CI_REGISTRY** | 8.10 | 0.5 | If the Container Registry is enabled it returns the address of GitLab's Container Registry | | **CI_REGISTRY_IMAGE** | 8.10 | 0.5 | If the Container Registry is enabled for the project it returns the address of the registry tied to the specific project | | **CI_REGISTRY_PASSWORD** | 9.0 | all | The password to use to push containers to the GitLab Container Registry | @@ -73,6 +75,7 @@ future GitLab releases.** | **CI_SERVER_NAME** | all | all | The name of CI server that is used to coordinate jobs | | **CI_SERVER_REVISION** | all | all | GitLab revision that is used to schedule jobs | | **CI_SERVER_VERSION** | all | all | GitLab version that is used to schedule jobs | +| **CI_SHARED_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. | | **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job | | **GET_SOURCES_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to fetch sources running a job | | **GITLAB_CI** | all | all | Mark that job is executed in GitLab CI environment | @@ -149,14 +152,15 @@ script: ## Secret variables ->**Notes:** -- This feature requires GitLab Runner 0.4.0 or higher. -- Group-level secret variables added in GitLab 9.4. -- Be aware that secret variables are not masked, and their values can be shown - in the job logs if explicitly asked to do so. If your project is public or - internal, you can set the pipelines private from your project's Pipelines - settings. Follow the discussion in issue [#13784][ce-13784] for masking the - secret variables. +NOTE: **Note:** +Group-level secret variables were added in GitLab 9.4. + +CAUTION: **Important:** +Be aware that secret variables are not masked, and their values can be shown +in the job logs if explicitly asked to do so. If your project is public or +internal, you can set the pipelines private from your [project's Pipelines +settings](../../user/project/pipelines/settings.md#visibility-of-pipelines). +Follow the discussion in issue [#13784][ce-13784] for masking the secret variables. GitLab CI allows you to define per-project or per-group secret variables that are set in the pipeline environment. The secret variables are stored out of @@ -171,6 +175,8 @@ Likewise, group-level secret variables can be added by going to your group's **Settings > CI/CD**, then finding the section called **Secret variables**. Any variables of [subgroups] will be inherited recursively. + + Once you set them, they will be available for all subsequent pipelines. You can also [protect your variables](#protected-secret-variables). @@ -202,7 +208,7 @@ are set in the build environment. These variables are only defined for the project services that you are using to learn which variables they define. An example project service that defines deployment variables is -[Kubernetes Service](../../user/project/integrations/kubernetes.md). +[Kubernetes Service](../../user/project/integrations/kubernetes.md#deployment-variables). ## Debug tracing @@ -439,7 +445,7 @@ export CI_REGISTRY_USER="gitlab-ci-token" export CI_REGISTRY_PASSWORD="longalfanumstring" ``` -[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 +[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables" [eep]: https://about.gitlab.com/gitlab-ee/ "Available only in GitLab Enterprise Edition Premium" [envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md diff --git a/doc/ci/variables/img/secret_variables.png b/doc/ci/variables/img/secret_variables.png Binary files differnew file mode 100644 index 00000000000..f70935069d9 --- /dev/null +++ b/doc/ci/variables/img/secret_variables.png diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index aad81843299..6ad70707594 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -95,6 +95,12 @@ be an array or a multi-line string. `after_script` is used to define the command that will be run after for all jobs. This has to be an array or a multi-line string. +> **Note:** +The `before_script` and the main `script` are concatenated and run in a single context/container. +The `after_script` is run separately, so depending on the executor, changes done +outside of the working tree might not be visible, e.g. software installed in the +`before_script`. + ### stages `stages` is used to define stages that can be used by jobs. @@ -1570,6 +1576,11 @@ Read more on [GitLab Pages user documentation](../../user/project/pages/index.md Each instance of GitLab CI has an embedded debug tool called Lint. You can find the link under `/ci/lint` of your gitlab instance. +## Using reserved keywords + +If you get validation error when using specific values (e.g., `true` or `false`), +try to quote them, or change them to a different form (e.g., `/bin/true`). + ## Skipping jobs If your commit message contains `[ci skip]` or `[skip ci]`, using any diff --git a/doc/development/README.md b/doc/development/README.md index 3096d9f25f0..0cafc112b6b 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -1,71 +1,96 @@ -# Development +--- +comments: false +--- -## Outside of docs +# GitLab development guides -- [CONTRIBUTING.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md) main contributing guide -- [PROCESS.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/PROCESS.md) contributing process -- [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) to install a development version +## Get started! -## Styleguides - -- [API styleguide](api_styleguide.md) Use this styleguide if you are - contributing to the API. -- [Documentation styleguide](doc_styleguide.md) Use this styleguide if you are - contributing to documentation. -- [Writing documentation](writing_documentation.md) - - [Distinction between general documentation and technical articles](writing_documentation.md#distinction-between-general-documentation-and-technical-articles) -- [SQL Migration Style Guide](migration_style_guide.md) for creating safe SQL migrations -- [Testing standards and style guidelines](testing.md) -- [UX guide](ux_guide/index.md) for building GitLab with existing CSS styles and elements -- [Frontend guidelines](fe_guide/index.md) -- [SQL guidelines](sql.md) for working with SQL queries -- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers -- [`Gemfile` guidelines](gemfile.md) +- Setup GitLab's development environment with [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) +- [GitLab contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md) +- [Architecture](architecture.md) of GitLab +- [Rake tasks](rake_tasks.md) for development -## Process +## Processes +- [GitLab core team & GitLab Inc. contribution process](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/PROCESS.md) - [Generate a changelog entry with `bin/changelog`](changelog.md) -- [Limit conflicts with EE when developing on CE](limit_ee_conflicts.md) - [Code review guidelines](code_review.md) for reviewing code and having code reviewed. -- [Merge request performance guidelines](merge_request_performance_guidelines.md) - for ensuring merge requests do not negatively impact GitLab performance +- [Limit conflicts with EE when developing on CE](limit_ee_conflicts.md) -## Backend howtos +## UX and frontend guides -- [Architecture](architecture.md) of GitLab +- [UX guide](ux_guide/index.md) for building GitLab with existing CSS styles and elements +- [Frontend guidelines](fe_guide/index.md) + +## Backend guides + +- [API styleguide](api_styleguide.md) Use this styleguide if you are + contributing to the API. +- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers +- [Working with Gitaly](gitaly.md) +- [Manage feature flags](feature_flags.md) +- [View sent emails or preview mailers](emails.md) +- [Shell commands](shell_commands.md) in the GitLab codebase +- [`Gemfile` guidelines](gemfile.md) +- [Sidekiq debugging](sidekiq_debugging.md) - [Gotchas](gotchas.md) to avoid +- [Issue and merge requests state models](object_state_models.md) - [How to dump production data to staging](db_dump.md) + +## Performance guides + - [Instrumentation](instrumentation.md) - [Performance guidelines](performance.md) -- [Rake tasks](rake_tasks.md) for development -- [Shell commands](shell_commands.md) in the GitLab codebase -- [Sidekiq debugging](sidekiq_debugging.md) -- [Object state models](object_state_models.md) -- [Building a package for testing purposes](build_test_package.md) -- [Manage feature flags](feature_flags.md) -- [View sent emails or preview mailers](emails.md) +- [Merge request performance guidelines](merge_request_performance_guidelines.md) + for ensuring merge requests do not negatively impact GitLab performance + +## Databases guides -## Databases +### Migrations -- [Merge Request Checklist](database_merge_request_checklist.md) - [What requires downtime?](what_requires_downtime.md) +- [SQL guidelines](sql.md) for working with SQL queries +- [Migrations style guide](migration_style_guide.md) for creating safe SQL migrations +- [Post deployment migrations](post_deployment_migrations.md) +- [Background migrations](background_migrations.md) +- [Swapping tables](swapping_tables.md) + +### Best practices + +- [Merge Request checklist](database_merge_request_checklist.md) - [Adding database indexes](adding_database_indexes.md) -- [Post Deployment Migrations](post_deployment_migrations.md) -- [Foreign Keys & Associations](foreign_keys.md) -- [Serializing Data](serializing_data.md) -- [Polymorphic Associations](polymorphic_associations.md) -- [Single Table Inheritance](single_table_inheritance.md) -- [Background Migrations](background_migrations.md) -- [Storing SHA1 Hashes As Binary](sha1_as_binary.md) -- [Iterating Tables In Batches](iterating_tables_in_batches.md) -- [Ordering Table Columns](ordering_table_columns.md) -- [Verifying Database Capabilities](verifying_database_capabilities.md) -- [Hash Indexes](hash_indexes.md) -- [Swapping Tables](swapping_tables.md) - -## i18n - -- [Internationalization for GitLab](i18n_guide.md) +- [Foreign keys & associations](foreign_keys.md) +- [Single table inheritance](single_table_inheritance.md) +- [Polymorphic associations](polymorphic_associations.md) +- [Serializing data](serializing_data.md) +- [Hash indexes](hash_indexes.md) +- [Storing SHA1 hashes as binary](sha1_as_binary.md) +- [Iterating tables in batches](iterating_tables_in_batches.md) +- [Ordering table columns](ordering_table_columns.md) +- [Verifying database capabilities](verifying_database_capabilities.md) + +## Testing guides + +- [Testing standards and style guidelines](testing_guide/index.md) +- [Frontend testing standards and style guidelines](testing_guide/frontend_testing.md) + +## Documentation guides + +- [Documentation styleguide](doc_styleguide.md): Use this styleguide if you are + contributing to the documentation. +- [Writing documentation](writing_documentation.md) + - [Distinction between general documentation and technical articles](writing_documentation.md#distinction-between-general-documentation-and-technical-articles) + +## Internationalization (i18n) guides + +- [Introduction](i18n/index.md) +- [Externalization](i18n/externalization.md) +- [Translation](i18n/translation.md) + +## Build guides + +- [Building a package for testing purposes](build_test_package.md) ## Compliance diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index f83a60e49e8..5452b0e7a2f 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -215,14 +215,29 @@ same time will ensure that both existing and new data is migrated. In the next release we can remove the `after_commit` hooks and related code. We will also need to add a post-deployment migration that consumes any remaining -jobs. Such a migration would look like this: +jobs and manually run on any un-migrated rows. Such a migration would look like +this: ```ruby class ConsumeRemainingExtractServicesUrlJobs < ActiveRecord::Migration disable_ddl_transaction! + class Service < ActiveRecord::Base + include ::EachBatch + + self.table_name = 'services' + end + def up + # This must be included Gitlab::BackgroundMigration.steal('ExtractServicesUrl') + + # This should be included, but can be skipped - see below + Service.where(url: nil).each_batch(of: 50) do |batch| + range = batch.pluck('MIN(id)', 'MAX(id)').first + + Gitlab::BackgroundMigration::ExtractServicesUrl.new.perform(*range) + end end def down @@ -230,6 +245,15 @@ class ConsumeRemainingExtractServicesUrlJobs < ActiveRecord::Migration end ``` +The final step runs for any un-migrated rows after all of the jobs have been +processed. This is in case a Sidekiq process running the background migrations +received SIGKILL, leading to the jobs being lost. (See +[more reliable Sidekiq queue][reliable-sidekiq] for more information.) + +If the application does not depend on the data being 100% migrated (for +instance, the data is advisory, and not mission-critical), then this final step +can be skipped. + This migration will then process any jobs for the ExtractServicesUrl migration and continue once all jobs have been processed. Once done you can safely remove the `services.properties` column. @@ -254,6 +278,9 @@ for more details. 1. Make sure that background migration jobs are idempotent. 1. Make sure that tests you write are not false positives. +1. Make sure that if the data being migrated is critical and cannot be lost, the + clean-up migration also checks the final state of the data before completing. [migrations-readme]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md [issue-rspec-hooks]: https://gitlab.com/gitlab-org/gitlab-ce/issues/35351 +[reliable-sidekiq]: https://gitlab.com/gitlab-org/gitlab-ce/issues/36791 diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md index 798f40eef3d..0e4ffbd7910 100644 --- a/doc/development/doc_styleguide.md +++ b/doc/development/doc_styleguide.md @@ -459,11 +459,11 @@ Rendered example: ### cURL commands - Use `https://gitlab.example.com/api/v4/` as an endpoint. -- Wherever needed use this private token: `9koXpg98eAheJpvBs5tK`. +- Wherever needed use this personal access token: `9koXpg98eAheJpvBs5tK`. - Always put the request first. `GET` is the default so you don't have to include it. - Use double quotes to the URL when it includes additional parameters. -- Prefer to use examples using the private token and don't pass data of +- Prefer to use examples using the personal access token and don't pass data of username and password. | Methods | Description | diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md new file mode 100644 index 00000000000..932a44f65e4 --- /dev/null +++ b/doc/development/ee_features.md @@ -0,0 +1,382 @@ +# Guidelines for implementing Enterprise Edition feature + +- **Write the code and the tests.**: As with any code, EE features should have + good test coverage to prevent regressions. +- **Write documentation.**: Add documentation to the `doc/` directory. Describe + the feature and include screenshots, if applicable. +- **Submit a MR to the `www-gitlab-com` project.**: Add the new feature to the + [EE features list][ee-features-list]. + +## Act as CE when unlicensed + +Since the implementation of [GitLab CE features to work with unlicensed EE instance][ee-as-ce] +GitLab Enterprise Edition should work like GitLab Community Edition +when no license is active. So EE features always should be guarded by +`project.feature_available?` or `group.feature_available?` (or +`License.feature_available?` if it is a system-wide feature). + +CE specs should remain untouched as much as possible and extra specs +should be added for EE. Licensed features can be stubbed using the +spec helper `stub_licensed_features` in `EE::LicenseHelpers`. + +[ee-as-ce]: https://gitlab.com/gitlab-org/gitlab-ee/issues/2500 + +## Separation of EE code + +We want a [single code base][] eventually, but before we reach the goal, +we still need to merge changes from GitLab CE to EE. To help us get there, +we should make sure that we no longer edit CE files in place in order to +implement EE features. + +Instead, all EE codes should be put inside the `ee/` top-level directory, and +tests should be put inside `spec/ee/`. We don't use `ee/spec` for now due to +technical limitation. The rest of codes should be as close as to the CE files. + +[single code base]: https://gitlab.com/gitlab-org/gitlab-ee/issues/2952#note_41016454 + +### EE-only features + +If the feature being developed is not present in any form in CE, we don't +need to put the codes under `EE` namespace. For example, an EE model could +go into: `ee/app/models/awesome.rb` using `Awesome` as the class name. This +is applied not only to models. Here's a list of other examples: + +- `ee/app/controllers/foos_controller.rb` +- `ee/app/finders/foos_finder.rb` +- `ee/app/helpers/foos_helper.rb` +- `ee/app/mailers/foos_mailer.rb` +- `ee/app/models/foo.rb` +- `ee/app/policies/foo_policy.rb` +- `ee/app/serializers/foo_entity.rb` +- `ee/app/serializers/foo_serializer.rb` +- `ee/app/services/foo/create_service.rb` +- `ee/app/validators/foo_attr_validator.rb` +- `ee/app/workers/foo_worker.rb` + +### EE features based on CE features + +For features that build on existing CE features, write a module in the +`EE` namespace and `prepend` it in the CE class. This makes conflicts +less likely to happen during CE to EE merges because only one line is +added to the CE class - the `prepend` line. + +Since the module would require an `EE` namespace, the file should also be +put in an `ee/` sub-directory. For example, we want to extend the user model +in EE, so we have a module called `::EE::User` put inside +`ee/app/models/ee/user.rb`. + +This is also not just applied to models. Here's a list of other examples: + +- `ee/app/controllers/ee/foos_controller.rb` +- `ee/app/finders/ee/foos_finder.rb` +- `ee/app/helpers/ee/foos_helper.rb` +- `ee/app/mailers/ee/foos_mailer.rb` +- `ee/app/models/ee/foo.rb` +- `ee/app/policies/ee/foo_policy.rb` +- `ee/app/serializers/ee/foo_entity.rb` +- `ee/app/serializers/ee/foo_serializer.rb` +- `ee/app/services/ee/foo/create_service.rb` +- `ee/app/validators/ee/foo_attr_validator.rb` +- `ee/app/workers/ee/foo_worker.rb` + +#### Overriding CE methods + +To override a method present in the CE codebase, use `prepend`. It +lets you override a method in a class with a method from a module, while +still having access the class's implementation with `super`. + +There are a few gotchas with it: + +- you should always add a `raise NotImplementedError unless defined?(super)` + guard clause in the "overrider" method to ensure that if the method gets + renamed in CE, the EE override won't be silently forgotten. +- when the "overrider" would add a line in the middle of the CE + implementation, you should refactor the CE method and split it in + smaller methods. Or create a "hook" method that is empty in CE, + and with the EE-specific implementation in EE. +- when the original implementation contains a guard clause (e.g. + `return unless condition`), we cannot easily extend the behaviour by + overriding the method, because we can't know when the overridden method + (i.e. calling `super` in the overriding method) would want to stop early. + In this case, we shouldn't just override it, but update the original method + to make it call the other method we want to extend, like a [template method + pattern](https://en.wikipedia.org/wiki/Template_method_pattern). + For example, given this base: + ``` ruby + class Base + def execute + return unless enabled? + + # ... + # ... + end + end + ``` + Instead of just overriding `Base#execute`, we should update it and extract + the behaviour into another method: + ``` ruby + class Base + def execute + return unless enabled? + + do_something + end + + private + + def do_something + # ... + # ... + end + end + ``` + Then we're free to override that `do_something` without worrying about the + guards: + ``` ruby + module EE::Base + def do_something + # Follow the above pattern to call super and extend it + end + end + ``` + This would require updating CE first, or make sure this is back ported to CE. + +When prepending, place them in the `ee/` specific sub-directory, and +wrap class or module in `module EE` to avoid naming conflicts. + +For example to override the CE implementation of +`ApplicationController#after_sign_out_path_for`: + +```ruby +def after_sign_out_path_for(resource) + current_application_settings.after_sign_out_path.presence || new_user_session_path +end +``` + +Instead of modifying the method in place, you should add `prepend` to +the existing file: + +```ruby +class ApplicationController < ActionController::Base + prepend EE::ApplicationController + # ... + + def after_sign_out_path_for(resource) + current_application_settings.after_sign_out_path.presence || new_user_session_path + end + + # ... +end +``` + +And create a new file in the `ee/` sub-directory with the altered +implementation: + +```ruby +module EE + class ApplicationController + def after_sign_out_path_for(resource) + raise NotImplementedError unless defined?(super) + + if Gitlab::Geo.secondary? + Gitlab::Geo.primary_node.oauth_logout_url(@geo_logout_state) + else + super + end + end + end +end +``` + +#### Use self-descriptive wrapper methods + +When it's not possible/logical to modify the implementation of a +method. Wrap it in a self-descriptive method and use that method. + +For example, in CE only an `admin` is allowed to access all private +projects/groups, but in EE also an `auditor` has full private +access. It would be incorrect to override the implementation of +`User#admin?`, so instead add a method `full_private_access?` to +`app/models/users.rb`. The implementation in CE will be: + +```ruby +def full_private_access? + admin? +end +``` + +In EE, the implementation `ee/app/models/ee/users.rb` would be: + +```ruby +def full_private_access? + raise NotImplementedError unless defined?(super) + super || auditor? +end +``` + +In `lib/gitlab/visibility_level.rb` this method is used to return the +allowed visibilty levels: + +```ruby +def levels_for_user(user = nil) + if user.full_private_access? + [PRIVATE, INTERNAL, PUBLIC] + elsif # ... +end +``` + +See [CE MR][ce-mr-full-private] and [EE MR][ee-mr-full-private] for +full implementation details. + +[ce-mr-full-private]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12373 +[ee-mr-full-private]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2199 + +### Code in `app/controllers/` + +In controllers, the most common type of conflict is with `before_action` that +has a list of actions in CE but EE adds some actions to that list. + +The same problem often occurs for `params.require` / `params.permit` calls. + +**Mitigations** + +Separate CE and EE actions/keywords. For instance for `params.require` in +`ProjectsController`: + +```ruby +def project_params + params.require(:project).permit(project_params_attributes) +end + +# Always returns an array of symbols, created however best fits the use case. +# It _should_ be sorted alphabetically. +def project_params_attributes + %i[ + description + name + path + ] +end + +``` + +In the `EE::ProjectsController` module: + +```ruby +def project_params_attributes + super + project_params_attributes_ee +end + +def project_params_attributes_ee + %i[ + approvals_before_merge + approver_group_ids + approver_ids + ... + ] +end +``` + +### Code in `app/models/` + +EE-specific models should `extend EE::Model`. + +For example, if EE has a specific `Tanuki` model, you would +place it in `ee/app/models/ee/tanuki.rb`. + +### Code in `app/views/` + +It's a very frequent problem that EE is adding some specific view code in a CE +view. For instance the approval code in the project's settings page. + +**Mitigations** + +Blocks of code that are EE-specific should be moved to partials. This +avoids conflicts with big chunks of HAML code that that are not fun to +resolve when you add the indentation to the equation. + +EE-specific views should be placed in `ee/app/views/ee/`, using extra +sub-directories if appropriate. + +### Code in `lib/` + +Place EE-specific logic in the top-level `EE` module namespace. Namespace the +class beneath the `EE` module just as you would normally. + +For example, if CE has LDAP classes in `lib/gitlab/ldap/` then you would place +EE-specific LDAP classes in `ee/lib/ee/gitlab/ldap`. + +### Code in `spec/` + +When you're testing EE-only features, avoid adding examples to the +existing CE specs. Also do no change existing CE examples, since they +should remain working as-is when EE is running without a license. + +Instead place EE specs in the `spec/ee/spec` folder. + +## JavaScript code in `assets/javascripts/` + +To separate EE-specific JS-files we can also move the files into an `ee` folder. + +For example there can be an +`app/assets/javascripts/protected_branches/protected_branches_bundle.js` and an +EE counterpart +`ee/app/assets/javascripts/protected_branches/protected_branches_bundle.js`. + +That way we can create a separate webpack bundle in `webpack.config.js`: + +```javascript + protected_branches: '~/protected_branches', + ee_protected_branches: 'ee/protected_branches/protected_branches_bundle.js', +``` + +With the separate bundle in place, we can decide which bundle to load inside the +view, using the `page_specific_javascript_bundle_tag` helper. + +```haml +- content_for :page_specific_javascripts do + = page_specific_javascript_bundle_tag('protected_branches') +``` + +## SCSS code in `assets/stylesheets` + +To separate EE-specific styles in SCSS files, if a component you're adding styles for +is limited to only EE, it is better to have a separate SCSS file in appropriate directory +within `app/assets/stylesheets`. + +In some cases, this is not entirely possible or creating dedicated SCSS file is an overkill, +e.g. a text style of some component is different for EE. In such cases, +styles are usually kept in stylesheet that is common for both CE and EE, and it is wise +to isolate such ruleset from rest of CE rules (along with adding comment describing the same) +to avoid conflicts during CE to EE merge. + +#### Bad +```scss +.section-body { + .section-title { + background: $gl-header-color; + } + + &.ee-section-body { + .section-title { + background: $gl-header-color-cyan; + } + } +} +``` + +#### Good +```scss +.section-body { + .section-title { + background: $gl-header-color; + } +} + +/* EE-specific styles */ +.section-body.ee-section-body { + .section-title { + background: $gl-header-color-cyan; + } +} +``` diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index d84801f91d4..8f956681693 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -29,34 +29,6 @@ For our currently-supported browsers, see our [requirements][requirements]. ## Development Process -When you are assigned an issue please follow the next steps: - -### Divide a big feature into small Merge Requests -1. Big Merge Request are painful to review. In order to make this process easier we -must break a big feature into smaller ones and create a Merge Request for each step. -1. First step is to create a branch from `master`, let's call it `new-feature`. This branch -will be the recipient of all the smaller Merge Requests. Only this one will be merged to master. -1. Don't do any work on this one, let's keep it synced with master. -1. Create a new branch from `new-feature`, let's call it `new-feature-step-1`. We advise you -to clearly identify which step the branch represents. -1. Do the first part of the modifications in this branch. The target branch of this Merge Request -should be `new-feature`. -1. Once `new-feature-step-1` gets merged into `new-feature` we can continue our work. Create a new -branch from `new-feature`, let's call it `new-feature-step-2` and repeat the process done before. - -```shell -master -└─ new-feature - ├─ new-feature-step-1 - ├─ new-feature-step-2 - └─ new-feature-step-3 -``` - -**Tips** -- Make sure `new-feature` branch is always synced with `master`: merge master frequently. -- Do the same for the feature branch you have opened. This can be accomplished by merging `master` into `new-feature` and `new-feature` into `new-feature-step-*` -- Avoid rewriting history. - ### Share your work early 1. Before writing code guarantee your vision of the architecture is aligned with GitLab's architecture. @@ -82,7 +54,8 @@ or make changes to our frontend development guidelines. --- -## [Testing](testing.md) +## [Testing](../testing_guide/frontend_testing.md) + How we write frontend tests, run the GitLab test suite, and debug test related issues. @@ -98,6 +71,9 @@ Vue specific design patterns and practices. --- +## [Vue Resource](vue_resource.md) +Vue resource specific practices and gotchas. + ## [Icons](icons.md) How we use SVG for our Icons. @@ -130,6 +106,10 @@ Frontend security practices. ## [Accessibility](accessibility.md) Our accessibility standards and resources. +## [Internationalization (i18n) and Translations](../i18n/externalization.md) +Frontend internationalization support is described in [this document](../i18n/). +The [externalization part of the guide](../i18n/externalization.md) explains the helpers/methods available. + [rails]: http://rubyonrails.org/ [haml]: http://haml.info/ diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index c8d23609280..10f4c5a0902 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -88,16 +88,31 @@ followed by any global declarations, then a blank newline prior to any imports o 1. Use ES module syntax to import modules ```javascript // bad - require('foo'); + const SomeClass = require('some_class'); // good - import Foo from 'foo'; + import SomeClass from 'some_class'; // bad - module.exports = Foo; + module.exports = SomeClass; // good - export default Foo; + export default SomeClass; + ``` + + Import statements are following usual naming guidelines, for example object literals use camel case: + + ```javascript + // some_object file + export default { + key: 'value', + }; + + // bad + import ObjectLiteral from 'some_object'; + + // good + import objectLiteral from 'some_object'; ``` 1. Relative paths: when importing a module in the same directory, a child @@ -285,6 +300,13 @@ A forEach will cause side effects, it will be mutating the array being iterated. 1. **Extensions**: Use `.vue` extension for Vue components. 1. **Reference Naming**: Use camelCase for their instances: ```javascript + // bad + import CardBoard from 'cardBoard' + + components: { + CardBoard: + }; + // good import cardBoard from 'cardBoard' @@ -470,7 +492,25 @@ On those a default key should not be provided. ``` #### Ordering -1. Order for a Vue Component: + +1. Tag order in `.vue` file + + ``` + <script> + // ... + </script> + + <template> + // ... + </template> + + // We don't use scoped styles but there are few instances of this + <style> + // ... + </style> + ``` + +1. Properties in a Vue Component: 1. `name` 1. `props` 1. `mixins` @@ -490,6 +530,7 @@ On those a default key should not be provided. 1. `beforeDestroy` 1. `destroyed` + #### Vue and Bootstrap 1. Tooltips: Do not rely on `has-tooltip` class name for Vue components diff --git a/doc/development/fe_guide/testing.md b/doc/development/fe_guide/testing.md index 867c83f1e72..98e499b8c0f 100644 --- a/doc/development/fe_guide/testing.md +++ b/doc/development/fe_guide/testing.md @@ -1,254 +1 @@ -# Frontend Testing - -There are two types of test suites you'll encounter while developing frontend code -at GitLab. We use Karma and Jasmine for JavaScript unit and integration testing, and RSpec -feature tests with Capybara for e2e (end-to-end) integration testing. - -Unit and feature tests need to be written for all new features. -Most of the time, you should use rspec for your feature tests. -There are cases where the behaviour you are testing is not worth the time spent running the full application, -for example, if you are testing styling, animation, edge cases or small actions that don't involve the backend, -you should write an integration test using Jasmine. - - - -_This diagram demonstrates the relative priority of each test type we use_ - -Regression tests should be written for bug fixes to prevent them from recurring in the future. - -See [the Testing Standards and Style Guidelines](../testing.md) -for more information on general testing practices at GitLab. - -## Karma test suite - -GitLab uses the [Karma][karma] test runner with [Jasmine][jasmine] as its test -framework for our JavaScript unit and integration tests. For integration tests, -we generate HTML files using RSpec (see `spec/javascripts/fixtures/*.rb` for examples). -Some fixtures are still HAML templates that are translated to HTML files using the same mechanism (see `static_fixtures.rb`). -Adding these static fixtures should be avoided as they are harder to keep up to date with real views. -The existing static fixtures will be migrated over time. -Please see [gitlab-org/gitlab-ce#24753](https://gitlab.com/gitlab-org/gitlab-ce/issues/24753) to track our progress. -Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin. - -JavaScript tests live in `spec/javascripts/`, matching the folder structure -of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js` -has a corresponding `spec/javascripts/behaviors/autosize_spec.js` file. - -Keep in mind that in a CI environment, these tests are run in a headless -browser and you will not have access to certain APIs, such as -[`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification), -which will have to be stubbed. - -### Best practice - -#### Naming unit tests - -When writing describe test blocks to test specific functions/methods, -please use the method name as the describe block name. - -```javascript -// Good -describe('methodName', () => { - it('passes', () => { - expect(true).toEqual(true); - }); -}); - -// Bad -describe('#methodName', () => { - it('passes', () => { - expect(true).toEqual(true); - }); -}); - -// Bad -describe('.methodName', () => { - it('passes', () => { - expect(true).toEqual(true); - }); -}); -``` -#### Testing Promises - -When testing Promises you should always make sure that the test is asynchronous and rejections are handled. -Your Promise chain should therefore end with a call of the `done` callback and `done.fail` in case an error occurred. - -```javascript -// Good -it('tests a promise', (done) => { - promise - .then((data) => { - expect(data).toBe(asExpected); - }) - .then(done) - .catch(done.fail); -}); - -// Good -it('tests a promise rejection', (done) => { - promise - .then(done.fail) - .catch((error) => { - expect(error).toBe(expectedError); - }) - .then(done) - .catch(done.fail); -}); - -// Bad (missing done callback) -it('tests a promise', () => { - promise - .then((data) => { - expect(data).toBe(asExpected); - }) -}); - -// Bad (missing catch) -it('tests a promise', (done) => { - promise - .then((data) => { - expect(data).toBe(asExpected); - }) - .then(done) -}); - -// Bad (use done.fail in asynchronous tests) -it('tests a promise', (done) => { - promise - .then((data) => { - expect(data).toBe(asExpected); - }) - .then(done) - .catch(fail) -}); - -// Bad (missing catch) -it('tests a promise rejection', (done) => { - promise - .catch((error) => { - expect(error).toBe(expectedError); - }) - .then(done) -}); -``` - -#### Stubbing - -For unit tests, you should stub methods that are unrelated to the current unit you are testing. -If you need to use a prototype method, instantiate an instance of the class and call it there instead of mocking the instance completely. - -For integration tests, you should stub methods that will effect the stability of the test if they -execute their original behaviour. i.e. Network requests. - -### Vue.js unit tests -See this [section][vue-test]. - -### Running frontend tests - -`rake karma` runs the frontend-only (JavaScript) tests. -It consists of two subtasks: - -- `rake karma:fixtures` (re-)generates fixtures -- `rake karma:tests` actually executes the tests - -As long as the fixtures don't change, `rake karma:tests` (or `yarn karma`) -is sufficient (and saves you some time). - -### Live testing and focused testing - -While developing locally, it may be helpful to keep karma running so that you -can get instant feedback on as you write tests and modify code. To do this -you can start karma with `npm run karma-start`. It will compile the javascript -assets and run a server at `http://localhost:9876/` where it will automatically -run the tests on any browser which connects to it. You can enter that url on -multiple browsers at once to have it run the tests on each in parallel. - -While karma is running, any changes you make will instantly trigger a recompile -and retest of the entire test suite, so you can see instantly if you've broken -a test with your changes. You can use [jasmine focused][jasmine-focus] or -excluded tests (with `fdescribe` or `xdescribe`) to get karma to run only the -tests you want while you're working on a specific feature, but make sure to -remove these directives when you commit your code. - -## RSpec Feature Integration Tests - -Information on setting up and running RSpec integration tests with -[Capybara][capybara] can be found in the -[general testing guide](../testing.md). - -## Gotchas - -### Errors due to use of unsupported JavaScript features - -Similar errors will be thrown if you're using JavaScript features not yet -supported by the PhantomJS test runner which is used for both Karma and RSpec -tests. We polyfill some JavaScript objects for older browsers, but some -features are still unavailable: - -- Array.from -- Array.first -- Async functions -- Generators -- Array destructuring -- For..Of -- Symbol/Symbol.iterator -- Spread - -Until these are polyfilled appropriately, they should not be used. Please -update this list with additional unsupported features. - -### RSpec errors due to JavaScript - -By default RSpec unit tests will not run JavaScript in the headless browser -and will simply rely on inspecting the HTML generated by rails. - -If an integration test depends on JavaScript to run correctly, you need to make -sure the spec is configured to enable JavaScript when the tests are run. If you -don't do this you'll see vague error messages from the spec runner. - -To enable a JavaScript driver in an `rspec` test, add `:js` to the -individual spec or the context block containing multiple specs that need -JavaScript enabled: - -```ruby -# For one spec -it 'presents information about abuse report', :js do - # assertions... -end - -describe "Admin::AbuseReports", :js do - it 'presents information about abuse report' do - # assertions... - end - it 'shows buttons for adding to abuse report' do - # assertions... - end -end -``` - -### Spinach errors due to missing JavaScript - -> **Note:** Since we are discouraging the use of Spinach when writing new -> feature tests, you shouldn't ever need to use this. This information is kept -> available for legacy purposes only. - -In Spinach, the JavaScript driver is enabled differently. In the `*.feature` -file for the failing spec, add the `@javascript` flag above the Scenario: - -``` -@javascript -Scenario: Developer can approve merge request - Given I am a "Shop" developer - And I visit project "Shop" merge requests page - And merge request 'Bug NS-04' must be approved - And I click link "Bug NS-04" - When I click link "Approve" - Then I should see approved merge request "Bug NS-04" -``` - -[capybara]: http://teamcapybara.github.io/capybara/ -[jasmine]: https://jasmine.github.io/ -[jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html -[jasmine-jquery]: https://github.com/velesin/jasmine-jquery -[karma]: http://karma-runner.github.io/ -[vue-test]:https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components +This document was moved to [../testing_guide/frontend_testing.md](../testing_guide/frontend_testing.md). diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 2607353782a..f88f0753687 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -179,6 +179,7 @@ itself, please read this guide: [State Management][state-management] The Service is a class used only to communicate with the server. It does not store or manipulate any data. It is not aware of the store or the components. We use [vue-resource][vue-resource-repo] to communicate with the server. +Refer to [vue resource](vue_resource.md) for more details. Vue Resource should only be imported in the service file. @@ -189,55 +190,6 @@ Vue Resource should only be imported in the service file. Vue.use(VueResource); ``` -#### Vue-resource gotchas -#### Headers -Headers are being parsed into a plain object in an interceptor. -In Vue-resource 1.x `headers` object was changed into an `Headers` object. In order to not change all old code, an interceptor was added. - -If you need to write a unit test that takes the headers in consideration, you need to include an interceptor to parse the headers after your test interceptor. -You can see an example in `spec/javascripts/environments/environment_spec.js`: - ```javascript - import { headersInterceptor } from './helpers/vue_resource_helper'; - - beforeEach(() => { - Vue.http.interceptors.push(myInterceptor); - Vue.http.interceptors.push(headersInterceptor); - }); - - afterEach(() => { - Vue.http.interceptors = _.without(Vue.http.interceptors, myInterceptor); - Vue.http.interceptors = _.without(Vue.http.interceptors, headersInterceptor); - }); - ``` - -#### `.json()` -When making a request to the server, you will most likely need to access the body of the response. -Use `.json()` to convert. Because `.json()` returns a Promise the follwoing structure should be used: - - ```javascript - service.get('url') - .then(resp => resp.json()) - .then((data) => { - this.store.storeData(data); - }) - .catch(() => new Flash('Something went wrong')); - ``` - -When using `Poll` (`app/assets/javascripts/lib/utils/poll.js`), the `successCallback` needs to handle `.json()` as a Promise: - ```javascript - successCallback: (response) => { - return response.json().then((data) => { - // handle the response - }); - } - ``` - -#### CSRF token -We use a Vue Resource interceptor to manage the CSRF token. -`app/assets/javascripts/vue_shared/vue_resource_interceptor.js` holds all our common interceptors. -Note: You don't need to load `app/assets/javascripts/vue_shared/vue_resource_interceptor.js` -since it's already being loaded by `common_vue.js`. - ### End Result The following example shows an application: @@ -428,7 +380,7 @@ is a good example of this pattern. ## Style guide -Please refer to the Vue section of our [style guide](style_guide_js.md#vuejs) +Please refer to the Vue section of our [style guide](style_guide_js.md#vue-js) for best practices while writing your Vue components and templates. ## Testing Vue Components @@ -769,7 +721,6 @@ describe('component', () => { [component-system]: https://vuejs.org/v2/guide/#Composing-with-Components [state-management]: https://vuejs.org/v2/guide/state-management.html#Simple-State-Management-from-Scratch [one-way-data-flow]: https://vuejs.org/v2/guide/components.html#One-Way-Data-Flow -[vue-resource-repo]: https://github.com/pagekit/vue-resource [vue-resource-interceptor]: https://github.com/pagekit/vue-resource/blob/develop/docs/http.md#interceptors [vue-test]: https://vuejs.org/v2/guide/unit-testing.html [issue-boards-service]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/boards/services/board_service.js.es6 diff --git a/doc/development/fe_guide/vue_resource.md b/doc/development/fe_guide/vue_resource.md new file mode 100644 index 00000000000..c376c5c32bf --- /dev/null +++ b/doc/development/fe_guide/vue_resource.md @@ -0,0 +1,72 @@ +# Vue Resouce +In Vue applications we use [vue-resource][vue-resource-repo] to communicate with the server. + +## HTTP Status Codes + +### `.json()` +When making a request to the server, you will most likely need to access the body of the response. +Use `.json()` to convert. Because `.json()` returns a Promise the follwoing structure should be used: + + ```javascript + service.get('url') + .then(resp => resp.json()) + .then((data) => { + this.store.storeData(data); + }) + .catch(() => new Flash('Something went wrong')); + ``` + + +When using `Poll` (`app/assets/javascripts/lib/utils/poll.js`), the `successCallback` needs to handle `.json()` as a Promise: + ```javascript + successCallback: (response) => { + return response.json().then((data) => { + // handle the response + }); + } + ``` + +### 204 +Some endpoints - usually `delete` endpoints - return `204` as the success response. +When handling `204 - No Content` responses, we cannot use `.json()` since it tries to parse the non-existant body content. + +When handling `204` responses, do not use `.json`, otherwise the promise will throw an error and will enter the `catch` statement: + +```javascript + Vue.http.delete('path') + .then(() => { + // success! + }) + .catch(() => { + // handle error + }) +``` + +## Headers +Headers are being parsed into a plain object in an interceptor. +In Vue-resource 1.x `headers` object was changed into an `Headers` object. In order to not change all old code, an interceptor was added. + +If you need to write a unit test that takes the headers in consideration, you need to include an interceptor to parse the headers after your test interceptor. +You can see an example in `spec/javascripts/environments/environment_spec.js`: + ```javascript + import { headersInterceptor } from './helpers/vue_resource_helper'; + + beforeEach(() => { + Vue.http.interceptors.push(myInterceptor); + Vue.http.interceptors.push(headersInterceptor); + }); + + afterEach(() => { + Vue.http.interceptors = _.without(Vue.http.interceptors, myInterceptor); + Vue.http.interceptors = _.without(Vue.http.interceptors, headersInterceptor); + }); + ``` + +## CSRF token +We use a Vue Resource interceptor to manage the CSRF token. +`app/assets/javascripts/vue_shared/vue_resource_interceptor.js` holds all our common interceptors. +Note: You don't need to load `app/assets/javascripts/vue_shared/vue_resource_interceptor.js` +since it's already being loaded by `common_vue.js`. + + +[vue-resource-repo]: https://github.com/pagekit/vue-resource diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md new file mode 100644 index 00000000000..ca2048c7019 --- /dev/null +++ b/doc/development/gitaly.md @@ -0,0 +1,102 @@ +# GitLab Developers Guide to Working with Gitaly + +[Gitaly](https://gitlab.com/gitlab-org/gitaly) is a high-level Git RPC service used by GitLab CE/EE, +Workhorse and GitLab-Shell. All Rugged operations in GitLab CE/EE are currently being phased out to +be replaced by Gitaly API calls. + +Visit the [Gitaly Migration Board](https://gitlab.com/gitlab-org/gitaly/boards/331341) for current +status of the migration. + +## Feature Flags + +Gitaly makes heavy use of [feature flags](feature_flags.md). + +Each Rugged-to-Gitaly migration goes through a [series of phases](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/MIGRATION_PROCESS.md): + +* **Opt-In**: by default the Rugged implementation is used. + * Production instances can choose to enable the Gitaly endpoint by enabling the feature flag. + * For testing purposes, you may wish to enable all feature flags by default. This can be done by exporting the following + environment variable: `GITALY_FEATURE_DEFAULT_ON=1`. + * On developer instances (ie, when `Rails.env.development?` is true), the Gitaly endpoint + is enabled by default, but can be _disabled_ using feature flags. +* **Opt-Out**: by default, the Gitaly endpoint is used, but the feature can be explicitly disabled using the feature flag. +* **Mandatory**: The migration is complete and cannot be disabled. The old codepath is removed. + +### Enabling and Disabling Feature + +In the Rails console, type: + +```ruby +Feature.enable(:gitaly_feature_name) +Feature.disable(:gitaly_feature_name) +``` + +Where `gitaly_feature_name` is the name of the Gitaly feature. This can be determined by finding the appropriate +`gitaly_migrate` code block, for example: + +```ruby +gitaly_migrate(:tag_names) do +... +end +``` + +Since Gitaly features are always prefixed with `gitaly_`, the name of the feature flag in this case would be `gitaly_tag_names`. + +## Gitaly-Related Test Failures + +If your test-suite is failing with Gitaly issues, as a first step, try running: + +```shell +rm -rf tmp/tests/gitaly +``` + +## `TooManyInvocationsError` errors + +During development and testing, you may experience `Gitlab::GitalyClient::TooManyInvocationsError` failures. +The `GitalyClient` will attempt to block against potential n+1 issues by raising this error +when Gitaly is called more than 30 times in a single Rails request or Sidekiq execution. + +As a temporary measure, export `GITALY_DISABLE_REQUEST_LIMITS=1` to suppress the error. This will disable the n+1 detection +in your development environment. + +Please raise an issue in the GitLab CE or EE repositories to report the issue. Include the labels ~Gitaly +~performance ~"technical debt". Please ensure that the issue contains the full stack trace and error message of the +`TooManyInvocationsError`. Also include any known failing tests if possible. + +Isolate the source of the n+1 problem. This will normally be a loop that results in Gitaly being called for each +element in an array. If you are unable to isolate the problem, please contact a member +of the [Gitaly Team](https://gitlab.com/groups/gl-gitaly/group_members) for assistance. + +Once the source has been found, wrap it in an `allow_n_plus_1_calls` block, as follows: + +```ruby +# n+1: link to n+1 issue +Gitlab::GitalyClient.allow_n_plus_1_calls do + # original code + commits.each { |commit| ... } +end +``` + +Once the code is wrapped in this block, this code-path will be excluded from n+1 detection. + +## Request counts + +Commits and other git data, is now fetched through Gitaly. These fetches can, +much like with a database, be batched. This improves performance for the client +and for Gitaly itself and therefore for the users too. To keep performance stable +and guard performance regressions, Gitaly calls can be counted and the call count +can be tested against. This requires the `:request_store` flag to be set. + +```ruby +describe 'Gitaly Request count tests' do + context 'when the request store is activated', :request_store do + it 'correctly counts the gitaly requests made' do + expect { subject }.to change { Gitlab::GitalyClient.get_request_count }.by(10) + end + end +end +``` + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md new file mode 100644 index 00000000000..7c38260406d --- /dev/null +++ b/doc/development/i18n/externalization.md @@ -0,0 +1,316 @@ +# Internationalization for GitLab + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10669) in GitLab 9.2. + +For working with internationalization (i18n), +[GNU gettext](https://www.gnu.org/software/gettext/) is used given it's the most +used tool for this task and there are a lot of applications that will help us to +work with it. + +## Setting up GitLab Development Kit (GDK) + +In order to be able to work on the [GitLab Community Edition](https://gitlab.com/gitlab-org/gitlab-ce) +project you must download and configure it through [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/set-up-gdk.md). + +Once you have the GitLab project ready, you can start working on the translation. + +## Tools + +The following tools are used: + +1. [`gettext_i18n_rails`](https://github.com/grosser/gettext_i18n_rails): this + gem allow us to translate content from models, views and controllers. Also + it gives us access to the following raketasks: + - `rake gettext:find`: Parses almost all the files from the + Rails application looking for content that has been marked for + translation. Finally, it updates the PO files with the new content that + it has found. + - `rake gettext:pack`: Processes the PO files and generates the + MO files that are binary and are finally used by the application. + +1. [`gettext_i18n_rails_js`](https://github.com/webhippie/gettext_i18n_rails_js): + this gem is useful to make the translations available in JavaScript. It + provides the following raketask: + - `rake gettext:po_to_json`: Reads the contents from the PO files and + generates JSON files containing all the available translations. + +1. PO editor: there are multiple applications that can help us to work with PO + files, a good option is [Poedit](https://poedit.net/download) which is + available for macOS, GNU/Linux and Windows. + +## Preparing a page for translation + +We basically have 4 types of files: + +1. Ruby files: basically Models and Controllers. +1. HAML files: these are the view files. +1. ERB files: used for email templates. +1. JavaScript files: we mostly need to work with VUE JS templates. + +### Ruby files + +If there is a method or variable that works with a raw string, for instance: + +```ruby +def hello + "Hello world!" +end +``` + +Or: + +```ruby +hello = "Hello world!" +``` + +You can easily mark that content for translation with: + +```ruby +def hello + _("Hello world!") +end +``` + +Or: + +```ruby +hello = _("Hello world!") +``` + +### HAML files + +Given the following content in HAML: + +```haml +%h1 Hello world! +``` + +You can mark that content for translation with: + +```haml +%h1= _("Hello world!") +``` + +### ERB files + +Given the following content in ERB: + +```erb +<h1>Hello world!</h1> +``` + +You can mark that content for translation with: + +```erb +<h1><%= _("Hello world!") %></h1> +``` + +### JavaScript files + +In JavaScript we added the `__()` (double underscore parenthesis) function +for translations. + +### Updating the PO files with the new content + +Now that the new content is marked for translation, we need to update the PO +files with the following command: + +```sh +bundle exec rake gettext:find +``` + +This command will update the `locale/**/gitlab.edit.po` file with the +new content that the parser has found. + +New translations will be added with their default content and will be marked +fuzzy. To use the translation, look for the `#, fuzzy` mention in `gitlab.edit.po` +and remove it. + +We need to make sure we remove the `fuzzy` translations before generating the +`locale/**/gitlab.po` file. When they aren't removed, the resulting `.po` will +be treated as a binary file which could overwrite translations that were merged +before the new translations. + +When we are just preparing a page to be translated, but not actually adding any +translations. There's no need to generate `.po` files. + +Translations that aren't used in the source code anymore will be marked with +`~#`; these can be removed to keep our translation files clutter-free. + +### Validating PO files + +To make sure we keep our translation files up to date, there's a linter that is +running on CI as part of the `static-analysis` job. + +To lint the adjustments in PO files locally you can run `rake gettext:lint`. + +The linter will take the following into account: + +- Valid PO-file syntax +- Variable usage + - Only one unnamed (`%d`) variable, since the order of variables might change + in different languages + - All variables used in the message-id are used in the translation + - There should be no variables used in a translation that aren't in the + message-id +- Errors during translation. + +The errors are grouped per file, and per message ID: + +``` +Errors in `locale/zh_HK/gitlab.po`: + PO-syntax errors + SimplePoParser::ParserErrorSyntax error in lines + Syntax error in msgctxt + Syntax error in msgid + Syntax error in msgstr + Syntax error in message_line + There should be only whitespace until the end of line after the double quote character of a message text. + Parseing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}' + SimplePoParser filtered backtrace: SimplePoParser::ParserError +Errors in `locale/zh_TW/gitlab.po`: + 1 pipeline + <%d 條流水線> is using unknown variables: [%d] + Failure translating to zh_TW with []: too few arguments +``` + +In this output the `locale/zh_HK/gitlab.po` has syntax errors. +The `locale/zh_TW/gitlab.po` has variables that are used in the translation that +aren't in the message with id `1 pipeline`. + +## Working with special content + + +### Just marking content for parsing + +- In Ruby/HAML: + + ```ruby + _('Subscribe') + ``` + +- In JavaScript: + + ```js + import { __ } from '../../../locale'; + const label = __('Subscribe'); + ``` + + +Sometimes there are some dynamic translations that can't be found by the +parser when running `bundle exec rake gettext:find`. For these scenarios you can +use the [`_N` method](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#unfound-translations-with-rake-gettextfind). + +There is also and alternative method to [translate messages from validation errors](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#option-a). + +### Interpolation + +- In Ruby/HAML: + + ```ruby + _("Hello %{name}") % { name: 'Joe' } => 'Hello Joe' + ``` + +- In JavaScript: + + ```js + import { __, sprintf } from '../../../locale'; + sprintf(__('Hello %{username}'), { username: 'Joe' }) => 'Hello Joe' + ``` + +### Plurals + +- In Ruby/HAML: + + ```ruby + n_('Apple', 'Apples', 3) => 'Apples' + ``` + + Using interpolation: + ```ruby + n_("There is a mouse.", "There are %d mice.", size) % size + ``` + +- In JavaScript: + + ```js + n__('Apple', 'Apples', 3) => 'Apples' + ``` + + Using interpolation: + + ```js + n__('Last day', 'Last %d days', 30) => 'Last 30 days' + ``` + +### Namespaces + +Sometimes you need to add some context to the text that you want to translate +(if the word occurs in a sentence and/or the word is ambiguous). + +- In Ruby/HAML: + + ```ruby + s_('OpenedNDaysAgo|Opened') + ``` + + In case the translation is not found it will return `Opened`. + +- In JavaScript: + + ```js + s__('OpenedNDaysAgo|Opened') + ``` + +## Adding a new language + +Let's suppose you want to add translations for a new language, let's say French. + +1. The first step is to register the new language in `lib/gitlab/i18n.rb`: + + ```ruby + ... + AVAILABLE_LANGUAGES = { + ..., + 'fr' => 'Français' + }.freeze + ... + ``` + +1. Next, you need to add the language: + + ```sh + bundle exec rake gettext:add_language[fr] + ``` + + If you want to add a new language for a specific region, the command is similar, + you just need to separate the region with an underscore (`_`). For example: + + ```sh + bundle exec rake gettext:add_language[en_GB] + ``` + + Please note that you need to specify the region part in capitals. + +1. Now that the language is added, a new directory has been created under the + path: `locale/fr/`. You can now start using your PO editor to edit the PO file + located in: `locale/fr/gitlab.edit.po`. + +1. After you're done updating the translations, you need to process the PO files + in order to generate the binary MO files and finally update the JSON files + containing the translations: + + ```sh + bundle exec rake gettext:compile + ``` + +1. In order to see the translated content we need to change our preferred language + which can be found under the user's **Settings** (`/profile`). + +1. After checking that the changes are ok, you can proceed to commit the new files. + For example: + + ```sh + git add locale/fr/ app/assets/javascripts/locale/fr/ + git commit -m "Add French translations for Cycle Analytics page" + ``` diff --git a/doc/development/i18n/img/crowdin-editor.png b/doc/development/i18n/img/crowdin-editor.png Binary files differnew file mode 100644 index 00000000000..5c31d8f0cec --- /dev/null +++ b/doc/development/i18n/img/crowdin-editor.png diff --git a/doc/development/i18n/index.md b/doc/development/i18n/index.md new file mode 100644 index 00000000000..4cb2624c098 --- /dev/null +++ b/doc/development/i18n/index.md @@ -0,0 +1,76 @@ +# Translate GitLab to your language + +The text in GitLab's user interface is in American English by default. +Each string can be translated to other languages. +As each string is translated, it is added to the languages translation file, +and will be available in future releases of GitLab. + +Contributions to translations are always needed. +Many strings are not yet available for translation because they have not been externalized. +Helping externalize strings benefits all languages. +Some translations are incomplete or inconsistent. +Translating strings will help complete and improve each language. + +## How to contribute + +There are many ways you can contribute in translating GitLab. + +### Externalize strings + +Before a string can be translated, it must be externalized. +This is the process where English strings in the GitLab source code are wrapped in a function that +retrieves the translated string for the user's language. + +As new features are added and existing features are updated, the surrounding strings are being +externalized, however, there are many parts of GitLab that still need more work to externalize all +strings. + +See [Externalization for GitLab](externalization.md). + +### Translate strings + +The translation process is managed at [translate.gitlab.com](https://translate.gitlab.com) +using [Crowdin](https://crowdin.com/). +You will need to create an account before you can submit translations. +Once you are signed in, select the language you wish to contribute translations to. + +Voting for translations is also valuable, helping to confirm good and flag inaccurate translations. + +See [Translation guidelines](translation.md). + +### Proof reading + +Proof reading helps ensure the accuracy and consistency of translations. +All translations are proof read before being accepted. +If a translations requires changes, you will be notified with a comment explaining why. + +Community assistance proof reading translations is encouraged and appreciated. +Requests to become a proof reader will be considered on the merits of previous translations. + +- Bulgarian +- Chinese Simplified + - [Huang Tao](https://crowdin.com/profile/htve) +- Chinese Traditional + - [Huang Tao](https://crowdin.com/profile/htve) +- Chinese Traditional, Hong Kong + - [Huang Tao](https://crowdin.com/profile/htve) +- Dutch +- Esperanto +- French +- German +- Italian +- Japanese +- Korean + - [Huang Tao](https://crowdin.com/profile/htve) +- Portuguese, Brazilian +- Russian + - [Alexy Lustin](https://crowdin.com/profile/lustin) + - [Nikita Grylov](https://crowdin.com/profile/nixel2007) +- Spanish +- Ukrainian + +If you would like to be added as a proof reader, please [open an issue](https://gitlab.com/gitlab-org/gitlab-ce/issues). + +## Release + +Translations are typically included in the next major or minor release. diff --git a/doc/development/i18n/translation.md b/doc/development/i18n/translation.md new file mode 100644 index 00000000000..b34ec754742 --- /dev/null +++ b/doc/development/i18n/translation.md @@ -0,0 +1,76 @@ +# Translating GitLab + +For managing the translation process we use [Crowdin](https://crowdin.com). + +## Using Crowdin + +The first step is to get familiar with Crowdin. + +### Sign In + +To contribute translations at [translate.gitlab.com](https://translate.gitlab.com) +you must create a Crowdin account. +You may create a new account or use any of their supported sign in services. + +### Language Selections + +GitLab is being translated into many languages. + +1. Select the language you would like to contribute translations to by clicking the flag +1. You will see a list of files and folders. + Click `gitlab.pot` to open the translation editor. + +### Translation Editor + +The online translation editor is the easiest way to contribute translations. + + + +1. Strings for translation are listed in the left panel +1. Translations are entered into the central panel. + Multiple translations will be required for strings that contains plurals. + The string to be translated is shown above with glossary terms highlighted. + If the string to be translated is not clear, you can 'Request Context' + +A glossary of common terms is available in the right panel by clicking Terms. +Comments can be added to discuss a translation with the community. + +Remember to **Save** each translation. + +## Translation Guidelines + +Be sure to check the following guidelines before you translate any strings. + +### Technical terms + +Technical terms should be treated like proper nouns and not be translated. +This helps maintain a logical connection and consistency between tools (e.g. `git` client) and +GitLab. + +Technical terms that should always be in English are noted in the glossary when using +[translate.gitlab.com](https://translate.gitlab.com). + +### Formality + +The level of formality used in software varies by language. +For example, in French we translate `you` as the informal `tu`. + +You can refer to other translated strings and notes in the glossary to assist determining a +suitable level of formality. + +### Inclusive language + +[Diversity] is one of GitLab's values. +We ask you to avoid translations which exclude people based on their gender or ethnicity. +In languages which distinguish between a male and female form, +use both or choose a neutral formulation. + +For example in German, the word "user" can be translated into "Benutzer" (male) or "Benutzerin" (female). +Therefore "create a new user" would translate into "Benutzer(in) anlegen". + +[Diversity]: https://about.gitlab.com/handbook/values/#diversity + +### Updating the glossary + +To propose additions to the glossary please +[open an issue](https://gitlab.com/gitlab-org/gitlab-ce/issues). diff --git a/doc/development/i18n_guide.md b/doc/development/i18n_guide.md index bd0ef39ca62..f6e949b5fd8 100644 --- a/doc/development/i18n_guide.md +++ b/doc/development/i18n_guide.md @@ -1,297 +1 @@ -# Internationalization for GitLab - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10669) in GitLab 9.2. - -For working with internationalization (i18n) we use -[GNU gettext](https://www.gnu.org/software/gettext/) given it's the most used -tool for this task and we have a lot of applications that will help us to work -with it. - -## Setting up GitLab Development Kit (GDK) - -In order to be able to work on the [GitLab Community Edition](https://gitlab.com/gitlab-org/gitlab-ce) project we must download and -configure it through [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit), we can do it by following this [guide](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/set-up-gdk.md). - -Once we have the GitLab project ready we can start working on the -translation of the project. - -## Tools - -We use a couple of gems: - -1. [`gettext_i18n_rails`](https://github.com/grosser/gettext_i18n_rails): this - gem allow us to translate content from models, views and controllers. Also - it gives us access to the following raketasks: - - `rake gettext:find`: Parses almost all the files from the - Rails application looking for content that has been marked for - translation. Finally, it updates the PO files with the new content that - it has found. - - `rake gettext:pack`: Processes the PO files and generates the - MO files that are binary and are finally used by the application. - -1. [`gettext_i18n_rails_js`](https://github.com/webhippie/gettext_i18n_rails_js): - this gem is useful to make the translations available in JavaScript. It - provides the following raketask: - - `rake gettext:po_to_json`: Reads the contents from the PO files and - generates JSON files containing all the available translations. - -1. PO editor: there are multiple applications that can help us to work with PO - files, a good option is [Poedit](https://poedit.net/download) which is - available for macOS, GNU/Linux and Windows. - -## Preparing a page for translation - -We basically have 4 types of files: - -1. Ruby files: basically Models and Controllers. -1. HAML files: these are the view files. -1. ERB files: used for email templates. -1. JavaScript files: we mostly need to work with VUE JS templates. - -### Ruby files - -If there is a method or variable that works with a raw string, for instance: - -```ruby -def hello - "Hello world!" -end -``` - -Or: - -```ruby -hello = "Hello world!" -``` - -You can easily mark that content for translation with: - -```ruby -def hello - _("Hello world!") -end -``` - -Or: - -```ruby -hello = _("Hello world!") -``` - -### HAML files - -Given the following content in HAML: - -```haml -%h1 Hello world! -``` - -You can mark that content for translation with: - -```haml -%h1= _("Hello world!") -``` - -### ERB files - -Given the following content in ERB: - -```erb -<h1>Hello world!</h1> -``` - -You can mark that content for translation with: - -```erb -<h1><%= _("Hello world!") %></h1> -``` - -### JavaScript files - -In JavaScript we added the `__()` (double underscore parenthesis) function -for translations. - -### Updating the PO files with the new content - -Now that the new content is marked for translation, we need to update the PO -files with the following command: - -```sh -bundle exec rake gettext:find -``` - -This command will update the `locale/**/gitlab.edit.po` file with the -new content that the parser has found. - -New translations will be added with their default content and will be marked -fuzzy. To use the translation, look for the `#, fuzzy` mention in `gitlab.edit.po` -and remove it. - -We need to make sure we remove the `fuzzy` translations before generating the -`locale/**/gitlab.po` file. When they aren't removed, the resulting `.po` will -be treated as a binary file which could overwrite translations that were merged -before the new translations. - -When we are just preparing a page to be translated, but not actually adding any -translations. There's no need to generate `.po` files. - -Translations that aren't used in the source code anymore will be marked with -`~#`; these can be removed to keep our translation files clutter-free. - -### Validating PO files - -To make sure we keep our translation files up to date, there's a linter that is -running on CI as part of the `static-analysis` job. - -To lint the adjustments in PO files locally you can run `rake gettext:lint`. - -The linter will take the following into account: - -- Valid PO-file syntax -- Variable usage - - Only one unnamed (`%d`) variable, since the order of variables might change - in different languages - - All variables used in the message-id are used in the translation - - There should be no variables used in a translation that aren't in the - message-id -- Errors during translation. - -The errors are grouped per file, and per message ID: - -``` -Errors in `locale/zh_HK/gitlab.po`: - PO-syntax errors - SimplePoParser::ParserErrorSyntax error in lines - Syntax error in msgctxt - Syntax error in msgid - Syntax error in msgstr - Syntax error in message_line - There should be only whitespace until the end of line after the double quote character of a message text. - Parseing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}' - SimplePoParser filtered backtrace: SimplePoParser::ParserError -Errors in `locale/zh_TW/gitlab.po`: - 1 pipeline - <%d 條流水線> is using unknown variables: [%d] - Failure translating to zh_TW with []: too few arguments -``` - -In this output the `locale/zh_HK/gitlab.po` has syntax errors. -The `locale/zh_TW/gitlab.po` has variables that are used in the translation that -aren't in the message with id `1 pipeline`. - -## Working with special content - -### Interpolation - -- In Ruby/HAML: - - ```ruby - _("Hello %{name}") % { name: 'Joe' } - ``` - -- In JavaScript: Not supported at this moment. - -### Plurals - -- In Ruby/HAML: - - ```ruby - n_('Apple', 'Apples', 3) => 'Apples' - ``` - - Using interpolation: - ```ruby - n_("There is a mouse.", "There are %d mice.", size) % size - ``` - -- In JavaScript: - - ```js - n__('Apple', 'Apples', 3) => 'Apples' - ``` - - Using interpolation: - - ```js - n__('Last day', 'Last %d days', 30) => 'Last 30 days' - ``` - -### Namespaces - -Sometimes you need to add some context to the text that you want to translate -(if the word occurs in a sentence and/or the word is ambiguous). - -- In Ruby/HAML: - - ```ruby - s_('OpenedNDaysAgo|Opened') - ``` - - In case the translation is not found it will return `Opened`. - -- In JavaScript: - - ```js - s__('OpenedNDaysAgo|Opened') - ``` - -### Just marking content for parsing - -Sometimes there are some dynamic translations that can't be found by the -parser when running `bundle exec rake gettext:find`. For these scenarios you can -use the [`_N` method](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#unfound-translations-with-rake-gettextfind). - -There is also and alternative method to [translate messages from validation errors](https://github.com/grosser/gettext_i18n_rails/blob/c09e38d481e0899ca7d3fc01786834fa8e7aab97/Readme.md#option-a). - -## Adding a new language - -Let's suppose you want to add translations for a new language, let's say French. - -1. The first step is to register the new language in `lib/gitlab/i18n.rb`: - - ```ruby - ... - AVAILABLE_LANGUAGES = { - ..., - 'fr' => 'Français' - }.freeze - ... - ``` - -1. Next, you need to add the language: - - ```sh - bundle exec rake gettext:add_language[fr] - ``` - - If you want to add a new language for a specific region, the command is similar, - you just need to separate the region with an underscore (`_`). For example: - - ```sh - bundle exec rake gettext:add_language[en_GB] - ``` - - Please note that you need to specify the region part in capitals. - -1. Now that the language is added, a new directory has been created under the - path: `locale/fr/`. You can now start using your PO editor to edit the PO file - located in: `locale/fr/gitlab.edit.po`. - -1. After you're done updating the translations, you need to process the PO files - in order to generate the binary MO files and finally update the JSON files - containing the translations: - - ```sh - bundle exec rake gettext:compile - ``` - -1. In order to see the translated content we need to change our preferred language - which can be found under the user's **Settings** (`/profile`). - -1. After checking that the changes are ok, you can proceed to commit the new files. - For example: - - ```sh - git add locale/fr/ app/assets/javascripts/locale/fr/ - git commit -m "Add French translations for Cycle Analytics page" - ``` +This document was moved to [a new location](i18n/index.md). diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 9a5811d8474..902b1c74a42 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -56,6 +56,7 @@ Libraries with the following licenses are acceptable for use: - [ISC License][ISC] (also known as the OpenBSD License): A permissive (non-copyleft) license as defined by the Open Source Initiative. - [Creative Commons Zero (CC0)][CC0]: A public domain dedication, recommended as a way to disclaim copyright on your work to the maximum extent possible. - [Unlicense][UNLICENSE]: Another public domain dedication. +- [OWFa 1.0][OWFa1]: An open-source license and patent grant designed for specifications. ## Unacceptable Licenses @@ -65,6 +66,7 @@ Libraries with the following licenses are unacceptable for use: - [GNU AGPLv3][AGPLv3]: AGPL-licensed libraries cannot be linked to from non-GPL projects. - [Open Software License (OSL)][OSL]: is a copyleft license. In addition, the FSF [recommend against its use][OSL-GNU]. - [Facebook BSD + PATENTS][Facebook]: is a 3-clause BSD license with a patent grant that has been deemed [Category X][x-list] by the Apache foundation. +- [WTFPL][WTFPL]: is a public domain dedication [rejected by the OSI (3.2)][WTFPL-OSI]. Also has a strong language which is not in accordance with our diversity policy. ## Requesting Approval for Licenses @@ -104,7 +106,10 @@ Gems which are included only in the "development" or "test" groups by Bundler ar [OSL-GNU]: https://www.gnu.org/licenses/license-list.en.html#OSL [Org-Repo]: https://gitlab.com/gitlab-com/organization [UNLICENSE]: https://unlicense.org +[OWFa1]: http://www.openwebfoundation.org/legal/the-owf-1-0-agreements/owfa-1-0 [Facebook]: https://code.facebook.com/pages/850928938376556 [x-list]: https://www.apache.org/legal/resolved.html#category-x [Acceptable-Licenses]: #acceptable-licenses [Unacceptable-Licenses]: #unacceptable-licenses +[WTFPL]: https://wtfpl.net +[WTFPL-OSI]: https://opensource.org/minutes20090304 diff --git a/doc/development/profiling.md b/doc/development/profiling.md index 933033a09e0..af79353b721 100644 --- a/doc/development/profiling.md +++ b/doc/development/profiling.md @@ -27,3 +27,13 @@ Bullet will log query problems to both the Rails log as well as the Chrome console. As a follow up to finding `N+1` queries with Bullet, consider writing a [QueryRecoder test](query_recorder.md) to prevent a regression. + +## GitLab Profiler + + +[Gitlab-Profiler](https://gitlab.com/gitlab-com/gitlab-profiler) was built to +help developers understand why specific URLs of their application may be slow +and to provide hard data that can help reduce load times. + +For GitLab.com, you can find the latest results here: +<http://redash.gitlab.com/dashboard/gitlab-profiler-statistics> diff --git a/doc/development/testing.md b/doc/development/testing.md index 83269303005..45b1519ece8 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -1,556 +1 @@ -# Testing Standards and Style Guidelines - -This guide outlines standards and best practices for automated testing of GitLab -CE and EE. - -It is meant to be an _extension_ of the [thoughtbot testing -styleguide](https://github.com/thoughtbot/guides/tree/master/style/testing). If -this guide defines a rule that contradicts the thoughtbot guide, this guide -takes precedence. Some guidelines may be repeated verbatim to stress their -importance. - -## Definitions - -### Unit tests - -Formal definition: https://en.wikipedia.org/wiki/Unit_testing - -These kind of tests ensure that a single unit of code (a method) works as -expected (given an input, it has a predictable output). These tests should be -isolated as much as possible. For example, model methods that don't do anything -with the database shouldn't need a DB record. Classes that don't need database -records should use stubs/doubles as much as possible. - -| Code path | Tests path | Testing engine | Notes | -| --------- | ---------- | -------------- | ----- | -| `app/finders/` | `spec/finders/` | RSpec | | -| `app/helpers/` | `spec/helpers/` | RSpec | | -| `app/db/{post_,}migrate/` | `spec/migrations/` | RSpec | More details at [`spec/migrations/README.md`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md). | -| `app/policies/` | `spec/policies/` | RSpec | | -| `app/presenters/` | `spec/presenters/` | RSpec | | -| `app/routing/` | `spec/routing/` | RSpec | | -| `app/serializers/` | `spec/serializers/` | RSpec | | -| `app/services/` | `spec/services/` | RSpec | | -| `app/tasks/` | `spec/tasks/` | RSpec | | -| `app/uploaders/` | `spec/uploaders/` | RSpec | | -| `app/views/` | `spec/views/` | RSpec | | -| `app/workers/` | `spec/workers/` | RSpec | | -| `app/assets/javascripts/` | `spec/javascripts/` | Karma | More details in the [JavaScript](#javascript) section. | - -### Integration tests - -Formal definition: https://en.wikipedia.org/wiki/Integration_testing - -These kind of tests ensure that individual parts of the application work well together, without the overhead of the actual app environment (i.e. the browser). These tests should assert at the request/response level: status code, headers, body. They're useful to test permissions, redirections, what view is rendered etc. - -| Code path | Tests path | Testing engine | Notes | -| --------- | ---------- | -------------- | ----- | -| `app/controllers/` | `spec/controllers/` | RSpec | | -| `app/mailers/` | `spec/mailers/` | RSpec | | -| `lib/api/` | `spec/requests/api/` | RSpec | | -| `lib/ci/api/` | `spec/requests/ci/api/` | RSpec | | -| `app/assets/javascripts/` | `spec/javascripts/` | Karma | More details in the [JavaScript](#javascript) section. | - -#### About controller tests - -In an ideal world, controllers should be thin. However, when this is not the -case, it's acceptable to write a system/feature test without JavaScript instead -of a controller test. The reason is that testing a fat controller usually -involves a lot of stubbing, things like: - -```ruby -controller.instance_variable_set(:@user, user) -``` - -and use methods which are deprecated in Rails 5 ([#23768]). - -[#23768]: https://gitlab.com/gitlab-org/gitlab-ce/issues/23768 - -#### About Karma - -As you may have noticed, Karma is both in the Unit tests and the Integration -tests category. That's because Karma is a tool that provides an environment to -run JavaScript tests, so you can either run unit tests (e.g. test a single -JavaScript method), or integration tests (e.g. test a component that is composed -of multiple components). - -### System tests or Feature tests - -Formal definition: https://en.wikipedia.org/wiki/System_testing. - -These kind of tests ensure the application works as expected from a user point -of view (aka black-box testing). These tests should test a happy path for a -given page or set of pages, and a test case should be added for any regression -that couldn't have been caught at lower levels with better tests (i.e. if a -regression is found, regression tests should be added at the lowest-level -possible). - -| Tests path | Testing engine | Notes | -| ---------- | -------------- | ----- | -| `spec/features/` | [Capybara] + [RSpec] | If your spec has the `:js` metadata, the browser driver will be [Poltergeist], otherwise it's using [RackTest]. | -| `features/` | Spinach | Spinach tests are deprecated, [you shouldn't add new Spinach tests](#spinach-feature-tests). | - -[Capybara]: https://github.com/teamcapybara/capybara -[RSpec]: https://github.com/rspec/rspec-rails#feature-specs -[Poltergeist]: https://github.com/teamcapybara/capybara#poltergeist -[RackTest]: https://github.com/teamcapybara/capybara#racktest - -#### Best practices - -- Create only the necessary records in the database -- Test a happy path and a less happy path but that's it -- Every other possible path should be tested with Unit or Integration tests -- Test what's displayed on the page, not the internals of ActiveRecord models. - For instance, if you want to verify that a record was created, add - expectations that its attributes are displayed on the page, not that - `Model.count` increased by one. -- It's ok to look for DOM elements but don't abuse it since it makes the tests - more brittle - -If we're confident that the low-level components work well (and we should be if -we have enough Unit & Integration tests), we shouldn't need to duplicate their -thorough testing at the System test level. - -It's very easy to add tests, but a lot harder to remove or improve tests, so one -should take care of not introducing too many (slow and duplicated) specs. - -The reasons why we should follow these best practices are as follows: - -- System tests are slow to run since they spin up the entire application stack - in a headless browser, and even slower when they integrate a JS driver -- When system tests run with a JavaScript driver, the tests are run in a - different thread than the application. This means it does not share a - database connection and your test will have to commit the transactions in - order for the running application to see the data (and vice-versa). In that - case we need to truncate the database after each spec instead of simply - rolling back a transaction (the faster strategy that's in use for other kind - of tests). This is slower than transactions, however, so we want to use - truncation only when necessary. - -### Black-box tests or End-to-end tests - -GitLab consists of [multiple pieces] such as [GitLab Shell], [GitLab Workhorse], -[Gitaly], [GitLab Pages], [GitLab Runner], and GitLab Rails. All theses pieces -are configured and packaged by [GitLab Omnibus]. - -[GitLab QA] is a tool that allows to test that all these pieces integrate well -together by building a Docker image for a given version of GitLab Rails and -running feature tests (i.e. using Capybara) against it. - -The actual test scenarios and steps are [part of GitLab Rails] so that they're -always in-sync with the codebase. - -[multiple pieces]: ./architecture.md#components -[GitLab Shell]: https://gitlab.com/gitlab-org/gitlab-shell -[GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse -[Gitaly]: https://gitlab.com/gitlab-org/gitaly -[GitLab Pages]: https://gitlab.com/gitlab-org/gitlab-pages -[GitLab Runner]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner -[GitLab Omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab -[GitLab QA]: https://gitlab.com/gitlab-org/gitlab-qa -[part of GitLab Rails]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa - -## How to test at the correct level? - -As many things in life, deciding what to test at each level of testing is a -trade-off: - -- Unit tests are usually cheap, and you should consider them like the basement - of your house: you need them to be confident that your code is behaving - correctly. However if you run only unit tests without integration / system - tests, you might [miss] the [big] [picture]! -- Integration tests are a bit more expensive, but don't abuse them. A system test - is often better than an integration test that is stubbing a lot of internals. -- System tests are expensive (compared to unit tests), even more if they require - a JavaScript driver. Make sure to follow the guidelines in the [Speed](#test-speed) - section. - -Another way to see it is to think about the "cost of tests", this is well -explained [in this article][tests-cost] and the basic idea is that the cost of a -test includes: - -- The time it takes to write the test -- The time it takes to run the test every time the suite runs -- The time it takes to understand the test -- The time it takes to fix the test if it breaks and the underlying code is OK -- Maybe, the time it takes to change the code to make the code testable. - -[miss]: https://twitter.com/ThePracticalDev/status/850748070698651649 -[big]: https://twitter.com/timbray/status/822470746773409794 -[picture]: https://twitter.com/withzombies/status/829716565834752000 -[tests-cost]: https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e - -## Frontend testing - -Please consult the [dedicated "Frontend testing" guide](./fe_guide/testing.md). - -## RSpec - -### General Guidelines - -- Use a single, top-level `describe ClassName` block. -- Use `.method` to describe class methods and `#method` to describe instance - methods. -- Use `context` to test branching logic. -- Don't assert against the absolute value of a sequence-generated attribute (see [Gotchas](gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). -- Try to match the ordering of tests to the ordering within the class. -- Try to follow the [Four-Phase Test][four-phase-test] pattern, using newlines - to separate phases. -- Use `Gitlab.config.gitlab.host` rather than hard coding `'localhost'` -- Don't assert against the absolute value of a sequence-generated attribute (see - [Gotchas](gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). -- Don't supply the `:each` argument to hooks since it's the default. -- On `before` and `after` hooks, prefer it scoped to `:context` over `:all` - -[four-phase-test]: https://robots.thoughtbot.com/four-phase-test - -### Automatic retries and flaky tests detection - -On our CI, we use [rspec-retry] to automatically retry a failing example a few -times (see [`spec/spec_helper.rb`] for the precise retries count). - -We also use a home-made `RspecFlaky::Listener` listener which records flaky -examples in a JSON report file on `master` (`retrieve-tests-metadata` and `update-tests-metadata` jobs), and warns when a new flaky example -is detected in any other branch (`flaky-examples-check` job). In the future, the -`flaky-examples-check` job will not be allowed to fail. - -[rspec-retry]: https://github.com/NoRedInk/rspec-retry -[`spec/spec_helper.rb`]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/spec_helper.rb - -### `let` variables - -GitLab's RSpec suite has made extensive use of `let` variables to reduce -duplication. However, this sometimes [comes at the cost of clarity][lets-not], -so we need to set some guidelines for their use going forward: - -- `let` variables are preferable to instance variables. Local variables are - preferable to `let` variables. -- Use `let` to reduce duplication throughout an entire spec file. -- Don't use `let` to define variables used by a single test; define them as - local variables inside the test's `it` block. -- Don't define a `let` variable inside the top-level `describe` block that's - only used in a more deeply-nested `context` or `describe` block. Keep the - definition as close as possible to where it's used. -- Try to avoid overriding the definition of one `let` variable with another. -- Don't define a `let` variable that's only used by the definition of another. - Use a helper method instead. - -[lets-not]: https://robots.thoughtbot.com/lets-not - -#### `set` variables - -In some cases there is no need to recreate the same object for tests again for -each example. For example, a project is needed to test issues on the same -project, one project will do for the entire file. This can be achieved by using -`set` in the same way you would use `let`. - -`rspec-set` only works on ActiveRecord objects, and before new examples it -reloads or recreates the model, _only_ if needed. That is, when you changed -properties or destroyed the object. - -There is one gotcha; you can't reference a model defined in a `let` block in a -`set` block. - -### Time-sensitive tests - -[Timecop](https://github.com/travisjeffery/timecop) is available in our -Ruby-based tests for verifying things that are time-sensitive. Any test that -exercises or verifies something time-sensitive should make use of Timecop to -prevent transient test failures. - -Example: - -```ruby -it 'is overdue' do - issue = build(:issue, due_date: Date.tomorrow) - - Timecop.freeze(3.days.from_now) do - expect(issue).to be_overdue - end -end -``` - -### System / Feature tests - -- Feature specs should be named `ROLE_ACTION_spec.rb`, such as - `user_changes_password_spec.rb`. -- Use only one `feature` block per feature spec file. -- Use scenario titles that describe the success and failure paths. -- Avoid scenario titles that add no information, such as "successfully". -- Avoid scenario titles that repeat the feature title. - -### Table-based / Parameterized tests - -This style of testing is used to exercise one piece of code with a comprehensive -range of inputs. By specifying the test case once, alongside a table of inputs -and the expected output for each, your tests can be made easier to read and more -compact. - -We use the [rspec-parameterized](https://github.com/tomykaira/rspec-parameterized) -gem. A short example, using the table syntax and checking Ruby equality for a -range of inputs, might look like this: - -```ruby -describe "#==" do - using Rspec::Parameterized::TableSyntax - - let(:project1) { create(:project) } - let(:project2) { create(:project) } - where(:a, :b, :result) do - 1 | 1 | true - 1 | 2 | false - true | true | true - true | false | false - project1 | project1 | true - project2 | project2 | true - project 1 | project2 | false - end - - with_them do - it { expect(a == b).to eq(result) } - - it 'is isomorphic' do - expect(b == a).to eq(result) - end - end -end -``` - -### Matchers - -Custom matchers should be created to clarify the intent and/or hide the -complexity of RSpec expectations.They should be placed under -`spec/support/matchers/`. Matchers can be placed in subfolder if they apply to -a certain type of specs only (e.g. features, requests etc.) but shouldn't be if -they apply to multiple type of specs. - -#### have_gitlab_http_status - -Prefer `have_gitlab_http_status` over `have_http_status` because the former -could also show the response body whenever the status mismatched. This would -be very useful whenever some tests start breaking and we would love to know -why without editing the source and rerun the tests. - -This is especially useful whenever it's showing 500 internal server error. - -### Shared contexts - -All shared contexts should be be placed under `spec/support/shared_contexts/`. -Shared contexts can be placed in subfolder if they apply to a certain type of -specs only (e.g. features, requests etc.) but shouldn't be if they apply to -multiple type of specs. - -Each file should include only one context and have a descriptive name, e.g. -`spec/support/shared_contexts/controllers/githubish_import_controller_shared_context.rb`. - -### Shared examples - -All shared examples should be be placed under `spec/support/shared_examples/`. -Shared examples can be placed in subfolder if they apply to a certain type of -specs only (e.g. features, requests etc.) but shouldn't be if they apply to -multiple type of specs. - -Each file should include only one context and have a descriptive name, e.g. -`spec/support/shared_examples/controllers/githubish_import_controller_shared_example.rb`. - -### Helpers - -Helpers are usually modules that provide some methods to hide the complexity of -specific RSpec examples. You can define helpers in RSpec files if they're not -intended to be shared with other specs. Otherwise, they should be be placed -under `spec/support/helpers/`. Helpers can be placed in subfolder if they apply -to a certain type of specs only (e.g. features, requests etc.) but shouldn't be -if they apply to multiple type of specs. - -Helpers should follow the Rails naming / namespacing convention. For instance -`spec/support/helpers/cycle_analytics_helpers.rb` should define: - -```ruby -module Spec - module Support - module Helpers - module CycleAnalyticsHelpers - def create_commit_referencing_issue(issue, branch_name: random_git_name) - project.repository.add_branch(user, branch_name, 'master') - create_commit("Commit for ##{issue.iid}", issue.project, user, branch_name) - end - end - end - end -end -``` - -Helpers should not change the RSpec config. For instance, the helpers module -described above should not include: - -```ruby -RSpec.configure do |config| - config.include Spec::Support::Helpers::CycleAnalyticsHelpers -end -``` - -### Factories - -GitLab uses [factory_girl] as a test fixture replacement. - -- Factory definitions live in `spec/factories/`, named using the pluralization - of their corresponding model (`User` factories are defined in `users.rb`). -- There should be only one top-level factory definition per file. -- FactoryGirl methods are mixed in to all RSpec groups. This means you can (and - should) call `create(...)` instead of `FactoryGirl.create(...)`. -- Make use of [traits] to clean up definitions and usages. -- When defining a factory, don't define attributes that are not required for the - resulting record to pass validation. -- When instantiating from a factory, don't supply attributes that aren't - required by the test. -- Factories don't have to be limited to `ActiveRecord` objects. - [See example](https://gitlab.com/gitlab-org/gitlab-ce/commit/0b8cefd3b2385a21cfed779bd659978c0402766d). - -[factory_girl]: https://github.com/thoughtbot/factory_girl -[traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits - -### Fixtures - -All fixtures should be be placed under `spec/fixtures/`. - -### Config - -RSpec config files are files that change the RSpec config (i.e. -`RSpec.configure do |config|` blocks). They should be placed under -`spec/support/config/`. - -Each file should be related to a specific domain, e.g. -`spec/support/config/capybara.rb`, `spec/support/config/carrierwave.rb`, etc. - -Helpers can be included in the `spec/support/config/rspec.rb` file. If a -helpers module applies only to a certain kind of specs, it should add modifiers -to the `config.include` call. For instance if -`spec/support/helpers/cycle_analytics_helpers.rb` applies to `:lib` and -`type: :model` specs only, you would write the following: - -```ruby -RSpec.configure do |config| - config.include Spec::Support::Helpers::CycleAnalyticsHelpers, :lib - config.include Spec::Support::Helpers::CycleAnalyticsHelpers, type: :model -end -``` - -## Testing Rake Tasks - -To make testing Rake tasks a little easier, there is a helper that can be included -in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use -`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures -a few other things to make testing Rake tasks easier. - -At a minimum, requiring the Rake helper will redirect `stdout`, include the -runtime task helpers, and include the `RakeHelpers` Spec support module. - -The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make -executing tasks simple. See `spec/support/rake_helpers.rb` for all available -methods. - -Example: - -```ruby -require 'rake_helper' - -describe 'gitlab:shell rake tasks' do - before do - Rake.application.rake_require 'tasks/gitlab/shell' - - stub_warn_user_is_not_gitlab - end - - describe 'install task' do - it 'invokes create_hooks task' do - expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke) - - run_rake_task('gitlab:shell:install') - end - end -end -``` - -## Test speed - -GitLab has a massive test suite that, without [parallelization], can take hours -to run. It's important that we make an effort to write tests that are accurate -and effective _as well as_ fast. - -Here are some things to keep in mind regarding test performance: - -- `double` and `spy` are faster than `FactoryGirl.build(...)` -- `FactoryGirl.build(...)` and `.build_stubbed` are faster than `.create`. -- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`, - `spy`, or `double` will do. Database persistence is slow! -- Don't mark a feature as requiring JavaScript (through `@javascript` in - Spinach or `:js` in RSpec) unless it's _actually_ required for the test - to be valid. Headless browser testing is slow! - -[parallelization]: #test-suite-parallelization-on-the-ci - -### Test suite parallelization on the CI - -Our current CI parallelization setup is as follows: - -1. The `knapsack` job in the prepare stage that is supposed to ensure we have a - `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file: - - The `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file is fetched - from S3, if it's not here we initialize the file with `{}`. -1. Each `rspec x y` job are run with `knapsack rspec` and should have an evenly - distributed share of tests: - - It works because the jobs have access to the - `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` since the "artifacts - from all previous stages are passed by default". [^1] - - the jobs set their own report path to - `KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`. - - if knapsack is doing its job, test files that are run should be listed under - `Report specs`, not under `Leftover specs`. -1. The `update-knapsack` job takes all the - `knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json` - files from the `rspec x y` jobs and merge them all together into a single - `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that is then - uploaded to S3. - -After that, the next pipeline will use the up-to-date -`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy -is used for Spinach tests as well. - -### Monitoring - -The GitLab test suite is [monitored] for the `master` branch, and any branch -that includes `rspec-profile` in their name. - -A [public dashboard] is available for everyone to see. Feel free to look at the -slowest test files and try to improve them. - -[monitored]: ./performance.md#rspec-profiling -[public dashboard]: https://redash.gitlab.com/public/dashboards/l1WhHXaxrCWM5Ai9D7YDqHKehq6OU3bx5gssaiWe?org_slug=default - -## CI setup - -- On CE and EE, the test suite runs both PostgreSQL and MySQL. -- Rails logging to `log/test.log` is disabled by default in CI [for - performance reasons][logging]. To override this setting, provide the - `RAILS_ENABLE_TEST_LOG` environment variable. - -[logging]: https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4 - -## Spinach (feature) tests - -GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426) -for its feature/integration tests in September 2012. - -As of March 2016, we are [trying to avoid adding new Spinach -tests](https://gitlab.com/gitlab-org/gitlab-ce/issues/14121) going forward, -opting for [RSpec feature](#features-integration) specs. - -Adding new Spinach scenarios is acceptable _only if_ the new scenario requires -no more than one new `step` definition. If more than that is required, the -test should be re-implemented using RSpec instead. - ---- - -[Return to Development documentation](README.md) - -[^1]: /ci/yaml/README.html#dependencies +This document was moved to [testing_guide/index.md](testing_guide/index.md). diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md new file mode 100644 index 00000000000..8b7b015427f --- /dev/null +++ b/doc/development/testing_guide/best_practices.md @@ -0,0 +1,303 @@ +# Testing best practices + +## Test speed + +GitLab has a massive test suite that, without [parallelization], can take hours +to run. It's important that we make an effort to write tests that are accurate +and effective _as well as_ fast. + +Here are some things to keep in mind regarding test performance: + +- `double` and `spy` are faster than `FactoryGirl.build(...)` +- `FactoryGirl.build(...)` and `.build_stubbed` are faster than `.create`. +- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`, + `spy`, or `double` will do. Database persistence is slow! +- Don't mark a feature as requiring JavaScript (through `@javascript` in + Spinach or `:js` in RSpec) unless it's _actually_ required for the test + to be valid. Headless browser testing is slow! + +[parallelization]: ci.md#test-suite-parallelization-on-the-ci + +## RSpec + +### General guidelines + +- Use a single, top-level `describe ClassName` block. +- Use `.method` to describe class methods and `#method` to describe instance + methods. +- Use `context` to test branching logic. +- Don't assert against the absolute value of a sequence-generated attribute (see [Gotchas](../gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). +- Try to match the ordering of tests to the ordering within the class. +- Try to follow the [Four-Phase Test][four-phase-test] pattern, using newlines + to separate phases. +- Use `Gitlab.config.gitlab.host` rather than hard coding `'localhost'` +- Don't assert against the absolute value of a sequence-generated attribute (see + [Gotchas](../gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). +- Don't supply the `:each` argument to hooks since it's the default. +- On `before` and `after` hooks, prefer it scoped to `:context` over `:all` +- When using `evaluate_script("$('.js-foo').testSomething()")` (or `execute_script`) which acts on a given element, + use a Capyabara matcher beforehand (e.g. `find('.js-foo')`) to ensure the element actually exists. + +[four-phase-test]: https://robots.thoughtbot.com/four-phase-test + +### System / Feature tests + +NOTE: **Note:** Before writing a new system test, [please consider **not** +writing one](testing_levels.md#consider-not-writing-a-system-test)! + +- Feature specs should be named `ROLE_ACTION_spec.rb`, such as + `user_changes_password_spec.rb`. +- Use scenario titles that describe the success and failure paths. +- Avoid scenario titles that add no information, such as "successfully". +- Avoid scenario titles that repeat the feature title. +- Create only the necessary records in the database +- Test a happy path and a less happy path but that's it +- Every other possible path should be tested with Unit or Integration tests +- Test what's displayed on the page, not the internals of ActiveRecord models. + For instance, if you want to verify that a record was created, add + expectations that its attributes are displayed on the page, not that + `Model.count` increased by one. +- It's ok to look for DOM elements but don't abuse it since it makes the tests + more brittle + +#### Debugging Capybara + +Sometimes you may need to debug Capybara tests by observing browser behavior. + +You can pause Capybara and view the website on the browser by using the +`live_debug` method in your spec. The current page will be automatically opened +in your default browser. +You may need to sign in first (the current user's credentials are displayed in +the terminal). + +To resume the test run, press any key. + +For example: + +``` +$ bin/rspec spec/features/auto_deploy_spec.rb:34 +Running via Spring preloader in process 8999 +Run options: include {:locations=>{"./spec/features/auto_deploy_spec.rb"=>[34]}} + +Current example is paused for live debugging +The current user credentials are: user2 / 12345678 +Press any key to resume the execution of the example! +Back to the example! +. + +Finished in 34.51 seconds (files took 0.76702 seconds to load) +1 example, 0 failures +``` + +### `let` variables + +GitLab's RSpec suite has made extensive use of `let` variables to reduce +duplication. However, this sometimes [comes at the cost of clarity][lets-not], +so we need to set some guidelines for their use going forward: + +- `let` variables are preferable to instance variables. Local variables are + preferable to `let` variables. +- Use `let` to reduce duplication throughout an entire spec file. +- Don't use `let` to define variables used by a single test; define them as + local variables inside the test's `it` block. +- Don't define a `let` variable inside the top-level `describe` block that's + only used in a more deeply-nested `context` or `describe` block. Keep the + definition as close as possible to where it's used. +- Try to avoid overriding the definition of one `let` variable with another. +- Don't define a `let` variable that's only used by the definition of another. + Use a helper method instead. + +[lets-not]: https://robots.thoughtbot.com/lets-not + +### `set` variables + +In some cases there is no need to recreate the same object for tests again for +each example. For example, a project is needed to test issues on the same +project, one project will do for the entire file. This can be achieved by using +`set` in the same way you would use `let`. + +`rspec-set` only works on ActiveRecord objects, and before new examples it +reloads or recreates the model, _only_ if needed. That is, when you changed +properties or destroyed the object. + +There is one gotcha; you can't reference a model defined in a `let` block in a +`set` block. + +### Time-sensitive tests + +[Timecop](https://github.com/travisjeffery/timecop) is available in our +Ruby-based tests for verifying things that are time-sensitive. Any test that +exercises or verifies something time-sensitive should make use of Timecop to +prevent transient test failures. + +Example: + +```ruby +it 'is overdue' do + issue = build(:issue, due_date: Date.tomorrow) + + Timecop.freeze(3.days.from_now) do + expect(issue).to be_overdue + end +end +``` + +### Table-based / Parameterized tests + +This style of testing is used to exercise one piece of code with a comprehensive +range of inputs. By specifying the test case once, alongside a table of inputs +and the expected output for each, your tests can be made easier to read and more +compact. + +We use the [rspec-parameterized](https://github.com/tomykaira/rspec-parameterized) +gem. A short example, using the table syntax and checking Ruby equality for a +range of inputs, might look like this: + +```ruby +describe "#==" do + using RSpec::Parameterized::TableSyntax + + let(:project1) { create(:project) } + let(:project2) { create(:project) } + where(:a, :b, :result) do + 1 | 1 | true + 1 | 2 | false + true | true | true + true | false | false + project1 | project1 | true + project2 | project2 | true + project 1 | project2 | false + end + + with_them do + it { expect(a == b).to eq(result) } + + it 'is isomorphic' do + expect(b == a).to eq(result) + end + end +end +``` + +### Matchers + +Custom matchers should be created to clarify the intent and/or hide the +complexity of RSpec expectations.They should be placed under +`spec/support/matchers/`. Matchers can be placed in subfolder if they apply to +a certain type of specs only (e.g. features, requests etc.) but shouldn't be if +they apply to multiple type of specs. + +#### `have_gitlab_http_status` + +Prefer `have_gitlab_http_status` over `have_http_status` because the former +could also show the response body whenever the status mismatched. This would +be very useful whenever some tests start breaking and we would love to know +why without editing the source and rerun the tests. + +This is especially useful whenever it's showing 500 internal server error. + +### Shared contexts + +All shared contexts should be be placed under `spec/support/shared_contexts/`. +Shared contexts can be placed in subfolder if they apply to a certain type of +specs only (e.g. features, requests etc.) but shouldn't be if they apply to +multiple type of specs. + +Each file should include only one context and have a descriptive name, e.g. +`spec/support/shared_contexts/controllers/githubish_import_controller_shared_context.rb`. + +### Shared examples + +All shared examples should be be placed under `spec/support/shared_examples/`. +Shared examples can be placed in subfolder if they apply to a certain type of +specs only (e.g. features, requests etc.) but shouldn't be if they apply to +multiple type of specs. + +Each file should include only one context and have a descriptive name, e.g. +`spec/support/shared_examples/controllers/githubish_import_controller_shared_example.rb`. + +### Helpers + +Helpers are usually modules that provide some methods to hide the complexity of +specific RSpec examples. You can define helpers in RSpec files if they're not +intended to be shared with other specs. Otherwise, they should be be placed +under `spec/support/helpers/`. Helpers can be placed in subfolder if they apply +to a certain type of specs only (e.g. features, requests etc.) but shouldn't be +if they apply to multiple type of specs. + +Helpers should follow the Rails naming / namespacing convention. For instance +`spec/support/helpers/cycle_analytics_helpers.rb` should define: + +```ruby +module Spec + module Support + module Helpers + module CycleAnalyticsHelpers + def create_commit_referencing_issue(issue, branch_name: random_git_name) + project.repository.add_branch(user, branch_name, 'master') + create_commit("Commit for ##{issue.iid}", issue.project, user, branch_name) + end + end + end + end +end +``` + +Helpers should not change the RSpec config. For instance, the helpers module +described above should not include: + +```ruby +RSpec.configure do |config| + config.include Spec::Support::Helpers::CycleAnalyticsHelpers +end +``` + +### Factories + +GitLab uses [factory_girl] as a test fixture replacement. + +- Factory definitions live in `spec/factories/`, named using the pluralization + of their corresponding model (`User` factories are defined in `users.rb`). +- There should be only one top-level factory definition per file. +- FactoryGirl methods are mixed in to all RSpec groups. This means you can (and + should) call `create(...)` instead of `FactoryGirl.create(...)`. +- Make use of [traits] to clean up definitions and usages. +- When defining a factory, don't define attributes that are not required for the + resulting record to pass validation. +- When instantiating from a factory, don't supply attributes that aren't + required by the test. +- Factories don't have to be limited to `ActiveRecord` objects. + [See example](https://gitlab.com/gitlab-org/gitlab-ce/commit/0b8cefd3b2385a21cfed779bd659978c0402766d). + +[factory_girl]: https://github.com/thoughtbot/factory_girl +[traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits + +### Fixtures + +All fixtures should be be placed under `spec/fixtures/`. + +### Config + +RSpec config files are files that change the RSpec config (i.e. +`RSpec.configure do |config|` blocks). They should be placed under +`spec/support/config/`. + +Each file should be related to a specific domain, e.g. +`spec/support/config/capybara.rb`, `spec/support/config/carrierwave.rb`, etc. + +Helpers can be included in the `spec/support/config/rspec.rb` file. If a +helpers module applies only to a certain kind of specs, it should add modifiers +to the `config.include` call. For instance if +`spec/support/helpers/cycle_analytics_helpers.rb` applies to `:lib` and +`type: :model` specs only, you would write the following: + +```ruby +RSpec.configure do |config| + config.include Spec::Support::Helpers::CycleAnalyticsHelpers, :lib + config.include Spec::Support::Helpers::CycleAnalyticsHelpers, type: :model +end +``` + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/testing_guide/ci.md b/doc/development/testing_guide/ci.md new file mode 100644 index 00000000000..e90de55068d --- /dev/null +++ b/doc/development/testing_guide/ci.md @@ -0,0 +1,52 @@ +# GitLab tests in the Continuous Integration (CI) context + +### Test suite parallelization on the CI + +Our current CI parallelization setup is as follows: + +1. The `knapsack` job in the prepare stage that is supposed to ensure we have a + `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file: + - The `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file is fetched + from S3, if it's not here we initialize the file with `{}`. +1. Each `rspec x y` job are run with `knapsack rspec` and should have an evenly + distributed share of tests: + - It works because the jobs have access to the + `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` since the "artifacts + from all previous stages are passed by default". [^1] + - the jobs set their own report path to + `KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`. + - if knapsack is doing its job, test files that are run should be listed under + `Report specs`, not under `Leftover specs`. +1. The `update-knapsack` job takes all the + `knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json` + files from the `rspec x y` jobs and merge them all together into a single + `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that is then + uploaded to S3. + +After that, the next pipeline will use the up-to-date +`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy +is used for Spinach tests as well. + +### Monitoring + +The GitLab test suite is [monitored] for the `master` branch, and any branch +that includes `rspec-profile` in their name. + +A [public dashboard] is available for everyone to see. Feel free to look at the +slowest test files and try to improve them. + +[monitored]: ../performance.md#rspec-profiling +[public dashboard]: https://redash.gitlab.com/public/dashboards/l1WhHXaxrCWM5Ai9D7YDqHKehq6OU3bx5gssaiWe?org_slug=default + +## CI setup + +- On CE and EE, the test suite runs both PostgreSQL and MySQL. +- Rails logging to `log/test.log` is disabled by default in CI [for + performance reasons][logging]. To override this setting, provide the + `RAILS_ENABLE_TEST_LOG` environment variable. + +[logging]: https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4 + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md new file mode 100644 index 00000000000..bbb2313ea7b --- /dev/null +++ b/doc/development/testing_guide/flaky_tests.md @@ -0,0 +1,74 @@ +# Flaky tests + +## What's a flaky test? + +It's a test that sometimes fails, but if you retry it enough times, it passes, +eventually. + +## Automatic retries and flaky tests detection + +On our CI, we use [rspec-retry] to automatically retry a failing example a few +times (see [`spec/spec_helper.rb`] for the precise retries count). + +We also use a home-made `RspecFlaky::Listener` listener which records flaky +examples in a JSON report file on `master` (`retrieve-tests-metadata` and `update-tests-metadata` jobs), and warns when a new flaky example +is detected in any other branch (`flaky-examples-check` job). In the future, the +`flaky-examples-check` job will not be allowed to fail. + +This was originally implemented in: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13021. + +[rspec-retry]: https://github.com/NoRedInk/rspec-retry +[`spec/spec_helper.rb`]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/spec_helper.rb + +## Problems we had in the past at GitLab + +- [`rspec-retry` is bitting us when some API specs fail](https://gitlab.com/gitlab-org/gitlab-ce/issues/29242): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9825 +- [Sporadic RSpec failures due to `PG::UniqueViolation`](https://gitlab.com/gitlab-org/gitlab-ce/issues/28307#note_24958837): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9846 + - Follow-up: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10688 + - [Capybara.reset_session! should be called before requests are blocked](https://gitlab.com/gitlab-org/gitlab-ce/issues/33779): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12224 +- FFaker generates funky data that tests are not ready to handle (and tests should be predictable so that's bad!): + - [Make `spec/mailers/notify_spec.rb` more robust](https://gitlab.com/gitlab-org/gitlab-ce/issues/20121): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10015 + - [Transient failure in spec/requests/api/commits_spec.rb](https://gitlab.com/gitlab-org/gitlab-ce/issues/27988#note_25342521): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9944 + - [Replace FFaker factory data with sequences](https://gitlab.com/gitlab-org/gitlab-ce/issues/29643): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10184 + - [Transient failure in spec/finders/issues_finder_spec.rb](https://gitlab.com/gitlab-org/gitlab-ce/issues/30211#note_26707685): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10404 + +### Time-sensitive flaky tests + +- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10046 +- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10306 + +### Array order expectation + +- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10148 + +### Feature tests + +- [Be sure to create all the data the test need before starting exercize](https://gitlab.com/gitlab-org/gitlab-ce/issues/32622#note_31128195): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12059 +- [Bis](https://gitlab.com/gitlab-org/gitlab-ce/issues/34609#note_34048715): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12604 +- [Bis](https://gitlab.com/gitlab-org/gitlab-ce/issues/34698#note_34276286): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12664 +- [Assert against the underlying database state instead of against a page's content](https://gitlab.com/gitlab-org/gitlab-ce/issues/31437): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10934 + +#### Capybara viewport size related issues + +- [Transient failure of spec/features/issues/filtered_search/filter_issues_spec.rb](https://gitlab.com/gitlab-org/gitlab-ce/issues/29241#note_26743936): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10411 + +#### Capybara JS driver related issues + +- [Don't wait for AJAX when no AJAX request is fired](https://gitlab.com/gitlab-org/gitlab-ce/issues/30461): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10454 +- [Bis](https://gitlab.com/gitlab-org/gitlab-ce/issues/34647): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12626 + +#### PhantomJS / WebKit related issues + +- Memory is through the roof! (TL;DR: Load images but block images requests!): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12003 + +## Resources + +- [Flaky Tests: Are You Sure You Want to Rerun Them?](http://semaphoreci.com/blog/2017/04/20/flaky-tests.html) +- [How to Deal With and Eliminate Flaky Tests](https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests) +- [Tips on Treating Flakiness in your Rails Test Suite](http://semaphoreci.com/blog/2017/08/03/tips-on-treating-flakiness-in-your-test-suite.html) +- ['Flaky' tests: a short story](https://www.ombulabs.com/blog/rspec/continuous-integration/how-to-track-down-a-flaky-test.html) +- [Using Insights to Discover Flaky, Slow, and Failed Tests](https://circleci.com/blog/using-insights-to-discover-flaky-slow-and-failed-tests/) + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md new file mode 100644 index 00000000000..0c63f51cb45 --- /dev/null +++ b/doc/development/testing_guide/frontend_testing.md @@ -0,0 +1,254 @@ +# Frontend testing standards and style guidelines + +There are two types of test suites you'll encounter while developing frontend code +at GitLab. We use Karma and Jasmine for JavaScript unit and integration testing, +and RSpec feature tests with Capybara for e2e (end-to-end) integration testing. + +Unit and feature tests need to be written for all new features. +Most of the time, you should use [RSpec] for your feature tests. + +Regression tests should be written for bug fixes to prevent them from recurring +in the future. + +See the [Testing Standards and Style Guidelines](index.md) page for more +information on general testing practices at GitLab. + +## Karma test suite + +GitLab uses the [Karma][karma] test runner with [Jasmine] as its test +framework for our JavaScript unit and integration tests. For integration tests, +we generate HTML files using RSpec (see `spec/javascripts/fixtures/*.rb` for examples). +Some fixtures are still HAML templates that are translated to HTML files using the same mechanism (see `static_fixtures.rb`). +Adding these static fixtures should be avoided as they are harder to keep up to date with real views. +The existing static fixtures will be migrated over time. +Please see [gitlab-org/gitlab-ce#24753](https://gitlab.com/gitlab-org/gitlab-ce/issues/24753) to track our progress. +Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin. + +JavaScript tests live in `spec/javascripts/`, matching the folder structure +of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js` +has a corresponding `spec/javascripts/behaviors/autosize_spec.js` file. + +Keep in mind that in a CI environment, these tests are run in a headless +browser and you will not have access to certain APIs, such as +[`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification), +which will have to be stubbed. + +### Best practices + +#### Naming unit tests + +When writing describe test blocks to test specific functions/methods, +please use the method name as the describe block name. + +```javascript +// Good +describe('methodName', () => { + it('passes', () => { + expect(true).toEqual(true); + }); +}); + +// Bad +describe('#methodName', () => { + it('passes', () => { + expect(true).toEqual(true); + }); +}); + +// Bad +describe('.methodName', () => { + it('passes', () => { + expect(true).toEqual(true); + }); +}); +``` +#### Testing promises + +When testing Promises you should always make sure that the test is asynchronous and rejections are handled. +Your Promise chain should therefore end with a call of the `done` callback and `done.fail` in case an error occurred. + +```javascript +// Good +it('tests a promise', (done) => { + promise + .then((data) => { + expect(data).toBe(asExpected); + }) + .then(done) + .catch(done.fail); +}); + +// Good +it('tests a promise rejection', (done) => { + promise + .then(done.fail) + .catch((error) => { + expect(error).toBe(expectedError); + }) + .then(done) + .catch(done.fail); +}); + +// Bad (missing done callback) +it('tests a promise', () => { + promise + .then((data) => { + expect(data).toBe(asExpected); + }) +}); + +// Bad (missing catch) +it('tests a promise', (done) => { + promise + .then((data) => { + expect(data).toBe(asExpected); + }) + .then(done) +}); + +// Bad (use done.fail in asynchronous tests) +it('tests a promise', (done) => { + promise + .then((data) => { + expect(data).toBe(asExpected); + }) + .then(done) + .catch(fail) +}); + +// Bad (missing catch) +it('tests a promise rejection', (done) => { + promise + .catch((error) => { + expect(error).toBe(expectedError); + }) + .then(done) +}); +``` + +#### Stubbing + +For unit tests, you should stub methods that are unrelated to the current unit you are testing. +If you need to use a prototype method, instantiate an instance of the class and call it there instead of mocking the instance completely. + +For integration tests, you should stub methods that will effect the stability of the test if they +execute their original behaviour. i.e. Network requests. + +### Vue.js unit tests + +See this [section][vue-test]. + +### Running frontend tests + +`rake karma` runs the frontend-only (JavaScript) tests. +It consists of two subtasks: + +- `rake karma:fixtures` (re-)generates fixtures +- `rake karma:tests` actually executes the tests + +As long as the fixtures don't change, `rake karma:tests` (or `yarn karma`) +is sufficient (and saves you some time). + +### Live testing and focused testing + +While developing locally, it may be helpful to keep karma running so that you +can get instant feedback on as you write tests and modify code. To do this +you can start karma with `npm run karma-start`. It will compile the javascript +assets and run a server at `http://localhost:9876/` where it will automatically +run the tests on any browser which connects to it. You can enter that url on +multiple browsers at once to have it run the tests on each in parallel. + +While karma is running, any changes you make will instantly trigger a recompile +and retest of the entire test suite, so you can see instantly if you've broken +a test with your changes. You can use [jasmine focused][jasmine-focus] or +excluded tests (with `fdescribe` or `xdescribe`) to get karma to run only the +tests you want while you're working on a specific feature, but make sure to +remove these directives when you commit your code. + +## RSpec feature integration tests + +Information on setting up and running RSpec integration tests with +[Capybara] can be found in the [Testing Best Practices](best_practices.md). + +## Gotchas + +### Errors due to use of unsupported JavaScript features + +Similar errors will be thrown if you're using JavaScript features not yet +supported by the PhantomJS test runner which is used for both Karma and RSpec +tests. We polyfill some JavaScript objects for older browsers, but some +features are still unavailable: + +- Array.from +- Array.first +- Async functions +- Generators +- Array destructuring +- For..Of +- Symbol/Symbol.iterator +- Spread + +Until these are polyfilled appropriately, they should not be used. Please +update this list with additional unsupported features. + +### RSpec errors due to JavaScript + +By default RSpec unit tests will not run JavaScript in the headless browser +and will simply rely on inspecting the HTML generated by rails. + +If an integration test depends on JavaScript to run correctly, you need to make +sure the spec is configured to enable JavaScript when the tests are run. If you +don't do this you'll see vague error messages from the spec runner. + +To enable a JavaScript driver in an `rspec` test, add `:js` to the +individual spec or the context block containing multiple specs that need +JavaScript enabled: + +```ruby +# For one spec +it 'presents information about abuse report', :js do + # assertions... +end + +describe "Admin::AbuseReports", :js do + it 'presents information about abuse report' do + # assertions... + end + it 'shows buttons for adding to abuse report' do + # assertions... + end +end +``` + +### Spinach errors due to missing JavaScript + +NOTE: **Note:** Since we are discouraging the use of Spinach when writing new +feature tests, you shouldn't ever need to use this. This information is kept +available for legacy purposes only. + +In Spinach, the JavaScript driver is enabled differently. In the `*.feature` +file for the failing spec, add the `@javascript` flag above the Scenario: + +``` +@javascript +Scenario: Developer can approve merge request + Given I am a "Shop" developer + And I visit project "Shop" merge requests page + And merge request 'Bug NS-04' must be approved + And I click link "Bug NS-04" + When I click link "Approve" + Then I should see approved merge request "Bug NS-04" +``` + +[jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html +[jasmine-jquery]: https://github.com/velesin/jasmine-jquery +[karma]: http://karma-runner.github.io/ +[vue-test]:https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components +[RSpec]: https://github.com/rspec/rspec-rails#feature-specs +[Capybara]: https://github.com/teamcapybara/capybara +[Karma]: http://karma-runner.github.io/ +[Jasmine]: https://jasmine.github.io/ + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/fe_guide/img/testing_triangle.png b/doc/development/testing_guide/img/testing_triangle.png Binary files differindex 7a9a848c2ee..7a9a848c2ee 100644 --- a/doc/development/fe_guide/img/testing_triangle.png +++ b/doc/development/testing_guide/img/testing_triangle.png diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md new file mode 100644 index 00000000000..8045bbad7ba --- /dev/null +++ b/doc/development/testing_guide/index.md @@ -0,0 +1,91 @@ +# Testing standards and style guidelines + +This document describes various guidelines and best practices for automated +testing of the GitLab project. + +It is meant to be an _extension_ of the [thoughtbot testing +styleguide](https://github.com/thoughtbot/guides/tree/master/style/testing). If +this guide defines a rule that contradicts the thoughtbot guide, this guide +takes precedence. Some guidelines may be repeated verbatim to stress their +importance. + +## Overview + +GitLab is built on top of [Ruby on Rails][rails], and we're using [RSpec] for all +the backend tests, with [Capybara] for end-to-end integration testing. +On the frontend side, we're using [Karma] and [Jasmine] for JavaScript unit and +integration testing. + +Following are two great articles that everyone should read to understand what +automated testing means, and what are its principles: + +- [Five Factor Testing](https://www.devmynd.com/blog/five-factor-testing): Why do we need tests? +- [Principles of Automated Testing](http://www.lihaoyi.com/post/PrinciplesofAutomatedTesting.html): Levels of testing. Prioritize tests. Cost of tests. + +--- + +## [Testing levels](testing_levels.md) + +Learn about the different testing levels, and how to decide at what level your +changes should be tested. + +--- + +## [Testing best practices](best_practices.md) + +Everything you should know about how to write good tests: RSpec, FactoryGirl, +system tests, parameterized tests etc. + +--- + +## [Frontend testing standards and style guidelines](frontend_testing.md) + +Everything you should know about how to write good Frontend tests: Karma, +testing promises, stubbing etc. + +--- + +## [Flaky tests](flaky_tests.md) + +What are flaky tests, the different kind of flaky tests we encountered, and what +we do about them. + +--- + +## [GitLab tests in the Continuous Integration (CI) context](ci.md) + +How GitLab test suite is run in the CI context: setup, caches, artifacts, +parallelization, monitoring. + +--- + +## [Testing Rake tasks](testing_rake_tasks.md) + +Everything you should know about how to test Rake tasks. + +--- + +## Spinach (feature) tests + +GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426) +for its feature/integration tests in September 2012. + +As of March 2016, we are [trying to avoid adding new Spinach +tests](https://gitlab.com/gitlab-org/gitlab-ce/issues/14121) going forward, +opting for [RSpec feature](#features-integration) specs. + +Adding new Spinach scenarios is acceptable _only if_ the new scenario requires +no more than one new `step` definition. If more than that is required, the +test should be re-implemented using RSpec instead. + +--- + +[Return to Development documentation](../README.md) + +[^1]: /ci/yaml/README.html#dependencies + +[rails]: http://rubyonrails.org/ +[RSpec]: https://github.com/rspec/rspec-rails#feature-specs +[Capybara]: https://github.com/teamcapybara/capybara +[Karma]: http://karma-runner.github.io/ +[Jasmine]: https://jasmine.github.io/ diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md new file mode 100644 index 00000000000..1cbd4350284 --- /dev/null +++ b/doc/development/testing_guide/testing_levels.md @@ -0,0 +1,173 @@ +# Testing levels + + + +_This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._ + +## Unit tests + +Formal definition: https://en.wikipedia.org/wiki/Unit_testing + +These kind of tests ensure that a single unit of code (a method) works as +expected (given an input, it has a predictable output). These tests should be +isolated as much as possible. For example, model methods that don't do anything +with the database shouldn't need a DB record. Classes that don't need database +records should use stubs/doubles as much as possible. + +| Code path | Tests path | Testing engine | Notes | +| --------- | ---------- | -------------- | ----- | +| `app/finders/` | `spec/finders/` | RSpec | | +| `app/helpers/` | `spec/helpers/` | RSpec | | +| `app/db/{post_,}migrate/` | `spec/migrations/` | RSpec | More details at [`spec/migrations/README.md`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/migrations/README.md). | +| `app/policies/` | `spec/policies/` | RSpec | | +| `app/presenters/` | `spec/presenters/` | RSpec | | +| `app/routing/` | `spec/routing/` | RSpec | | +| `app/serializers/` | `spec/serializers/` | RSpec | | +| `app/services/` | `spec/services/` | RSpec | | +| `app/tasks/` | `spec/tasks/` | RSpec | | +| `app/uploaders/` | `spec/uploaders/` | RSpec | | +| `app/views/` | `spec/views/` | RSpec | | +| `app/workers/` | `spec/workers/` | RSpec | | +| `app/assets/javascripts/` | `spec/javascripts/` | Karma | More details in the [Frontent Testing guide](frontend_testing.md) section. | + +## Integration tests + +Formal definition: https://en.wikipedia.org/wiki/Integration_testing + +These kind of tests ensure that individual parts of the application work well together, without the overhead of the actual app environment (i.e. the browser). These tests should assert at the request/response level: status code, headers, body. They're useful to test permissions, redirections, what view is rendered etc. + +| Code path | Tests path | Testing engine | Notes | +| --------- | ---------- | -------------- | ----- | +| `app/controllers/` | `spec/controllers/` | RSpec | | +| `app/mailers/` | `spec/mailers/` | RSpec | | +| `lib/api/` | `spec/requests/api/` | RSpec | | +| `lib/ci/api/` | `spec/requests/ci/api/` | RSpec | | +| `app/assets/javascripts/` | `spec/javascripts/` | Karma | More details in the [JavaScript](#javascript) section. | + +### About controller tests + +In an ideal world, controllers should be thin. However, when this is not the +case, it's acceptable to write a system/feature test without JavaScript instead +of a controller test. The reason is that testing a fat controller usually +involves a lot of stubbing, things like: + +```ruby +controller.instance_variable_set(:@user, user) +``` + +and use methods which are deprecated in Rails 5 ([#23768]). + +[#23768]: https://gitlab.com/gitlab-org/gitlab-ce/issues/23768 + +### About Karma + +As you may have noticed, Karma is both in the Unit tests and the Integration +tests category. That's because Karma is a tool that provides an environment to +run JavaScript tests, so you can either run unit tests (e.g. test a single +JavaScript method), or integration tests (e.g. test a component that is composed +of multiple components). + +## System tests or feature tests + +Formal definition: https://en.wikipedia.org/wiki/System_testing. + +These kind of tests ensure the application works as expected from a user point +of view (aka black-box testing). These tests should test a happy path for a +given page or set of pages, and a test case should be added for any regression +that couldn't have been caught at lower levels with better tests (i.e. if a +regression is found, regression tests should be added at the lowest-level +possible). + +| Tests path | Testing engine | Notes | +| ---------- | -------------- | ----- | +| `spec/features/` | [Capybara] + [RSpec] | If your spec has the `:js` metadata, the browser driver will be [Poltergeist], otherwise it's using [RackTest]. | +| `features/` | Spinach | Spinach tests are deprecated, [you shouldn't add new Spinach tests](#spinach-feature-tests). | + +### Consider **not** writing a system test! + +If we're confident that the low-level components work well (and we should be if +we have enough Unit & Integration tests), we shouldn't need to duplicate their +thorough testing at the System test level. + +It's very easy to add tests, but a lot harder to remove or improve tests, so one +should take care of not introducing too many (slow and duplicated) specs. + +The reasons why we should follow these best practices are as follows: + +- System tests are slow to run since they spin up the entire application stack + in a headless browser, and even slower when they integrate a JS driver +- When system tests run with a JavaScript driver, the tests are run in a + different thread than the application. This means it does not share a + database connection and your test will have to commit the transactions in + order for the running application to see the data (and vice-versa). In that + case we need to truncate the database after each spec instead of simply + rolling back a transaction (the faster strategy that's in use for other kind + of tests). This is slower than transactions, however, so we want to use + truncation only when necessary. + +[Poltergeist]: https://github.com/teamcapybara/capybara#poltergeist +[RackTest]: https://github.com/teamcapybara/capybara#racktest + +## Black-box tests or end-to-end tests + +GitLab consists of [multiple pieces] such as [GitLab Shell], [GitLab Workhorse], +[Gitaly], [GitLab Pages], [GitLab Runner], and GitLab Rails. All theses pieces +are configured and packaged by [GitLab Omnibus]. + +[GitLab QA] is a tool that allows to test that all these pieces integrate well +together by building a Docker image for a given version of GitLab Rails and +running feature tests (i.e. using Capybara) against it. + +The actual test scenarios and steps are [part of GitLab Rails] so that they're +always in-sync with the codebase. + +[multiple pieces]: ../architecture.md#components +[GitLab Shell]: https://gitlab.com/gitlab-org/gitlab-shell +[GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse +[Gitaly]: https://gitlab.com/gitlab-org/gitaly +[GitLab Pages]: https://gitlab.com/gitlab-org/gitlab-pages +[GitLab Runner]: https://gitlab.com/gitlab-org/gitlab-runner +[GitLab Omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab +[GitLab QA]: https://gitlab.com/gitlab-org/gitlab-qa +[part of GitLab Rails]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa + +## How to test at the correct level? + +As many things in life, deciding what to test at each level of testing is a +trade-off: + +- Unit tests are usually cheap, and you should consider them like the basement + of your house: you need them to be confident that your code is behaving + correctly. However if you run only unit tests without integration / system + tests, you might [miss] the [big] [picture]! +- Integration tests are a bit more expensive, but don't abuse them. A system test + is often better than an integration test that is stubbing a lot of internals. +- System tests are expensive (compared to unit tests), even more if they require + a JavaScript driver. Make sure to follow the guidelines in the [Speed](#test-speed) + section. + +Another way to see it is to think about the "cost of tests", this is well +explained [in this article][tests-cost] and the basic idea is that the cost of a +test includes: + +- The time it takes to write the test +- The time it takes to run the test every time the suite runs +- The time it takes to understand the test +- The time it takes to fix the test if it breaks and the underlying code is OK +- Maybe, the time it takes to change the code to make the code testable. + +### Frontend-related tests + +There are cases where the behaviour you are testing is not worth the time spent +running the full application, for example, if you are testing styling, animation, +edge cases or small actions that don't involve the backend, +you should write an integration test using Jasmine. + +[miss]: https://twitter.com/ThePracticalDev/status/850748070698651649 +[big]: https://twitter.com/timbray/status/822470746773409794 +[picture]: https://twitter.com/withzombies/status/829716565834752000 +[tests-cost]: https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/testing_guide/testing_rake_tasks.md b/doc/development/testing_guide/testing_rake_tasks.md new file mode 100644 index 00000000000..60163f1a230 --- /dev/null +++ b/doc/development/testing_guide/testing_rake_tasks.md @@ -0,0 +1,39 @@ +# Testing Rake tasks + +To make testing Rake tasks a little easier, there is a helper that can be included +in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use +`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures +a few other things to make testing Rake tasks easier. + +At a minimum, requiring the Rake helper will redirect `stdout`, include the +runtime task helpers, and include the `RakeHelpers` Spec support module. + +The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make +executing tasks simple. See `spec/support/rake_helpers.rb` for all available +methods. + +Example: + +```ruby +require 'rake_helper' + +describe 'gitlab:shell rake tasks' do + before do + Rake.application.rake_require 'tasks/gitlab/shell' + + stub_warn_user_is_not_gitlab + end + + describe 'install task' do + it 'invokes create_hooks task' do + expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke) + + run_rake_task('gitlab:shell:install') + end + end +end +``` + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/ux_guide/animation.md b/doc/development/ux_guide/animation.md index 5dae4bcc905..d190ee1b0ff 100644 --- a/doc/development/ux_guide/animation.md +++ b/doc/development/ux_guide/animation.md @@ -39,6 +39,12 @@ When information is updating in place, a quick, subtle animation is needed. The  +### Skeleton loading + +Skeleton loading is explained in the [component section](components.html#skeleton-loading) of the UX guide. It includes a horizontally pulsating animation that shows motion as if it's growing. It's timing is a slower `linear 1s`. + + + ### Moving transitions When elements move on screen, there should be a quick animation so it is clear to users what moved where. The timing of this animation differs based on the amount of movement and change. Consider animations between `200ms` and `400ms`. @@ -51,7 +57,9 @@ View the [interactive example](http://codepen.io/awhildy/full/ALyKPE/) here.  #### Autoscroll the page + Another example of a moving transition is when you have to autoscroll the page to keep an active element visible. View the [interactive example](http://codepen.io/awhildy/full/PbxgVo/) here. -
\ No newline at end of file + + diff --git a/doc/development/ux_guide/components.md b/doc/development/ux_guide/components.md index ac7c1b6207d..fa31c496b30 100644 --- a/doc/development/ux_guide/components.md +++ b/doc/development/ux_guide/components.md @@ -42,6 +42,37 @@ By default, tooltips should be placed below the referring element. However, if t --- +## Popovers + +Popovers provide additional, useful, unique information about the referring elements and can provide one or multiple actionable elements. They inform the user of additional information within the context of their original view, but without forcing the user to act upon it like a modal. Popovers are different from tooltips, which do not provide rich markup and actionable items. A popover can contain a header section with a different background color. + +Popovers are summoned: + +* Upon hover or touch on an element + +### Usage +A popover should be used: +* When you don't want to let the user lose context, but still want to provide additional useful unique information about referring elements +* When it isn’t critical for the user to act upon the information +* When you want to give a user a summary of extended information and the option to switch context if they want to dive in deeper. + +### Styling + +A popover can contain a header section with a different background color if that improves readability and separation of content within. + + + +This example shows two sections, where each section includes an actionable element. The first section shows a summary of the content shown when clicking the "read more" link. With this information the user can decide to dive deeper or start their GitLab Enterprise Edition trial immediately. + +### Placement +By default, tooltips should be placed below the referring element. However, if there isn’t enough space in the viewport or it blocks related content, the tooltip should be moved to the side or above as needed. + + + +In this example we let the user know more about the setting they are deciding over, without loosing context. If they want to know even more they can do so, but with the expectation of opening that content in a new view. + +--- + ## Anchor links Anchor links are used for navigational actions and lone, secondary commands (such as 'Reset filters' on the Issues List) when deemed appropriate by the UX team. @@ -204,6 +235,25 @@ Cover blocks are generally used to create a heading element for a page, such as --- +## Skeleton loading + +Skeleton loading is a way to convey to the user what kind of content is currently being loaded. It's a paradigm with which content can independently and asynchronously be loaded, while still adhering to the structure and look of the completely loaded view. + +### Requirements + +* A skeleton should represent an organism in a recognisable way +* Atom elements within organisms (for reference see this article on [atomic design methodology](http://atomicdesign.bradfrost.com/chapter-2/)) may be represented in a maximum of 3 repetitions, if applicable. +* Skeletons should only be presented in grayscale using the HEX colors: `#fafafa` or `#ffffff` (except for shadows) +* Animate the grey atoms in a pulsating way to show motion, as if "loading". The pulse animation transitions colors horizontally from left to right, starting with `#f2f2f2` to `#fafafa`. + + + +### Usage + +Skeleton loading can replace any existing UI elements for the period in which they are loaded and should aim for maintaining a similar structure visually. + +--- + ## Panels > TODO: Catalog how we are currently using panels and rationalize how they relate to alerts diff --git a/doc/development/ux_guide/illustrations.md b/doc/development/ux_guide/illustrations.md new file mode 100644 index 00000000000..7e16c300921 --- /dev/null +++ b/doc/development/ux_guide/illustrations.md @@ -0,0 +1,86 @@ +# Illustrations + +The illustrations should always align with topics and goals in specific context. + +## Principles + +#### Be simple. +- For clarity, we use simple and specific elements to create our illustrations. + +#### Be optimistic. +- We are an open-minded, optimistic, and friendly team. We should reflect those values in our illustrations to connect with our brand experience. + +#### Be gentle. +- Our illustrations assist users in understanding context and guide users in the right direction. Illustrations are supportive, so they should be obvious but not aggressive. + + +## Style + +#### Shapes +- All illustrations are geometric rather than organic. +- The illustrations are made by circles, rectangles, squares, and triangles. + +<img src="img/illustrations-geometric.png" width=224px alt="Example for geometric" /> + +#### Stroke +- Standard border thickness: **4px** +- Depending on the situation, border thickness can be changed to **3px**. For example, when the illustration size is small, an illustration with 4px border thickness would look tight. In this case, the border thickness can be changed to 3px. +- We use **rounded caps** and **rounded corner**. + +| Do | Don't | +| -------- | -------- | +| <img src="img/illustrations-caps-do.png" width= 133px alt="Do: caps and corner" /> | <img src="img/illustrations-caps-don't.png" width= 133px alt="Don't: caps and corner"/> | + +#### Radius +- Standard corner radius: **10px** +- Depending on the situation, corner radius can be changed to **5px**. For example, when the illustration size is small, an illustration with 10px corner radius would be over-rounded. In this case, the corner radius can be changed to 5px. + +<img src="img/illustrations-border-radius.png" width= 464px alt="Example for border radius"/> + +#### Size +Depends on the situation, the illustration size can be the 3 types below: + +**Large** +* Use case: Empty states, error pages(e.g. 404, 403) +* For vertical layout, the illustration should not larger than **430*300 px**. +* For horizontal layout, the illustration should not larger than **430*380 px**. + +| Vertical layout | Horizontal layout | +| --------------- | ----------------- | +| <img src="img/illustration-size-large-vertical.png" /> | <img src="img/illustration-size-large-horizontal.png" /> + +**Medium** +* Use case: Banner +* The illustration should not larger than **240*160 px** +* The illustration should keep simple and clear. We recommend not including too many elements in the medium size illustration. + +<img src="img/illustration-size-medium.png" width=983px /> + +**Small** +* Use case: Graphics for explanatory text, graphics for status. +* The illustration should not larger than **160*90 px**. +* The illustration should keep simple and clear. We recommend not including too many elements in the small size illustration. + +<img src="img/illustration-size-small.png" width=983px /> + +**Illustration on mobile** +- Keep the proportions in original ratio. + +#### Colors palette + +For consistency, we recommend choosing colors from our color palette. + +| Orange | Purple | Grey | +| -------- | -------- | -------- | +| <img src="img/illustrations-color-orange.png" width= 160px alt="Orange" /> | <img src="img/illustrations-color-purple.png" width= 160px alt="Purple" /> | <img src="img/illustrations-color-grey.png" width= 160px alt="Grey" /> | +| #FC6D26 | #6B4FBB | #EEEEEE | + +#### Don't +- Don't include the typography in the illustration. +- Don't include tanuki in the illustration. If necessary, we recommend having tanuki monochromatic. + +--- + +| Orange | Purple | +| -------- | -------- | +| <img src="img/illustrations-palette-oragne.png" width= 160px alt="Palette - Orange" /> | <img src="img/illustrations-palette-purple.png" width= 160px alt="Palette - Purple" /> | diff --git a/doc/development/ux_guide/img/illustration-size-large-horizontal.png b/doc/development/ux_guide/img/illustration-size-large-horizontal.png Binary files differnew file mode 100644 index 00000000000..8aa835adccc --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-large-horizontal.png diff --git a/doc/development/ux_guide/img/illustration-size-large-vertical.png b/doc/development/ux_guide/img/illustration-size-large-vertical.png Binary files differnew file mode 100644 index 00000000000..813b6a065e5 --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-large-vertical.png diff --git a/doc/development/ux_guide/img/illustration-size-medium.png b/doc/development/ux_guide/img/illustration-size-medium.png Binary files differnew file mode 100644 index 00000000000..55cfe1dcb91 --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-medium.png diff --git a/doc/development/ux_guide/img/illustration-size-small.png b/doc/development/ux_guide/img/illustration-size-small.png Binary files differnew file mode 100644 index 00000000000..0124f58f48e --- /dev/null +++ b/doc/development/ux_guide/img/illustration-size-small.png diff --git a/doc/development/ux_guide/img/illustrations-border-radius.png b/doc/development/ux_guide/img/illustrations-border-radius.png Binary files differnew file mode 100644 index 00000000000..4e2fef5c7f5 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-border-radius.png diff --git a/doc/development/ux_guide/img/illustrations-caps-do.png b/doc/development/ux_guide/img/illustrations-caps-do.png Binary files differnew file mode 100644 index 00000000000..7a2c74382f6 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-caps-do.png diff --git a/doc/development/ux_guide/img/illustrations-caps-don't.png b/doc/development/ux_guide/img/illustrations-caps-don't.png Binary files differnew file mode 100644 index 00000000000..848f72dbe30 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-caps-don't.png diff --git a/doc/development/ux_guide/img/illustrations-color-grey.png b/doc/development/ux_guide/img/illustrations-color-grey.png Binary files differnew file mode 100644 index 00000000000..63855026c2b --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-grey.png diff --git a/doc/development/ux_guide/img/illustrations-color-orange.png b/doc/development/ux_guide/img/illustrations-color-orange.png Binary files differnew file mode 100644 index 00000000000..96765c8c28c --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-orange.png diff --git a/doc/development/ux_guide/img/illustrations-color-purple.png b/doc/development/ux_guide/img/illustrations-color-purple.png Binary files differnew file mode 100644 index 00000000000..745d2c853ba --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-color-purple.png diff --git a/doc/development/ux_guide/img/illustrations-geometric.png b/doc/development/ux_guide/img/illustrations-geometric.png Binary files differnew file mode 100644 index 00000000000..33f05547bac --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-geometric.png diff --git a/doc/development/ux_guide/img/illustrations-palette-oragne.png b/doc/development/ux_guide/img/illustrations-palette-oragne.png Binary files differnew file mode 100644 index 00000000000..15f35912646 --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-palette-oragne.png diff --git a/doc/development/ux_guide/img/illustrations-palette-purple.png b/doc/development/ux_guide/img/illustrations-palette-purple.png Binary files differnew file mode 100644 index 00000000000..e0f5839705e --- /dev/null +++ b/doc/development/ux_guide/img/illustrations-palette-purple.png diff --git a/doc/development/ux_guide/img/popover-placement-above.png b/doc/development/ux_guide/img/popover-placement-above.png Binary files differnew file mode 100644 index 00000000000..1aa044bfc9c --- /dev/null +++ b/doc/development/ux_guide/img/popover-placement-above.png diff --git a/doc/development/ux_guide/img/popover-placement-below.png b/doc/development/ux_guide/img/popover-placement-below.png Binary files differnew file mode 100644 index 00000000000..2d6ab8a1618 --- /dev/null +++ b/doc/development/ux_guide/img/popover-placement-below.png diff --git a/doc/development/ux_guide/img/skeleton-loading.gif b/doc/development/ux_guide/img/skeleton-loading.gif Binary files differnew file mode 100644 index 00000000000..5877139171d --- /dev/null +++ b/doc/development/ux_guide/img/skeleton-loading.gif diff --git a/doc/development/ux_guide/index.md b/doc/development/ux_guide/index.md index 8a849f239dc..42bcf234e12 100644 --- a/doc/development/ux_guide/index.md +++ b/doc/development/ux_guide/index.md @@ -21,6 +21,11 @@ Guidance on the timing, curving and motion for GitLab. --- +### [Illustrations](illustrations.md) +Guidelines for principals and styles related to illustrations for GitLab. + +--- + ### [Copy](copy.md) Conventions on text and messaging within labels, buttons, and other components. diff --git a/doc/development/ux_guide/resources.md b/doc/development/ux_guide/resources.md index 2f760c94414..db57258237f 100644 --- a/doc/development/ux_guide/resources.md +++ b/doc/development/ux_guide/resources.md @@ -10,4 +10,8 @@ you can use during GitLab development. ## Design repository All design files are stored in the [gitlab-design](https://gitlab.com/gitlab-org/gitlab-design) -repository and maintained by GitLab UX designers.
\ No newline at end of file +repository and maintained by GitLab UX designers. + +## [brand.ai](https://brand.ai/git-lab/primary-brand) + +We are in the process of capturing our UI paradigms on brand.ai, see https://brand.ai/git-lab/primary-brand diff --git a/doc/development/ux_guide/users.md b/doc/development/ux_guide/users.md index cbd7c17de41..fce882a45f1 100644 --- a/doc/development/ux_guide/users.md +++ b/doc/development/ux_guide/users.md @@ -1,4 +1,5 @@ -## UX Personas +# UX Personas + * [Nazim Ramesh](#nazim-ramesh) - Small to medium size organisations using GitLab CE * [James Mackey](#james-mackey) @@ -7,16 +8,16 @@ * [Karolina Plaskaty](#karolina-plaskaty) - Using GitLab.com for personal/hobby projects - Would like to use GitLab at work - - Working for a medium to large size organisation + - Working for a medium to large size organisation -<hr> +--- -### Nazim Ramesh +## Nazim Ramesh - Small to medium size organisations using GitLab CE <img src="img/nazim-ramesh.png" width="300px"> -#### Demographics +### Demographics - **Age**<br>32 years old - **Location**<br>Germany @@ -26,7 +27,7 @@ - **Frequently used programming languages**<br>JavaScript, SQL, PHP - **Hobbies / interests**<br>Functional programming, open source, gaming, web development and web security. -#### Motivations +### Motivations Nazim works for a software development company which currently hires around 80 people. When Nazim first joined the company, the engineering team were using Subversion (SVN) as their primary form of source control. However, Nazim felt SVN was not flexible enough to work with many feature branches and noticed that developers with less experience of source control struggled with the central-repository nature of SVN. Armed with a wishlist of features, Nazim began comparing source control tools. A search for “self-hosted Git server repository management” returned GitLab. In his own words, Nazim explains why he wanted the engineering team to start using GitLab: > @@ -39,48 +40,48 @@ In his role as a full-stack web developer, Nazim could recommend products that h “The biggest challenge...why should we change anything at all from the status quo? We needed to switch from SVN to Git. They knew they needed to learn Git and a Git workflow...using Git was scary to my colleagues...they thought it was more complex than SVN to use.” > -Undeterred, Nazim decided to migrate a couple of projects across to GitLab. +Undeterred, Nazim decided to migrate a couple of projects across to GitLab. > “Old SVN users couldn’t see the benefits of Git at first. It took a month or two to convince them.” > -Slowly, by showing his colleagues how easy it was to use Git, the majority of the team’s projects were migrated to GitLab. +Slowly, by showing his colleagues how easy it was to use Git, the majority of the team’s projects were migrated to GitLab. -The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company’s decision to move to GitLab. +The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company’s decision to move to GitLab. -#### Frustrations -##### Adoption to GitLab has been slow +### Frustrations +#### Adoption to GitLab has been slow Not only has the engineering team had to get to grips with Git, they’ve also had to adapt to using GitLab. Due to lack of training and existing skills in other tools, the full feature set of GitLab CE is not being utilised. Nazim sold GitLab to his manager as an ‘all in one’ tool which would replace multiple tools used within the company, thus saving costs. Nazim hasn’t had the time to integrate the legacy tools to GitLab and he’s struggling to convince his peers to change their habits. -##### Missing Features +#### Missing Features Nazim’s company want GitLab to be able to do everything. There isn’t a large budget for software, so they’re selective about what tools are implemented. It needs to add real value to the company. In order for GitLab to be widely adopted and to meet the requirements of different roles within the company, it needs a host of features. When an individual within Nazim’s company wants to know if GitLab has a specific feature or does a particular thing, Nazim is the person to ask. He becomes the point of contact to investigate, build or sometimes just raise the feature request. Nazim gets frustrated when GitLab isn’t able to do what he or his colleagues need it to do. -##### Regressions and bugs +#### Regressions and bugs Nazim often has to calm down his colleagues, when a release contains regressions or new bugs. As he puts it “every new version adds something awesome, but breaks something”. He feels that “old issues for "minor" annoyances get quickly buried in the mass of open issues and linger for a very long time. More generally, I have the feeling that GitLab focus on adding new functionalities, but overlook a bunch of annoying minor regressions or introduced bugs.” Due to limited resource and expertise within the team, not only is it difficult to remain up-to-date with the frequent release cycle, it’s also counterproductive to fix workflows every month. -##### Uses too much RAM and CPU +#### Uses too much RAM and CPU > “Memory usages mean that if we host it from a cloud based host like AWS, we spend almost as much on the instance as what we would pay GitHub” > -##### UI/UX +#### UI/UX GitLab’s interface initially attracted Nazim when he was comparing version control software. He thought it would help his less technical colleagues to adapt to using Git and perhaps, GitLab could be rolled out to other areas of the business, beyond engineering. However, using GitLab’s interface daily has left him frustrated at the lack of personalisation / control over his user experience. He’s also regularly lost in a maze of navigation. Whilst he acknowledges that GitLab listens to its users and that the interface is improving, he becomes annoyed when the changes are too progressive. “Too frequent UI changes. Most of them tend to turn out great after a few cycles of fixes, but the frequency is still far too high for me to feel comfortable to always stay on the current release.” -#### Goals +### Goals * To convince his colleagues to fully adopt GitLab CE, thus improving workflow and collaboration. * To use a feature rich version control platform that covers all stages of the development lifecycle, in order to reduce dependencies on other tools. * To use an intuitive and stable product, so he can spend more time on his core job responsibilities and less time bug-fixing, guiding colleagues, etc. -<hr> +--- -### James Mackey +## James Mackey - Medium to large size organisations using CE or EE - Small organisations using EE <img src="img/james-mackey.png" width="300px"> -#### Demographics +### Demographics - **Age**<br>36 years old - **Location**<br>US @@ -90,7 +91,7 @@ GitLab’s interface initially attracted Nazim when he was comparing version con - **Frequently used programming languages**<br>JavaScript, SQL, Node.js, Java, PHP, Python - **Hobbies / interests**<br>DevOps, open source, web development, science, automation and electronics. -#### Motivations +### Motivations James works for a research company which currently hires around 800 staff. He began using GitLab.com back in 2013 for his own open source, hobby projects and loved “the simplicity of installation, administration and use”. After using GitLab for over a year, he began to wonder about using it at work. James explains: > @@ -99,7 +100,7 @@ James works for a research company which currently hires around 800 staff. He be James and his colleagues also reviewed competitor products including GitHub Enterprise, but they found it “less innovative and with considerable costs...GitLab had the features we wanted at a much lower cost per head than GitHub”. -The company James works for provides employees with a discretionary budget to spend how they want on software, so James and his team decided to upgrade to EE. +The company James works for provides employees with a discretionary budget to spend how they want on software, so James and his team decided to upgrade to EE. James feels partially responsible for his organisation’s decision to start using GitLab. @@ -107,33 +108,33 @@ James feels partially responsible for his organisation’s decision to start usi “It's still up to the teams themselves [to decide] which tools to use. We just had a great experience moving our daily development to GitLab, so other teams have followed the path or are thinking about switching.” > -#### Frustrations -##### Third Party Integration +### Frustrations +#### Third Party Integration Some of GitLab EE’s features are too basic, in particular, issues boards which do not have the level of reporting that James and his team need. Subsequently, they still need to use GitLab EE in conjunction with other tools, such as JIRA. Whilst James feels it isn’t essential for GitLab to meet all his needs (his company are happy for him to use, and pay for, multiple tools), he sometimes isn’t sure what is/isn’t possible with plugins and what level of custom development he and his team will need to do. -##### UX/UI +#### UX/UI James and his team use CI quite heavily for several projects. Whilst they’ve welcomed improvements to the builds and pipelines interface, they still have some difficulty following build process on the different tabs under Pipelines. Some confusion has arisen from not knowing where to find different pieces of information or how to get to the next stages logs from the current stage’s log output screen. They feel more intuitive linking and flow may alleviate the problem. Generally, they feel GitLab’s navigation needs to reviewed and optimised. -##### Permissions +#### Permissions > “There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for Gitlab CI/build pipelines.” > -#### Goals +### Goals * To be able to integrate third party tools easily with GitLab EE and to create custom integrations and patches where needed. * To use GitLab EE primarily for code hosting, merge requests, continuous integration and issue management. James and his team want to be able to understand and use these particular features easily. * To able to share one instance of GitLab EE with multiple teams across the business. Advanced user management, the ability to separate permissions on different parts of the source code, etc are important to James. -<hr> +--- -### Karolina Plaskaty +## Karolina Plaskaty - Using GitLab.com for personal/hobby projects - Would like to use GitLab at work -- Working for a medium to large size organisation +- Working for a medium to large size organisation <img src="img/karolina-plaskaty.png" width="300px"> -#### Demographics +### Demographics - **Age**<br>26 years old - **Location**<br>UK @@ -143,22 +144,22 @@ James and his team use CI quite heavily for several projects. Whilst they’ve w - **Frequently used programming languages**<br>JavaScript and SQL - **Hobbies / interests**<br>Web development, mobile development, UX, open source, gaming and travel. -#### Motivations +### Motivations Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab’s values and the fact that it isn’t a “behemoth of a company”. She explains that “displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose Gitlab personally and to recommend it at work.” She’s also an avid reader of GitLab’s blog. Karolina works for a software development company which currently hires around 500 people. Karolina would love to use GitLab at work but the company has used GitHub Enterprise for a number of years. She describes management at her company as “old fashioned” and explains that it’s “less of a technical issue and more of a cultural issue” to convince upper management to move to GitLab. Karolina is also relatively new to the company so she’s apprehensive about pushing too hard to change version control platforms. -#### Frustrations -##### Unable to use GitLab at work +### Frustrations +#### Unable to use GitLab at work Karolina wants to use GitLab at work but isn’t sure how to approach the subject with management. In her current role, she doesn’t feel that she has the authority to request GitLab. -##### Performance +#### Performance GitLab.com is frequently slow and unavailable. Karolina has also heard that GitLab is a “memory hog” which has deterred her from running GitLab on her own machine for just hobby / personal projects. -##### UX/UI +#### UX/UI Karolina has an interest in UX and therefore has strong opinions about how GitLab should look and feel. She feels the interface is cluttered, “it has too many links/buttons” and the navigation “feels a bit weird sometimes. I get lost if I don’t pay attention.” As Karolina also enjoys contributing to open-source projects, it’s important to her that GitLab is well designed for public repositories, she doesn’t feel that GitLab currently achieves this. -#### Goals +### Goals * To develop her programming experience and to learn from other developers. * To contribute to both her own and other open source projects. -* To use a fast and intuitive version control platform.
\ No newline at end of file +* To use a fast and intuitive version control platform. diff --git a/doc/development/verifying_database_capabilities.md b/doc/development/verifying_database_capabilities.md index cc6d62957e3..ffdeff47d4a 100644 --- a/doc/development/verifying_database_capabilities.md +++ b/doc/development/verifying_database_capabilities.md @@ -24,3 +24,15 @@ else run_query end ``` + +# Read-only database + +The database can be used in read-only mode. In this case we have to +make sure all GET requests don't attempt any write operations to the +database. If one of those requests wants to write to the database, it needs +to be wrapped in a `Gitlab::Database.read_only?` or `Gitlab::Database.read_write?` +guard, to make sure it doesn't for read-only databases. + +We have a Rails Middleware that filters any potentially writing +operations (the CUD operations of CRUD) and prevent the user from trying +to update the database and getting a 500 error (see `Gitlab::Middleware::ReadOnly`). diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md index b1eb020a592..68ba3dd2da3 100644 --- a/doc/development/writing_documentation.md +++ b/doc/development/writing_documentation.md @@ -1,6 +1,6 @@ # Writing documentation - - **General Documentation**: written by the developers responsible by creating features. Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. + - **General Documentation**: written by the [developers responsible by creating features](#contributing-to-docs). Should be submitted in the same merge request containing code. Feature proposals (by GitLab contributors) should also be accompanied by its respective documentation. They can be later improved by PMs and Technical Writers. - **Technical Articles**: written by any [GitLab Team](https://about.gitlab.com/team/) member, GitLab contributors, or [Community Writers](https://about.gitlab.com/handbook/product/technical-writing/community-writers/). - **Indexes per topic**: initially prepared by the Technical Writing Team, and kept up-to-date by developers and PMs in the same merge request containing code. They gather all resources for that topic in a single page (user and admin documentation, articles, and third-party docs). @@ -69,6 +69,51 @@ Use the [writing method](https://about.gitlab.com/handbook/product/technical-wri All the docs follow the same [styleguide](doc_styleguide.md). +### Contributing to docs + +Whenever a feature is changed, updated, introduced, or deprecated, the merge +request introducing these changes must be accompanied by the documentation +(either updating existing ones or creating new ones). This is also valid when +changes are introduced to the UI. + +The one resposible for writing the first piece of documentation is the developer who +wrote the code. It's the job of the Product Manager to ensure all features are +shipped with its docs, whether is a small or big change. At the pace GitLab evolves, +this is the only way to keep the docs up-to-date. If you have any questions about it, +please ask a Technical Writer. Otherwise, when your content is ready, assign one of +them to review it for you. + +We use the [monthly release blog post](https://about.gitlab.com/handbook/marketing/blog/release-posts/#monthly-releases) as a changelog checklist to ensure everything +is documented. + +### Feature overview and use cases + +Every major feature (regardless if present in GitLab Community or Enterprise editions) +should present, at the beginning of the document, two main sections: **overview** and +**use cases**. Every GitLab EE-only feature should also contain these sections. + +**Overview**: at the name suggests, the goal here is to provide an overview of the feature. +Describe what is it, what it does, why it is important/cool/nice-to-have, +what problem it solves, and what you can do with this feature that you couldn't +do before. + +**Use cases**: provide at least two, ideally three, use cases for every major feature. +You should answer this question: what can you do with this feature/change? Use cases +are examples of how this feauture or change can be used in real life. + +Examples: +- CE and EE: [Issues](../user/project/issues/index.md#use-cases) +- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview) +- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview) +- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview) + +Note that if you don't have anything to add between the doc title (`<h1>`) and +the header `## Overview`, you can omit the header, but keep the content of the +overview there. + +> **Overview** and **use cases** are required to **every** Enterprise Edition feature, +and for every **major** feature present in Community Edition. + ### Markdown Currently GitLab docs use Redcarpet as [markdown](../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 3d893ba53dd..4e15f7cfd49 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab basics Step-by-step guides on the basics of working with Git and GitLab. diff --git a/doc/gitlab-basics/add-merge-request.md b/doc/gitlab-basics/add-merge-request.md index bf01fe51dc3..5cc014419ad 100644 --- a/doc/gitlab-basics/add-merge-request.md +++ b/doc/gitlab-basics/add-merge-request.md @@ -3,31 +3,28 @@ Merge requests are useful to integrate separate changes that you've made to a project, on different branches. This is a brief guide on how to create a merge request. For more information, check the -[merge requests documentation](../user/project/merge_requests.md). +[merge requests documentation](../user/project/merge_requests/index.md). --- 1. Before you start, you should have already [created a branch](create-branch.md) and [pushed your changes](basic-git-commands.md) to GitLab. - -1. You can then go to the project where you'd like to merge your changes and - click on the **Merge requests** tab. - -  - +1. Go to the project where you'd like to merge your changes and click on the + **Merge requests** tab. 1. Click on **New merge request** on the right side of the screen. - -  - -1. Select a source branch and click on the **Compare branches and continue** button. +1. From there on, you have the option to select the source branch and the target + branch you'd like to compare to. The default target project is the upstream + repository, but you can choose to compare across any of its forks.  +1. When ready, click on the **Compare branches and continue** button. 1. At a minimum, add a title and a description to your merge request. Optionally, select a user to review your merge request and to accept or close it. You may also select a milestone and labels.  -1. When ready, click on the **Submit merge request** button. Your merge request - will be ready to be approved and published. +1. When ready, click on the **Submit merge request** button. + +Your merge request will be ready to be approved and merged. diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md index 67ef189fee9..e18711f3392 100644 --- a/doc/gitlab-basics/create-project.md +++ b/doc/gitlab-basics/create-project.md @@ -17,7 +17,7 @@ [Project Templates](https://gitlab.com/gitlab-org/project-templates): this will kickstart your repository code and CI automatically. Otherwise, if you have a project in a different repository, you can [import it] by - clicking an **Import project from** button provided this is enabled in + clicking on the **Import project** tab, provided this is enabled in your GitLab instance. Ask your administrator if not. 1. Provide the following information: diff --git a/doc/gitlab-basics/img/create_new_project_info.png b/doc/gitlab-basics/img/create_new_project_info.png Binary files differindex ef8753e224b..ce4f7d1204b 100644 --- a/doc/gitlab-basics/img/create_new_project_info.png +++ b/doc/gitlab-basics/img/create_new_project_info.png diff --git a/doc/gitlab-basics/img/merge_request_new.png b/doc/gitlab-basics/img/merge_request_new.png Binary files differdeleted file mode 100644 index 6fcd7bebada..00000000000 --- a/doc/gitlab-basics/img/merge_request_new.png +++ /dev/null diff --git a/doc/gitlab-basics/img/merge_request_select_branch.png b/doc/gitlab-basics/img/merge_request_select_branch.png Binary files differindex 9f6b93943a9..57ea0e65f34 100644 --- a/doc/gitlab-basics/img/merge_request_select_branch.png +++ b/doc/gitlab-basics/img/merge_request_select_branch.png diff --git a/doc/gitlab-basics/img/project_navbar.png b/doc/gitlab-basics/img/project_navbar.png Binary files differdeleted file mode 100644 index be6f38ede32..00000000000 --- a/doc/gitlab-basics/img/project_navbar.png +++ /dev/null diff --git a/doc/install/README.md b/doc/install/README.md index 656f8720361..540cb0d3f38 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Installation GitLab can be installed via various ways. Check the [installation methods][methods] diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index 5c128f54a76..f9ba1508705 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -1,11 +1,12 @@ # Database MySQL >**Note:** -We do not recommend using MySQL due to various issues. For example, case +- We do not recommend using MySQL due to various issues. For example, case [(in)sensitivity](https://dev.mysql.com/doc/refman/5.0/en/case-sensitivity.html) and [problems](https://bugs.mysql.com/bug.php?id=65830) that [suggested](https://bugs.mysql.com/bug.php?id=50909) [fixes](https://bugs.mysql.com/bug.php?id=65830) [have](https://bugs.mysql.com/bug.php?id=63164). +- We recommend using MySQL version 5.6 or later. Please see the following [issue][ce-38152]. ## Initial database setup @@ -13,7 +14,7 @@ and [problems](https://bugs.mysql.com/bug.php?id=65830) that # Install the database packages sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev -# Ensure you have MySQL version 5.5.14 or later +# Ensure you have MySQL version 5.6 or later mysql --version # Pick a MySQL root password (can be anything), type it and press enter @@ -293,3 +294,4 @@ Details can be found in the [PostgreSQL][postgres-text-type] and [postgres-text-type]: http://www.postgresql.org/docs/9.2/static/datatype-character.html [mysql-text-types]: http://dev.mysql.com/doc/refman/5.7/en/string-type-overview.html +[ce-38152]: https://gitlab.com/gitlab-org/gitlab-ce/issues/38152 diff --git a/doc/install/installation.md b/doc/install/installation.md index 200cd94f43c..2a004152d5e 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -80,7 +80,7 @@ Make sure you have the right version of Git installed # Install Git sudo apt-get install -y git-core - # Make sure Git is version 2.13.0 or higher + # Make sure Git is version 2.13.6 or higher git --version Is the system packaged Git too old? Remove it and compile from source. @@ -121,7 +121,7 @@ The use of Ruby version managers such as [RVM], [rbenv] or [chruby] with GitLab in production, frequently leads to hard to diagnose problems. For example, GitLab Shell is called from OpenSSH, and having a version manager can prevent pushing and pulling over SSH. Version managers are not supported and we strongly -advise everyone to follow the instructions below to use a system Ruby. +advise everyone to follow the instructions below to use a system Ruby. Linux distributions generally have older versions of Ruby available, so these instructions are designed to install Ruby from the official source code. @@ -133,9 +133,9 @@ Remove the old Ruby 1.8 if present: Download Ruby and compile it: mkdir /tmp/ruby && cd /tmp/ruby - curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.3.tar.gz - echo '1014ee699071aa2ddd501907d18cbe15399c997d ruby-2.3.3.tar.gz' | shasum -c - && tar xzf ruby-2.3.3.tar.gz - cd ruby-2.3.3 + curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.5.tar.gz + echo '3247e217d6745c27ef23bdc77b6abdb4b57a118f ruby-2.3.5.tar.gz' | shasum -c - && tar xzf ruby-2.3.5.tar.gz + cd ruby-2.3.5 ./configure --disable-install-rdoc make sudo make install @@ -299,9 +299,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-0-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 10-1-stable gitlab -**Note:** You can change `10-0-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `10-1-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It diff --git a/doc/install/kubernetes/gitlab_chart.md b/doc/install/kubernetes/gitlab_chart.md index 177124c8291..fa564d83785 100644 --- a/doc/install/kubernetes/gitlab_chart.md +++ b/doc/install/kubernetes/gitlab_chart.md @@ -1,9 +1,16 @@ # GitLab Helm Chart > **Note**: -* This chart will be replaced by the [gitlab-omnibus](gitlab_omnibus.md) chart, once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). +* This chart is deprecated, and is being replaced by the [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). For more information on available charts, please see our [overview](index.md#chart-overview). * These charts have been tested on Google Container Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). -The `gitlab` Helm chart deploys just GitLab into your Kubernetes cluster, and offers extensive configuration options. This chart requires advanced knowledge of Kubernetes to successfully use. For most deployments we **strongly recommended** the [gitlab-omnibus](gitlab_omnibus.md) chart, which will replace this chart once it supports [additional configuration options](https://gitlab.com/charts/charts.gitlab.io/issues/68). Due to the difficulty in supporting upgrades to the `omnibus-gitlab` chart, migrating will require exporting data out of this instance and importing it into the new deployment. + +For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). + +## Introduction + +The `gitlab` Helm chart deploys just GitLab into your Kubernetes cluster, and offers extensive configuration options. This chart requires advanced knowledge of Kubernetes to successfully use. We **strongly recommend** the [gitlab-omnibus](gitlab_omnibus.md) chart. + +This chart is deprecated, and will be replaced by the [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). Due to the difficulty in supporting upgrades, migrating will require exporting data out of this instance and importing it into the new deployment. This chart includes the following: @@ -15,8 +22,6 @@ This chart includes the following: - Optional PostgreSQL deployment using the [PostgreSQL Chart](https://github.com/kubernetes/charts/tree/master/stable/postgresql) (defaults to enabled) - Optional Ingress (defaults to disabled) -For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). - ## Prerequisites - _At least_ 3 GB of RAM available on your cluster. 41GB of storage and 2 CPU are also required. diff --git a/doc/install/kubernetes/gitlab_omnibus.md b/doc/install/kubernetes/gitlab_omnibus.md index 19e2a257c94..6659c3cf7b2 100644 --- a/doc/install/kubernetes/gitlab_omnibus.md +++ b/doc/install/kubernetes/gitlab_omnibus.md @@ -1,7 +1,6 @@ # GitLab-Omnibus Helm Chart > **Note:** -* This Helm chart is in beta, while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being worked on. -* GitLab is working on a [cloud native set of Charts](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md) which will eventually replace these. +* This Helm chart is in beta, and will be deprecated by the [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). * These charts have been tested on Google Container Engine and Azure Container Service. Other Kubernetes installations may work as well, if not please [open an issue](https://gitlab.com/charts/charts.gitlab.io/issues). This work is based partially on: https://github.com/lwolf/kubernetes-gitlab/. GitLab would like to thank Sergey Nuzhdin for his work. @@ -12,6 +11,8 @@ For more information on available GitLab Helm Charts, please see our [overview]( This chart provides an easy way to get started with GitLab, provisioning an installation with nearly all functionality enabled. SSL is automatically provisioned via [Let's Encrypt](https://letsencrypt.org/). +This Helm chart is in beta, and will be deprecated by the [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md) once available. Due to the difficulty in supporting upgrades, migrating will require exporting data out of this instance and importing it into the new deployment. + The deployment includes: - A [GitLab Omnibus](https://docs.gitlab.com/omnibus/) Pod, including Mattermost, Container Registry, and Prometheus @@ -23,9 +24,8 @@ The deployment includes: ### Limitations -* This chart is suited for small to medium size deployments, because [High Availability](https://docs.gitlab.com/ee/administration/high_availability/) and [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html) will not be supported. -* It is in beta. Additional features to support production deployments, like backups, are [in development](https://gitlab.com/charts/charts.gitlab.io/issues/68). Once completed, this chart will be generally available. -* A new generation of [cloud native charts](index.md#upcoming-cloud-native-helm-charts) is in development, and will eventually deprecate these. Due to the difficulty in supporting upgrades to the new architecture, migrating will require exporting data out of this instance and importing it into the new deployment. We do not expect these to be production ready before the second half of 2018. +* This chart is in beta, and suited for small to medium size deployments. [High Availability](https://docs.gitlab.com/ee/administration/high_availability/) and [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html) are not supported. +* A new generation [cloud native GitLab chart](index.md#cloud-native-gitlab-chart) is in development, and will deprecate this chart. Due to the difficulty in supporting upgrades to the new architecture, migrating will require exporting data out of this instance and importing it into the new deployment. We plan to release the new chart in beta by the end of 2017. For more information on available GitLab Helm Charts, please see our [overview](index.md#chart-overview). @@ -148,13 +148,36 @@ helm install --name gitlab --set baseDomain=gitlab.io,baseIP=1.1.1.1,gitlab=ee,g ## Updating GitLab using the Helm Chart +>**Note**: If you are upgrading from a previous version to 0.1.35 or above, you will need to change the access mode values for GitLab's storage. To do this, set the following in `values.yaml` or on the CLI: +``` +gitlabDataAccessMode=ReadWriteMany +gitlabRegistryAccessMode=ReadWriteMany +gitlabConfigAccessMode=ReadWriteMany +``` + Once your GitLab Chart is installed, configuration changes and chart updates -should we done using `helm upgrade`: +should be done using `helm upgrade`: ```bash helm upgrade -f values.yaml gitlab gitlab/gitlab-omnibus ``` +## Upgrading from CE to EE using the Helm Chart + +If you have installed the Community Edition using this chart, upgrading to Enterprise Edition is easy. + +If you are using a `values.yaml` file to specify the configuration options, edit the file and set `gitlab=ee`. If you would like to run a specific version of GitLab EE, set `gitlabEEImage` to be the desired GitLab [docker image](https://hub.docker.com/r/gitlab/gitlab-ee/tags/). Then you can use `helm upgrade` to update your GitLab instance to EE: + +```bash +helm upgrade -f values.yaml gitlab gitlab/gitlab-omnibus +``` + +You can also upgrade and specify these options via the command line: + +```bash +helm upgrade gitlab --set gitlab=ee,gitlabEEImage=gitlab/gitlab-ee:9.5.5-ee.0 gitlab/gitlab-omnibus +``` + ## Uninstalling GitLab using the Helm Chart To uninstall the GitLab Chart, run the following: @@ -163,5 +186,13 @@ To uninstall the GitLab Chart, run the following: helm delete gitlab ``` +## Troubleshooting + +### Storage errors when updating `gitlab-omnibus` versions prior to 0.1.35 + +Users upgrading `gitlab-omnibus` from a version prior to 0.1.35, may see an error like: `Error: UPGRADE FAILED: PersistentVolumeClaim "gitlab-gitlab-config-storage" is invalid: spec: Forbidden: field is immutable after creation`. + +This is due to a change in the access mode for GitLab storage in version 0.1.35. To successfully upgrade, the access mode flags must be set to `ReadWriteMany` as detailed in the [update section](#updating-gitlab-using-the-helm-chart). + [kube-srv]: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services---service-types [storageclass]: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#storageclasses diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index 467d5b92e0c..dd350820c18 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -9,25 +9,25 @@ should be deployed, upgraded, and configured. ## Chart Overview -* **[GitLab-Omnibus](#gitlab-omnibus-chart-recommended)**: The best way to run GitLab on Kubernetes today. It is suited for small to medium deployments, and is in beta while support for backups and other features are added. -* **[Upcoming Cloud Native Charts](#upcoming-cloud-native-helm-charts)**: The next generation of charts, currently in development. Will support large deployments, with horizontal scaling of individual GitLab components. +* **[GitLab-Omnibus](gitlab_omnibus.md)**: The best way to run GitLab on Kubernetes today, suited for small to medium deployments. The chart is in beta and will be deprecated by the [cloud native GitLab chart](#cloud-native-gitlab-chart). +* **[Cloud Native GitLab Chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md)**: The next generation GitLab chart, currently in development. Will support large deployments with horizontal scaling of individual GitLab components. * Other Charts - * [GitLab Runner Chart](#gitlab-runner-chart): For deploying just the GitLab Runner. - * [Advanced GitLab Installation](#advanced-gitlab-installation): Provides additional deployment options, but provides less functionality out-of-the-box. It's beta, no longer actively developed, and will be deprecated by [gitlab-omnibus](#gitlab-omnibus-chart-recommended) once it supports these options. - * [Community Contributed Charts](#community-contributed-charts): Community contributed charts, deprecated by the official GitLab charts. + * [GitLab Runner Chart](gitlab_runner_chart.md): For deploying just the GitLab Runner. + * [Advanced GitLab Installation](gitlab_chart.md): Deprecated, being replaced by the [cloud native GitLab chart](#cloud-native-gitlab-chart). Provides additional deployment options, but provides less functionality out-of-the-box. + * [Community Contributed Charts](#community-contributed-charts): Community contributed charts, deprecated by the official GitLab chart. ## GitLab-Omnibus Chart (Recommended) > **Note**: This chart is in beta while [additional features](https://gitlab.com/charts/charts.gitlab.io/issues/68) are being added. This chart is the best available way to operate GitLab on Kubernetes. It deploys and configures nearly all features of GitLab, including: a [Runner](https://docs.gitlab.com/runner/), [Container Registry](../../user/project/container_registry.html#gitlab-container-registry), [Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/), [automatic SSL](https://github.com/kubernetes/charts/tree/master/stable/kube-lego), and a [load balancer](https://github.com/kubernetes/ingress/tree/master/controllers/nginx). It is based on our [GitLab Omnibus Docker Images](https://docs.gitlab.com/omnibus/docker/README.html). -Once the [cloud native charts](#upcoming-cloud-native-helm-charts) are ready for production use, this chart will be deprecated. Due to the difficulty in supporting upgrades to the new architecture, migrating will require exporting data out of this instance and importing it into the new deployment. +Once the [cloud native GitLab chart](#cloud-native-gitlab-chart) is ready for production use, this chart will be deprecated. Due to the difficulty in supporting upgrades to the new architecture, migrating will require exporting data out of this instance and importing it into the new deployment. -Learn more about the [gitlab-omnibus chart.](gitlab_omnibus.md) +Learn more about the [gitlab-omnibus chart](gitlab_omnibus.md). -## Upcoming Cloud Native Charts +## Cloud Native GitLab Chart -GitLab is working towards building a [cloud native deployment method](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). A key part of this effort is to isolate each service into its [own Docker container and Helm chart](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420), rather than utilizing the all-in-one container image of the [current charts](#official-gitlab-helm-charts-recommended). +GitLab is working towards building a [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). A key part of this effort is to isolate each service into its [own Docker container and Helm chart](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420), rather than utilizing the all-in-one container image of the [current chart](#gitlab-omnibus-chart-recommended). By offering individual containers and charts, we will be able to provide a number of benefits: * Easier horizontal scaling of each service, @@ -35,7 +35,9 @@ By offering individual containers and charts, we will be able to provide a numbe * Potential for rolling updates and canaries within a service, * and plenty more. -This is a large project and will be worked on over the span of multiple releases. For the most up-to-date status and release information, please see our [tracking issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420). We do not expect these to be production ready before the second half of 2018. +This is a large project and will be worked on over the span of multiple releases. For the most up-to-date status and release information, please see our [tracking issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420). We are planning to launch this chart in beta by the end of 2017. + +Learn more about the [cloud native GitLab chart](https://gitlab.com/charts/helm.gitlab.io/blob/master/README.md). ## Other Charts @@ -55,7 +57,7 @@ Learn more about the [gitlab chart.](gitlab_chart.md) ### Community Contributed Charts -The community has also [contributed GitLab charts](https://github.com/kubernetes/charts/tree/master/stable/gitlab-ce) to the [Helm Stable Repository](https://github.com/kubernetes/charts#repository-structure). These charts should be considered [deprecated](https://github.com/kubernetes/charts/issues/1138) in favor of the [official Charts](#official-gitlab-helm-charts-recommended). +The community has also contributed GitLab [CE](https://github.com/kubernetes/charts/tree/master/stable/gitlab-ce) and [EE](https://github.com/kubernetes/charts/tree/master/stable/gitlab-ee) charts to the [Helm Stable Repository](https://github.com/kubernetes/charts#repository-structure). These charts should be considered [deprecated](https://github.com/kubernetes/charts/issues/1138) in favor of the [official Charts](gitlab_omnibus.md). [chart]: https://github.com/kubernetes/charts [helm]: https://github.com/kubernetes/helm/blob/master/README.md diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md index 713d11b75e4..2f5d4142d04 100644 --- a/doc/install/relative_url.md +++ b/doc/install/relative_url.md @@ -1,11 +1,11 @@ -## Install GitLab under a relative URL +# Install GitLab under a relative URL -_**Note:** +NOTE: **Note:** This document describes how to run GitLab under a relative URL for installations from source. If you are using an Omnibus package, [the steps are different][omnibus-rel]. Use this guide along with the [installation guide](installation.md) if you are installing GitLab for the -first time._ +first time. --- @@ -33,7 +33,7 @@ serve GitLab under a relative URL is: After all the changes you need to recompile the assets and [restart GitLab]. -### Relative URL requirements +## Relative URL requirements If you configure GitLab with a relative URL, the assets (JavaScript, CSS, fonts, images, etc.) will need to be recompiled, which is a task which consumes a lot @@ -43,11 +43,11 @@ least 2GB of RAM available on your system, while we recommend 4GB RAM, and 4 or See the [requirements](requirements.md) document for more information. -### Enable relative URL in GitLab +## Enable relative URL in GitLab -_**Note:** +NOTE: **Note:** Do not make any changes to your web server configuration file regarding -relative URL. The relative URL support is implemented by GitLab Workhorse._ +relative URL. The relative URL support is implemented by GitLab Workhorse. --- @@ -115,7 +115,7 @@ Make sure to follow all steps below: 1. [Restart GitLab][] for the changes to take effect. -### Disable relative URL in GitLab +## Disable relative URL in GitLab To disable the relative URL: diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 17fe80fa93d..7bf126eec5d 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -121,8 +121,8 @@ Existing users using GitLab with MySQL/MariaDB are advised to ### PostgreSQL Requirements -As of GitLab 9.3, PostgreSQL 9.2 or newer is required, and earlier versions are -not supported. We highly recommend users to use at least PostgreSQL 9.6 as this +As of GitLab 10.0, PostgreSQL 9.6 or newer (but less than 10) is required, and earlier versions are +not supported. We highly recommend users to use PostgreSQL 9.6 as this is the PostgreSQL version used for development and testing. Users using PostgreSQL must ensure the `pg_trgm` extension is loaded into every @@ -184,7 +184,7 @@ Runner. We recommend using a separate machine for each GitLab Runner, if you plan to use the CI features. -[security reasons]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/security/index.md +[security reasons]: https://gitlab.com/gitlab-org/gitlab-runner/blob/master/docs/security/index.md ## Supported web browsers diff --git a/doc/integration/README.md b/doc/integration/README.md index 09d96bdd338..54e78bdef54 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Integration GitLab integrates with multiple third-party services to allow external issue diff --git a/doc/integration/azure.md b/doc/integration/azure.md index 5e3e9f5ab77..f3c9c498634 100644 --- a/doc/integration/azure.md +++ b/doc/integration/azure.md @@ -74,6 +74,9 @@ To enable the Microsoft Azure OAuth2 OmniAuth provider you must register your ap tenant_id: "TENANT ID" } } ``` + The `base_azure_url` is optional and can be added for different locales; + e.g. `base_azure_url: "https://login.microsoftonline.de"`. + 1. Replace 'CLIENT ID', 'CLIENT SECRET' and 'TENANT ID' with the values you got above. 1. Save the configuration file. diff --git a/doc/integration/google.md b/doc/integration/google.md index d5b523e6dc0..727ca13ebcf 100644 --- a/doc/integration/google.md +++ b/doc/integration/google.md @@ -1,83 +1,92 @@ # Google OAuth2 OmniAuth Provider -To enable the Google OAuth2 OmniAuth provider you must register your application with Google. Google will generate a client ID and secret key for you to use. - -1. Sign in to the [Google Developers Console](https://console.developers.google.com/) with the Google account you want to use to register GitLab. - -1. Select "Create Project". - -1. Provide the project information - - Project name: 'GitLab' works just fine here. - - Project ID: Must be unique to all Google Developer registered applications. Google provides a randomly generated Project ID by default. You can use the randomly generated ID or choose a new one. -1. Refresh the page. You should now see your new project in the list. Click on the project. - -1. Select the "Google APIs" tab in the Overview. - -1. Select and enable the following Google APIs - listed under "Popular APIs" - - Enable `Contacts API` - - Enable `Google+ API` +To enable the Google OAuth2 OmniAuth provider you must register your application +with Google. Google will generate a client ID and secret key for you to use. + +## Enabling Google OAuth + +In Google's side: + +1. Navigate to the [cloud resource manager](https://console.cloud.google.com/cloud-resource-manager) page +1. Select **Create Project** +1. Provide the project information: + - **Project name** - "GitLab" works just fine here. + - **Project ID** - Must be unique to all Google Developer registered applications. + Google provides a randomly generated Project ID by default. You can use + the randomly generated ID or choose a new one. +1. Refresh the page and you should see your new project in the list +1. Go to the [Google API Console](https://console.developers.google.com/apis/dashboard) +1. Select the previously created project form the upper left corner +1. Select **Credentials** from the sidebar +1. Select **OAuth consent screen** and fill the form with the required information +1. In the **Credentials** tab, select **Create credentials > OAuth client ID** +1. Fill in the required information + - **Application type** - Choose "Web Application" + - **Name** - Use the default one or provide your own + - **Authorized JavaScript origins** -This isn't really used by GitLab but go + ahead and put `https://gitlab.example.com` + - **Authorized redirect URIs** - Enter your domain name followed by the + callback URIs one at a time: -1. Select "Credentials" in the submenu. + ``` + https://gitlab.example.com/users/auth/google_oauth2/callback + https://gitlab.exampl.com/-/google_api/auth/callback + ``` -1. Select "Create New Client ID". +1. You should now be able to see a Client ID and Client secret. Note them down + or keep this page open as you will need them later. +1. From the **Dashboard** select **ENABLE APIS AND SERVICES > Google Cloud APIs > Container Engine API > Enable** -1. Fill in the required information - - Application type: "Web Application" - - Authorized JavaScript origins: This isn't really used by GitLab but go ahead and put 'https://gitlab.example.com' here. - - Authorized redirect URI: 'https://gitlab.example.com/users/auth/google_oauth2/callback' -1. Under the heading "Client ID for web application" you should see a Client ID and Client secret (see screenshot). Keep this page open as you continue configuration.  +On your GitLab server: -1. On your GitLab server, open the configuration file. +1. Open the configuration file. - For omnibus package: + For Omnibus GitLab: ```sh - sudo editor /etc/gitlab/gitlab.rb + sudo editor /etc/gitlab/gitlab.rb ``` For installations from source: ```sh - cd /home/git/gitlab - - sudo -u git -H editor config/gitlab.yml + cd /home/git/gitlab + sudo -u git -H editor config/gitlab.yml ``` -1. See [Initial OmniAuth Configuration](omniauth.md#initial-omniauth-configuration) for initial settings. +1. See [Initial OmniAuth Configuration](omniauth.md#initial-omniauth-configuration) for initial settings. +1. Add the provider configuration: -1. Add the provider configuration: - - For omnibus package: + For Omnibus GitLab: ```ruby - gitlab_rails['omniauth_providers'] = [ - { - "name" => "google_oauth2", - "app_id" => "YOUR_APP_ID", - "app_secret" => "YOUR_APP_SECRET", - "args" => { "access_type" => "offline", "approval_prompt" => '' } - } - ] + gitlab_rails['omniauth_providers'] = [ + { + "name" => "google_oauth2", + "app_id" => "YOUR_APP_ID", + "app_secret" => "YOUR_APP_SECRET", + "args" => { "access_type" => "offline", "approval_prompt" => '' } + } + ] ``` For installations from source: ``` - - { name: 'google_oauth2', app_id: 'YOUR_APP_ID', - app_secret: 'YOUR_APP_SECRET', - args: { access_type: 'offline', approval_prompt: '' } } + - { name: 'google_oauth2', app_id: 'YOUR_APP_ID', + app_secret: 'YOUR_APP_SECRET', + args: { access_type: 'offline', approval_prompt: '' } } ``` -1. Change 'YOUR_APP_ID' to the client ID from the Google Developer page from step 10. - -1. Change 'YOUR_APP_SECRET' to the client secret from the Google Developer page from step 10. - -1. Make sure that you configure GitLab to use an FQDN as Google will not accept raw IP addresses. +1. Change `YOUR_APP_ID` to the client ID from the Google Developer page +1. Similarly, change `YOUR_APP_SECRET` to the client secret +1. Make sure that you configure GitLab to use an FQDN as Google will not accept + raw IP addresses. For Omnibus packages: ```ruby - external_url 'https://gitlab.example.com' + external_url 'https://gitlab.example.com' ``` For installations from source: @@ -88,21 +97,13 @@ To enable the Google OAuth2 OmniAuth provider you must register your application ``` 1. Save the configuration file. - 1. [Reconfigure][] or [restart GitLab][] for the changes to take effect if you installed GitLab via Omnibus or from source respectively. -On the sign in page there should now be a Google icon below the regular sign in form. Click the icon to begin the authentication process. Google will ask the user to sign in and authorize the GitLab application. If everything goes well the user will be returned to GitLab and will be signed in. - -## Further Configuration - -This further configuration is not required for Google authentication to function but it is strongly recommended. Taking these steps will increase usability for users by providing a little more recognition and branding. - -At this point, when users first try to authenticate to your GitLab installation with Google they will see a generic application name on the prompt screen. The prompt informs the user that "Project Default Service Account" would like to access their account. "Project Default Service Account" isn't very recognizable and may confuse or cause users to be concerned. This is easily changeable. - -1. Select 'Consent screen' in the left menu. (See steps 1, 4 and 5 above for instructions on how to get here if you closed your window). -1. Scroll down until you find "Product Name". Change the product name to something more descriptive. -1. Add any additional information as you wish - homepage, logo, privacy policy, etc. None of this is required, but it may help your users. +On the sign in page there should now be a Google icon below the regular sign in +form. Click the icon to begin the authentication process. Google will ask the +user to sign in and authorize the GitLab application. If everything goes well +the user will be returned to GitLab and will be signed in. [reconfigure]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure [restart GitLab]: ../administration/restart_gitlab.md#installations-from-source diff --git a/doc/integration/saml.md b/doc/integration/saml.md index b5b245c626f..3ae98adc465 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -132,14 +132,17 @@ On the sign in page there should now be a SAML button below the regular sign in Click the icon to begin the authentication process. If everything goes well the user will be returned to GitLab and will be signed in. -## External Groups +## Marking Users as External based on SAML Groups >**Note:** This setting is only available on GitLab 8.7 and above. -SAML login includes support for external groups. You can define in the SAML -settings which groups, to which your users belong in your IdP, you wish to be -marked as [external](../user/permissions.md). +SAML login includes support for automatically identifying whether a user should +be considered an [external](../user/permissions.md) user based on the user's group +membership in the SAML identity provider. This feature **does not** allow you to +automatically add users to GitLab [Groups](../user/group/index.md), it simply +allows you to mark users as External if they are members of certain groups in the +Identity Provider. ### Requirements diff --git a/doc/integration/trello_power_up.md b/doc/integration/trello_power_up.md index d264486a872..834d63d1166 100644 --- a/doc/integration/trello_power_up.md +++ b/doc/integration/trello_power_up.md @@ -39,4 +39,4 @@ Learn more about generating a personal access token in the [Personal Access Token Documentation][personal-access-token-documentation]. Don't forget to check the API scope checkbox! -[personal-access-token-documentation]: ../user/profile/personal_access_tokens.html +[personal-access-token-documentation]: ../user/profile/personal_access_tokens.md diff --git a/doc/intro/README.md b/doc/intro/README.md index 7485912d1a2..d9acc5bdeac 100644 --- a/doc/intro/README.md +++ b/doc/intro/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Get started with GitLab ## Organize diff --git a/doc/legal/README.md b/doc/legal/README.md index 56d72ae3859..6413f1d645f 100644 --- a/doc/legal/README.md +++ b/doc/legal/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Legal - [Corporate contributor license agreement](corporate_contributor_license_agreement.md) diff --git a/doc/legal/corporate_contributor_license_agreement.md b/doc/legal/corporate_contributor_license_agreement.md index 7f08188bd65..ebb24ba0a7f 100644 --- a/doc/legal/corporate_contributor_license_agreement.md +++ b/doc/legal/corporate_contributor_license_agreement.md @@ -1,29 +1,2 @@ -# Corporate contributor license agreement - -You accept and agree to the following terms and conditions for Your present and future Contributions submitted to GitLab B.V.. Except for the license granted herein to GitLab B.V. and recipients of software distributed by GitLab B.V., You reserve all right, title, and interest in and to Your Contributions. - -1. Definitions. - - "You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with GitLab B.V.. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. - - "Contribution" shall mean the code, documentation or other original works of authorship, including any modifications or additions to an existing work, that is submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution." - -2. Grant of Copyright License. - -Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works. - -3. Grant of Patent License. - -Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed. - -4. You represent that You are legally entitled to grant the above license. You represent further that each of Your employees is authorized to submit Contributions on Your behalf, but excluding employees that are designated in writing by You as "Not authorized to submit Contributions on behalf of [name of Your corporation here]." Such designations of exclusion for unauthorized employees are to be submitted via email to legal@gitlab.com. - -5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). - -6. You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. - -7. Should You wish to submit work that is not Your original creation, You may submit it to GitLab B.V. separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [named here]". - -8. It is Your responsibility to notify GitLab.com when any change is required to the list of designated employees excluded from submitting Contributions on Your behalf per Section 4. Such notification should be sent via email to legal@gitlab.com. - -This text is licensed under the [Creative Commons Attribution 3.0 License](https://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. +This document has been replaced by a Developer Certificate of Origin and License, +as described in [Contributing.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md).
\ No newline at end of file diff --git a/doc/legal/individual_contributor_license_agreement.md b/doc/legal/individual_contributor_license_agreement.md index 59803aea080..ebb24ba0a7f 100644 --- a/doc/legal/individual_contributor_license_agreement.md +++ b/doc/legal/individual_contributor_license_agreement.md @@ -1,25 +1,2 @@ -# Individual contributor license agreement - -You accept and agree to the following terms and conditions for Your present and future Contributions submitted to GitLab B.V.. Except for the license granted herein to GitLab B.V. and recipients of software distributed by GitLab B.V., You reserve all right, title, and interest in and to Your Contributions. - -1. Definitions. - - "You" (or "Your") shall mean the copyright owner or legal entity authorized by the copyright owner that is making this Agreement with GitLab B.V.. For legal entities, the entity making a Contribution and all other entities that control, are controlled by, or are under common control with that entity are considered to be a single Contributor. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. - - "Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is intentionally submitted by You to GitLab B.V. for inclusion in, or documentation of, any of the products owned or managed by GitLab B.V. (the "Work"). For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to GitLab B.V. or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, GitLab B.V. for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by You as "Not a Contribution." - -2. Grant of Copyright License. Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute Your Contributions and such derivative works. - -3. Grant of Patent License. Subject to the terms and conditions of this Agreement, You hereby grant to GitLab B.V. and to recipients of software distributed by GitLab B.V. a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by You that are necessarily infringed by Your Contribution(s) alone or by combination of Your Contribution(s) with the Work to which such Contribution(s) was submitted. If any entity institutes patent litigation against You or any other entity (including a cross-claim or counterclaim in a lawsuit) alleging that your Contribution, or the Work to which you have contributed, constitutes direct or contributory patent infringement, then any patent licenses granted to that entity under this Agreement for that Contribution or Work shall terminate as of the date such litigation is filed. - -4. You represent that you are legally entitled to grant the above license. If your employer(s) has rights to intellectual property that you create that includes your Contributions, you represent that you have received permission to make Contributions on behalf of that employer, that your employer has waived such rights for your Contributions to GitLab B.V., or that your employer has executed a separate Corporate CLA with GitLab B.V.. - -5. You represent that each of Your Contributions is Your original creation (see section 7 for submissions on behalf of others). You represent that Your Contribution submissions include complete details of any third-party license or other restriction (including, but not limited to, related patents and trademarks) of which you are personally aware and which are associated with any part of Your Contributions. - -6. You are not expected to provide support for Your Contributions, except to the extent You desire to provide support. You may provide support for free, for a fee, or not at all. Unless required by applicable law or agreed to in writing, You provide Your Contributions on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON- INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. - -7. Should You wish to submit work that is not Your original creation, You may submit it to GitLab B.V. separately from any Contribution, identifying the complete details of its source and of any license or other restriction (including, but not limited to, related patents, trademarks, and license agreements) of which you are personally aware, and conspicuously marking the work as "Submitted on behalf of a third-party: [insert_name_here]". - -8. You agree to notify GitLab B.V. of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect. - -This text is licensed under the [Creative Commons Attribution 3.0 License](https://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. +This document has been replaced by a Developer Certificate of Origin and License, +as described in [Contributing.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md).
\ No newline at end of file diff --git a/doc/migrate_ci_to_ce/README.md b/doc/migrate_ci_to_ce/README.md index 2e7782736ff..9347a834510 100644 --- a/doc/migrate_ci_to_ce/README.md +++ b/doc/migrate_ci_to_ce/README.md @@ -372,8 +372,10 @@ CREATE TABLE ``` To fix that you need to apply this SQL statement before doing final backup: -``` -# Omnibus + +```sql +## Omnibus GitLab + gitlab-ci-rails dbconsole <<EOF -- ALTER TABLES - DROP DEFAULTS ALTER TABLE ONLY ci_application_settings ALTER COLUMN id DROP DEFAULT; @@ -427,7 +429,8 @@ ALTER TABLE ONLY ci_variables ALTER COLUMN id SET DEFAULT nextval('ci_variables_ ALTER TABLE ONLY ci_web_hooks ALTER COLUMN id SET DEFAULT nextval('ci_web_hooks_id_seq'::regclass); EOF -# Source +## Source installations + cd /home/gitlab_ci/gitlab-ci sudo -u gitlab_ci -H bundle exec rails dbconsole production <<EOF ... COPY SQL STATEMENTS FROM ABOVE ... diff --git a/doc/policy/maintenance.md b/doc/policy/maintenance.md new file mode 100644 index 00000000000..8d0afa9e692 --- /dev/null +++ b/doc/policy/maintenance.md @@ -0,0 +1,86 @@ +# GitLab Maintenance Policy + +## Versioning + +GitLab follows the [Semantic Versioning](http://semver.org/) for its releases: +`(Major).(Minor).(Patch)` in a [pragmatic way]. + +- **Major version**: Whenever there is something significant or any backwards + incompatible changes are introduced to the public API. +- **Minor version**: When new, backwards compatible functionality is introduced + to the public API or a minor feature is introduced, or when a set of smaller + features is rolled out. +- **Patch number**: When backwards compatible bug fixes are introduced that fix + incorrect behavior. + +For example, for GitLab version 10.5.7: + +* `10` represents major version +* `5` represents minor version +* `7` represents patch number + +## Patch releases + +Patch releases usually only include bug fixes and are only done for the current +stable release. That said, in some cases, we may backport it to previous stable +release, depending on the severity of the bug. + +For instance, if we release `10.1.1` with a fix for a severe bug introduced in +`10.0.0`, we could backport the fix to a new `10.0.x` patch release. + +### Security releases + +Security releases are a special kind of patch release that only include security +fixes and patches (see below). + +Our current policy is to support one stable release at any given time, but for +medium-level security issues, we may backport security fixes to the previous two +monthly releases. + +For very serious security issues, there is +[precedent](https://about.gitlab.com/2016/05/02/cve-2016-4340-patches/) +to backport security fixes to even more monthly releases of GitLab. +This decision is made on a case-by-case basis. + +## Upgrade recommendations + +We encourage everyone to run the latest stable release to ensure that you can +easily upgrade to the most secure and feature-rich GitLab experience. In order +to make sure you can easily run the most recent stable release, we are working +hard to keep the update process simple and reliable. + +If you are unable to follow our monthly release cycle, there are a couple of +cases you need to consider. + +It is considered safe to jump between patch versions and minor versions within +one major version. For example, it is safe to: + +* Upgrade the patch version: + * `8.9.0` -> `8.9.7` + * `8.9.0` -> `8.9.1` + * `8.9.2` -> `8.9.6` +* Upgrade the minor version: + * `8.9.4` -> `8.12.3` + * `9.2.3` -> `9.5.5` + +Upgrading the major version requires more attention. +We cannot guarantee that upgrading between major versions will be seamless. As previously mentioned, major versions are reserved for backwards incompatible changes. + +We recommend that you first upgrade to the latest available minor version within +your major version. By doing this, you can address any deprecation messages +that could possibly change behaviour in the next major release. + +Please see the table below for some examples: + +| Latest stable version | Your version | Recommended upgrade path | Note | +| -------------- | ------------ | ------------------------ | ---------------- | +| 9.4.5 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.4.5` | `8.17.7` is the last version in version `8` | +| 10.1.4 | 8.13.4 | `8.13.4` -> `8.17.7` -> `9.5.8` -> `10.1.4` | `8.17.7` is the last version in version `8`, `9.5.8` is the last version in version `9` | + +More information about the release procedures can be found in our +[release-tools documentation][rel]. You may also want to read our +[Responsible Disclosure Policy][disclosure]. + +[rel]: https://gitlab.com/gitlab-org/release-tools/blob/master/doc/ +[disclosure]: https://about.gitlab.com/disclosure/ +[pragmatic way]: https://gist.github.com/jashkenas/cbd2b088e20279ae2c8e diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md index 2b81ebc9c59..2f916f5dea7 100644 --- a/doc/raketasks/README.md +++ b/doc/raketasks/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Rake tasks - [Backup restore](backup_restore.md) diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index ae69d7f92f2..54c3e20d61d 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -136,44 +136,54 @@ In the example below we use Amazon S3 for storage, but Fog also lets you use for AWS, Google, OpenStack Swift, Rackspace and Aliyun as well. A local driver is [also available](#uploading-to-locally-mounted-shares). -For omnibus packages, add the following to `/etc/gitlab/gitlab.rb`: +#### Using Amazon S3 -```ruby -gitlab_rails['backup_upload_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-west-1', - 'aws_access_key_id' => 'AKIAKIAKI', - 'aws_secret_access_key' => 'secret123' - # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key - # 'use_iam_profile' => true -} -gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' -``` +For Omnibus GitLab packages: + +1. Add the following to `/etc/gitlab/gitlab.rb`: -Make sure to run `sudo gitlab-ctl reconfigure` after editing `/etc/gitlab/gitlab.rb` to reflect the changes. + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-west-1', + 'aws_access_key_id' => 'AKIAKIAKI', + 'aws_secret_access_key' => 'secret123' + # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key + # 'use_iam_profile' => true + } + gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' + ``` + +1. [Reconfigure GitLab] for the changes to take effect + +--- For installations from source: -```yaml - backup: - # snip - upload: - # Fog storage connection settings, see http://fog.io/storage/ . - connection: - provider: AWS - region: eu-west-1 - aws_access_key_id: AKIAKIAKI - aws_secret_access_key: 'secret123' - # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty - # ie. aws_access_key_id: '' - # use_iam_profile: 'true' - # The remote 'directory' to store your backups. For S3, this would be the bucket name. - remote_directory: 'my.s3.bucket' - # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional - # encryption: 'AES256' - # Specifies Amazon S3 storage class to use for backups, this is optional - # storage_class: 'STANDARD' -``` +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + # snip + upload: + # Fog storage connection settings, see http://fog.io/storage/ . + connection: + provider: AWS + region: eu-west-1 + aws_access_key_id: AKIAKIAKI + aws_secret_access_key: 'secret123' + # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty + # ie. aws_access_key_id: '' + # use_iam_profile: 'true' + # The remote 'directory' to store your backups. For S3, this would be the bucket name. + remote_directory: 'my.s3.bucket' + # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional + # encryption: 'AES256' + # Specifies Amazon S3 storage class to use for backups, this is optional + # storage_class: 'STANDARD' + ``` + +1. [Restart GitLab] for the changes to take effect If you are uploading your backups to S3 you will probably want to create a new IAM user with restricted access rights. To give the upload user access only for @@ -226,6 +236,50 @@ with the name of your bucket: } ``` +#### Using Google Cloud Storage + +If you want to use Google Cloud Storage to save backups, you'll have to create +an access key from the Google console first: + +1. Go to the storage settings page https://console.cloud.google.com/storage/settings +1. Select "Interoperability" and create an access key +1. Make note of the "Access Key" and "Secret" and replace them in the + configurations below +1. Make sure you already have a bucket created + +For Omnibus GitLab packages: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'Google', + 'google_storage_access_key_id' => 'Access Key', + 'google_storage_secret_access_key' => 'Secret' + } + gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket' + ``` + +1. [Reconfigure GitLab] for the changes to take effect + +--- + +For installations from source: + +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + upload: + connection: + provider: 'Google' + google_storage_access_key_id: 'Access Key' + google_storage_secret_access_key: 'Secret' + remote_directory: 'my.google.bucket' + ``` + +1. [Restart GitLab] for the changes to take effect + ### Uploading to locally mounted shares You may also send backups to a mounted share (`NFS` / `CIFS` / `SMB` / etc.) by @@ -370,7 +424,7 @@ This is recommended to reduce cron spam. ## Restore -GitLab provides a simple command line interface to backup your whole installation, +GitLab provides a simple command line interface to restore your whole installation, and is flexible enough to fit your needs. The [restore prerequisites section](#restore-prerequisites) includes crucial @@ -445,6 +499,14 @@ Restoring repositories: Deleting tmp directories...[DONE] ``` +Next, restore `/home/git/gitlab/.secret` if necessary as mentioned above. + +Restart GitLab: + +```shell +sudo service gitlab restart +``` + ### Restore for Omnibus installations This procedure assumes that: @@ -480,10 +542,12 @@ restore: sudo gitlab-rake gitlab:backup:restore BACKUP=1493107454_2017_04_25_9.1.0 ``` +Next, restore `/etc/gitlab/gitlab-secrets.json` if necessary as mentioned above. + Restart and check GitLab: ```shell -sudo gitlab-ctl start +sudo gitlab-ctl restart sudo gitlab-rake gitlab:check SANITIZE=true ``` @@ -544,3 +608,6 @@ The rake task runs this as the `gitlab` user which does not have the superuser a Those objects have no influence on the database backup/restore but they give this annoying warning. For more information see similar questions on postgresql issue tracker[here](http://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](http://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](http://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql). + +[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure +[restart GitLab]: ../administration/restart_gitlab.md#installations-from-source diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md index 3ae46019daf..5554a0c8b78 100644 --- a/doc/raketasks/user_management.md +++ b/doc/raketasks/user_management.md @@ -149,18 +149,3 @@ cp config/secrets.yml.bak config/secrets.yml sudo /etc/init.d/gitlab start ``` - -## Clear authentication tokens for all users. Important! Data loss! - -Clear authentication tokens for all users in the GitLab database. This -task is useful if your users' authentication tokens might have been exposed in -any way. All the existing tokens will become invalid, and new tokens are -automatically generated upon sign-in or user modification. - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:users:clear_all_authentication_tokens - -# installation from source -bundle exec rake gitlab:users:clear_all_authentication_tokens RAILS_ENV=production -``` diff --git a/doc/security/README.md b/doc/security/README.md index 0fea6be8b55..d397ff104ab 100644 --- a/doc/security/README.md +++ b/doc/security/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Security - [Password length limits](password_length_limits.md) diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 793de9d777c..33a2d7a88a7 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -1,4 +1,4 @@ -# SSH +# GitLab and SSH keys Git is a distributed version control system, which means you can work locally but you can also share or "push" your changes to other servers. @@ -114,7 +114,7 @@ custom name continue onto the next step. If you manually copied your public SSH key make sure you copied the entire key starting with `ssh-rsa` and ending with your email. - + 1. Optionally you can test your setup by running `ssh -T git@example.com` (replacing `example.com` with your GitLab domain) and verifying that you receive a `Welcome to GitLab` message. @@ -172,7 +172,7 @@ dummy user account. If you are a project master or owner, you can add a deploy key in the project settings under the section 'Repository'. Specify a title for the new deploy key and paste a public SSH key. After this, the machine that uses -the corresponding private SSH key has read-only or read-write (if enabled) +the corresponding private SSH key has read-only or read-write (if enabled) access to the project. You can't add the same deploy key twice using the form. @@ -232,7 +232,7 @@ something is wrong with your SSH setup. - Ensure that you generated your SSH key pair correctly and added the public SSH key to your GitLab profile -- Try manually registering your private SSH key using `ssh-agent` as documented +- Try manually registering your private SSH key using `ssh-agent` as documented earlier in this document - Try to debug the connection by running `ssh -Tv git@example.com` (replacing `example.com` with your GitLab domain) diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index 0399ebec86a..f2a9b1d769b 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -1,6 +1,26 @@ # System hooks -Your GitLab instance can perform HTTP POST requests on the following events: `project_create`, `project_destroy`, `project_rename`, `project_transfer`, `project_update`, `user_add_to_team`, `user_remove_from_team`, `user_create`, `user_destroy`, `key_create`, `key_destroy`, `group_create`, `group_destroy`, `user_add_to_group` and `user_remove_from_group`. +Your GitLab instance can perform HTTP POST requests on the following events: + +- `project_create` +- `project_destroy` +- `project_rename` +- `project_transfer` +- `project_update` +- `user_add_to_team` +- `user_remove_from_team` +- `user_create` +- `user_destroy` +- `user_rename` +- `key_create` +- `key_destroy` +- `group_create` +- `group_destroy` +- `group_rename` +- `user_add_to_group` +- `user_remove_from_group` + +The triggers for most of these are self-explanatory, but `project_update` and `project_rename` deserve some clarification: `project_update` is fired any time an attribute of a project is changed (name, description, tags, etc.) *unless* the `path` attribute is also changed. In that case, a `project_rename` is triggered instead (so that, for instance, if all you care about is the repo URL, you can just listen for `project_rename`). System hooks can be used, e.g. for logging or changing information in a LDAP server. @@ -70,6 +90,9 @@ X-Gitlab-Event: System Hook } ``` +Note that `project_rename` is not triggered if the namespace changes. +Please refer to `group_rename` and `user_rename` for that case. + **Project transferred:** ```json @@ -173,6 +196,21 @@ X-Gitlab-Event: System Hook } ``` +**User renamed:** + +```json +{ + "event_name": "user_rename", + "created_at": "2017-11-01T11:21:04Z", + "updated_at": "2017-11-01T14:04:47Z", + "name": "new-name", + "email": "best-email@example.tld", + "user_id": 58, + "username": "new-exciting-name", + "old_username": "old-boring-name" +} +``` + **Key added** ```json @@ -207,13 +245,15 @@ X-Gitlab-Event: System Hook "updated_at": "2012-07-21T07:38:22Z", "event_name": "group_create", "name": "StoreCloud", - "owner_email": "johnsmith@gmail.com", - "owner_name": "John Smith", + "owner_email": null, + "owner_name": null, "path": "storecloud", "group_id": 78 } ``` +`owner_name` and `owner_email` are always `null`. Please see https://gitlab.com/gitlab-org/gitlab-ce/issues/39675. + **Group removed:** ```json @@ -222,13 +262,35 @@ X-Gitlab-Event: System Hook "updated_at": "2012-07-21T07:38:22Z", "event_name": "group_destroy", "name": "StoreCloud", - "owner_email": "johnsmith@gmail.com", - "owner_name": "John Smith", + "owner_email": null, + "owner_name": null, "path": "storecloud", "group_id": 78 } ``` +`owner_name` and `owner_email` are always `null`. Please see https://gitlab.com/gitlab-org/gitlab-ce/issues/39675. + +**Group renamed:** + +```json +{ + "event_name": "group_rename", + "created_at": "2017-10-30T15:09:00Z", + "updated_at": "2017-11-01T10:23:52Z", + "name": "Better Name", + "path": "better-name", + "full_path": "parent-group/better-name", + "group_id": 64, + "owner_name": null, + "owner_email": null, + "old_path": "old-name", + "old_full_path": "parent-group/old-name" +} +``` + +`owner_name` and `owner_email` are always `null`. Please see https://gitlab.com/gitlab-org/gitlab-ce/issues/39675. + **New Group Member:** ```json diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index fac91935a45..597c98fbf6b 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -11,6 +11,7 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Security Webcast with Yubico](https://about.gitlab.com/2016/08/31/gitlab-and-yubico-security-webcast/) - **Integrations:** - [GitLab as OAuth2 authentication service provider](../../integration/oauth_provider.md#introduction-to-oauth) + - [GitLab as OpenID Connect identity provider](../../integration/openid_connect_provider.md) ## GitLab administrators diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 0b7c11ea700..1cfdabac248 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -144,6 +144,12 @@ has a `.gitlab-ci.yml` or not: All you need to do is remove your existing `.gitlab-ci.yml`, and you can even do that in a branch to test Auto DevOps before committing to `master`. +NOTE: **Note:** +If you are a GitLab Administrator, you can enable Auto DevOps instance wide +in **Admin Area > Settings > Continuous Integration and Deployment**. Doing that, +all the projects that haven't explicitly set an option will have Auto DevOps +enabled by default. + ## Stages of Auto DevOps The following sections describe the stages of Auto DevOps. Read them carefully @@ -511,13 +517,13 @@ sudo gitlab-rails console Then run: ```ruby -Feature.get(:auto_devops_banner_disabled).disable +Feature.get(:auto_devops_banner_disabled).enable ``` Or through the HTTP API with an admin access token: ```sh -curl --data "value=true" --header "PRIVATE-TOKEN: private_token" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled +curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled ``` [ce-37115]: https://gitlab.com/gitlab-org/gitlab-ce/issues/37115 diff --git a/doc/university/README.md b/doc/university/README.md index 170582bcd0c..55865ac23e8 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab University GitLab University is the best place to learn about **Version Control with Git and GitLab**. @@ -51,10 +55,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 1.5. Migrating from other Source Control -1. [Migrating from BitBucket/Stash](https://docs.gitlab.com/ee/workflow/importing/import_projects_from_bitbucket.html) -1. [Migrating from GitHub](https://docs.gitlab.com/ee/workflow/importing/import_projects_from_github.html) -1. [Migrating from SVN](https://docs.gitlab.com/ee/workflow/importing/migrating_from_svn.html) -1. [Migrating from Fogbugz](https://docs.gitlab.com/ee/workflow/importing/import_projects_from_fogbugz.html) +1. [Migrating from BitBucket/Stash](https://docs.gitlab.com/ee/user/project/import/bitbucket.html) +1. [Migrating from GitHub](https://docs.gitlab.com/ee/user/project/import/github.html) +1. [Migrating from SVN](https://docs.gitlab.com/ee/user/project/import/svn.html) +1. [Migrating from Fogbugz](https://docs.gitlab.com/ee/user/project/import/fogbugz.html) #### 1.6. GitLab Inc. @@ -76,13 +80,13 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project - Being part of our Great Community and Contributing to GitLab 1. [Getting Started with the GitLab Development Kit (GDK)](https://about.gitlab.com/2016/06/08/getting-started-with-gitlab-development-kit/) 1. [Contributing Technical Articles to the GitLab Blog](https://about.gitlab.com/2016/01/26/call-for-writers/) -1. [GitLab Training Workshops](https://about.gitlab.com/training) +1. [GitLab Training Workshops](https://docs.gitlab.com/ce/university/training/end-user/) +1. [GitLab Professional Services](https://about.gitlab.com/services/) #### 1.8 GitLab Training Material 1. [Git and GitLab Terminology](glossary/README.md) 1. [Git and GitLab Workshop - Slides](https://docs.google.com/presentation/d/1JzTYD8ij9slejV2-TO-NzjCvlvj6mVn9BORePXNJoMI/edit?usp=drive_web) -1. [Git and GitLab Revision](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/university/training/end-user) --- diff --git a/doc/university/bookclub/booklist.md b/doc/university/bookclub/booklist.md index c4229832e9f..26c3851276b 100644 --- a/doc/university/bookclub/booklist.md +++ b/doc/university/bookclub/booklist.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Books List of books and resources, that may be worth reading. diff --git a/doc/university/bookclub/index.md b/doc/university/bookclub/index.md index 022a61f4429..63238685b2b 100644 --- a/doc/university/bookclub/index.md +++ b/doc/university/bookclub/index.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # The GitLab Book Club The Book Club is a casual meet-up to read and discuss books we like. diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 9544de41b9a..c6a91c8d5c2 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -1,4 +1,8 @@ -## What is the Glossary +--- +comments: false +--- + +# What is the Glossary This contains a simplified list and definitions of some of the terms that you will encounter in your day to day activities when working with GitLab. Please add any terms that you discover that you think would be useful for others. @@ -456,7 +460,7 @@ A route table contains rules (called routes) that determine where network traffi ### Runners -Actual build machines/containers that [run and execute tests](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner) you have specified to be run on GitLab CI. +Actual build machines/containers that [run and execute tests](https://gitlab.com/gitlab-org/gitlab-runner) you have specified to be run on GitLab CI. ### Sidekiq diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 6b8f3cd3d1d..54625996dff 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # High Availability on AWS diff --git a/doc/university/process/README.md b/doc/university/process/README.md index 04f2d52514f..fdf6224f7f6 100644 --- a/doc/university/process/README.md +++ b/doc/university/process/README.md @@ -1,8 +1,12 @@ --- +comments: false +--- + +--- title: University | Process --- -## Suggesting improvements +# Suggesting improvements If you would like to teach a class or participate or help in any way please submit a merge request and assign it to [Job](https://gitlab.com/u/JobV). diff --git a/doc/university/support/README.md b/doc/university/support/README.md index 567dadb3b47..25d5fe351ca 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -1,5 +1,9 @@ +--- +comments: false +--- -## Support Boot Camp + +# Support Boot Camp **Goal:** Prepare new Service Engineers at GitLab diff --git a/doc/university/training/end-user/README.md b/doc/university/training/end-user/README.md index 03c62a81b10..a882bf0eb48 100644 --- a/doc/university/training/end-user/README.md +++ b/doc/university/training/end-user/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Training diff --git a/doc/university/training/gitlab_flow.md b/doc/university/training/gitlab_flow.md index a7db1f2e069..02a6ad48a38 100755..100644 --- a/doc/university/training/gitlab_flow.md +++ b/doc/university/training/gitlab_flow.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Flow - A simplified branching strategy diff --git a/doc/university/training/index.md b/doc/university/training/index.md index 03179ff5a77..14f096b130f 100755..100644 --- a/doc/university/training/index.md +++ b/doc/university/training/index.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Training Material All GitLab training material is stored in markdown format. Slides are diff --git a/doc/university/training/topics/additional_resources.md b/doc/university/training/topics/additional_resources.md index 3ed601625cf..d01634df744 100755..100644 --- a/doc/university/training/topics/additional_resources.md +++ b/doc/university/training/topics/additional_resources.md @@ -1,4 +1,8 @@ -## Additional Resources +--- +comments: false +--- + +# Additional Resources 1. GitLab Documentation [http://docs.gitlab.com](http://docs.gitlab.com/) 2. GUI Clients [http://git-scm.com/downloads/guis](http://git-scm.com/downloads/guis) diff --git a/doc/university/training/topics/agile_git.md b/doc/university/training/topics/agile_git.md index e6e4fea9b51..251af99bed7 100755..100644 --- a/doc/university/training/topics/agile_git.md +++ b/doc/university/training/topics/agile_git.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Agile and Git ---------- diff --git a/doc/university/training/topics/bisect.md b/doc/university/training/topics/bisect.md index a60c4365e0c..2d5ab107fe6 100755..100644 --- a/doc/university/training/topics/bisect.md +++ b/doc/university/training/topics/bisect.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Bisect ---------- diff --git a/doc/university/training/topics/cherry_picking.md b/doc/university/training/topics/cherry_picking.md index af7a70a2818..df23024b6ee 100755..100644 --- a/doc/university/training/topics/cherry_picking.md +++ b/doc/university/training/topics/cherry_picking.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Cherry Pick ---------- diff --git a/doc/university/training/topics/env_setup.md b/doc/university/training/topics/env_setup.md index 8149379b36f..b7bec83ed8a 100755..100644 --- a/doc/university/training/topics/env_setup.md +++ b/doc/university/training/topics/env_setup.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Configure your environment ---------- diff --git a/doc/university/training/topics/explore_gitlab.md b/doc/university/training/topics/explore_gitlab.md index b65457728c0..84a1879cd92 100755..100644 --- a/doc/university/training/topics/explore_gitlab.md +++ b/doc/university/training/topics/explore_gitlab.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Explore GitLab projects ---------- diff --git a/doc/university/training/topics/feature_branching.md b/doc/university/training/topics/feature_branching.md index 4b34406ea75..0df5f26dbea 100755..100644 --- a/doc/university/training/topics/feature_branching.md +++ b/doc/university/training/topics/feature_branching.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Feature branching ---------- diff --git a/doc/university/training/topics/getting_started.md b/doc/university/training/topics/getting_started.md index ec7bb2631aa..153b45fb4da 100755..100644 --- a/doc/university/training/topics/getting_started.md +++ b/doc/university/training/topics/getting_started.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Getting Started ---------- diff --git a/doc/university/training/topics/git_add.md b/doc/university/training/topics/git_add.md index 9ffb4b9c859..651366e0d49 100755..100644 --- a/doc/university/training/topics/git_add.md +++ b/doc/university/training/topics/git_add.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git Add ---------- diff --git a/doc/university/training/topics/git_intro.md b/doc/university/training/topics/git_intro.md index ca1ff29d93b..7e502d6dad4 100755..100644 --- a/doc/university/training/topics/git_intro.md +++ b/doc/university/training/topics/git_intro.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git introduction ---------- diff --git a/doc/university/training/topics/git_log.md b/doc/university/training/topics/git_log.md index 32ebceff491..f2709ae3890 100755..100644 --- a/doc/university/training/topics/git_log.md +++ b/doc/university/training/topics/git_log.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git Log ---------- @@ -49,8 +53,8 @@ git log --since=1.month.ago --until=3.weeks.ago ``` cd ~/workspace -git clone git@gitlab.com:gitlab-org/gitlab-ci-multi-runner.git -cd gitlab-ci-multi-runner +git clone git@gitlab.com:gitlab-org/gitlab-runner.git +cd gitlab-runner git log --author="Travis" git log --since=1.month.ago --until=3.weeks.ago git log --since=1.month.ago --until=1.day.ago --author="Travis" diff --git a/doc/university/training/topics/gitlab_flow.md b/doc/university/training/topics/gitlab_flow.md index 8e5d3baf959..b8049b5c80e 100755..100644 --- a/doc/university/training/topics/gitlab_flow.md +++ b/doc/university/training/topics/gitlab_flow.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Flow ---------- diff --git a/doc/university/training/topics/merge_conflicts.md b/doc/university/training/topics/merge_conflicts.md index 77807b3e7ef..9a1ce550868 100755..100644 --- a/doc/university/training/topics/merge_conflicts.md +++ b/doc/university/training/topics/merge_conflicts.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Merge conflicts ---------- diff --git a/doc/university/training/topics/merge_requests.md b/doc/university/training/topics/merge_requests.md index 5b446f02f63..4e8c9de85a1 100755..100644 --- a/doc/university/training/topics/merge_requests.md +++ b/doc/university/training/topics/merge_requests.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Merge requests ---------- diff --git a/doc/university/training/topics/rollback_commits.md b/doc/university/training/topics/rollback_commits.md index cf647284604..0db1d93d1dc 100755..100644 --- a/doc/university/training/topics/rollback_commits.md +++ b/doc/university/training/topics/rollback_commits.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Rollback Commits ---------- diff --git a/doc/university/training/topics/stash.md b/doc/university/training/topics/stash.md index c1bdda32645..5b27ac12f77 100755..100644 --- a/doc/university/training/topics/stash.md +++ b/doc/university/training/topics/stash.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Git Stash ---------- diff --git a/doc/university/training/topics/subtree.md b/doc/university/training/topics/subtree.md index 5d869af64c1..b5a892dc17b 100755..100644 --- a/doc/university/training/topics/subtree.md +++ b/doc/university/training/topics/subtree.md @@ -1,8 +1,8 @@ -## Subtree +--- +comments: false +--- ----------- - -## Subtree +# Subtree * Used when there are nested repositories. * Not recommended when the amount of dependencies is too large diff --git a/doc/university/training/topics/tags.md b/doc/university/training/topics/tags.md index e9607b5a875..ab48d52d3c3 100755..100644 --- a/doc/university/training/topics/tags.md +++ b/doc/university/training/topics/tags.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Tags ---------- diff --git a/doc/university/training/topics/unstage.md b/doc/university/training/topics/unstage.md index 17dbb64b9e6..fc72949ade9 100755..100644 --- a/doc/university/training/topics/unstage.md +++ b/doc/university/training/topics/unstage.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Unstage ---------- diff --git a/doc/university/training/user_training.md b/doc/university/training/user_training.md index 9e38df26b6a..90e1d2ba5e8 100755..100644 --- a/doc/university/training/user_training.md +++ b/doc/university/training/user_training.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Git Workshop --- diff --git a/doc/update/10.0-to-10.1.md b/doc/update/10.0-to-10.1.md new file mode 100644 index 00000000000..af815d26a74 --- /dev/null +++ b/doc/update/10.0-to-10.1.md @@ -0,0 +1,360 @@ +--- +comments: false +--- + +# From 10.0 to 10.1 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + +```bash +sudo service gitlab stop +``` + +### 2. Backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Update Ruby + +NOTE: GitLab 9.0 and higher only support Ruby 2.3.x and dropped support for Ruby 2.1.x. Be +sure to upgrade your interpreter if necessary. + +You can check which version you are running with `ruby -v`. + +Download and compile Ruby: + +```bash +mkdir /tmp/ruby && cd /tmp/ruby +curl --remote-name --progress https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.5.tar.gz +echo '3247e217d6745c27ef23bdc77b6abdb4b57a118f ruby-2.3.5.tar.gz' | shasum -c - && tar xzf ruby-2.3.5.tar.gz +cd ruby-2.3.5 +./configure --disable-install-rdoc +make +sudo make install +``` + +Install Bundler: + +```bash +sudo gem install bundler --no-ri --no-rdoc +``` + +### 4. Update Node + +GitLab now runs [webpack](http://webpack.js.org) to compile frontend assets and +it has a minimum requirement of node v4.3.0. + +You can check which version you are running with `node -v`. If you are running +a version older than `v4.3.0` you will need to update to a newer version. You +can find instructions to install from community maintained packages or compile +from source at the nodejs.org website. + +<https://nodejs.org/en/download/> + + +Since 8.17, GitLab requires the use of yarn `>= v0.17.0` to manage +JavaScript dependencies. + +```bash +curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - +echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list +sudo apt-get update +sudo apt-get install yarn +``` + +More information can be found on the [yarn website](https://yarnpkg.com/en/docs/install). + +### 5. Update Go + +NOTE: GitLab 9.2 and higher only supports Go 1.8.3 and dropped support for Go +1.5.x through 1.7.x. Be sure to upgrade your installation if necessary. + +You can check which version you are running with `go version`. + +Download and install Go: + +```bash +# Remove former Go installation folder +sudo rm -rf /usr/local/go + +curl --remote-name --progress https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz +echo '1862f4c3d3907e59b04a757cfda0ea7aa9ef39274af99a784f5be843c80c6772 go1.8.3.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.8.3.linux-amd64.tar.gz +sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ +rm go1.8.3.linux-amd64.tar.gz +``` + +### 6. Get latest code + +```bash +cd /home/git/gitlab + +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +sudo -u git -H git checkout -- locale +``` + +For GitLab Community Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-1-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +cd /home/git/gitlab + +sudo -u git -H git checkout 10-1-stable-ee +``` + +### 7. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) +sudo -u git -H bin/compile +``` + +### 8. Update gitlab-workhorse + +Install and compile gitlab-workhorse. GitLab-Workhorse uses +[GNU Make](https://www.gnu.org/software/make/). +If you are not using Linux you may have to run `gmake` instead of +`make` below. + +```bash +cd /home/git/gitlab-workhorse + +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) +sudo -u git -H make +``` + +### 9. Update Gitaly + +#### New Gitaly configuration options required + +In order to function Gitaly needs some additional configuration information. Below we assume you installed Gitaly in `/home/git/gitaly` and GitLab Shell in `/home/git/gitlab-shell`. + +```shell +echo ' +[gitaly-ruby] +dir = "/home/git/gitaly/ruby" + +[gitlab-shell] +dir = "/home/git/gitlab-shell" +' | sudo -u git tee -a /home/git/gitaly/config.toml +``` + +#### Check Gitaly configuration + +Due to a bug in the `rake gitlab:gitaly:install` script your Gitaly +configuration file may contain syntax errors. The block name +`[[storages]]`, which may occur more than once in your `config.toml` +file, should be `[[storage]]` instead. + +```shell +sudo -u git -H sed -i.pre-10.1 's/\[\[storages\]\]/[[storage]]/' /home/git/gitaly/config.toml +``` + +#### Compile Gitaly + +```shell +cd /home/git/gitaly +sudo -u git -H git fetch --all --tags +sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) +sudo -u git -H make +``` + +### 10. Update MySQL permissions + +If you are using MySQL you need to grant the GitLab user the necessary +permissions on the database: + +```bash +mysql -u root -p -e "GRANT TRIGGER ON \`gitlabhq_production\`.* TO 'git'@'localhost';" +``` + +If you use MySQL with replication, or just have MySQL configured with binary logging, +you will need to also run the following on all of your MySQL servers: + +```bash +mysql -u root -p -e "SET GLOBAL log_bin_trust_function_creators = 1;" +``` + +You can make this setting permanent by adding it to your `my.cnf`: + +``` +log_bin_trust_function_creators=1 +``` + +### 11. Update configuration files + +#### New configuration options for `gitlab.yml` + +There might be configuration options available for [`gitlab.yml`][yaml]. View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +cd /home/git/gitlab + +git diff origin/10-0-stable:config/gitlab.yml.example origin/10-1-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +cd /home/git/gitlab + +# For HTTPS configurations +git diff origin/10-0-stable:lib/support/nginx/gitlab-ssl origin/10-1-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/10-0-stable:lib/support/nginx/gitlab origin/10-1-stable:lib/support/nginx/gitlab +``` + +If you are using Strict-Transport-Security in your installation to continue using it you must enable it in your Nginx +configuration as GitLab application no longer handles setting it. + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/lib/support/init.d/gitlab.default.example#L38 + +#### SMTP configuration + +If you're installing from source and use SMTP to deliver mail, you will need to add the following line +to config/initializers/smtp_settings.rb: + +```ruby +ActionMailer::Base.delivery_method = :smtp +``` + +See [smtp_settings.rb.sample] as an example. + +[smtp_settings.rb.sample]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/config/initializers/smtp_settings.rb.sample#L13 + +#### Init script + +There might be new configuration options available for [`gitlab.default.example`][gl-example]. View them with the command below and apply them manually to your current `/etc/default/gitlab`: + +```sh +cd /home/git/gitlab + +git diff origin/10-0-stable:lib/support/init.d/gitlab.default.example origin/10-1-stable:lib/support/init.d/gitlab.default.example +``` + +Ensure you're still up-to-date with the latest init script changes: + +```bash +cd /home/git/gitlab + +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +For Ubuntu 16.04.1 LTS: + +```bash +sudo systemctl daemon-reload +``` + +### 12. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Compile GetText PO files + +sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production + +# Update node dependencies and recompile assets +sudo -u git -H bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production + +# Clean up cache +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +**MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). + +### 13. Start application + +```bash +sudo service gitlab start +sudo service nginx restart +``` + +### 14. Check application status + +Check if GitLab and its environment are configured correctly: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +To make sure you didn't miss anything run a more thorough check: + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (10.0) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 9.5 to 10.0](9.5-to-10.0.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +[yaml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/config/gitlab.yml.example +[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/10-1-stable/lib/support/init.d/gitlab.default.example diff --git a/doc/update/2.6-to-3.0.md b/doc/update/2.6-to-3.0.md index 97cd277b424..8f18bd93cea 100644 --- a/doc/update/2.6-to-3.0.md +++ b/doc/update/2.6-to-3.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 2.6 to 3.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/2.6-to-3.0.md) for the most up to date instructions.* diff --git a/doc/update/2.9-to-3.0.md b/doc/update/2.9-to-3.0.md index a890aa885d5..6a3c2387683 100644 --- a/doc/update/2.9-to-3.0.md +++ b/doc/update/2.9-to-3.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 2.9 to 3.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/2.9-to-3.0.md) for the most up to date instructions.* diff --git a/doc/update/3.0-to-3.1.md b/doc/update/3.0-to-3.1.md index e32508745a2..1f25b8265c9 100644 --- a/doc/update/3.0-to-3.1.md +++ b/doc/update/3.0-to-3.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 3.0 to 3.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/3.0-to-3.1.md) for the most up to date instructions.* diff --git a/doc/update/3.1-to-4.0.md b/doc/update/3.1-to-4.0.md index b370464390e..1a53ddeb4bd 100644 --- a/doc/update/3.1-to-4.0.md +++ b/doc/update/3.1-to-4.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 3.1 to 4.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/3.1-to-4.0.md) for the most up to date instructions.* diff --git a/doc/update/4.0-to-4.1.md b/doc/update/4.0-to-4.1.md index 7124424bb60..40a133e796e 100644 --- a/doc/update/4.0-to-4.1.md +++ b/doc/update/4.0-to-4.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 4.0 to 4.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/4.0-to-4.1.md) for the most up to date instructions.* diff --git a/doc/update/4.1-to-4.2.md b/doc/update/4.1-to-4.2.md index 8ed5b333a2e..1fd6c58bda7 100644 --- a/doc/update/4.1-to-4.2.md +++ b/doc/update/4.1-to-4.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 4.1 to 4.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/4.1-to-4.2.md) for the most up to date instructions.* diff --git a/doc/update/4.2-to-5.0.md b/doc/update/4.2-to-5.0.md index 1ec39218ba8..311664b2bc1 100644 --- a/doc/update/4.2-to-5.0.md +++ b/doc/update/4.2-to-5.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 4.2 to 5.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/4.2-to-5.0.md) for the most up to date instructions.* diff --git a/doc/update/5.0-to-5.1.md b/doc/update/5.0-to-5.1.md index 9c9950fb2c6..7067ea4c40c 100644 --- a/doc/update/5.0-to-5.1.md +++ b/doc/update/5.0-to-5.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.0 to 5.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.0-to-5.1.md) for the most up to date instructions.* diff --git a/doc/update/5.1-to-5.2.md b/doc/update/5.1-to-5.2.md index 2aab47d2d7c..4faf5fa549e 100644 --- a/doc/update/5.1-to-5.2.md +++ b/doc/update/5.1-to-5.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.1 to 5.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.1-to-5.2.md) for the most up to date instructions.* diff --git a/doc/update/5.1-to-5.4.md b/doc/update/5.1-to-5.4.md index e80f1b89c63..212343bac3f 100644 --- a/doc/update/5.1-to-5.4.md +++ b/doc/update/5.1-to-5.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.1 to 5.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.1-to-5.4.md) for the most up to date instructions.* diff --git a/doc/update/5.1-to-6.0.md b/doc/update/5.1-to-6.0.md index 1ee175383da..865d38e0ca4 100644 --- a/doc/update/5.1-to-6.0.md +++ b/doc/update/5.1-to-6.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.1 to 6.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.1-to-6.0.md) for the most up to date instructions.* diff --git a/doc/update/5.2-to-5.3.md b/doc/update/5.2-to-5.3.md index 2ae50510f63..ed4f3ebdd53 100644 --- a/doc/update/5.2-to-5.3.md +++ b/doc/update/5.2-to-5.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.2 to 5.3 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.2-to-5.3.md) for the most up to date instructions.* diff --git a/doc/update/5.3-to-5.4.md b/doc/update/5.3-to-5.4.md index 842e3bb6791..7277250eb32 100644 --- a/doc/update/5.3-to-5.4.md +++ b/doc/update/5.3-to-5.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.3 to 5.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.3-to-5.4.md) for the most up to date instructions.* diff --git a/doc/update/5.4-to-6.0.md b/doc/update/5.4-to-6.0.md index 44715984f0c..dacdf05cc9c 100644 --- a/doc/update/5.4-to-6.0.md +++ b/doc/update/5.4-to-6.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 5.4 to 6.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/5.4-to-6.0.md) for the most up to date instructions.* diff --git a/doc/update/6.0-to-6.1.md b/doc/update/6.0-to-6.1.md index 0c672abeb05..a3c52a1cfb4 100644 --- a/doc/update/6.0-to-6.1.md +++ b/doc/update/6.0-to-6.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.0 to 6.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.0-to-6.1.md) for the most up to date instructions.* diff --git a/doc/update/6.1-to-6.2.md b/doc/update/6.1-to-6.2.md index d3760cf0619..36a395bf01e 100644 --- a/doc/update/6.1-to-6.2.md +++ b/doc/update/6.1-to-6.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.1 to 6.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.1-to-6.2.md) for the most up to date instructions.* diff --git a/doc/update/6.2-to-6.3.md b/doc/update/6.2-to-6.3.md index 91105de2e29..02e87a08b8f 100644 --- a/doc/update/6.2-to-6.3.md +++ b/doc/update/6.2-to-6.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.2 to 6.3 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.2-to-6.3.md) for the most up to date instructions.* diff --git a/doc/update/6.3-to-6.4.md b/doc/update/6.3-to-6.4.md index 20b58ed8b25..285ed06bdad 100644 --- a/doc/update/6.3-to-6.4.md +++ b/doc/update/6.3-to-6.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.3 to 6.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.3-to-6.4.md) for the most up to date instructions.* diff --git a/doc/update/6.4-to-6.5.md b/doc/update/6.4-to-6.5.md index 5ee0f040b5d..e07c98a5ad4 100644 --- a/doc/update/6.4-to-6.5.md +++ b/doc/update/6.4-to-6.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.4 to 6.5 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.4-to-6.5.md) for the most up to date instructions.* diff --git a/doc/update/6.5-to-6.6.md b/doc/update/6.5-to-6.6.md index fa3712f83ad..3f79b19644e 100644 --- a/doc/update/6.5-to-6.6.md +++ b/doc/update/6.5-to-6.6.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.5 to 6.6 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.5-to-6.6.md) for the most up to date instructions.* diff --git a/doc/update/6.6-to-6.7.md b/doc/update/6.6-to-6.7.md index 9c85ed091c5..a0542d20d49 100644 --- a/doc/update/6.6-to-6.7.md +++ b/doc/update/6.6-to-6.7.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.6 to 6.7 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.6-to-6.7.md) for the most up to date instructions.* diff --git a/doc/update/6.7-to-6.8.md b/doc/update/6.7-to-6.8.md index 687c1265d9b..acf004577f1 100644 --- a/doc/update/6.7-to-6.8.md +++ b/doc/update/6.7-to-6.8.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.7 to 6.8 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.7-to-6.8.md) for the most up to date instructions.* diff --git a/doc/update/6.8-to-6.9.md b/doc/update/6.8-to-6.9.md index 0205b0c896a..3d7b1e5346b 100644 --- a/doc/update/6.8-to-6.9.md +++ b/doc/update/6.8-to-6.9.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.8 to 6.9 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.8-to-6.9.md) for the most up to date instructions.* diff --git a/doc/update/6.9-to-7.0.md b/doc/update/6.9-to-7.0.md index 4b6e3989893..27063948028 100644 --- a/doc/update/6.9-to-7.0.md +++ b/doc/update/6.9-to-7.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.9 to 7.0 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.9-to-7.0.md) for the most up to date instructions.* diff --git a/doc/update/6.x-or-7.x-to-7.14.md b/doc/update/6.x-or-7.x-to-7.14.md index 1e39fe47ef9..41d0e78b7d8 100644 --- a/doc/update/6.x-or-7.x-to-7.14.md +++ b/doc/update/6.x-or-7.x-to-7.14.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 6.x or 7.x to 7.14 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/6.x-or-7.x-to-7.14.md) for the most up to date instructions.* diff --git a/doc/update/7.0-to-7.1.md b/doc/update/7.0-to-7.1.md index 2e9457aa142..308e8aeb985 100644 --- a/doc/update/7.0-to-7.1.md +++ b/doc/update/7.0-to-7.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.0 to 7.1 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.0-to-7.1.md) for the most up to date instructions.* diff --git a/doc/update/7.1-to-7.2.md b/doc/update/7.1-to-7.2.md index e5045b5570f..07f92ac3af6 100644 --- a/doc/update/7.1-to-7.2.md +++ b/doc/update/7.1-to-7.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.1 to 7.2 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.1-to-7.2.md) for the most up to date instructions.* diff --git a/doc/update/7.10-to-7.11.md b/doc/update/7.10-to-7.11.md index 89213ba7178..39eeefc0e32 100644 --- a/doc/update/7.10-to-7.11.md +++ b/doc/update/7.10-to-7.11.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.10 to 7.11 ### 0. Stop server diff --git a/doc/update/7.11-to-7.12.md b/doc/update/7.11-to-7.12.md index 3865186918c..530066e5fdb 100644 --- a/doc/update/7.11-to-7.12.md +++ b/doc/update/7.11-to-7.12.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.11 to 7.12 ### 0. Double-check your Git version diff --git a/doc/update/7.12-to-7.13.md b/doc/update/7.12-to-7.13.md index 4c8d8f1f741..8f413a2079a 100644 --- a/doc/update/7.12-to-7.13.md +++ b/doc/update/7.12-to-7.13.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.12 to 7.13 ### 0. Double-check your Git version diff --git a/doc/update/7.13-to-7.14.md b/doc/update/7.13-to-7.14.md index 934898da5a1..a8980662855 100644 --- a/doc/update/7.13-to-7.14.md +++ b/doc/update/7.13-to-7.14.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.13 to 7.14 ### 0. Double-check your Git version diff --git a/doc/update/7.14-to-8.0.md b/doc/update/7.14-to-8.0.md index 25fa6d93f06..513afccff50 100644 --- a/doc/update/7.14-to-8.0.md +++ b/doc/update/7.14-to-8.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.14 to 8.0 ### 0. Double-check your Git version diff --git a/doc/update/7.2-to-7.3.md b/doc/update/7.2-to-7.3.md index d3391ddd225..a16f9de54e4 100644 --- a/doc/update/7.2-to-7.3.md +++ b/doc/update/7.2-to-7.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.2 to 7.3 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.2-to-7.3.md) for the most up to date instructions.* diff --git a/doc/update/7.3-to-7.4.md b/doc/update/7.3-to-7.4.md index 6d632dc3c8e..734c655f1d1 100644 --- a/doc/update/7.3-to-7.4.md +++ b/doc/update/7.3-to-7.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.3 to 7.4 *Make sure you view this [upgrade guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/update/7.3-to-7.4.md) for the most up to date instructions.* diff --git a/doc/update/7.4-to-7.5.md b/doc/update/7.4-to-7.5.md index ec50706d421..7a3a49ff948 100644 --- a/doc/update/7.4-to-7.5.md +++ b/doc/update/7.4-to-7.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.4 to 7.5 ### 0. Stop server diff --git a/doc/update/7.5-to-7.6.md b/doc/update/7.5-to-7.6.md index 331f5de080e..f0dfb177b79 100644 --- a/doc/update/7.5-to-7.6.md +++ b/doc/update/7.5-to-7.6.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.5 to 7.6 ### 0. Stop server diff --git a/doc/update/7.6-to-7.7.md b/doc/update/7.6-to-7.7.md index 918b10fbd95..85de6b0c546 100644 --- a/doc/update/7.6-to-7.7.md +++ b/doc/update/7.6-to-7.7.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.6 to 7.7 ### 0. Stop server diff --git a/doc/update/7.7-to-7.8.md b/doc/update/7.7-to-7.8.md index 84e0464a824..7cee5f79a13 100644 --- a/doc/update/7.7-to-7.8.md +++ b/doc/update/7.7-to-7.8.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.7 to 7.8 ### 0. Stop server diff --git a/doc/update/7.8-to-7.9.md b/doc/update/7.8-to-7.9.md index b0dc2ba1dbb..5a8b689dbc1 100644 --- a/doc/update/7.8-to-7.9.md +++ b/doc/update/7.8-to-7.9.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.8 to 7.9 ### 0. Stop server diff --git a/doc/update/7.9-to-7.10.md b/doc/update/7.9-to-7.10.md index 8f7f84b41ba..99df51dbb99 100644 --- a/doc/update/7.9-to-7.10.md +++ b/doc/update/7.9-to-7.10.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 7.9 to 7.10 ### 0. Stop server diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md index 6ee0c0656ee..f612606af68 100644 --- a/doc/update/8.0-to-8.1.md +++ b/doc/update/8.0-to-8.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.0 to 8.1 **NOTE:** GitLab 8.0 introduced several significant changes related to diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md index 4c9ff5c5c0a..2d0b19abd74 100644 --- a/doc/update/8.1-to-8.2.md +++ b/doc/update/8.1-to-8.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.1 to 8.2 **NOTE:** GitLab 8.0 introduced several significant changes related to diff --git a/doc/update/8.10-to-8.11.md b/doc/update/8.10-to-8.11.md index e538983e603..df3e34f5cc6 100644 --- a/doc/update/8.10-to-8.11.md +++ b/doc/update/8.10-to-8.11.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.10 to 8.11 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.11-to-8.12.md b/doc/update/8.11-to-8.12.md index 604166beb56..9d6a1f42375 100644 --- a/doc/update/8.11-to-8.12.md +++ b/doc/update/8.11-to-8.12.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.11 to 8.12 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.12-to-8.13.md b/doc/update/8.12-to-8.13.md index d83965131f5..6225dee9802 100644 --- a/doc/update/8.12-to-8.13.md +++ b/doc/update/8.12-to-8.13.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.12 to 8.13 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.13-to-8.14.md b/doc/update/8.13-to-8.14.md index aaadcec8ac0..d2508e3f980 100644 --- a/doc/update/8.13-to-8.14.md +++ b/doc/update/8.13-to-8.14.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.13 to 8.14 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.14-to-8.15.md b/doc/update/8.14-to-8.15.md index a68fe3bb605..daf8d0f2ca6 100644 --- a/doc/update/8.14-to-8.15.md +++ b/doc/update/8.14-to-8.15.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.14 to 8.15 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.15-to-8.16.md b/doc/update/8.15-to-8.16.md index 9f8f0f714d4..3668142edd2 100644 --- a/doc/update/8.15-to-8.16.md +++ b/doc/update/8.15-to-8.16.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.15 to 8.16 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.16-to-8.17.md b/doc/update/8.16-to-8.17.md index 74ffe0bc846..ee2e31c2aec 100644 --- a/doc/update/8.16-to-8.17.md +++ b/doc/update/8.16-to-8.17.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.16 to 8.17 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.17-to-9.0.md b/doc/update/8.17-to-9.0.md index baab217b6b7..2e0c26a9092 100644 --- a/doc/update/8.17-to-9.0.md +++ b/doc/update/8.17-to-9.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.17 to 9.0 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.2-to-8.3.md b/doc/update/8.2-to-8.3.md index 4b3c5bf6d64..3a0d647cbfe 100644 --- a/doc/update/8.2-to-8.3.md +++ b/doc/update/8.2-to-8.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.2 to 8.3 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.3-to-8.4.md b/doc/update/8.3-to-8.4.md index 8b89455ca87..f5162dd5ff5 100644 --- a/doc/update/8.3-to-8.4.md +++ b/doc/update/8.3-to-8.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.3 to 8.4 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.4-to-8.5.md b/doc/update/8.4-to-8.5.md index 0eedfaee2db..9e2f98add8d 100644 --- a/doc/update/8.4-to-8.5.md +++ b/doc/update/8.4-to-8.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.4 to 8.5 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.5-to-8.6.md b/doc/update/8.5-to-8.6.md index 851056161bb..55d8178c407 100644 --- a/doc/update/8.5-to-8.6.md +++ b/doc/update/8.5-to-8.6.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.5 to 8.6 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.6-to-8.7.md b/doc/update/8.6-to-8.7.md index 34c727260aa..49db6f2967c 100644 --- a/doc/update/8.6-to-8.7.md +++ b/doc/update/8.6-to-8.7.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.6 to 8.7 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.7-to-8.8.md b/doc/update/8.7-to-8.8.md index 6feeb1919de..ee7ec6f7614 100644 --- a/doc/update/8.7-to-8.8.md +++ b/doc/update/8.7-to-8.8.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.7 to 8.8 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.8-to-8.9.md b/doc/update/8.8-to-8.9.md index 61cdf8854d4..7508443c30a 100644 --- a/doc/update/8.8-to-8.9.md +++ b/doc/update/8.8-to-8.9.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.8 to 8.9 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/8.9-to-8.10.md b/doc/update/8.9-to-8.10.md index 42132f690d8..915e7db819a 100644 --- a/doc/update/8.9-to-8.10.md +++ b/doc/update/8.9-to-8.10.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 8.9 to 8.10 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.0-to-9.1.md b/doc/update/9.0-to-9.1.md index 6f1870a1366..f60bd92e236 100644 --- a/doc/update/9.0-to-9.1.md +++ b/doc/update/9.0-to-9.1.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.0 to 9.1 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.1-to-9.2.md b/doc/update/9.1-to-9.2.md index ce72b313031..2fff6544797 100644 --- a/doc/update/9.1-to-9.2.md +++ b/doc/update/9.1-to-9.2.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.1 to 9.2 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.2-to-9.3.md b/doc/update/9.2-to-9.3.md index 779ced0cf75..1b36cf53f4c 100644 --- a/doc/update/9.2-to-9.3.md +++ b/doc/update/9.2-to-9.3.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.2 to 9.3 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.3-to-9.4.md b/doc/update/9.3-to-9.4.md index 78d8a6c7de5..210b6eb607d 100644 --- a/doc/update/9.3-to-9.4.md +++ b/doc/update/9.3-to-9.4.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.3 to 9.4 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.4-to-9.5.md b/doc/update/9.4-to-9.5.md index a7255142ef5..1bfc1167c36 100644 --- a/doc/update/9.4-to-9.5.md +++ b/doc/update/9.4-to-9.5.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.4 to 9.5 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/9.5-to-10.0.md b/doc/update/9.5-to-10.0.md index 8581e6511f2..8d1cf0f737b 100644 --- a/doc/update/9.5-to-10.0.md +++ b/doc/update/9.5-to-10.0.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # From 9.5 to 10.0 Make sure you view this update guide from the tag (version) of GitLab you would diff --git a/doc/update/mysql_to_postgresql.md b/doc/update/mysql_to_postgresql.md index 5dc8e6f65f8..fff47180099 100644 --- a/doc/update/mysql_to_postgresql.md +++ b/doc/update/mysql_to_postgresql.md @@ -1,80 +1,267 @@ -*** NOTE: These instructions should be considered deprecated. In GitLab 10.0 we will be releasing new migration instructions using [pgloader](http://pgloader.io/). +--- +last_updated: 2017-10-05 +--- -# Migrating GitLab from MySQL to Postgres -*Make sure you view this [guide from the `master` branch](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/mysql_to_postgresql.md#migrating-gitlab-from-mysql-to-postgres) for the most up to date instructions.* +# Migrating from MySQL to PostgreSQL -If you are replacing MySQL with Postgres while keeping GitLab on the same server all you need to do is to export from MySQL, convert the resulting SQL file, and import it into Postgres. If you are also moving GitLab to another server, or if you are switching to omnibus-gitlab, you may want to use a GitLab backup file. The second part of this documents explains the procedure to do this. +> **Note:** This guide assumes you have a working Omnibus GitLab instance with +> MySQL and want to migrate to bundled PostgreSQL database. -## Export from MySQL and import into Postgres +## Prerequisites -Use this if you are keeping GitLab on the same server. +First, we'll need to enable the bundled PostgreSQL database with up-to-date +schema. Next, we'll use [pgloader](http://pgloader.io) to migrate the data +from the old MySQL database to the new PostgreSQL one. -``` -sudo service gitlab stop +Here's what you'll need to have installed: -# Update /home/git/gitlab/config/database.yml +- pgloader 3.4.1+ +- Omnibus GitLab +- MySQL -git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab -cd mysql-postgresql-converter -mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p -python db_converter.py gitlabhq_production.mysql gitlabhq_production.psql -ed -s gitlabhq_production.psql < move_drop_indexes.ed +## Enable bundled PostgreSQL database -# Import the database dump as the application database user -sudo -u git psql -f gitlabhq_production.psql -d gitlabhq_production +1. Stop GitLab: -# Install gems for PostgreSQL (note: the line below states '--without ... mysql') -sudo -u git -H bundle install --without development test mysql --deployment + ``` bash + sudo gitlab-ctl stop + ``` -sudo service gitlab start -``` +1. Edit `/etc/gitlab/gitlab.rb` to enable bundled PostgreSQL: -## Converting a GitLab backup file from MySQL to Postgres -**Note:** Please make sure to have Python 2.7.x (or higher) installed. + ``` + postgresql['enable'] = true + ``` -GitLab backup files (`<timestamp>_gitlab_backup.tar`) contain a SQL dump. Using the lanyrd database converter we can replace a MySQL database dump inside the tar file with a Postgres database dump. This can be useful if you are moving to another server. +1. Edit `/etc/gitlab/gitlab.rb` to use the bundled PostgreSQL. Please check + all the settings beginning with `db_`, such as `gitlab_rails['db_adapter']` + and alike. You could just comment all of them out so that we'll just use + the defaults. -``` -# Stop GitLab -sudo service gitlab stop +1. [Reconfigure GitLab] for the changes to take effect: + + ``` bash + sudo gitlab-ctl reconfigure + ``` + +1. Start Unicorn and PostgreSQL so that we can prepare the schema: + + ``` bash + sudo gitlab-ctl start unicorn + sudo gitlab-ctl start postgresql + ``` + +1. Run the following commands to prepare the schema: + + ``` bash + sudo gitlab-rake db:create db:migrate + ``` + +1. Stop Unicorn to prevent other database access from interfering with the loading of data: + + ``` bash + sudo gitlab-ctl stop unicorn + ``` + +After these steps, you'll have a fresh PostgreSQL database with up-to-date schema. + +## Migrate data from MySQL to PostgreSQL + +Now, you can use pgloader to migrate the data from MySQL to PostgreSQL: -# Create the backup -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +1. Save the following snippet in a `commands.load` file, and edit with your + database `username`, `password` and `host`: -# Note the filename of the backup that was created. We will call it -# TIMESTAMP_gitlab_backup.tar below. + ``` + LOAD DATABASE + FROM mysql://username:password@host/gitlabhq_production + INTO postgresql://gitlab-psql@unix://var/opt/gitlab/postgresql:/gitlabhq_production -# Move the backup file we will convert to its own directory -sudo -u git -H mkdir -p tmp/backups/postgresql -sudo -u git -H mv tmp/backups/TIMESTAMP_gitlab_backup.tar tmp/backups/postgresql/ + WITH include no drop, truncate, disable triggers, create no tables, + create no indexes, preserve index names, no foreign keys, + data only -# Create a separate database dump with PostgreSQL compatibility -cd tmp/backups/postgresql -sudo -u git -H mysqldump --compatible=postgresql --default-character-set=utf8 -r gitlabhq_production.mysql -u root gitlabhq_production -p + ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public' -# Clone the database converter -sudo -u git -H git clone https://github.com/gitlabhq/mysql-postgresql-converter.git -b gitlab + ; + ``` -# Convert gitlabhq_production.mysql -sudo -u git -H mkdir db -sudo -u git -H python mysql-postgresql-converter/db_converter.py gitlabhq_production.mysql db/database.sql -sudo -u git -H ed -s db/database.sql < mysql-postgresql-converter/move_drop_indexes.ed +1. Start the migration: -# Compress database backup -# Warning: If you have Gitlab 7.12.0 or older skip this step and import the database.sql directly into the backup with: -# sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql -# The compressed databasedump is not supported at 7.12.0 and older. -sudo -u git -H gzip db/database.sql + ``` bash + sudo -u gitlab-psql pgloader commands.load + ``` -# Replace the MySQL dump in TIMESTAMP_gitlab_backup.tar. +1. Once the migration finishes, you should see a summary table that looks like +the following: -# Warning: if you forget to replace TIMESTAMP below, tar will create a new file -# 'TIMESTAMP_gitlab_backup.tar' without giving an error. -sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql.gz + ``` + table name read imported errors total time + ----------------------------------------------- --------- --------- --------- -------------- + fetch meta data 119 119 0 0.388s + Truncate 119 119 0 1.134s + ----------------------------------------------- --------- --------- --------- -------------- + public.abuse_reports 0 0 0 0.490s + public.appearances 0 0 0 0.488s + public.approvals 0 0 0 0.273s + public.application_settings 1 1 0 0.266s + public.approvers 0 0 0 0.339s + public.approver_groups 0 0 0 0.357s + public.audit_events 1 1 0 0.410s + public.award_emoji 0 0 0 0.441s + public.boards 0 0 0 0.505s + public.broadcast_messages 0 0 0 0.498s + public.chat_names 0 0 0 0.576s + public.chat_teams 0 0 0 0.617s + public.ci_builds 0 0 0 0.611s + public.ci_group_variables 0 0 0 0.620s + public.ci_pipelines 0 0 0 0.599s + public.ci_pipeline_schedules 0 0 0 0.622s + public.ci_pipeline_schedule_variables 0 0 0 0.573s + public.ci_pipeline_variables 0 0 0 0.594s + public.ci_runners 0 0 0 0.533s + public.ci_runner_projects 0 0 0 0.584s + public.ci_sources_pipelines 0 0 0 0.564s + public.ci_stages 0 0 0 0.595s + public.ci_triggers 0 0 0 0.569s + public.ci_trigger_requests 0 0 0 0.596s + public.ci_variables 0 0 0 0.565s + public.container_repositories 0 0 0 0.605s + public.conversational_development_index_metrics 0 0 0 0.571s + public.deployments 0 0 0 0.607s + public.emails 0 0 0 0.602s + public.deploy_keys_projects 0 0 0 0.557s + public.events 160 160 0 0.677s + public.environments 0 0 0 0.567s + public.features 0 0 0 0.639s + public.events_for_migration 160 160 0 0.582s + public.feature_gates 0 0 0 0.579s + public.forked_project_links 0 0 0 0.660s + public.geo_nodes 0 0 0 0.686s + public.geo_event_log 0 0 0 0.626s + public.geo_repositories_changed_events 0 0 0 0.677s + public.geo_node_namespace_links 0 0 0 0.618s + public.geo_repository_renamed_events 0 0 0 0.696s + public.gpg_keys 0 0 0 0.704s + public.geo_repository_deleted_events 0 0 0 0.638s + public.historical_data 0 0 0 0.729s + public.geo_repository_updated_events 0 0 0 0.634s + public.index_statuses 0 0 0 0.746s + public.gpg_signatures 0 0 0 0.667s + public.issue_assignees 80 80 0 0.769s + public.identities 0 0 0 0.655s + public.issue_metrics 80 80 0 0.781s + public.issues 80 80 0 0.720s + public.labels 0 0 0 0.795s + public.issue_links 0 0 0 0.707s + public.label_priorities 0 0 0 0.793s + public.keys 0 0 0 0.734s + public.lfs_objects 0 0 0 0.812s + public.label_links 0 0 0 0.725s + public.licenses 0 0 0 0.813s + public.ldap_group_links 0 0 0 0.751s + public.members 52 52 0 0.830s + public.lfs_objects_projects 0 0 0 0.738s + public.merge_requests_closing_issues 0 0 0 0.825s + public.lists 0 0 0 0.769s + public.merge_request_diff_commits 0 0 0 0.840s + public.merge_request_metrics 0 0 0 0.837s + public.merge_requests 0 0 0 0.753s + public.merge_request_diffs 0 0 0 0.771s + public.namespaces 30 30 0 0.874s + public.merge_request_diff_files 0 0 0 0.775s + public.notes 0 0 0 0.849s + public.milestones 40 40 0 0.799s + public.oauth_access_grants 0 0 0 0.979s + public.namespace_statistics 0 0 0 0.797s + public.oauth_applications 0 0 0 0.899s + public.notification_settings 72 72 0 0.818s + public.oauth_access_tokens 0 0 0 0.807s + public.pages_domains 0 0 0 0.958s + public.oauth_openid_requests 0 0 0 0.832s + public.personal_access_tokens 0 0 0 0.965s + public.projects 8 8 0 0.987s + public.path_locks 0 0 0 0.925s + public.plans 0 0 0 0.923s + public.project_features 8 8 0 0.985s + public.project_authorizations 66 66 0 0.969s + public.project_import_data 8 8 0 1.002s + public.project_statistics 8 8 0 1.001s + public.project_group_links 0 0 0 0.949s + public.project_mirror_data 0 0 0 0.972s + public.protected_branch_merge_access_levels 0 0 0 1.017s + public.protected_branches 0 0 0 0.969s + public.protected_branch_push_access_levels 0 0 0 0.991s + public.protected_tags 0 0 0 1.009s + public.protected_tag_create_access_levels 0 0 0 0.985s + public.push_event_payloads 0 0 0 1.041s + public.push_rules 0 0 0 0.999s + public.redirect_routes 0 0 0 1.020s + public.remote_mirrors 0 0 0 1.034s + public.releases 0 0 0 0.993s + public.schema_migrations 896 896 0 1.057s + public.routes 38 38 0 1.021s + public.services 0 0 0 1.055s + public.sent_notifications 0 0 0 1.003s + public.slack_integrations 0 0 0 1.022s + public.spam_logs 0 0 0 1.024s + public.snippets 0 0 0 1.058s + public.subscriptions 0 0 0 1.069s + public.taggings 0 0 0 1.099s + public.timelogs 0 0 0 1.104s + public.system_note_metadata 0 0 0 1.038s + public.tags 0 0 0 1.034s + public.trending_projects 0 0 0 1.140s + public.uploads 0 0 0 1.129s + public.todos 80 80 0 1.085s + public.users_star_projects 0 0 0 1.153s + public.u2f_registrations 0 0 0 1.061s + public.web_hooks 0 0 0 1.179s + public.users 26 26 0 1.163s + public.user_agent_details 0 0 0 1.068s + public.web_hook_logs 0 0 0 1.080s + ----------------------------------------------- --------- --------- --------- -------------- + COPY Threads Completion 4 4 0 2.008s + Reset Sequences 113 113 0 0.304s + Install Comments 0 0 0 0.000s + ----------------------------------------------- --------- --------- --------- -------------- + Total import time 1894 1894 0 12.497s + ``` + + If there is no output for more than 30 minutes, it's possible pgloader encountered an error. See + the [troubleshooting guide](#Troubleshooting) for more details. + +1. Start GitLab: + + ``` bash + sudo gitlab-ctl start + ``` + +Now, you can verify that everything worked by visiting GitLab. + +## Troubleshooting + +### Permissions + +Note that the PostgreSQL user that you use for the above MUST have **superuser** privileges. Otherwise, you may see +a similar message to the following: -# Done! TIMESTAMP_gitlab_backup.tar can now be restored into a Postgres GitLab -# installation. -# See https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/raketasks/backup_restore.md for more information about backups. ``` +debugger invoked on a CL-POSTGRES-ERROR:INSUFFICIENT-PRIVILEGE in thread + #<THREAD "lparallel" RUNNING {10078A3513}>: + Database error 42501: permission denied: "RI_ConstraintTrigger_a_20937" is a system trigger + QUERY: ALTER TABLE ci_builds DISABLE TRIGGER ALL; + 2017-08-23T00:36:56.782000Z ERROR Database error 42501: permission denied: "RI_ConstraintTrigger_c_20864" is a system trigger + QUERY: ALTER TABLE approver_groups DISABLE TRIGGER ALL; +``` + +### Experiencing 500 errors after the migration + +If you experience 500 errors after the migration, try to clear the cache: + +``` bash +sudo gitlab-rake cache:clear +``` + +[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index b2679d1ff22..e1857ce99c6 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Universal update guide for patch versions ## Select Version to Install diff --git a/doc/update/upgrader.md b/doc/update/upgrader.md index eb7f14a96d5..746d6bf93e7 100644 --- a/doc/update/upgrader.md +++ b/doc/update/upgrader.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # GitLab Upgrader (deprecated) *DEPRECATED* We recommend to [switch to the Omnibus package and repository server](https://about.gitlab.com/update/) instead of using this script. diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index 70934f9960a..843fb4ce26b 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -18,7 +18,7 @@ traffic until the system is ready or restart the container as needed. To access monitoring resources, the client IP needs to be included in a whitelist. -[Read how to add IPs to a whitelist for the monitoring endpoints.][admin]. +[Read how to add IPs to a whitelist for the monitoring endpoints][admin]. ## Using the endpoint diff --git a/doc/user/discussions/img/discussion_lock_system_notes.png b/doc/user/discussions/img/discussion_lock_system_notes.png Binary files differnew file mode 100644 index 00000000000..8e8e8e0bc3d --- /dev/null +++ b/doc/user/discussions/img/discussion_lock_system_notes.png diff --git a/doc/user/discussions/img/image_resolved_discussion.png b/doc/user/discussions/img/image_resolved_discussion.png Binary files differnew file mode 100755 index 00000000000..ed00b5c77fe --- /dev/null +++ b/doc/user/discussions/img/image_resolved_discussion.png diff --git a/doc/user/discussions/img/lock_form_member.png b/doc/user/discussions/img/lock_form_member.png Binary files differnew file mode 100644 index 00000000000..01c6308d24c --- /dev/null +++ b/doc/user/discussions/img/lock_form_member.png diff --git a/doc/user/discussions/img/lock_form_non_member.png b/doc/user/discussions/img/lock_form_non_member.png Binary files differnew file mode 100644 index 00000000000..3bb70b69580 --- /dev/null +++ b/doc/user/discussions/img/lock_form_non_member.png diff --git a/doc/user/discussions/img/onion_skin_view.png b/doc/user/discussions/img/onion_skin_view.png Binary files differnew file mode 100755 index 00000000000..91c3b396844 --- /dev/null +++ b/doc/user/discussions/img/onion_skin_view.png diff --git a/doc/user/discussions/img/start_image_discussion.gif b/doc/user/discussions/img/start_image_discussion.gif Binary files differnew file mode 100644 index 00000000000..43efbf2fbb2 --- /dev/null +++ b/doc/user/discussions/img/start_image_discussion.gif diff --git a/doc/user/discussions/img/swipe_view.png b/doc/user/discussions/img/swipe_view.png Binary files differnew file mode 100755 index 00000000000..82d6e52173c --- /dev/null +++ b/doc/user/discussions/img/swipe_view.png diff --git a/doc/user/discussions/img/turn_off_lock.png b/doc/user/discussions/img/turn_off_lock.png Binary files differnew file mode 100644 index 00000000000..dd05b398a8b --- /dev/null +++ b/doc/user/discussions/img/turn_off_lock.png diff --git a/doc/user/discussions/img/turn_on_lock.png b/doc/user/discussions/img/turn_on_lock.png Binary files differnew file mode 100644 index 00000000000..9597da4e14d --- /dev/null +++ b/doc/user/discussions/img/turn_on_lock.png diff --git a/doc/user/discussions/img/two_up_view.png b/doc/user/discussions/img/two_up_view.png Binary files differnew file mode 100755 index 00000000000..d9e90708e87 --- /dev/null +++ b/doc/user/discussions/img/two_up_view.png diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index efea99eb120..2206b2860f4 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -153,12 +153,82 @@ comments in greater detail.  +## Image discussions + +> [Introduced][ce-14061] in GitLab 10.1. + +Sometimes a discussion is revolved around an image. With image discussions, +you can easily target a specific coordinate of an image and start a discussion +around it. Image discussions are available in merge requests and commit detail views. + +To start an image discussion, hover your mouse over the image. Your mouse pointer +should convert into an icon, indicating that the image is available for commenting. +Simply click anywhere on the image to create a new discussion. + + + +After you click on the image, a comment form will be displayed that would be the start +of your discussion. Once you save your comment, you will see a new badge displayed on +top of your image. This badge represents your discussion. + +>**Note:** +This discussion badge is typically associated with a number that is only used as a visual +reference for each discussion. In the merge request discussion tab, +this badge will be indicated with a comment icon since each discussion will render a new +image section. + +Image discussions also work on diffs that replace an existing image. In this diff view +mode, you can toggle the different view modes and still see the discussion point badges. + +| 2-up | Swipe | Onion Skin | +| :-----------: | :----------: | :----------: | +|  |  |  | + +Image discussions also work well with resolvable discussions. Resolved discussions +on diffs (not on the merge request discussion tab) will appear collapsed on page +load and will have a corresponding badge counter to match the counter on the image. + + + +## Lock discussions + +> [Introduced][ce-14531] in GitLab 10.1. + +For large projects with many contributors, it may be useful to stop discussions +in issues or merge requests in these scenarios: + +- The project maintainer has already resolved the discussion and it is not helpful +for continued feedback. The project maintainer has already directed new conversation +to newer issues or merge requests. +- The people participating in the discussion are trolling, abusive, or otherwise +being unproductive. + +In these cases, a user with Master permissions or higher in the project can lock (and unlock) +an issue or a merge request, using the "Lock" section in the sidebar: + +| Unlock | Lock | +| :-----------: | :----------: | +|  |  | + +System notes indicate locking and unlocking. + + + +In a locked issue or merge request, only team members can add new comments and +edit existing comments. Non-team members are restricted from adding or editing comments. + +| Team member | Non-team member | +| :-----------: | :----------: | +|  |  | + [ce-5022]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5022 [ce-7125]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7125 [ce-7527]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7527 [ce-7180]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7180 [ce-8266]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8266 [ce-14053]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14053 +[ce-14061]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14061 +[ce-14531]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14531 [resolve-discussion-button]: img/resolve_discussion_button.png [resolve-comment-button]: img/resolve_comment_button.png [discussion-view]: img/discussion_view.png diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 44ee994a26b..c03700a3501 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -25,6 +25,7 @@ The following table depicts the various user permission levels in a project. | Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | | View confidential issues | (✓) [^2] | ✓ | ✓ | ✓ | ✓ | | Leave comments | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | +| Lock discussions (issues and merge requests) | | | | ✓ | ✓ | | See a list of jobs | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | | See a job log | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | | Download and browse job artifacts | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | @@ -53,7 +54,7 @@ The following table depicts the various user permission levels in a project. | Create or update commit status | | | ✓ | ✓ | ✓ | | Update a container registry | | | ✓ | ✓ | ✓ | | Remove a container registry image | | | ✓ | ✓ | ✓ | -| Create new milestones | | | | ✓ | ✓ | +| Create/edit/delete project milestones | | | ✓ | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | | Push to protected branches | | | | ✓ | ✓ | | Enable/disable branch protection | | | | ✓ | ✓ | @@ -71,9 +72,11 @@ The following table depicts the various user permission levels in a project. | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | | Remove project | | | | | ✓ | +| Delete issues | | | | | ✓ | | Force push to protected branches [^4] | | | | | | | Remove protected branches [^4] | | | | | | | Remove pages | | | | | ✓ | +| Manage clusters | | | | ✓ | ✓ | ## Project features permissions @@ -141,6 +144,7 @@ group. | Manage group members | | | | | ✓ | | Remove group | | | | | ✓ | | Manage group labels | | ✓ | ✓ | ✓ | ✓ | +| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | ### Subgroup permissions diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 5ebb88bf324..5fcc0501dc1 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -52,7 +52,7 @@ You can edit your account settings by navigating from the up-right corner menu b From there, you can: - Update your personal information -- Manage [private tokens](../../api/README.md#private-tokens), email tokens, [2FA](account/two_factor_authentication.md) +- Manage [2FA](account/two_factor_authentication.md) - Change your username and [delete your account](account/delete_account.md) - Manage applications that can [use GitLab as an OAuth provider](../../integration/oauth_provider.md#introduction-to-oauth) diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index f28c034e74c..9b4fdd65e2f 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -2,17 +2,15 @@ > [Introduced][ce-3749] in GitLab 8.8. -Personal access tokens are useful if you need access to the [GitLab API][api]. -Instead of using your private token which grants full access to your account, -personal access tokens could be a better fit because of their -[granular permissions](#limiting-scopes-of-a-personal-access-token). +Personal access tokens are the preferred way for third party applications and scripts to +authenticate with the [GitLab API][api], if using [OAuth2](../../api/oauth2.md) is not practical. You can also use them to authenticate against Git over HTTP. They are the only accepted method of authentication when you have [Two-Factor Authentication (2FA)][2fa] enabled. Once you have your token, [pass it to the API][usage] using either the -`private_token` parameter or the `PRIVATE-TOKEN` header. +`private_token` parameter or the `Private-Token` header. The expiration of personal access tokens happens on the date you define, at midnight UTC. @@ -49,12 +47,14 @@ the following table. |`read_user` | Allows access to the read-only endpoints under `/users`. Essentially, any of the `GET` requests in the [Users API][users] are allowed ([introduced][ce-5951] in GitLab 8.15). | | `api` | Grants complete access to the API (read/write) ([introduced][ce-5951] in GitLab 8.15). Required for accessing Git repositories over HTTP when 2FA is enabled. | | `read_registry` | Allows to read [container registry] images if a project is private and authorization is required ([introduced][ce-11845] in GitLab 9.3). | +| `sudo` | Allows performing API actions as any user in the system (if the authenticated user is an admin) ([introduced][ce-14838] in GitLab 10.2). | [2fa]: ../account/two_factor_authentication.md [api]: ../../api/README.md [ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749 [ce-5951]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951 [ce-11845]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845 +[ce-14838]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14838 [container registry]: ../project/container_registry.md [users]: ../../api/users.md -[usage]: ../../api/README.md#basic-usage +[usage]: ../../api/README.md#personal-access-tokens diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md new file mode 100644 index 00000000000..7d9e771f570 --- /dev/null +++ b/doc/user/project/clusters/index.md @@ -0,0 +1,90 @@ +# Connecting GitLab with GKE + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in 10.1. + +CAUTION: **Warning:** +The Cluster integration is currently in **Beta**. + +Connect your project to Google Container Engine (GKE) in a few steps. + +With a cluster associated to your project, you can use Review Apps, deploy your +applications, run your pipelines, and much more in an easy way. + +NOTE: **Note:** +The Cluster integration will eventually supersede the +[Kubernetes integration](../integrations/kubernetes.md). For the moment, +you can create only one cluster. + +## Prerequisites + +In order to be able to manage your GKE cluster through GitLab, the following +prerequisites must be met: + +- The [Google authentication integration](../../../integration/google.md) must + be enabled in GitLab at the instance level. If that's not the case, ask your + administrator to enable it. +- Your associated Google account must have the right privileges to manage + clusters on GKE. That would mean that a + [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account) + must be set up. +- You must have Master [permissions] in order to be able to access the **Cluster** + page. + +If all of the above requirements are met, you can proceed to add a new cluster. + +## Adding a cluster + +NOTE: **Note:** +You need Master [permissions] and above to add a cluster. + +To add a new cluster: + +1. Navigate to your project's **CI/CD > Cluster** page. +1. Connect your Google account if you haven't done already by clicking the + "Sign-in with Google" button. +1. Fill in the requested values: + - **Cluster name** (required) - The name you wish to give the cluster. + - **GCP project ID** (required) - The ID of the project you created in your GCP + console that will host the Kubernetes cluster. This must **not** be confused + with the project name. Learn more about [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects). + - **Zone** - The zone under which the cluster will be created. Read more about + [the available zones](https://cloud.google.com/compute/docs/regions-zones/). + - **Number of nodes** - The number of nodes you wish the cluster to have. + - **Machine type** - The machine type of the Virtual Machine instance that + the cluster will be based on. Read more about [the available machine types](https://cloud.google.com/compute/docs/machine-types). + - **Project namespace** - The unique namespace for this project. By default you + don't have to fill it in; by leaving it blank, GitLab will create one for you. +1. Click the **Create cluster** button. + +After a few moments your cluster should be created. If something goes wrong, +you will be notified. + +Now, you can proceed to [enable the Cluster integration](#enabling-or-disabling-the-cluster-integration). + +## Enabling or disabling the Cluster integration + +After you have successfully added your cluster information, you can enable the +Cluster integration: + +1. Click the "Enabled/Disabled" switch +1. Hit **Save** for the changes to take effect + +You can now start using your Kubernetes cluster for your deployments. + +To disable the Cluster integration, follow the same procedure. + +## Removing the Cluster integration + +NOTE: **Note:** +You need Master [permissions] and above to remove a cluster integration. + +NOTE: **Note:** +When you remove a cluster, you only remove its relation to GitLab, not the +cluster itself. To remove the cluster, you can do so by visiting the GKE +dashboard or using `kubectl`. + +To remove the Cluster integration from your project, simply click on the +**Remove integration** button. You will then be able to follow the procedure +and [add a cluster](#adding-a-cluster) again. + +[permissions]: ../../permissions.md diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 5c615daf464..2c4dfcff4a6 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -17,25 +17,25 @@ have its own space to store its Docker images. You can read more about Docker Registry at https://docs.docker.com/registry/introduction/. ---- - ## Enable the Container Registry for your project +NOTE: **Note:** +If you cannot find the Container Registry entry under your project's settings, +that means that it is not enabled in your GitLab instance. Ask your administrator +to enable it. + 1. First, ask your system administrator to enable GitLab Container Registry following the [administration documentation](../../administration/container_registry.md). If you are using GitLab.com, this is enabled by default so you can start using the Registry immediately. - -1. Go to your project's settings and enable the **Container Registry** feature - on your project. For new projects this might be enabled by default. For - existing projects (prior GitLab 8.8), you will have to explicitly enable it. - -  - +1. Go to your [project's General settings](settings/index.md#sharing-and-permissions) + and enable the **Container Registry** feature on your project. For new + projects this might be enabled by default. For existing projects + (prior GitLab 8.8), you will have to explicitly enable it. 1. Hit **Save changes** for the changes to take effect. You should now be able - to see the **Registry** link in the project menu. + to see the **Registry** link in the sidebar. -  + ## Build and push images diff --git a/doc/user/project/img/container_registry.png b/doc/user/project/img/container_registry.png Binary files differnew file mode 100644 index 00000000000..abbaf838538 --- /dev/null +++ b/doc/user/project/img/container_registry.png diff --git a/doc/user/project/img/container_registry_enable.png b/doc/user/project/img/container_registry_enable.png Binary files differdeleted file mode 100644 index d067a8be1ca..00000000000 --- a/doc/user/project/img/container_registry_enable.png +++ /dev/null diff --git a/doc/user/project/img/container_registry_tab.png b/doc/user/project/img/container_registry_tab.png Binary files differdeleted file mode 100644 index a85237271d9..00000000000 --- a/doc/user/project/img/container_registry_tab.png +++ /dev/null diff --git a/doc/user/project/img/issue_board.png b/doc/user/project/img/issue_board.png Binary files differindex cf7f519f783..5f6dc9e4e8b 100644 --- a/doc/user/project/img/issue_board.png +++ b/doc/user/project/img/issue_board.png diff --git a/doc/user/project/img/issue_board_move_issue_card_list.png b/doc/user/project/img/issue_board_move_issue_card_list.png Binary files differindex c6b17ada40e..3666dbb87ab 100644 --- a/doc/user/project/img/issue_board_move_issue_card_list.png +++ b/doc/user/project/img/issue_board_move_issue_card_list.png diff --git a/doc/user/project/img/labels_assign_label_in_new_issue.png b/doc/user/project/img/labels_assign_label_in_new_issue.png Binary files differdeleted file mode 100644 index badfbed0bbe..00000000000 --- a/doc/user/project/img/labels_assign_label_in_new_issue.png +++ /dev/null diff --git a/doc/user/project/img/labels_default.png b/doc/user/project/img/labels_default.png Binary files differindex 474953d565b..7934e3bfb5e 100644 --- a/doc/user/project/img/labels_default.png +++ b/doc/user/project/img/labels_default.png diff --git a/doc/user/project/img/labels_filter.png b/doc/user/project/img/labels_filter.png Binary files differindex 3aca77f0070..6a1ebfc2ecb 100644 --- a/doc/user/project/img/labels_filter.png +++ b/doc/user/project/img/labels_filter.png diff --git a/doc/user/project/img/labels_filter_by_priority.png b/doc/user/project/img/labels_filter_by_priority.png Binary files differindex 5609a1f6d7f..419e555e709 100644 --- a/doc/user/project/img/labels_filter_by_priority.png +++ b/doc/user/project/img/labels_filter_by_priority.png diff --git a/doc/user/project/img/labels_new_label.png b/doc/user/project/img/labels_new_label.png Binary files differindex b44b4bd296d..e26425d0188 100644 --- a/doc/user/project/img/labels_new_label.png +++ b/doc/user/project/img/labels_new_label.png diff --git a/doc/user/project/img/labels_prioritize.png b/doc/user/project/img/labels_prioritize.png Binary files differindex 3e888f36364..d602a3c90ec 100644 --- a/doc/user/project/img/labels_prioritize.png +++ b/doc/user/project/img/labels_prioritize.png diff --git a/doc/user/project/img/project_repository_settings.png b/doc/user/project/img/project_repository_settings.png Binary files differindex 1aa7efc36f1..aa4d4452c87 100644 --- a/doc/user/project/img/project_repository_settings.png +++ b/doc/user/project/img/project_repository_settings.png diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 016f98966e3..6423beefc77 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -42,6 +42,11 @@ The importer will create any new namespaces (groups) if they don't exist or in the case the namespace is taken, the repository will be imported under the user's namespace that started the import process. +The importer will also import branches on forks of projects related to open pull +requests. These branches will be imported with a naming scheume similar to +GH-SHA-Username/Pull-Request-number/fork-name/branch. This may lead to a discrepency +in branches compared to the GitHub Repository. + ## Importing your GitHub repositories The importer page is visible when you create a new project. diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index 8da6e2a8207..e2b285678c3 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -4,7 +4,7 @@ 1. [From ClearCase](clearcase.md) 1. [From CVS](cvs.md) 1. [From FogBugz](fogbugz.md) -1. [From GitHub.com of GitHub Enterprise](github.md) +1. [From GitHub.com or GitHub Enterprise](github.md) 1. [From GitLab.com](gitlab_com.md) 1. [From Gitea](gitea.md) 1. [From Perforce](perforce.md) diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 03bbc46bd8c..97d0d529886 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -63,6 +63,8 @@ common actions on issues or merge requests browse, and download job artifacts - [Pipeline settings](pipelines/settings.md): Set up Git strategy (choose the default way your repository is fetched from GitLab in a job), timeout (defines the maximum amount of time in minutes that a job is able run), custom path for `.gitlab-ci.yml`, test coverage parsing, pipeline's visibility, and much more + - [GKE cluster integration](clusters/index.md): Connecting your GitLab project + with Google Container Engine - [GitLab Pages](pages/index.md): Build, test, and deploy your static website with GitLab Pages diff --git a/doc/user/project/integrations/img/kubernetes_configuration.png b/doc/user/project/integrations/img/kubernetes_configuration.png Binary files differindex 349a2dc8456..e535e2b8d46 100644 --- a/doc/user/project/integrations/img/kubernetes_configuration.png +++ b/doc/user/project/integrations/img/kubernetes_configuration.png diff --git a/doc/user/project/integrations/img/webhook_logs.png b/doc/user/project/integrations/img/webhook_logs.png Binary files differindex 917068d9398..803678db6b6 100755..100644 --- a/doc/user/project/integrations/img/webhook_logs.png +++ b/doc/user/project/integrations/img/webhook_logs.png diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index f4000523938..e9738b683f9 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -1,3 +1,7 @@ +--- +last_updated: 2017-09-25 +--- + # GitLab Kubernetes / OpenShift integration GitLab can be configured to interact with Kubernetes, or other systems using the @@ -6,62 +10,114 @@ Kubernetes API (such as OpenShift). Each project can be configured to connect to a different Kubernetes cluster, see the [configuration](#configuration) section. -If you have a single cluster that you want to use for all your projects, -you can pre-fill the settings page with a default template. To configure the -template, see the [Services Templates](services_templates.md) document. - ## Configuration Navigate to the [Integrations page](project_services.md#accessing-the-project-services) -of your project and select the **Kubernetes** service to configure it. +of your project and select the **Kubernetes** service to configure it. Fill in +all the needed parameters, check the "Active" checkbox and hit **Save changes** +for the changes to take effect.  -The Kubernetes service takes the following arguments: - -1. API URL -1. Custom CA bundle -1. Kubernetes namespace -1. Service token - -The API URL is the URL that GitLab uses to access the Kubernetes API. Kubernetes -exposes several APIs - we want the "base" URL that is common to all of them, -e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. - -GitLab authenticates against Kubernetes using service tokens, which are -scoped to a particular `namespace`. If you don't have a service token yet, -you can follow the -[Kubernetes documentation](http://kubernetes.io/docs/user-guide/service-accounts/) -to create one. You can also view or create service tokens in the -[Kubernetes dashboard](http://kubernetes.io/docs/user-guide/ui/) - visit -`Config -> Secrets`. - -Fill in the service token and namespace according to the values you just got. -If the API is using a self-signed TLS certificate, you'll also need to include -the `ca.crt` contents as the `Custom CA bundle`. +The Kubernetes service takes the following parameters: + +- **API URL** - + It's the URL that GitLab uses to access the Kubernetes API. Kubernetes + exposes several APIs, we want the "base" URL that is common to all of them, + e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`. +- **CA certificate** (optional) - + If the API is using a self-signed TLS certificate, you'll also need to include + the `ca.crt` contents here. +- **Project namespace** (optional) - The following apply: + - By default you don't have to fill it in; by leaving it blank, GitLab will + create one for you. + - Each project should have a unique namespace. + - The project namespace is not necessarily the namespace of the secret, if + you're using a secret with broader permissions, like the secret from `default`. + - You should **not** use `default` as the project namespace. + - If you or someone created a secret specifically for the project, usually + with limited permissions, the secret's namespace and project namespace may + be the same. +- **Token** - + GitLab authenticates against Kubernetes using service tokens, which are + scoped to a particular `namespace`. If you don't have a service token yet, + you can follow the + [Kubernetes documentation](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) + to create one. You can also view or create service tokens in the + [Kubernetes dashboard](https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#config) + (under **Config > Secrets**). + +TIP: **Tip:** +If you have a single cluster that you want to use for all your projects, +you can pre-fill the settings page with a default template. To configure the +template, see [Services Templates](services_templates.md). ## Deployment variables -The Kubernetes service exposes following +The Kubernetes service exposes the following [deployment variables](../../../ci/variables/README.md#deployment-variables) in the -GitLab CI build environment: +GitLab CI/CD build environment: -- `KUBE_URL` - equal to the API URL -- `KUBE_TOKEN` +- `KUBE_URL` - Equal to the API URL. +- `KUBE_TOKEN` - The Kubernetes token. - `KUBE_NAMESPACE` - The Kubernetes namespace is auto-generated if not specified. The default value is `<project_name>-<project_id>`. You can overwrite it to use different one if needed, otherwise the `KUBE_NAMESPACE` variable will receive the default value. -- `KUBE_CA_PEM_FILE` - only present if a custom CA bundle was specified. Path +- `KUBE_CA_PEM_FILE` - Only present if a custom CA bundle was specified. Path to a file containing PEM data. -- `KUBE_CA_PEM` (deprecated)- only if a custom CA bundle was specified. Raw PEM data. -- `KUBECONFIG` - Path to a file containing kubeconfig for this deployment. CA bundle would be embedded if specified. +- `KUBE_CA_PEM` (deprecated) - Only if a custom CA bundle was specified. Raw PEM data. +- `KUBECONFIG` - Path to a file containing `kubeconfig` for this deployment. + CA bundle would be embedded if specified. + +## What you can get with the Kubernetes integration + +Here's what you can do with GitLab if you enable the Kubernetes integration. + +### Deploy Boards (EEP) + +> Available in [GitLab Enterprise Edition Premium][ee]. -## Web terminals +GitLab's Deploy Boards offer a consolidated view of the current health and +status of each CI [environment](../../../ci/environments.md) running on Kubernetes, +displaying the status of the pods in the deployment. Developers and other +teammates can view the progress and status of a rollout, pod by pod, in the +workflow they already use without any need to access Kubernetes. ->**NOTE:** -Added in GitLab 8.15. You must be the project owner or have `master` permissions -to use terminals. Support is currently limited to the first container in the +[> Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) + +### Canary Deployments (EEP) + +> Available in [GitLab Enterprise Edition Premium][ee]. + +Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments) +and visualize your canary deployments right inside the Deploy Board, without +the need to leave GitLab. + +[> Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) + +### Kubernetes monitoring + +Automatically detect and monitor Kubernetes metrics. Automatic monitoring of +[NGINX ingress](./prometheus_library/nginx.md) is also supported. + +[> Read more about Kubernetes monitoring](prometheus_library/kubernetes.md) + +### Auto DevOps + +Auto DevOps automatically detects, builds, tests, deploys, and monitors your +applications. + +To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring) +you will need the Kubernetes project integration enabled. + +[> Read more about Auto DevOps](../../../topics/autodevops/index.md) + +### Web terminals + +NOTE: **Note:** +Introduced in GitLab 8.15. You must be the project owner or have `master` permissions +to use terminals. Support is limited to the first container in the first pod of your environment. When enabled, the Kubernetes service adds [web terminal](../../../ci/environments.md#web-terminals) @@ -70,3 +126,5 @@ Docker and Kubernetes, so you get a new shell session within your existing containers. To use this integration, you should deploy to Kubernetes using the deployment variables above, ensuring any pods you create are labelled with `app=$CI_ENVIRONMENT_SLUG`. GitLab will do the rest! + +[ee]: https://about.gitlab.com/gitlab-ee/ diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index 51989ccaaea..a0405161495 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -43,6 +43,7 @@ Click on the service links to see further configuration instructions and details | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | | [Microsoft teams](microsoft_teams.md) | Receive notifications for actions that happen on GitLab into a room on Microsoft Teams using Office 365 Connectors | +| Packagist | Update your project on Packagist, the main Composer repository | | Pipelines emails | Email the pipeline status to a list of recipients | | [Slack Notifications](slack.md) | Send GitLab events (e.g. issue created) to Slack as notifications | | [Slack slash commands](slack_slash_commands.md) | Use slash commands in Slack to control GitLab | diff --git a/doc/user/project/integrations/prometheus_library/cloudwatch.md b/doc/user/project/integrations/prometheus_library/cloudwatch.md index cc5cee36d28..34a0b97a171 100644 --- a/doc/user/project/integrations/prometheus_library/cloudwatch.md +++ b/doc/user/project/integrations/prometheus_library/cloudwatch.md @@ -1,8 +1,13 @@ # Monitoring AWS Resources + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 GitLab has support for automatically detecting and monitoring AWS resources, starting with the [Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/). This is provided by leveraging the official [Cloudwatch exporter](https://github.com/prometheus/cloudwatch_exporter), which translates [Cloudwatch metrics](https://aws.amazon.com/cloudwatch/) into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/prometheus_library/haproxy.md b/doc/user/project/integrations/prometheus_library/haproxy.md index d4b5911a91c..518018e5839 100644 --- a/doc/user/project/integrations/prometheus_library/haproxy.md +++ b/doc/user/project/integrations/prometheus_library/haproxy.md @@ -3,6 +3,10 @@ GitLab has support for automatically detecting and monitoring HAProxy. This is provided by leveraging the [HAProxy Exporter](https://github.com/prometheus/haproxy_exporter), which translates HAProxy statistics into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 4d39ae0c4fa..518683965e8 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -1,7 +1,13 @@ # Monitoring Kubernetes + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935) in GitLab 9.0 -GitLab has support for automatically detecting and monitoring Kubernetes metrics. Kubernetes exposes Node level metrics out of the box via the built-in [Prometheus metrics support in cAdvisor](https://github.com/google/cadvisor). No additional services or exporters are needed. +GitLab has support for automatically detecting and monitoring Kubernetes metrics. + +## Requirements + +The [Prometheus](../prometheus.md) and [Kubernetes](../kubernetes.md) +integration services must be enabled. ## Metrics supported @@ -23,4 +29,4 @@ Prometheus server up and running. You have two options here: In order to isolate and only display relevant metrics for a given environment however, GitLab needs a method to detect which labels are associated. To do this, GitLab will [look for an `environment` label](metrics.md#identifying-environments). -If you are using [GitLab Auto-Deploy][../../../ci/autodeploy/index.md] and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. +If you are using [GitLab Auto-Deploy](../../../../ci/autodeploy/index.md) and one of the two [provided Kubernetes monitoring solutions](../prometheus.md#getting-started-with-prometheus-monitoring), the `environment` label will be automatically added. diff --git a/doc/user/project/integrations/prometheus_library/nginx.md b/doc/user/project/integrations/prometheus_library/nginx.md index bab22f9a384..7fb8369d3c1 100644 --- a/doc/user/project/integrations/prometheus_library/nginx.md +++ b/doc/user/project/integrations/prometheus_library/nginx.md @@ -1,8 +1,13 @@ # Monitoring NGINX + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12621) in GitLab 9.4 GitLab has support for automatically detecting and monitoring NGINX. This is provided by leveraging the [NGINX VTS exporter](https://github.com/hnlq715/nginx-vts-exporter), which translates [VTS statistics](https://github.com/vozlt/nginx-module-vts) into a Prometheus readable form. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/prometheus_library/nginx_ingress.md b/doc/user/project/integrations/prometheus_library/nginx_ingress.md index 17a47cfa646..e6f13d0630b 100644 --- a/doc/user/project/integrations/prometheus_library/nginx_ingress.md +++ b/doc/user/project/integrations/prometheus_library/nginx_ingress.md @@ -1,8 +1,13 @@ # Monitoring NGINX Ingress Controller + > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13438) in GitLab 9.5 GitLab has support for automatically detecting and monitoring the Kubernetes NGINX ingress controller. This is provided by leveraging the built in Prometheus metrics included in [version 0.9.0](https://github.com/kubernetes/ingress/blob/master/controllers/nginx/Changelog.md#09-beta1) of the ingress. +## Requirements + +The [Prometheus service](../prometheus/index.md) must be enabled. + ## Metrics supported | Name | Query | diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 47eb0b34f66..5896f8f72a0 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -76,6 +76,7 @@ X-Gitlab-Event: Push Hook "user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80", "project_id": 15, "project":{ + "id": 15, "name":"Diaspora", "description":"", "web_url":"http://example.com/mike/diaspora", @@ -156,6 +157,7 @@ X-Gitlab-Event: Tag Push Hook "user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80", "project_id": 1, "project":{ + "id": 1, "name":"Example", "description":"", "web_url":"http://example.com/jsmith/example", @@ -205,7 +207,8 @@ X-Gitlab-Event: Issue Hook "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" }, - "project":{ + "project": { + "id": 1, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlabhq/gitlab-test", @@ -221,7 +224,7 @@ X-Gitlab-Event: Issue Hook "ssh_url":"git@example.com:gitlabhq/gitlab-test.git", "http_url":"http://example.com/gitlabhq/gitlab-test.git" }, - "repository":{ + "repository": { "name": "Gitlab Test", "url": "http://example.com/gitlabhq/gitlab-test.git", "description": "Aut reprehenderit ut est.", @@ -266,7 +269,37 @@ X-Gitlab-Event: Issue Hook "description": "API related issues", "type": "ProjectLabel", "group_id": 41 - }] + }], + "changes": { + "updated_by_id": [null, 1], + "updated_at": ["2017-09-15 16:50:55 UTC", "2017-09-15 16:52:00 UTC"], + "labels": { + "previous": [{ + "id": 206, + "title": "API", + "color": "#ffffff", + "project_id": 14, + "created_at": "2013-12-03T17:15:43Z", + "updated_at": "2013-12-03T17:15:43Z", + "template": false, + "description": "API related issues", + "type": "ProjectLabel", + "group_id": 41 + }], + "current": [{ + "id": 205, + "title": "Platform", + "color": "#123123", + "project_id": 14, + "created_at": "2013-12-03T17:15:43Z", + "updated_at": "2013-12-03T17:15:43Z", + "template": false, + "description": "Platform related issues", + "type": "ProjectLabel", + "group_id": 41 + }] + } + } } ``` @@ -305,6 +338,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlabhq/gitlab-test", @@ -384,6 +418,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlab-org/gitlab-test", @@ -510,6 +545,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlab-org/gitlab-test", @@ -588,6 +624,7 @@ X-Gitlab-Event: Note Hook }, "project_id": 5, "project":{ + "id": 5, "name":"Gitlab Test", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/gitlab-org/gitlab-test", @@ -661,6 +698,29 @@ X-Gitlab-Event: Merge Request Hook "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" }, + "project": { + "id": 1, + "name":"Gitlab Test", + "description":"Aut reprehenderit ut est.", + "web_url":"http://example.com/gitlabhq/gitlab-test", + "avatar_url":null, + "git_ssh_url":"git@example.com:gitlabhq/gitlab-test.git", + "git_http_url":"http://example.com/gitlabhq/gitlab-test.git", + "namespace":"GitlabHQ", + "visibility_level":20, + "path_with_namespace":"gitlabhq/gitlab-test", + "default_branch":"master", + "homepage":"http://example.com/gitlabhq/gitlab-test", + "url":"http://example.com/gitlabhq/gitlab-test.git", + "ssh_url":"git@example.com:gitlabhq/gitlab-test.git", + "http_url":"http://example.com/gitlabhq/gitlab-test.git" + }, + "repository": { + "name": "Gitlab Test", + "url": "http://example.com/gitlabhq/gitlab-test.git", + "description": "Aut reprehenderit ut est.", + "homepage": "http://example.com/gitlabhq/gitlab-test" + }, "object_attributes": { "id": 99, "target_branch": "master", @@ -679,7 +739,7 @@ X-Gitlab-Event: Merge Request Hook "target_project_id": 14, "iid": 1, "description": "", - "source":{ + "source": { "name":"Awesome Project", "description":"Aut reprehenderit ut est.", "web_url":"http://example.com/awesome_space/awesome_project", @@ -729,6 +789,48 @@ X-Gitlab-Event: Merge Request Hook "username": "user1", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" } + }, + "labels": [{ + "id": 206, + "title": "API", + "color": "#ffffff", + "project_id": 14, + "created_at": "2013-12-03T17:15:43Z", + "updated_at": "2013-12-03T17:15:43Z", + "template": false, + "description": "API related issues", + "type": "ProjectLabel", + "group_id": 41 + }], + "changes": { + "updated_by_id": [null, 1], + "updated_at": ["2017-09-15 16:50:55 UTC", "2017-09-15 16:52:00 UTC"], + "labels": { + "previous": [{ + "id": 206, + "title": "API", + "color": "#ffffff", + "project_id": 14, + "created_at": "2013-12-03T17:15:43Z", + "updated_at": "2013-12-03T17:15:43Z", + "template": false, + "description": "API related issues", + "type": "ProjectLabel", + "group_id": 41 + }], + "current": [{ + "id": 205, + "title": "Platform", + "color": "#123123", + "project_id": 14, + "created_at": "2013-12-03T17:15:43Z", + "updated_at": "2013-12-03T17:15:43Z", + "template": false, + "description": "Platform related issues", + "type": "ProjectLabel", + "group_id": 41 + }] + } } } ``` @@ -754,6 +856,7 @@ X-Gitlab-Event: Wiki Page Hook "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon" }, "project": { + "id": 1, "name": "awesome-project", "description": "This is awesome", "web_url": "http://example.com/root/awesome-project", @@ -825,6 +928,7 @@ X-Gitlab-Event: Pipeline Hook "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" }, "project":{ + "id": 1, "name": "Gitlab Test", "description": "Atque in sunt eos similique dolores voluptatem.", "web_url": "http://192.168.64.1:3005/gitlab-org/gitlab-test", @@ -1015,7 +1119,7 @@ X-Gitlab-Event: Build Hook ## Testing webhooks -You can trigger the webhook manually. Sample data from the project will be used.Sample data will take from the project. +You can trigger the webhook manually. Sample data from the project will be used.Sample data will take from the project. > For example: for triggering `Push Events` your project should have at least one commit.  @@ -1036,6 +1140,18 @@ From this page, you can repeat delivery with the same data by clicking `Resend R >**Note:** If URL or secret token of the webhook were updated, data will be delivered to the new address. +### Receiving duplicate or multiple web hook requests triggered by one event + +When GitLab sends a webhook it expects a response in 10 seconds (set default value). If it does not receive one, it'll retry the webhook. +If the endpoint doesn't send its HTTP response within those 10 seconds, GitLab may decide the hook failed and retry it. + +If you are receiving multiple requests, you can try increasing the default value to wait for the HTTP response after sending the webhook +by uncommenting or adding the following setting to your `/etc/gitlab/gitlab.rb`: + +``` +gitlab_rails['webhook_timeout'] = 10 +``` + ## Example webhook receiver If you want to see GitLab's webhooks in action for testing purposes you can use diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index e2cc67726e0..96a5a23ee13 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -12,6 +12,8 @@ Other interesting links: - [GitLab Issue Board landing page on about.gitlab.com][landing] - [YouTube video introduction to Issue Boards][youtube] + + ## Overview The Issue Board builds on GitLab's existing @@ -89,10 +91,6 @@ two defaults: - **Backlog** (default): shows all open issues that does not belong to one of lists. Always appears on the very left. - **Closed** (default): shows all closed issues. Always appears on the very right. - - ---- - In short, here's a list of actions you can take in an Issue Board: - [Create a new list](#creating-a-new-list). diff --git a/doc/user/project/issues/automatic_issue_closing.md b/doc/user/project/issues/automatic_issue_closing.md index d6f3a7d5555..402a2a3c727 100644 --- a/doc/user/project/issues/automatic_issue_closing.md +++ b/doc/user/project/issues/automatic_issue_closing.md @@ -1,8 +1,10 @@ # Automatic issue closing ->**Note:** -This is the user docs. In order to change the default issue closing pattern, -follow the steps in the [administration docs]. +>**Notes:** +> - This is the user docs. In order to change the default issue closing pattern, +> follow the steps in the [administration docs]. +> - For performance reasons, automatic issue closing is disabled for the very +> first push from an existing repository. When a commit or merge request resolves one or more issues, it is possible to automatically have these issues closed when the commit or merge request lands @@ -19,7 +21,7 @@ When not specified, the default issue closing pattern as shown below will be used: ```bash -((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing))(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) +((?:[Cc]los(?:e[sd]?|ing)|[Ff]ix(?:e[sd]|ing)?|[Rr]esolv(?:e[sd]?|ing)|[Ii]mplement(?:s|ed|ing)?)(:?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?)|([A-Z][A-Z0-9_]+-\d+))+) ``` Note that `%{issue_ref}` is a complex regular expression defined inside GitLab's @@ -34,6 +36,7 @@ This translates to the following keywords: - Close, Closes, Closed, Closing, close, closes, closed, closing - Fix, Fixes, Fixed, Fixing, fix, fixes, fixed, fixing - Resolve, Resolves, Resolved, Resolving, resolve, resolves, resolved, resolving +- Implement, Implements, Implemented, Implementing, implement, implements, implemented, implementing --- diff --git a/doc/user/project/issues/deleting_issues.md b/doc/user/project/issues/deleting_issues.md new file mode 100644 index 00000000000..d7442104c53 --- /dev/null +++ b/doc/user/project/issues/deleting_issues.md @@ -0,0 +1,11 @@ +# Deleting Issues + +> [Introduced][ce-2982] in GitLab 8.6 + +Please read through the [GitLab Issue Documentation](index.md) for an overview on GitLab Issues. + +You can delete an issue by editing it and clicking on the delete button. + + + +>**Note:** Only [project owners](../../permissions.md) can delete issues.
\ No newline at end of file diff --git a/doc/user/project/issues/img/button_close_issue.png b/doc/user/project/issues/img/button_close_issue.png Binary files differindex 8fb2e23f58a..05d257ce9bf 100755..100644 --- a/doc/user/project/issues/img/button_close_issue.png +++ b/doc/user/project/issues/img/button_close_issue.png diff --git a/doc/user/project/issues/img/closing_and_related_issues.png b/doc/user/project/issues/img/closing_and_related_issues.png Binary files differindex c6543e85fdb..c6543e85fdb 100755..100644 --- a/doc/user/project/issues/img/closing_and_related_issues.png +++ b/doc/user/project/issues/img/closing_and_related_issues.png diff --git a/doc/user/project/issues/img/confidential_issues_create.png b/doc/user/project/issues/img/confidential_issues_create.png Binary files differindex 0a141eb39f8..0a141eb39f8 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_create.png +++ b/doc/user/project/issues/img/confidential_issues_create.png diff --git a/doc/user/project/issues/img/confidential_issues_search_guest.png b/doc/user/project/issues/img/confidential_issues_search_guest.png Binary files differindex dc1b4ba8ad7..dc1b4ba8ad7 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_search_guest.png +++ b/doc/user/project/issues/img/confidential_issues_search_guest.png diff --git a/doc/user/project/issues/img/confidential_issues_search_master.png b/doc/user/project/issues/img/confidential_issues_search_master.png Binary files differindex fc01f4da9db..fc01f4da9db 100755..100644 --- a/doc/user/project/issues/img/confidential_issues_search_master.png +++ b/doc/user/project/issues/img/confidential_issues_search_master.png diff --git a/doc/user/project/issues/img/delete_issue.png b/doc/user/project/issues/img/delete_issue.png Binary files differnew file mode 100644 index 00000000000..a356f52044e --- /dev/null +++ b/doc/user/project/issues/img/delete_issue.png diff --git a/doc/user/project/issues/img/due_dates_create.png b/doc/user/project/issues/img/due_dates_create.png Binary files differindex ece35d44213..ece35d44213 100755..100644 --- a/doc/user/project/issues/img/due_dates_create.png +++ b/doc/user/project/issues/img/due_dates_create.png diff --git a/doc/user/project/issues/img/due_dates_edit_sidebar.png b/doc/user/project/issues/img/due_dates_edit_sidebar.png Binary files differindex d1c7d1eb7e9..d1c7d1eb7e9 100755..100644 --- a/doc/user/project/issues/img/due_dates_edit_sidebar.png +++ b/doc/user/project/issues/img/due_dates_edit_sidebar.png diff --git a/doc/user/project/issues/img/due_dates_issues_index_page.png b/doc/user/project/issues/img/due_dates_issues_index_page.png Binary files differindex 94679436b32..94679436b32 100755..100644 --- a/doc/user/project/issues/img/due_dates_issues_index_page.png +++ b/doc/user/project/issues/img/due_dates_issues_index_page.png diff --git a/doc/user/project/issues/img/due_dates_todos.png b/doc/user/project/issues/img/due_dates_todos.png Binary files differindex 4c124c97f67..4c124c97f67 100755..100644 --- a/doc/user/project/issues/img/due_dates_todos.png +++ b/doc/user/project/issues/img/due_dates_todos.png diff --git a/doc/user/project/issues/img/group_issues_list_view.png b/doc/user/project/issues/img/group_issues_list_view.png Binary files differindex 5d20e8cbc89..bba964076d0 100644 --- a/doc/user/project/issues/img/group_issues_list_view.png +++ b/doc/user/project/issues/img/group_issues_list_view.png diff --git a/doc/user/project/issues/img/issue_board.png b/doc/user/project/issues/img/issue_board.png Binary files differindex 1759b28a9ef..87b1016cc76 100755..100644 --- a/doc/user/project/issues/img/issue_board.png +++ b/doc/user/project/issues/img/issue_board.png diff --git a/doc/user/project/issues/img/issue_template.png b/doc/user/project/issues/img/issue_template.png Binary files differindex c63229a4af2..0e4c8df897b 100755..100644 --- a/doc/user/project/issues/img/issue_template.png +++ b/doc/user/project/issues/img/issue_template.png diff --git a/doc/user/project/issues/img/issues_main_view.png b/doc/user/project/issues/img/issues_main_view.png Binary files differindex 4faa42e40ee..a929916c682 100644 --- a/doc/user/project/issues/img/issues_main_view.png +++ b/doc/user/project/issues/img/issues_main_view.png diff --git a/doc/user/project/issues/img/issues_main_view_numbered.jpg b/doc/user/project/issues/img/issues_main_view_numbered.jpg Binary files differindex 4b5d7fba459..b4b68476d24 100644 --- a/doc/user/project/issues/img/issues_main_view_numbered.jpg +++ b/doc/user/project/issues/img/issues_main_view_numbered.jpg diff --git a/doc/user/project/issues/img/mention_in_issue.png b/doc/user/project/issues/img/mention_in_issue.png Binary files differindex c762a812138..c762a812138 100755..100644 --- a/doc/user/project/issues/img/mention_in_issue.png +++ b/doc/user/project/issues/img/mention_in_issue.png diff --git a/doc/user/project/issues/img/mention_in_merge_request.png b/doc/user/project/issues/img/mention_in_merge_request.png Binary files differindex 681e086d6e0..681e086d6e0 100755..100644 --- a/doc/user/project/issues/img/mention_in_merge_request.png +++ b/doc/user/project/issues/img/mention_in_merge_request.png diff --git a/doc/user/project/issues/img/merge_request_closes_issue.png b/doc/user/project/issues/img/merge_request_closes_issue.png Binary files differindex 6fd27738843..6fd27738843 100755..100644 --- a/doc/user/project/issues/img/merge_request_closes_issue.png +++ b/doc/user/project/issues/img/merge_request_closes_issue.png diff --git a/doc/user/project/issues/img/new_issue.png b/doc/user/project/issues/img/new_issue.png Binary files differindex e72ac49d6b9..07d65a93070 100755..100644 --- a/doc/user/project/issues/img/new_issue.png +++ b/doc/user/project/issues/img/new_issue.png diff --git a/doc/user/project/issues/img/new_issue_from_issue_board.png b/doc/user/project/issues/img/new_issue_from_issue_board.png Binary files differindex 9c2b3ff50fa..da892eff0a6 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_issue_board.png +++ b/doc/user/project/issues/img/new_issue_from_issue_board.png diff --git a/doc/user/project/issues/img/new_issue_from_open_issue.png b/doc/user/project/issues/img/new_issue_from_open_issue.png Binary files differindex 2aed5372830..c6f3f0617ab 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_open_issue.png +++ b/doc/user/project/issues/img/new_issue_from_open_issue.png diff --git a/doc/user/project/issues/img/new_issue_from_projects_dashboard.png b/doc/user/project/issues/img/new_issue_from_projects_dashboard.png Binary files differindex cddf36b7457..4b9535f6b15 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_projects_dashboard.png +++ b/doc/user/project/issues/img/new_issue_from_projects_dashboard.png diff --git a/doc/user/project/issues/img/new_issue_from_tracker_list.png b/doc/user/project/issues/img/new_issue_from_tracker_list.png Binary files differindex 7e5413f0b7d..66793cb44fa 100755..100644 --- a/doc/user/project/issues/img/new_issue_from_tracker_list.png +++ b/doc/user/project/issues/img/new_issue_from_tracker_list.png diff --git a/doc/user/project/issues/img/project_issues_list_view.png b/doc/user/project/issues/img/project_issues_list_view.png Binary files differindex 2fcc9e8d9da..584a81aab8a 100644 --- a/doc/user/project/issues/img/project_issues_list_view.png +++ b/doc/user/project/issues/img/project_issues_list_view.png diff --git a/doc/user/project/issues/img/sidebar_confidential_issue.png b/doc/user/project/issues/img/sidebar_confidential_issue.png Binary files differindex d99a1ca756e..d99a1ca756e 100755..100644 --- a/doc/user/project/issues/img/sidebar_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_confidential_issue.png diff --git a/doc/user/project/issues/img/sidebar_move_issue.png b/doc/user/project/issues/img/sidebar_move_issue.png Binary files differindex 111f7861364..1e688cec894 100644 --- a/doc/user/project/issues/img/sidebar_move_issue.png +++ b/doc/user/project/issues/img/sidebar_move_issue.png diff --git a/doc/user/project/issues/img/sidebar_not_confidential_issue.png b/doc/user/project/issues/img/sidebar_not_confidential_issue.png Binary files differindex 2e6cbbc5b3a..2e6cbbc5b3a 100755..100644 --- a/doc/user/project/issues/img/sidebar_not_confidential_issue.png +++ b/doc/user/project/issues/img/sidebar_not_confidential_issue.png diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 0f187946a4a..3e81dcb78c6 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -90,6 +90,10 @@ Learn distinct ways to [close issues](closing_issues.md) in GitLab. Read through the [documentation on moving issues](moving_issues.md). +## Deleting issues + +Read through the [documentation on deleting issues](deleting_issues.md) + ## Create a merge request from an issue Learn more about it on the [GitLab Issues Functionalities documentation](issues_functionalities.md#18-new-merge-request). diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 8ec7adad172..21a2e1213ec 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -20,8 +20,6 @@ Head over a single project and navigate to **Issues > Labels**. The first time you visit this page, you'll notice that there are no labels created yet. - - Creating a new label from scratch is as easy as pressing the **New label** button. From there on you can choose the name, give it an optional description, a color and you are set. @@ -32,21 +30,23 @@ When you are ready press the **Create label** button to create the new label. --- -## Default Labels - -It's possible to populate the labels for your project from a set of predefined labels. - -### Generate GitLab's predefined label set +## Default labels - +The very first time you visit the labels area, it's gonna be empty. In that +case, it's possible to populate the labels for your project from a set of +predefined labels. Click the link to 'Generate a default set of labels' and GitLab will -generate a set of predefined labels for you. There are 8 default generated labels -in total and you can see them in the screenshot below. - - +generate them for you. There are 8 default generated labels in total: ---- +- bug +- confirmed +- critical +- discussion +- documentation +- enhancement +- suggestion +- support ## Labels Overview @@ -102,30 +102,25 @@ If you work on a large or popular project, try subscribing only to the labels that are relevant to you. You’ll notice it’ll be much easier to focus on what’s important. -## Create a new label right from the issue tracker - -> Introduced in GitLab 8.6. +## Create a new label when inside an issue -There are times when you are already in the issue tracker searching for a +There are times when you are already inside an issue searching to assign a label, only to realize it doesn't exist. Instead of going to the **Labels** page and being distracted from your original purpose, you can create new labels on the fly. -Select **Create new** from the labels dropdown list, provide a name, pick a -color and hit **Create**. +Expand the issue sidebar and select **Create new label** from the labels dropdown +list. Provide a name, pick a color and hit **Create**. The new label will be +ready to used right away! -  ## Assigning labels to issues and merge requests There are generally two ways to assign a label to an issue or merge request. -You can assign a label when you first create or edit an issue or merge request. - - - ---- +The first one is to assign a label when you first create or edit an issue or +merge request. The second way is by using the right sidebar when inside an issue or merge request. Expand it and hit **Edit** in the labels area. Start typing the name diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md index 64b94d81024..22ef11e4049 100644 --- a/doc/user/project/merge_requests/cherry_pick_changes.md +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -2,24 +2,19 @@ > [Introduced][ce-3514] in GitLab 8.7. ---- - GitLab implements Git's powerful feature to [cherry-pick any commit][git-cherry-pick] -with introducing a **Cherry-pick** button in Merge Requests and commit details. +with introducing a **Cherry-pick** button in merge requests and commit details. -## Cherry-picking a Merge Request +## Cherry-picking a merge request -After the Merge Request has been merged, a **Cherry-pick** button will be available -to cherry-pick the changes introduced by that Merge Request: +After the merge request has been merged, a **Cherry-pick** button will be available +to cherry-pick the changes introduced by that merge request.  ---- - -You can cherry-pick the changes directly into the selected branch or you can opt to -create a new Merge Request with the cherry-pick changes: - - +After you click that button, a modal will appear where you can choose to +cherry-pick the changes directly into the selected branch or you can opt to +create a new merge request with the cherry-pick changes ## Cherry-picking a Commit @@ -27,15 +22,9 @@ You can cherry-pick a Commit from the Commit details page:  ---- - -Similar to cherry-picking a Merge Request, you can opt to cherry-pick the changes -directly into the target branch or create a new Merge Request to cherry-pick the -changes: - - - ---- +Similar to cherry-picking a merge request, you can opt to cherry-pick the changes +directly into the target branch or create a new merge request to cherry-pick the +changes. Please note that when cherry-picking merge commits, the mainline will always be the first parent. If you want to use a different mainline then you need to do that diff --git a/doc/user/project/merge_requests/fast_forward_merge.md b/doc/user/project/merge_requests/fast_forward_merge.md new file mode 100644 index 00000000000..085170d9f03 --- /dev/null +++ b/doc/user/project/merge_requests/fast_forward_merge.md @@ -0,0 +1,35 @@ +# Fast-forward merge requests + +Retain a linear Git history and a way to accept merge requests without +creating merge commits. + +## Overview + +When the fast-forward merge ([`--ff-only`][ffonly]) setting is enabled, no merge +commits will be created and all merges are fast-forwarded, which means that +merging is only allowed if the branch could be fast-forwarded. + +When a fast-forward merge is not possible, the user must rebase the branch manually. + +## Use cases + +Sometimes, a workflow policy might mandate a clean commit history without +merge commits. In such cases, the fast-forward merge is the perfect candidate. + +## Enabling fast-forward merges + +1. Navigate to your project's **Settings** and search for the 'Merge method' +1. Select the **Fast-forward merge** option +1. Hit **Save changes** for the changes to take effect + +Now, when you visit the merge request page, you will be able to accept it +**only if a fast-forward merge is possible**. + + + +If the target branch is ahead of the source branch, you need to rebase the +source branch locally before you will be able to do a fast-forward merge. + + + +[ffonly]: https://git-scm.com/docs/git-merge#git-merge---ff-only diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png b/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png Binary files differindex 5ab094ab367..7dc344f8cf6 100644 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_commit.png diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png b/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png Binary files differdeleted file mode 100644 index 42dcb9203ec..00000000000 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_commit_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png b/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png Binary files differindex 71227747182..811b0998f85 100644 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png +++ b/doc/user/project/merge_requests/img/cherry_pick_changes_mr.png diff --git a/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png b/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png Binary files differdeleted file mode 100644 index 604eb22f51c..00000000000 --- a/doc/user/project/merge_requests/img/cherry_pick_changes_mr_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/commit_compare.png b/doc/user/project/merge_requests/img/commit_compare.png Binary files differdeleted file mode 100644 index e612a39716e..00000000000 --- a/doc/user/project/merge_requests/img/commit_compare.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/ff_merge_mr.png b/doc/user/project/merge_requests/img/ff_merge_mr.png Binary files differnew file mode 100644 index 00000000000..241cc990343 --- /dev/null +++ b/doc/user/project/merge_requests/img/ff_merge_mr.png diff --git a/doc/user/project/merge_requests/img/ff_merge_rebase_locally.png b/doc/user/project/merge_requests/img/ff_merge_rebase_locally.png Binary files differnew file mode 100644 index 00000000000..fb412296efc --- /dev/null +++ b/doc/user/project/merge_requests/img/ff_merge_rebase_locally.png diff --git a/doc/user/project/merge_requests/img/merge_request.png b/doc/user/project/merge_requests/img/merge_request.png Binary files differnew file mode 100644 index 00000000000..f9ca6348953 --- /dev/null +++ b/doc/user/project/merge_requests/img/merge_request.png diff --git a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png Binary files differindex 33f5a4a7a02..d7f0535d3c5 100644 --- a/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png +++ b/doc/user/project/merge_requests/img/merge_when_pipeline_succeeds_enable.png diff --git a/doc/user/project/merge_requests/img/revert_changes_commit_modal.png b/doc/user/project/merge_requests/img/revert_changes_commit_modal.png Binary files differdeleted file mode 100644 index ef7b6dae553..00000000000 --- a/doc/user/project/merge_requests/img/revert_changes_commit_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/revert_changes_mr_modal.png b/doc/user/project/merge_requests/img/revert_changes_mr_modal.png Binary files differdeleted file mode 100644 index f6540c9dd33..00000000000 --- a/doc/user/project/merge_requests/img/revert_changes_mr_modal.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/versions.png b/doc/user/project/merge_requests/img/versions.png Binary files differindex 33c58d2abff..3883fb4bc1c 100644 --- a/doc/user/project/merge_requests/img/versions.png +++ b/doc/user/project/merge_requests/img/versions.png diff --git a/doc/user/project/merge_requests/img/versions_compare.png b/doc/user/project/merge_requests/img/versions_compare.png Binary files differindex db978ea7b1d..f5bd85dc7c1 100644 --- a/doc/user/project/merge_requests/img/versions_compare.png +++ b/doc/user/project/merge_requests/img/versions_compare.png diff --git a/doc/user/project/merge_requests/img/versions_dropdown.png b/doc/user/project/merge_requests/img/versions_dropdown.png Binary files differindex 889a2d93e6c..cc70a5bf14b 100644 --- a/doc/user/project/merge_requests/img/versions_dropdown.png +++ b/doc/user/project/merge_requests/img/versions_dropdown.png diff --git a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png Binary files differindex 047b0b4620f..0c492aca363 100644 --- a/doc/user/project/merge_requests/img/wip_blocked_accept_button.png +++ b/doc/user/project/merge_requests/img/wip_blocked_accept_button.png diff --git a/doc/user/project/merge_requests/img/wip_mark_as_wip.png b/doc/user/project/merge_requests/img/wip_mark_as_wip.png Binary files differindex 8bd206bc24a..e405879b28a 100644 --- a/doc/user/project/merge_requests/img/wip_mark_as_wip.png +++ b/doc/user/project/merge_requests/img/wip_mark_as_wip.png diff --git a/doc/user/project/merge_requests/img/wip_unmark_as_wip.png b/doc/user/project/merge_requests/img/wip_unmark_as_wip.png Binary files differindex c0bfa6a35a2..d7f8c419945 100644 --- a/doc/user/project/merge_requests/img/wip_unmark_as_wip.png +++ b/doc/user/project/merge_requests/img/wip_unmark_as_wip.png diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 26c6277d33a..4b2e042251b 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -3,6 +3,8 @@ Merge requests allow you to exchange changes you made to source code and collaborate with other people on the same project. + + ## Overview A Merge Request (**MR**) is the basis of GitLab as a code collaboration @@ -23,14 +25,15 @@ With GitLab merge requests, you can: - Organize your issues and merge requests consistently throughout the project with [labels](../../project/labels.md) - Add a time estimation and the time spent with that merge request with [Time Tracking](../../../workflow/time_tracking.html#time-tracking) - [Resolve merge conflicts from the UI](#resolve-conflicts) +- Enable [fast-forward merge requests](#fast-forward-merge-requests) +- Enable [semi-linear history merge requests](#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch + With **[GitLab Enterprise Edition][ee]**, you can also: - View the deployment process across projects with [Multi-Project Pipeline Graphs](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html#multi-project-pipeline-graphs) (available only in GitLab Enterprise Edition Premium) - Request [approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers (available in GitLab Enterprise Edition Starter) -- Enable [fast-forward merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/fast_forward_merge.html) (available in GitLab Enterprise Edition Starter) - [Squash and merge](https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html) for a cleaner commit history (available in GitLab Enterprise Edition Starter) -- Enable [semi-linear history merge requests](https://docs.gitlab.com/ee/user/project/merge_requests/index.html#semi-linear-history-merge-requests) as another security layer to guarantee the pipeline is passing in the target branch (available in GitLab Enterprise Edition Starter) - Analise the impact of your changes with [Code Quality reports](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html) (available in GitLab Enterprise Edition Starter) ## Use cases @@ -89,6 +92,22 @@ in a merged merge requests or a commit. [Learn more about cherry-picking changes.](cherry_pick_changes.md) +## Semi-linear history merge requests + +A merge commit is created for every merge, but the branch is only merged if +a fast-forward merge is possible. This ensures that if the merge request build +succeeded, the target branch build will also succeed after merging. + +Navigate to a project's settings, select the **Merge commit with semi-linear +history** option under **Merge Requests: Merge method** and save your changes. + +## Fast-forward merge requests + +If you prefer a linear Git history and a way to accept merge requests without +creating merge commits, you can configure this on a per-project basis. + +[Read more about fast-forward merge requests.](fast_forward_merge.md) + ## Merge when pipeline succeeds When reviewing a merge request that looks ready to merge but still has one or @@ -254,4 +273,4 @@ git checkout origin/merge-requests/1 ``` [protected branches]: ../protected_branches.md -[ee]: https://about.gitlab.com/gitlab-ee/ "GitLab Enterprise Edition"
\ No newline at end of file +[ee]: https://about.gitlab.com/gitlab-ee/ "GitLab Enterprise Edition" diff --git a/doc/user/project/merge_requests/revert_changes.md b/doc/user/project/merge_requests/revert_changes.md index 5ead9f4177f..8cf8a59dbfe 100644 --- a/doc/user/project/merge_requests/revert_changes.md +++ b/doc/user/project/merge_requests/revert_changes.md @@ -2,51 +2,39 @@ > [Introduced][ce-1990] in GitLab 8.5. ---- - GitLab implements Git's powerful feature to [revert any commit][git-revert] -with introducing a **Revert** button in Merge Requests and commit details. +with introducing a **Revert** button in merge requests and commit details. ## Reverting a Merge Request -_**Note:** The **Revert** button will only be available for Merge Requests -created since GitLab 8.5. However, you can still revert a Merge Request -by reverting the merge commit from the list of Commits page._ +NOTE: **Note:** +The **Revert** button will only be available for merge requests +created since GitLab 8.5. However, you can still revert a merge request +by reverting the merge commit from the list of Commits page. After the Merge Request has been merged, a **Revert** button will be available -to revert the changes introduced by that Merge Request: - - - ---- - -You can revert the changes directly into the selected branch or you can opt to -create a new Merge Request with the revert changes: +to revert the changes introduced by that merge request. - + ---- +After you click that button, a modal will appear where you can choose to +revert the changes directly into the selected branch or you can opt to +create a new merge request with the revert changes. -After the Merge Request has been reverted, the **Revert** button will not be +After the merge request has been reverted, the **Revert** button will not be available anymore. ## Reverting a Commit You can revert a Commit from the Commit details page: - - ---- - -Similar to reverting a Merge Request, you can opt to revert the changes -directly into the target branch or create a new Merge Request to revert the -changes: - - + ---- +Similar to reverting a merge request, you can opt to revert the changes +directly into the target branch or create a new merge request to revert the +changes. -After the Commit has been reverted, the **Revert** button will not be available +After the commit has been reverted, the **Revert** button will not be available anymore. Please note that when reverting merge commits, the mainline will always be the diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 876b98a4dc5..83adbd8cce2 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -29,7 +29,8 @@ In addition to that you will be able to filter issues or merge requests by group ## Milestone promotion -You will be able to promote a project milestone to a group milestone [in the future](https://gitlab.com/gitlab-org/gitlab-ce/issues/35833). +Project milestones can be promoted to group milestones if its project belongs to a group. When a milestone is promoted all other milestones across the group projects with the same title will be merged into it, which means all milestone's children like issues, merge requests and boards will be moved into the new promoted milestone. +The promote button can be found in the milestone view or milestones list. ## Special milestone filters diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 46fa4378fe7..453e10184f0 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -62,7 +62,7 @@ which is highly recommendable and much faster than hardcoding. If you set up a GitLab Pages project on GitLab.com, it will automatically be accessible under a -[subdomain of `namespace.pages.io`](https://docs.gitlab.com/ce/user/project/pages/). +[subdomain of `namespace.pages.io`](introduction.md#gitlab-pages-on-gitlab-com). The `namespace` is defined by your username on GitLab.com, or the group name you created this project under. @@ -73,6 +73,8 @@ Pages wildcard domain. This guide is valid for any GitLab instance, you just need to replace Pages wildcard domain on GitLab.com (`*.gitlab.io`) with your own. +Learn more about [namespaces](../../group/index.md#namespaces). + ### Practical examples #### Project Websites diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index 53fd1786cfa..0096f8507d2 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -1,9 +1,14 @@ +--- +last_updated: 2017-09-28 +--- + # GitLab Pages from A to Z: Part 3 -> **Article [Type](../../../development/writing_documentation.html#types-of-technical-articles)**: user guide || +> **[Article Type](../../../development/writing_documentation.md#types-of-technical-articles)**: user guide || > **Level**: beginner || > **Author**: [Marcia Ramos](https://gitlab.com/marcia) || -> **Publication date:** 2017/02/22 +> **Publication date:** 2017-02-22 || +> **Last updated**: 2017-09-28 - [Part 1: Static sites and GitLab Pages domains](getting_started_part_one.md) - [Part 2: Quick start guide - Setting up GitLab Pages](getting_started_part_two.md) @@ -16,6 +21,21 @@ As described in the previous part of this series, setting up GitLab Pages with c These steps assume you've already [set your site up](getting_started_part_two.md) and and it's served under the default Pages domain `namespace.gitlab.io`, or `namespace.gitlab.io/project-name`. +### Adding your custom domain to GitLab Pages + +To use one or more custom domain with your Pages site, there are two things +you should consider first, which we'll cover in this guide: + +1. Either if you're adding a **root domain** or a **subdomain**, for which +you'll need to set up [DNS records](#dns-records) +1. Whether you want to add an [SSL/TLS certificate](#ssl-tls-certificates) or not + +To finish the association, you need to [add your domain to your project's Pages settings](#add-your-custom-domain-to-gitlab-pages-settings). + +Let's start from the beginning with [DNS records](#dns-records). +If you already know how they work and want to skip the introduction to DNS, +you may be interested in skipping it until the [TL;DR](#tl-dr) section below. + ### DNS Records A Domain Name System (DNS) web service routes visitors to websites @@ -99,6 +119,29 @@ domain. E.g., **do not** point your `subdomain.domain.com` to `namespace.gitlab.io.` or `namespace.gitlab.io/`. > - GitLab Pages IP on GitLab.com [has been changed](https://about.gitlab.com/2017/03/06/we-are-changing-the-ip-of-gitlab-pages-on-gitlab-com/) from `104.208.235.32` to `52.167.214.135`. +### Add your custom domain to GitLab Pages settings + +Once you've set the DNS record, you'll need navigate to your project's +**Setting > Pages** and click **+ New domain** to add your custom domain to +GitLab Pages. You can choose whether to add an [SSL/TLS certificate](#ssl-tls-certificates) +to make your website accessible under HTTPS or leave it blank. If don't add a certificate, +your site will be accessible only via HTTP: + + + +You can add more than one alias (custom domains and subdomains) to the same project. +An alias can be understood as having many doors leading to the same room. + +All the aliases you've set to your site will be listed on **Setting > Pages**. +From that page, you can view, add, and remove them. + +Note that [DNS propagation may take some time (up to 24h)](http://www.inmotionhosting.com/support/domain-names/dns-nameserver-changes/domain-names-dns-changes), +although it's usually a matter of minutes to complete. Until it does, visit attempts +to your domain will respond with a 404. + +Read through the [general documentation on GitLab Pages](introduction.md#add-a-custom-domain-to-your-pages-website) to learn more about adding +custom domains to GitLab Pages sites. + ### SSL/TLS Certificates Every GitLab Pages project on GitLab.com will be available under diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 9ecf7a3a8e7..3ab88948fbd 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -3,7 +3,7 @@ > **Notes:** > - This feature was [introduced][ee-80] in GitLab EE 8.3. > - Custom CNAMEs with TLS support were [introduced][ee-173] in GitLab EE 8.5. -> - GitLab Pages [were ported][ce-14605] to Community Edition in GitLab 8.17. +> - GitLab Pages [was ported][ce-14605] to Community Edition in GitLab 8.17. > - This document is about the user guide. To learn how to enable GitLab Pages > across your GitLab instance, visit the [administrator documentation](../../../administration/pages/index.md). @@ -28,7 +28,8 @@ In general there are two types of pages one might create: - Pages per project (`username.example.io/projectname` or `groupname.example.io/projectname`) In GitLab, usernames and groupnames are unique and we often refer to them -as namespaces. There can be only one namespace in a GitLab instance. Below you +as [namespaces](../../group/index.md#namespaces). There can be only one namespace +in a GitLab instance. Below you can see the connection between the type of GitLab Pages, what the project name that is created on GitLab looks like and the website URL it will be ultimately be served on. @@ -98,6 +99,9 @@ The steps to create a project page for a user or a group are identical: A user's project will be served under `http(s)://username.example.io/projectname` whereas a group's project under `http(s)://groupname.example.io/projectname`. +For practical examples for group and project Pages, read through the guide +[GitLab Pages from A to Z: Part 1 - Static sites and GitLab Pages domains](getting_started_part_one.md#practical-examples). + ## Quick Start Read through [GitLab Pages Quick Start Guide][pages-quick] or watch the video tutorial on @@ -111,6 +115,9 @@ The key thing about GitLab Pages is the `.gitlab-ci.yml` file, something that gives you absolute control over the build process. You can actually watch your website being built live by following the CI job traces. +For a simplified user guide on setting up GitLab CI/CD for Pages, read through +the article [GitLab Pages from A to Z: Part 4 - Creating and Tweaking `.gitlab-ci.yml` for GitLab Pages](getting_started_part_four.md#creating-and-tweaking-gitlab-ci-yml-for-gitlab-pages) + > **Note:** > Before reading this section, make sure you familiarize yourself with GitLab CI > and the specific syntax of[`.gitlab-ci.yml`][yaml] by @@ -167,7 +174,7 @@ job, the contents of the `public` directory will be served by GitLab Pages. #### How `.gitlab-ci.yml` looks like when the static content is in your repository -Supposedly your repository contained the following files: +Supposed your repository contained the following files: ``` ├── index.html @@ -311,6 +318,9 @@ Visit the GitLab Pages group for a full list of example projects: ### Add a custom domain to your Pages website +For a complete guide on Pages domains, read through the article +[GitLab Pages from A to Z: Part 3 - Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](getting_started_part_three.md#setting-up-custom-domains-dns-records-and-ssl-tls-certificates) + If this setting is enabled by your GitLab administrator, you should be able to see the **New Domain** button when visiting your project's settings through the gear icon in the top right and then navigating to **Pages**. @@ -349,6 +359,9 @@ private key when adding a new domain.  +For a complete guide on Pages domains, read through the article +[GitLab Pages from A to Z: Part 3 - Setting Up Custom Domains - DNS Records and SSL/TLS Certificates](getting_started_part_three.md#setting-up-custom-domains-dns-records-and-ssl-tls-certificates) + ### Custom error codes pages You can provide your own 403 and 404 error pages by creating the `403.html` and @@ -387,6 +400,8 @@ If you are using GitLab.com to host your website, then: The rest of the guide still applies. +See also: [GitLab Pages from A to Z: Part 1 - Static sites and GitLab Pages domains](getting_started_part_one.md#gitlab-pages-domain). + ## Limitations When using Pages under the general domain of a GitLab instance (`*.example.io`), @@ -404,7 +419,7 @@ You can only create the highest level group website. ## Redirects in GitLab Pages Since you cannot use any custom server configuration files, like `.htaccess` or -any `.conf` file for that matter, if you want to redirect a web page to another +any `.conf` file, if you want to redirect a page to another location, you can use the [HTTP meta refresh tag][metarefresh]. Some static site generators provide plugins for that functionality so that you @@ -419,7 +434,7 @@ Sure. All you need to do is download the artifacts archive from the job page. ### Can I use GitLab Pages if my project is private? -Yes. GitLab Pages don't care whether you set your project's visibility level +Yes. GitLab Pages doesn't care whether you set your project's visibility level to private, internal or public. ### Do I need to create a user/group website before creating a project website? diff --git a/doc/user/project/pipelines/img/job_artifacts_browser.png b/doc/user/project/pipelines/img/job_artifacts_browser.png Binary files differindex 145fe156bbb..d3d8de5ac60 100644 --- a/doc/user/project/pipelines/img/job_artifacts_browser.png +++ b/doc/user/project/pipelines/img/job_artifacts_browser.png diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index 4e93e680fd2..f9a268fb789 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -50,6 +50,11 @@ For more examples on artifacts, follow the [artifacts reference in With GitLab 9.2, PDFs, images, videos and other formats can be previewed directly in the job artifacts browser without the need to download them. +>**Note:** +With [GitLab 10.1][ce-14399], HTML files in a public project can be previewed +directly in a new tab without the need to download them when +[GitLab Pages](../../../administration/pages/index.md) is enabled + After a job finishes, if you visit the job's specific page, there are three buttons. You can download the artifacts archive or browse its contents, whereas the **Keep** button appears only if you have set an [expiry date] to the @@ -64,7 +69,9 @@ archive. If your artifacts contained directories, then you are also able to browse inside them. Below you can see how browsing looks like. In this case we have browsed inside -the archive and at this point there is one directory and one HTML file. +the archive and at this point there is one directory, a couple files, and +one HTML file that you can view directly online when +[GitLab Pages](../../../administration/pages/index.md) is enabled (opens in a new tab).  @@ -158,3 +165,4 @@ information in the UI. [expiry date]: ../../../ci/yaml/README.md#artifacts-expire_in +[ce-14399]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14399 diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 6a5d2d40927..e81e935e37d 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -32,7 +32,7 @@ do. | `/wip` | Toggle the Work In Progress status | | <code>/estimate <1w 3d 2h 14m></code> | Set time estimate | | `/remove_estimate` | Remove estimated time | -| <code>/spend <1h 30m | -1h 5m></code> | Add or subtract spent time | +| <code>/spend <time(1h 30m | -1h 5m)> <date(YYYY-MM-DD)></code> | Add or subtract spent time; optionally, specify the date that time was spent on | | `/remove_time_spent` | Remove time spent | | `/target_branch <Branch Name>` | Set target branch for current merge request | | `/award :emoji:` | Toggle award for :emoji: | diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index e1d3aebb8b3..26c55891b3c 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -37,7 +37,7 @@ This feature allows merged branches to be deleted in bulk. Only branches that have been merged and [are not protected][protected] will be deleted as part of this operation. -It's particularly useful to clean up old branches that were not deleting +It's particularly useful to clean up old branches that were not deleted automatically when a merge request was merged. [ce-6449]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6449 "Add button to delete all merged branches" diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index dfe43c6b691..6b9976d133c 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -26,7 +26,7 @@ to be uploaded to GitLab. For a signature to be verified three conditions need to be met: 1. The public key needs to be added your GitLab account -1. One of the emails in the GPG key matches your **primary** email +1. One of the emails in the GPG key matches a **verified** email address you use in GitLab 1. The committer's email matches the verified email from the gpg key ## Generating a GPG key @@ -94,7 +94,7 @@ started: ``` 1. Enter you real name, the email address to be associated with this key (should - match the primary email address you use in GitLab) and an optional comment + match a verified email address you use in GitLab) and an optional comment (press <kbd>Enter</kbd> to skip): ``` @@ -113,25 +113,25 @@ started: 1. Use the following command to list the private GPG key you just created: ``` - gpg --list-secret-keys --keyid-format 0xLONG mr@robot.sh + gpg --list-secret-keys --keyid-format LONG mr@robot.sh ``` Replace `mr@robot.sh` with the email address you entered above. 1. Copy the GPG key ID that starts with `sec`. In the following example, that's - `0x30F2B65B9246B6CA`: + `30F2B65B9246B6CA`: ``` - sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + sec rsa4096/30F2B65B9246B6CA 2017-08-18 [SC] D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA uid [ultimate] Mr. Robot <mr@robot.sh> - ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` 1. Export the public key of that ID (replace your key ID from the previous step): ``` - gpg --armor --export 0x30F2B65B9246B6CA + gpg --armor --export 30F2B65B9246B6CA ``` 1. Finally, copy the public key and [add it in your profile settings](#adding-a-gpg-key-to-your-account) @@ -167,28 +167,28 @@ key to use. 1. Use the following command to list the private GPG key you just created: ``` - gpg --list-secret-keys --keyid-format 0xLONG mr@robot.sh + gpg --list-secret-keys --keyid-format LONG mr@robot.sh ``` Replace `mr@robot.sh` with the email address you entered above. 1. Copy the GPG key ID that starts with `sec`. In the following example, that's - `0x30F2B65B9246B6CA`: + `30F2B65B9246B6CA`: ``` - sec rsa4096/0x30F2B65B9246B6CA 2017-08-18 [SC] + sec rsa4096/30F2B65B9246B6CA 2017-08-18 [SC] D5E4F29F3275DC0CDA8FFC8730F2B65B9246B6CA uid [ultimate] Mr. Robot <mr@robot.sh> - ssb rsa4096/0xB7ABC0813E4028C0 2017-08-18 [E] + ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` 1. Tell Git to use that key to sign the commits: ``` - git config --global user.signingkey 0x30F2B65B9246B6CA + git config --global user.signingkey 30F2B65B9246B6CA ``` - Replace `0x30F2B65B9246B6CA` with your GPG key ID. + Replace `30F2B65B9246B6CA` with your GPG key ID. ## Signing commits diff --git a/doc/user/project/repository/img/contributors_graph.png b/doc/user/project/repository/img/contributors_graph.png Binary files differindex c31da7aa1ff..c31da7aa1ff 100755..100644 --- a/doc/user/project/repository/img/contributors_graph.png +++ b/doc/user/project/repository/img/contributors_graph.png diff --git a/doc/user/project/repository/img/repo_graph.png b/doc/user/project/repository/img/repo_graph.png Binary files differindex 28da8ad9589..28da8ad9589 100755..100644 --- a/doc/user/project/repository/img/repo_graph.png +++ b/doc/user/project/repository/img/repo_graph.png diff --git a/doc/user/project/repository/web_editor.md b/doc/user/project/repository/web_editor.md index d47a3acdbe9..db0c3ed9d59 100644 --- a/doc/user/project/repository/web_editor.md +++ b/doc/user/project/repository/web_editor.md @@ -105,7 +105,7 @@ merge request. Once you click it, a new branch will be created that diverges from the default branch of your project, by default `master`. The branch name will be based on -the title of the issue and as suffix it will have its ID. Thus, the example +the title of the issue and as a prefix, it will have its internal ID. Thus, the example screenshot above will yield a branch named `2-et-cum-et-sed-expedita-repellat-consequatur-ut-assumenda-numquam-rerum`. diff --git a/doc/user/project/settings/img/general_settings.png b/doc/user/project/settings/img/general_settings.png Binary files differindex 96f5b84871f..96f5b84871f 100755..100644 --- a/doc/user/project/settings/img/general_settings.png +++ b/doc/user/project/settings/img/general_settings.png diff --git a/doc/user/project/settings/img/merge_requests_settings.png b/doc/user/project/settings/img/merge_requests_settings.png Binary files differindex b1f2dfa7376..b1f2dfa7376 100755..100644 --- a/doc/user/project/settings/img/merge_requests_settings.png +++ b/doc/user/project/settings/img/merge_requests_settings.png diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 22c343dc027..a234a647b77 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -23,7 +23,7 @@ Add an [issue description template](../description_templates.md#description-temp Set up your project's merge request settings: -- Set up the merge request method (merge commit, [fast-forward merge](https://docs.gitlab.com/ee/user/project/merge_requests/fast_forward_merge.html#fast-forward-merge-requests)). _Fast-forward is available in [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/)._ +- Set up the merge request method (merge commit, [fast-forward merge](../merge_requests/fast_forward_merge.html)). - Merge request [description templates](../description_templates.md#description-templates). - Enable [merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals), _available in [GitLab Enterprise Edition Starter](https://about.gitlab.com/gitlab-ee/)_. - Enable [merge only of pipeline succeeds](../merge_requests/merge_when_pipeline_succeeds.md). @@ -42,3 +42,11 @@ Learn how to [export a project](import_export.md#importing-the-project) in GitLa ### Advanced settings Here you can run housekeeping, archive, rename, transfer, or remove a project. + +#### Archiving a project + +>**Note:** Only Project Owners and Admin users have the permission to archive a project + +It's possible to mark a project as archived via the Project Settings. An archived project will be hidden by default in the project listings. + +An archived project can be fully restored and will therefore retain it's repository and all associated resources whilst in an archived state. diff --git a/doc/user/search/img/issues_any_assignee.png b/doc/user/search/img/issues_any_assignee.png Binary files differindex 2f902bcc66c..2f902bcc66c 100755..100644 --- a/doc/user/search/img/issues_any_assignee.png +++ b/doc/user/search/img/issues_any_assignee.png diff --git a/doc/user/search/img/issues_assigned_to_you.png b/doc/user/search/img/issues_assigned_to_you.png Binary files differindex 36c670eedd5..36c670eedd5 100755..100644 --- a/doc/user/search/img/issues_assigned_to_you.png +++ b/doc/user/search/img/issues_assigned_to_you.png diff --git a/doc/user/search/img/issues_author.png b/doc/user/search/img/issues_author.png Binary files differindex 792f9746db6..792f9746db6 100755..100644 --- a/doc/user/search/img/issues_author.png +++ b/doc/user/search/img/issues_author.png diff --git a/doc/user/search/img/issues_mrs_shortcut.png b/doc/user/search/img/issues_mrs_shortcut.png Binary files differindex 6380b337b54..6380b337b54 100755..100644 --- a/doc/user/search/img/issues_mrs_shortcut.png +++ b/doc/user/search/img/issues_mrs_shortcut.png diff --git a/doc/user/search/img/left_menu_bar.png b/doc/user/search/img/left_menu_bar.png Binary files differindex d68a71cba8e..d68a71cba8e 100755..100644 --- a/doc/user/search/img/left_menu_bar.png +++ b/doc/user/search/img/left_menu_bar.png diff --git a/doc/user/search/img/project_search.png b/doc/user/search/img/project_search.png Binary files differindex 3150b40de29..3150b40de29 100755..100644 --- a/doc/user/search/img/project_search.png +++ b/doc/user/search/img/project_search.png diff --git a/doc/user/search/img/search_issues_board.png b/doc/user/search/img/search_issues_board.png Binary files differindex 84048ae6a02..84048ae6a02 100755..100644 --- a/doc/user/search/img/search_issues_board.png +++ b/doc/user/search/img/search_issues_board.png diff --git a/doc/user/search/img/sort_projects.png b/doc/user/search/img/sort_projects.png Binary files differindex 9bf2770b299..9bf2770b299 100755..100644 --- a/doc/user/search/img/sort_projects.png +++ b/doc/user/search/img/sort_projects.png diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 673e08287a3..272f7807ac0 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -1,3 +1,7 @@ +--- +comments: false +--- + # Workflow - [Automatic issue closing](../user/project/issues/automatic_issue_closing.md) @@ -36,6 +40,7 @@ - [Revert changes in the UI](../user/project/merge_requests/revert_changes.md) - [Merge requests versions](../user/project/merge_requests/versions.md) - ["Work In Progress" merge requests](../user/project/merge_requests/work_in_progress_merge_requests.md) + - [Fast-forward merge requests](../user/project/merge_requests/fast_forward_merge.md) - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) - [Importing from SVN, GitHub, Bitbucket, etc](importing/README.md) - [Todos](todos.md) diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 9d466ae1971..23b67310d25 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -1,6 +1,6 @@  -## Introduction +# Introduction to GitLab Flow Version management with git makes branching and merging much easier than older versioning systems such as SVN. This allows a wide variety of branching strategies and workflows. diff --git a/doc/workflow/shortcuts.md b/doc/workflow/shortcuts.md index 87416008e98..2e1bd6bfe5c 100644 --- a/doc/workflow/shortcuts.md +++ b/doc/workflow/shortcuts.md @@ -9,7 +9,7 @@ You can see GitLab's keyboard shortcuts by using 'shift + ?' | <kbd>n</kbd> | Main navigation | | <kbd>s</kbd> | Focus search | | <kbd>f</kbd> | Focus filter | -| <kbd>p b</kbd> | Show/hide the Performance Bar | +| <kbd>p</kbd> + <kbd>b</kbd> | Show/hide the Performance Bar | | <kbd>?</kbd> | Show/hide this dialog | | <kbd>⌘</kbd> + <kbd>shift</kbd> + <kbd>p</kbd> | Toggle markdown preview | | <kbd>↑</kbd> | Edit last comment (when focused on an empty textarea) | |
