summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2019-09-17 14:16:34 +0000
committerGitLab Bot <gitlab-bot@gitlab.com>2019-09-17 14:16:34 +0000
commit4ab54c2233e91f60a80e5b6fa2181e6899fdcc3e (patch)
tree2b256ff8dfe63dafe7f42b0d995f9e74fd1dc48b /doc
parentbd860c22f6a4b9473cbddd34a53eead8235a7ea1 (diff)
downloadgitlab-ce-4ab54c2233e91f60a80e5b6fa2181e6899fdcc3e.tar.gz
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r--doc/administration/geo/replication/docker_registry.md11
-rw-r--r--doc/ci/yaml/README.md2
-rw-r--r--doc/development/README.md1
-rw-r--r--doc/development/changelog.md10
-rw-r--r--doc/development/documentation/index.md30
-rw-r--r--doc/development/pipelines.md216
-rw-r--r--doc/development/testing_guide/end_to_end/index.md18
7 files changed, 235 insertions, 53 deletions
diff --git a/doc/administration/geo/replication/docker_registry.md b/doc/administration/geo/replication/docker_registry.md
index 537f8f57420..12cd39c0213 100644
--- a/doc/administration/geo/replication/docker_registry.md
+++ b/doc/administration/geo/replication/docker_registry.md
@@ -63,6 +63,11 @@ We need to make Docker Registry send notification events to the
notification secret in `registry.notification_secret` section of
`/etc/gitlab/gitlab.rb` file.
+ NOTE: **Note:**
+ If you use GitLab HA, you will also have to specify
+ the notification secret in `registry.notification_secret` section of
+ `/etc/gitlab/gitlab.rb` file for every web node.
+
1. Reconfigure the **primary** node for the change to take effect:
```sh
@@ -94,10 +99,8 @@ generate a short-lived JWT that is pull-only-capable to access the
1. Edit `/etc/gitlab/gitlab.rb`:
```ruby
- gitlab_rails['registry_replication'] = {
- enabled: true,
- primary_api_url: 'http://primary.example.com:5000/' # internal address to the primary registry, will be used by GitLab to directly communicate with primary registry API
- }
+ gitlab_rails['geo_registry_replication_enabled'] = true
+ gitlab_rails['geo_registry_replication_primary_api_url'] = 'http://primary.example.com:5000/' # internal address to the primary registry, will be used by GitLab to directly communicate with primary registry API
```
1. Reconfigure the **secondary** node for the change to take effect:
diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md
index 83ac38f3aa3..aa188facebd 100644
--- a/doc/ci/yaml/README.md
+++ b/doc/ci/yaml/README.md
@@ -2104,7 +2104,7 @@ staging:
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23464) in GitLab 12.3.
-`interruptible` is used to indicate that a job should be canceled if made redundant by a newer run of the same job. Defaults to `true`.
+`interruptible` is used to indicate that a job should be canceled if made redundant by a newer run of the same job. Defaults to `false`.
This value will only be used if the [automatic cancellation of redundant pipelines feature](../../user/project/pipelines/settings.md#auto-cancel-pending-pipelines)
is enabled.
diff --git a/doc/development/README.md b/doc/development/README.md
index bbe73570f49..e8bebc79124 100644
--- a/doc/development/README.md
+++ b/doc/development/README.md
@@ -18,6 +18,7 @@ description: 'Learn how to contribute to GitLab.'
- [Generate a changelog entry with `bin/changelog`](changelog.md)
- [Code review guidelines](code_review.md) for reviewing code and having code reviewed
- [Database review guidelines](database_review.md) for reviewing database-related changes and complex SQL queries
+- [Pipelines for the GitLab project](pipelines.md)
- [Automatic CE->EE merge](automatic_ce_ee_merge.md)
- [Guidelines for implementing Enterprise Edition features](ee_features.md)
- [Security process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer)
diff --git a/doc/development/changelog.md b/doc/development/changelog.md
index 5667f58b0c3..afc18885a80 100644
--- a/doc/development/changelog.md
+++ b/doc/development/changelog.md
@@ -102,6 +102,13 @@ Its simplest usage is to provide the value for `title`:
bin/changelog 'Hey DZ, I added a feature to GitLab!'
```
+If you want to generate a changelog entry for GitLab EE, you will need to pass
+the `--ee` option:
+
+```text
+bin/changelog --ee 'Hey DZ, I added a feature to GitLab!'
+```
+
At this point the script would ask you to select the category of the change (mapped to the `type` field in the entry):
```text
@@ -131,9 +138,6 @@ author:
type:
```
-If you're working on the GitLab EE repository, the entry will be added to
-`ee/changelogs/unreleased/` instead.
-
### Arguments
| Argument | Shorthand | Purpose |
diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md
index c158ec63cdd..87892722d5e 100644
--- a/doc/development/documentation/index.md
+++ b/doc/development/documentation/index.md
@@ -152,20 +152,6 @@ disqus_identifier: 'https://docs.gitlab.com/my-old-location/README.html'
Note: it is necessary to include the file name in the `disqus_identifier` URL,
even if it's `index.html` or `README.html`.
-## Branch naming
-
-If your contribution contains **only** documentation changes, you can speed up
-the CI process by following these branch naming conventions:
-
-| Branch name | Valid example |
-|:----------------------|:-----------------------------|
-| Starting with `docs/` | `docs/update-api-issues` |
-| Starting with `docs-` | `docs-update-api-issues` |
-| Ending in `-docs` | `123-update-api-issues-docs` |
-
-If your branch name matches any of the above, it will run only the docs
-tests. If not, the whole application test suite will run (including docs tests).
-
## Merge requests for GitLab documentation
Before getting started, make sure you read the introductory section
@@ -173,7 +159,6 @@ Before getting started, make sure you read the introductory section
[documentation workflow](workflow.md).
- Use the current [merge request description template](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/merge_request_templates/Documentation.md)
-- Use the correct [branch name](#branch-naming)
- Label the MR `Documentation`
- Assign the correct milestone (see note below)
@@ -283,10 +268,6 @@ Several [rspec tests](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/feat
are run to ensure GitLab documentation renders and works correctly. In particular, that [main docs landing page](../../README.md) will work correctly from `/help`.
For example, [GitLab.com's `/help`](https://gitlab.com/help).
-CAUTION: **Caution:**
-Because the rspec tests only run in a full pipeline, and not a special [docs-only pipeline](#branch-naming), it is possible
-to merge changes that will break `master` from a merge request with a successful docs-only pipeline run.
-
## Docs site architecture
See the [Docs site architecture](site_architecture/index.md) page to learn
@@ -309,14 +290,9 @@ The live preview is currently enabled for the following projects:
- <https://gitlab.com/gitlab-org/gitlab>
- <https://gitlab.com/gitlab-org/gitlab-runner>
-If your branch contains only documentation changes, you can use
-[special branch names](#branch-naming) to avoid long-running pipelines.
-
-For [docs-only changes](#branch-naming), the review app is run automatically.
-For all other branches, you can use the manual `review-docs-deploy-manual` job
-in your merge request. You will need at least Maintainer permissions to be able
-to run it. In the mini pipeline graph, you should see a `>>` icon. Clicking it will
-reveal the `review-docs-deploy-manual` job. Click the play button to start the job.
+If your merge request has docs changes, you can use the manual `review-docs-deploy` job
+to deploy the docs review app for your merge request.
+You will need at least Maintainer permissions to be able to run it.
![Manual trigger a docs build](img/manual_build_docs.png)
diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md
new file mode 100644
index 00000000000..448fb0f9f5a
--- /dev/null
+++ b/doc/development/pipelines.md
@@ -0,0 +1,216 @@
+# Pipelines for the GitLab project
+
+Pipelines for `gitlab-org/gitlab` and `gitlab-org/gitlab-foss` (as well as the
+`dev` instance's mirrors) are configured in the usual
+[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml)
+which itself includes files under
+[`.gitlab/ci/`](https://gitlab.com/gitlab-org/gitlab/tree/master/.gitlab/ci)
+for easier maintenance.
+
+We're striving to [dogfood](https://about.gitlab.com/handbook/engineering/#dogfooding)
+GitLab [CI/CD features and best-practices](../ci/yaml/README.md)
+as much as possible.
+
+## Stages
+
+The current stages are:
+
+- `prepare`: This stage includes jobs that prepare artifacts that are needed by
+ jobs in subsequent stages.
+- `quick-test`: This stage includes test jobs that should run first and fail the
+ pipeline early (currently used to run Geo tests when the branch name starts
+ with `geo-`, `geo/`, or ends with `-geo`).
+- `test`: This stage includes most of the tests, DB/migration jobs, and static analysis jobs.
+- `review-prepare`: This stage includes a job that build the CNG images that are
+ later used by the (Helm) Review App deployment (see
+ [Review Apps](testing_guide/review_apps.md) for details).
+- `review`: This stage includes jobs that deploy the GitLab and Docs Review Apps.
+- `qa`: This stage includes jobs that perform QA tasks against the Review App
+ that is deployed in the previous stage.
+- `post-test`: This stage includes jobs that build reports or gather data from
+ the previous stages' jobs (e.g. coverage, Knapsack metadata etc.).
+- `pages`: This stage includes a job that deploys the various reports as
+ GitLab pages (e.g. <https://gitlab-org.gitlab.io/gitlab/coverage-ruby/>,
+ <https://gitlab-org.gitlab.io/gitlab/coverage-javascript/>,
+ <https://gitlab-org.gitlab.io/gitlab/webpack-report/>).
+
+## Default image
+
+The default image is currently
+`dev.gitlab.org:5005/gitlab/gitlab-build-images:ruby-2.6.3-golang-1.11-git-2.22-chrome-73.0-node-12.x-yarn-1.16-postgresql-9.6-graphicsmagick-1.3.33`.
+It includes Ruby 2.6.3, Go 1.11, Git 2.22, Chrome 73, Node 12, Yarn 1.16,
+PostgreSQL 9.6, and Graphics Magick 1.3.33.
+
+The images used in our pipelines are configured in the
+[`gitlab-org/gitlab-build-images`](https://gitlab.com/gitlab-org/gitlab-build-images)
+project, which is push-mirrored to <https://dev.gitlab.org/gitlab/gitlab-build-images>
+for redundancy.
+
+The current version of the build images can be found in the
+["Used by GitLab CE/EE section"](https://gitlab.com/gitlab-org/gitlab-build-images/blob/master/.gitlab-ci.yml).
+
+## Default variables
+
+In addition to the [predefined variables](../ci/variables/predefined_variables.md),
+each pipeline includes the following [variables](../ci/variables/README.md):
+
+- `RAILS_ENV: "test"`
+- `NODE_ENV: "test"`
+- `SIMPLECOV: "true"`
+- `GIT_DEPTH: "20"`
+- `GIT_SUBMODULE_STRATEGY: "none"`
+- `GET_SOURCES_ATTEMPTS: "3"`
+- `KNAPSACK_RSPEC_SUITE_REPORT_PATH: knapsack/${CI_PROJECT_NAME}/rspec_report-master.json`
+- `EE_KNAPSACK_RSPEC_SUITE_REPORT_PATH: knapsack/${CI_PROJECT_NAME}/rspec_report-master-ee.json`
+- `FLAKY_RSPEC_SUITE_REPORT_PATH: rspec_flaky/report-suite.json`
+- `BUILD_ASSETS_IMAGE: "false"`
+- `ES_JAVA_OPTS: "-Xms256m -Xmx256m"`
+- `ELASTIC_URL: "http://elastic:changeme@docker.elastic.co-elasticsearch-elasticsearch:9200"`
+
+## Common job definitions
+
+Most of the jobs [extend from a few CI definitions](../ci/yaml/README.md#extends)
+that are scoped to a single
+[configuration parameter](../ci/yaml/README.md#configuration-parameters).
+
+These common definitions are:
+
+- `.default-tags`: Ensures a job has the `gitlab-org` tag to ensure it's using
+ our dedicated runners.
+- `.default-retry`: Allows a job to retry upon `unknown_failure`, `api_failure`,
+ `runner_system_failure`.
+- `.default-before_script`: Allows a job to use a default `before_script` definition
+ suitable for Ruby/Rails tasks that may need a database running (e.g. tests).
+- `.default-cache`: Allows a job to use a default `cache` definition suitable for
+ Ruby/Rails and frontend tasks.
+- `.default-only`: Restricts the cases where a job is created. This currently
+ includes `master`, `/^[\d-]+-stable(-ee)?$/` (stable branches),
+ `/^\d+-\d+-auto-deploy-\d+$/` (security branches), `merge_requests`, `tags`.
+ Note that jobs won't be created for branches with this default configuration.
+- `.only-review`: Only creates a job for the `gitlab-org` namespace and if
+ Kubernetes integration is available. Also, prevents a job from being created
+ for `master` and auto-deploy branches.
+- `.only-review-schedules`: Same as `.only-review` but also restrict a job to
+ only run for [schedules](../user/project/pipelines/schedules.md).
+- `.use-pg`: Allows a job to use the `postgres:9.6.14` and `redis:alpine` services.
+- `.use-pg-10`: Allows a job to use the `postgres:10.9` and `redis:alpine` services.
+- `.only-ee`: Only creates a job for the `gitlab` project.
+
+## Changes detection
+
+If a job extends from `.default-only` (and most of the jobs should), it can restrict
+the cases where it should be created
+[based on the changes](../ci/yaml/README.md#onlychangesexceptchanges)
+from a commit or MR by extending from the following CI definitions:
+
+- `.only-code-changes`: Allows a job to only be created upon code-related changes.
+- `.only-qa-changes`: Allows a job to only be created upon QA-related changes.
+- `.only-docs-changes`: Allows a job to only be created upon docs-related changes.
+- `.only-code-qa-changes`: Allows a job to only be created upon code-related or QA-related changes.
+
+**See <https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab/ci/global.gitlab-ci.yml>
+for the list of exact patterns.**
+
+## Directed acyclic graph
+
+We're using the [`needs:`](../ci/yaml/README.md#needs) keyword to
+execute jobs out of order for the following jobs:
+
+```mermaid
+graph RL;
+ A[setup-test-env];
+ B["gitlab:assets:compile<br/>(master only)"];
+ C[gitlab:assets:compile pull-cache];
+ D["cache gems<br/>(master and tags only)"];
+ E[review-build-cng];
+ F[build-qa-image];
+ G[review-deploy];
+ G2["schedule:review-deploy<br/>(master only)"];
+ H[karma];
+ I[jest];
+ J["compile-assets<br/>(master only)"];
+ K[compile-assets pull-cache];
+ L[webpack-dev-server];
+ M[coverage];
+ N[pages];
+ O[static-analysis];
+ P["package-and-qa-manual:master<br/>(master schedule only)"];
+ Q[package-and-qa];
+ R[package-and-qa-manual];
+
+subgraph "`prepare` stage"
+ A
+ F
+ J
+ K
+ end
+
+subgraph "`test` stage"
+ B --> |needs| A;
+ C --> |needs| A;
+ D --> |needs| A;
+ H -.-> |depends on| A;
+ H -.-> |depends on| J;
+ H -.-> |depends on| K;
+ I -.-> |depends on| A;
+ I -.-> |depends on| J;
+ I -.-> |depends on| K;
+ L -.-> |depends on| A;
+ L -.-> |depends on| J;
+ L -.-> |depends on| K;
+ downtime_check --> |needs and depends on| A;
+ db:* --> |needs| A;
+ gitlab:setup --> |needs| A;
+ O -.-> |depends on| A;
+ O -.-> |depends on| B;
+ O -.-> |depends on| C;
+ downtime_check --> |needs and depends on| A;
+ end
+
+subgraph "`review-prepare` stage"
+ E --> |needs| C;
+ X["schedule:review-build-cng<br/>(master schedule only)"] --> |needs| B;
+ end
+
+subgraph "`review` stage"
+ G --> |needs| E;
+ G2 --> |needs| E;
+ end
+
+subgraph "`qa` stage"
+ Q --> |needs| C;
+ Q --> |needs| F;
+ R --> |needs| C;
+ R --> |needs| F;
+ P --> |needs| B;
+ P --> |needs| F;
+ review-qa-smoke -.-> |depends on| G;
+ review-qa-all -.-> |depends on| G;
+ review-qa-performance -.-> |depends on| G;
+ X2["schedule:review-performance<br/>(master only)"] -.-> |depends on| G2;
+ dast -.-> |depends on| G;
+ end
+
+subgraph "`post-test` stage"
+ M
+ end
+
+subgraph "`pages` stage"
+ N -.-> |depends on| B;
+ N -.-> |depends on| H;
+ N -.-> |depends on| M;
+ end
+```
+
+## Test jobs
+
+Consult [GitLab tests in the Continuous Integration (CI) context](testing_guide/ci.md)
+for more information.
+
+## Review app jobs
+
+Consult the [Review Apps](testing_guide/review_apps.md) dedicated page for more information.
+
+---
+
+[Return to Development documentation](README.md)
diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md
index f6a2f642274..c00be77ce8c 100644
--- a/doc/development/testing_guide/end_to_end/index.md
+++ b/doc/development/testing_guide/end_to_end/index.md
@@ -7,24 +7,6 @@ as expected across the entire software stack and architecture, including
integration of all micro-services and components that are supposed to work
together.
-## Branch naming
-
-If your contribution contains **only** changes under the
-[`qa/` folder](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa), you can
-speed up the CI process by following some branch naming conventions. You have
-three choices:
-
-| Branch name | Valid example |
-|:----------------------|:-----------------------------|
-| Starting with `qa/` | `qa/new-oauth-login-test` |
-| Starting with `qa-` | `qa-new-oauth-login-test` |
-| Ending in `-qa` | `123-new-oauth-login-test-qa` |
-
-If your branch name matches any of the above, it will run only the QA-related
-jobs.
-If it does not, the whole application test suite will run (including QA-related
-jobs).
-
## How do we test GitLab?
We use [Omnibus GitLab][omnibus-gitlab] to build GitLab packages and then we