diff options
author | Jacob Vosmaer <contact@jacobvosmaer.nl> | 2016-04-12 18:23:42 +0200 |
---|---|---|
committer | Jacob Vosmaer <contact@jacobvosmaer.nl> | 2016-04-12 18:23:42 +0200 |
commit | a1c8fdfbeef5a74dfdf86f1e7954a7a8eb8577eb (patch) | |
tree | 528c69cf876d2473fd9cfbfcf94d58a938fe0f7f /doc | |
parent | ab9dfa8fd681ac558cf988aa2cdb5bd69feea757 (diff) | |
parent | 2082879d2f3f91b038863f7c67c658d678924564 (diff) | |
download | gitlab-ce-a1c8fdfbeef5a74dfdf86f1e7954a7a8eb8577eb.tar.gz |
Merge branch 'master' of https://gitlab.com/gitlab-org/gitlab-ce into git-http-controller
Diffstat (limited to 'doc')
41 files changed, 1410 insertions, 964 deletions
diff --git a/doc/README.md b/doc/README.md index 08d0a6a5bfb..724c7cca0f1 100644 --- a/doc/README.md +++ b/doc/README.md @@ -3,7 +3,7 @@ ## User documentation - [API](api/README.md) Automate GitLab via a simple and powerful API. -- [CI](ci/README.md) +- [CI](ci/README.md) GitLab Continuous Integration (CI) getting started, .gitlab-ci.yml options, and examples. - [GitLab as OAuth2 authentication service provider](integration/oauth_provider.md). It allows you to login to other applications from GitLab. - [GitLab Basics](gitlab-basics/README.md) Find step by step how to start working on your commandline and on GitLab. - [Importing to GitLab](workflow/importing/README.md). @@ -19,10 +19,12 @@ ## Administrator documentation +- [Authentication/Authorization](administration/auth/README.md) Configure + external authentication with LDAP, SAML, CAS and additional Omniauth providers. - [Custom git hooks](hooks/custom_hooks.md) Custom git hooks (on the filesystem) for when webhooks aren't enough. - [Install](install/README.md) Requirements, directory structures and installation from source. - [Restart GitLab](administration/restart_gitlab.md) Learn how to restart GitLab and its components -- [Integration](integration/README.md) How to integrate with systems such as JIRA, Redmine, LDAP and Twitter. +- [Integration](integration/README.md) How to integrate with systems such as JIRA, Redmine, Twitter. - [Issue closing](customization/issue_closing.md) Customize how to close an issue from commit messages. - [Libravatar](customization/libravatar.md) Use Libravatar for user avatars. - [Log system](logs/logs.md) Log system. @@ -45,4 +47,3 @@ contributing to documentation. - [Development](development/README.md) Explains the architecture and the guidelines for shell commands. - [Legal](legal/README.md) Contributor license agreements. -- [Release](release/README.md) How to make the monthly and security releases. diff --git a/doc/administration/auth/README.md b/doc/administration/auth/README.md new file mode 100644 index 00000000000..07e548aaabe --- /dev/null +++ b/doc/administration/auth/README.md @@ -0,0 +1,11 @@ +# Authentication and Authorization + +GitLab integrates with the following external authentication and authorization +providers. + +- [LDAP](ldap.md) Includes Active Directory, Apple Open Directory, Open LDAP, + and 389 Server +- [OmniAuth](../../integration/omniauth.md) Sign in via Twitter, GitHub, GitLab.com, Google, + Bitbucket, Facebook, Shibboleth, Crowd and Azure +- [SAML](../../integration/saml.md) Configure GitLab as a SAML 2.0 Service Provider +- [CAS](../../integration/cas.md) Configure GitLab to sign in using CAS diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md new file mode 100644 index 00000000000..10096779844 --- /dev/null +++ b/doc/administration/auth/ldap.md @@ -0,0 +1,277 @@ +# LDAP + +GitLab integrates with LDAP to support user authentication. +This integration works with most LDAP-compliant directory +servers, including Microsoft Active Directory, Apple Open Directory, Open LDAP, +and 389 Server. GitLab EE includes enhanced integration, including group +membership syncing. + +## Security + +GitLab assumes that LDAP users are not able to change their LDAP 'mail', 'email' +or 'userPrincipalName' attribute. An LDAP user who is allowed to change their +email on the LDAP server can potentially +[take over any account](#enabling-ldap-sign-in-for-existing-gitlab-users) +on your GitLab server. + +We recommend against using LDAP integration if your LDAP users are +allowed to change their 'mail', 'email' or 'userPrincipalName' attribute on +the LDAP server. + +### User deletion + +If a user is deleted from the LDAP server, they will be blocked in GitLab, as +well. Users will be immediately blocked from logging in. However, there is an +LDAP check cache time (sync time) of one hour (see note). This means users that +are already logged in or are using Git over SSH will still be able to access +GitLab for up to one hour. Manually block the user in the GitLab Admin area to +immediately block all access. + +>**Note**: GitLab EE supports a configurable sync time, with a default +of one hour. + +## Configuration + +To enable LDAP integration you need to add your LDAP server settings in +`/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml`. + +>**Note**: In GitLab EE, you can configure multiple LDAP servers to connect to +one GitLab server. + +Prior to version 7.4, GitLab used a different syntax for configuring +LDAP integration. The old LDAP integration syntax still works but may be +removed in a future version. If your `gitlab.rb` or `gitlab.yml` file contains +LDAP settings in both the old syntax and the new syntax, only the __old__ +syntax will be used by GitLab. + +The configuration inside `gitlab_rails['ldap_servers']` below is sensitive to +incorrect indentation. Be sure to retain the indentation given in the example. +Copy/paste can sometimes cause problems. + +**Omnibus configuration** + +```ruby +gitlab_rails['ldap_enabled'] = true +gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below +main: # 'main' is the GitLab 'provider ID' of this LDAP server + ## label + # + # A human-friendly name for your LDAP server. It is OK to change the label later, + # for instance if you find out it is too large to fit on the web page. + # + # Example: 'Paris' or 'Acme, Ltd.' + label: 'LDAP' + + host: '_your_ldap_server' + port: 389 + uid: 'sAMAccountName' + method: 'plain' # "tls" or "ssl" or "plain" + bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' + password: '_the_password_of_the_bind_user' + + # Set a timeout, in seconds, for LDAP queries. This helps avoid blocking + # a request if the LDAP server becomes unresponsive. + # A value of 0 means there is no timeout. + timeout: 10 + + # This setting specifies if LDAP server is Active Directory LDAP server. + # For non AD servers it skips the AD specific queries. + # If your LDAP server is not AD, set this to false. + active_directory: true + + # If allow_username_or_email_login is enabled, GitLab will ignore everything + # after the first '@' in the LDAP username submitted by the user on login. + # + # Example: + # - the user enters 'jane.doe@example.com' and 'p@ssw0rd' as LDAP credentials; + # - GitLab queries the LDAP server with 'jane.doe' and 'p@ssw0rd'. + # + # If you are using "uid: 'userPrincipalName'" on ActiveDirectory you need to + # disable this setting, because the userPrincipalName contains an '@'. + allow_username_or_email_login: false + + # To maintain tight control over the number of active users on your GitLab installation, + # enable this setting to keep new users blocked until they have been cleared by the admin + # (default: false). + block_auto_created_users: false + + # Base where we can search for users + # + # Ex. ou=People,dc=gitlab,dc=example + # + base: '' + + # Filter LDAP users + # + # Format: RFC 4515 https://tools.ietf.org/search/rfc4515 + # Ex. (employeeType=developer) + # + # Note: GitLab does not support omniauth-ldap's custom filter syntax. + # + user_filter: '' + + # LDAP attributes that GitLab will use to create an account for the LDAP user. + # The specified attribute can either be the attribute name as a string (e.g. 'mail'), + # or an array of attribute names to try in order (e.g. ['mail', 'email']). + # Note that the user's LDAP login will always be the attribute specified as `uid` above. + attributes: + # The username will be used in paths for the user's own projects + # (like `gitlab.example.com/username/project`) and when mentioning + # them in issues, merge request and comments (like `@username`). + # If the attribute specified for `username` contains an email address, + # the GitLab username will be the part of the email address before the '@'. + username: ['uid', 'userid', 'sAMAccountName'] + email: ['mail', 'email', 'userPrincipalName'] + + # If no full name could be found at the attribute specified for `name`, + # the full name is determined using the attributes specified for + # `first_name` and `last_name`. + name: 'cn' + first_name: 'givenName' + last_name: 'sn' + + ## EE only + + # Base where we can search for groups + # + # Ex. ou=groups,dc=gitlab,dc=example + # + group_base: '' + + # The CN of a group containing GitLab administrators + # + # Ex. administrators + # + # Note: Not `cn=administrators` or the full DN + # + admin_group: '' + + # The LDAP attribute containing a user's public SSH key + # + # Ex. ssh_public_key + # + sync_ssh_keys: false + +# GitLab EE only: add more LDAP servers +# Choose an ID made of a-z and 0-9 . This ID will be stored in the database +# so that GitLab can remember which LDAP server a user belongs to. +# uswest2: +# label: +# host: +# .... +EOS +``` + +**Source configuration** + +Use the same format as `gitlab_rails['ldap_servers']` for the contents under +`servers:` in the example below: + +``` +production: + # snip... + ldap: + enabled: false + servers: + main: # 'main' is the GitLab 'provider ID' of this LDAP server + ## label + # + # A human-friendly name for your LDAP server. It is OK to change the label later, + # for instance if you find out it is too large to fit on the web page. + # + # Example: 'Paris' or 'Acme, Ltd.' + label: 'LDAP' + # snip... +``` + +## Using an LDAP filter to limit access to your GitLab server + +If you want to limit all GitLab access to a subset of the LDAP users on your +LDAP server, the first step should be to narrow the configured `base`. However, +it is sometimes necessary to filter users further. In this case, you can set up +an LDAP user filter. The filter must comply with +[RFC 4515](https://tools.ietf.org/search/rfc4515). + +**Omnibus configuration** + +```ruby +gitlab_rails['ldap_servers'] = YAML.load <<-EOS +main: + # snip... + user_filter: '(employeeType=developer)' +EOS +``` + +**Source configuration** + +```yaml +production: + ldap: + servers: + main: + # snip... + user_filter: '(employeeType=developer)' +``` + +Tip: If you want to limit access to the nested members of an Active Directory +group you can use the following syntax: + +``` +(memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com) +``` + +Please note that GitLab does not support the custom filter syntax used by +omniauth-ldap. + +## Enabling LDAP sign-in for existing GitLab users + +When a user signs in to GitLab with LDAP for the first time, and their LDAP +email address is the primary email address of an existing GitLab user, then +the LDAP DN will be associated with the existing user. If the LDAP email +attribute is not found in GitLab's database, a new user is created. + +In other words, if an existing GitLab user wants to enable LDAP sign-in for +themselves, they should check that their GitLab email address matches their +LDAP email address, and then sign into GitLab via their LDAP credentials. + +## Limitations + +### TLS Client Authentication + +Not implemented by `Net::LDAP`. +You should disable anonymous LDAP authentication and enable simple or SASL +authentication. The TLS client authentication setting in your LDAP server cannot +be mandatory and clients cannot be authenticated with the TLS protocol. + +### TLS Server Authentication + +Not supported by GitLab's configuration options. +When setting `method: ssl`, the underlying authentication method used by +`omniauth-ldap` is `simple_tls`. This method establishes TLS encryption with +the LDAP server before any LDAP-protocol data is exchanged but no validation of +the LDAP server's SSL certificate is performed. + +## Troubleshooting + +### Invalid credentials when logging in + +- Make sure the user you are binding with has enough permissions to read the user's +tree and traverse it. +- Check that the `user_filter` is not blocking otherwise valid users. +- Run the following check command to make sure that the LDAP settings are + correct and GitLab can see your users: + + ```bash + # For Omnibus installations + sudo gitlab-rake gitlab:ldap:check + + # For installations from source + sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production + ``` + +### Connection Refused + +If you are getting 'Connection Refused' errors when trying to connect to the +LDAP server please double-check the LDAP `port` and `method` settings used by +GitLab. Common combinations are `method: 'plain'` and `port: 389`, OR +`method: 'ssl'` and `port: 636`. diff --git a/doc/api/issues.md b/doc/api/issues.md index 18d64c41986..1c635a6cdcf 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -76,8 +76,9 @@ Example response: "title" : "Consequatur vero maxime deserunt laboriosam est voluptas dolorem.", "created_at" : "2016-01-04T15:31:51.081Z", "iid" : 6, - "labels" : [] - }, + "labels" : [], + "subscribed" : false + } ] ``` @@ -152,7 +153,8 @@ Example response: "id" : 41, "title" : "Ut commodi ullam eos dolores perferendis nihil sunt.", "updated_at" : "2016-01-04T15:31:46.176Z", - "created_at" : "2016-01-04T15:31:46.176Z" + "created_at" : "2016-01-04T15:31:46.176Z", + "subscribed" : false } ] ``` @@ -213,7 +215,8 @@ Example response: "id" : 41, "title" : "Ut commodi ullam eos dolores perferendis nihil sunt.", "updated_at" : "2016-01-04T15:31:46.176Z", - "created_at" : "2016-01-04T15:31:46.176Z" + "created_at" : "2016-01-04T15:31:46.176Z", + "subscribed": false } ``` @@ -237,6 +240,7 @@ POST /projects/:id/issues | `assignee_id` | integer | no | The ID of a user to assign issue | | `milestone_id` | integer | no | The ID of a milestone to assign issue | | `labels` | string | no | Comma-separated label names for an issue | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. `2016-03-11T03:45:40Z` | ```bash curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/4/issues?title=Issues%20with%20auth&labels=bug @@ -266,7 +270,8 @@ Example response: }, "description" : null, "updated_at" : "2016-01-07T12:44:33.959Z", - "milestone" : null + "milestone" : null, + "subscribed" : true } ``` @@ -322,7 +327,8 @@ Example response: ], "id" : 85, "assignee" : null, - "milestone" : null + "milestone" : null, + "subscribed" : true } ``` diff --git a/doc/api/labels.md b/doc/api/labels.md index 6496ffe9fd1..3730c07c5a7 100644 --- a/doc/api/labels.md +++ b/doc/api/labels.md @@ -8,9 +8,9 @@ Get all labels for a given project. GET /projects/:id/labels ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | integer | yes | The ID of the project | ```bash curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/1/labels @@ -22,35 +22,43 @@ Example response: [ { "name" : "bug", - "color" : "#d9534f" + "color" : "#d9534f", + "description": "Bug reported by user", + "open_issues_count": 1, + "closed_issues_count": 0, + "open_merge_requests_count": 1 }, { "color" : "#d9534f", - "name" : "confirmed" + "name" : "confirmed", + "description": "Confirmed issue", + "open_issues_count": 2, + "closed_issues_count": 5, + "open_merge_requests_count": 0 }, { "name" : "critical", - "color" : "#d9534f" - }, - { - "color" : "#428bca", - "name" : "discussion" + "color" : "#d9534f", + "description": "Criticalissue. Need fix ASAP", + "open_issues_count": 1, + "closed_issues_count": 3, + "open_merge_requests_count": 1 }, { "name" : "documentation", - "color" : "#f0ad4e" + "color" : "#f0ad4e", + "description": "Issue about documentation", + "open_issues_count": 1, + "closed_issues_count": 0, + "open_merge_requests_count": 2 }, { "color" : "#5cb85c", - "name" : "enhancement" - }, - { - "color" : "#428bca", - "name" : "suggestion" - }, - { - "color" : "#f0ad4e", - "name" : "support" + "name" : "enhancement", + "description": "Enhancement proposal", + "open_issues_count": 1, + "closed_issues_count": 0, + "open_merge_requests_count": 1 } ] ``` @@ -66,11 +74,12 @@ and 409 if the label already exists. POST /projects/:id/labels ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | -| `name` | string | yes | The name of the label | -| `color` | string | yes | The color of the label in 6-digit hex notation with leading `#` sign | +| Attribute | Type | Required | Description | +| ------------- | ------- | -------- | ---------------------------- | +| `id` | integer | yes | The ID of the project | +| `name` | string | yes | The name of the label | +| `color` | string | yes | The color of the label in 6-digit hex notation with leading `#` sign | +| `description` | string | no | The description of the label | ```bash curl --data "name=feature&color=#5843AD" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels" @@ -81,7 +90,8 @@ Example response: ```json { "name" : "feature", - "color" : "#5843AD" + "color" : "#5843AD", + "description":null } ``` @@ -97,10 +107,10 @@ In case of an error, an additional error message is returned. DELETE /projects/:id/labels ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | -| `name` | string | yes | The name of the label | +| Attribute | Type | Required | Description | +| --------- | ------- | -------- | --------------------- | +| `id` | integer | yes | The ID of the project | +| `name` | string | yes | The name of the label | ```bash curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels?name=bug" @@ -112,6 +122,7 @@ Example response: { "title" : "feature", "color" : "#5843AD", + "description": "New feature proposal", "updated_at" : "2015-11-03T21:22:30.737Z", "template" : false, "project_id" : 1, @@ -133,15 +144,16 @@ In case of an error, an additional error message is returned. PUT /projects/:id/labels ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer | yes | The ID of the project | -| `name` | string | yes | The name of the existing label | -| `new_name` | string | yes if `color` if not provided | The new name of the label | -| `color` | string | yes if `new_name` is not provided | The new color of the label in 6-digit hex notation with leading `#` sign | +| Attribute | Type | Required | Description | +| --------------- | ------- | --------------------------------- | ------------------------------- | +| `id` | integer | yes | The ID of the project | +| `name` | string | yes | The name of the existing label | +| `new_name` | string | yes if `color` if not provided | The new name of the label | +| `color` | string | yes if `new_name` is not provided | The new color of the label in 6-digit hex notation with leading `#` sign | +| `description` | string | no | The new description of the label | ```bash -curl -X PUT --data "name=documentation&new_name=docs&color=#8E44AD" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels" +curl -X PUT --data "name=documentation&new_name=docs&color=#8E44AD&description=Documentation" -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/1/labels" ``` Example response: @@ -149,6 +161,7 @@ Example response: ```json { "color" : "#8E44AD", - "name" : "docs" + "name" : "docs", + "description": "Documentation" } ``` diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index b20a6300b7a..20db73ea6c0 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -66,7 +66,8 @@ Parameters: "due_date": null }, "merge_when_build_succeeds": true, - "merge_status": "can_be_merged" + "merge_status": "can_be_merged", + "subscribed" : false } ] ``` @@ -128,7 +129,8 @@ Parameters: "due_date": null }, "merge_when_build_succeeds": true, - "merge_status": "can_be_merged" + "merge_status": "can_be_merged", + "subscribed" : true } ``` @@ -227,6 +229,7 @@ Parameters: }, "merge_when_build_succeeds": true, "merge_status": "can_be_merged", + "subscribed" : true, "changes": [ { "old_path": "VERSION", @@ -304,7 +307,8 @@ Parameters: "due_date": null }, "merge_when_build_succeeds": true, - "merge_status": "can_be_merged" + "merge_status": "can_be_merged", + "subscribed" : true } ``` @@ -373,7 +377,8 @@ Parameters: "due_date": null }, "merge_when_build_succeeds": true, - "merge_status": "can_be_merged" + "merge_status": "can_be_merged", + "subscribed" : true } ``` @@ -466,7 +471,8 @@ Parameters: "due_date": null }, "merge_when_build_succeeds": true, - "merge_status": "can_be_merged" + "merge_status": "can_be_merged", + "subscribed" : true } ``` @@ -530,7 +536,8 @@ Parameters: "due_date": null }, "merge_when_build_succeeds": true, - "merge_status": "can_be_merged" + "merge_status": "can_be_merged", + "subscribed" : true } ``` diff --git a/doc/api/milestones.md b/doc/api/milestones.md index a6828728264..e4202025f80 100644 --- a/doc/api/milestones.md +++ b/doc/api/milestones.md @@ -7,8 +7,24 @@ Returns a list of project milestones. ``` GET /projects/:id/milestones GET /projects/:id/milestones?iid=42 +GET /projects/:id/milestones?state=active +GET /projects/:id/milestones?state=closed ``` +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `iid` | integer | optional | Return only the milestone having the given `iid` | +| `state` | string | optional | Return only `active` or `closed` milestones` | + +```bash +curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/milestones +``` + +Example Response: + ```json [ { @@ -25,10 +41,6 @@ GET /projects/:id/milestones?iid=42 ] ``` -Parameters: - -- `id` (required) - The ID of a project -- `iid` (optional) - Return the milestone having the given `iid` ## Get single milestone diff --git a/doc/api/notes.md b/doc/api/notes.md index d4d63e825ab..2e0936f11b5 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -32,6 +32,7 @@ Parameters: "created_at": "2013-09-30T13:46:01Z" }, "created_at": "2013-10-02T09:22:45Z", + "updated_at": "2013-10-02T10:22:45Z", "system": true, "upvote": false, "downvote": false, @@ -51,6 +52,7 @@ Parameters: "created_at": "2013-09-30T13:46:01Z" }, "created_at": "2013-10-02T09:56:03Z", + "updated_at": "2013-10-02T09:56:03Z", "system": true, "upvote": false, "downvote": false, @@ -103,6 +105,53 @@ Parameters: - `note_id` (required) - The ID of a note - `body` (required) - The content of a note +### Delete an issue note + +Deletes an existing note of an issue. On success, this API method returns 200 +and the deleted note. If the note does not exist, the API returns 404. + +``` +DELETE /projects/:id/issues/:issue_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `issue_id` | integer | yes | The ID of an issue | +| `note_id` | integer | yes | The ID of a note | + +```bash +curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/issues/11/notes/636 +``` + +Example Response: + +```json +{ + "id": 636, + "body": "This is a good idea.", + "attachment": null, + "author": { + "id": 1, + "username": "pipin", + "email": "admin@example.com", + "name": "Pip", + "state": "active", + "created_at": "2013-09-30T13:46:01Z", + "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", + "web_url": "https://gitlab.example.com/u/pipin" + }, + "created_at": "2016-04-05T22:10:44.164Z", + "system": false, + "noteable_id": 11, + "noteable_type": "Issue", + "upvote": false, + "downvote": false +} +``` + ## Snippets ### List all snippet notes @@ -180,6 +229,53 @@ Parameters: - `note_id` (required) - The ID of a note - `body` (required) - The content of a note +### Delete a snippet note + +Deletes an existing note of a snippet. On success, this API method returns 200 +and the deleted note. If the note does not exist, the API returns 404. + +``` +DELETE /projects/:id/snippets/:snippet_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `snippet_id` | integer | yes | The ID of a snippet | +| `note_id` | integer | yes | The ID of a note | + +```bash +curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/snippets/52/notes/1659 +``` + +Example Response: + +```json +{ + "id": 1659, + "body": "This is a good idea.", + "attachment": null, + "author": { + "id": 1, + "username": "pipin", + "email": "admin@example.com", + "name": "Pip", + "state": "active", + "created_at": "2013-09-30T13:46:01Z", + "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", + "web_url": "https://gitlab.example.com/u/pipin" + }, + "created_at": "2016-04-06T16:51:53.239Z", + "system": false, + "noteable_id": 52, + "noteable_type": "Snippet", + "upvote": false, + "downvote": false +} +``` + ## Merge Requests ### List all merge request notes @@ -223,6 +319,7 @@ Parameters: "created_at": "2013-09-30T13:46:01Z" }, "created_at": "2013-10-02T08:57:14Z", + "updated_at": "2013-10-02T08:57:14Z", "system": false, "upvote": false, "downvote": false, @@ -259,3 +356,50 @@ Parameters: - `merge_request_id` (required) - The ID of a merge request - `note_id` (required) - The ID of a note - `body` (required) - The content of a note + +### Delete a merge request note + +Deletes an existing note of a merge request. On success, this API method returns +200 and the deleted note. If the note does not exist, the API returns 404. + +``` +DELETE /projects/:id/merge_requests/:merge_request_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `merge_request_id` | integer | yes | The ID of a merge request | +| `note_id` | integer | yes | The ID of a note | + +```bash +curl -X DELETE -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/merge_requests/7/notes/1602 +``` + +Example Response: + +```json +{ + "id": 1602, + "body": "This is a good idea.", + "attachment": null, + "author": { + "id": 1, + "username": "pipin", + "email": "admin@example.com", + "name": "Pip", + "state": "active", + "created_at": "2013-09-30T13:46:01Z", + "avatar_url": "http://www.gravatar.com/avatar/5224fd70153710e92fb8bcf79ac29d67?s=80&d=identicon", + "web_url": "https://gitlab.example.com/u/pipin" + }, + "created_at": "2016-04-05T22:11:59.923Z", + "system": false, + "noteable_id": 7, + "noteable_type": "MergeRequest", + "upvote": false, + "downvote": false +} +``` diff --git a/doc/api/projects.md b/doc/api/projects.md index 3703f4b327a..ab716c229dc 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -491,6 +491,172 @@ Parameters: - `id` (required) - The ID of the project to be forked +### Archive a project + +Archives the project if the user is either admin or the project owner of this project. This action is +idempotent, thus archiving an already archived project will not change the project. + +Status code 201 with the project as body is given when successful, in case the user doesn't +have the proper access rights, code 403 is returned. Status 404 is returned if the project +doesn't exist, or is hidden to the user. + +``` +POST /projects/:id/archive +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the project | + +```bash +curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/archive" +``` + +Example response: + +```json +{ + "id": 3, + "description": null, + "default_branch": "master", + "public": false, + "visibility_level": 0, + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site", + "tag_list": [ + "example", + "disapora project" + ], + "owner": { + "id": 3, + "name": "Diaspora", + "created_at": "2013-09-30T13: 46: 02Z" + }, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "builds_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "created_at": "2013-09-30T13: 46: 02Z", + "last_activity_at": "2013-09-30T13: 46: 02Z", + "creator_id": 3, + "namespace": { + "created_at": "2013-09-30T13: 46: 02Z", + "description": "", + "id": 3, + "name": "Diaspora", + "owner_id": 1, + "path": "diaspora", + "updated_at": "2013-09-30T13: 46: 02Z" + }, + "permissions": { + "project_access": { + "access_level": 10, + "notification_level": 3 + }, + "group_access": { + "access_level": 50, + "notification_level": 3 + } + }, + "archived": true, + "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png", + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b" +} +``` + +### Unarchive a project + +Unarchives the project if the user is either admin or the project owner of this project. This action is +idempotent, thus unarchiving an non-archived project will not change the project. + +Status code 201 with the project as body is given when successful, in case the user doesn't +have the proper access rights, code 403 is returned. Status 404 is returned if the project +doesn't exist, or is hidden to the user. + +``` +POST /projects/:id/archive +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the project | + +```bash +curl -X POST -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/unarchive" +``` + +Example response: + +```json +{ + "id": 3, + "description": null, + "default_branch": "master", + "public": false, + "visibility_level": 0, + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-project-site.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site", + "tag_list": [ + "example", + "disapora project" + ], + "owner": { + "id": 3, + "name": "Diaspora", + "created_at": "2013-09-30T13: 46: 02Z" + }, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "builds_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "created_at": "2013-09-30T13: 46: 02Z", + "last_activity_at": "2013-09-30T13: 46: 02Z", + "creator_id": 3, + "namespace": { + "created_at": "2013-09-30T13: 46: 02Z", + "description": "", + "id": 3, + "name": "Diaspora", + "owner_id": 1, + "path": "diaspora", + "updated_at": "2013-09-30T13: 46: 02Z" + }, + "permissions": { + "project_access": { + "access_level": 10, + "notification_level": 3 + }, + "group_access": { + "access_level": 50, + "notification_level": 3 + } + }, + "archived": false, + "avatar_url": "http://example.com/uploads/project/avatar/3/uploads/avatar.png", + "shared_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8bc4a7a29eb76ea83cf79e4908c2b" +} +``` + ### Remove project Removes a project including all associated resources (issues, merge requests etc.) @@ -614,8 +780,10 @@ Parameters: - `id` (required) - The ID or NAMESPACE/PROJECT_NAME of a project - `user_id` (required) - The ID of a team member -This method is idempotent and can be called multiple times with the same parameters. -Revoking team membership for a user who is not currently a team member is considered success. +This method removes the project member if the user has the proper access rights to do so. +It returns a status code 403 if the member does not have the proper rights to perform this action. +In all other cases this method is idempotent and revoking team membership for a user who is not +currently a team member is considered success. Please note that the returned JSON currently differs slightly. Thus you should not rely on the returned JSON structure. diff --git a/doc/api/settings.md b/doc/api/settings.md index 001de76c7af..1e745115dc8 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -26,7 +26,6 @@ Example response: "default_branch_protection" : 2, "restricted_visibility_levels" : [], "signin_enabled" : true, - "twitter_sharing_enabled" : true, "after_sign_out_path" : null, "max_attachment_size" : 10, "user_oauth_applications" : true, @@ -57,7 +56,6 @@ PUT /application/settings | `sign_in_text` | string | no | Text on login page | | `home_page_url` | string | no | Redirect to this URL when not logged in | | `default_branch_protection` | integer | no | Determine if developers can push to master. Can take `0` _(not protected, both developers and masters can push new commits, force push or delete the branch)_, `1` _(partially protected, developers can push new commits, but cannot force push or delete the branch, masters can do anything)_ or `2` _(fully protected, developers cannot push new commits, force push or delete the branch, masters can do anything)_ as a parameter. Default is `1`. | -| `twitter_sharing_enabled` | boolean | no | Allow users to share project creation on Twitter | | `restricted_visibility_levels` | array of integers | no | Selected levels cannot be used by non-admin users for projects or snippets. Can take `0` _(Private)_, `1` _(Internal)_ and `2` _(Public)_ as a parameter. Default is null which means there is no restriction. | | `max_attachment_size` | integer | no | Limit attachment size in MB | | `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | @@ -85,7 +83,6 @@ Example response: "updated_at": "2015-06-30T13:22:42.210Z", "home_page_url": "", "default_branch_protection": 2, - "twitter_sharing_enabled": true, "restricted_visibility_levels": [], "max_attachment_size": 10, "session_expire_delay": 10080, diff --git a/doc/api/tags.md b/doc/api/tags.md index 17d12e9cc62..ac9fac92f4c 100644 --- a/doc/api/tags.md +++ b/doc/api/tags.md @@ -38,6 +38,50 @@ Parameters: ] ``` +## Get a single repository tag + +Get a specific repository tag determined by its name. It returns `200` together +with the tag information if the tag exists. It returns `404` if the tag does not +exist. + +``` +GET /projects/:id/repository/tags/:tag_name +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | +| `tag_name` | string | yes | The name of the tag | + +```bash +curl -H "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v3/projects/5/repository/tags/v1.0.0 +``` + +Example Response: + +```json +{ + "name": "v5.0.0", + "message": null, + "commit": { + "id": "60a8ff033665e1207714d6670fcd7b65304ec02f", + "message": "v5.0.0\n", + "parent_ids": [ + "f61c062ff8bcbdb00e0a1b3317a91aed6ceee06b" + ], + "authored_date": "2015-02-01T21:56:31.000+01:00", + "author_name": "Arthur Verschaeve", + "author_email": "contact@arthurverschaeve.be", + "committed_date": "2015-02-01T21:56:31.000+01:00", + "committer_name": "Arthur Verschaeve", + "committer_email": "contact@arthurverschaeve.be" + }, + "release": null +} +``` + ## Create a new tag Creates a new tag in the repository that points to the supplied ref. @@ -148,4 +192,4 @@ Parameters: "tag_name": "1.0.0", "description": "Amazing release. Wow" } -```
\ No newline at end of file +``` diff --git a/doc/api/users.md b/doc/api/users.md index 383e7c76ab0..7d2b4897cff 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -69,6 +69,7 @@ GET /users "state": "blocked", "created_at": "2012-05-23T08:01:01Z", "bio": null, + "location": null, "skype": "", "linkedin": "", "twitter": "", @@ -126,6 +127,7 @@ Parameters: "created_at": "2012-05-23T08:00:58Z", "is_admin": false, "bio": null, + "location": null, "skype": "", "linkedin": "", "twitter": "", @@ -154,6 +156,7 @@ Parameters: "confirmed_at": "2012-05-23T08:00:58Z", "last_sign_in_at": "2015-03-23T08:00:58Z", "bio": null, + "location": null, "skype": "", "linkedin": "", "twitter": "", @@ -191,6 +194,7 @@ Parameters: - `extern_uid` (optional) - External UID - `provider` (optional) - External provider name - `bio` (optional) - User's biography +- `location` (optional) - User's location - `admin` (optional) - User is admin - true or false (default) - `can_create_group` (optional) - User can create groups - true or false - `confirm` (optional) - Require confirmation - true (default) or false @@ -218,6 +222,7 @@ Parameters: - `extern_uid` - External UID - `provider` - External provider name - `bio` - User's biography +- `location` (optional) - User's location - `admin` (optional) - User is admin - true or false (default) - `can_create_group` (optional) - User can create groups - true or false - `external` (optional) - Flags the user as external - true or false(default) @@ -260,6 +265,7 @@ GET /user "state": "active", "created_at": "2012-05-23T08:00:58Z", "bio": null, + "location": null, "skype": "", "linkedin": "", "twitter": "", diff --git a/doc/ci/build_artifacts/README.md b/doc/ci/build_artifacts/README.md index 71db5aa5dc8..9553bb11e9d 100644 --- a/doc/ci/build_artifacts/README.md +++ b/doc/ci/build_artifacts/README.md @@ -1,7 +1,10 @@ # Introduction to build artifacts Artifacts is a list of files and directories which are attached to a build -after it completes successfully. +after it completes successfully. This feature is enabled by default in all GitLab installations. + +_If you are searching for ways to use artifacts, jump to +[Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._ Since GitLab 8.2 and [GitLab Runner] 0.7.0, build artifacts that are created by GitLab Runner are uploaded to GitLab and are downloadable as a single archive @@ -16,13 +19,9 @@ The artifacts browser will be available only for new artifacts that are sent to GitLab using GitLab Runner version 1.0 and up. It will not be possible to browse old artifacts already uploaded to GitLab. -## Enabling build artifacts - -_If you are searching for ways to use artifacts, jump to -[Defining artifacts in `.gitlab-ci.yml`](#defining-artifacts-in-gitlab-ciyml)._ +## Disabling build artifacts -The artifacts feature is enabled by default in all GitLab installations. -To disable it site-wide, follow the steps below. +To disable artifacts site-wide, follow the steps below. --- diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 210f9c3e849..d790015aca1 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -57,7 +57,7 @@ before_script: # WARNING: Use this only with the Docker executor, if you use it with shell # you will overwrite your user's SSH config. - mkdir -p ~/.ssh - - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config` + - '[[ -f /.dockerinit ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' ``` As a final step, add the _public_ key from the one you created earlier to the diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 4316f3c1f64..7da9b31e30d 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -38,7 +38,7 @@ services: - postgres before_script: - - bundle_install + - bundle install stages: - build diff --git a/doc/development/README.md b/doc/development/README.md index 1b281809afc..3f3ef068f96 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -2,11 +2,15 @@ - [Architecture](architecture.md) of GitLab - [CI setup](ci_setup.md) for testing GitLab +- [Code review guidelines](code_review.md) for reviewing code and having code + reviewed. - [Gotchas](gotchas.md) to avoid - [How to dump production data to staging](db_dump.md) +- [Instrumentation](instrumentation.md) - [Migration Style Guide](migration_style_guide.md) for creating safe migrations - [Rake tasks](rake_tasks.md) for development - [Shell commands](shell_commands.md) in the GitLab codebase - [Sidekiq debugging](sidekiq_debugging.md) - [SQL guidelines](sql.md) for SQL guidelines +- [Testing standards and style guidelines](testing.md) - [UI guide](ui_guide.md) for building GitLab with existing css styles and elements diff --git a/doc/development/code_review.md b/doc/development/code_review.md new file mode 100644 index 00000000000..40ae55ab905 --- /dev/null +++ b/doc/development/code_review.md @@ -0,0 +1,78 @@ +# Code Review Guidelines + +This guide contains advice and best practices for performing code review, and +having your code reviewed. + +All merge requests for GitLab CE and EE, whether written by a GitLab team member +or a volunteer contributor, must go through a code review process to ensure the +code is effective, understandable, and maintainable. + +Any developer can, and is encouraged to, perform code review on merge requests +of colleagues and contributors. However, the final decision to accept a merge +request is up to one of our merge request "endbosses", denoted on the +[team page](https://about.gitlab.com/team). + +## Everyone + +- Accept that many programming decisions are opinions. Discuss tradeoffs, which + you prefer, and reach a resolution quickly. +- Ask questions; don't make demands. ("What do you think about naming this + `:user_id`?") +- Ask for clarification. ("I didn't understand. Can you clarify?") +- Avoid selective ownership of code. ("mine", "not mine", "yours") +- Avoid using terms that could be seen as referring to personal traits. ("dumb", + "stupid"). Assume everyone is attractive, intelligent, and well-meaning. +- Be explicit. Remember people don't always understand your intentions online. +- Be humble. ("I'm not sure - let's look it up.") +- Don't use hyperbole. ("always", "never", "endlessly", "nothing") +- Be careful about the use of sarcasm. Everything we do is public; what seems + like good-natured ribbing to you and a long-time colleague might come off as + mean and unwelcoming to a person new to the project. +- Consider one-on-one chats or video calls if there are too many "I didn't + understand" or "Alternative solution:" comments. Post a follow-up comment + summarizing one-on-one discussion. + +## Having your code reviewed + +- The first reviewer of your code is _you_. Before you perform that first push + of your shiny new branch, read through the entire diff. Does it make sense? + Did you include something unrelated to the overall purpose of the changes? Did + you forget to remove any debugging code? +- Be grateful for the reviewer's suggestions. ("Good call. I'll make that + change.") +- Don't take it personally. The review is of the code, not of you. +- Explain why the code exists. ("It's like that because of these reasons. Would + it be more clear if I rename this class/file/method/variable?") +- Extract unrelated changes and refactorings into future merge requests/issues. +- Seek to understand the reviewer's perspective. +- Try to respond to every comment. +- Push commits based on earlier rounds of feedback as isolated commits to the + branch. Do not squash until the branch is ready to merge. Reviewers should be + able to read individual updates based on their earlier feedback. + +## Reviewing code + +Understand why the change is necessary (fixes a bug, improves the user +experience, refactors the existing code). Then: + +- Communicate which ideas you feel strongly about and those you don't. +- Identify ways to simplify the code while still solving the problem. +- Offer alternative implementations, but assume the author already considered + them. ("What do you think about using a custom validator here?") +- Seek to understand the author's perspective. +- If you don't understand a piece of code, _say so_. There's a good chance + someone else would be confused by it as well. +- After a round of line notes, it can be helpful to post a summary note such as + "LGTM :thumbsup:", or "Just a couple things to address." +- Avoid accepting a merge request before the build succeeds ("Merge when build + succeeds" is fine). + +## Credits + +Largely based on the [thoughtbot code review guide]. + +[thoughtbot code review guide]: https://github.com/thoughtbot/guides/tree/master/code-review + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/instrumentation.md b/doc/development/instrumentation.md new file mode 100644 index 00000000000..c1cf2e77c26 --- /dev/null +++ b/doc/development/instrumentation.md @@ -0,0 +1,36 @@ +# Instrumenting Ruby Code + +GitLab Performance Monitoring allows instrumenting of custom blocks of Ruby +code. This can be used to measure the time spent in a specific part of a larger +chunk of code. The resulting data is stored as a field in the transaction that +executed the block. + +To start measuring a block of Ruby code you should use `Gitlab::Metrics.measure` +and give it a name: + +```ruby +Gitlab::Metrics.measure(:foo) do + ... +end +``` + +3 values are measured for a block: + +1. The real time elapsed, stored in NAME_real_time. +2. The CPU time elapsed, stored in NAME_cpu_time. +3. The call count, stored in NAME_call_count. + +Both the real and CPU timings are measured in milliseconds. + +Multiple calls to the same block will result in the final values being the sum +of all individual values. Take this code for example: + +```ruby +3.times do + Gitlab::Metrics.measure(:sleep) do + sleep 1 + end +end +``` + +Here the final value of `sleep_real_time` will be `3`, _not_ `1`. diff --git a/doc/development/scss_styleguide.md b/doc/development/scss_styleguide.md index 6c48c25448b..a79f4073cde 100644 --- a/doc/development/scss_styleguide.md +++ b/doc/development/scss_styleguide.md @@ -72,9 +72,9 @@ p { margin: 0; padding: 0; } ### Colors -HEX (hexadecimal) colors short-form should use shortform where possible, and -should use lower case letters to differenciate between letters and numbers, e. -g. `#E3E3E3` vs. `#e3e3e3`. +HEX (hexadecimal) colors should use shorthand where possible, and should use +lower case letters to differentiate between letters and numbers, e.g. `#E3E3E3` +vs. `#e3e3e3`. ```scss // Bad @@ -160,6 +160,7 @@ is slightly more performant. ``` ### Selectors with a `js-` Prefix + Do not use any selector prefixed with `js-` for styling purposes. These selectors are intended for use only with JavaScript to allow for removal or renaming without breaking styling. @@ -187,8 +188,28 @@ CSSComb globally (system-wide). Run it in the GitLab directory with Note that this won't fix every problem, but it should fix a majority. +### Ignoring issues + +If you want a line or set of lines to be ignored by the linter, you can use +`// scss-lint:disable RuleName` ([more info][disabling-linters]): + +```scss +// This lint rule is disabled because the class name comes from a gem. +// scss-lint:disable SelectorFormat +.ui_charcoal { + background-color: #333; +} +// scss-lint:enable SelectorFormat +``` + +Make sure a comment is added on the line above the `disable` rule, otherwise the +linter will throw a warning. `DisableLinterReason` is enabled to make sure the +style guide isn't being ignored, and to communicate to others why the style +guide is ignored in this instance. + [csscomb]: https://github.com/csscomb/csscomb.js [node]: https://github.com/nodejs/node [npm]: https://www.npmjs.com/ [scss-lint]: https://github.com/brigade/scss-lint [scss-lint-documentation]: https://github.com/brigade/scss-lint/blob/master/lib/scss_lint/linter/README.md +[disabling-linters]: https://github.com/brigade/scss-lint#disabling-linters-via-source diff --git a/doc/development/testing.md b/doc/development/testing.md new file mode 100644 index 00000000000..672e3fb4649 --- /dev/null +++ b/doc/development/testing.md @@ -0,0 +1,136 @@ +# Testing Standards and Style Guidelines + +This guide outlines standards and best practices for automated testing of GitLab +CE and EE. + +It is meant to be an _extension_ of the [thoughtbot testing +styleguide](https://github.com/thoughtbot/guides/tree/master/style/testing). If +this guide defines a rule that contradicts the thoughtbot guide, this guide +takes precedence. Some guidelines may be repeated verbatim to stress their +importance. + +## Factories + +GitLab uses [factory_girl] as a test fixture replacement. + +- Factory definitions live in `spec/factories/`, named using the pluralization + of their corresponding model (`User` factories are defined in `users.rb`). +- There should be only one top-level factory definition per file. +- FactoryGirl methods are mixed in to all RSpec groups. This means you can (and + should) call `create(...)` instead of `FactoryGirl.create(...)`. +- Make use of [traits] to clean up definitions and usages. +- When defining a factory, don't define attributes that are not required for the + resulting record to pass validation. +- When instantiating from a factory, don't supply attributes that aren't + required by the test. +- Factories don't have to be limited to `ActiveRecord` objects. + [See example](https://gitlab.com/gitlab-org/gitlab-ce/commit/0b8cefd3b2385a21cfed779bd659978c0402766d). + +[factory_girl]: https://github.com/thoughtbot/factory_girl +[traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits + +## JavaScript + +GitLab uses [Teaspoon] to run its [Jasmine] JavaScript specs. They can be run on +the command line via `bundle exec teaspoon`, or via a web browser at +`http://localhost:3000/teaspoon` when the Rails server is running. + +- JavaScript tests live in `spec/javascripts/`, matching the folder structure of + `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js.coffee` has a corresponding + `spec/javascripts/behaviors/autosize_spec.js.coffee` file. +- Haml fixtures required for JavaScript tests live in + `spec/javascripts/fixtures`. They should contain the bare minimum amount of + markup necessary for the test. + + > **Warning:** Keep in mind that a Rails view may change and + invalidate your test, but everything will still pass because your fixture + doesn't reflect the latest view. + +- Keep in mind that in a CI environment, these tests are run in a headless + browser and you will not have access to certain APIs, such as + [`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification), + which will have to be stubbed. + +[Teaspoon]: https://github.com/modeset/teaspoon +[Jasmine]: https://github.com/jasmine/jasmine + +## RSpec + +### General Guidelines + +- Use a single, top-level `describe ClassName` block. +- Use `described_class` instead of repeating the class name being described. +- Use `.method` to describe class methods and `#method` to describe instance + methods. +- Use `context` to test branching logic. +- Don't `describe` symbols (see [Gotchas](gotchas.md#dont-describe-symbols)). +- Prefer `not_to` to `to_not`. +- Try to match the ordering of tests to the ordering within the class. +- Try to follow the [Four-Phase Test][four-phase-test] pattern, using newlines + to separate phases. + +[four-phase-test]: https://robots.thoughtbot.com/four-phase-test + +### `let` variables + +GitLab's RSpec suite has made extensive use of `let` variables to reduce +duplication. However, this sometimes [comes at the cost of clarity][lets-not], +so we need to set some guidelines for their use going forward: + +- `let` variables are preferable to instance variables. Local variables are + preferable to `let` variables. +- Use `let` to reduce duplication throughout an entire spec file. +- Don't use `let` to define variables used by a single test; define them as + local variables inside the test's `it` block. +- Don't define a `let` variable inside the top-level `describe` block that's + only used in a more deeply-nested `context` or `describe` block. Keep the + definition as close as possible to where it's used. +- Try to avoid overriding the definition of one `let` variable with another. +- Don't define a `let` variable that's only used by the definition of another. + Use a helper method instead. + +[lets-not]: https://robots.thoughtbot.com/lets-not + +### Test speed + +GitLab has a massive test suite that, without parallelization, can take more +than an hour to run. It's important that we make an effort to write tests that +are accurate and effective _as well as_ fast. + +Here are some things to keep in mind regarding test performance: + +- `double` and `spy` are faster than `FactoryGirl.build(...)` +- `FactoryGirl.build(...)` and `.build_stubbed` are faster than `.create`. +- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`, + `spy`, or `double` will do. Database persistence is slow! +- Use `create(:empty_project)` instead of `create(:project)` when you don't need + the underlying Git repository. Filesystem operations are slow! +- Don't mark a feature as requiring JavaScript (through `@javascript` in + Spinach or `js: true` in RSpec) unless it's _actually_ required for the test + to be valid. Headless browser testing is slow! + +### Features / Integration + +- Feature specs live in `spec/features/` and should be named + `ROLE_ACTION_spec.rb`, such as `user_changes_password_spec.rb`. +- Use only one `feature` block per feature spec file. +- Use scenario titles that describe the success and failure paths. +- Avoid scenario titles that add no information, such as "successfully." +- Avoid scenario titles that repeat the feature title. + +## Spinach (feature) tests + +GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426) +for its feature/integration tests in September 2012. + +As of March 2016, we are [trying to avoid adding new Spinach +tests](https://gitlab.com/gitlab-org/gitlab-ce/issues/14121) going forward, +opting for [RSpec feature](#features-integration) specs. + +Adding new Spinach scenarios is acceptable _only if_ the new scenario requires +no more than one new `step` definition. If more than that is required, the +test should be re-implemented using RSpec instead. + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/ui_guide.md b/doc/development/ui_guide.md index 2f01defc11d..a3e260a5f89 100644 --- a/doc/development/ui_guide.md +++ b/doc/development/ui_guide.md @@ -1,9 +1,5 @@ # UI Guide for building GitLab -## Best practices for creating new pages in GitLab - -TODO: write some best practices when develop GitLab features. - ## GitLab UI development kit We created a page inside GitLab where you can check commonly used html and css elements. diff --git a/doc/incoming_email/README.md b/doc/incoming_email/README.md index 4cfb8402943..5a9a1582877 100644 --- a/doc/incoming_email/README.md +++ b/doc/incoming_email/README.md @@ -1,36 +1,99 @@ # Reply by email -GitLab can be set up to allow users to comment on issues and merge requests by replying to notification emails. +GitLab can be set up to allow users to comment on issues and merge requests by +replying to notification emails. -## Get a mailbox +## Requirement -Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises. +Reply by email requires an IMAP-enabled email account. GitLab allows you to use +three strategies for this feature: +- using email sub-addressing +- using a dedicated email address +- using a catch-all mailbox -If you want to use Gmail / Google Apps with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255). +### Email sub-addressing -To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these instructions](./postfix.md). +**If your provider or server supports email sub-addressing, we recommend using it.** + +[Sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing) is +a feature where any email to `user+some_arbitrary_tag@example.com` will end up +in the mailbox for `user@example.com`, and is supported by providers such as +Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix +mail server which you can run on-premises. + +### Dedicated email address + +This solution is really simple to set up: you just have to create an email +address dedicated to receive your users' replies to GitLab notifications. + +### Catch-all mailbox + +A [catch-all mailbox](https://en.wikipedia.org/wiki/Catch-all) for a domain will +"catch all" the emails addressed to the domain that do not exist in the mail +server. + +## How it works? + +### 1. GitLab sends a notification email + +When GitLab sends a notification and Reply by email is enabled, the `Reply-To` +header is set to the address defined in your GitLab configuration, with the +`%{key}` placeholder (if present) replaced by a specific "reply key". In +addition, this "reply key" is also added to the `References` header. + +### 2. You reply to the notification email + +When you reply to the notification email, your email client will: + +- send the email to the `Reply-To` address it got from the notification email +- set the `In-Reply-To` header to the value of the `Message-ID` header from the + notification email +- set the `References` header to the value of the `Message-ID` plus the value of + the notification email's `References` header. + +### 3. GitLab receives your reply to the notification email + +When GitLab receives your reply, it will look for the "reply key" in the +following headers, in this order: + +1. the `To` header +1. the `References` header + +If it finds a reply key, it will be able to leave your reply as a comment on +the entity the notification was about (issue, merge request, commit...). + +For more details about the `Message-ID`, `In-Reply-To`, and `References headers`, +please consult [RFC 5322](https://tools.ietf.org/html/rfc5322#section-3.6.4). ## Set it up +If you want to use Gmail / Google Apps with Reply by email, make sure you have +[IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) +and [allowed less secure apps to access the account](https://support.google.com/accounts/answer/6010255). + +To set up a basic Postfix mail server with IMAP access on Ubuntu, follow +[these instructions](./postfix.md). + ### Omnibus package installations -1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the feature and fill in the details for your specific IMAP server and email account: +1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the + feature and fill in the details for your specific IMAP server and email account: ```ruby # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com gitlab_rails['incoming_email_enabled'] = true - - # The email address including a placeholder for the key that references the item being replied to. - # The `%{key}` placeholder is added after the user part, before the `@`. + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). gitlab_rails['incoming_email_address'] = "incoming+%{key}@gitlab.example.com" - + # Email account username # With third party providers, this is usually the full email address. # With self-hosted email servers, this is usually the user part of the email address. gitlab_rails['incoming_email_email'] = "incoming" # Email account password gitlab_rails['incoming_email_password'] = "[REDACTED]" - + # IMAP server host gitlab_rails['incoming_email_host'] = "gitlab.example.com" # IMAP server port @@ -47,18 +110,18 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these ```ruby # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com gitlab_rails['incoming_email_enabled'] = true - + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com" - + # Email account username # With third party providers, this is usually the full email address. # With self-hosted email servers, this is usually the user part of the email address. gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com" # Email account password gitlab_rails['incoming_email_password'] = "[REDACTED]" - + # IMAP server host gitlab_rails['incoming_email_host'] = "imap.gmail.com" # IMAP server port @@ -72,8 +135,6 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these gitlab_rails['incoming_email_mailbox_name'] = "inbox" ``` - As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`. - 1. Reconfigure GitLab and restart mailroom for the changes to take effect: ```sh @@ -97,7 +158,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these cd /home/git/gitlab ``` -1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account: +1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature + and fill in the details for your specific IMAP server and email account: ```sh sudo editor config/gitlab.yml @@ -109,7 +171,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these enabled: true # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). address: "incoming+%{key}@gitlab.example.com" # Email account username @@ -138,7 +200,7 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these enabled: true # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. - # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). address: "gitlab-incoming+%{key}@gmail.com" # Email account username @@ -161,8 +223,6 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these mailbox: "inbox" ``` - As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`. - 1. Enable `mail_room` in the init script at `/etc/default/gitlab`: ```sh @@ -195,8 +255,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these incoming_email: enabled: true - # The email address including a placeholder for the key that references the item being replied to. - # The `%{key}` placeholder is added after the user part, before the `@`. + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The placeholder can be omitted but if present, it must appear in the "user" part of the address (before the `@`). address: "gitlab-incoming+%{key}@gmail.com" # Email account username diff --git a/doc/install/installation.md b/doc/install/installation.md index bffbc776500..f8f7d6a9ebe 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -227,9 +227,9 @@ sudo usermod -aG redis git ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-6-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-7-stable gitlab -**Note:** You can change `8-6-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `8-7-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It @@ -283,9 +283,13 @@ sudo usermod -aG redis git # Copy the example Rack attack config sudo -u git -H cp config/initializers/rack_attack.rb.example config/initializers/rack_attack.rb - # Configure Git global settings for git user, used when editing via web editor + # Configure Git global settings for git user + # 'autocrlf' is needed for the web editor sudo -u git -H git config --global core.autocrlf input + # Disable 'git gc --auto' because GitLab already runs 'git gc' when needed + sudo -u git -H git config --global gc.auto 0 + # Configure Redis connection settings sudo -u git -H cp config/resque.yml.example config/resque.yml diff --git a/doc/integration/README.md b/doc/integration/README.md index 7c8f785a61f..6fe04aa2a06 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -19,26 +19,15 @@ See the documentation below for details on how to configure these services. GitLab Enterprise Edition contains [advanced Jenkins support][jenkins]. +[jenkins]: http://doc.gitlab.com/ee/integration/jenkins.html + + ## Project services Integration with services such as Campfire, Flowdock, Gemnasium, HipChat, Pivotal Tracker, and Slack are available in the form of a [Project Service][]. -You can find these within GitLab in the Services page under Project Settings if -you are at least a master on the project. -Project Services are a bit like plugins in that they allow a lot of freedom in -adding functionality to GitLab. For example there is also a service that can -send an email every time someone pushes new commits. -Because GitLab is open source we can ship with the code and tests for all -plugins. This allows the community to keep the plugins up to date so that they -always work in newer GitLab versions. - -For an overview of what projects services are available without logging in, -please see the [project_services directory][projects-code]. - -[jenkins]: http://doc.gitlab.com/ee/integration/jenkins.html [Project Service]: ../project_services/project_services.md -[projects-code]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/models/project_services ## SSL certificate errors diff --git a/doc/integration/github.md b/doc/integration/github.md index 886784a27c9..1890edd7a4c 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -17,7 +17,7 @@ GitHub will generate an application ID and secret key for you to use. - Application name: This can be anything. Consider something like "\<Organization\>'s GitLab" or "\<Your Name\>'s GitLab" or something else descriptive. - Homepage URL: The URL to your GitLab installation. 'https://gitlab.company.com' - Application description: Fill this in if you wish. - - Authorization callback URL: 'https://gitlab.company.com/' + - Default authorization callback URL is '${YOUR_DOMAIN}/import/github/callback' 1. Select "Register application". 1. You should now see a Client ID and Client Secret near the top right of the page (see screenshot). diff --git a/doc/integration/ldap.md b/doc/integration/ldap.md index cf1f98492ea..30f0c15dacc 100644 --- a/doc/integration/ldap.md +++ b/doc/integration/ldap.md @@ -1,228 +1,3 @@ # GitLab LDAP integration -GitLab can be configured to allow your users to sign with their LDAP credentials to integrate with e.g. Active Directory. - -The first time a user signs in with LDAP credentials, GitLab will create a new GitLab user associated with the LDAP Distinguished Name (DN) of the LDAP user. - -GitLab user attributes such as nickname and email will be copied from the LDAP user entry. - -## Security - -GitLab assumes that LDAP users are not able to change their LDAP 'mail', 'email' or 'userPrincipalName' attribute. -An LDAP user who is allowed to change their email on the LDAP server can [take over any account](#enabling-ldap-sign-in-for-existing-gitlab-users) on your GitLab server. - -We recommend against using GitLab LDAP integration if your LDAP users are allowed to change their 'mail', 'email' or 'userPrincipalName' attribute on the LDAP server. - -If a user is deleted from the LDAP server, they will be blocked in GitLab as well. -Users will be immediately blocked from logging in. However, there is an LDAP check -cache time of one hour. The means users that are already logged in or are using Git -over SSH will still be able to access GitLab for up to one hour. Manually block -the user in the GitLab Admin area to immediately block all access. - -## Configuring GitLab for LDAP integration - -To enable GitLab LDAP integration you need to add your LDAP server settings in `/etc/gitlab/gitlab.rb` or `/home/git/gitlab/config/gitlab.yml`. -In GitLab Enterprise Edition you can have multiple LDAP servers connected to one GitLab server. - -Please note that before version 7.4, GitLab used a different syntax for configuring LDAP integration. -The old LDAP integration syntax still works in GitLab 7.4. -If your `gitlab.rb` or `gitlab.yml` file contains LDAP settings in both the old syntax and the new syntax, only the __old__ syntax will be used by GitLab. - -```ruby -# For omnibus packages -gitlab_rails['ldap_enabled'] = true -gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below -main: # 'main' is the GitLab 'provider ID' of this LDAP server - ## label - # - # A human-friendly name for your LDAP server. It is OK to change the label later, - # for instance if you find out it is too large to fit on the web page. - # - # Example: 'Paris' or 'Acme, Ltd.' - label: 'LDAP' - - host: '_your_ldap_server' - port: 389 - uid: 'sAMAccountName' - method: 'plain' # "tls" or "ssl" or "plain" - bind_dn: '_the_full_dn_of_the_user_you_will_bind_with' - password: '_the_password_of_the_bind_user' - - # Set a timeout, in seconds, for LDAP queries. This helps avoid blocking - # a request if the LDAP server becomes unresponsive. - # A value of 0 means there is no timeout. - timeout: 10 - - # This setting specifies if LDAP server is Active Directory LDAP server. - # For non AD servers it skips the AD specific queries. - # If your LDAP server is not AD, set this to false. - active_directory: true - - # If allow_username_or_email_login is enabled, GitLab will ignore everything - # after the first '@' in the LDAP username submitted by the user on login. - # - # Example: - # - the user enters 'jane.doe@example.com' and 'p@ssw0rd' as LDAP credentials; - # - GitLab queries the LDAP server with 'jane.doe' and 'p@ssw0rd'. - # - # If you are using "uid: 'userPrincipalName'" on ActiveDirectory you need to - # disable this setting, because the userPrincipalName contains an '@'. - allow_username_or_email_login: false - - # To maintain tight control over the number of active users on your GitLab installation, - # enable this setting to keep new users blocked until they have been cleared by the admin - # (default: false). - block_auto_created_users: false - - # Base where we can search for users - # - # Ex. ou=People,dc=gitlab,dc=example - # - base: '' - - # Filter LDAP users - # - # Format: RFC 4515 https://tools.ietf.org/search/rfc4515 - # Ex. (employeeType=developer) - # - # Note: GitLab does not support omniauth-ldap's custom filter syntax. - # - user_filter: '' - - # LDAP attributes that GitLab will use to create an account for the LDAP user. - # The specified attribute can either be the attribute name as a string (e.g. 'mail'), - # or an array of attribute names to try in order (e.g. ['mail', 'email']). - # Note that the user's LDAP login will always be the attribute specified as `uid` above. - attributes: - # The username will be used in paths for the user's own projects - # (like `gitlab.example.com/username/project`) and when mentioning - # them in issues, merge request and comments (like `@username`). - # If the attribute specified for `username` contains an email address, - # the GitLab username will be the part of the email address before the '@'. - username: ['uid', 'userid', 'sAMAccountName'] - email: ['mail', 'email', 'userPrincipalName'] - - # If no full name could be found at the attribute specified for `name`, - # the full name is determined using the attributes specified for - # `first_name` and `last_name`. - name: 'cn' - first_name: 'givenName' - last_name: 'sn' - -# GitLab EE only: add more LDAP servers -# Choose an ID made of a-z and 0-9 . This ID will be stored in the database -# so that GitLab can remember which LDAP server a user belongs to. -# uswest2: -# label: -# host: -# .... -EOS -``` - -If you are getting 'Connection Refused' errors when trying to connect to the LDAP server please double-check the LDAP `port` and `method` settings used by GitLab. -Common combinations are `method: 'plain'` and `port: 389`, OR `method: 'ssl'` and `port: 636`. - -If you are using a GitLab installation from source you can find the LDAP settings in `/home/git/gitlab/config/gitlab.yml`: - -``` -production: - # snip... - ldap: - enabled: false - servers: - main: # 'main' is the GitLab 'provider ID' of this LDAP server - ## label - # - # A human-friendly name for your LDAP server. It is OK to change the label later, - # for instance if you find out it is too large to fit on the web page. - # - # Example: 'Paris' or 'Acme, Ltd.' - label: 'LDAP' - # snip... -``` - -## Enabling LDAP sign-in for existing GitLab users - -When a user signs in to GitLab with LDAP for the first time, and their LDAP email address is the primary email address of an existing GitLab user, then the LDAP DN will be associated with the existing user. - -If the LDAP email attribute is not found in GitLab's database, a new user is created. - -In other words, if an existing GitLab user wants to enable LDAP sign-in for themselves, they should check that their GitLab email address matches their LDAP email address, and then sign into GitLab via their LDAP credentials. - -GitLab recognizes the following LDAP attributes as email addresses: `mail`, `email` and `userPrincipalName`. - -If multiple LDAP email attributes are present, e.g. `mail: foo@bar.com` and `email: foo@example.com`, then the first attribute found wins -- in this case `foo@bar.com`. - -## Using an LDAP filter to limit access to your GitLab server - -If you want to limit all GitLab access to a subset of the LDAP users on your LDAP server you can set up an LDAP user filter. -The filter must comply with [RFC 4515](https://tools.ietf.org/search/rfc4515). - -```ruby -# For omnibus packages; new LDAP server syntax -gitlab_rails['ldap_servers'] = YAML.load <<-EOS -main: - # snip... - user_filter: '(employeeType=developer)' -EOS -``` - -```yaml -# For installations from source; new LDAP server syntax -production: - ldap: - servers: - main: - # snip... - user_filter: '(employeeType=developer)' -``` - -Tip: if you want to limit access to the nested members of an Active Directory group you can use the following syntax: - -``` -(memberOf:1.2.840.113556.1.4.1941:=CN=My Group,DC=Example,DC=com) -``` - -Please note that GitLab does not support the custom filter syntax used by omniauth-ldap. - -## Limitations - -GitLab's LDAP client is based on [omniauth-ldap](https://gitlab.com/gitlab-org/omniauth-ldap) -which encapsulates Ruby's `Net::LDAP` class. It provides a pure-Ruby implementation -of the LDAP client protocol. As a result, GitLab is limited by `omniauth-ldap` and may impact your LDAP -server settings. - -### TLS Client Authentication -Not implemented by `Net::LDAP`. -So you should disable anonymous LDAP authentication and enable simple or SASL -authentication. TLS client authentication setting in your LDAP server cannot be -mandatory and clients cannot be authenticated with the TLS protocol. - -### TLS Server Authentication -Not supported by GitLab's configuration options. -When setting `method: ssl`, the underlying authentication method used by -`omniauth-ldap` is `simple_tls`. This method establishes TLS encryption with -the LDAP server before any LDAP-protocol data is exchanged but no validation of -the LDAP server's SSL certificate is performed. - -## Troubleshooting - -### Invalid credentials when logging in - -Make sure the user you are binding with has enough permissions to read the user's -tree and traverse it. - -Also make sure that the `user_filter` is not blocking otherwise valid users. - -To make sure that the LDAP settings are correct and GitLab can see your users, -execute the following command: - - -```bash -# For Omnibus installations -sudo gitlab-rake gitlab:ldap:check - -# For installations from source -sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production -``` - +This document was moved under [`administration/auth/ldap`](../administration/auth/ldap.md). diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 1c3dc707f6d..8a7205caaa4 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -131,8 +131,75 @@ On the sign in page there should now be a SAML button below the regular sign in Click the icon to begin the authentication process. If everything goes well the user will be returned to GitLab and will be signed in. +## External Groups + +>**Note:** +This setting is only available on GitLab 8.7 and above. + +SAML login includes support for external groups. You can define in the SAML +settings which groups, to which your users belong in your IdP, you wish to be +marked as [external](../permissions/permissions.md). + +### Requirements + +First you need to tell GitLab where to look for group information. For this you +need to make sure that your IdP server sends a specific `AttributeStament` along +with the regular SAML response. Here is an example: + +```xml +<saml:AttributeStatement> + <saml:Attribute Name="Groups"> + <saml:AttributeValue xsi:type="xs:string">SecurityGroup</saml:AttributeValue> + <saml:AttributeValue xsi:type="xs:string">Developers</saml:AttributeValue> + <saml:AttributeValue xsi:type="xs:string">Designers</saml:AttributeValue> + </saml:Attribute> +</saml:AttributeStatement> +``` + +The name of the attribute can be anything you like, but it must contain the groups +to which a user belongs. In order to tell GitLab where to find these groups, you need +to add a `groups_attribute:` element to your SAML settings. You will also need to +tell GitLab which groups are external via the `external_groups:` element: + +```yaml +{ name: 'saml', + label: 'Our SAML Provider', + groups_attribute: 'Groups', + external_groups: ['Freelancers', 'Interns'], + args: { + assertion_consumer_service_url: 'https://gitlab.example.com/users/auth/saml/callback', + idp_cert_fingerprint: '43:51:43:a1:b5:fc:8b:b7:0a:3a:a9:b1:0f:66:73:a8', + idp_sso_target_url: 'https://login.example.com/idp', + issuer: 'https://gitlab.example.com', + name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' + } } +``` + ## Customization +### `auto_sign_in_with_provider` + +You can add this setting to your GitLab configuration to automatically redirect you +to your SAML server for authentication, thus removing the need to click a button +before actually signing in. + +For omnibus package: + +```ruby +gitlab_rails['omniauth_auto_sign_in_with_provider'] = 'saml' +``` + +For installations from source: + +```yaml +omniauth: + auto_sign_in_with_provider: saml +``` + +Please keep in mind that every sign in attempt will be redirected to the SAML server, +so you will not be able to sign in using local credentials. Make sure that at least one +of the SAML users has admin permissions. + ### `attribute_statements` >**Note:** @@ -205,6 +272,10 @@ To bypass this you can add `skip_before_action :verify_authenticity_token` to th where it can then be seen in the usual logs, or as a flash message in the login screen. +That file is located at `/opt/gitlab/embedded/service/gitlab-rails/app/controllers` +for Omnibus installations and by default on `/home/git/gitlab/app/controllers` for +installations from source. + ### Invalid audience This error means that the IdP doesn't recognize GitLab as a valid sender and diff --git a/doc/markdown/markdown.md b/doc/markdown/markdown.md index e6eb1cf3819..4f199b6af6f 100644 --- a/doc/markdown/markdown.md +++ b/doc/markdown/markdown.md @@ -31,7 +31,7 @@ _GitLab uses the [Redcarpet Ruby library][redcarpet] for Markdown processing._ -For GitLab we developed something we call "GitLab Flavored Markdown" (GFM). It extends the standard Markdown in a few significant ways to add some useful functionality. +GitLab uses "GitLab Flavored Markdown" (GFM). It extends the standard Markdown in a few significant ways to add some useful functionality. It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). You can use GFM in @@ -47,10 +47,10 @@ You can also use other rich text files in GitLab. You might have to install a de GFM honors the markdown specification in how [paragraphs and line breaks are handled](https://daringfireball.net/projects/markdown/syntax#p). -A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines. +A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines. Line-breaks, or softreturns, are rendered if you end a line with two or more spaces - Roses are red [followed by two or more spaces] + Roses are red [followed by two or more spaces] Violets are blue Sugar is sweet @@ -67,7 +67,7 @@ It is not reasonable to italicize just _part_ of a word, especially when you're perform_complicated_task do_this_and_do_that_and_another_thing -perform_complicated_task +perform_complicated_task do_this_and_do_that_and_another_thing ## URL auto-linking diff --git a/doc/permissions/permissions.md b/doc/permissions/permissions.md index 3d375e47c8e..6219693b8a8 100644 --- a/doc/permissions/permissions.md +++ b/doc/permissions/permissions.md @@ -52,10 +52,11 @@ documentation](../workflow/add-user/add-user.md). | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | | Remove project | | | | | ✓ | -| Force push to protected branches | | | | | | -| Remove protected branches | | | | | | +| Force push to protected branches [^2] | | | | | | +| Remove protected branches [^2] | | | | | | [^1]: If **Allow guest to access builds** is enabled in CI settings +[^2]: Not allowed for Guest, Reporter, Developer, Master, or Owner ## Group diff --git a/doc/project_services/project_services.md b/doc/project_services/project_services.md index 3fea2cff0b9..a5af620d9be 100644 --- a/doc/project_services/project_services.md +++ b/doc/project_services/project_services.md @@ -1,7 +1,24 @@ # Project Services Project services allow you to integrate GitLab with other applications. Below -is list of the currently supported ones. Click on the service links to see +is list of the currently supported ones. + +You can find these within GitLab in the Services page under Project Settings if +you are at least a master on the project. +Project Services are a bit like plugins in that they allow a lot of freedom in +adding functionality to GitLab. For example there is also a service that can +send an email every time someone pushes new commits. + +Because GitLab is open source we can ship with the code and tests for all +plugins. This allows the community to keep the plugins up to date so that they +always work in newer GitLab versions. + +For an overview of what projects services are available without logging in, +please see the [project_services directory][projects-code]. + +[projects-code]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/app/models/project_services + +Click on the service links to see further configuration instructions and details. Contributions are welcome. ## Services diff --git a/doc/release/README.md b/doc/release/README.md deleted file mode 100644 index 52eca7c02a6..00000000000 --- a/doc/release/README.md +++ /dev/null @@ -1,10 +0,0 @@ -## Release cycle - -Since 2011 a minor or major version of GitLab is released on the 22nd of every month. Patch and security releases are published when needed. New features are detailed on the [blog](https://about.gitlab.com/blog/) and in the [changelog](CHANGELOG). Features that will likely be in the next releases can be found on the [direction page](https://about.gitlab.com/direction/). - -## Release process documentation - -- [Monthly release](monthly.md), every month on the 22nd. -- [Patch release](patch.md), if there are serious regressions. -- [Security](security.md), for security problems. -- [Master](master.md), update process for the master branch. diff --git a/doc/release/howto_rc1.md b/doc/release/howto_rc1.md deleted file mode 100644 index 07c703142d4..00000000000 --- a/doc/release/howto_rc1.md +++ /dev/null @@ -1,55 +0,0 @@ -# How to create RC1 - -The RC1 release comes with the task to update the installation and upgrade docs. Be mindful that there might already be merge requests for this on GitLab or GitHub. - -### 1. Update the installation guide - -1. Check if it references the correct branch `x-x-stable` (doesn't exist yet, but that is okay) -1. Check the [GitLab Shell version](/lib/tasks/gitlab/check.rake#L782) -1. Check the [Git version](/lib/tasks/gitlab/check.rake#L794) -1. There might be other changes. Ask around. - -### 2. Create update guides - -[Follow this guide](howto_update_guides.md) to create update guides. - -### 3. Code quality indicators - -Make sure the code quality indicators are green / good. - -- [![Build status](http://ci.gitlab.org/projects/1/status.png?ref=master)](http://ci.gitlab.org/projects/1?ref=master) on ci.gitlab.org (master branch) - -- [![Build Status](https://semaphoreapp.com/api/v1/projects/2f1a5809-418b-4cc2-a1f4-819607579fe7/243338/badge.png)](https://semaphoreapp.com/gitlabhq/gitlabhq) (master branch) - -- [![Code Climate](https://codeclimate.com/github/gitlabhq/gitlabhq.png)](https://codeclimate.com/github/gitlabhq/gitlabhq) - -- [![Dependency Status](https://gemnasium.com/gitlabhq/gitlabhq.png)](https://gemnasium.com/gitlabhq/gitlabhq) this button can be yellow (small updates are available) but must not be red (a security fix or an important update is available) - -- [![Coverage Status](https://coveralls.io/repos/gitlabhq/gitlabhq/badge.png?branch=master)](https://coveralls.io/r/gitlabhq/gitlabhq) - -### 4. Run release tool - -**Make sure EE `master` has latest changes from CE `master`** - -Get release tools - -``` -git clone git@dev.gitlab.org:gitlab/release-tools.git -cd release-tools -``` - -Release candidate creates stable branch from master. -So we need to sync master branch between all CE, EE and CI remotes. - -``` -bundle exec rake sync -``` - -Create release candidate and stable branch: - -``` -bundle exec rake release["x.x.0.rc1"] -``` - -Now developers can use master for merging new features. -So you should use stable branch for future code changes related to release. diff --git a/doc/release/howto_update_guides.md b/doc/release/howto_update_guides.md deleted file mode 100644 index 23d0959c33d..00000000000 --- a/doc/release/howto_update_guides.md +++ /dev/null @@ -1,55 +0,0 @@ -# Create update guides - -1. Create: CE update guide from previous version. Like `7.3-to-7.4.md` -1. Create: CE to EE update guide in EE repository for latest version. -1. Update: `6.x-or-7.x-to-7.x.md` to latest version. -1. Create: CI update guide from previous version - -It's best to copy paste the previous guide and make changes where necessary. -The typical steps are listed below with any points you should specifically look at. - -#### 0. Any major changes? - -List any major changes here, so the user is aware of them before starting to upgrade. For instance: - -- Database updates -- Web server changes -- File structure changes - -#### 1. Stop server - -#### 2. Make backup - -#### 3. Do users need to update dependencies like `git`? - -- Check if the [GitLab Shell version](/lib/tasks/gitlab/check.rake#L782) changed since the last release. - -- Check if the [Git version](/lib/tasks/gitlab/check.rake#L794) changed since the last release. - -#### 4. Get latest code - -#### 5. Does GitLab shell need to be updated? - -#### 6. Install libs, migrations, etc. - -#### 7. Any config files updated since last release? - -Check if any of these changed since last release: - -- [lib/support/nginx/gitlab](/lib/support/nginx/gitlab) -- [lib/support/nginx/gitlab-ssl](/lib/support/nginx/gitlab-ssl) -- <https://gitlab.com/gitlab-org/gitlab-shell/commits/master/config.yml.example> -- [config/gitlab.yml.example](/config/gitlab.yml.example) -- [config/unicorn.rb.example](/config/unicorn.rb.example) -- [config/database.yml.mysql](/config/database.yml.mysql) -- [config/database.yml.postgresql](/config/database.yml.postgresql) -- [config/initializers/rack_attack.rb.example](/config/initializers/rack_attack.rb.example) -- [config/resque.yml.example](/config/resque.yml.example) - -#### 8. Need to update init script? - -Check if the `init.d/gitlab` script changed since last release: [lib/support/init.d/gitlab](/lib/support/init.d/gitlab) - -#### 9. Start application - -#### 10. Check application status diff --git a/doc/release/master.md b/doc/release/master.md deleted file mode 100644 index 9163e652003..00000000000 --- a/doc/release/master.md +++ /dev/null @@ -1,62 +0,0 @@ -# How to push GitLab CE master branch to all remotes. - -The source code of GitLab is available on multiple servers (with GitLab.com as the canonical source). -Synchronization between the repo's is done by the lead developer if there is no rush. -This happens a few times per workday on average. -If somebody else with access to all repo's wants to do it the instructions are below. -This is just to distribute changes, not to make them. - -## Add this to `.bashrc` or [your dotfiles](https://github.com/dosire/dotfiles/commit/52803ce3ac60d57632164b7713ff0041e86fa26c) - -```bash -gpa () -{ - git push origin ${1:-master} && git push gh ${1:-master} && git push gl ${1:-master} -} -``` - -## Then add remotes to your local repo - -```bash -cd my-gitlab-ce-repo - -git remote add origin git@dev.gitlab.org:gitlab/gitlabhq.git -git remote add gh git@github.com:gitlabhq/gitlabhq.git -git remote add gl git@gitlab.com:gitlab-org/gitlab-ce.git -``` - -## Push to all remotes - -```bash -gpa -``` - -# Yanking packages from packages.gitlab.com - -In case something went wrong with the release and there is a need to remove the packages you can yank the packages by following the -procedure described in [package cloud documentation](https://packagecloud.io/docs#yank_pkg). - -You need to have: - -1. `package_cloud` gem installed (sudo gem install package_cloud) -1. Email and password for packages.gitlab.com -1. Make sure that you are supplying the url to packages.gitlab.com (default is packagecloud.io) - -Example of yanking a package: - -```bash -package_cloud yank --url https://packages.gitlab.com gitlab/gitlab-ce/el/6 gitlab-ce-7.10.2~omnibus-1.x86_64.rpm -``` - -If you are attempting this for the first time the output will look something like: - -```bash -Looking for repository at gitlab/gitlab-ce... No config file exists at /Users/marin/.packagecloud. Login to create one. -Email: -marin@gitlab.com -Password: - -Got your token. Writing a config file to /Users/marin/.packagecloud... success! -success! -Attempting to yank package at gitlab/gitlab-ce/el/6/gitlab-ce-7.10.2~omnibus-1.x86_64.rpm...done! -``` diff --git a/doc/release/monthly.md b/doc/release/monthly.md deleted file mode 100644 index 907c19e65a0..00000000000 --- a/doc/release/monthly.md +++ /dev/null @@ -1,245 +0,0 @@ -# Monthly Release - -NOTE: This is a guide used by the GitLab the company to release GitLab. -As an end user you do not need to use this guide. - -The process starts 7 working days before the release. -The release manager doesn't have to perform all the work but must ensure someone is assigned. -The current release manager must schedule the appointment of the next release manager. -The new release manager should create overall issue to track the progress. -The release manager should be the only person pushing/merging commits to the x-y-stable branches. - -## Release Manager - -A release manager is selected that coordinates all releases the coming month, -including the patch releases for previous releases. -The release manager has to make sure all the steps below are done and delegated where necessary. -This person should also make sure this document is kept up to date and issues are created and updated. - -## Take vacations into account - -The time is measured in weekdays to compensate for weekends. -Do everything on time to prevent problems due to rush jobs or too little testing time. -Make sure that you take into account any vacations of maintainers. -If the release is falling behind immediately warn the team. - -## Create an overall issue and follow it - -Create an issue in the GitLab CE project. Name it "Release x.x" and tag it with -the `release` label for easier searching. Replace the dates with actual dates -based on the number of workdays before the release. All steps from issue -template are explained below: - -``` -### Xth: (7 working days before the 22nd) - -- [ ] Triage the [Omnibus milestone] - -### Xth: (6 working days before the 22nd) - -- [ ] Determine QA person and notify this person -- [ ] Check the tasks in [how to rc1 guide](https://dev.gitlab.org/gitlab/gitlabhq/blob/master/doc/release/howto_rc1.md) and delegate tasks if necessary -- [ ] Merge CE `master` into EE `master` via merge request (#LINK) -- [ ] Create CE and EE RC1 versions (#LINK) -- [ ] Build RC1 packages - -### Xth: (5 working days before the 22nd) - -- [ ] Do QA and fix anything coming out of it (#LINK) -- [ ] Close the [Omnibus milestone] -- [ ] Prepare the [blog post] - -### Xth: (4 working days before the 22nd) - -- [ ] Update GitLab.com with RC1 -- [ ] Create the regression issue in the CE issue tracker: - - ``` - This is a meta issue to index possible regressions in this monthly release - and any patch versions. - - Please do not raise or discuss issues directly in this issue but link to - issues that might warrant a patch release. If there is a Merge Request - that fixes the issue, please link to that as well. - - Please only post one regression issue and/or merge request per comment. - Comments will be updated by the release manager as they are addressed. - ``` - -- [ ] Tweet about RC1 release: - - ``` - GitLab x.y.0.rc1 is available: https://packages.gitlab.com/gitlab/unstable - Use at your own risk. Please link regressions issues from - LINK_TO_REGRESSION_ISSUE - ``` - -### Xth: (3 working days before the 22nd) - -- [ ] Merge `x-y-stable` into `x-y-stable-ee` -- [ ] Check that everyone is mentioned on the [blog post] using `@all` - -### Xth: (2 working days before the 22nd) - -- [ ] Check that MVP is added to the [MVP page] - -### Xth: (1 working day before the 22nd) - -- [ ] Merge `x-y-stable` into `x-y-stable-ee` -- [ ] Create CE and EE release candidates -- [ ] Create Omnibus tags and build packages for the latest release candidates -- [ ] Update GitLab.com with the latest RC - -### 22nd before 1200 CET: - -Release before 1200 CET / 2AM PST, to make sure the majority of our users -get the new version on the 22nd and there is sufficient time in the European -workday to quickly fix any issues. - -- [ ] Merge `x-y-stable` into `x-y-stable-ee` -- [ ] Create the 'x.y.0' tag with the [release tools](https://dev.gitlab.org/gitlab/release-tools) -- [ ] Create the 'x.y.0' version on version.gitlab.com -- [ ] Try to do before 1100 CET: Create and push Omnibus tags for x.y.0 (will auto-release the packages) -- [ ] Try to do before 1200 CET: Publish the release [blog post] -- [ ] Tweet about the release -- [ ] Schedule a second Tweet of the release announcement with the same text at 1800 CET / 8AM PST - -[Omnibus milestone]: LINK_TO_OMNIBUS_MILESTONE -[blog post]: LINK_TO_WIP_BLOG_POST -[MVP page]: https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/source/mvp/index.html -``` - -- - - - -## Update changelog - -Any changes not yet added to the changelog are added by lead developer and in that merge request the complete team is -asked if there is anything missing. - -There are three changelogs that need to be updated: CE, EE and CI. - -## Create RC1 (CE, EE, CI) - -[Follow this How-to guide](howto_rc1.md) to create RC1. - -## Prepare CHANGELOG for next release - -Once the stable branches have been created, update the CHANGELOG in `master` with the upcoming version, usually X.X.X.pre. - -On creating the stable branches, notify the core team and developers. - -## QA - -Create issue on dev.gitlab.org `gitlab` repository, named "GitLab X.X QA" in order to keep track of the progress. - -Use the omnibus packages created for RC1 of Enterprise Edition using [this guide](https://dev.gitlab.org/gitlab/gitlab-ee/blob/master/doc/release/manual_testing.md). - -**NOTE** Upgrader can only be tested when tags are pushed to all repositories. Do not forget to confirm it is working before releasing. Note that in the issue. - -#### Fix anything coming out of the QA - -Create an issue with description of a problem, if it is quick fix fix it yourself otherwise contact the team for advice. - -**NOTE** If there is a problem that cannot be fixed in a timely manner, reverting the feature is an option! If the feature is reverted, -create an issue about it in order to discuss the next steps after the release. - -## Update GitLab.com with RC1 - -Use the omnibus EE packages created for RC1. -If there are big database migrations consider testing them with the production db on a VM. -Try to deploy in the morning. -It is important to do this as soon as possible, so we can catch any errors before we release the full version. - -## Create a regressions issue - -On [the GitLab CE issue tracker on GitLab.com](https://gitlab.com/gitlab-org/gitlab-ce/issues/) create an issue titled "GitLab X.X regressions" add the following text: - -This is a meta issue to discuss possible regressions in this monthly release and any patch versions. -Please do not raise issues directly in this issue but link to issues that might warrant a patch release. -The decision to create a patch release or not is with the release manager who is assigned to this issue. -The release manager will comment here about the plans for patch releases. - -Assign the issue to the release manager and at mention all members of GitLab core team. If there are any known bugs in the release add them immediately. - -## Tweet about RC1 - -Tweet about the RC release: - -> GitLab x.x.0.rc1 is out. This release candidate is only suitable for testing. Please link regressions issues from LINK_TO_REGRESSION_ISSUE - -## Prepare the blog post - -1. The blog post template for this release should already exist and might have comments that were added during the month. -1. Fill out as much of the blog post template as you can. -1. Make sure the blog post contains information about the GitLab CI release. -1. Check the changelog of CE and EE for important changes. -1. Also check the CI changelog -1. Add a proposed tweet text to the blog post WIP MR description. -1. Create a WIP MR for the blog post -1. Make sure merge request title starts with `WIP` so it can not be accidentally merged until ready. -1. Ask Dmitriy (or a team member with OS X) to add screenshots to the WIP MR. -1. Decide with core team who will be the MVP user. -1. Create WIP MR for adding MVP to MVP page on website -1. Add a note if there are security fixes: This release fixes an important security issue and we advise everyone to upgrade as soon as possible. -1. Create a merge request on [GitLab.com](https://gitlab.com/gitlab-com/www-gitlab-com/tree/master) -1. Assign to one reviewer who will fix spelling issues by editing the branch (either with a git client or by using the online editor) -1. Comment to the reviewer: '@person Please mention the whole team as soon as you are done (3 workdays before release at the latest)' -1. Create a new merge request with complete copy of the [release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/release_blog_template.md) for the next release using the branch name `release-x-x-x`. - -## Create CE, EE, CI stable versions - -Get release tools - -``` -git clone git@dev.gitlab.org:gitlab/release-tools.git -cd release-tools -``` - -Bump version, create release tag and push to remotes: - -``` -bundle exec rake release["x.x.0"] -``` - -This will create correct version and tag and push to all CE, EE and CI remotes. - -Update [installation.md](/doc/install/installation.md) to the newest version in master. - - -## Create Omnibus tags and build packages - -Follow the [release doc in the Omnibus repository](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/release.md). -This can happen before tagging because Omnibus uses tags in its own repo and SHA1's to refer to the GitLab codebase. - -## Update GitLab.com with the stable version - -- Deploy the package (should not need downtime because of the small difference with RC1) -- Deploy the package for gitlab.com/ci - -## Release CE, EE and CI - -__1. Publish packages for new release__ - -Update `downloads/index.html` and `downloads/archive/index.html` in `www-gitlab-com` repository. - -__2. Publish blog for new release__ - -Doublecheck the everyone has been mentioned in the blog post. -Merge the [blog merge request](#1-prepare-the-blog-post) in `www-gitlab-com` repository. - -__3. Tweet to blog__ - -Send out a tweet to share the good news with the world. -List the most important features and link to the blog post. - -Proposed tweet "Release of GitLab X.X & CI Y.Y! FEATURE, FEATURE and FEATURE <link-to-blog-post> #gitlab" - -Consider creating a post on Hacker News. - -## Release new AMIs - -[Follow this guide](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md) - -## Create a WIP blogpost for the next release - -Create a WIP blogpost using [release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/release_blog_template.md). diff --git a/doc/release/patch.md b/doc/release/patch.md deleted file mode 100644 index 1c921439156..00000000000 --- a/doc/release/patch.md +++ /dev/null @@ -1,81 +0,0 @@ -# Things to do when doing a patch release - -NOTE: This is a guide for GitLab developers. If you are trying to install GitLab -see the latest stable [installation guide](install/installation.md) and if you -are trying to upgrade, see the [upgrade guides](update). - -## When to do a patch release - -Patch releases are done as-needed in order to fix regressions in the current -major release that cannot or should not wait until the next major release. -What's included and when to release is at the discretion of the release manager. - -## Release Procedure - -### Create a patch issue - -Create an issue in the GitLab CE project. Name it "Release x.y.z", tag it with -the `release` label, and assign it to the milestone of the corresponding major -release. - -Use the following template: - -``` -- Picked into respective `stable` branches: -- [ ] Merge `x-y-stable` into `x-y-stable-ee` -- [ ] release-tools: `x.y.z` -- omnibus-gitlab - - [ ] `x.y.z+ee.0` - - [ ] `x.y.z+ce.0` -- [ ] Deploy -- [ ] Add patch notice to [x.y regressions]() -- [ ] [Blog post]() -- [ ] [Tweet]() -- [ ] Add entry to version.gitlab.com -``` - -Update the issue with links to merge requests that need to be/have been picked -into the `stable` branches. - -### Preparation - -1. Verify that the issue can be reproduced -1. Note in the 'GitLab X.X regressions' that you will create a patch -1. Fix the issue on a feature branch, do this on the private GitLab development server -1. If it is a security issue, then assign it to the release manager and apply a 'security' label -1. Consider creating and testing workarounds -1. After the branch is merged into master, cherry pick the commit(s) into the current stable branch -1. Make sure that the build has passed and all tests are passing -1. In a separate commit in the master branch update the CHANGELOG -1. For EE, update the CHANGELOG-EE if it is EE specific fix. Otherwise, merge the stable CE branch and add to CHANGELOG-EE "Merge community edition changes for version X.X.X" -1. Merge CE stable branch into EE stable branch - -### Bump version - -Get release tools - -``` -git clone git@dev.gitlab.org:gitlab/release-tools.git -cd release-tools -``` - -Bump all versions in stable branch, even if the changes affect only EE, CE, or CI. Since all the versions are synced now, -it doesn't make sense to say upgrade CE to 7.2, EE to 7.3 and CI to 7.1. - -Create release tag and push to remotes: - -``` -bundle exec rake release["x.x.x"] -``` - -## Release - -1. [Build new packages with the latest version](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/release.md) -1. Apply the patch to GitLab.com and the private GitLab development server -1. Apply the patch to ci.gitLab.com and the private GitLab CI development server -1. Create and publish a blog post, see [patch release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/patch_release_blog_template.md) -1. Send tweets about the release from `@gitlab`, tweet should include the most important feature that the release is addressing and link to the blog post -1. Note in the 'GitLab X.X regressions' issue that the patch was published (CE only) -1. Create the 'x.y.0' version on version.gitlab.com -1. [Create new AMIs](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md) -1. Create a new patch release issue for the next potential release diff --git a/doc/release/security.md b/doc/release/security.md deleted file mode 100644 index 118c016ba4f..00000000000 --- a/doc/release/security.md +++ /dev/null @@ -1,76 +0,0 @@ -# Things to do when doing an out-of-bound security release - -NOTE: This is a guide for GitLab developers. If you are trying to install GitLab see the latest stable [installation guide](install/installation.md) and if you are trying to upgrade, see the [upgrade guides](update). - -## When to do a security release - -Do a security release when there is a critical issue that needs to be addresses before the next monthly release. Otherwise include it in the monthly release and note there was a security fix in the release announcement. - -## Security vulnerability disclosure - -Please report suspected security vulnerabilities in private to <support@gitlab.com>, also see the [disclosure section on the GitLab.com website](https://about.gitlab.com/disclosure/). Please do NOT create publicly viewable issues for suspected security vulnerabilities. - -## Release Procedure - -1. Verify that the issue can be reproduced -1. Acknowledge the issue to the researcher that disclosed it -1. Inform the release manager that there needs to be a security release -1. Do the steps from [patch release document](../release/patch.md), starting with "Create an issue on private GitLab development server" -1. The MR with the security fix should get a 'security' label and be assigned to the release manager -1. Build the package for GitLab.com and do a deploy -1. Build the package for ci.gitLab.com and do a deploy -1. [Create new AMIs](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md) -1. Create feature branches for the blog post on GitLab.com and link them from the code branch -1. Merge and publish the blog posts -1. Send tweets about the release from `@gitlabhq` -1. Send out an email to [the community google mailing list](https://groups.google.com/forum/#!forum/gitlabhq) -1. Post a signed copy of our complete announcement to [oss-security](http://www.openwall.com/lists/oss-security/) and request a CVE number. CVE is only needed for bugs that allow someone to own the server (Remote Code Execution) or access to code of projects they are not a member of. -1. Add the security researcher to the [Security Researcher Acknowledgments list](https://about.gitlab.com/vulnerability-acknowledgements/) -1. Thank the security researcher in an email for their cooperation -1. Update the blog post and the CHANGELOG when we receive the CVE number - -The timing of the code merge into master should be coordinated in advance. - -After the merge we strive to publish the announcements within 60 minutes. - -## Blog post template - -XXX Security Advisory for GitLab - -A recently discovered critical vulnerability in GitLab allows [unauthenticated API access|remote code execution|unauthorized access to repositories|XXX|PICKSOMETHING]. All users should update GitLab and gitlab-shell immediately. We [have|haven't|XXX|PICKSOMETHING|] heard of this vulnerability being actively exploited. - -### Version affected - -GitLab Community Edition XXX and lower - -GitLab Enterprise Edition XXX and lower - -### Fixed versions - -GitLab Community Edition XXX and up - -GitLab Enterprise Edition XXX and up - -### Impact - -On GitLab installations which use MySQL as their database backend it is possible for an attacker to assume the identity of any existing GitLab user in certain API calls. This attack can be performed by [unauthenticated|authenticated|XXX|PICKSOMETHING] users. - -### Workarounds - -If you are unable to upgrade you should apply the following patch and restart GitLab. - -XXX - -### Credit - -We want to thank XXX of XXX for the responsible disclosure of this vulnerability. - -## Email template - -We just announced a security advisory for GitLab at XXX - -Please contact us at support@gitlab.com if you have any questions. - -## Tweet template - -We just announced a security advisory for GitLab at XXX diff --git a/doc/update/8.5-to-8.6.md b/doc/update/8.5-to-8.6.md index 712e9fdf93a..b9abcbd2c12 100644 --- a/doc/update/8.5-to-8.6.md +++ b/doc/update/8.5-to-8.6.md @@ -62,7 +62,26 @@ sudo -u git -H git checkout v0.7.1 sudo -u git -H make ``` -### 6. Install libs, migrations, etc. +### 6. Updates for PostgreSQL Users + +Starting with 8.6 users using GitLab in combination with PostgreSQL are required +to have the `pg_trgm` extension enabled for all GitLab databases. If you're +using GitLab's Omnibus packages there's nothing you'll need to do manually as +this extension is enabled automatically. Users who install GitLab without using +Omnibus (e.g. by building from source) have to enable this extension manually. +To enable this extension run the following SQL command as a PostgreSQL super +user for _every_ GitLab database: + +```sql +CREATE EXTENSION IF NOT EXISTS pg_trgm; +``` + +Certain operating systems might require the installation of extra packages for +this extension to be available. For example, users using Ubuntu will have to +install the `postgresql-contrib` package in order for this extension to be +available. + +### 7. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -84,7 +103,7 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS ``` -### 7. Update configuration files +### 8. Update configuration files #### New configuration options for `gitlab.yml` @@ -120,25 +139,6 @@ Ensure you're still up-to-date with the latest init script changes: sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab -### 8. Updates for PostgreSQL Users - -Starting with 8.6 users using GitLab in combination with PostgreSQL are required -to have the `pg_trgm` extension enabled for all GitLab databases. If you're -using GitLab's Omnibus packages there's nothing you'll need to do manually as -this extension is enabled automatically. Users who install GitLab without using -Omnibus (e.g. by building from source) have to enable this extension manually. -To enable this extension run the following SQL command as a PostgreSQL super -user for _every_ GitLab database: - -```sql -CREATE EXTENSION IF NOT EXISTS pg_trgm; -``` - -Certain operating systems might require the installation of extra packages for -this extension to be available. For example, users using Ubuntu will have to -install the `postgresql-contrib` package in order for this extension to be -available. - ### 9. Start application sudo service gitlab start diff --git a/doc/update/8.6-to-8.7.md b/doc/update/8.6-to-8.7.md new file mode 100644 index 00000000000..8599133a726 --- /dev/null +++ b/doc/update/8.6-to-8.7.md @@ -0,0 +1,154 @@ +# From 8.6 to 8.7 + +Make sure you view this update guide from the tag (version) of GitLab you would +like to install. In most cases this should be the highest numbered production +tag (without rc in it). You can select the tag in the version dropdown at the +top left corner of GitLab (below the menu bar). + +If the highest number stable branch is unclear please check the +[GitLab Blog](https://about.gitlab.com/blog/archives.html) for installation +guide links by version. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-7-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-7-stable-ee +``` + +### 4. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch --all +sudo -u git -H git checkout v2.7.0 +``` + +### 5. Update gitlab-workhorse + +Install and compile gitlab-workhorse. This requires +[Go 1.5](https://golang.org/dl) which should already be on your system from +GitLab 8.1. + +```bash +cd /home/git/gitlab-workhorse +sudo -u git -H git fetch --all +sudo -u git -H git checkout v0.7.1 +sudo -u git -H make +``` + +### 6. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Optional: clean up old gems +sudo -u git -H bundle clean + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production + +``` + +### 7. Update configuration files + +#### Git configuration + +Disable `git gc --auto` because GitLab runs `git gc` for us already. + +```sh +sudo -u git -H git config --global gc.auto 0 +``` + +#### Nginx configuration + +Ensure you're still up-to-date with the latest NGINX configuration changes: + +```sh +# For HTTPS configurations +git diff origin/8-6-stable:lib/support/nginx/gitlab-ssl origin/8-7-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-6-stable:lib/support/nginx/gitlab origin/8-7-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-workhorse listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-7-stable/lib/support/init.d/gitlab.default.example#L37 + +#### Init script + +Ensure you're still up-to-date with the latest init script changes: + + sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab + +### 8. Start application + + sudo service gitlab start + sudo service nginx restart + +### 9. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.6) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 8.5 to 8.6](8.5-to-8.6.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. diff --git a/doc/web_hooks/web_hooks.md b/doc/web_hooks/web_hooks.md index afdf1a682e2..22e207b6d32 100644 --- a/doc/web_hooks/web_hooks.md +++ b/doc/web_hooks/web_hooks.md @@ -58,13 +58,13 @@ X-Gitlab-Event: Push Hook "path_with_namespace":"mike/diaspora", "default_branch":"master", "homepage":"http://example.com/mike/diaspora", - "url":"git@example.com:mike/diasporadiaspora.git", + "url":"git@example.com:mike/diaspora.git", "ssh_url":"git@example.com:mike/diaspora.git", "http_url":"http://example.com/mike/diaspora.git" }, "repository":{ "name": "Diaspora", - "url": "git@example.com:mike/diasporadiaspora.git", + "url": "git@example.com:mike/diaspora.git", "description": "", "homepage": "http://example.com/mike/diaspora", "git_http_url":"http://example.com/mike/diaspora.git", @@ -113,7 +113,6 @@ Triggered when you create (or delete) tags to the repository. X-Gitlab-Event: Tag Push Hook ``` - **Request body:** ```json @@ -143,7 +142,7 @@ X-Gitlab-Event: Tag Push Hook "http_url":"http://example.com/jsmith/example.git" }, "repository":{ - "name": "jsmith", + "name": "Example", "url": "ssh://git@example.com/jsmith/example.git", "description": "", "homepage": "http://example.com/jsmith/example", @@ -478,7 +477,7 @@ X-Gitlab-Event: Note Hook }, "repository":{ "name":"diaspora", - "url":"git@example.com:mike/diasporadiaspora.git", + "url":"git@example.com:mike/diaspora.git", "description":"", "homepage":"http://example.com/mike/diaspora" }, diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md index 36cb9da2380..9dc1e9b47e3 100644 --- a/doc/workflow/lfs/lfs_administration.md +++ b/doc/workflow/lfs/lfs_administration.md @@ -23,6 +23,10 @@ In `/etc/gitlab/gitlab.rb`: ```ruby gitlab_rails['lfs_enabled'] = false + +# Optionally, change the storage path location. Defaults to +# `#{gitlab_rails['shared_path']}/lfs-objects`. Which evaluates to +# `/var/opt/gitlab/gitlab-rails/shared/lfs-objects` by default. gitlab_rails['lfs_storage_path'] = "/mnt/storage/lfs-objects" ``` |