diff options
| author | Jacob Vosmaer <contact@jacobvosmaer.nl> | 2015-12-08 12:35:54 +0100 |
|---|---|---|
| committer | Jacob Vosmaer <contact@jacobvosmaer.nl> | 2015-12-08 12:35:54 +0100 |
| commit | 6d2be0212c444c6a3d25ae6a3c75822fa1c8614f (patch) | |
| tree | 66c12a4f7863ded395300a56de689b5cb669e84b /doc | |
| parent | ad37f58ebd97fee3c06a531e4067c8adb4c9ecc7 (diff) | |
| parent | f5430e48b42227f1c1874ca27c6907f0f704be28 (diff) | |
| download | gitlab-ce-6d2be0212c444c6a3d25ae6a3c75822fa1c8614f.tar.gz | |
Merge branch 'master' into sync-all-repos
Diffstat (limited to 'doc')
97 files changed, 2169 insertions, 556 deletions
diff --git a/doc/README.md b/doc/README.md index a0ff856ebb6..58ab5dd08e0 100644 --- a/doc/README.md +++ b/doc/README.md @@ -17,20 +17,22 @@ ## CI Documentation -+ [Quick Start](ci/quick_start/README.md) -+ [Configuring project (.gitlab-ci.yml)](ci/yaml/README.md) -+ [Configuring runner](ci/runners/README.md) -+ [Configuring deployment](ci/deployment/README.md) -+ [Using Docker Images](ci/docker/using_docker_images.md) -+ [Using Docker Build](ci/docker/using_docker_build.md) -+ [Using Variables](ci/variables/README.md) +- [Quick Start](ci/quick_start/README.md) +- [Configuring project (.gitlab-ci.yml)](ci/yaml/README.md) +- [Configuring runner](ci/runners/README.md) +- [Configuring deployment](ci/deployment/README.md) +- [Using Docker Images](ci/docker/using_docker_images.md) +- [Using Docker Build](ci/docker/using_docker_build.md) +- [Using Variables](ci/variables/README.md) +- [User permissions](ci/permissions/README.md) +- [API](ci/api/README.md) ### CI Examples -+ [Test and deploy Ruby applications to Heroku](ci/examples/test-and-deploy-ruby-application-to-heroku.md) -+ [Test and deploy Python applications to Heroku](ci/examples/test-and-deploy-python-application-to-heroku.md) -+ [Test Clojure applications](ci/examples/test-clojure-application.md) -+ Help your favorite programming language and GitLab by sending a merge request with a guide for that language. +- [Test and deploy Ruby applications to Heroku](ci/examples/test-and-deploy-ruby-application-to-heroku.md) +- [Test and deploy Python applications to Heroku](ci/examples/test-and-deploy-python-application-to-heroku.md) +- [Test Clojure applications](ci/examples/test-clojure-application.md) +- Help your favorite programming language and GitLab by sending a merge request with a guide for that language. ## Administrator documentation @@ -48,11 +50,7 @@ - [Welcome message](customization/welcome_message.md) Add a custom welcome message to the sign-in page. - [Reply by email](incoming_email/README.md) Allow users to comment on issues and merge requests by replying to notification emails. - [Migrate GitLab CI to CE/EE](migrate_ci_to_ce/README.md) Follow this guide to migrate your existing GitLab CI data to GitLab CE/EE. - -### Administrator documentation - -+ [User permissions](permissions/permissions.md) -+ [API](api/README.md) +- [Git LFS configuration](workflow/lfs/lfs_administration.md) ## Contributor documentation diff --git a/doc/api/README.md b/doc/api/README.md index 6b8528de50c..25a31b235cc 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -10,6 +10,7 @@ - [Repositories](repositories.md) - [Repository Files](repository_files.md) - [Commits](commits.md) +- [Tags](tags.md) - [Branches](branches.md) - [Merge Requests](merge_requests.md) - [Issues](issues.md) diff --git a/doc/api/commits.md b/doc/api/commits.md index eb8d6a43592..93d62b751e6 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -22,7 +22,8 @@ Parameters: "author_name": "Dmitriy Zaporozhets", "author_email": "dzaporozhets@sphereconsultinginc.com", "created_at": "2012-09-20T11:50:22+03:00", - "message": "Replace sanitize with escape once" + "message": "Replace sanitize with escape once", + "allow_failure": false }, { "id": "6104942438c14ec7bd21c6cd5bd995272b3faff6", @@ -31,7 +32,8 @@ Parameters: "author_name": "randx", "author_email": "dmitriy.zaporozhets@gmail.com", "created_at": "2012-09-20T09:06:12+03:00", - "message": "Sanitize for network graph" + "message": "Sanitize for network graph", + "allow_failure": false } ] ``` @@ -62,7 +64,8 @@ Parameters: "authored_date": "2012-09-20T09:06:12+03:00", "parent_ids": [ "ae1d9fb46aa2b07ee9836d49862ec4e2c46fbbba" - ] + ], + "status": "running" } ``` @@ -156,3 +159,84 @@ Parameters: "line_type": "new" } ``` + +## Get the status of a commit + +Get the statuses of a commit in a project. + +``` +GET /projects/:id/repository/commits/:sha/statuses +``` + +Parameters: + +- `id` (required) - The ID of a project +- `sha` (required) - The commit SHA +- `ref` (optional) - Filter by ref name, it can be branch or tag +- `stage` (optional) - Filter by stage +- `name` (optional) - Filer by status name, eg. jenkins +- `all` (optional) - The flag to return all statuses, not only latest ones + +```json +[ + { + "id": 13, + "sha": "b0b3a907f41409829b307a28b82fdbd552ee5a27", + "ref": "test", + "status": "success", + "name": "ci/jenkins", + "target_url": "http://jenkins/project/url", + "description": "Jenkins success", + "created_at": "2015-10-12T09:47:16.250Z", + "started_at": "2015-10-12T09:47:16.250Z", + "finished_at": "2015-10-12T09:47:16.262Z", + "author": { + "id": 1, + "username": "admin", + "email": "admin@local.host", + "name": "Administrator", + "blocked": false, + "created_at": "2012-04-29T08:46:00Z" + } + } +] +``` + +## Post the status to commit + +Adds or updates a status of a commit. + +``` +POST /projects/:id/statuses/:sha +``` + +- `id` (required) - The ID of a project +- `sha` (required) - The commit SHA +- `state` (required) - The state of the status. Can be: pending, running, success, failed, canceled +- `ref` (optional) - The ref (branch or tag) to which the status refers +- `name` or `context` (optional) - The label to differentiate this status from the status of other systems. Default: "default" +- `target_url` (optional) - The target URL to associate with this status +- `description` (optional) - The short description of the status + +```json +{ + "id": 13, + "sha": "b0b3a907f41409829b307a28b82fdbd552ee5a27", + "ref": "test", + "status": "success", + "name": "ci/jenkins", + "target_url": "http://jenkins/project/url", + "description": "Jenkins success", + "created_at": "2015-10-12T09:47:16.250Z", + "started_at": "2015-10-12T09:47:16.250Z", + "finished_at": "2015-10-12T09:47:16.262Z", + "author": { + "id": 1, + "username": "admin", + "email": "admin@local.host", + "name": "Administrator", + "blocked": false, + "created_at": "2012-04-29T08:46:00Z" + } +} +``` diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index bb551fc67f7..82f2cef969f 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -3,8 +3,9 @@ ## List merge requests Get all merge requests for this project. -The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). -The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. +The `state` parameter can be used to get only merge requests with a given state (`opened`, `closed`, or `merged`) or all of them (`all`). +The pagination parameters `page` and `per_page` can be used to restrict the list of merge requests. With GitLab 8.2 the return fields `upvotes` and +`downvotes` are deprecated and always return `0`. ``` GET /projects/:id/merge_requests @@ -57,7 +58,7 @@ Parameters: ## Get single MR -Shows information about a single merge request. +Shows information about a single merge request. With GitLab 8.2 the return fields `upvotes` and `downvotes` are deprecated and always return `0`. ``` GET /projects/:id/merge_request/:merge_request_id @@ -100,9 +101,48 @@ Parameters: } ``` +## Get single MR commits + +Get a list of merge request commits. + +``` +GET /projects/:id/merge_request/:merge_request_id/commits +``` + +Parameters: + +- `id` (required) - The ID of a project +- `merge_request_id` (required) - The ID of MR + + +```json +[ + { + "id": "ed899a2f4b50b4370feeea94676502b42383c746", + "short_id": "ed899a2f4b5", + "title": "Replace sanitize with escape once", + "author_name": "Dmitriy Zaporozhets", + "author_email": "dzaporozhets@sphereconsultinginc.com", + "created_at": "2012-09-20T11:50:22+03:00", + "message": "Replace sanitize with escape once" + }, + { + "id": "6104942438c14ec7bd21c6cd5bd995272b3faff6", + "short_id": "6104942438c", + "title": "Sanitize for network graph", + "author_name": "randx", + "author_email": "dmitriy.zaporozhets@gmail.com", + "created_at": "2012-09-20T09:06:12+03:00", + "message": "Sanitize for network graph" + } +] +``` + ## Get single MR changes -Shows information about the merge request including its files and changes +Shows information about the merge request including its files and changes. +With GitLab 8.2 the return fields `upvotes` and `downvotes` are deprecated and +always return `0`. ``` GET /projects/:id/merge_request/:merge_request_id/changes @@ -156,7 +196,7 @@ Parameters: "updated_at": "2015-02-02T19:49:26.013Z", "due_date": null }, - "files": [ + "changes": [ { "old_path": "VERSION", "new_path": "VERSION", @@ -173,7 +213,8 @@ Parameters: ## Create MR -Creates a new merge request. +Creates a new merge request. With GitLab 8.2 the return fields `upvotes` and ` +downvotes` are deprecated and always return `0`. ``` POST /projects/:id/merge_requests @@ -188,6 +229,7 @@ Parameters: - `title` (required) - Title of MR - `description` (optional) - Description of MR - `target_project_id` (optional) - The target project (numeric id) +- `labels` (optional) - Labels for MR as a comma-separated list ```json { @@ -224,7 +266,8 @@ If an error occurs, an error number and a message explaining the reason is retur ## Update MR -Updates an existing merge request. You can change the target branch, title, or even close the MR. +Updates an existing merge request. You can change the target branch, title, or even close the MR. With GitLab 8.2 the return fields `upvotes` and `downvotes` +are deprecated and always return `0`. ``` PUT /projects/:id/merge_request/:merge_request_id @@ -239,6 +282,7 @@ Parameters: - `title` - Title of MR - `description` - Description of MR - `state_event` - New state (close|reopen|merge) +- `labels` (optional) - Labels for MR as a comma-separated list ```json { @@ -274,7 +318,8 @@ If an error occurs, an error number and a message explaining the reason is retur ## Accept MR -Merge changes submitted with MR using this API. +Merge changes submitted with MR using this API. With GitLab 8.2 the return +fields `upvotes` and `downvotes` are deprecated and always return `0`. If merge success you get `200 OK`. diff --git a/doc/api/notes.md b/doc/api/notes.md index c683cb883d4..4d7ef288df8 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -6,7 +6,8 @@ Notes are comments on snippets, issues or merge requests. ### List project issue notes -Gets a list of all notes for a single issue. +Gets a list of all notes for a single issue. With GitLab 8.2 the return fields +`upvote` and `downvote` are deprecated and always return `false`. ``` GET /projects/:id/issues/:issue_id/notes @@ -34,7 +35,9 @@ Parameters: "created_at": "2013-10-02T09:22:45Z", "system": true, "upvote": false, - "downvote": false + "downvote": false, + "noteable_id": 377, + "noteable_type": "Issue" }, { "id": 305, @@ -49,9 +52,11 @@ Parameters: "created_at": "2013-09-30T13:46:01Z" }, "created_at": "2013-10-02T09:56:03Z", - "system": false, + "system": true, "upvote": false, - "downvote": false + "downvote": false, + "noteable_id": 121, + "noteable_type": "Issue" } ] ``` @@ -218,7 +223,12 @@ Parameters: "state": "active", "created_at": "2013-09-30T13:46:01Z" }, - "created_at": "2013-10-02T08:57:14Z" + "created_at": "2013-10-02T08:57:14Z", + "system": false, + "upvote": false, + "downvote": false, + "noteable_id": 2, + "noteable_type": "MergeRequest" } ``` diff --git a/doc/api/projects.md b/doc/api/projects.md index 96485857035..42919a312ae 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -60,6 +60,7 @@ Parameters: "path_with_namespace": "diaspora/diaspora-client", "issues_enabled": true, "merge_requests_enabled": true, + "builds_enabled": true, "wiki_enabled": true, "snippets_enabled": false, "created_at": "2013-09-30T13: 46: 02Z", @@ -101,6 +102,7 @@ Parameters: "path_with_namespace": "brightbox/puppet", "issues_enabled": true, "merge_requests_enabled": true, + "builds_enabled": true, "wiki_enabled": true, "snippets_enabled": false, "created_at": "2013-09-30T13:46:02Z", @@ -191,6 +193,7 @@ Parameters: "path_with_namespace": "diaspora/diaspora-project-site", "issues_enabled": true, "merge_requests_enabled": true, + "builds_enabled": true, "wiki_enabled": true, "snippets_enabled": false, "created_at": "2013-09-30T13: 46: 02Z", @@ -242,9 +245,17 @@ Parameters: "target_id": 830, "target_type": "Issue", "author_id": 1, - "author_username": "john", "data": null, - "target_title": "Public project search field" + "target_title": "Public project search field", + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "author_username": "root" }, { "title": null, @@ -253,6 +264,14 @@ Parameters: "target_id": null, "target_type": null, "author_id": 1, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, "author_username": "john", "data": { "before": "50d4420237a9de7be1304607147aec22e4a14af7", @@ -289,9 +308,56 @@ Parameters: "target_id": 840, "target_type": "Issue", "author_id": 1, - "author_username": "john", "data": null, - "target_title": "Finish & merge Code search PR" + "target_title": "Finish & merge Code search PR", + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "author_username": "root" + }, + { + "title": null, + "project_id": 15, + "action_name": "commented on", + "target_id": 1312, + "target_type": "Note", + "author_id": 1, + "data": null, + "target_title": null, + "created_at": "2015-12-04T10:33:58.089Z", + "note": { + "id": 1312, + "body": "What an awesome day!", + "attachment": null, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "created_at": "2015-12-04T10:33:56.698Z", + "system": false, + "upvote": false, + "downvote": false, + "noteable_id": 377, + "noteable_type": "Issue" + }, + "author": { + "name": "Dmitriy Zaporozhets", + "username": "root", + "id": 1, + "state": "active", + "avatar_url": "http://localhost:3000/uploads/user/avatar/1/fox_avatar.png", + "web_url": "http://localhost:3000/u/root" + }, + "author_username": "root" } ] ``` @@ -312,6 +378,7 @@ Parameters: - `description` (optional) - short project description - `issues_enabled` (optional) - `merge_requests_enabled` (optional) +- `builds_enabled` (optional) - `wiki_enabled` (optional) - `snippets_enabled` (optional) - `public` (optional) - if `true` same as setting visibility_level = 20 @@ -334,6 +401,7 @@ Parameters: - `default_branch` (optional) - 'master' by default - `issues_enabled` (optional) - `merge_requests_enabled` (optional) +- `builds_enabled` (optional) - `wiki_enabled` (optional) - `snippets_enabled` (optional) - `public` (optional) - if `true` same as setting visibility_level = 20 @@ -357,6 +425,7 @@ Parameters: - `default_branch` (optional) - `issues_enabled` (optional) - `merge_requests_enabled` (optional) +- `builds_enabled` (optional) - `wiki_enabled` (optional) - `snippets_enabled` (optional) - `public` (optional) - if `true` same as setting visibility_level = 20 diff --git a/doc/api/repositories.md b/doc/api/repositories.md index 33167453802..b6cca5d4e2a 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -1,79 +1,5 @@ # Repositories -## List project repository tags - -Get a list of repository tags from a project, sorted by name in reverse alphabetical order. - -``` -GET /projects/:id/repository/tags -``` - -Parameters: - -- `id` (required) - The ID of a project - -```json -[ - { - "commit": { - "author_name": "John Smith", - "author_email": "john@example.com", - "authored_date": "2012-05-28T04:42:42-07:00", - "committed_date": "2012-05-28T04:42:42-07:00", - "committer_name": "Jack Smith", - "committer_email": "jack@example.com", - "id": "2695effb5807a22ff3d138d593fd856244e155e7", - "message": "Initial commit", - "parents_ids": [ - "2a4b78934375d7f53875269ffd4f45fd83a84ebe" - ] - }, - "name": "v1.0.0", - "message": null - } -] -``` - -## Create a new tag - -Creates new tag in the repository that points to the supplied ref. - -``` -POST /projects/:id/repository/tags -``` - -Parameters: - -- `id` (required) - The ID of a project -- `tag_name` (required) - The name of a tag -- `ref` (required) - Create tag using commit SHA, another tag name, or branch name. -- `message` (optional) - Creates annotated tag. - -```json -{ - "commit": { - "author_name": "John Smith", - "author_email": "john@example.com", - "authored_date": "2012-05-28T04:42:42-07:00", - "committed_date": "2012-05-28T04:42:42-07:00", - "committer_name": "Jack Smith", - "committer_email": "jack@example.com", - "id": "2695effb5807a22ff3d138d593fd856244e155e7", - "message": "Initial commit", - "parents_ids": [ - "2a4b78934375d7f53875269ffd4f45fd83a84ebe" - ] - }, - "name": "v1.0.0", - "message": null -} -``` -The message will be `nil` when creating a lightweight tag otherwise -it will contain the annotation. - -It returns 200 if the operation succeed. In case of an error, -405 with an explaining error message is returned. - ## List repository tree Get a list of repository files and directories in a project. diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index 25311b07107..623063f357b 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -23,7 +23,8 @@ Example response: "content": "IyA9PSBTY2hlbWEgSW5mb3...", "ref": "master", "blob_id": "79f7bbd25901e8334750839545a9bd021f0e4c83", - "commit_id": "d5a3ff139356ce33e37e73add446f16869741b50" + "commit_id": "d5a3ff139356ce33e37e73add446f16869741b50", + "last_commit_id": "570e7b2abdd848b95f2f578043fc23bd6f6fd24d" } ``` diff --git a/doc/api/tags.md b/doc/api/tags.md new file mode 100644 index 00000000000..085d387e824 --- /dev/null +++ b/doc/api/tags.md @@ -0,0 +1,131 @@ +# Tags + +## List project repository tags + +Get a list of repository tags from a project, sorted by name in reverse alphabetical order. + +``` +GET /projects/:id/repository/tags +``` + +Parameters: + +- `id` (required) - The ID of a project + +```json +[ + { + "commit": { + "author_name": "John Smith", + "author_email": "john@example.com", + "authored_date": "2012-05-28T04:42:42-07:00", + "committed_date": "2012-05-28T04:42:42-07:00", + "committer_name": "Jack Smith", + "committer_email": "jack@example.com", + "id": "2695effb5807a22ff3d138d593fd856244e155e7", + "message": "Initial commit", + "parents_ids": [ + "2a4b78934375d7f53875269ffd4f45fd83a84ebe" + ] + }, + "release": { + "tag_name": "1.0.0", + "description": "Amazing release. Wow" + }, + "name": "v1.0.0", + "message": null + } +] +``` + +## Create a new tag + +Creates a new tag in the repository that points to the supplied ref. + +``` +POST /projects/:id/repository/tags +``` + +Parameters: + +- `id` (required) - The ID of a project +- `tag_name` (required) - The name of a tag +- `ref` (required) - Create tag using commit SHA, another tag name, or branch name. +- `message` (optional) - Creates annotated tag. +- `release_description` (optional) - Add release notes to the git tag and store it in the GitLab database. + +```json +{ + "commit": { + "author_name": "John Smith", + "author_email": "john@example.com", + "authored_date": "2012-05-28T04:42:42-07:00", + "committed_date": "2012-05-28T04:42:42-07:00", + "committer_name": "Jack Smith", + "committer_email": "jack@example.com", + "id": "2695effb5807a22ff3d138d593fd856244e155e7", + "message": "Initial commit", + "parents_ids": [ + "2a4b78934375d7f53875269ffd4f45fd83a84ebe" + ] + }, + "release": { + "tag_name": "1.0.0", + "description": "Amazing release. Wow" + }, + "name": "v1.0.0", + "message": null +} +``` +The message will be `nil` when creating a lightweight tag otherwise +it will contain the annotation. + +It returns 200 if the operation succeed. In case of an error, +405 with an explaining error message is returned. + + +## Create a new release + +Add release notes to the existing git tag. It returns 201 if the release is +created successfully. If the tag does not exist, 404 is returned. If there +already exists a release for the given tag, 409 is returned. + +``` +POST /projects/:id/repository/tags/:tag_name/release +``` + +Parameters: + +- `id` (required) - The ID of a project +- `tag_name` (required) - The name of a tag +- `description` (required) - Release notes with markdown support + +```json +{ + "tag_name": "1.0.0", + "description": "Amazing release. Wow" +} +``` + +## Update a release + +Updates the release notes of a given release. It returns 200 if the release is +successfully updated. If the tag or the release does not exist, it returns 404 +with a proper error message. + +``` +PUT /projects/:id/repository/tags/:tag_name/release +``` + +Parameters: + +- `id` (required) - The ID of a project +- `tag_name` (required) - The name of a tag +- `description` (required) - Release notes with markdown support + +```json +{ + "tag_name": "1.0.0", + "description": "Amazing release. Wow" +} +```
\ No newline at end of file diff --git a/doc/ci/api/README.md b/doc/ci/api/README.md index 33c5b172e98..cf9710ede57 100644 --- a/doc/ci/api/README.md +++ b/doc/ci/api/README.md @@ -25,7 +25,7 @@ GitLab CI API has 4 authentication methods: Authentication is done by sending the `private-token` of a valid user and the `url` of an -authorized Gitlab instance via a query string along with the API +authorized GitLab instance via a query string along with the API request: GET http://gitlab.example.com/ci/api/v1/projects?private_token=QVy1PB7sTxfy4pqfZM1U&url=http://demo.gitlab.com/ diff --git a/doc/ci/api/projects.md b/doc/ci/api/projects.md index 54584db0938..74a4c64d000 100644 --- a/doc/ci/api/projects.md +++ b/doc/ci/api/projects.md @@ -1,7 +1,7 @@ # Projects API This API is intended to aid in the setup and configuration of -projects on Gitlab CI. +projects on GitLab CI. __Authentication is done by GitLab user token & GitLab url__ @@ -88,25 +88,23 @@ authorized. Parameters: - * `id` (required) - The ID of the Gitlab CI project + * `id` (required) - The ID of the GitLab CI project ### Create Project -Creates a Gitlab CI project using Gitlab project details. +Creates a GitLab CI project using GitLab project details. POST /ci/projects Parameters: * `name` (required) - The name of the project - * `gitlab_id` (required) - The ID of the project on the Gitlab instance - * `path` (required) - The gitlab project path - * `ssh_url_to_repo` (required) - The gitlab SSH url to the repo + * `gitlab_id` (required) - The ID of the project on the GitLab instance * `default_ref` (optional) - The branch to run on (default to `master`) ### Update Project -Updates a Gitlab CI project using Gitlab project details that the +Updates a GitLab CI project using GitLab project details that the authenticated user has access to. PUT /ci/projects/:id @@ -114,20 +112,17 @@ authenticated user has access to. Parameters: * `name` - The name of the project - * `gitlab_id` - The ID of the project on the Gitlab instance - * `path` - The gitlab project path - * `ssh_url_to_repo` - The gitlab SSH url to the repo * `default_ref` - The branch to run on (default to `master`) ### Remove Project -Removes a Gitlab CI project that the authenticated user has access to. +Removes a GitLab CI project that the authenticated user has access to. DELETE /ci/projects/:id Parameters: - * `id` (required) - The ID of the Gitlab CI project + * `id` (required) - The ID of the GitLab CI project ### Link Project to Runner @@ -138,8 +133,8 @@ authorized user). Parameters: - * `id` (required) - The ID of the Gitlab CI project - * `runner_id` (required) - The ID of the Gitlab CI runner + * `id` (required) - The ID of the GitLab CI project + * `runner_id` (required) - The ID of the GitLab CI runner ### Remove Project from Runner @@ -150,5 +145,5 @@ via authorized user). Parameters: - * `id` (required) - The ID of the Gitlab CI project - * `runner_id` (required) - The ID of the Gitlab CI runner
\ No newline at end of file + * `id` (required) - The ID of the GitLab CI project + * `runner_id` (required) - The ID of the GitLab CI runner
\ No newline at end of file diff --git a/doc/ci/api/runners.md b/doc/ci/api/runners.md index e9f88ee066e..c383dc4bcc9 100644 --- a/doc/ci/api/runners.md +++ b/doc/ci/api/runners.md @@ -6,7 +6,7 @@ __Authentication is done by GitLab user token & GitLab url__ -Used to get information about all runners registered on the Gitlab CI +Used to get information about all runners registered on the GitLab CI instance. GET /ci/runners @@ -31,7 +31,7 @@ Returns: __Authentication is done with a Shared runner registration token or a project Specific runner registration token__ -Used to make Gitlab CI aware of available runners. +Used to make GitLab CI aware of available runners. POST /ci/runners/register diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index a698fbc8184..4b1788a9af0 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -1,6 +1,6 @@ # Using Docker Build -GitLab CI can allows you to use Docker Engine to build and test docker-based projects. +GitLab CI allows you to use Docker Engine to build and test docker-based projects. **This also allows to you to use `docker-compose` and other docker-enabled tools.** @@ -35,7 +35,7 @@ GitLab Runner then executes build scripts as `gitlab-runner` user. ```bash $ sudo gitlab-runner register -n \ - --url http://gitlab.com/ci \ + --url https://gitlab.com/ci \ --token RUNNER_TOKEN \ --executor shell --description "My Runner" @@ -84,7 +84,7 @@ In order to do that follow the steps: ```bash $ sudo gitlab-runner register -n \ - --url http://gitlab.com/ci \ + --url https://gitlab.com/ci \ --token RUNNER_TOKEN \ --executor docker \ --description "My Docker Runner" \ @@ -108,5 +108,4 @@ In order to do that follow the steps: ``` 1. However, by enabling `--docker-privileged` you are effectively disables all security mechanisms of containers and exposing your host to privilege escalation which can lead to container breakout. -For more information you could be interested in checking out [Runtime privilege](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration). - +For more information, check out [Runtime privilege](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration).
\ No newline at end of file diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 191e3a8144d..1feae62b1c7 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -60,11 +60,11 @@ This is image that have fully preconfigured `wordpress` and have `MySQL` server ``` Next time when you run your application the `tutum/wordpress` will be started -and you will have access to it from your build container under hostname: `tutum_wordpress`. +and you will have access to it from your build container under hostname: `tutum__wordpress`. Alias hostname for the service is made from the image name: 1. Everything after `:` is stripped, -2. '/' is replaced to `_`. +2. '/' is replaced with `__`. ### Configuring services Many services accept environment variables, which allow you to easily change database names or set account names depending on the environment. @@ -90,7 +90,7 @@ you need to set MYSQL_ALLOW_EMPTY_PASSWORD. - mysql variables: - MYSQL_ALLOW_EMPTY_PASSWORD: yes + MYSQL_ALLOW_EMPTY_PASSWORD: "yes" ``` For other possible configuration variables check the @@ -190,7 +190,7 @@ This will create two service containers (MySQL and PostgreSQL). 1. Create a build container and execute script in its context: ``` -$ cat build_script | docker run -n build -i -l mysql:service-mysql -l postgres:service-postgres ruby:2.1 /bin/bash +$ docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` This will create build container that has two service containers linked. The build_script is piped using STDIN to bash interpreter which executes the build script in container. diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index e0b9fa0e25d..1cf41aea391 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -1,5 +1,5 @@ # Build script examples -+ [Test and deploy Ruby Application to Heroku](test-and-deploy-ruby-application-to-heroku.md) -+ [Test and deploy Python Application to Heroku](test-and-deploy-python-application-to-heroku.md) -+ [Test Clojure applications](examples/test-clojure-application.md) ++ [Test and deploy a Ruby application to Heroku](test-and-deploy-ruby-application-to-heroku.md) ++ [Test and deploy a Python application to Heroku](test-and-deploy-python-application-to-heroku.md) ++ [Test a Clojure application](test-clojure-application.md) diff --git a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md index 036b03dd6b9..a236da53fe9 100644 --- a/doc/ci/examples/test-and-deploy-python-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-python-application-to-heroku.md @@ -1,7 +1,7 @@ ## Test and Deploy a python application This example will guide you how to run tests in your Python application and deploy it automatically as Heroku application. -You can checkout the example [source](https://gitlab.com/ayufan/python-getting-started) and check [CI status](https://ci.gitlab.com/projects/4080). +You can checkout the example [source](https://gitlab.com/ayufan/python-getting-started) and check [CI status](https://gitlab.com/ayufan/python-getting-started/builds?scope=all). ### Configure project This is what the `.gitlab-ci.yml` file looks like for this project: diff --git a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md index d2a872f1934..e52e1547461 100644 --- a/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md +++ b/doc/ci/examples/test-and-deploy-ruby-application-to-heroku.md @@ -1,7 +1,7 @@ ## Test and Deploy a ruby application This example will guide you how to run tests in your Ruby application and deploy it automatiacally as Heroku application. -You can checkout the example [source](https://gitlab.com/ayufan/ruby-getting-started) and check [CI status](https://ci.gitlab.com/projects/4050). +You can checkout the example [source](https://gitlab.com/ayufan/ruby-getting-started) and check [CI status](https://gitlab.com/ayufan/ruby-getting-started/builds?scope=all). ### Configure project This is what the `.gitlab-ci.yml` file looks like for this project: @@ -64,4 +64,4 @@ gitlab-ci-multi-runner register \ With the command above, you create a runner that uses [ruby:2.1](https://registry.hub.docker.com/u/library/ruby/) image and uses [postgres](https://registry.hub.docker.com/u/library/postgres/) database. -To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password.
\ No newline at end of file +To access PostgreSQL database you need to connect to `host: postgres` as user `postgres` without password. diff --git a/doc/ci/examples/test-clojure-application.md b/doc/ci/examples/test-clojure-application.md index eaee94a10f1..56b746ce025 100644 --- a/doc/ci/examples/test-clojure-application.md +++ b/doc/ci/examples/test-clojure-application.md @@ -1,8 +1,8 @@ -## Test Clojure applications +## Test a Clojure application This example will guide you how to run tests in your Clojure application. -You can checkout the example [source](https://gitlab.com/dzaporozhets/clojure-web-application) and check [CI status](https://ci.gitlab.com/projects/6306). +You can checkout the example [source](https://gitlab.com/dzaporozhets/clojure-web-application) and check [CI status](https://gitlab.com/dzaporozhets/clojure-web-application/builds?scope=all). ### Configure project diff --git a/doc/ci/img/builds_tab.png b/doc/ci/img/builds_tab.png Binary files differnew file mode 100644 index 00000000000..d088b8b329d --- /dev/null +++ b/doc/ci/img/builds_tab.png diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index a87a1f806fc..a9b36139de9 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,44 +1,62 @@ # Quick Start -To start building projects with GitLab CI a few steps needs to be done. +Starting from version 8.0, GitLab Continuous Integration (CI) is fully +integrated into GitLab itself and is enabled by default on all projects. -## 1. Install GitLab and CI +This guide assumes that you: -First you need to have a working GitLab and GitLab CI instance. +- have a working GitLab instance of version 8.0 or higher or are using + [GitLab.com](https://gitlab.com/users/sign_in) +- have a project in GitLab that you would like to use CI for -You can omit this step if you use [GitLab.com](http://GitLab.com/). +In brief, the steps needed to have a working CI can be summed up to: -## 2. Create repository on GitLab +1. Create a new project +1. Add `.gitlab-ci.yml` to the git repository and push to GitLab +1. Configure a Runner -Once you login on your GitLab add a new repository where you will store your source code. -Push your application to that repository. +From there on, on every push to your git repository the build will be +automagically started by the Runner and will appear under the project's +`/builds` page. -## 3. Add project to CI +Now, let's break it down to pieces and work on solving the GitLab CI puzzle. -The next part is to login to GitLab CI. -Point your browser to the URL you have set GitLab or use [gitlab.com/ci](http://gitlab.com/ci/). +## Creating a `.gitlab-ci.yml` file -On the first screen you will see a list of GitLab's projects that you have access to: +Before you create `.gitlab-ci.yml` let's first explain in brief what this is +all about. - +### What is `.gitlab-ci.yml` -Click **Add Project to CI**. -This will create project in CI and authorize GitLab CI to fetch sources from GitLab. +The `.gitlab-ci.yml` file is where you configure what CI does with your project. +It lives in the root of your repository. -> GitLab CI creates unique token that is used to configure GitLab CI service in GitLab. -> This token allows to access GitLab's repository and configures GitLab to trigger GitLab CI webhook on **Push events** and **Tag push events**. -> You can see that token by going to Project's Settings > Services > GitLab CI. -> You will see there token, the same token is assigned in GitLab CI settings of project. +On any push to your repository, GitLab will look for the `.gitlab-ci.yml` +file and start builds on _Runners_ according to the contents of the file, +for that commit. -## 4. Create project's configuration - .gitlab-ci.yml +Because `.gitlab-ci.yml` is in the repository, it is version controlled, +old versions still build succesfully, forks can easily make use of CI, +branches can have separate builds and you have a single source of truth for CI. +You can read more about the reasons why we are using `.gitlab-ci.yml` +[in our blog about it][blog-ci]. -The next: You have to define how your project will be built. -GitLab CI uses [YAML](https://en.wikipedia.org/wiki/YAML) file to store build configuration. -You need to create `.gitlab-ci.yml` in root directory of your repository: +**Note:** `.gitlab-ci.yml` is a [YAML](https://en.wikipedia.org/wiki/YAML) file +so you have to pay extra attention to the identation. Always use spaces, not +tabs. + +### Creating a simple `.gitlab-ci.yml` file + +You need to create a file named `.gitlab-ci.yml` in the root directory of your +repository. Below is an example for a Ruby on Rails project. ```yaml before_script: - - bundle install + - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs + - ruby -v + - which ruby + - gem install bundler --no-ri --no-rdoc + - bundle install --jobs $(nproc) "${FLAGS[@]}" rspec: script: @@ -49,71 +67,131 @@ rubocop: - bundle exec rubocop ``` -This is the simplest possible build configuration that will work for most Ruby applications: -1. Define two jobs `rspec` and `rubocop` with two different commands to be executed. -1. Before every job execute commands defined by `before_script`. +This is the simplest possible build configuration that will work for most Ruby +applications: + +1. Define two jobs `rspec` and `rubocop` (the names are arbitrary) with + different commands to be executed. +1. Before every job, the commands defined by `before_script` are executed. -The `.gitlab-ci.yml` defines set of jobs with constrains how and when they should be run. -The jobs are defined as top-level elements with name and always have to contain the `script`. -Jobs are used to create builds, which are then picked by [runners](../runners/README.md) and executed within environment of the runner. -What is important that each job is run independently from each other. +The `.gitlab-ci.yml` file defines sets of jobs with constraints of how and when +they should be run. The jobs are defined as top-level elements with a name (in +our case `rspec` and `rubocop`) and always have to contain the `script` keyword. +Jobs are used to create builds, which are then picked by +[Runners](../runners/README.md) and executed within the environment of the Runner. -For more information and complete `.gitlab-ci.yml` syntax, please check the [Configuring project (.gitlab-ci.yml)](../yaml/README.md). +What is important is that each job is run independently from each other. -## 5. Add file and push .gitlab-ci.yml to repository +If you want to check whether your `.gitlab-ci.yml` file is valid, there is a +Lint tool under the page `/ci/lint` of your GitLab instance. You can also find +the link under **Settings > CI settings** in your project. -Once you created `.gitlab-ci.yml` you should add it to git repository and push it to GitLab. +For more information and a complete `.gitlab-ci.yml` syntax, please check +[the documentation on .gitlab-ci.yml](../yaml/README.md). + +### Push `.gitlab-ci.yml` to GitLab + +Once you've created `.gitlab-ci.yml`, you should add it to your git repository +and push it to GitLab. ```bash git add .gitlab-ci.yml -git commit +git commit -m "Add .gitlab-ci.yml" git push origin master ``` -If you refresh the project's page on GitLab CI you will notice a one new commit: +Now if you go to the **Builds** page you will see that the builds are pending. + +You can also go to the **Commits** page and notice the little clock icon next +to the commit SHA. + + + +Clicking on the clock icon you will be directed to the builds page for that +specific commit. + + + +Notice that there are two jobs pending which are named after what we wrote in +`.gitlab-ci.yml`. The red triangle indicates that there is no Runner configured +yet for these builds. + +The next step is to configure a Runner so that it picks the pending jobs. + +## Configuring a Runner + +In GitLab, Runners run the builds that you define in `.gitlab-ci.yml`. +A Runner can be a virtual machine, a VPS, a bare-metal machine, a docker +container or even a cluster of containers. GitLab and the Runners communicate +through an API, so the only needed requirement is that the machine on which the +Runner is configured to has Internet access. + +A Runner can be specific to a certain project or serve multiple projects in +GitLab. If it serves all projects it's called a _Shared Runner_. + +Find more information about different Runners in the +[Runners](../runners/README.md) documentation. + +You can find whether any Runners are assigned to your project by going to +**Settings > Runners**. Setting up a Runner is easy and straightforward. The +official Runner supported by GitLab is written in Go and can be found at +<https://gitlab.com/gitlab-org/gitlab-ci-multi-runner>. + +In order to have a functional Runner you need to follow two steps: + +1. [Install it][runner-install] +2. [Configure it](../runners/README.md#registering-a-specific-runner) + +Follow the links above to set up your own Runner or use a Shared Runner as +described in the next section. + +For other types of unofficial Runners written in other languages, see the +[instructions for the various GitLab Runners](https://about.gitlab.com/gitlab-ci/#gitlab-runner). + +Once the Runner has been set up, you should see it on the Runners page of your +project, following **Settings > Runners**. - + -However the commit has status **pending** which means that commit was not yet picked by runner. +### Shared Runners -## 6. Configure runner +If you use [GitLab.com](https://gitlab.com/) you can use **Shared Runners** +provided by GitLab Inc. -In GitLab CI, Runners run your builds. -A runner is a machine (can be virtual, bare-metal or VPS) that picks up builds through the coordinator API of GitLab CI. +These are special virtual machines that run on GitLab's infrastructure and can +build any project. -A runner can be specific to a certain project or serve any project in GitLab CI. -A runner that serves all projects is called a shared runner. -More information about different runner types can be found in [Configuring runner](../runners/README.md). +To enable **Shared Runners** you have to go to your project's +**Settings > Runners** and click **Enable shared runners**. -To check if you have runners assigned to your project go to **Runners**. You will find there information how to setup project specific runner: +[Read more on Shared Runners](../runners/README.md). -1. Install GitLab Runner software. Checkout the [GitLab Runner](https://about.gitlab.com/gitlab-ci/#gitlab-runner) section to install it. -1. Specify following URL during runner setup: https://gitlab.com/ci/ -1. Use the following registration token during setup: TOKEN +## Seeing the status of your build -If you do it correctly your runner should be shown under **Runners activated for this project**: +After configuring the Runner succesfully, you should see the status of your +last commit change from _pending_ to either _running_, _success_ or _failed_. - +You can view all builds, by going to the **Builds** page in your project. -### Shared runners + -If you use [gitlab.com/ci](http://gitlab.com/ci/) you can use **Shared runners** provided by GitLab Inc. -These are special virtual machines that are run on GitLab's infrastructure that can build any project. -To enable **Shared runners** you have to go to **Runners** and click **Enable shared runners** for this project. +By clicking on a Build ID, you will be able to see the log of that build. +This is important to diagnose why a build failed or acted differently than +you expected. -## 7. Check status of commit + -If everything went OK and you go to commit, the status of the commit should change from **pending** to either **running**, **success** or **failed**. +You are also able to view the status of any commit in the various pages in +GitLab, such as **Commits** and **Merge Requests**. - +## Next steps -You can click **Build ID** to view build log for specific job. +Awesome! You started using CI in GitLab! -## 8. Congratulations! +Next you can look into doing more with the CI. Many people are using GitLab +to package, containerize, test and deploy software. -You managed to build your first project using GitLab CI. -You may need to tune your `.gitlab-ci.yml` file to implement build plan for your project. -A few examples how it can be done you can find on [Examples](../examples/README.md) page. +Visit our various languages examples at <https://gitlab.com/groups/gitlab-examples>. -GitLab CI also offers **the Lint** tool to verify validity of your `.gitlab-ci.yml` which can be useful to troubleshoot potential problems. -The Lint is available from project's settings or by adding `/lint` to GitLab CI url. +[runner-install]: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/tree/master#installation +[blog-ci]: https://about.gitlab.com/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/ diff --git a/doc/ci/quick_start/build_status.png b/doc/ci/quick_start/build_status.png Binary files differdeleted file mode 100644 index 333259e6acd..00000000000 --- a/doc/ci/quick_start/build_status.png +++ /dev/null diff --git a/doc/ci/quick_start/commit_status.png b/doc/ci/quick_start/commit_status.png Binary files differdeleted file mode 100644 index 725b79e6f91..00000000000 --- a/doc/ci/quick_start/commit_status.png +++ /dev/null diff --git a/doc/ci/quick_start/img/build_log.png b/doc/ci/quick_start/img/build_log.png Binary files differnew file mode 100644 index 00000000000..89e6cd40cb6 --- /dev/null +++ b/doc/ci/quick_start/img/build_log.png diff --git a/doc/ci/quick_start/img/builds_status.png b/doc/ci/quick_start/img/builds_status.png Binary files differnew file mode 100644 index 00000000000..b8e6c2a361a --- /dev/null +++ b/doc/ci/quick_start/img/builds_status.png diff --git a/doc/ci/quick_start/img/new_commit.png b/doc/ci/quick_start/img/new_commit.png Binary files differnew file mode 100644 index 00000000000..3d3c9d5c0bd --- /dev/null +++ b/doc/ci/quick_start/img/new_commit.png diff --git a/doc/ci/quick_start/img/runners_activated.png b/doc/ci/quick_start/img/runners_activated.png Binary files differnew file mode 100644 index 00000000000..eafcfd6ecd5 --- /dev/null +++ b/doc/ci/quick_start/img/runners_activated.png diff --git a/doc/ci/quick_start/img/single_commit_status_pending.png b/doc/ci/quick_start/img/single_commit_status_pending.png Binary files differnew file mode 100644 index 00000000000..23b3bb5acfc --- /dev/null +++ b/doc/ci/quick_start/img/single_commit_status_pending.png diff --git a/doc/ci/quick_start/img/status_pending.png b/doc/ci/quick_start/img/status_pending.png Binary files differnew file mode 100644 index 00000000000..a049ec2a5ba --- /dev/null +++ b/doc/ci/quick_start/img/status_pending.png diff --git a/doc/ci/quick_start/new_commit.png b/doc/ci/quick_start/new_commit.png Binary files differdeleted file mode 100644 index 3839e893c17..00000000000 --- a/doc/ci/quick_start/new_commit.png +++ /dev/null diff --git a/doc/ci/quick_start/projects.png b/doc/ci/quick_start/projects.png Binary files differdeleted file mode 100644 index 0b3430a69db..00000000000 --- a/doc/ci/quick_start/projects.png +++ /dev/null diff --git a/doc/ci/quick_start/runners.png b/doc/ci/quick_start/runners.png Binary files differdeleted file mode 100644 index 25b4046bc00..00000000000 --- a/doc/ci/quick_start/runners.png +++ /dev/null diff --git a/doc/ci/quick_start/runners_activated.png b/doc/ci/quick_start/runners_activated.png Binary files differdeleted file mode 100644 index c934bd12f41..00000000000 --- a/doc/ci/quick_start/runners_activated.png +++ /dev/null diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 04c6bf1e3a3..022afb70042 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -15,21 +15,27 @@ The API_TOKEN will take the Secure Variable value: `SECURE`. ### Predefined variables (Environment Variables) -| Variable | Description | +| Variable | Runner | Description | |-------------------------|-------------| -| **CI** | Mark that build is executed in CI environment | -| **GITLAB_CI** | Mark that build is executed in GitLab CI environment | -| **CI_SERVER** | Mark that build is executed in CI environment | -| **CI_SERVER_NAME** | CI server that is used to coordinate builds | -| **CI_SERVER_VERSION** | Not yet defined | -| **CI_SERVER_REVISION** | Not yet defined | -| **CI_BUILD_REF** | The commit revision for which project is built | -| **CI_BUILD_BEFORE_SHA** | The first commit that were included in push request | -| **CI_BUILD_REF_NAME** | The branch or tag name for which project is built | -| **CI_BUILD_ID** | The unique id of the current build that GitLab CI uses internally | -| **CI_BUILD_REPO** | The URL to clone the Git repository | -| **CI_PROJECT_ID** | The unique id of the current project that GitLab CI uses internally | -| **CI_PROJECT_DIR** | The full path where the repository is cloned and where the build is ran | +| **CI** | 0.4 | Mark that build is executed in CI environment | +| **GITLAB_CI** | all | Mark that build is executed in GitLab CI environment | +| **CI_SERVER** | all | Mark that build is executed in CI environment | +| **CI_SERVER_NAME** | all | CI server that is used to coordinate builds | +| **CI_SERVER_VERSION** | all | Not yet defined | +| **CI_SERVER_REVISION** | all | Not yet defined | +| **CI_BUILD_REF** | all | The commit revision for which project is built | +| **CI_BUILD_TAG** | 0.5 | The commit tag name. Present only when building tags. | +| **CI_BUILD_NAME** | 0.5 | The name of the build as defined in `.gitlab-ci.yml` | +| **CI_BUILD_STAGE** | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | +| **CI_BUILD_BEFORE_SHA** | all | The first commit that were included in push request | +| **CI_BUILD_REF_NAME** | all | The branch or tag name for which project is built | +| **CI_BUILD_ID** | all | The unique id of the current build that GitLab CI uses internally | +| **CI_BUILD_REPO** | all | The URL to clone the Git repository | +| **CI_BUILD_TRIGGERED** | 0.5 | The flag to indicate that build was triggered | +| **CI_PROJECT_ID** | all | The unique id of the current project that GitLab CI uses internally | +| **CI_PROJECT_DIR** | all | The full path where the repository is cloned and where the build is ran | + +**Some of the variables are only available when using runner with at least defined version.** Example values: @@ -39,6 +45,10 @@ export CI_BUILD_ID="50" export CI_BUILD_REF="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_BUILD_REF_NAME="master" export CI_BUILD_REPO="https://gitlab.com/gitlab-org/gitlab-ce.git" +export CI_BUILD_TAG="1.0.0" +export CI_BUILD_NAME="spec:other" +export CI_BUILD_STAGE="test" +export CI_BUILD_TRIGGERED="true" export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" export CI_PROJECT_ID="34" export CI_SERVER="yes" diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 4caeccacb7f..3dbf1afc7a9 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -56,6 +56,7 @@ There are a few `keywords` that can't be used as job names: | types | optional | Alias for `stages` | | before_script | optional | Define commands prepended for each job's script | | variables | optional | Define build variables | +| cache | optional | Define list of files that should be cached between subsequent runs | ### image and services This allows to specify a custom Docker image and a list of services that can be used for time of the build. @@ -110,6 +111,19 @@ These variables can be later used in all executed commands and scripts. The YAML-defined variables are also set to all created service containers, thus allowing to fine tune them. +### cache +`cache` is used to specify list of files and directories which should be cached between builds. + +**The global setting allows to specify default cached files for all jobs.** + +To cache all git untracked files and files in `binaries`: +``` +cache: + untracked: true + paths: + - binaries/ +``` + ## Jobs `.gitlab-ci.yml` allows you to specify an unlimited number of jobs. Each job has to have a unique `job_name`, which is not one of the keywords mentioned above. @@ -140,6 +154,9 @@ job_name: | except | optional | Defines a list of git refs for which build is not created | | tags | optional | Defines a list of tags which are used to select runner | | allow_failure | optional | Allow build to fail. Failed build doesn't contribute to commit status | +| when | optional | Define when to run build. Can be `on_success`, `on_failure` or `always` | +| artifacts | optional | Define list build artifacts | +| cache | optional | Define list of files that should be cached between subsequent runs | ### script `script` is a shell script which is executed by runner. The shell script is prepended with `before_script`. @@ -168,7 +185,7 @@ This are two parameters that allow for setting a refs policy to limit when jobs There are a few rules that apply to usage of refs policy: -1. `only` and `except` are exclusive. If both `only` and `except` are defined in job specification only `only` is taken into account. +1. `only` and `except` are inclusive. If both `only` and `except` are defined in job specification the ref is filtered by `only` and `except`. 1. `only` and `except` allow for using the regexp expressions. 1. `only` and `except` allow for using special keywords: `branches` and `tags`. These names can be used for example to exclude all tags and all branches. @@ -181,6 +198,18 @@ job: - branches # use special keyword ``` +1. `only` and `except` allow for specify repository path to filter jobs for forks. +The repository path can be used to have jobs executed only for parent repository. + +```yaml +job: + only: + - branches@gitlab-org/gitlab-ce + except: + - master@gitlab-org/gitlab-ce +``` +The above will run `job` for all branches on `gitlab-org/gitlab-ce`, except master . + ### tags `tags` is used to select specific runners from the list of all runners that are allowed to run this project. @@ -196,9 +225,130 @@ job: The above specification will make sure that `job` is built by a runner that have `ruby` AND `postgres` tags defined. +### when +`when` is used to implement jobs that are run in case of failure or despite the failure. + +`when` can be set to one of the following values: + +1. `on_success` - execute build only when all builds from prior stages succeeded. This is the default. +1. `on_failure` - execute build only when at least one build from prior stages failed. +1. `always` - execute build despite the status of builds from prior stages. + +``` +stages: +- build +- cleanup_build +- test +- deploy +- cleanup + +build: + stage: build + script: + - make build + +cleanup_build: + stage: cleanup_build + script: + - cleanup build when failed + when: on_failure + +test: + stage: test + script: + - make test + +deploy: + stage: deploy + script: + - make deploy + +cleanup: + stage: cleanup + script: + - cleanup after builds + when: always +``` + +The above script will: +1. Execute `cleanup_build` only when the `build` failed, +2. Always execute `cleanup` as the last step in pipeline. + +### artifacts +`artifacts` is used to specify list of files and directories which should be attached to build after success. + +1. Send all files in `binaries` and `.config`: + + artifacts: + paths: + - binaries/ + - .config + +2. Send all git untracked files: + + artifacts: + untracked: true + +3. Send all git untracked files and files in `binaries`: + + artifacts: + untracked: true + paths: + - binaries/ + +The artifacts will be send after the build success to GitLab and will be accessible in GitLab interface to download. + +This feature requires GitLab Runner v0.7.0 or higher. + +### cache +`cache` is used to specify list of files and directories which should be cached between builds. + +1. Cache all files in `binaries` and `.config`: + + rspec: + script: test + cache: + paths: + - binaries/ + - .config + +2. Cache all git untracked files: + + rspec: + script: test + cache: + untracked: true + +3. Cache all git untracked files and files in `binaries`: + + rspec: + script: test + cache: + untracked: true + paths: + - binaries/ + +4. Locally defined cache overwrites globally defined options. This will cache only `binaries/`: + + cache: + paths: + - my/files + + rspec: + script: test + cache: + paths: + - binaries/ + +The cache is provided on best effort basis, so don't expect that cache will be present. +For implementation details please check GitLab Runner. + +This feature requires GitLab Runner v0.7.0 or higher. + + ## Validate the .gitlab-ci.yml Each instance of GitLab CI has an embedded debug tool called Lint. You can find the link to the Lint in the project's settings page or use short url `/lint`. ## Skipping builds -There is one more way to skip all builds, if your commit message contains tag [ci skip]. In this case, commit will be created but builds will be skipped
\ No newline at end of file +There is one more way to skip all builds, if your commit message contains tag [ci skip]. In this case, commit will be created but builds will be skipped diff --git a/doc/customization/libravatar.md b/doc/customization/libravatar.md index 54c1780c3ab..bd2c242afc2 100644 --- a/doc/customization/libravatar.md +++ b/doc/customization/libravatar.md @@ -2,7 +2,7 @@ GitLab by default supports [Gravatar](https://gravatar.com) avatar service. Libravatar is a service which delivers your avatar (profile picture) to other websites and their API is -[heavily based on gravatar](http://wiki.libravatar.org/api/). +[heavily based on gravatar](https://wiki.libravatar.org/api/). This means that it is not complicated to switch to Libravatar avatar service or even self hosted Libravatar server. @@ -31,7 +31,7 @@ the configuration options as follows: ## Self-hosted -If you are [running your own libravatar service](http://wiki.libravatar.org/running_your_own/) the URL will be different in the configuration +If you are [running your own libravatar service](https://wiki.libravatar.org/running_your_own/) the URL will be different in the configuration but the important part is to provide the same placeholders so GitLab can parse the URL correctly. For example, you host a service on `http://libravatar.example.com` the `plain_url` you need to supply in `gitlab.yml` is @@ -63,7 +63,7 @@ Run `sudo gitlab-ctl reconfigure` for changes to take effect. ## Default URL for missing images -[Libravatar supports different sets](http://wiki.libravatar.org/api/) of `missing images` for emails not found on the Libravatar service. +[Libravatar supports different sets](https://wiki.libravatar.org/api/) of `missing images` for emails not found on the Libravatar service. In order to use a different set other than `identicon`, replace `&d=identicon` portion of the URL with another supported set. For example, you can use `retro` set in which case the URL would look like: `plain_url: "http://cdn.libravatar.org/avatar/%{hash}?s=%{size}&d=retro"` diff --git a/doc/customization/welcome_message.md b/doc/customization/welcome_message.md index 6c141d1fb7a..e993230bb88 100644 --- a/doc/customization/welcome_message.md +++ b/doc/customization/welcome_message.md @@ -8,31 +8,5 @@ It is possible to add a markdown-formatted welcome message to your GitLab sign-in page. Users of GitLab Enterprise Edition should use the [branded login page feature](/ee/customization/branded_login_page.html) instead. -## Omnibus-gitlab example - -In `/etc/gitlab/gitlab.rb`: - -```ruby -gitlab_rails['extra_sign_in_text'] = <<'EOS' -# ACME GitLab -Welcome to the [ACME](http://www.example.com) GitLab server! -EOS -``` - -Run `sudo gitlab-ctl reconfigure` for changes to take effect. - -## Installation from source - -In `/home/git/gitlab/config/gitlab.yml`: - -```yaml -# snip -production: - # snip - extra: - sign_in_text: | - # ACME GitLab - Welcome to the [ACME](http://www.example.com) GitLab server! -``` - -Run `sudo service gitlab reload` for the change to take effect. +The welcome message (extra_sign_in_text) can now be set/changed in the Admin UI. +Admin area > Settings
\ No newline at end of file diff --git a/doc/development/README.md b/doc/development/README.md index 6bc8e1888db..d5bf166ad32 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -8,3 +8,4 @@ - [UI guide](ui_guide.md) for building GitLab with existing css styles and elements - [Migration Style Guide](migration_style_guide.md) for creating safe migrations - [How to dump production data to staging](dump_db.md) +- [Benchmarking](benchmarking.md) diff --git a/doc/development/architecture.md b/doc/development/architecture.md index c00d290371e..6101a71a8de 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -146,7 +146,7 @@ nginx Apache httpd -- [Explanation of Apache logs](http://httpd.apache.org/docs/2.2/logs.html). +- [Explanation of Apache logs](https://httpd.apache.org/docs/2.2/logs.html). - `/var/log/apache2/` contains error and output logs (on Ubuntu). - `/var/log/httpd/` contains error and output logs (on RHEL). diff --git a/doc/development/benchmarking.md b/doc/development/benchmarking.md new file mode 100644 index 00000000000..88e18ee95f9 --- /dev/null +++ b/doc/development/benchmarking.md @@ -0,0 +1,69 @@ +# Benchmarking + +GitLab CE comes with a set of benchmarks that are executed for every build. This +makes it easier to measure performance of certain components over time. + +Benchmarks are written as RSpec tests using a few extra helpers. To write a +benchmark, first tag the top-level `describe`: + +```ruby +describe MaruTheCat, benchmark: true do + +end +``` + +This ensures the benchmark is executed separately from other test collections. +It also exposes the various RSpec matchers used for writing benchmarks to the +test group. + +Next, lets write the actual benchmark: + +```ruby +describe MaruTheCat, benchmark: true do + let(:maru) { MaruTheChat.new } + + describe '#jump_in_box' do + benchmark_subject { maru.jump_in_box } + + it { is_expected.to iterate_per_second(9000) } + end +end +``` + +Here `benchmark_subject` is a small wrapper around RSpec's `subject` method that +makes it easier to specify the subject of a benchmark. Using RSpec's regular +`subject` would require us to write the following instead: + +```ruby +subject { -> { maru.jump_in_box } } +``` + +The `iterate_per_second` matcher defines the amount of times per second a +subject should be executed. The higher the amount of iterations the better. + +By default the allowed standard deviation is a maximum of 30%. This can be +adjusted by chaining the `with_maximum_stddev` on the `iterate_per_second` +matcher: + +```ruby +it { is_expected.to iterate_per_second(9000).with_maximum_stddev(50) } +``` + +This can be useful if the code in question depends on external resources of +which the performance can vary a lot (e.g. physical HDDs, network calls, etc). +However, in most cases 30% should be enough so only change this when really +needed. + +## Benchmarks Location + +Benchmarks should be stored in `spec/benchmarks` and should follow the regular +Rails specs structure. That is, model benchmarks go in `spec/benchmark/models`, +benchmarks for code in the `lib` directory go in `spec/benchmarks/lib`, etc. + +## Underlying Technology + +The benchmark setup uses [benchmark-ips][benchmark-ips] which takes care of the +heavy lifting such as warming up code, calculating iterations, standard +deviation, etc. + +[benchmark-ips]: https://github.com/evanphx/benchmark-ips diff --git a/doc/development/profiling.md b/doc/development/profiling.md new file mode 100644 index 00000000000..e244ad4e881 --- /dev/null +++ b/doc/development/profiling.md @@ -0,0 +1,27 @@ +# Profiling + +To make it easier to track down performance problems GitLab comes with a set of +profiling tools, some of these are available by default while others need to be +explicitly enabled. + +## Sherlock + +Sherlock is a custom profiling tool built into GitLab. Sherlock is _only_ +available when running GitLab in development mode _and_ when setting the +environment variable `ENABLE_SHERLOCK` to a non empty value. For example: + + ENABLE_SHERLOCK=1 bundle exec rails s + +Recorded transactions can be found by navigating to `/sherlock/transactions`. + +## Bullet + +Bullet is a Gem that can be used to track down N+1 query problems. Because +Bullet adds quite a bit of logging noise it's disabled by default. To enable +Bullet, set the environment variable `ENABLE_BULLET` to a non-empty value before +starting GitLab. For example: + + ENABLE_BULLET=true bundle exec rails s + +Bullet will log query problems to both the Rails log as well as the Chrome +console. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index a4a980cf0e0..9f3fd69fc4e 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -9,7 +9,7 @@ bundle exec rake setup ``` The `setup` task is a alias for `gitlab:setup`. -This tasks calls `db:setup` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and fianlly it calls `db:seed_fu` to seed the database. +This tasks calls `db:setup` to create the database, calls `add_limits_mysql` that adds limits to the database schema in case of a MySQL database and finally it calls `db:seed_fu` to seed the database. Note: `db:setup` calls `db:seed` but this does nothing. ## Run tests diff --git a/doc/development/shared_files.md b/doc/development/shared_files.md new file mode 100644 index 00000000000..fcd905b54a4 --- /dev/null +++ b/doc/development/shared_files.md @@ -0,0 +1,33 @@ +# Shared files + +Historically, GitLab has been storing shared files in many different +directories: `public/uploads`, `builds`, `tmp/repositories`, `tmp/rebase` (EE), +etc. Having so many shared directories makes it difficult to deploy GitLab on +shared storage (e.g. NFS). Working towards GitLab 9.0 we are consolidating +these different directories under the `shared` directory. + +This means that if GitLab will start storing puppies in some future version +then we should put them in `shared/puppies`. Temporary puppy files should be +stored in `shared/tmp`. + +In the GitLab application code you can get the full path to the `shared` +directory with `Gitlab.config.shared.path`. + +## What is not a 'shared file' + +Files that belong to only one process, or on only one server, should not go in +`shared`. Examples include PID files and sockets. + +## Temporary files and shared storage + +Sometimes you create a temporary file on disk with the intention of it becoming +'official'. For example you might be first streaming an upload from a user to +disk in a temporary file so you can perform some checks on it. When the checks +pass, you make the file official. In scenarios like this please follow these +rules: + +- Store the temporary file under `shared/tmp`, i.e. on the same filesystem you + want the official file to be on. +- Use move/rename operations when operating on the file instead of copy + operations where possible, because renaming a file is much faster than + copying it. diff --git a/doc/development/shell_commands.md b/doc/development/shell_commands.md index 2d1d0fb4154..65cdd74bdb6 100644 --- a/doc/development/shell_commands.md +++ b/doc/development/shell_commands.md @@ -35,6 +35,16 @@ Gitlab::Popen.popen(%W(find /some/path -not -path /some/path -mmin +120 -delete) This coding style could have prevented CVE-2013-4490. +## Always use the configurable git binary path for git commands + +```ruby +# Wrong +system(*%W(git branch -d -- #{branch_name})) + +# Correct +system(*%W(#{Gitlab.config.git.bin_path} branch -d -- #{branch_name})) +``` + ## Bypass the shell by splitting commands into separate tokens When we pass shell commands as a single string to Ruby, Ruby will let `/bin/sh` evaluate the entire string. Essentially, we are asking the shell to evaluate a one-line script. This creates a risk for shell injection attacks. It is better to split the shell command into tokens ourselves. Sometimes we use the scripting capabilities of the shell to change the working directory or set environment variables. All of this can also be achieved securely straight from Ruby @@ -81,9 +91,9 @@ In the GitLab codebase, we avoid the option/argument ambiguity by _always_ using ```ruby # Wrong -system(*%W(git branch -d #{branch_name})) +system(*%W(#{Gitlab.config.git.bin_path} branch -d #{branch_name})) # Correct -system(*%W(git branch -d -- #{branch_name})) +system(*%W(#{Gitlab.config.git.bin_path} branch -d -- #{branch_name})) ``` This coding style could have prevented CVE-2013-4582. @@ -94,9 +104,9 @@ Capturing the output of shell commands with backticks reads nicely, but you are ```ruby # Wrong -logs = `cd #{repo_dir} && git log` +logs = `cd #{repo_dir} && #{Gitlab.config.git.bin_path} log` # Correct -logs, exit_status = Gitlab::Popen.popen(%W(git log), repo_dir) +logs, exit_status = Gitlab::Popen.popen(%W(#{Gitlab.config.git.bin_path} log), repo_dir) # Wrong user = `whoami` @@ -108,7 +118,7 @@ In other repositories, such as gitlab-shell you can also use `IO.popen`. ```ruby # Safe IO.popen example -logs = IO.popen(%W(git log), chdir: repo_dir) { |p| p.read } +logs = IO.popen(%W(#{Gitlab.config.git.bin_path} log), chdir: repo_dir) { |p| p.read } ``` Note that unlike `Gitlab::Popen.popen`, `IO.popen` does not capture standard error. diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index b904c70e980..493e1d1b09c 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -23,3 +23,5 @@ Step-by-step guides on the basics of working with Git and GitLab. * [Add an image](add-image.md) * [Create a Merge Request](add-merge-request.md) + +* [Create an Issue](create-issue.md) diff --git a/doc/gitlab-basics/create-issue.md b/doc/gitlab-basics/create-issue.md new file mode 100644 index 00000000000..87f078def04 --- /dev/null +++ b/doc/gitlab-basics/create-issue.md @@ -0,0 +1,27 @@ +# How to create an Issue in GitLab + +The Issue Tracker is a good place to add things that need to be improved or solved in a project. + +To create an Issue, sign in to GitLab. + +Go to the project where you'd like to create the Issue: + + + +Click on "Issues" on the left side of your screen: + + + +Click on the "+ new issue" button on the right side of your screen: + + + +Add a title and a description to your issue: + + + +You may assign the Issue to a user, add a milestone and add labels (they are all optional). Then click on "submit new issue": + + + +Your Issue will now be added to the Issue Tracker and will be ready to be reviewed. You can comment on it and mention the people involved. You can also link Issues to the Merge Requests where the Issues are solved. To do this, you can use an [Issue closing pattern](http://doc.gitlab.com/ce/customization/issue_closing.html). diff --git a/doc/hooks/custom_hooks.md b/doc/hooks/custom_hooks.md index f7d4f3de68b..0f2665a3bf7 100644 --- a/doc/hooks/custom_hooks.md +++ b/doc/hooks/custom_hooks.md @@ -2,12 +2,12 @@ **Note: Custom git hooks must be configured on the filesystem of the GitLab server. Only GitLab server administrators will be able to complete these tasks. -Please explore webhooks as an option if you do not have filesystem access.** +Please explore webhooks as an option if you do not have filesystem access. For a user configurable Git Hooks interface, please see [GitLab Enterprise Edition Git Hooks](http://doc.gitlab.com/ee/git_hooks/git_hooks.html).** Git natively supports hooks that are executed on different actions. Examples of server-side git hooks include pre-receive, post-receive, and update. See -[Git SCM Server-Side Hooks](http://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks) +[Git SCM Server-Side Hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks) for more information about each hook type. As of gitlab-shell version 2.2.0 (which requires GitLab 7.5+), GitLab diff --git a/doc/incoming_email/README.md b/doc/incoming_email/README.md index 01ab22321ed..86d205ba7a5 100644 --- a/doc/incoming_email/README.md +++ b/doc/incoming_email/README.md @@ -4,32 +4,75 @@ GitLab can be set up to allow users to comment on issues and merge requests by r ## Get a mailbox -Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises. +Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Google Apps, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises. -If you want to use Gmail with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255). +If you want to use Gmail / Google Apps with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255). To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these instructions](./postfix.md). ## Set it up -In this example, we'll use the Gmail address `gitlab-incoming@gmail.com`. - ### Omnibus package installations -1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the feature, enter the email address including a placeholder for the `key` that references the item being replied to and fill in the details for your specific IMAP server and email account: +1. Find the `incoming_email` section in `/etc/gitlab/gitlab.rb`, enable the feature and fill in the details for your specific IMAP server and email account: ```ruby + # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com gitlab_rails['incoming_email_enabled'] = true - gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com" - gitlab_rails['incoming_email_host'] = "imap.gmail.com" # IMAP server host - gitlab_rails['incoming_email_port'] = 993 # IMAP server port - gitlab_rails['incoming_email_ssl'] = true # Whether the IMAP server uses SSL - gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com" # Email account username. Usually the full email address. - gitlab_rails['incoming_email_password'] = "password" # Email account password - gitlab_rails['incoming_email_mailbox_name'] = "inbox" # The name of the mailbox where incoming mail will end up. Usually "inbox". + + # The email address including a placeholder for the key that references the item being replied to. + # The `%{key}` placeholder is added after the user part, before the `@`. + gitlab_rails['incoming_email_address'] = "incoming+%{key}@gitlab.example.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + gitlab_rails['incoming_email_email'] = "incoming" + # Email account password + gitlab_rails['incoming_email_password'] = "[REDACTED]" + + # IMAP server host + gitlab_rails['incoming_email_host'] = "gitlab.example.com" + # IMAP server port + gitlab_rails['incoming_email_port'] = 143 + # Whether the IMAP server uses SSL + gitlab_rails['incoming_email_ssl'] = false + # Whether the IMAP server uses StartTLS + gitlab_rails['incoming_email_start_tls'] = false + + # The mailbox where incoming mail will end up. Usually "inbox". + gitlab_rails['incoming_email_mailbox_name'] = "inbox" ``` - As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-incoming@gmail.com`. + ```ruby + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + gitlab_rails['incoming_email_enabled'] = true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. + gitlab_rails['incoming_email_address'] = "gitlab-incoming+%{key}@gmail.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + gitlab_rails['incoming_email_email'] = "gitlab-incoming@gmail.com" + # Email account password + gitlab_rails['incoming_email_password'] = "[REDACTED]" + + # IMAP server host + gitlab_rails['incoming_email_host'] = "imap.gmail.com" + # IMAP server port + gitlab_rails['incoming_email_port'] = 993 + # Whether the IMAP server uses SSL + gitlab_rails['incoming_email_ssl'] = true + # Whether the IMAP server uses StartTLS + gitlab_rails['incoming_email_start_tls'] = false + + # The mailbox where incoming mail will end up. Usually "inbox". + gitlab_rails['incoming_email_mailbox_name'] = "inbox" + ``` + + As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`. 1. Reconfigure GitLab for the changes to take effect: @@ -53,155 +96,146 @@ In this example, we'll use the Gmail address `gitlab-incoming@gmail.com`. cd /home/git/gitlab ``` -1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `key` that references the item being replied to: +1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account: ```sh sudo editor config/gitlab.yml ``` ```yaml + # Configuration for Postfix mail server, assumes mailbox incoming@gitlab.example.com incoming_email: enabled: true - address: "gitlab-incoming+%{key}@gmail.com" - ``` - - As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-incoming@gmail.com`. -2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`: + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. + address: "incoming+%{key}@gitlab.example.com" - ```sh - sudo cp config/mail_room.yml.example config/mail_room.yml - ``` + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "incoming" + # Email account password + password: "[REDACTED]" -3. Uncomment the configuration options in `config/mail_room.yml` and fill in the details for your specific IMAP server and email account: + # IMAP server host + host: "gitlab.example.com" + # IMAP server port + port: 143 + # Whether the IMAP server uses SSL + ssl: false + # Whether the IMAP server uses StartTLS + start_tls: false - ```sh - sudo editor config/mail_room.yml + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" ``` ```yaml - :mailboxes: - - - # IMAP server host - :host: "imap.gmail.com" - # IMAP server port - :port: 993 - # Whether the IMAP server uses SSL - :ssl: true - # Whether the IMAP server uses StartTLS - :start_tls: false - # Email account username. Usually the full email address. - :email: "gitlab-incoming@gmail.com" - # Email account password - :password: "[REDACTED]" - # The name of the mailbox where incoming mail will end up. Usually "inbox". - :name: "inbox" - # Always "sidekiq". - :delivery_method: sidekiq - # Always true. - :delete_after_delivery: true - :delivery_options: - # The URL to the Redis server used by Sidekiq. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "resque:gitlab". - :namespace: resque:gitlab - # Always "incoming_email". - :queue: incoming_email - # Always "EmailReceiverWorker" - :worker: EmailReceiverWorker + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com + incoming_email: + enabled: true + + # The email address including the `%{key}` placeholder that will be replaced to reference the item being replied to. + # The `%{key}` placeholder is added after the user part, after a `+` character, before the `@`. + address: "gitlab-incoming+%{key}@gmail.com" + + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "gitlab-incoming@gmail.com" + # Email account password + password: "[REDACTED]" + + # IMAP server host + host: "imap.gmail.com" + # IMAP server port + port: 993 + # Whether the IMAP server uses SSL + ssl: true + # Whether the IMAP server uses StartTLS + start_tls: false + + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" ``` -5. Edit the init script configuration at `/etc/default/gitlab` to enable `mail_room`: + As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `incoming@gitlab.example.com`/`gitlab-incoming@gmail.com`. + +1. Enable `mail_room` in the init script at `/etc/default/gitlab`: ```sh sudo mkdir -p /etc/default echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab ``` -6. Restart GitLab: +1. Restart GitLab: ```sh sudo service gitlab restart ``` -7. Verify that everything is configured correctly: +1. Verify that everything is configured correctly: ```sh sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production ``` -8. Reply by email should now be working. +1. Reply by email should now be working. ### Development 1. Go to the GitLab installation directory. -1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `key` that references the item being replied to: +1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account: ```yaml + # Configuration for Gmail / Google Apps, assumes mailbox gitlab-incoming@gmail.com incoming_email: enabled: true + + # The email address including a placeholder for the key that references the item being replied to. + # The `%{key}` placeholder is added after the user part, before the `@`. address: "gitlab-incoming+%{key}@gmail.com" - ``` - As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. + # Email account username + # With third party providers, this is usually the full email address. + # With self-hosted email servers, this is usually the user part of the email address. + user: "gitlab-incoming@gmail.com" + # Email account password + password: "[REDACTED]" -2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`: + # IMAP server host + host: "imap.gmail.com" + # IMAP server port + port: 993 + # Whether the IMAP server uses SSL + ssl: true + # Whether the IMAP server uses StartTLS + start_tls: false - ```sh - sudo cp config/mail_room.yml.example config/mail_room.yml + # The mailbox where incoming mail will end up. Usually "inbox". + mailbox: "inbox" ``` -3. Uncomment the configuration options in `config/mail_room.yml` and fill in the details for your specific IMAP server and email account: - - ```yaml - :mailboxes: - - - # IMAP server host - :host: "imap.gmail.com" - # IMAP server port - :port: 993 - # Whether the IMAP server uses SSL - :ssl: true - # Whether the IMAP server uses StartTLS - :start_tls: false - # Email account username. Usually the full email address. - :email: "gitlab-incoming@gmail.com" - # Email account password - :password: "[REDACTED]" - # The name of the mailbox where incoming mail will end up. Usually "inbox". - :name: "inbox" - # Always "sidekiq". - :delivery_method: sidekiq - # Always true. - :delete_after_delivery: true - :delivery_options: - # The URL to the Redis server used by Sidekiq. Should match the URL in config/resque.yml. - :redis_url: redis://localhost:6379 - # Always "resque:gitlab". - :namespace: resque:gitlab - # Always "incoming_email". - :queue: incoming_email - # Always "EmailReceiverWorker" - :worker: EmailReceiverWorker - ``` + As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. -4. Uncomment the `mail_room` line in your `Procfile`: +1. Uncomment the `mail_room` line in your `Procfile`: ```yaml mail_room: bundle exec mail_room -q -c config/mail_room.yml ``` -6. Restart GitLab: +1. Restart GitLab: ```sh bundle exec foreman start ``` -7. Verify that everything is configured correctly: +1. Verify that everything is configured correctly: ```sh bundle exec rake gitlab:incoming_email:check RAILS_ENV=development ``` -8. Reply by email should now be working. +1. Reply by email should now be working. diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index c565e90da2f..513ad69ec26 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -2,7 +2,7 @@ ## Note -We do not recommend using MySQL due to various issues. For example, case [(in)sensitivity](https://dev.mysql.com/doc/refman/5.0/en/case-sensitivity.html) and [problems](http://bugs.mysql.com/bug.php?id=65830) that [suggested](http://bugs.mysql.com/bug.php?id=50909) [fixes](http://bugs.mysql.com/bug.php?id=65830) [have](http://bugs.mysql.com/bug.php?id=63164). +We do not recommend using MySQL due to various issues. For example, case [(in)sensitivity](https://dev.mysql.com/doc/refman/5.0/en/case-sensitivity.html) and [problems](https://bugs.mysql.com/bug.php?id=65830) that [suggested](https://bugs.mysql.com/bug.php?id=50909) [fixes](https://bugs.mysql.com/bug.php?id=65830) [have](https://bugs.mysql.com/bug.php?id=63164). ## MySQL diff --git a/doc/install/installation.md b/doc/install/installation.md index 039bb3c2561..618391e16d2 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -106,7 +106,7 @@ Then select 'Internet Site' and press enter to confirm the hostname. ## 2. Ruby -The use of Ruby version managers such as [RVM](http://rvm.io/), [rbenv](https://github.com/sstephenson/rbenv) or [chruby](https://github.com/postmodern/chruby) with GitLab in production frequently leads to hard to diagnose problems. For example, GitLab Shell is called from OpenSSH and having a version manager can prevent pushing and pulling over SSH. Version managers are not supported and we strongly advise everyone to follow the instructions below to use a system Ruby. +The use of Ruby version managers such as [RVM](https://rvm.io/), [rbenv](https://github.com/sstephenson/rbenv) or [chruby](https://github.com/postmodern/chruby) with GitLab in production frequently leads to hard to diagnose problems. For example, GitLab Shell is called from OpenSSH and having a version manager can prevent pushing and pulling over SSH. Version managers are not supported and we strongly advise everyone to follow the instructions below to use a system Ruby. Remove the old Ruby 1.8 if present @@ -115,8 +115,9 @@ Remove the old Ruby 1.8 if present Download Ruby and compile it: mkdir /tmp/ruby && cd /tmp/ruby - curl -L --progress http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.6.tar.gz | tar xz - cd ruby-2.1.6 + curl -O --progress https://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.7.tar.gz + echo 'e2e195a4a58133e3ad33b955c829bb536fa3c075 ruby-2.1.7.tar.gz' | shasum -c - && tar xzf ruby-2.1.7.tar.gz + cd ruby-2.1.7 ./configure --disable-install-rdoc make sudo make install @@ -127,15 +128,17 @@ Install the Bundler Gem: ## 3. Go -Since GitLab 8.0, Git HTTP requests are handled by gitlab-git-http-server. -This is a small daemon written in Go. -To install gitlab-git-http-server we need a Go compiler. +Since GitLab 8.0, Git HTTP requests are handled by gitlab-workhorse (formerly +gitlab-git-http-server). This is a small daemon written in Go. To install +gitlab-workhorse we need a Go compiler. The instructions below assume you +use 64-bit Linux. You can find downloads for other platforms at the [Go download +page](https://golang.org/dl). - curl -O --progress https://storage.googleapis.com/golang/go1.5.linux-amd64.tar.gz - echo '5817fa4b2252afdb02e11e8b9dc1d9173ef3bd5a go1.5.linux-amd64.tar.gz' | shasum -c - && \ - sudo tar -C /usr/local -xzf go1.5.linux-amd64.tar.gz + curl -O --progress https://storage.googleapis.com/golang/go1.5.1.linux-amd64.tar.gz + echo '46eecd290d8803887dec718c691cc243f2175fe0 go1.5.1.linux-amd64.tar.gz' | shasum -c - && \ + sudo tar -C /usr/local -xzf go1.5.1.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ - rm go1.5.linux-amd64.tar.gz + rm go1.5.1.linux-amd64.tar.gz ## 4. System Users @@ -207,9 +210,9 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da ### Clone the Source # Clone GitLab repository - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-0-stable gitlab + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-2-stable gitlab -**Note:** You can change `8-0-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! +**Note:** You can change `8-2-stable` to `master` if you want the *bleeding edge* version, but never install master on a production server! ### Configure It @@ -242,6 +245,9 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da # Change the permissions of the directory where CI build traces are stored sudo chmod -R u+rwX builds/ + # Change the permissions of the directory where CI artifacts are stored + sudo chmod -R u+rwX shared/artifacts/ + # Copy the example Unicorn config sudo -u git -H cp config/unicorn.rb.example config/unicorn.rb @@ -249,8 +255,8 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da nproc # Enable cluster mode if you expect to have a high load instance - # Ex. change amount of workers to 3 for 2GB RAM server # Set the number of workers to at least the number of cores + # Ex. change amount of workers to 3 for 2GB RAM server sudo -u git -H editor config/unicorn.rb # Copy the example Rack attack config @@ -291,7 +297,7 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da ### Install Gems -**Note:** As of bundler 1.5.2, you can invoke `bundle install -jN` (where `N` the number of your processor cores) and enjoy the parallel gems installation with measurable difference in completion time (~60% faster). Check the number of your cores with `nproc`. For more information check this [post](http://robots.thoughtbot.com/parallel-gem-installing-using-bundler). First make sure you have bundler >= 1.5.2 (run `bundle -v`) as it addresses some [issues](https://devcenter.heroku.com/changelog-items/411) that were [fixed](https://github.com/bundler/bundler/pull/2817) in 1.5.2. +**Note:** As of bundler 1.5.2, you can invoke `bundle install -jN` (where `N` the number of your processor cores) and enjoy the parallel gems installation with measurable difference in completion time (~60% faster). Check the number of your cores with `nproc`. For more information check this [post](https://robots.thoughtbot.com/parallel-gem-installing-using-bundler). First make sure you have bundler >= 1.5.2 (run `bundle -v`) as it addresses some [issues](https://devcenter.heroku.com/changelog-items/411) that were [fixed](https://github.com/bundler/bundler/pull/2817) in 1.5.2. # For PostgreSQL (note, the option says "without ... mysql") sudo -u git -H bundle install --deployment --without development test mysql aws kerberos @@ -306,7 +312,7 @@ We recommend using a PostgreSQL database. For MySQL check [MySQL setup guide](da GitLab Shell is an SSH access and repository management software developed specially for GitLab. # Run the installation task for gitlab-shell (replace `REDIS_URL` if needed): - sudo -u git -H bundle exec rake gitlab:shell:install[v2.6.5] REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production + sudo -u git -H bundle exec rake gitlab:shell:install REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production # By default, the gitlab-shell config is generated from your main GitLab config. # You can review (and modify) the gitlab-shell config as follows: @@ -314,17 +320,22 @@ GitLab Shell is an SSH access and repository management software developed speci **Note:** If you want to use HTTPS, see [Using HTTPS](#using-https) for the additional steps. -**Note:** Make sure your hostname can be resolved on the machine itself by either a proper DNS record or an additional line in /etc/hosts ("127.0.0.1 hostname"). This might be necessary for example if you set up gitlab behind a reverse proxy. If the hostname cannot be resolved, the final installation check will fail with "Check GitLab API access: FAILED. code: 401" and pushing commits will be rejected with "[remote rejected] master -> master (hook declined)". +**Note:** Make sure your hostname can be resolved on the machine itself by either a proper DNS record or an additional line in /etc/hosts ("127.0.0.1 hostname"). This might be necessary for example if you set up GitLab behind a reverse proxy. If the hostname cannot be resolved, the final installation check will fail with "Check GitLab API access: FAILED. code: 401" and pushing commits will be rejected with "[remote rejected] master -> master (hook declined)". -### Install gitlab-git-http-server +### Install gitlab-workhorse cd /home/git - sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-git-http-server.git - cd gitlab-git-http-server + sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git + cd gitlab-workhorse + sudo -u git -H git checkout 0.4.2 sudo -u git -H make ### Initialize Database and Activate Advanced Features + # Go to GitLab installation folder + + cd /home/git/gitlab + sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production # Type 'yes' to create the database tables. @@ -341,11 +352,6 @@ The `secrets.yml` file stores encryption keys for sessions and secure variables. Backup `secrets.yml` someplace safe, but don't store it in the same place as your database backups. Otherwise your secrets are exposed if one of your backups is compromised. -### Install schedules - - # Setup schedules - sudo -u gitlab_ci -H bundle exec whenever -w RAILS_ENV=production - ### Install Init Script Download the init script (will be `/etc/init.d/gitlab`): @@ -489,7 +495,7 @@ See the [omniauth integration document](../integration/omniauth.md) ### Build your projects GitLab can build your projects. To enable that feature you need GitLab Runners to do that for you. -Checkout the [Gitlab Runner section](https://about.gitlab.com/gitlab-ci/#gitlab-runner) to install it +Checkout the [GitLab Runner section](https://about.gitlab.com/gitlab-ci/#gitlab-runner) to install it ### Custom Redis Connection diff --git a/doc/install/requirements.md b/doc/install/requirements.md index aa0d03b75bc..c0ccdd37458 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -109,8 +109,4 @@ On a very active server (10,000 active users) the Sidekiq process can use 1GB+ o - Firefox (Latest released version and [latest ESR version](https://www.mozilla.org/en-US/firefox/organizations/)) - Safari 7+ (known problem: required fields in html5 do not work) - Opera (Latest released version) -- IE 10+ - -### Common UI problems with IE - -If you experience UI issues with Internet Explorer, please make sure that you have the `Compatibility View` mode disabled.
\ No newline at end of file +- Internet Explorer (IE) 10+ but please make sure that you have the `Compatibility View` mode disabled.
\ No newline at end of file diff --git a/doc/integration/facebook.md b/doc/integration/facebook.md new file mode 100644 index 00000000000..bc1f1673086 --- /dev/null +++ b/doc/integration/facebook.md @@ -0,0 +1,97 @@ +# Facebook OAuth2 OmniAuth Provider + +To enable the Facebook OmniAuth provider you must register your application with Facebook. Facebook will generate an app ID and secret key for you to use. + +1. Sign in to the [Facebook Developer Platform](https://developers.facebook.com/). + +1. Choose "My Apps" > "Add a New App" + +1. Select the type "Website" + +1. Enter a name for your app. This can be anything. Consider something like "<Organization>'s GitLab" or "<Your Name>'s GitLab" or +something else descriptive. + +1. Choose "Create New Facebook App ID" + +1. Select a Category, for example "Productivity" + +1. Choose "Create App ID" + +1. Enter the address of your GitLab installation at the bottom of the package + +  + +1. Choose "Next" + +1. Choose "Skip Quick Start" in the upper right corner + +1. Choose "Settings" in the menu on the left + +1. Fill in a contact email for your app + +  + +1. Choose "Save Changes" + +1. Choose "Status & Review" in the menu on the left + +1. Change the switch on the right from No to Yes + +1. Choose "Confirm" when prompted to make the app public + +1. Choose "Dashboard" in the menu on the left + +1. Choose "Show" next to the hidden "App Secret" + +1. You should now see an app key and app secret (see screenshot). Keep this page open as you continue configuration. + +  + +1. On your GitLab server, open the configuration file. + + For omnibus package: + + ```sh + sudo editor /etc/gitlab/gitlab.rb + ``` + + For installations from source: + + ```sh + cd /home/git/gitlab + + sudo -u git -H editor config/gitlab.yml + ``` + +1. See [Initial OmniAuth Configuration](omniauth.md#initial-omniauth-configuration) for initial settings. + +1. Add the provider configuration: + + For omnibus package: + + ```ruby + gitlab_rails['omniauth_providers'] = [ + { + "name" => "facebook", + "app_id" => "YOUR_APP_ID", + "app_secret" => "YOUR_APP_SECRET" + } + ] + ``` + + For installations from source: + + ``` + - { name: 'facebook', app_id: 'YOUR_APP_ID', + app_secret: 'YOUR_APP_SECRET' } + ``` + +1. Change 'YOUR_APP_ID' to the API key from Facebook page in step 10. + +1. Change 'YOUR_APP_SECRET' to the API secret from the Facebook page in step 10. + +1. Save the configuration file. + +1. Restart GitLab for the changes to take effect. + +On the sign in page there should now be a Facebook icon below the regular sign in form. Click the icon to begin the authentication process. Facebook will ask the user to sign in and authorize the GitLab application. If everything goes well the user will be returned to GitLab and will be signed in. diff --git a/doc/integration/facebook_api_keys.png b/doc/integration/facebook_api_keys.png Binary files differnew file mode 100644 index 00000000000..d6c44ac0f11 --- /dev/null +++ b/doc/integration/facebook_api_keys.png diff --git a/doc/integration/facebook_app_settings.png b/doc/integration/facebook_app_settings.png Binary files differnew file mode 100644 index 00000000000..30dd21e198a --- /dev/null +++ b/doc/integration/facebook_app_settings.png diff --git a/doc/integration/facebook_website_url.png b/doc/integration/facebook_website_url.png Binary files differnew file mode 100644 index 00000000000..dc3088bb2fa --- /dev/null +++ b/doc/integration/facebook_website_url.png diff --git a/doc/integration/ldap.md b/doc/integration/ldap.md index 3bc5df21ef4..7e2920b8865 100644 --- a/doc/integration/ldap.md +++ b/doc/integration/ldap.md @@ -71,7 +71,7 @@ main: # 'main' is the GitLab 'provider ID' of this LDAP server # Filter LDAP users # - # Format: RFC 4515 http://tools.ietf.org/search/rfc4515 + # Format: RFC 4515 https://tools.ietf.org/search/rfc4515 # Ex. (employeeType=developer) # # Note: GitLab does not support omniauth-ldap's custom filter syntax. @@ -145,7 +145,7 @@ If multiple LDAP email attributes are present, e.g. `mail: foo@bar.com` and `ema ## Using an LDAP filter to limit access to your GitLab server If you want to limit all GitLab access to a subset of the LDAP users on your LDAP server you can set up an LDAP user filter. -The filter must comply with [RFC 4515](http://tools.ietf.org/search/rfc4515). +The filter must comply with [RFC 4515](https://tools.ietf.org/search/rfc4515). ```ruby # For omnibus packages; new LDAP server syntax @@ -173,3 +173,23 @@ Tip: if you want to limit access to the nested members of an Active Directory gr ``` Please note that GitLab does not support the custom filter syntax used by omniauth-ldap. + +## Limitations + +GitLab's LDAP client is based on [omniauth-ldap](https://gitlab.com/gitlab-org/omniauth-ldap) +which encapsulates Ruby's `Net::LDAP` class. It provides a pure-Ruby implementation +of the LDAP client protocol. As a result, GitLab is limited by `omniauth-ldap` and may impact your LDAP +server settings. + +### TLS Client Authentication +Not implemented by `Net::LDAP`. +So you should disable anonymous LDAP authentication and enable simple or SASL +authentication. TLS client authentication setting in your LDAP server cannot be +mandatory and clients cannot be authenticated with the TLS protocol. + +### TLS Server Authentication +Not supported by GitLab's configuration options. +When setting `method: ssl`, the underlying authentication method used by +`omniauth-ldap` is `simple_tls`. This method establishes TLS encryption with +the LDAP server before any LDAP-protocol data is exchanged but no validation of +the LDAP server's SSL certificate is performed.
\ No newline at end of file diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index c5cecbc2f2d..f2b1721fc03 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -36,7 +36,7 @@ If you want to change these settings: ``` gitlab_rails['omniauth_enabled'] = true gitlab_rails['omniauth_allow_single_sign_on'] = false - gitlab_rails['block_auto_created_users'] = true + gitlab_rails['omniauth_block_auto_created_users'] = true ``` * **For installations from source** @@ -73,8 +73,9 @@ Now we can choose one or more of the Supported Providers below to continue confi - [Bitbucket](bitbucket.md) - [GitLab.com](gitlab.md) - [Google](google.md) -- [Shibboleth](shibboleth.md) +- [Facebook](facebook.md) - [Twitter](twitter.md) +- [Shibboleth](shibboleth.md) - [SAML](saml.md) - [Crowd](crowd.md) diff --git a/doc/legal/corporate_contributor_license_agreement.md b/doc/legal/corporate_contributor_license_agreement.md index 13bf15fcf45..7b94506c297 100644 --- a/doc/legal/corporate_contributor_license_agreement.md +++ b/doc/legal/corporate_contributor_license_agreement.md @@ -22,4 +22,4 @@ You accept and agree to the following terms and conditions for Your present and 8. It is your responsibility to notify GitLab B.V. when any change is required to the list of designated employees authorized to submit Contributions on behalf of the Corporation, or to the Corporation's Point of Contact with GitLab B.V.. -This text is licensed under the [Creative Commons Attribution 3.0 License](http://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. +This text is licensed under the [Creative Commons Attribution 3.0 License](https://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. diff --git a/doc/legal/individual_contributor_license_agreement.md b/doc/legal/individual_contributor_license_agreement.md index 72b01433dd0..f97c252fd7c 100644 --- a/doc/legal/individual_contributor_license_agreement.md +++ b/doc/legal/individual_contributor_license_agreement.md @@ -22,4 +22,4 @@ You accept and agree to the following terms and conditions for Your present and 8. You agree to notify GitLab B.V. of any facts or circumstances of which you become aware that would make these representations inaccurate in any respect. -This text is licensed under the [Creative Commons Attribution 3.0 License](http://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. +This text is licensed under the [Creative Commons Attribution 3.0 License](https://creativecommons.org/licenses/by/3.0/) and the original source is the Google Open Source Programs Office. diff --git a/doc/markdown/markdown.md b/doc/markdown/markdown.md index d83838127c9..bc8e7d155e7 100644 --- a/doc/markdown/markdown.md +++ b/doc/markdown/markdown.md @@ -43,7 +43,7 @@ You can also use other rich text files in GitLab. You might have to install a de ## Newlines -GFM honors the markdown specification in how [paragraphs and line breaks are handled](http://daringfireball.net/projects/markdown/syntax#p). +GFM honors the markdown specification in how [paragraphs and line breaks are handled](https://daringfireball.net/projects/markdown/syntax#p). A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines. Line-breaks, or softreturns, are rendered if you end a line with two or more spaces @@ -72,14 +72,14 @@ do_this_and_do_that_and_another_thing GFM will autolink almost any URL you copy and paste into your text. - * http://www.google.com + * https://www.google.com * https://google.com/ * ftp://ftp.us.debian.org/debian/ * smb://foo/bar/baz * irc://irc.freenode.net/gitlab * http://localhost:3000 -* http://www.google.com +* https://www.google.com * https://google.com/ * ftp://ftp.us.debian.org/debian/ * smb://foo/bar/baz @@ -274,7 +274,7 @@ The IDs are generated from the content of the header according to the following 1. All spaces are converted to hyphens 1. Two or more hyphens in a row are converted to one 1. If a header with the same ID has already been generated, a unique - incrementing number is appended. + incrementing number is appended, starting at 1. For example: @@ -291,8 +291,8 @@ Would generate the following link IDs: 1. `this-header-has-spaces-in-it` 1. `this-header-has-a-in-it` 1. `this-header-has-unicode-in-it-한글` +1. `this-header-has-spaces-in-it` 1. `this-header-has-spaces-in-it-1` -1. `this-header-has-spaces-in-it-2` Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. @@ -390,7 +390,7 @@ There are two ways to create links, inline-style and reference-style. [arbitrary case-insensitive reference text]: https://www.mozilla.org [1]: http://slashdot.org - [link text itself]: http://www.reddit.com + [link text itself]: https://www.reddit.com [I'm an inline-style link](https://www.google.com) @@ -406,7 +406,7 @@ Some text to show that the reference links can follow later. [arbitrary case-insensitive reference text]: https://www.mozilla.org [1]: http://slashdot.org -[link text itself]: http://www.reddit.com +[link text itself]: https://www.reddit.com **Note** @@ -583,5 +583,5 @@ By including colons in the header row, you can align the text within that column ## References - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). -- The [Markdown Syntax Guide](http://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. +- The [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. - [Dillinger.io](http://dillinger.io) is a handy tool for testing standard markdown. diff --git a/doc/migrate_ci_to_ce/README.md b/doc/migrate_ci_to_ce/README.md index 2725bf343ee..5ec0a2069b5 100644 --- a/doc/migrate_ci_to_ce/README.md +++ b/doc/migrate_ci_to_ce/README.md @@ -28,13 +28,20 @@ upgrade to 8.0 until you finish the migration procedure. ### Before upgrading -If you have GitLab CI installed using omnibus-gitlab packages but *you don't want to migrate your existing data*: +If you have GitLab CI installed using omnibus-gitlab packages but **you don't want to migrate your existing data**: ```bash mv /var/opt/gitlab/gitlab-ci/builds /var/opt/gitlab/gitlab-ci/builds.$(date +%s) ``` -and run `sudo gitlab-ctl reconfigure`. +run `sudo gitlab-ctl reconfigure` and you can reach CI at `gitlab.example.com/ci`. + +If you want to migrate your existing data, continue reading. + +#### 0. Updating Omnibus from versions prior to 7.13 + +If you are updating from older versions you should first update to 7.14 and then to 8.0. +Otherwise it's pretty likely that you will encounter problems described in the [Troubleshooting](#troubleshooting). #### 1. Verify that backups work @@ -43,6 +50,7 @@ Make sure that the backup script on both servers can connect to the database. ``` # On your CI server: # Omnibus +sudo chown gitlab-ci:gitlab-ci /var/opt/gitlab/gitlab-ci/builds sudo gitlab-ci-rake backup:create # Source @@ -143,7 +151,7 @@ sudo gitlab-ctl stop ci-sidekiq # Source sudo service gitlab_ci stop cd /home/gitlab_ci/gitlab-ci -sudo -u gitlab_ci -H bundle exec whenever --clear-crontab +sudo -u gitlab_ci -H bundle exec whenever --clear-crontab RAILS_ENV=production ``` ### II. Moving data @@ -176,6 +184,7 @@ will need this file later. ``` # On your CI server: # Omnibus +sudo chown gitlab-ci:gitlab-ci /var/opt/gitlab/gitlab-ci/builds sudo gitlab-ci-rake backup:create # Source @@ -222,6 +231,7 @@ be no CI data yet because you turned CI on the GitLab server off earlier. ``` # On your GitLab server: # Omnibus +sudo chown git:git /var/opt/gitlab/gitlab-ci/builds sudo gitlab-rake ci:migrate # Source @@ -319,3 +329,107 @@ You should also make sure that you can: If something went wrong and you need to restore a backup, consult the [Backup restoration](../raketasks/backup_restore.md) guide. + +### Troubleshooting + +#### show:secrets problem (Omnibus-only) +If you see errors like this: +``` +Missing `secret_key_base` or `db_key_base` for 'production' environment. The secrets will be generated and stored in `config/secrets.yml` +rake aborted! +Errno::EACCES: Permission denied @ rb_sysopen - config/secrets.yml +``` + +This can happen if you are updating from versions prior to 7.13 straight to 8.0. +The fix for this is to update to Omnibus 7.14 first and then update it to 8.0. + +#### Permission denied when accessing /var/opt/gitlab/gitlab-ci/builds +To fix that issue you have to change builds/ folder permission before doing final backup: +``` +sudo chown -R gitlab-ci:gitlab-ci /var/opt/gitlab/gitlab-ci/builds +``` + +Then before executing `ci:migrate` you need to fix builds folder permission: +``` +sudo chown git:git /var/opt/gitlab/gitlab-ci/builds +``` + +#### Problems when importing CI database to GitLab +If you were migrating CI database from MySQL to PostgreSQL manually you can see errros during import about missing sequences: +``` +ALTER SEQUENCE +ERROR: relation "ci_builds_id_seq" does not exist +ERROR: relation "ci_commits_id_seq" does not exist +ERROR: relation "ci_events_id_seq" does not exist +ERROR: relation "ci_jobs_id_seq" does not exist +ERROR: relation "ci_projects_id_seq" does not exist +ERROR: relation "ci_runner_projects_id_seq" does not exist +ERROR: relation "ci_runners_id_seq" does not exist +ERROR: relation "ci_services_id_seq" does not exist +ERROR: relation "ci_taggings_id_seq" does not exist +ERROR: relation "ci_tags_id_seq" does not exist +CREATE TABLE +``` + +To fix that you need to apply this SQL statement before doing final backup: +``` +# Omnibus +gitlab-ci-rails dbconsole <<EOF +-- ALTER TABLES - DROP DEFAULTS +ALTER TABLE ONLY ci_application_settings ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_builds ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_commits ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_events ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_jobs ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_projects ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_runner_projects ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_runners ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_services ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_taggings ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_tags ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_trigger_requests ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_triggers ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_variables ALTER COLUMN id DROP DEFAULT; +ALTER TABLE ONLY ci_web_hooks ALTER COLUMN id DROP DEFAULT; + +-- ALTER SEQUENCES +ALTER SEQUENCE ci_application_settings_id_seq OWNED BY ci_application_settings.id; +ALTER SEQUENCE ci_builds_id_seq OWNED BY ci_builds.id; +ALTER SEQUENCE ci_commits_id_seq OWNED BY ci_commits.id; +ALTER SEQUENCE ci_events_id_seq OWNED BY ci_events.id; +ALTER SEQUENCE ci_jobs_id_seq OWNED BY ci_jobs.id; +ALTER SEQUENCE ci_projects_id_seq OWNED BY ci_projects.id; +ALTER SEQUENCE ci_runner_projects_id_seq OWNED BY ci_runner_projects.id; +ALTER SEQUENCE ci_runners_id_seq OWNED BY ci_runners.id; +ALTER SEQUENCE ci_services_id_seq OWNED BY ci_services.id; +ALTER SEQUENCE ci_taggings_id_seq OWNED BY ci_taggings.id; +ALTER SEQUENCE ci_tags_id_seq OWNED BY ci_tags.id; +ALTER SEQUENCE ci_trigger_requests_id_seq OWNED BY ci_trigger_requests.id; +ALTER SEQUENCE ci_triggers_id_seq OWNED BY ci_triggers.id; +ALTER SEQUENCE ci_variables_id_seq OWNED BY ci_variables.id; +ALTER SEQUENCE ci_web_hooks_id_seq OWNED BY ci_web_hooks.id; + +-- ALTER TABLES - RE-APPLY DEFAULTS +ALTER TABLE ONLY ci_application_settings ALTER COLUMN id SET DEFAULT nextval('ci_application_settings_id_seq'::regclass); +ALTER TABLE ONLY ci_builds ALTER COLUMN id SET DEFAULT nextval('ci_builds_id_seq'::regclass); +ALTER TABLE ONLY ci_commits ALTER COLUMN id SET DEFAULT nextval('ci_commits_id_seq'::regclass); +ALTER TABLE ONLY ci_events ALTER COLUMN id SET DEFAULT nextval('ci_events_id_seq'::regclass); +ALTER TABLE ONLY ci_jobs ALTER COLUMN id SET DEFAULT nextval('ci_jobs_id_seq'::regclass); +ALTER TABLE ONLY ci_projects ALTER COLUMN id SET DEFAULT nextval('ci_projects_id_seq'::regclass); +ALTER TABLE ONLY ci_runner_projects ALTER COLUMN id SET DEFAULT nextval('ci_runner_projects_id_seq'::regclass); +ALTER TABLE ONLY ci_runners ALTER COLUMN id SET DEFAULT nextval('ci_runners_id_seq'::regclass); +ALTER TABLE ONLY ci_services ALTER COLUMN id SET DEFAULT nextval('ci_services_id_seq'::regclass); +ALTER TABLE ONLY ci_taggings ALTER COLUMN id SET DEFAULT nextval('ci_taggings_id_seq'::regclass); +ALTER TABLE ONLY ci_tags ALTER COLUMN id SET DEFAULT nextval('ci_tags_id_seq'::regclass); +ALTER TABLE ONLY ci_trigger_requests ALTER COLUMN id SET DEFAULT nextval('ci_trigger_requests_id_seq'::regclass); +ALTER TABLE ONLY ci_triggers ALTER COLUMN id SET DEFAULT nextval('ci_triggers_id_seq'::regclass); +ALTER TABLE ONLY ci_variables ALTER COLUMN id SET DEFAULT nextval('ci_variables_id_seq'::regclass); +ALTER TABLE ONLY ci_web_hooks ALTER COLUMN id SET DEFAULT nextval('ci_web_hooks_id_seq'::regclass); +EOF + +# Source +cd /home/gitlab_ci/gitlab-ci +sudo -u gitlab_ci -H bundle exec rails dbconsole production <<EOF +... COPY SQL STATEMENTS FROM ABOVE ... +EOF +``` diff --git a/doc/operations/unicorn.md b/doc/operations/unicorn.md index 31b432cd411..bad61151bda 100644 --- a/doc/operations/unicorn.md +++ b/doc/operations/unicorn.md @@ -52,7 +52,7 @@ leak memory, probably because it does not handle user requests.) To make these memory leaks manageable, GitLab comes with the [unicorn-worker-killer gem](https://github.com/kzk/unicorn-worker-killer). This -gem [monkey-patches](http://en.wikipedia.org/wiki/Monkey_patch) the Unicorn +gem [monkey-patches](https://en.wikipedia.org/wiki/Monkey_patch) the Unicorn workers to do a memory self-check after every 16 requests. If the memory of the Unicorn worker exceeds a pre-set limit then the worker process exits. The Unicorn master then automatically replaces the worker process. @@ -78,9 +78,9 @@ threshold is a random value between 200 and 250 MB. The master process (PID ``` One other thing that stands out in the log snippet above, taken from -Gitlab.com, is that 'worker 4' was serving requests for only 23 seconds. This +GitLab.com, is that 'worker 4' was serving requests for only 23 seconds. This is a normal value for our current GitLab.com setup and traffic. The high frequency of Unicorn memory restarts on some GitLab sites can be a source of confusion for administrators. Usually they are a [red -herring](http://en.wikipedia.org/wiki/Red_herring). +herring](https://en.wikipedia.org/wiki/Red_herring). diff --git a/doc/permissions/permissions.md b/doc/permissions/permissions.md index 7a6a1958445..bcd00cfc6bf 100644 --- a/doc/permissions/permissions.md +++ b/doc/permissions/permissions.md @@ -6,6 +6,9 @@ If a user is both in a project group and in the project itself, the highest perm If a user is a GitLab administrator they receive all permissions. +On public projects the Guest role is not enforced. +All users will be able to create issues, leave comments, and pull or download the project code. + To add or import a user, you can follow the [project users and members documentation](doc/workflow/add-user/add-user.md). diff --git a/doc/public_access/public_access.md b/doc/public_access/public_access.md index bd439f7c6f3..6e22ea7b72a 100644 --- a/doc/public_access/public_access.md +++ b/doc/public_access/public_access.md @@ -1,44 +1,59 @@ # Public access -GitLab allows you to open selected projects to be accessed **publicly** or **internally**. +GitLab allows you to change your projects' visibility in order be accessed +**publicly** or **internally**. -Projects with either of these visibility levels will be listed in the [public access directory](/public). +Projects with either of these visibility levels will be listed in the +public access directory (`/public` under your GitLab instance). +Here is the [GitLab.com example](https://gitlab.com/public). Internal projects will only be available to authenticated users. -## Public projects +## Visibility of projects + +### Public projects Public projects can be cloned **without any** authentication. -It will also be listed on the [public access directory](/public). +They will also be listed on the public access directory (`/public`). -**Any logged in user** will have [Guest](../permissions/permissions) permissions on the repository. +**Any logged in user** will have [Guest](../permissions/permissions) +permissions on the repository. -## Internal projects +### Internal projects Internal projects can be cloned by any logged in user. -It will also be listed on the [public access directory](/public) for logged in users. +They will also be listed on the public access directory (`/public`) for logged +in users. -Any logged in user will have [Guest](../permissions/permissions) permissions on the repository. +Any logged in user will have [Guest](../permissions/permissions) permissions on +the repository. -## How to change project visibility +### How to change project visibility -1. Go to your project dashboard -1. Click on the "Edit" tab -1. Change "Visibility Level" +1. Go to your project's **Settings** +1. Change "Visibility Level" to either Public, Internal or Private ## Visibility of users -The public page of users, located at `/u/username` is visible if either: +The public page of a user, located at `/u/username`, is always visible whether +you are logged in or not. + +When visiting the public page of a user, you can only see the projects which +you are privileged to. -- You are logged in. -- You are logged out, and the target user is authorized to (is Guest, Reporter, etc.) at least one public project. +## Visibility of groups -Otherwise, you will be redirected to the sign in page. +The public page of a group, located at `/groups/groupname`, is always visible +to everyone. -When visiting the public page of an user, you will only see listed projects which you can view yourself. +Logged out users will be able to see the description and the avatar of the +group as well as all public projects belonging to that group. ## Restricting the use of public or internal projects -In the Admin area under Settings you can disable public projects or public and internal projects for the entire GitLab installation to prevent people making code public by accident. The restricted visibility settings do not apply to admin users. +In the Admin area under **Settings** (`/admin/application_settings`), you can +restrict the use of visibility levels for users when they create a project or a +snippet. This is useful to prevent people exposing their repositories to public +by accident. The restricted visibility settings do not apply to admin users. diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 4ff5e74d438..b4d2786bd76 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -7,14 +7,16 @@ A backup creates an archive file that contains the database, all repositories and all attachments. This archive will be saved in backup_path (see `config/gitlab.yml`). The filename will be `[TIMESTAMP]_gitlab_backup.tar`. This timestamp can be used to restore an specific backup. -You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. The best way to migrate your repositories from one server to another is through backup restore. +You can only restore a backup to exactly the same version of GitLab that you created it +on, for example 7.2.1. The best way to migrate your repositories from one server to +another is through backup restore. You need to keep a separate copy of `/etc/gitlab/gitlab-secrets.json` (for omnibus packages) or `/home/git/gitlab/.secret` (for installations from source). This file contains the database encryption key used for two-factor authentication. If you restore a GitLab backup without restoring the database encryption key, users who have two-factor -authentication enabled will loose access to your GitLab server. +authentication enabled will lose access to your GitLab server. If you are interested in GitLab CI backup please follow to the [CI backup documentation](https://gitlab.com/gitlab-org/gitlab-ci/blob/master/doc/raketasks/backup_restore.md)* @@ -27,7 +29,8 @@ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production ``` Also you can choose what should be backed up by adding environment variable SKIP. Available options: db, -uploads (attachments), repositories. Use a comma to specify several options at the same time. +uploads (attachments), repositories, builds(CI build output logs), artifacts (CI build artifacts), lfs (LFS objects). +Use a comma to specify several options at the same time. ``` sudo gitlab-rake gitlab:backup:create SKIP=db,uploads @@ -95,6 +98,8 @@ For installations from source: aws_secret_access_key: 'secret123' # The remote 'directory' to store your backups. For S3, this would be the bucket name. remote_directory: 'my.s3.bucket' + # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional + # encryption: 'AES256' ``` If you are uploading your backups to S3 you will probably want to create a new @@ -269,9 +274,6 @@ sudo gitlab-rake gitlab:backup:restore BACKUP=1393513186 # Start GitLab sudo gitlab-ctl start -# Create satellites -sudo gitlab-rake gitlab:satellites:create - # Check GitLab sudo gitlab-rake gitlab:check SANITIZE=true ``` @@ -369,7 +371,11 @@ For more information see similar questions on postgresql issue tracker[here](htt ## Note This documentation is for GitLab CE. -We backup GitLab.com and make sure your data is secure, but you can't use these methods -to export / backup your data yourself from GitLab.com. +We backup GitLab.com and make sure your data is secure, but you can't use these methods to export / backup your data yourself from GitLab.com. Issues are stored in the database. They can't be stored in Git itself. + +To migrate your repositories from one server to another with an up-to-date version of +GitLab, you can use the [import rake task](import.md) to do a mass import of the +repository. Note that if you do an import rake task, rather than a backup restore, you +will have all your repositories, but not any other data. diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md index 96d67f7b5d6..8fbcbb983e9 100644 --- a/doc/raketasks/cleanup.md +++ b/doc/raketasks/cleanup.md @@ -12,7 +12,8 @@ sudo gitlab-rake gitlab:cleanup:dirs bundle exec rake gitlab:cleanup:dirs RAILS_ENV=production ``` -Remove repositories (global only for now) from `/home/git/repositories` if they don't exist in GitLab database. +Rename repositories from `/home/git/repositories` if they don't exist in GitLab database. +The repositories get a `+orphaned+TIMESTAMP` suffix so that they cannot block new repositories from being created. ``` # omnibus-gitlab diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md index 4fbd20762da..629d38efc53 100644 --- a/doc/raketasks/user_management.md +++ b/doc/raketasks/user_management.md @@ -56,3 +56,17 @@ bundle exec rake gitlab:import:all_users_to_all_groups RAILS_ENV=production ``` block_auto_created_users: false ``` + +## Disable Two-factor Authentication (2FA) for all users + +This task will disable 2FA for all users that have it enabled. This can be +useful if GitLab's `.secret` file has been lost and users are unable to login, +for example. + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:two_factor:disable_for_all_users + +# installation from source +bundle exec rake gitlab:two_factor:disable_for_all_users RAILS_ENV=production +``` diff --git a/doc/release/README.md b/doc/release/README.md index 1342b90f3b3..52eca7c02a6 100644 --- a/doc/release/README.md +++ b/doc/release/README.md @@ -1,4 +1,8 @@ -GitLab has the following updates: +## Release cycle + +Since 2011 a minor or major version of GitLab is released on the 22nd of every month. Patch and security releases are published when needed. New features are detailed on the [blog](https://about.gitlab.com/blog/) and in the [changelog](CHANGELOG). Features that will likely be in the next releases can be found on the [direction page](https://about.gitlab.com/direction/). + +## Release process documentation - [Monthly release](monthly.md), every month on the 22nd. - [Patch release](patch.md), if there are serious regressions. diff --git a/doc/release/monthly.md b/doc/release/monthly.md index c56e99a7005..aff3f066b24 100644 --- a/doc/release/monthly.md +++ b/doc/release/monthly.md @@ -25,69 +25,88 @@ If the release is falling behind immediately warn the team. ## Create an overall issue and follow it -Create issue for GitLab CE project(internal). Name it "Release x.x.x" for easier searching. -Replace the dates with actual dates based on the number of workdays before the release. -All steps from issue template are explained below +Create an issue in the GitLab CE project. Name it "Release x.x" and tag it with +the `release` label for easier searching. Replace the dates with actual dates +based on the number of workdays before the release. All steps from issue +template are explained below: ``` -Xth: (7 working days before the 22nd) +### Xth: (7 working days before the 22nd) -- [ ] Triage the omnibus-gitlab milestone +- [ ] Triage the [Omnibus milestone] -Xth: (6 working days before the 22nd) +### Xth: (6 working days before the 22nd) -- [ ] Merge CE master in to EE master via merge request (#LINK) - [ ] Determine QA person and notify this person - [ ] Check the tasks in [how to rc1 guide](https://dev.gitlab.org/gitlab/gitlabhq/blob/master/doc/release/howto_rc1.md) and delegate tasks if necessary -- [ ] Create CE, EE, CI RC1 versions (#LINK) -- [ ] Build RC1 packages (EE first) (#LINK) +- [ ] Merge CE `master` into EE `master` via merge request (#LINK) +- [ ] Create CE and EE RC1 versions (#LINK) +- [ ] Build RC1 packages -Xth: (5 working days before the 22nd) +### Xth: (5 working days before the 22nd) - [ ] Do QA and fix anything coming out of it (#LINK) -- [ ] Close the omnibus-gitlab milestone -- [ ] Prepare the blog post (#LINK) +- [ ] Close the [Omnibus milestone] +- [ ] Prepare the [blog post] -Xth: (4 working days before the 22nd) +### Xth: (4 working days before the 22nd) -- [ ] Update GitLab.com with rc1 (#LINK) (https://dev.gitlab.org/cookbooks/chef-repo/blob/master/doc/administration.md#deploy-the-package) -- [ ] Update ci.gitLab.com with rc1 (#LINK) (https://dev.gitlab.org/cookbooks/chef-repo/blob/master/doc/administration.md#deploy-the-package) -- [ ] Create regression issues (CE, CI) (#LINK) -- [ ] Tweet about rc1 (#LINK), proposed text: +- [ ] Update GitLab.com with RC1 +- [ ] Create the regression issue in the CE issue tracker: -> GitLab x.x.0.rc1 is available https://packages.gitlab.com/gitlab/unstable Use at your own risk. Please link regressions issues from LINK_TO_REGRESSION_ISSUE + ``` + This is a meta issue to index possible regressions in this monthly release + and any patch versions. -Xth: (3 working days before the 22nd) + Please do not raise or discuss issues directly in this issue but link to + issues that might warrant a patch release. If there is a Merge Request + that fixes the issue, please link to that as well. -- [ ] Merge CE stable branch into EE stable branch + Please only post one regression issue and/or merge request per comment. + Comments will be updated by the release manager as they are addressed. + ``` -Xth: (2 working days before the 22nd) +- [ ] Tweet about RC1 release: -- [ ] Check that everyone is mentioned on the blog post using `@all` (the reviewer should have done this one working day ago) -- [ ] Check that MVP is added to the mvp page (source/mvp/index.html in www-gitlab-com) + ``` + GitLab x.y.0.rc1 is available: https://packages.gitlab.com/gitlab/unstable + Use at your own risk. Please link regressions issues from + LINK_TO_REGRESSION_ISSUE + ``` -Xth: (1 working day before the 22nd) +### Xth: (3 working days before the 22nd) -- [ ] Merge CE stable into EE stable -- [ ] Create CE, EE, CI release candidates (#LINK) (hopefully final ones with the same commit as the release tomorrow) +- [ ] Merge `x-y-stable` into `x-y-stable-ee` +- [ ] Check that everyone is mentioned on the [blog post] using `@all` + +### Xth: (2 working days before the 22nd) + +- [ ] Check that MVP is added to the [MVP page] + +### Xth: (1 working day before the 22nd) + +- [ ] Merge `x-y-stable` into `x-y-stable-ee` +- [ ] Create CE and EE release candidates - [ ] Create Omnibus tags and build packages for the latest release candidates -- [ ] Update GitLab.com with the latest RC (#LINK) -- [ ] Update ci.gitLab.com with the latest RC (#LINK) +- [ ] Update GitLab.com with the latest RC -22nd before 12AM CET: +### 22nd before 1200 CET: -Release before 12AM CET / 3AM PST, to make sure the majority of our users +Release before 1200 CET / 2AM PST, to make sure the majority of our users get the new version on the 22nd and there is sufficient time in the European workday to quickly fix any issues. -- [ ] Merge CE stable into EE stable (#LINK) -- [ ] Create the 'x.y.0' tag with the [release tools](https://dev.gitlab.org/gitlab/release-tools) (#LINK) +- [ ] Merge `x-y-stable` into `x-y-stable-ee` +- [ ] Create the 'x.y.0' tag with the [release tools](https://dev.gitlab.org/gitlab/release-tools) - [ ] Create the 'x.y.0' version on version.gitlab.com -- [ ] Try to do before 11AM CET: Create and push omnibus tags for x.y.0 (will auto-release the packages) (#LINK) -- [ ] Try to do before 12AM CET: Publish the release blog post (#LINK) -- [ ] Tweet about the release (blog post) (#LINK) -- [ ] Schedule a second tweet of the release announcement with the same text at 6PM CET / 9AM PST - +- [ ] Try to do before 1100 CET: Create and push Omnibus tags for x.y.0 (will auto-release the packages) +- [ ] Try to do before 1200 CET: Publish the release [blog post] +- [ ] Tweet about the release +- [ ] Schedule a second Tweet of the release announcement with the same text at 1800 CET / 8AM PST + +[Omnibus milestone]: LINK_TO_OMNIBUS_MILESTONE +[blog post]: LINK_TO_WIP_BLOG_POST +[MVP page]: https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/source/mvp/index.html ``` - - - @@ -140,7 +159,7 @@ Please do not raise issues directly in this issue but link to issues that might The decision to create a patch release or not is with the release manager who is assigned to this issue. The release manager will comment here about the plans for patch releases. -Assign the issue to the release manager and at mention all members of gitlab core team. If there are any known bugs in the release add them immediately. +Assign the issue to the release manager and at mention all members of GitLab core team. If there are any known bugs in the release add them immediately. ## Tweet about RC1 @@ -223,4 +242,4 @@ Consider creating a post on Hacker News. ## Create a WIP blogpost for the next release -Create a WIP blogpost using [release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/release_blog_template.md).
\ No newline at end of file +Create a WIP blogpost using [release blog template](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/doc/release_blog_template.md). diff --git a/doc/release/patch.md b/doc/release/patch.md index 6aa11b283df..3022e375aca 100644 --- a/doc/release/patch.md +++ b/doc/release/patch.md @@ -1,21 +1,46 @@ # Things to do when doing a patch release -NOTE: This is a guide for GitLab developers. If you are trying to install GitLab see the latest stable [installation guide](install/installation.md) and if you are trying to upgrade, see the [upgrade guides](update). +NOTE: This is a guide for GitLab developers. If you are trying to install GitLab +see the latest stable [installation guide](install/installation.md) and if you +are trying to upgrade, see the [upgrade guides](update). ## When to do a patch release -Do a patch release when there is a critical regression that needs to be addresses before the next monthly release. - -Otherwise include it in the monthly release and note there was a regression fix in the release announcement. +Patch releases are done as-needed in order to fix regressions in the current +major release that cannot or should not wait until the next major release. +What's included and when to release is at the discretion of the release manager. ## Release Procedure +### Create a patch issue + +Create an issue in the GitLab CE project. Name it "Release x.y.z", tag it with +the `release` label, and assign it to the milestone of the corresponding major +release. + +Use the following template: + +``` +- Picked into respective `stable` branches: +- [ ] Merge `x-y-stable` into `x-y-stable-ee` +- [ ] release-tools: `x.y.z` +- gitlab-omnibus + - [ ] `x.y.z+ee.0` + - [ ] `x.y.z+ce.0` +- [ ] Deploy +- [ ] Add patch notice to [x.y regressions]() +- [ ] [Blog post]() +- [ ] [Tweet]() +- [ ] Add entry to version.gitlab.com +``` + +Update the issue with links to merge requests that need to be/have been picked +into the `stable` branches. + ### Preparation 1. Verify that the issue can be reproduced 1. Note in the 'GitLab X.X regressions' that you will create a patch -1. Create an issue on private GitLab development server -1. Name the issue "Release X.X.X CE and X.X.X EE", this will make searching easier 1. Fix the issue on a feature branch, do this on the private GitLab development server 1. If it is a security issue, then assign it to the release manager and apply a 'security' label 1. Consider creating and testing workarounds @@ -25,7 +50,6 @@ Otherwise include it in the monthly release and note there was a regression fix 1. For EE, update the CHANGELOG-EE if it is EE specific fix. Otherwise, merge the stable CE branch and add to CHANGELOG-EE "Merge community edition changes for version X.X.X" 1. Merge CE stable branch into EE stable branch - ### Bump version Get release tools @@ -54,4 +78,4 @@ bundle exec rake release["x.x.x"] 1. Note in the 'GitLab X.X regressions' issue that the patch was published (CE only) 1. Create the 'x.y.0' version on version.gitlab.com 1. [Create new AMIs](https://dev.gitlab.org/gitlab/AMI/blob/master/README.md) -1. Create a new patch release issue for the next potential release
\ No newline at end of file +1. Create a new patch release issue for the next potential release diff --git a/doc/release/security.md b/doc/release/security.md index 60bcfbb6da5..b1a62b333e6 100644 --- a/doc/release/security.md +++ b/doc/release/security.md @@ -8,7 +8,7 @@ Do a security release when there is a critical issue that needs to be addresses ## Security vulnerability disclosure -Please report suspected security vulnerabilities in private to <support@gitlab.com>, also see the [disclosure section on the GitLab.com website](http://about.gitlab.com/disclosure/). Please do NOT create publicly viewable issues for suspected security vulnerabilities. +Please report suspected security vulnerabilities in private to <support@gitlab.com>, also see the [disclosure section on the GitLab.com website](https://about.gitlab.com/disclosure/). Please do NOT create publicly viewable issues for suspected security vulnerabilities. ## Release Procedure @@ -25,7 +25,7 @@ Please report suspected security vulnerabilities in private to <support@gitlab.c 1. Send tweets about the release from `@gitlabhq` 1. Send out an email to [the community google mailing list](https://groups.google.com/forum/#!forum/gitlabhq) 1. Post a signed copy of our complete announcement to [oss-security](http://www.openwall.com/lists/oss-security/) and request a CVE number. CVE is only needed for bugs that allow someone to own the server (Remote Code Execution) or access to code of projects they are not a member of. -1. Add the security researcher to the [Security Researcher Acknowledgments list](http://about.gitlab.com/vulnerability-acknowledgements/) +1. Add the security researcher to the [Security Researcher Acknowledgments list](https://about.gitlab.com/vulnerability-acknowledgements/) 1. Thank the security researcher in an email for their cooperation 1. Update the blog post and the CHANGELOG when we receive the CVE number diff --git a/doc/ssh/README.md b/doc/ssh/README.md index b6b8000af4e..fe5b45dd432 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -78,11 +78,11 @@ Deploy keys can be shared between projects, you just need to add them to each pr ### Eclipse -How to add your ssh key to Eclipse: http://wiki.eclipse.org/EGit/User_Guide#Eclipse_SSH_Configuration +How to add your ssh key to Eclipse: https://wiki.eclipse.org/EGit/User_Guide#Eclipse_SSH_Configuration ## Tip: Non-default OpenSSH key file names or locations -If, for whatever reason, you decide to specify a non-default location and filename for your Gitlab SSH key pair, you must configure your SSH client to find your Gitlab SSH private key for connections to your Gitlab server (perhaps gitlab.com). For OpenSSH clients, this is handled in the `~/.ssh/config` file with a stanza similar to the following: +If, for whatever reason, you decide to specify a non-default location and filename for your GitLab SSH key pair, you must configure your SSH client to find your GitLab SSH private key for connections to your GitLab server (perhaps gitlab.com). For OpenSSH clients, this is handled in the `~/.ssh/config` file with a stanza similar to the following: ``` # @@ -97,7 +97,7 @@ User mygitlabusername Another example ``` # -# Our company's internal Gitlab server +# Our company's internal GitLab server # Host my-gitlab.company.com RSAAuthentication yes @@ -109,4 +109,4 @@ Note in the gitlab.com example above a username was specified to override the de Due to the wide variety of SSH clients and their very large number of configuration options, further explanation of these topics is beyond the scope of this document. Public SSH keys need to be unique, as they will bind to your account. Your SSH key is the only identifier you'll -have when pushing code via SSH. That's why it needs to uniquely map to a single user. +have when pushing code via SSH. That's why it needs to uniquely map to a single user.
\ No newline at end of file diff --git a/doc/system_hooks/system_hooks.md b/doc/system_hooks/system_hooks.md index b0e4613cdef..5cb05b13b3e 100644 --- a/doc/system_hooks/system_hooks.md +++ b/doc/system_hooks/system_hooks.md @@ -48,16 +48,17 @@ X-Gitlab-Event: System Hook ```json { - "created_at": "2012-07-21T07:30:56Z", - "event_name": "user_add_to_team", - "project_access": "Master", - "project_id": 74, - "project_name": "StoreCloud", - "project_path": "storecloud", - "user_email": "johnsmith@gmail.com", - "user_name": "John Smith", - "user_id": 41, - "project_visibility": "private", + "created_at": "2012-07-21T07:30:56Z", + "event_name": "user_add_to_team", + "project_access": "Master", + "project_id": 74, + "project_name": "StoreCloud", + "project_path": "storecloud", + "project_path_with_namespace": "jsmith/storecloud", + "user_email": "johnsmith@gmail.com", + "user_name": "John Smith", + "user_id": 41, + "project_visibility": "private", } ``` @@ -65,16 +66,17 @@ X-Gitlab-Event: System Hook ```json { - "created_at": "2012-07-21T07:30:56Z", - "event_name": "user_remove_from_team", - "project_access": "Master", - "project_id": 74, - "project_name": "StoreCloud", - "project_path": "storecloud", - "user_email": "johnsmith@gmail.com", - "user_name": "John Smith", - "user_id": 41, - "project_visibility": "private", + "created_at": "2012-07-21T07:30:56Z", + "event_name": "user_remove_from_team", + "project_access": "Master", + "project_id": 74, + "project_name": "StoreCloud", + "project_path": "storecloud", + "project_path_with_namespace": "jsmith/storecloud", + "user_email": "johnsmith@gmail.com", + "user_name": "John Smith", + "user_id": 41, + "project_visibility": "private", } ``` diff --git a/doc/update/6.x-or-7.x-to-7.14.md b/doc/update/6.x-or-7.x-to-7.14.md index b34fb12da6f..4516a102084 100644 --- a/doc/update/6.x-or-7.x-to-7.14.md +++ b/doc/update/6.x-or-7.x-to-7.14.md @@ -47,7 +47,7 @@ Download and compile Ruby: ```bash mkdir /tmp/ruby && cd /tmp/ruby -curl --progress http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.6.tar.gz | tar xz +curl --progress https://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.6.tar.gz | tar xz cd ruby-2.1.6 ./configure --disable-install-rdoc make diff --git a/doc/update/7.14-to-8.0.md b/doc/update/7.14-to-8.0.md index 0cb82fb718a..305017b7048 100644 --- a/doc/update/7.14-to-8.0.md +++ b/doc/update/7.14-to-8.0.md @@ -84,6 +84,7 @@ Now we download `gitlab-git-http-server` and install it in `/home/git/gitlab-git cd /home/git sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-git-http-server.git cd gitlab-git-http-server +sudo -u git -H git checkout 0.2.14 sudo -u git -H make ``` @@ -168,18 +169,28 @@ git diff origin/7-14-stable:lib/support/nginx/gitlab origin/8-0-stable:lib/suppo ``` If you are using Apache instead of NGINX please see the updated [Apache templates](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache). +Also note that because Apache does not support upstreams behind Unix sockets you will need to let gitlab-git-http-server listen on a TCP port. You can do this via [/etc/default/gitlab](https://gitlab.com/gitlab-org/gitlab-ce/blob/8-0-stable/lib/support/init.d/gitlab.default.example#L34). ### 9. Migrate GitLab CI to GitLab CE/EE Now, GitLab CE and EE has CI integrated. However, migrations don't happen automatically and you need to do it manually. Please follow the following guide [to migrate](../migrate_ci_to_ce/README.md) your GitLab CI instance to GitLab CE/EE. -### 10. Start application +### 10. Use Redis v2.4.0+ + +Previous versions of GitLab allowed Redis versions >= 2.0 to be used, but +Sidekiq jobs could fail due to lack of support for the SREM command. GitLab +8.0 now checks that Redis >= 2.4.0 is used. You can check your Redis version +with the following command: + + redis-cli info | grep redis_version + +### 11. Start application sudo service gitlab start sudo service nginx restart -### 11. Check application status +### 12. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/update/8.0-to-8.1.md b/doc/update/8.0-to-8.1.md new file mode 100644 index 00000000000..d57c0d0674d --- /dev/null +++ b/doc/update/8.0-to-8.1.md @@ -0,0 +1,171 @@ +# From 8.0 to 8.1 + +**NOTE:** GitLab 8.0 introduced several significant changes related to +installation and configuration which *are not duplicated here*. Be sure you're +already running a working version of 8.0 before proceeding with this guide. + +### 0. Double-check your Git version + +**This notice applies only to /usr/local/bin/git** + +If you compiled Git from source on your GitLab server then please double-check +that you are using a version that protects against CVE-2014-9390. For six +months after this vulnerability became known the GitLab installation guide +still contained instructions that would install the outdated, 'vulnerable' Git +version 2.1.2. + +Run the following command to get your current Git version: + +```sh +/usr/local/bin/git --version +``` + +If you see 'No such file or directory' then you did not install Git according +to the outdated instructions from the GitLab installation guide and you can go +to the next step 'Stop server' below. + +If you see a version string then it should be v1.8.5.6, v1.9.5, v2.0.5, v2.1.4, +v2.2.1 or newer. You can use the [instructions in the GitLab source +installation +guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies) +to install a newer version of Git. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-1-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-1-stable-ee +``` + +### 4. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch +sudo -u git -H git checkout v2.6.5 +``` + +### 5. Update gitlab-git-http-server + +```bash +cd /home/git/gitlab-git-http-server +sudo -u git -H git fetch origin +sudo -u git -H git checkout 0.3.0 +sudo -u git -H make +``` + +### 6. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production + +# Update init.d script +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +### 7. Update configuration files + +#### New configuration options for `gitlab.yml` + +There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +git diff origin/8-0-stable:config/gitlab.yml.example origin/8-1-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +View changes between the previous recommended Nginx configuration and the +current one: + +```sh +# For HTTPS configurations +git diff origin/8-0-stable:lib/support/nginx/gitlab-ssl origin/8-1-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-0-stable:lib/support/nginx/gitlab origin/8-1-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-git-http-server listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-1-stable/lib/support/init.d/gitlab.default.example#L34 + +### 8. Start application + + sudo service gitlab start + sudo service nginx restart + +### 9. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.0) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 7.14 to 8.0](7.14-to-8.0.md), except for the database migration +(The backup is already migrated to the previous version) + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +## Troubleshooting + +### "You appear to have cloned an empty repository." + +See the [7.14 to 8.0 update guide](7.14-to-8.0.md#troubleshooting). diff --git a/doc/update/8.1-to-8.2.md b/doc/update/8.1-to-8.2.md new file mode 100644 index 00000000000..7b228d6a22f --- /dev/null +++ b/doc/update/8.1-to-8.2.md @@ -0,0 +1,189 @@ +# From 8.1 to 8.2 + +**NOTE:** GitLab 8.0 introduced several significant changes related to +installation and configuration which *are not duplicated here*. Be sure you're +already running a working version of at least 8.0 before proceeding with this +guide. + +### 0. Double-check your Git version + +**This notice applies only to /usr/local/bin/git** + +If you compiled Git from source on your GitLab server then please double-check +that you are using a version that protects against CVE-2014-9390. For six +months after this vulnerability became known the GitLab installation guide +still contained instructions that would install the outdated, 'vulnerable' Git +version 2.1.2. + +Run the following command to get your current Git version: + +```sh +/usr/local/bin/git --version +``` + +If you see 'No such file or directory' then you did not install Git according +to the outdated instructions from the GitLab installation guide and you can go +to the next step 'Stop server' below. + +If you see a version string then it should be v1.8.5.6, v1.9.5, v2.0.5, v2.1.4, +v2.2.1 or newer. You can use the [instructions in the GitLab source +installation +guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md#1-packages-dependencies) +to install a newer version of Git. + +### 1. Stop server + + sudo service gitlab stop + +### 2. Backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +### 3. Get latest code + +```bash +sudo -u git -H git fetch --all +sudo -u git -H git checkout -- db/schema.rb # local changes will be restored automatically +``` + +For GitLab Community Edition: + +```bash +sudo -u git -H git checkout 8-2-stable +``` + +OR + +For GitLab Enterprise Edition: + +```bash +sudo -u git -H git checkout 8-2-stable-ee +``` + +### 4. Update gitlab-shell + +```bash +cd /home/git/gitlab-shell +sudo -u git -H git fetch +sudo -u git -H git checkout v2.6.8 +``` + +### 5. Replace gitlab-git-http-server with gitlab-workhorse + +Install and compile gitlab-workhorse. This requires [Go +1.5](https://golang.org/dl) which should already be on your system +from GitLab 8.1. + +```bash +cd /home/git +sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-workhorse.git +cd gitlab-workhorse +sudo -u git -H git checkout 0.4.2 +sudo -u git -H make +``` + +Update the GitLab init script and 'default' file. + +``` +cd /home/git/gitlab +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +test -e /etc/default/gitlab && \ + sudo sed -i.pre-8.2 's/^\([^=]*\)gitlab_git_http_server/\1gitlab_workhorse/' /etc/default/gitlab +``` + +Make sure that you also update your **NGINX configuration** to use +the new gitlab-workhorse.socket file. + +### 6. Install libs, migrations, etc. + +```bash +cd /home/git/gitlab + +# MySQL installations (note: the line below states '--without postgres') +sudo -u git -H bundle install --without postgres development test --deployment + +# PostgreSQL installations (note: the line below states '--without mysql') +sudo -u git -H bundle install --without mysql development test --deployment + +# Run database migrations +sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production + +# Clean up assets and cache +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production + +# Update init.d script +sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab +``` + +### 7. Update configuration files + +#### New configuration options for `gitlab.yml` + +There are new configuration options available for [`gitlab.yml`](config/gitlab.yml.example). View them with the command below and apply them manually to your current `gitlab.yml`: + +```sh +git diff origin/8-1-stable:config/gitlab.yml.example origin/8-2-stable:config/gitlab.yml.example +``` + +#### Nginx configuration + +View changes between the previous recommended Nginx configuration and the +current one: + +```sh +# For HTTPS configurations +git diff origin/8-1-stable:lib/support/nginx/gitlab-ssl origin/8-2-stable:lib/support/nginx/gitlab-ssl + +# For HTTP configurations +git diff origin/8-1-stable:lib/support/nginx/gitlab origin/8-2-stable:lib/support/nginx/gitlab +``` + +If you are using Apache instead of NGINX please see the updated [Apache templates]. +Also note that because Apache does not support upstreams behind Unix sockets you +will need to let gitlab-git-http-server listen on a TCP port. You can do this +via [/etc/default/gitlab]. + +[Apache templates]: https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache +[/etc/default/gitlab]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-2-stable/lib/support/init.d/gitlab.default.example#L34 + +### 8. Start application + + sudo service gitlab start + sudo service nginx restart + +### 9. Check application status + +Check if GitLab and its environment are configured correctly: + + sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production + +To make sure you didn't miss anything run a more thorough check: + + sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production + +If all items are green, then congratulations, the upgrade is complete! + +## Things went south? Revert to previous version (8.1) + +### 1. Revert the code to the previous version + +Follow the [upgrade guide from 8.0 to 8.1](8.0-to-8.1.md), except for the +database migration (the backup is already migrated to the previous version). + +### 2. Restore from the backup + +```bash +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +If you have more than one backup `*.tar` file(s) please add `BACKUP=timestamp_of_backup` to the command above. + +## Troubleshooting + +### "You appear to have cloned an empty repository." + +See the [7.14 to 8.0 update guide](7.14-to-8.0.md#troubleshooting). diff --git a/doc/update/mysql_to_postgresql.md b/doc/update/mysql_to_postgresql.md index a596ea38456..a7de5648c0e 100644 --- a/doc/update/mysql_to_postgresql.md +++ b/doc/update/mysql_to_postgresql.md @@ -60,6 +60,9 @@ sudo -u git -H python mysql-postgresql-converter/db_converter.py gitlabhq_produc sudo -u git -H ed -s db/database.sql < mysql-postgresql-converter/move_drop_indexes.ed # Compress database backup +# Warning: If you have Gitlab 7.12.0 or older skip this step and import the database.sql directly into the backup with: +# sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql +# The compressed databasedump is not supported at 7.12.0 and older. sudo -u git -H gzip db/database.sql # Replace the MySQL dump in TIMESTAMP_gitlab_backup.tar. @@ -71,4 +74,5 @@ sudo -u git -H tar rf TIMESTAMP_gitlab_backup.tar db/database.sql.gz # Done! TIMESTAMP_gitlab_backup.tar can now be restored into a Postgres GitLab # installation. +# See https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/raketasks/backup_restore.md for more information about backups. ``` diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index a66a863f6c4..957354decb7 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -6,7 +6,8 @@ For example from 7.14.0 to 7.14.3, also see the [semantic versioning specificati ### 0. Backup It's useful to make a backup just in case things go south: -(With MySQL, this may require granting "LOCK TABLES" privileges to the GitLab user on the database version) +(With MySQL, this may require granting "LOCK TABLES" privileges to the GitLab +user on the database version) ```bash cd /home/git/gitlab @@ -15,16 +16,22 @@ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production ### 1. Stop server - sudo service gitlab stop +```bash +sudo service gitlab stop +``` ### 2. Get latest code for the stable branch +In the commands below, replace `LATEST_TAG` with the latest GitLab tag you want +to update to, for example `v8.0.3`. Use `git tag -l 'v*.[0-9]' --sort='v:refname'` +to see a list of all tags. Make sure to update patch versions only (check your +current version with `cat VERSION`). + ```bash cd /home/git/gitlab sudo -u git -H git fetch --all sudo -u git -H git checkout -- Gemfile.lock db/schema.rb -LATEST_TAG=$(git describe --tags `git rev-list --tags --max-count=1`) -sudo -u git -H git checkout $LATEST_TAG -b $LATEST_TAG +sudo -u git -H git checkout LATEST_TAG -b LATEST_TAG ``` ### 3. Update gitlab-shell to the corresponding version @@ -35,36 +42,48 @@ sudo -u git -H git fetch sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` -b v`cat /home/git/gitlab/GITLAB_SHELL_VERSION` ``` -### 4. Install libs, migrations, etc. +### 4. Update gitlab-workhorse to the corresponding version + +```bash +cd /home/git/gitlab-workhorse +sudo -u git -H git fetch +sudo -u git -H git checkout v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` -b v`cat /home/git/gitlab/GITLAB_WORKHORSE_VERSION` +``` + +### 5. Install libs, migrations, etc. ```bash cd /home/git/gitlab -#PostgreSQL +# PostgreSQL sudo -u git -H bundle install --without development test mysql --deployment # MySQL sudo -u git -H bundle install --without development test postgres --deployment sudo -u git -H bundle exec rake db:migrate RAILS_ENV=production -sudo -u git -H bundle exec rake assets:clean RAILS_ENV=production -sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production -sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS_ENV=production ``` -### 5. Start application +### 6. Start application - sudo service gitlab start - sudo service nginx restart +```bash +sudo service gitlab start +sudo service nginx restart +``` -### 6. Check application status +### 7. Check application status Check if GitLab and its environment are configured correctly: - sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +```bash +sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production +``` To make sure you didn't miss anything run a more thorough check with: - sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +```bash +sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production +``` If all items are green, then congratulations upgrade complete! diff --git a/doc/web_hooks/web_hooks.md b/doc/web_hooks/web_hooks.md index c185ccfcac3..03746dd9df3 100644 --- a/doc/web_hooks/web_hooks.md +++ b/doc/web_hooks/web_hooks.md @@ -57,6 +57,9 @@ X-Gitlab-Event: Push Hook "name": "Jordi Mallach", "email": "jordi@softcatala.org" } + "added": ["CHANGELOG"], + "modified": ["app/controller/application.rb"], + "removed": [] }, { "id": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7", @@ -66,10 +69,14 @@ X-Gitlab-Event: Push Hook "author": { "name": "GitLab dev user", "email": "gitlabdev@dv6700.(none)" - } + }, + "added": ["CHANGELOG"], + "modified": ["app/controller/application.rb"], + "removed": [] } ], "total_commits_count": 4 + } ``` @@ -314,7 +321,8 @@ X-Gitlab-Event: Note Hook "name": "John Smith", "email": "john@example.com" } - } + }, + "work_in_progress": false } } ``` @@ -500,6 +508,7 @@ X-Gitlab-Event: Merge Request Hook "email": "gitlabdev@dv6700.(none)" } }, + "work_in_progress": false, "url": "http://example.com/diaspora/merge_requests/1", "action": "open" } @@ -537,4 +546,4 @@ When you press 'Test Hook' in GitLab, you should see something like this in the {"before":"077a85dd266e6f3573ef7e9ef8ce3343ad659c4e","after":"95cd4a99e93bc4bbabacfa2cd10e6725b1403c60",<SNIP>} example.com - - [14/May/2014:07:45:26 EDT] "POST / HTTP/1.1" 200 0 - -> / -```
\ No newline at end of file +``` diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 5b8d72dfd34..a6b4d951188 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -13,5 +13,8 @@ - [Project users](add-user/add-user.md) - [Protected branches](protected_branches.md) - [Web Editor](web_editor.md) +- [Releases](releases.md) +- [Milestones](milestones.md) - [Merge Requests](merge_requests.md) - ["Work In Progress" Merge Requests](wip_merge_requests.md) +- [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) diff --git a/doc/workflow/award_emoji.png b/doc/workflow/award_emoji.png Binary files differnew file mode 100644 index 00000000000..fb26ee04393 --- /dev/null +++ b/doc/workflow/award_emoji.png diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index f608674faf6..8965e5b3654 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -7,7 +7,7 @@ This allows a wide variety of branching strategies and workflows. Almost all of these are an improvement over the methods used before git. But many organizations end up with a workflow that is not clearly defined, overly complex or not integrated with issue tracking systems. Therefore we propose the GitLab flow as clearly defined set of best practices. -It combines [feature driven development](http://en.wikipedia.org/wiki/Feature-driven_development) and [feature branches](http://martinfowler.com/bliki/FeatureBranch.html) with issue tracking. +It combines [feature driven development](https://en.wikipedia.org/wiki/Feature-driven_development) and [feature branches](http://martinfowler.com/bliki/FeatureBranch.html) with issue tracking. Organizations coming to git from other version control systems frequently find it hard to develop an effective workflow. This article describes the GitLab flow that integrates the git workflow with an issue tracking system. @@ -26,7 +26,7 @@ After getting used to these three steps the branching model becomes the challeng Since many organizations new to git have no conventions how to work with it, it can quickly become a mess. The biggest problem they run into is that many long running branches that each contain part of the changes are around. People have a hard time figuring out which branch they should develop on or deploy to production. -Frequently the reaction to this problem is to adopt a standardized pattern such as [git flow](http://nvie.com/posts/a-successful-git-branching-model/) and [GitHub flow](http://scottchacon.com/2011/08/31/github-flow.html) +Frequently the reaction to this problem is to adopt a standardized pattern such as [git flow](http://nvie.com/posts/a-successful-git-branching-model/) and [GitHub flow](http://scottchacon.com/2011/08/31/github-flow.html). We think there is still room for improvement and will detail a set of practices we call GitLab flow. ## Git flow and its problems @@ -91,7 +91,7 @@ This workflow where commits only flow downstream ensures that everything has bee If you need to cherry-pick a commit with a hotfix it is common to develop it on a feature branch and merge it into master with a merge request, do not delete the feature branch. If master is good to go (it should be if you a practicing [continuous delivery](http://martinfowler.com/bliki/ContinuousDelivery.html)) you then merge it to the other branches. If this is not possible because more manual testing is required you can send merge requests from the feature branch to the downstream branches. -An 'extreme' version of environment branches are setting up an environment for each feature branch as done by [Teatro](http://teatro.io/). +An 'extreme' version of environment branches are setting up an environment for each feature branch as done by [Teatro](https://teatro.io/). ## Release branches with GitLab flow @@ -104,7 +104,7 @@ By branching as late as possible you minimize the time you have to apply bug fix After a release branch is announced, only serious bug fixes are included in the release branch. If possible these bug fixes are first merged into master and then cherry-picked into the release branch. This way you can't forget to cherry-pick them into master and encounter the same bug on subsequent releases. -This is called an 'upstream first' policy that is also practiced by [Google](http://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first) and [Red Hat](http://www.redhat.com/about/news/archive/2013/5/a-community-for-using-openstack-with-red-hat-rdo). +This is called an 'upstream first' policy that is also practiced by [Google](https://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first) and [Red Hat](https://www.redhat.com/about/news/archive/2013/5/a-community-for-using-openstack-with-red-hat-rdo). Every time a bug-fix is included in a release branch the patch version is raised (to comply with [Semantic Versioning](http://semver.org/)) by setting a new tag. Some projects also have a stable branch that points to the same commit as the latest released branch. In this flow it is not common to have a production branch (or git flow master branch). @@ -200,7 +200,7 @@ And to understand a change in context one can always look at the merge commit th After you merge multiple commits from a feature branch into the master branch this is harder to undo. If you would have squashed all the commits into one you could have just reverted this commit but as we indicated you should not rebase commits after they are pushed. -Fortunately [reverting a merge made some time ago](http://git-scm.com/blog/2010/03/02/undoing-merges.html) can be done with git. +Fortunately [reverting a merge made some time ago](https://git-scm.com/blog/2010/03/02/undoing-merges.html) can be done with git. This however, requires having specific merge commits for the commits your want to revert. If you revert a merge and you change your mind, revert the revert instead of merging again since git will not allow you to merge the code again otherwise. @@ -215,7 +215,7 @@ With git you can also rebase your feature branch commits to order them after the This prevents creating a merge commit when merging master into your feature branch and creates a nice linear history. However, just like with squashing you should never rebase commits you have pushed to a remote server. This makes it impossible to rebase work in progress that you already shared with your team which is something we recommend. -When using rebase to keep your feature branch updated you [need to resolve similar conflicts again and again](http://blogs.atlassian.com/2013/10/git-team-workflows-merge-or-rebase/). +When using rebase to keep your feature branch updated you [need to resolve similar conflicts again and again](https://blogs.atlassian.com/2013/10/git-team-workflows-merge-or-rebase/). You can reuse recorded resolutions (rerere) sometimes, but without rebasing you only have to solve the conflicts one time and you’re set. There has to be a better way to avoid many merge commits. @@ -244,13 +244,12 @@ Developing software happen in small messy steps and it is OK to have your histor You can use tools to view the network graphs of commits and understand the messy history that created your code. If you rebase code the history is incorrect, and there is no way for tools to remedy this because they can't deal with changing commit identifiers. -## Voting on merge requests +## Award emojis on issues and merge requests - + -It is common to voice approval or disapproval by using +1 or -1 emoticons. -In GitLab the +1 and -1 are aggregated and shown at the top of the merge request. -As a rule of thumb anything that doesn't have two times more +1's than -1's is suspect and should not be merged yet. +It is common to voice approval or disapproval by using +1 or -1. In GitLab you +can use emojis to give a virtual high five on issues and merge requests. ## Pushing and removing branches @@ -307,7 +306,7 @@ When initiating a feature branch, always start with an up to date master to bran If you know beforehand that your work absolutely depends on another branch you can also branch from there. If you need to merge in another branch after starting explain the reason in the merge commit. If you have not pushed your commits to a shared location yet you can also rebase on master or another feature branch. -Do not merge in upstream if your code will work and merge cleanly without doing so, Linus even says that [you should never merge in upstream at random points, only at major releases](http://lwn.net/Articles/328438/). +Do not merge in upstream if your code will work and merge cleanly without doing so, Linus even says that [you should never merge in upstream at random points, only at major releases](https://lwn.net/Articles/328438/). Merging only when needed prevents creating merge commits in your feature branch that later end up littering the master history. ### References diff --git a/doc/workflow/importing/import_projects_from_gitlab_com.md b/doc/workflow/importing/import_projects_from_gitlab_com.md index f4c4e955d46..1117db98e7e 100644 --- a/doc/workflow/importing/import_projects_from_gitlab_com.md +++ b/doc/workflow/importing/import_projects_from_gitlab_com.md @@ -2,12 +2,12 @@ You can import your existing GitLab.com projects to your GitLab instance. But keep in mind that it is possible only if GitLab support is enabled on your GitLab instance. -You can read more about Gitlab support [here](http://doc.gitlab.com/ce/integration/gitlab.html) +You can read more about GitLab support [here](http://doc.gitlab.com/ce/integration/gitlab.html) To get to the importer page you need to go to "New project" page.  -Click on the "Import projects from Gitlab.com" link and you will be redirected to GitLab.com +Click on the "Import projects from GitLab.com" link and you will be redirected to GitLab.com for permission to access your projects. After accepting, you'll be automatically redirected to the importer. diff --git a/doc/workflow/importing/migrating_from_svn.md b/doc/workflow/importing/migrating_from_svn.md index 485db4834e9..1938ccd0c26 100644 --- a/doc/workflow/importing/migrating_from_svn.md +++ b/doc/workflow/importing/migrating_from_svn.md @@ -6,9 +6,9 @@ Git is a distributed version control system. There are some major differences between the two, for more information consult your favorite search engine. Git has tools for migrating SVN repositories to git, namely `git svn`. You can read more about this at -[git documentation pages](http://git-scm.com/book/en/Git-and-Other-Systems-Git-and-Subversion). +[git documentation pages](https://git-scm.com/book/en/Git-and-Other-Systems-Git-and-Subversion). -Apart from the [official git documentation](http://git-scm.com/book/en/Git-and-Other-Systems-Migrating-to-Git) there is also +Apart from the [official git documentation](https://git-scm.com/book/en/Git-and-Other-Systems-Migrating-to-Git) there is also user created step by step guide for migrating from SVN to GitLab. [Benjamin New](https://github.com/leftclickben) wrote [a guide that shows how to do a migration](https://gist.github.com/leftclickben/322b7a3042cbe97ed2af). Mirrors can be found [here](https://gitlab.com/snippets/2168) and [here](https://gist.github.com/maxlazio/f1b593b0d00aa966e9ca). diff --git a/doc/workflow/lfs/lfs_administration.md b/doc/workflow/lfs/lfs_administration.md new file mode 100644 index 00000000000..5076b2697a3 --- /dev/null +++ b/doc/workflow/lfs/lfs_administration.md @@ -0,0 +1,41 @@ +# GitLab Git LFS Administration + +Documentation on how to use Git LFS are under [Managing large binary files with Git LFS doc](manage_large_binaries_with_git_lfs.md). + +## Requirements + +* Git LFS is supported in GitLab starting with version 8.2. +* Users need to install [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up. + +## Configuration + +Git LFS objects can be large in size. By default, they are stored on the server GitLab is installed on. + +There are two configuration options to help GitLab server administrators: + +* Enabling/disabling Git LFS support +* Changing the location of LFS object storage + +### Omnibus packages + +In `/etc/gitlab/gitlab.rb`: + +```ruby +gitlab_rails['lfs_enabled'] = false +gitlab_rails['lfs_storage_path'] = "/mnt/storage/lfs-objects" +``` + +### Installations from source + +In `config/gitlab.yml`: + +```yaml + lfs: + enabled: false + storage_path: /mnt/storage/lfs-objects +``` + +## Known limitations + +* Currently, storing GitLab Git LFS objects on a non-local storage (like S3 buckets) is not supported +* Currently, removing LFS objects from GitLab Git LFS storage is not supported diff --git a/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md new file mode 100644 index 00000000000..b59e92cb317 --- /dev/null +++ b/doc/workflow/lfs/manage_large_binaries_with_git_lfs.md @@ -0,0 +1,126 @@ +# Git LFS + +Managing large files such as audio, video and graphics files has always been one of the shortcomings of Git. +The general recommendation is to not have Git repositories larger than 1GB to preserve performance. + +GitLab already supports [managing large files with git annex](http://doc.gitlab.com/ee/workflow/git_annex.html) (EE only), however in certain +environments it is not always convenient to use different commands to differentiate between the large files and regular ones. + +Git LFS makes this simpler for the end user by removing the requirement to learn new commands. + +## How it works + +Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication to authorize client requests. +Once the request is authorized, Git LFS client receives instructions from where to fetch or where to push the large file. + +## GitLab server configuration + +Documentation for GitLab instance administrators is under [LFS administration doc](lfs_administration.md). + +## Requirements + +* Git LFS is supported in GitLab starting with version 8.2 +* [Git LFS client](https://git-lfs.github.com) version 1.0.1 and up + +## Known limitations + +* Git LFS v1 original API is not supported since it was deprecated early in LFS development +* When SSH is set as a remote, Git LFS objects still go through HTTPS +* Any Git LFS request will ask for HTTPS credentials to be provided so good Git credentials store is recommended +* Git LFS always assumes HTTPS so if you have GitLab server on HTTP you will have to add the URL to Git config manually (see #troubleshooting) + +## Using Git LFS + +Lets take a look at the workflow when you need to check large files into your Git repository with Git LFS: +For example, if you want to upload a very large file and check it into your Git repository: + +```bash +git clone git@gitlab.example.com:group/project.git +git lfs init # initialize the Git LFS project project +git lfs track "*.iso" # select the file extensions that you want to treat as large files +``` + +Once a certain file extension is marked for tracking as a LFS object you can use Git as usual without having to redo the command to track a file with the same extension: + +```bash +cp ~/tmp/debian.iso ./ # copy a large file into the current directory +git add . # add the large file to the project +git commit -am "Added Debian iso" # commit the file meta data +git push origin master # sync the git repo and large file to the GitLab server +``` + +Cloning the repository works the same as before. Git automatically detects the LFS-tracked files and clones them via HTTP. If you performed the git clone command with a SSH URL, you have to enter your GitLab credentials for HTTP authentication. + +```bash +git clone git@gitlab.example.com:group/project.git +``` + +If you already cloned the repository and you want to get the latest LFS object that are on the remote repository, eg. from branch `master`: + +```bash +git lfs fetch master +``` + +## Troubleshooting + +### error: Repository or object not found + +There are a couple of reasons why this error can occur: + +* You don't have permissions to access certain LFS object + +Check if you have permissions to push to the project or fetch from the project. + +* Project is not allowed to access the LFS object + +LFS object you are trying to push to the project or fetch from the project is not available to the project anymore. +Probably the object was removed from the server. + +* Local git repository is using deprecated LFS API + +### Invalid status for <url> : 501 + +Git LFS will log the failures into a log file. +To view this log file, while in project directory: + +```bash +git lfs logs last +``` + +If the status `error 501` is shown, it is because: + +* Git LFS support is not enabled on the GitLab server. Check with your GitLab administrator why Git LFS is not enabled on the server. See [LFS administration documentation](lfs_administration.md) for instructions on how to enable LFS support. + +* Git LFS client version is not supported by GitLab server. Check your Git LFS version with `git lfs version`. Check the Git config of the project for traces of deprecated API with `git lfs -l`. If `batch = false` is set in the config, remove the line and try to update your Git LFS client. Only version 1.0.1 and newer are supported. + +### getsockopt: connection refused + +If you push a LFS object to a project and you receive an error similar to: `Post <URL>/info/lfs/objects/batch: dial tcp IP: getsockopt: connection refused`, +the LFS client is trying to reach GitLab through HTTPS. However, your GitLab instance is being served on HTTP. + +This behaviour is caused by Git LFS using HTTPS connections by default when a `lfsurl` is not set in the Git config. + +To prevent this from happening, set the lfs url in project Git config: + +```bash + +git config --add lfs.url "http://gitlab.example.com/group/project.git/info/lfs/objects/batch" +``` + +### Credentials are always required when pushing an object + +Given that Git LFS uses HTTP Basic Authentication to authenticate the user pushing the LFS object on every push for every object, user HTTPS credentials are required. + +By default, Git has support for remembering the credentials for each repository you use. This is described in [Git credentials man pages](https://git-scm.com/docs/gitcredentials). + +For example, you can tell Git to remember the password for a period of time in which you expect to push the objects: + +```bash +git config --global credential.helper 'cache --timeout=3600' +``` + +This will remember the credentials for an hour after which Git operations will require re-authentication. + +If you are using OS X you can use `osxkeychain` to store and encrypt your credentials. For Windows, you can use `wincred` or Microsoft's [Git Credential Manager for Windows](https://github.com/Microsoft/Git-Credential-Manager-for-Windows/releases). + +More details about various methods of storing the user credentials can be found on [Git Credential Storage documentation](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage).
\ No newline at end of file diff --git a/doc/workflow/merge_requests.md b/doc/workflow/merge_requests.md index 751e19da7f1..6d57b5d98cd 100644 --- a/doc/workflow/merge_requests.md +++ b/doc/workflow/merge_requests.md @@ -38,3 +38,15 @@ To check out a particular merge request: ``` $ git checkout origin/merge-requests/1 ``` + +## Ignore whitespace changes in Merge Request diff view + + + +It you add `w=1` option to URL, you can see diff without whitespace changes. + + + +It is also working on commits compare view. + + diff --git a/doc/workflow/merge_requests/commit_compare.png b/doc/workflow/merge_requests/commit_compare.png Binary files differnew file mode 100644 index 00000000000..46b3a56a59b --- /dev/null +++ b/doc/workflow/merge_requests/commit_compare.png diff --git a/doc/workflow/merge_requests/merge_request_diff.png b/doc/workflow/merge_requests/merge_request_diff.png Binary files differnew file mode 100644 index 00000000000..ed08ae91bec --- /dev/null +++ b/doc/workflow/merge_requests/merge_request_diff.png diff --git a/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png b/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png Binary files differnew file mode 100644 index 00000000000..67d67a64d12 --- /dev/null +++ b/doc/workflow/merge_requests/merge_request_diff_without_whitespace.png diff --git a/doc/workflow/milestones.md b/doc/workflow/milestones.md new file mode 100644 index 00000000000..dff36899aec --- /dev/null +++ b/doc/workflow/milestones.md @@ -0,0 +1,13 @@ +# Milestones + +Milestones allow you to organize issues and merge requests into a cohesive group, optionally setting a due date. +A common use is keeping track of an upcoming software version. Milestones are created per-project. + + + +## Groups and milestones + +You can create a milestone for several projects in the same group simultaneously. +On the group's milestones page, you will be able to see the status of that milestone across all of the selected projects. + + diff --git a/doc/workflow/milestones/form.png b/doc/workflow/milestones/form.png Binary files differnew file mode 100644 index 00000000000..de44c1ffc1a --- /dev/null +++ b/doc/workflow/milestones/form.png diff --git a/doc/workflow/milestones/group_form.png b/doc/workflow/milestones/group_form.png Binary files differnew file mode 100644 index 00000000000..38862dcca68 --- /dev/null +++ b/doc/workflow/milestones/group_form.png diff --git a/doc/workflow/releases.md b/doc/workflow/releases.md new file mode 100644 index 00000000000..6176784fc57 --- /dev/null +++ b/doc/workflow/releases.md @@ -0,0 +1,20 @@ +# Releases + +You can turn any git tag into a release, by adding a note to it. +Release notes behave like any other markdown form in GitLab so you can write text and drag-n-drop files to it. +Release notes are stored in the database of GitLab. + +There are several ways to add release notes: + +* In the interface, when you create a new git tag with GitLab +* In the interface, by adding a note to an existing git tag +* with the GitLab API + +## New tag page with release notes text area + + + +## Tags page with button to add or edit release notes for existing git tag + + + diff --git a/doc/workflow/releases/new_tag.png b/doc/workflow/releases/new_tag.png Binary files differnew file mode 100644 index 00000000000..e2b64bfe17f --- /dev/null +++ b/doc/workflow/releases/new_tag.png diff --git a/doc/workflow/releases/tags.png b/doc/workflow/releases/tags.png Binary files differnew file mode 100644 index 00000000000..aca91906c68 --- /dev/null +++ b/doc/workflow/releases/tags.png diff --git a/doc/workflow/voting_slider.png b/doc/workflow/voting_slider.png Binary files differdeleted file mode 100644 index 4c660ef9593..00000000000 --- a/doc/workflow/voting_slider.png +++ /dev/null |
