diff options
Diffstat (limited to 'doc/development')
24 files changed, 576 insertions, 175 deletions
diff --git a/doc/development/README.md b/doc/development/README.md index efe37b8ba0b..43d3865da0e 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -7,7 +7,7 @@ description: 'Learn how to contribute to GitLab.' ## Get started! -- Setup GitLab's development environment with [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) +- Set up GitLab's development environment with [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) - [GitLab contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md) - [Architecture](architecture.md) of GitLab - [Rake tasks](rake_tasks.md) for development @@ -53,11 +53,14 @@ description: 'Learn how to contribute to GitLab.' ## Performance guides -- [Instrumentation](instrumentation.md) -- [Performance guidelines](performance.md) +- [Instrumentation](instrumentation.md) for Ruby code running in production + environments +- [Performance guidelines](performance.md) for writing code, benchmarks, and + certain patterns to avoid - [Merge request performance guidelines](merge_request_performance_guidelines.md) for ensuring merge requests do not negatively impact GitLab performance -- [Profiling](profiling.md) for profiling a URL +- [Profiling](profiling.md) a URL, measuring performance using Sherlock, or + tracking down N+1 queries using Bullet ## Database guides @@ -91,6 +94,7 @@ description: 'Learn how to contribute to GitLab.' - [Verifying database capabilities](verifying_database_capabilities.md) - [Database Debugging and Troubleshooting](database_debugging.md) - [Query Count Limits](query_count_limits.md) +- [Database helper modules](database_helpers.md) ## Testing guides diff --git a/doc/development/automatic_ce_ee_merge.md b/doc/development/automatic_ce_ee_merge.md index f6509b5c1dd..9dd78806a12 100644 --- a/doc/development/automatic_ce_ee_merge.md +++ b/doc/development/automatic_ce_ee_merge.md @@ -63,7 +63,7 @@ EE version of your CE merge request. For each commit (except on `master`), the `ee_compat_check` CI job tries to detect if the current branch's changes will conflict during the CE->EE merge. -The job reports what files are conflicting and how to setup a merge request +The job reports what files are conflicting and how to set up a merge request against EE. #### How the job works diff --git a/doc/development/contributing/community_roles.md b/doc/development/contributing/community_roles.md new file mode 100644 index 00000000000..c508969f7f4 --- /dev/null +++ b/doc/development/contributing/community_roles.md @@ -0,0 +1,12 @@ +### Community members & roles + +GitLab community members and their privileges/responsibilities. + +| Roles | Responsibilities | Requirements | +|-------|------------------|--------------| +| Maintainer | Accepts merge requests on several GitLab projects | Added to the [team page](https://about.gitlab.com/team/). An expert on code reviews and knows the product/code base | +| Reviewer | Performs code reviews on MRs | Added to the [team page](https://about.gitlab.com/team/) | +| Developer |Has access to GitLab internal infrastructure & issues (e.g. HR-related) | GitLab employee or a Core Team member (with an NDA) | +| Contributor | Can make contributions to all GitLab public projects | Have a GitLab.com account | + +[List of current reviewers/maintainers](https://about.gitlab.com/handbook/engineering/projects/#gitlab-ce)
\ No newline at end of file diff --git a/doc/development/contributing/design.md b/doc/development/contributing/design.md index 45fe8c26591..be7891061f9 100644 --- a/doc/development/contributing/design.md +++ b/doc/development/contributing/design.md @@ -1,13 +1,4 @@ -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - -- [Implement design & UI elements](#implement-design--ui-elements) -- [Style guides](#style-guides) - -<!-- END doctoc generated TOC please keep comment here to allow auto update --> - -## Implement design & UI elements +# Implement design & UI elements For guidance on UX implementation at GitLab, please refer to our [Design System](https://design.gitlab.com/). @@ -34,27 +25,27 @@ In order to complete a product discovery issue in a release, you must complete t ## Style guides -1. [Ruby](https://github.com/bbatsov/ruby-style-guide). - Important sections include [Source Code Layout][rss-source] and - [Naming][rss-naming]. Use: - - multi-line method chaining style **Option A**: dot `.` on the second line - - string literal quoting style **Option A**: single quoted by default -1. [Rails](https://github.com/bbatsov/rails-style-guide) -1. [Newlines styleguide][newlines-styleguide] -1. [Testing][testing] -1. [JavaScript styleguide][js-styleguide] -1. [SCSS styleguide][scss-styleguide] -1. [Shell commands](../shell_commands.md) created by GitLab - contributors to enhance security -1. [Database Migrations](../migration_style_guide.md) -1. [Markdown](http://www.cirosantilli.com/markdown-styleguide) -1. [Documentation styleguide](https://docs.gitlab.com/ee/development/documentation/styleguide.html) -1. Interface text should be written subjectively instead of objectively. It - should be the GitLab core team addressing a person. It should be written in - present time and never use past tense (has been/was). For example instead - of _prohibited this user from being saved due to the following errors:_ the - text should be _sorry, we could not create your account because:_ -1. Code should be written in [US English][us-english] +1. [Ruby](https://github.com/bbatsov/ruby-style-guide). + Important sections include [Source Code Layout][rss-source] and + [Naming][rss-naming]. Use: + - multi-line method chaining style **Option A**: dot `.` on the second line + - string literal quoting style **Option A**: single quoted by default +1. [Rails](https://github.com/bbatsov/rails-style-guide) +1. [Newlines styleguide][newlines-styleguide] +1. [Testing][testing] +1. [JavaScript styleguide][js-styleguide] +1. [SCSS styleguide][scss-styleguide] +1. [Shell commands](../shell_commands.md) created by GitLab + contributors to enhance security +1. [Database Migrations](../migration_style_guide.md) +1. [Markdown](http://www.cirosantilli.com/markdown-styleguide) +1. [Documentation styleguide](https://docs.gitlab.com/ee/development/documentation/styleguide.html) +1. Interface text should be written subjectively instead of objectively. It + should be the GitLab core team addressing a person. It should be written in + present time and never use past tense (has been/was). For example instead + of _prohibited this user from being saved due to the following errors:_ the + text should be _sorry, we could not create your account because:_ +1. Code should be written in [US English][us-english] This is also the style used by linting tools such as [RuboCop](https://github.com/bbatsov/rubocop), diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index eac7cb44c40..f4486ae3549 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -1,41 +1,4 @@ -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - -- [Contribute to GitLab](#contribute-to-gitlab) -- [Security vulnerability disclosure](#security-vulnerability-disclosure) -- [Code of conduct](#code-of-conduct) -- [Closing policy for issues and merge requests](#closing-policy-for-issues-and-merge-requests) -- [Helping others](#helping-others) -- [I want to contribute!](#i-want-to-contribute) -- [Contribution Flow](#contribution-flow) -- [Workflow labels](#workflow-labels) - - [Type labels](#type-labels) - - [Subject labels](#subject-labels) - - [Team labels](#team-labels) - - [Milestone labels](#milestone-labels) - - [Bug Priority labels](#bug-priority-labels) - - [Bug Severity labels](#bug-severity-labels) - - [Severity impact guidance](#severity-impact-guidance) - - [Label for community contributors](#label-for-community-contributors) -- [Implement design & UI elements](#implement-design--ui-elements) -- [Issue tracker](#issue-tracker) - - [Issue triaging](#issue-triaging) - - [Feature proposals](#feature-proposals) - - [Issue tracker guidelines](#issue-tracker-guidelines) - - [Issue weight](#issue-weight) - - [Regression issues](#regression-issues) - - [Technical and UX debt](#technical-and-ux-debt) - - [Stewardship](#stewardship) -- [Merge requests](#merge-requests) - - [Merge request guidelines](#merge-request-guidelines) - - [Contribution acceptance criteria](#contribution-acceptance-criteria) -- [Definition of done](#definition-of-done) -- [Style guides](#style-guides) - -<!-- END doctoc generated TOC please keep comment here to allow auto update --> - -## Contribute to GitLab +# Contribute to GitLab For a first-time step-by-step guide to the contribution process, see ["Contributing to GitLab"](https://about.gitlab.com/contributing/). diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 08042fa2aec..fed29d37b26 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -1,27 +1,4 @@ -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - -- [Workflow labels](#workflow-labels) - - [Type labels](#type-labels) - - [Subject labels](#subject-labels) - - [Team labels](#team-labels) - - [Release Scoping labels](#release-scoping-labels) - - [Priority labels](#priority-labels) - - [Severity labels](#severity-labels) - - [Severity impact guidance](#severity-impact-guidance) - - [Label for community contributors](#label-for-community-contributors) - - [Issue triaging](#issue-triaging) - - [Feature proposals](#feature-proposals) - - [Issue tracker guidelines](#issue-tracker-guidelines) - - [Issue weight](#issue-weight) - - [Regression issues](#regression-issues) - - [Technical and UX debt](#technical-and-ux-debt) - - [Stewardship](#stewardship) - -<!-- END doctoc generated TOC please keep comment here to allow auto update --> - -## Workflow labels +# Workflow labels To allow for asynchronous issue handling, we use [milestones][milestones-page] and [labels][labels-page]. Leads and product managers handle most of the @@ -45,7 +22,7 @@ labels, you can _always_ add the team and type, and often also the subject. [milestones-page]: https://gitlab.com/gitlab-org/gitlab-ce/milestones [labels-page]: https://gitlab.com/gitlab-org/gitlab-ce/labels -### Type labels +## Type labels Type labels are very important. They define what kind of issue this is. Every issue should have one or more. @@ -61,7 +38,7 @@ already reserved for subject labels). The descriptions on the [labels page][labels-page] explain what falls under each type label. -### Subject labels +## Subject labels Subject labels are labels that define what area or feature of GitLab this issue hits. They are not always necessary, but very convenient. @@ -75,7 +52,7 @@ issue is labeled with a subject label corresponding to your expertise. Subject labels are always all-lowercase. -### Team labels +## Team labels Team labels specify what team is responsible for this issue. Assigning a team label makes sure issues get the attention of the appropriate @@ -107,7 +84,7 @@ indicate if an issue needs backend work, frontend work, or both. Team labels are always capitalized so that they show up as the first label for any issue. -### Release Scoping labels +## Release Scoping labels Release Scoping labels help us clearly communicate expectations of the work for the release. There are three levels of Release Scoping labels: @@ -138,7 +115,7 @@ This label documents the planned timeline & urgency which is used to measure aga | ~P3 | Medium Priority | Within the next 3 releases (approx one quarter) | | ~P4 | Low Priority | Anything outside the next 3 releases (approx beyond one quarter) | -### Severity labels +## Severity labels Severity labels help us clearly communicate the impact of a ~bug on users. @@ -149,7 +126,7 @@ Severity labels help us clearly communicate the impact of a ~bug on users. | ~S3 | Major Severity | Broken Feature, workaround acceptable | Can create merge requests only from the Merge Requests page, not through the Issue. | | ~S4 | Low Severity | Functionality inconvenience or cosmetic issue | Label colors are incorrect / not being displayed. | -#### Severity impact guidance +### Severity impact guidance Severity levels can be applied further depending on the facet of the impact; e.g. Affected customers, GitLab.com availability, performance and etc. The below is a guideline. @@ -160,7 +137,7 @@ Severity levels can be applied further depending on the facet of the impact; e.g | ~S3 | A few users or a single paid customer affected | Limited impact on important portions of GitLab.com | Degradation is likely to occur in the near future | | ~S4 | No paid users/customer affected, or expected to in the near future | Minor impact on on GitLab.com | Degradation _may_ occur but it's not likely | -### Label for community contributors +## Label for community contributors Issues that are beneficial to our users, 'nice to haves', that we currently do not have the capacity for or want to give the priority to, are labeled as @@ -200,7 +177,7 @@ If you've decided that you would like to work on an issue, please @-mention the [appropriate product manager](https://about.gitlab.com/handbook/product/#who-to-talk-to-for-what) as soon as possible. The product manager will then pull in appropriate GitLab team members to further discuss scope, design, and technical considerations. This will -ensure that that your contribution is aligned with the GitLab product and minimize +ensure that your contribution is aligned with the GitLab product and minimize any rework and delay in getting it merged into master. GitLab team members who apply the ~"Accepting Merge Requests" label to an issue @@ -210,8 +187,7 @@ any potential community contributor to @-mention per above. [up-for-grabs]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name=Accepting+Merge+Requests&scope=all&sort=weight_asc&state=opened [firt-timers]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name%5B%5D=Accepting+Merge+Requests&scope=all&sort=upvotes_desc&state=opened&weight=1 - -### Issue triaging +## Issue triaging Our issue triage policies are [described in our handbook]. You are very welcome to help the GitLab team triage issues. We also organize [issue bash events] once @@ -224,7 +200,7 @@ on those issues. Please select someone with relevant experience from the the commit history for the affected files to find someone. We also use [GitLab Triage] to automate some triaging policies. This is -currently setup as a [scheduled pipeline] running on [quality/triage-ops] +currently set up as a [scheduled pipeline] running on [quality/triage-ops] project. [described in our handbook]: https://about.gitlab.com/handbook/engineering/issue-triage/ @@ -233,7 +209,7 @@ project. [scheduled pipeline]: https://gitlab.com/gitlab-org/quality/triage-ops/pipeline_schedules/10512/edit [quality/triage-ops]: https://gitlab.com/gitlab-org/quality/triage-ops -### Feature proposals +## Feature proposals To create a feature proposal for CE, open an issue on the [issue tracker of CE][ce-tracker]. @@ -250,7 +226,7 @@ code snippet right after your description in a new line: `~"feature proposal"`. Please keep feature proposals as small and simple as possible, complex ones might be edited to make them small and simple. -Please submit Feature Proposals using the ['Feature Proposal' issue template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/issue_templates/Feature%20Proposal.md) provided on the issue tracker. +Please submit Feature Proposals using the ['Feature Proposal' issue template](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab/issue_templates/Feature%20proposal.md) provided on the issue tracker. For changes in the interface, it is helpful to include a mockup. Issues that add to, or change, the interface should be given the ~"UX" label. This will allow the UX team to provide input and guidance. You may @@ -259,7 +235,7 @@ need to ask one of the [core team] members to add the label, if you do not have If you want to create something yourself, consider opening an issue first to discuss whether it is interesting to include this in GitLab. -### Issue tracker guidelines +## Issue tracker guidelines **[Search the issue tracker][ce-tracker]** for similar entries before submitting your own, there's a good chance somebody else had the same issue or @@ -271,7 +247,7 @@ The text in the parenthesis is there to help you with what to include. Omit it when submitting the actual issue. You can copy-paste it and then edit as you see fit. -### Issue weight +## Issue weight Issue weight allows us to get an idea of the amount of work required to solve one or multiple issues. This makes it possible to schedule work more accurately. @@ -293,7 +269,7 @@ is probably 1, adding a new Git Hook maybe 4 or 5, big features 7-9. issues or chunks. You can simply not set the weight of a parent issue and set weights to children issues. -### Regression issues +## Regression issues Every monthly release has a corresponding issue on the CE issue tracker to keep track of functionality broken by that release and any fixes that need to be @@ -313,7 +289,7 @@ addressed. [8.3 Regressions]: https://gitlab.com/gitlab-org/gitlab-ce/issues/4127 [update the notes]: https://gitlab.com/gitlab-org/release-tools/blob/master/doc/pro-tips.md#update-the-regression-issue -### Technical and UX debt +## Technical and UX debt In order to track things that can be improved in GitLab's codebase, we use the ~"technical debt" label in [GitLab's issue tracker][ce-tracker]. @@ -337,7 +313,7 @@ for a release by the appropriate person. Make sure to mention the merge request that the ~"technical debt" issue or ~"UX debt" issue is associated with in the description of the issue. -### Stewardship +## Stewardship For issues related to the open source stewardship of GitLab, there is the ~"stewardship" label. diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index 685287f7a0c..a286e74908c 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -1,15 +1,4 @@ -<!-- START doctoc generated TOC please keep comment here to allow auto update --> -<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> -**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - -- [Merge requests](#merge-requests) - - [Merge request guidelines](#merge-request-guidelines) - - [Contribution acceptance criteria](#contribution-acceptance-criteria) -- [Definition of done](#definition-of-done) - -<!-- END doctoc generated TOC please keep comment here to allow auto update --> - -## Merge requests +# Merge requests We welcome merge requests with fixes and improvements to GitLab code, tests, and/or documentation. The issues that are specifically suitable for @@ -36,7 +25,7 @@ some potentially easy issues. To start with GitLab development download the [GitLab Development Kit][gdk] and see the [Development section](../README.md) for some guidelines. -### Merge request guidelines +## Merge request guidelines If you can, please submit a merge request with the fix or improvements including tests. If you don't know how to fix the issue but can write a test @@ -114,7 +103,7 @@ Please ensure that your merge request meets the contribution acceptance criteria When having your code reviewed and when reviewing merge requests please take the [code review guidelines](../code_review.md) into account. -### Contribution acceptance criteria +## Contribution acceptance criteria 1. The change is as small as possible 1. Include proper tests and make all tests pass (unless it contains a test diff --git a/doc/development/database_debugging.md b/doc/development/database_debugging.md index 9c31265e417..b2c804b2ff0 100644 --- a/doc/development/database_debugging.md +++ b/doc/development/database_debugging.md @@ -33,7 +33,7 @@ If your test DB is giving you problems, it is safe to nuke it because it doesn't - `bundle exec rake db:migrate RAILS_ENV=development`: Execute any pending migrations that you may have picked up from a MR - `bundle exec rake db:migrate:status RAILS_ENV=development`: Check if all migrations are `up` or `down` - `bundle exec rake db:migrate:down VERSION=20170926203418 RAILS_ENV=development`: Tear down a migration - - `bundle exec rake db:migrate:up VERSION=20170926203418 RAILS_ENV=development`: Setup a migration + - `bundle exec rake db:migrate:up VERSION=20170926203418 RAILS_ENV=development`: Set up a migration - `bundle exec rake db:migrate:redo VERSION=20170926203418 RAILS_ENV=development`: Re-run a specific migration diff --git a/doc/development/database_helpers.md b/doc/development/database_helpers.md new file mode 100644 index 00000000000..21e4e725de6 --- /dev/null +++ b/doc/development/database_helpers.md @@ -0,0 +1,63 @@ +# Database helpers + +There are a number of useful helper modules defined in `/lib/gitlab/database/`. + +## Subquery + +In some cases it is not possible to perform an operation on a query. +For example: + +```ruby +Geo::EventLog.where('id < 100').limit(10).delete_all +``` + +Will give this error: + +> ActiveRecord::ActiveRecordError: delete_all doesn't support limit + +One solution would be to wrap it in another `where`: + +```ruby +Geo::EventLog.where(id: Geo::EventLog.where('id < 100').limit(10)).delete_all +``` + +This works with PostgreSQL, but with MySQL it gives this error: + +> ActiveRecord::StatementInvalid: Mysql2::Error: This version of MySQL +> doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery' + +Also, that query doesn't have very good performance. Using a +`INNER JOIN` with itself is better. + +So instead of this query: + +```sql +SELECT geo_event_log.* +FROM geo_event_log +WHERE geo_event_log.id IN + (SELECT geo_event_log.id + FROM geo_event_log + WHERE (id < 100) + LIMIT 10) +``` + +It's better to write: + +```sql +SELECT geo_event_log.* +FROM geo_event_log +INNER JOIN + (SELECT geo_event_log.* + FROM geo_event_log + WHERE (id < 100) + LIMIT 10) t2 ON geo_event_log.id = t2.id +``` + +And this is where `Gitlab::Database::Subquery.self_join` can help +you. So you can rewrite the above statement as: + +```ruby +Gitlab::Database::Subquery.self_join(Geo::EventLog.where('id < 100').limit(10)).delete_all +``` + +And this also works with MySQL, so you don't need to worry about that. diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 7ac211ed550..2db78e4a365 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -43,13 +43,13 @@ how to structure GitLab docs. Currently GitLab docs use Redcarpet as [markdown](../../user/markdown.md) engine, but there's an [open discussion](https://gitlab.com/gitlab-com/gitlab-docs/issues/50) for implementing Kramdown in the near future. -All the docs follow the [documentation style guidelines](styleguide.md). +All the docs follow the [documentation style guidelines](styleguide.md). See [Linting](#linting) for help to follow the guidelines. ## Documentation directory structure The documentation is structured based on the GitLab UI structure itself, separated by [`user`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/user), -[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). +[`administrator`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/administration), and [`contributor`](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/doc/development). In order to have a [solid site structure](https://searchengineland.com/seo-benefits-developing-solid-site-structure-277456) for our documentation, all docs should be linked. Every new document should be cross-linked to its related documentation, and linked from its topic-related index, when existent. @@ -223,6 +223,108 @@ redirect_from: 'https://docs.gitlab.com/my-old-location/README.html' Note: it is necessary to include the file name in the `redirect_from` URL, even if it's `index.html` or `README.html`. +## Linting + +To help adhere to the [documentation style guidelines](styleguide.md), and to improve the content + added to documentation, consider locally installing and running documentation linters. This will + help you catch common issues before raising merge requests for review of documentation. + +The following are some suggested linters you can install locally and sample configuration: + +- `proselint` +- `markdownlint` + +NOTE: **Note:** +This list does not limit what other linters you can add to your local documentation writing + toolchain. + +### `proselint` + +`proselint` checks for common problems with English prose. It provides a + [plethora of checks](http://proselint.com/checks/) that are helpful for technical writing. + +`proselint` can be used [on the command line](http://proselint.com/utility/), either on a single + Markdown file or on all Markdown files in a project. For example, to run `proselint` on all + documentation in the [`gitlab-ce` project](https://gitlab.com/gitlab-org/gitlab-ce), run the + following commands from within the `gitlab-ce` project: + +```sh +cd doc +proselint **/*.md +``` + +`proselint` can also be run from within editors using plugins. For example, the following plugins + are available: + +- [Sublime Text](https://packagecontrol.io/packages/SublimeLinter-contrib-proselint) +- [Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=PatrykPeszko.vscode-proselint) +- [Others](https://github.com/amperser/proselint#plugins-for-other-software) + +#### Sample `proselint` configuration + +All of the checks are good to use. However, excluding the `typography.symbols` checks might reduce + noise. The following sample `proselint` configuration disables the `typography.symbols` checks: + +```json +{ + "checks": { + "typography.symbols": false + } +} +``` + +A file with `proselint` configuration must be placed in a + [valid location](https://github.com/amperser/proselint#checks). For example, `~/.config/proselint/config`. + +### `markdownlint` + +`markdownlint` checks that certain rules ([example](https://github.com/DavidAnson/markdownlint/blob/master/README.md#rules--aliases)) + are followed for Markdown syntax. Our [style guidelines](styleguide.md) elaborate on which choices + must be made when selecting Markdown syntax for GitLab documentation and this tool helps + catch deviations from those guidelines. + +`markdownlint` can be used [on the command line](https://github.com/igorshubovych/markdownlint-cli#markdownlint-cli--), + either on a single Markdown file or on all Markdown files in a project. For example, to run + `markdownlint` on all documentation in the [`gitlab-ce` project](https://gitlab.com/gitlab-org/gitlab-ce), + run the following commands from within the `gitlab-ce` project: + +```sh +cd doc +markdownlint **/*.md +``` + +`markdownlint` can also be run from within editors using plugins. For example, the following plugins + are available: + +- [Sublime Text](https://packagecontrol.io/packages/SublimeLinter-contrib-markdownlint) +- [Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint) +- [Others](https://github.com/DavidAnson/markdownlint#related) + +#### Sample `markdownlint` configuration + +The following sample `markdownlint` configuration modifies the available default rules to: + +- Adhere to the [style guidelines](styleguide.md). +- Apply conventions found in the GitLab documentation. + +```json +{ + "default": true, + "header-style": { "style": "atx" }, + "ul-style": { "style": "dash" }, + "line-length": false, + "no-trailing-punctuation": false, + "ol-prefix": { "style": "one" }, + "blanks-around-fences": false, + "hr-style": { "style": "---" }, + "fenced-code-language": false +} +``` + +For [`markdownlint`](https://gitahub.com/DavidAnson/markdownlint/), this configuration must be + placed in a [valid location](https://github.com/igorshubovych/markdownlint-cli#configuration). For + example, `~/.markdownlintrc`. + ## Testing We treat documentation as code, thus have implemented some testing. @@ -278,7 +380,6 @@ for GitLab Team members. - Label the MR `Documentation` - Assign the correct milestone (see note below) - NOTE: **Note:** If the release version you want to add the documentation to has already been frozen or released, use the label `Pick into X.Y` to get it merged into @@ -411,6 +512,22 @@ The following GitLab features are used among others: Every GitLab instance includes the documentation, which is available from `/help` (`http://my-instance.com/help`), e.g., <https://gitlab.com/help>. +The documentation available online on docs.gitlab.com is continuously +deployed every hour from the `master` branch of CE, EE, Omnibus, and Runner. Therefore, +once a merge request gets merged, it will be available online on the same day, +but they will be shipped (and available on `/help`) within the milestone assigned +to the MR. + +For instance, let's say your merge request has a milestone set to 11.3, which +will be released on 2018-09-22. If it gets merged on 2018-09-15, it will be +available online on 2018-09-15, but, as the feature freeze date has passed, if +the MR does not have a "pick into 11.3" label, the milestone has to be changed +to 11.4 and it will be shipped with all GitLab packages only on 2018-10-22, +with GitLab 11.4. Meaning, it will only be available under `/help` from GitLab +11.4 onwards, but available on docs.gitlab.com on the same day it was merged. + +### Linking to `/help` + When you're building a new feature, you may need to link the documentation from GitLab, the application. This is normally done in files inside the `app/views/` directory with the help of the `help_page_path` helper method. diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md index 8083f219d4a..c43f91278de 100644 --- a/doc/development/documentation/styleguide.md +++ b/doc/development/documentation/styleguide.md @@ -10,6 +10,8 @@ GitLab documentation. Check the Check the GitLab handbook for the [writing styles guidelines](https://about.gitlab.com/handbook/communication/#writing-style-guidelines). +For help adhering to the guidelines, see [Linting](index.md#linting). + ## Files - [Directory structure](index.md#location-and-naming-documents): place the docs @@ -251,7 +253,7 @@ below. (in that order) that introduced it. The above quote would be then transformed to: ```md - > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/1242) in GitLab 8.3. + > [Introduced](<link-to-issue>) in GitLab 8.3. ``` - If the feature is only available in GitLab Enterprise Edition, don't forget to mention @@ -259,10 +261,22 @@ below. the feature is available in: ```md - > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/1242) - in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. + > [Introduced](<link-to-issue>) in [GitLab Starter](https://about.gitlab.com/pricing/) 10.3. ``` +#### Early versions of EE + +If the feature was created before GitLab 9.2 (before [different EE tiers were introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1851)): + +- Declare it as "Introduced in GitLab Enterprise Edition X.Y". +- Note which tier the feature is available in. + +For example: + +```md +> [Introduced](<link-to-issue>) in GitLab Enterprise Edition 9.0. Available in [GitLab Premium](https://about.gitlab.com/pricing/). +``` + ### Product badges When a feature is available in EE-only tiers, add the corresponding tier according to the diff --git a/doc/development/fe_guide/components.md b/doc/development/fe_guide/components.md index 66a8abe42f7..ee0c2d534ff 100644 --- a/doc/development/fe_guide/components.md +++ b/doc/development/fe_guide/components.md @@ -42,7 +42,7 @@ See also the [corresponding UX guide](../ux_guide/components.md#dropdowns). See also the [corresponding UX guide](../ux_guide/components.md#modals). -We have a reusable Vue component for modals: [vue_shared/components/gl-modal.vue](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/vue_shared/components/gl-modal.vue) +We have a reusable Vue component for modals: [vue_shared/components/gl_modal.vue](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/vue_shared/components/gl_modal.vue) Here is an example of how to use it: diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 6f757f1ce7b..417298205f5 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -65,13 +65,18 @@ In the rare case that you need the feature flag to be on automatically, use Feature.enabled?(:feature_flag, project, default_enabled: true) ``` +For more information about rolling out changes using feature flags, refer to the +[Rolling out changes using feature flags](rolling_out_changes_using_feature_flags.md) +guide. + ### Specs In the test environment `Feature.enabled?` is stubbed to always respond to `true`, so we make sure behavior under feature flag doesn't go untested in some non-specific contexts. -If you need to test the feature flag in a different state, you need to stub it with: +Whenever a feature flag is present, make sure to test _both_ states of the +feature flag. You can stub a feature flag as follows: ```ruby stub_feature_flags(my_feature_flag: false) diff --git a/doc/development/file_storage.md b/doc/development/file_storage.md index fdbd7f1fa37..6e014e8c751 100644 --- a/doc/development/file_storage.md +++ b/doc/development/file_storage.md @@ -45,6 +45,11 @@ In the case of Issues/MR/Notes Markdown attachments, there is a different approa instead of basing the path into a mutable variable `:project_path_with_namespace`, it's possible to use the hash of the project ID instead, if project migrates to the new approach (introduced in 10.2). +> Note: We provide an [all-in-one rake task] to migrate all uploads to object +> storage in one go. If a new Uploader class or model type is introduced, make +> sure you add a rake task invocation corresponding to it to the [category +> list]. + ### Path segments Files are stored at multiple locations and use different path schemes. @@ -137,3 +142,5 @@ end [CarrierWave]: https://github.com/carrierwaveuploader/carrierwave [Hashed Storage]: ../administration/repository_storage_types.md +[all-in-one rake task]: ../administration/raketasks/uploads/migrate.md +[category list]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/tasks/gitlab/uploads/migrate.rake diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index 4f9ca1920a5..f4784c19359 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -9,7 +9,7 @@ status of the migration. ## Developing new Git features -Starting with Gitlab 10.8, all new Git features should be developed in +Starting with GitLab 10.8, all new Git features should be developed in Gitaly. > This is a new process that is not clearly defined yet. If you want diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 7054ff39da0..5e13c725e15 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -30,6 +30,7 @@ are very appreciative of the work done by translators and proofreaders! - Korean - Chang-Ho Cha - [GitLab](https://gitlab.com/changho-cha), [Crowdin](https://crowdin.com/profile/zzazang) - Huang Tao - [GitLab](https://gitlab.com/htve), [Crowdin](https://crowdin.com/profile/htve) + - Ji Hun Oh - [GitLab](https://gitlab.com/Baw-Appie), [Crowdin](https://crowdin.com/profile/BawAppie) - Polish - Filip Mech - [GitLab](https://gitlab.com/mehenz), [Crowdin](https://crowdin.com/profile/mehenz) - Portuguese, Brazilian diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index 12badbe39b2..ee01c89e0ed 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -168,6 +168,7 @@ user objects for every username we can remove the need for running the same query for every mention of `@alice`. Caching data per transaction can be done using -[RequestStore](https://github.com/steveklabnik/request_store). Caching data in -Redis can be done using [Rails' caching +[RequestStore](https://github.com/steveklabnik/request_store) (use +`Gitlab::SafeRequestStore` to avoid having to remember to check +`RequestStore.active?`). Caching data in Redis can be done using [Rails' caching system](http://guides.rubyonrails.org/caching_with_rails.html). diff --git a/doc/development/module_with_instance_variables.md b/doc/development/module_with_instance_variables.md index 48a1b7f847e..7bdfa04fc57 100644 --- a/doc/development/module_with_instance_variables.md +++ b/doc/development/module_with_instance_variables.md @@ -60,7 +60,7 @@ as long as it's contained in the same module; that is, no other modules or objects are touching them, then it would be an acceptable use. We especially allow the case where a single instance variable is used with -`||=` to setup the value. This would look like: +`||=` to set up the value. This would look like: ``` ruby module M diff --git a/doc/development/ordering_table_columns.md b/doc/development/ordering_table_columns.md index 5d00e1f7a0c..e9c6481635b 100644 --- a/doc/development/ordering_table_columns.md +++ b/doc/development/ordering_table_columns.md @@ -1,32 +1,49 @@ -# Ordering Table Columns +# Ordering Table Columns in PostgreSQL Similar to C structures the space of a table is influenced by the order of columns. This is because the size of columns is aligned depending on the type of -the column. Take the following column order for example: +the following column. Let's consider an example: -* id (integer, 4 bytes) -* name (text, variable) -* user_id (integer, 4 bytes) +- `id` (integer, 4 bytes) +- `name` (text, variable) +- `user_id` (integer, 4 bytes) -Integers are aligned to the word size. This means that on a 64 bit platform the -actual size of each column would be: 8 bytes, variable, 8 bytes. This means that -each row will require at least 16 bytes for the two integers, and a variable -amount for the text field. If a table has a few rows this is not an issue, but -once you start storing millions of rows you can save space by using a different -order. For the above example a more ideal column order would be the following: +The first column is a 4-byte integer. The next is text of variable length. The +`text` data type requires 1-word alignment, and on 64-bit platform, 1 word is 8 +bytes. To meet the alignment requirements, four zeros are to be added right +after the first column, so `id` occupies 4 bytes, then 4 bytes of alignment +padding, and only next `name` is being stored. Therefore, in this case, 8 bytes +will be spent for storing a 4-byte integer. -* id (integer, 4 bytes) -* user_id (integer, 4 bytes) -* name (text, variable) +The space between rows is also subject to alignment padding. The `user_id` +column takes only 4 bytes, and on 64-bit platform, 4 zeroes will be added for +alignment padding, to allow storing the next row beginning with the "clear" word. -In this setup the `id` and `user_id` columns can be packed together, which means -we only need 8 bytes to store _both_ of them. This in turn each row will require -8 bytes less of space. +As a result, the actual size of each column would be (ommiting variable length +data and 24-byte tuple header): 8 bytes, variable, 8 bytes. This means that +each row will require at least 16 bytes for the two 4-byte integers. If a table +has a few rows this is not an issue. However, once you start storing millions of +rows you can save space by using a different order. For the above example, the +ideal column order would be the following: + +- `id` (integer, 4 bytes) +- `user_id` (integer, 4 bytes) +- `name` (text, variable) + +or + +- `name` (text, variable) +- `id` (integer, 4 bytes) +- `user_id` (integer, 4 bytes) + +In these examples, the `id` and `user_id` columns are packed together, which +means we only need 8 bytes to store _both_ of them. This in turn means each row +will require 8 bytes less space. For GitLab we require that columns of new tables are ordered based to use the least amount of space. An easy way of doing this is to order them based on the -type size in descending order with variable sizes (string and text columns for -example) at the end. +type size in descending order with variable sizes (`text`, `varchar`, arrays, +`json`, `jsonb`, and so on) at the end. ## Type Sizes @@ -36,7 +53,7 @@ of information we will list the sizes of common types here so it's easier to look them up. Here "word" refers to the word size, which is 4 bytes for a 32 bits platform and 8 bytes for a 64 bits platform. -| Type | Size | Aligned To | +| Type | Size | Alignment needed | |:-----------------|:-------------------------------------|:-----------| | smallint | 2 bytes | 1 word | | integer | 4 bytes | 1 word | @@ -58,7 +75,7 @@ always be at the end of a table. ## Real Example -Let's use the "events" table as an example, which currently has the following +Let's use the `events` table as an example, which currently has the following layout: | Column | Type | Size | @@ -89,8 +106,8 @@ divided into fixed size chunks as follows: | 8 bytes | updated_at | | 8 bytes | action, author_id | -This means that excluding the variable sized data we need at least 48 bytes per -row. +This means that excluding the variable sized data and tuple header, we need at +least 8 * 6 = 48 bytes per row. We can optimise this by using the following column order instead: @@ -120,8 +137,8 @@ This would produce the following chunks: | variable | title | | variable | data | -Here we only need 40 bytes per row excluding the variable sized data. 8 bytes -being saved may not sound like much, but for tables as large as the "events" -table it does begin to matter. For example, when storing 80 000 000 rows this -translates to a space saving of at least 610 MB: all by just changing the order -of a few columns. +Here we only need 40 bytes per row excluding the variable sized data and 24-byte +tuple header. 8 bytes being saved may not sound like much, but for tables as +large as the `events` table it does begin to matter. For example, when storing +80 000 000 rows this translates to a space saving of at least 610 MB, all by +just changing the order of a few columns. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index fc51b74da1d..2ad748d4802 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -1,6 +1,6 @@ # Rake tasks for developers -## Setup db with developer seeds +## Set up db with developer seeds Note that if your db user does not have advanced privileges you must create the db manually before running this command. diff --git a/doc/development/rolling_out_changes_using_feature_flags.md b/doc/development/rolling_out_changes_using_feature_flags.md new file mode 100644 index 00000000000..905aa26a40b --- /dev/null +++ b/doc/development/rolling_out_changes_using_feature_flags.md @@ -0,0 +1,153 @@ +# Rolling out changes using feature flags + +[Feature flags](feature_flags.md) can be used to gradually roll out changes, be +it a new feature, or a performance improvement. By using feature flags, we can +comfortably measure the impact of our changes, while still being able to easily +disable those changes, without having to revert an entire release. + +## When to use feature flags + +Starting with GitLab 11.4, developers are required to use feature flags for +non-trivial changes. Such changes include: + +* New features (e.g. a new merge request widget, epics, etc). +* Complex performance improvements that may require additional testing in + production, such as rewriting complex queries. +* Invasive changes to the user interface, such as a new navigation bar or the + removal of a sidebar. +* Adding support for importing projects from a third-party service. + +In all cases, those working on the changes can best decide if a feature flag is +necessary. For example, changing the color of a button doesn't need a feature +flag, while changing the navigation bar definitely needs one. In case you are +uncertain if a feature flag is necessary, simply ask about this in the merge +request, and those reviewing the changes will likely provide you with an answer. + +When using a feature flag for UI elements, make sure to _also_ use a feature +flag for the underlying backend code, if there is any. This ensures there is +absolutely no way to use the feature until it is enabled. + +## The cost of feature flags + +When reading the above, one might be tempted to think this procedure is going to +add a lot of work. Fortunately, this is not the case, and we'll show why. For +this example we'll specify the cost of the work to do as a number, ranging from +0 to infinity. The greater the number, the more expensive the work is. The cost +does _not_ translate to time, it's just a way of measuring complexity of one +change relative to another. + +Let's say we are building a new feature, and we have determined that the cost of +this is 10. We have also determined that the cost of adding a feature flag check +in a variety of places is 1. If we do not use feature flags, and our feature +works as intended, our total cost is 10. This however is the best case scenario. +Optimising for the best case scenario is guaranteed to lead to trouble, whereas +optimising for the worst case scenario is almost always better. + +To illustrate this, let's say our feature causes an outage, and there's no +immediate way to resolve it. This means we'd have to take the following steps to +resolve the outage: + +1. Revert the release. +1. Perform any cleanups that might be necessary, depending on the changes that + were made. +1. Revert the commit, ensuring the "master" branch remains stable. This is + especially necessary if solving the problem can take days or even weeks. +1. Pick the revert commit into the appropriate stable branches, ensuring we + don't block any future releases until the problem is resolved. + +As history has shown, these steps are time consuming, complex, often involve +many developers, and worst of all: our users will have a bad experience using +GitLab.com until the problem is resolved. + +Now let's say that all of this has an associated cost of 10. This means that in +the worst case scenario, which we should optimise for, our total cost is now 20. + +If we had used a feature flag, things would have been very different. We don't +need to revert a release, and because feature flags are disabled by default we +don't need to revert and pick any Git commits. In fact, all we have to do is +disable the feature, and _maybe_ perform some cleanup. Let's say that the cost +of this is 1. In this case, our best case cost is 11: 10 to build the feature, +and 1 to add the feature flag. The worst case cost is now 12: 10 to build the +feature, 1 to add the feature flag, and 1 to disable it. + +Here we can see that in the best case scenario the work necessary is only a tiny +bit more compared to not using a feature flag. Meanwhile, the process of +reverting our changes has been made significantly cheaper, to the point of being +trivial. + +In other words, feature flags do not slow down the development process. Instead, +they speed up the process as managing incidents now becomes _much_ easier. Once +continuous deployments are easier to perform, the time to iterate on a feature +is reduced even further, as you no longer need to wait weeks before your changes +are available on GitLab.com. + +## Rolling out changes + +The procedure of using feature flags is straightforward, and similar to not +using them. You add the necessary tests (make sure to test both the on and off +states of your feature flag(s)), make sure they all pass, have the code +reviewed, etc. You then submit your merge request, and add the ~"feature flag" +label. This label is used to signal to release managers that your changes are +hidden behind a feature flag and that it is safe to pick the MR into a stable +branch, without the need for an exception request. + +When the changes are deployed it is time to start rolling out the feature to our +users. The exact procedure of rolling out a change is unspecified, as this can +vary from change to change. However, in general we recommend rolling out changes +incrementally, instead of enabling them for everybody right away. We also +recommend you to _not_ enable a feature _before_ the code is being deployed. +This allows you to separate rolling out a feature from a deploy, making it +easier to measure the impact of both separately. + +GitLab's feature library (using +[Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature +Flags](feature_flags.md) guide) supports rolling out changes to a percentage of +users. This in turn can be controlled using [GitLab +chatops](https://docs.gitlab.com/ee/ci/chatops/). + +For example, to enable a feature for 25% of all users, run the following in +Slack: + +``` +/chatops run feature set new_navigation_bar 25 +``` + +This will enable the feature for GitLab.com, with `new_navigation_bar` being the +name of the feature. We can also enable the feature for <https://dev.gitlab.org> +or <https://staging.gitlab.com>: + +``` +/chatops run feature set new_navigation_bar 25 --dev +/chatops run feature set new_navigation_bar 25 --staging +``` + +If you are not certain what percentages to use, simply use the following steps: + +1. 25% +1. 50% +1. 75% +1. 100% + +Between every step you'll want to wait a little while and monitor the +appropriate graphs on <https://dashboards.gitlab.net>. The exact time to wait +may differ. For some features a few minutes is enough, while for others you may +want to wait several hours or even days. This is entirely up to you, just make +sure it is clearly communicated to your team, and the Production team if you +anticipate any potential problems. + +Once a change is deemed stable, submit a new merge request to remove the +feature flag. This ensures the change is available to all users and self-hosted +instances. Make sure to add the ~"feature flag" label to this merge request so +release managers are aware the changes are hidden behind a feature flag. If the +merge request has to be picked into a stable branch (e.g. after the 7th), make +sure to also add the appropriate "Pick into X" label (e.g. "Pick into 11.4"). + +One might be tempted to think this will delay the release of a feature by at +least one month (= one release). This is not the case. A feature flag does not +have to stick around for a specific amount of time (e.g. at least one release), +instead they should stick around until the feature is deemed stable. Stable +means it works on GitLab.com without causing any problems, such as outages. In +most cases this will translate to a feature (with a feature flag) being shipped +in RC1, followed by the feature flag being removed in RC2. This in turn means +the feature will be stable by the time we publish a stable package around the +22nd of the month. diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index 0cd63a54b55..67e4cfeda0e 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -59,6 +59,12 @@ parallelization, monitoring. --- +## [Review apps](review_apps.md) + +How review apps are set up for GitLab CE/EE and how to use them. + +--- + ## [Testing Rake tasks](testing_rake_tasks.md) Everything you should know about how to test Rake tasks. diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md new file mode 100644 index 00000000000..25c6371f3d7 --- /dev/null +++ b/doc/development/testing_guide/review_apps.md @@ -0,0 +1,82 @@ +# Review apps + +We currently have review apps available as a manual job in EE pipelines. Here is +[the first implementation](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6259). + +That said, [the Quality team is working](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6665) +on making Review Apps automatically deployed by each pipeline, both in CE and EE. + +## How does it work? + +1. On every EE [pipeline][gitlab-pipeline] during the `test` stage, you can + start the [`review` job][review-job] +1. The `review` job [triggers a pipeline][cng-pipeline] in the + [`CNG-mirror`][cng-mirror] [^1] project +1. The `CNG-mirror` pipeline creates the Docker images of each component (e.g. `gitlab-rails-ee`, + `gitlab-shell`, `gitaly` etc.) based on the commit from the + [GitLab pipeline][gitlab-pipeline] and store them in its + [registry][cng-mirror-registry] +1. Once all images are built, the review app is deployed using + [the official GitLab Helm chart][helm-chart] [^2] to the + [`review-apps-ee` Kubernetes cluster on GCP][review-apps-ee] + - The actual scripts used to deploy the review app can be found at + [`scripts/review_apps/review-apps.sh`][review-apps.sh] + - These scripts are basically + [our official Auto DevOps scripts][Auto-DevOps.gitlab-ci.yml] where the + default CNG images are overriden with the images built and stored in the + [`CNG-mirror` project's registry][cng-mirror-registry] +1. Once the `review` job succeeds, you should be able to use your review app + thanks to the direct link to it from the MR widget. The default username is + `root` and its password can be found in the 1Password secure note named + **gitlab-{ce,ee} review app's root password**. + +**Additional notes:** + +- The Kubernetes cluster is connected to the `gitlab-ee` project using [GitLab's + Kubernetes integration][gitlab-k8s-integration]. This basically allows to have + a link to the review app directly from the merge request widget. +- The manual `stop_review` in the `post-cleanup` stage can be used to stop a + review app manually, and is also started by GitLab once a branch is deleted +- [TBD] Review apps are cleaned up regularly using a pipeline schedule that runs + the [`scripts/review_apps/automated_cleanup.rb`][automated_cleanup.rb] script + +[^1]: We use the `CNG-mirror` project so that the `CNG`, (**C**loud **N**ative **G**itLab), project's registry is + not overloaded with a lot of transient Docker images. +[^2]: Since we're using [the official GitLab Helm chart][helm-chart], this means + you get the a dedicated environment for your branch that's very close to what it + would look in production + +## Frequently Asked Questions + +**Will it be too much to trigger CNG image builds on every test run? This could create thousands of unused docker images.** + + > We have to start somewhere and improve later. If we see this getting out of hand, we will revisit. + +**How big is the Kubernetes cluster?** + + > The cluster is currently setup with a single pool of preemptible + nodes, with a minimum of 1 node and a maximum of 30 nodes. + +**What are the machine running on the cluster?** + + > We're currently using `n1-standard-4` (4 vCPUs, 15 GB memory) machines. + +**How do we secure this from abuse? Apps are open to the world so we need to find a way to limit it to only us.** + + > This won't work for forks. We will add a root password to 1password shared vault. + +[gitlab-pipeline]: https://gitlab.com/gitlab-org/gitlab-ee/pipelines/29302122 +[review-job]: https://gitlab.com/gitlab-org/gitlab-ee/-/jobs/94294136 +[cng-mirror]: https://gitlab.com/gitlab-org/build/CNG-mirror +[cng-pipeline]: https://gitlab.com/gitlab-org/build/CNG-mirror/pipelines/29307727 +[cng-mirror-registry]: https://gitlab.com/gitlab-org/build/CNG-mirror/container_registry +[helm-chart]: https://gitlab.com/charts/gitlab/ +[review-apps-ee]: https://console.cloud.google.com/kubernetes/clusters/details/us-central1-b/review-apps-ee?project=gitlab-review-apps +[review-apps.sh]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/review-apps.sh +[automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb +[Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml +[gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html + +--- + +[Return to Testing documentation](index.md) diff --git a/doc/development/ux_guide/users.md b/doc/development/ux_guide/users.md index 6afb33cfc36..f9c395b2dff 100644 --- a/doc/development/ux_guide/users.md +++ b/doc/development/ux_guide/users.md @@ -154,7 +154,7 @@ He credits himself as being entirely responsible for moving his company to GitLa #### Updating to the latest release Matthieu introduced his company to GitLab. He is responsible for maintaining and managing the company's installation in addition to his day job. He feels updates are too frequent and he doesn't always have sufficient time to update GitLab. As a result, he's not up to date with releases. -Matthieu tried to set up automatic updates, however, as he isn't a Systems Administrator, he wasn't confident in his set-up. He feels he should be able to "upgrade without users even noticing" but hasn't figured out how to do this yet. Matthieu would like the "update process to be triggered from the Admin Panel, perhaps accompanied with a changelog and the option to skip updates." +Matthieu tried to set up automatic updates, however, as he isn't a Systems Administrator, he wasn't confident in his setup. He feels he should be able to "upgrade without users even noticing" but hasn't figured out how to do this yet. Matthieu would like the "update process to be triggered from the Admin Panel, perhaps accompanied with a changelog and the option to skip updates." Matthieu is looking for confirmation that his update procedure is "secure and efficient" so more tutorials related to this topic would be useful to him. @@ -242,7 +242,7 @@ Some of GitLab EE's features are too basic, in particular, issues boards which d James and his team use CI quite heavily for several projects. Whilst they've welcomed improvements to the builds and pipelines interface, they still have some difficulty following build process on the different tabs under Pipelines. Some confusion has arisen from not knowing where to find different pieces of information or how to get to the next stages logs from the current stage's log output screen. They feel more intuitive linking and flow may alleviate the problem. Generally, they feel GitLab's navigation needs to reviewed and optimized. #### Permissions ->"There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for Gitlab CI/build pipelines." +>"There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for GitLab CI/build pipelines." ### Goals @@ -290,7 +290,7 @@ JavaScript and SQL Web development, mobile development, UX, open source, gaming, and travel. ### Motivations -Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab's values and the fact that it isn't a "behemoth of a company". She explains that "displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose Gitlab personally and to recommend it at work." She's also an avid reader of GitLab's blog. +Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab's values and the fact that it isn't a "behemoth of a company". She explains that "displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose GitLab personally and to recommend it at work." She's also an avid reader of GitLab's blog. Karolina works for a software development company which currently hires around 500 people. Karolina would love to use GitLab at work but the company has used GitHub Enterprise for a number of years. She describes management at her company as "old fashioned" and explains that it's "less of a technical issue and more of a cultural issue" to convince upper management to move to GitLab. Karolina is also relatively new to the company so she's apprehensive about pushing too hard to change version control platforms. |