summaryrefslogtreecommitdiff
path: root/doc/development
diff options
context:
space:
mode:
authorConstance Okoghenun <cokoghenun@gitlab.com>2018-01-09 00:50:54 +0100
committerConstance Okoghenun <cokoghenun@gitlab.com>2018-01-09 00:50:54 +0100
commit85e8acedaf1d4e51335c59a0efa2f95576a4fde3 (patch)
tree9b21d76626568158cf746e2a7eb4853efe3aa0d5 /doc/development
parentcdcade0dd4e748bcf0119b307c06993e0669f507 (diff)
parentbd50ecbad8c00e7c9ab5c60fa8bc839a8905b4ab (diff)
downloadgitlab-ce-move-project-dropdown.tar.gz
Resolved conflictsmove-project-dropdown
Diffstat (limited to 'doc/development')
-rw-r--r--doc/development/README.md1
-rw-r--r--doc/development/doc_styleguide.md4
-rw-r--r--doc/development/fe_guide/axios.md2
-rw-r--r--doc/development/fe_guide/style_guide_scss.md2
-rw-r--r--doc/development/gitaly.md23
-rw-r--r--doc/development/testing_guide/end_to_end_tests.md80
-rw-r--r--doc/development/testing_guide/index.md8
-rw-r--r--doc/development/testing_guide/testing_levels.md3
-rw-r--r--doc/development/utilities.md92
-rw-r--r--doc/development/ux_guide/img/james-mackey.pngbin11147 -> 32552 bytes
-rw-r--r--doc/development/ux_guide/img/karolina-plaskaty.pngbin33498 -> 29553 bytes
-rw-r--r--doc/development/ux_guide/img/matthieu-poirier.pngbin0 -> 32819 bytes
-rw-r--r--doc/development/ux_guide/img/nazim-ramesh.pngbin31163 -> 27475 bytes
-rw-r--r--doc/development/ux_guide/users.md291
-rw-r--r--doc/development/what_requires_downtime.md57
-rw-r--r--doc/development/writing_documentation.md4
16 files changed, 490 insertions, 77 deletions
diff --git a/doc/development/README.md b/doc/development/README.md
index b624aa37c70..12cca9f84b7 100644
--- a/doc/development/README.md
+++ b/doc/development/README.md
@@ -27,6 +27,7 @@ comments: false
## Backend guides
+- [GitLab utilities](utilities.md)
- [API styleguide](api_styleguide.md) Use this styleguide if you are
contributing to the API.
- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers
diff --git a/doc/development/doc_styleguide.md b/doc/development/doc_styleguide.md
index db13e0e6249..9cb1f708a6a 100644
--- a/doc/development/doc_styleguide.md
+++ b/doc/development/doc_styleguide.md
@@ -18,6 +18,10 @@ like `docs.gitlab.com/user/project/merge_requests.html`. With this pattern,
you can immediately tell that you are navigating a user related documentation
and is about the project and its merge requests.
+Do not create summaries of similar types of content (e.g. an index of all articles, videos, etc.),
+rather organise content by its subject (e.g. everything related to CI goes together)
+and cross-link between any related content.
+
The table below shows what kind of documentation goes where.
| Directory | What belongs here |
diff --git a/doc/development/fe_guide/axios.md b/doc/development/fe_guide/axios.md
index 962fe3dcec9..1daa6758171 100644
--- a/doc/development/fe_guide/axios.md
+++ b/doc/development/fe_guide/axios.md
@@ -11,7 +11,7 @@ This exported module should be used instead of directly using `axios` to ensure
## Usage
```javascript
- import axios from '~/lib/utils/axios_utils';
+ import axios from './lib/utils/axios_utils';
axios.get(url)
.then((response) => {
diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md
index 77b308c4a43..86a8b4135af 100644
--- a/doc/development/fe_guide/style_guide_scss.md
+++ b/doc/development/fe_guide/style_guide_scss.md
@@ -216,7 +216,7 @@ If you want a line or set of lines to be ignored by the linter, you can use
```scss
// This lint rule is disabled because the class name comes from a gem.
// scss-lint:disable SelectorFormat
-.ui_charcoal {
+.ui_indigo {
background-color: #333;
}
// scss-lint:enable SelectorFormat
diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md
index ca2048c7019..26abf967dcf 100644
--- a/doc/development/gitaly.md
+++ b/doc/development/gitaly.md
@@ -97,6 +97,29 @@ describe 'Gitaly Request count tests' do
end
```
+## Running tests with a locally modified version of Gitaly
+
+Normally, gitlab-ce/ee tests use a local clone of Gitaly in `tmp/tests/gitaly`
+pinned at the version specified in GITALY_SERVER_VERSION. If you want
+to run tests locally against a modified version of Gitaly you can
+replace `tmp/tests/gitaly` with a symlink.
+
+```shell
+rm -rf tmp/tests/gitaly
+ln -s /path/to/gitaly tmp/tests/gitaly
+```
+
+Make sure you run `make` in your local Gitaly directory before running
+tests. Otherwise, Gitaly will fail to boot.
+
+If you make changes to your local Gitaly in between test runs you need
+to manually run `make` again.
+
+Note that CI tests will not use your locally modified version of
+Gitaly. To use a custom Gitaly version in CI you need to update
+GITALY_SERVER_VERSION. You can use the format `=revision` to use a
+non-tagged commit from https://gitlab.com/gitlab-org/gitaly in CI.
+
---
[Return to Development documentation](README.md)
diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end_tests.md
new file mode 100644
index 00000000000..abe5b06e0f0
--- /dev/null
+++ b/doc/development/testing_guide/end_to_end_tests.md
@@ -0,0 +1,80 @@
+# End-to-End Testing
+
+## What is End-to-End testing?
+
+End-to-End testing is a strategy used to check whether your application works
+as expected across entire software stack and architecture, including
+integration of all microservices and components that are supposed to work
+together.
+
+## How do we test GitLab?
+
+We use [Omnibus GitLab][omnibus-gitlab] to build GitLab packages and then we
+test these packages using [GitLab QA][gitlab-qa] project, which is entirely
+black-box, click-driven testing framework.
+
+### Testing nightly builds
+
+We run scheduled pipeline each night to test nightly builds created by Omnibus.
+You can find these nightly pipelines at [GitLab QA pipelines page][gitlab-qa-pipelines].
+
+### Testing code in merge requests
+
+It is possible to run end-to-end tests (eventually being run within a
+[GitLab QA pipeline][gitlab-qa-pipelines]) for a merge request by triggering
+the `package-qa` manual action, that should be present in a merge request
+widget.
+
+Mmanual action that starts end-to-end tests is also available in merge requests
+in Omnibus GitLab project.
+
+Below you can read more about how to use it and how does it work.
+
+#### How does it work?
+
+Currently, we are using _multi-project pipeline_-like approach to run QA
+pipelines.
+
+1. Developer triggers a manual action, that can be found in CE and EE merge
+requests. This starts a chain of pipelines in multiple projects.
+
+1. The script being executed triggers a pipeline in GitLab Omnibus and waits
+for the resulting status. We call this a _status attribution_.
+
+1. GitLab packages are being built in Omnibus pipeline. Packages are going to be
+pushed to Container Registry.
+
+1. When packages are ready, and available in the registry, a final step in the
+pipeline, that is now running in Omnibus, triggers a new pipeline in the GitLab
+QA project. It also waits for a resulting status.
+
+1. GitLab QA pulls images from the registry, spins-up containers and runs tests
+against a test environment that has been just orchestrated by the `gitlab-qa`
+tool.
+
+1. The result of the GitLab QA pipeline is being propagated upstream, through
+Omnibus, back to CE / EE merge request.
+
+#### How do I write tests?
+
+In order to write new tests, you first need to learn more about GitLab QA
+architecture. See the [documentation about it][gitlab-qa-architecture] in
+GitLab QA project.
+
+Once you decided where to put test environment orchestration scenarios and
+instance specs, take a look at the [relevant documentation][instance-qa-readme]
+and examples in [the `qa/` directory][instance-qa-examples].
+
+## Where can I ask for help?
+
+You can ask question in the `#qa` channel on Slack (GitLab internal) or you can
+find an issue you would like to work on in [the issue tracker][gitlab-qa-issues]
+and start a new discussion there.
+
+[omnibus-gitlab]: https://gitlab.com/gitlab-org/omnibus-gitlab
+[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa
+[gitlab-qa-pipelines]: https://gitlab.com/gitlab-org/gitlab-qa/pipelines
+[gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md
+[gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues
+[instance-qa-readme]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md
+[instance-qa-examples]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa
diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md
index 65386f231a0..74d09eb91ff 100644
--- a/doc/development/testing_guide/index.md
+++ b/doc/development/testing_guide/index.md
@@ -65,6 +65,13 @@ Everything you should know about how to test Rake tasks.
---
+## [End-to-end tests](end_to_end_tests.md)
+
+Everything you should know about how to run end-to-end tests using
+[GitLab QA][gitlab-qa] testing framework.
+
+---
+
## Spinach (feature) tests
GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426)
@@ -89,3 +96,4 @@ test should be re-implemented using RSpec instead.
[Capybara]: https://github.com/teamcapybara/capybara
[Karma]: http://karma-runner.github.io/
[Jasmine]: https://jasmine.github.io/
+[gitlab-qa]: https://gitlab.com/gitlab-org/gitlab-qa
diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md
index 1cbd4350284..4adf0dc7c7a 100644
--- a/doc/development/testing_guide/testing_levels.md
+++ b/doc/development/testing_guide/testing_levels.md
@@ -121,6 +121,9 @@ running feature tests (i.e. using Capybara) against it.
The actual test scenarios and steps are [part of GitLab Rails] so that they're
always in-sync with the codebase.
+Read a separate document about [end-to-end tests](end_to_end_tests.md) to
+learn more.
+
[multiple pieces]: ../architecture.md#components
[GitLab Shell]: https://gitlab.com/gitlab-org/gitlab-shell
[GitLab Workhorse]: https://gitlab.com/gitlab-org/gitlab-workhorse
diff --git a/doc/development/utilities.md b/doc/development/utilities.md
new file mode 100644
index 00000000000..951c3ef85ce
--- /dev/null
+++ b/doc/development/utilities.md
@@ -0,0 +1,92 @@
+# GitLab utilities
+
+We developed a number of utilities to ease development.
+
+## [`MergeHash`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/utils/merge_hash.rb)
+
+* Deep merges an array of hashes:
+
+ ``` ruby
+ Gitlab::Utils::MergeHash.merge(
+ [{ hello: ["world"] },
+ { hello: "Everyone" },
+ { hello: { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] } },
+ "Goodbye", "Hallo"]
+ )
+ ```
+
+ Gives:
+
+ ``` ruby
+ [
+ {
+ hello:
+ [
+ "world",
+ "Everyone",
+ { greetings: ['Bonjour', 'Hello', 'Hallo', 'Dzien dobry'] }
+ ]
+ },
+ "Goodbye"
+ ]
+ ```
+
+* Extracts all keys and values from a hash into an array:
+
+ ``` ruby
+ Gitlab::Utils::MergeHash.crush(
+ { hello: "world", this: { crushes: ["an entire", "hash"] } }
+ )
+ ```
+
+ Gives:
+
+ ``` ruby
+ [:hello, "world", :this, :crushes, "an entire", "hash"]
+ ```
+
+## [`StrongMemoize`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/utils/strong_memoize.rb)
+
+* Memoize the value even if it is `nil` or `false`.
+
+ We often do `@value ||= compute`, however this doesn't work well if
+ `compute` might eventually give `nil` and we don't want to compute again.
+ Instead we could use `defined?` to check if the value is set or not.
+ However it's tedious to write such pattern, and `StrongMemoize` would
+ help us use such pattern.
+
+ Instead of writing patterns like this:
+
+ ``` ruby
+ class Find
+ def result
+ return @result if defined?(@result)
+
+ @result = search
+ end
+ end
+ ```
+
+ We could write it like:
+
+ ``` ruby
+ class Find
+ include Gitlab::Utils::StrongMemoize
+
+ def result
+ strong_memoize(:result) do
+ search
+ end
+ end
+ end
+ ```
+
+* Clear memoization
+
+ ``` ruby
+ class Find
+ include Gitlab::Utils::StrongMemoize
+ end
+
+ Find.new.clear_memoization(:result)
+ ```
diff --git a/doc/development/ux_guide/img/james-mackey.png b/doc/development/ux_guide/img/james-mackey.png
index 6db257c5b39..c8f9097f69f 100644
--- a/doc/development/ux_guide/img/james-mackey.png
+++ b/doc/development/ux_guide/img/james-mackey.png
Binary files differ
diff --git a/doc/development/ux_guide/img/karolina-plaskaty.png b/doc/development/ux_guide/img/karolina-plaskaty.png
index 2e356c99762..ae2e98b7bad 100644
--- a/doc/development/ux_guide/img/karolina-plaskaty.png
+++ b/doc/development/ux_guide/img/karolina-plaskaty.png
Binary files differ
diff --git a/doc/development/ux_guide/img/matthieu-poirier.png b/doc/development/ux_guide/img/matthieu-poirier.png
new file mode 100644
index 00000000000..dd21948ebe2
--- /dev/null
+++ b/doc/development/ux_guide/img/matthieu-poirier.png
Binary files differ
diff --git a/doc/development/ux_guide/img/nazim-ramesh.png b/doc/development/ux_guide/img/nazim-ramesh.png
index 01ba0391630..cc3e197679d 100644
--- a/doc/development/ux_guide/img/nazim-ramesh.png
+++ b/doc/development/ux_guide/img/nazim-ramesh.png
Binary files differ
diff --git a/doc/development/ux_guide/users.md b/doc/development/ux_guide/users.md
index fce882a45f1..6afb33cfc36 100644
--- a/doc/development/ux_guide/users.md
+++ b/doc/development/ux_guide/users.md
@@ -1,127 +1,252 @@
# UX Personas
* [Nazim Ramesh](#nazim-ramesh)
- - Small to medium size organisations using GitLab CE
+ - Small to medium size organizations using GitLab CE
+* [Matthieu Poirier](#matthieu-poirier)
+ - Responsible for managing and maintaining GitLab installation
+ - Any size organization
+ - Using CE or EE
* [James Mackey](#james-mackey)
- - Medium to large size organisations using CE or EE
- - Small organisations using EE
+ - Medium to large size organizations using CE or EE
+ - Small organizations using EE
* [Karolina Plaskaty](#karolina-plaskaty)
- Using GitLab.com for personal/hobby projects
- Would like to use GitLab at work
- - Working for a medium to large size organisation
+ - Working within a medium to large size organization
---
## Nazim Ramesh
-- Small to medium size organisations using GitLab CE
+- Small to medium size organizations using GitLab CE
-<img src="img/nazim-ramesh.png" width="300px">
+![nazim-ramesh](img/nazim-ramesh.png)
### Demographics
-- **Age**<br>32 years old
-- **Location**<br>Germany
-- **Education**<br>Bachelor of Science in Computer Science
-- **Occupation**<br>Full-stack web developer
-- **Programming experience**<br>Over 10 years
-- **Frequently used programming languages**<br>JavaScript, SQL, PHP
-- **Hobbies / interests**<br>Functional programming, open source, gaming, web development and web security.
+**Age**
+
+32 years old
+
+**Location**
+
+Germany
+
+**Education**
+
+Bachelor of Science in Computer Science
+
+**Occupation**
+
+Full-stack web developer
+
+**Programming experience**
+
+Over 10 years
+
+**Frequently used programming languages**
+
+JavaScript, SQL, PHP
+
+**Hobbies / interests**
+
+Functional programming, open source, gaming, web development, and web security.
### Motivations
-Nazim works for a software development company which currently hires around 80 people. When Nazim first joined the company, the engineering team were using Subversion (SVN) as their primary form of source control. However, Nazim felt SVN was not flexible enough to work with many feature branches and noticed that developers with less experience of source control struggled with the central-repository nature of SVN. Armed with a wishlist of features, Nazim began comparing source control tools. A search for “self-hosted Git server repository management” returned GitLab. In his own words, Nazim explains why he wanted the engineering team to start using GitLab:
+Nazim works for a software development company which currently hires around 80 people. When Nazim first joined the company, the engineering team were using Subversion (SVN) as their primary form of source control. However, Nazim felt SVN was not flexible enough to work with many feature branches and noticed that developers with less experience of source control struggled with the central-repository nature of SVN. Armed with a wish list of features, Nazim began comparing source control tools. A search for "self-hosted Git server repository management" returned GitLab. In his own words, Nazim explains why he wanted the engineering team to start using GitLab:
->
-“I wanted them to switch away from SVN. I needed a server application to manage repositories. The common tools that were around just didn’t meet the requirements. Most of them were too simple or plain...GitLab provided all the required features. Also costs had to be low, since we don’t have a big budget for those things...the Community Edition was perfect in this regard.”
->
+>"I wanted them to switch away from SVN. I needed a server application to manage repositories. The common tools that were around just didn't meet the requirements. Most of them were too simple or plain...GitLab provided all the required features. Also, costs had to be low since we don't have a big budget for those things...the Community Edition was perfect in this regard."
In his role as a full-stack web developer, Nazim could recommend products that he would like the engineering team to use, but final approval lay with his line manager, Mike, VP of Engineering. Nazim recalls that he was met with reluctance from his colleagues when he raised moving to Git and using GitLab.
->
-“The biggest challenge...why should we change anything at all from the status quo? We needed to switch from SVN to Git. They knew they needed to learn Git and a Git workflow...using Git was scary to my colleagues...they thought it was more complex than SVN to use.”
->
+>"The biggest challenge...why should we change anything at all from the status quo? We needed to switch from SVN to Git. They knew they needed to learn Git and a Git workflow...using Git was scary to my colleagues...they thought it was more complex than SVN to use."
Undeterred, Nazim decided to migrate a couple of projects across to GitLab.
->
-“Old SVN users couldn’t see the benefits of Git at first. It took a month or two to convince them.”
->
+>"Old SVN users couldn't see the benefits of Git at first. It took a month or two to convince them."
-Slowly, by showing his colleagues how easy it was to use Git, the majority of the team’s projects were migrated to GitLab.
+Slowly, by showing his colleagues how easy it was to use Git, the majority of the team's projects were migrated to GitLab.
-The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company’s decision to move to GitLab.
+The engineering team have been using GitLab CE for around 2 years now. Nazim credits himself as being entirely responsible for his company's decision to move to GitLab.
### Frustrations
#### Adoption to GitLab has been slow
-Not only has the engineering team had to get to grips with Git, they’ve also had to adapt to using GitLab. Due to lack of training and existing skills in other tools, the full feature set of GitLab CE is not being utilised. Nazim sold GitLab to his manager as an ‘all in one’ tool which would replace multiple tools used within the company, thus saving costs. Nazim hasn’t had the time to integrate the legacy tools to GitLab and he’s struggling to convince his peers to change their habits.
+Not only has the engineering team had to get to grips with Git, they've also had to adapt to using GitLab. Due to lack of training and existing skills in other tools, the full feature set of GitLab CE is not being utilized. Nazim sold GitLab to his manager as an ‘all in one' tool which would replace multiple tools used within the company, thus saving costs. Nazim hasn't had the time to integrate the legacy tools to GitLab and he's struggling to convince his peers to change their habits.
#### Missing Features
-Nazim’s company want GitLab to be able to do everything. There isn’t a large budget for software, so they’re selective about what tools are implemented. It needs to add real value to the company. In order for GitLab to be widely adopted and to meet the requirements of different roles within the company, it needs a host of features. When an individual within Nazim’s company wants to know if GitLab has a specific feature or does a particular thing, Nazim is the person to ask. He becomes the point of contact to investigate, build or sometimes just raise the feature request. Nazim gets frustrated when GitLab isn’t able to do what he or his colleagues need it to do.
+Nazim's company want GitLab to be able to do everything. There isn't a large budget for software, so they're selective about what tools are implemented. It needs to add real value to the company. In order for GitLab to be widely adopted and to meet the requirements of different roles within the company, it needs a host of features. When an individual within Nazim's company wants to know if GitLab has a specific feature or does a particular thing, Nazim is the person to ask. He becomes the point of contact to investigate, build or sometimes just raise the feature request. Nazim gets frustrated when GitLab isn't able to do what he or his colleagues need it to do.
#### Regressions and bugs
-Nazim often has to calm down his colleagues, when a release contains regressions or new bugs. As he puts it “every new version adds something awesome, but breaks something”. He feels that “old issues for "minor" annoyances get quickly buried in the mass of open issues and linger for a very long time. More generally, I have the feeling that GitLab focus on adding new functionalities, but overlook a bunch of annoying minor regressions or introduced bugs.” Due to limited resource and expertise within the team, not only is it difficult to remain up-to-date with the frequent release cycle, it’s also counterproductive to fix workflows every month.
+Nazim often has to calm down his colleagues, when a release contains regressions or new bugs. As he puts it "every new version adds something awesome, but breaks something". He feels that "old issues for minor annoyances get quickly buried in the mass of open issues and linger for a very long time. More generally, I have the feeling that GitLab focus on adding new functionalities, but overlook a bunch of annoying minor regressions or introduced bugs." Due to limited resource and expertise within the team, not only is it difficult to remain up-to-date with the frequent release cycle, it's also counterproductive to fix workflows every month.
#### Uses too much RAM and CPU
->
-“Memory usages mean that if we host it from a cloud based host like AWS, we spend almost as much on the instance as what we would pay GitHub”
->
+>"Memory usages mean that if we host it from a cloud-based host like AWS, we spend almost as much on the instance as what we would pay GitHub"
+
#### UI/UX
-GitLab’s interface initially attracted Nazim when he was comparing version control software. He thought it would help his less technical colleagues to adapt to using Git and perhaps, GitLab could be rolled out to other areas of the business, beyond engineering. However, using GitLab’s interface daily has left him frustrated at the lack of personalisation / control over his user experience. He’s also regularly lost in a maze of navigation. Whilst he acknowledges that GitLab listens to its users and that the interface is improving, he becomes annoyed when the changes are too progressive. “Too frequent UI changes. Most of them tend to turn out great after a few cycles of fixes, but the frequency is still far too high for me to feel comfortable to always stay on the current release.”
+GitLab's interface initially attracted Nazim when he was comparing version control software. He thought it would help his less technical colleagues to adapt to using Git and perhaps, GitLab could be rolled out to other areas of the business, beyond engineering. However, using GitLab's interface daily has left him frustrated at the lack of personalization/control over his user experience. He's also regularly lost in a maze of navigation. Whilst he acknowledges that GitLab listens to its users and that the interface is improving, he becomes annoyed when the changes are too progressive. "Too frequent UI changes. Most of them tend to turn out great after a few cycles of fixes, but the frequency is still far too high for me to feel comfortable to always stay on the current release."
### Goals
* To convince his colleagues to fully adopt GitLab CE, thus improving workflow and collaboration.
-* To use a feature rich version control platform that covers all stages of the development lifecycle, in order to reduce dependencies on other tools.
+* To use a feature-rich version control platform that covers all stages of the development lifecycle, in order to reduce dependencies on other tools.
* To use an intuitive and stable product, so he can spend more time on his core job responsibilities and less time bug-fixing, guiding colleagues, etc.
---
+## Matthieu Poirier
+- Responsible for managing and maintaining GitLab installation
+- Any size organization
+- Using CE or EE
+
+![matthieu-poirier](img/matthieu-poirier.png)
+
+### Demographics
+
+**Age**
+
+42 years old
+
+**Location**
+
+France
+
+**Education**
+
+Masters Degree in Computer Science
+
+**Occupation**
+
+DevOps Engineer
+
+**Programming experience**
+
+Over 10 years
+
+**Frequently used programming languages**
+
+JavaScript, SQL, PHP and Node.js
+
+**Hobbies / interests**
+
+Functional programming, data analysis, building apps, and tools.
+
+### Motivations
+Matthieu works in DevOps for a web services company which currently hires 90 staff. When Matthieu first joined the company, he was responsible for managing a custom built in-house bug tracking tool and release management system. Over time, as the company grew, his colleagues requested more features and tools to help them in their day-to-day work. To meet their needs, Matthieu was forced to "hack together" a solution. In his own words, Matthieu explains that it became:
+
+>"...a huge pain managing access to all the individual pieces. In addition, they didn't have any integration with each other, nobody ended up using them and we couldn't do any workflows with merge requests and the like. I was sick of managing all those separate parts and wanted to move to a single platform that would handle it all."
+
+
+He further explains that he wanted to introduce "better, easier, more formal code reviews" and to start using continuous integration and deployment.
+
+Matthieu tried to find a platform which would consolidate the company's existing toolset, and centralize code, documentation, and issues. He discovered GitHub, but the price was an issue:
+
+>"We needed to host our code on-site and wanted GitHub Enterprise functionality without the GitHub Enterprise costs."
+
+
+Not only was GitLab cheaper than GitHub, it was also more cost-effective than maintaining multiple tools. Subsequently, Matthieu found it easy to sell the merits of GitLab to his manager.
+
+Matthieu describes GitLab as:
+
+>"the only tool that offers the real feeling of having everything you need in one place."
+
+
+He credits himself as being entirely responsible for moving his company to GitLab.
+
+### Frustrations
+#### Updating to the latest release
+Matthieu introduced his company to GitLab. He is responsible for maintaining and managing the company's installation in addition to his day job. He feels updates are too frequent and he doesn't always have sufficient time to update GitLab. As a result, he's not up to date with releases.
+
+Matthieu tried to set up automatic updates, however, as he isn't a Systems Administrator, he wasn't confident in his set-up. He feels he should be able to "upgrade without users even noticing" but hasn't figured out how to do this yet. Matthieu would like the "update process to be triggered from the Admin Panel, perhaps accompanied with a changelog and the option to skip updates."
+
+Matthieu is looking for confirmation that his update procedure is "secure and efficient" so more tutorials related to this topic would be useful to him.
+
+#### Configuration
+Matthieu dislikes using the combination of gitlab.rb and the UI for changing settings. He explains that it "would be nice to be able to configure more from the Admin UI rather than just the config files."
+
+#### Creating a backup
+Matthieu explains that the "backup solution is not well integrated into the UI", for example, he "cannot see if backups succeeded" or whether they have been rolled back to via the UI.
+
+#### Onboarding
+It's Matthieu's responsibility to get teams across his organization up and running with GitLab. He explains that whilst many teams might be leveraging GitLab, they are:
+
+>"..not aware of GitLab's powerful CI or our omnibus install of Mattermost...It would be nice to have a tutorial type walkthrough available when a new user logs in on how to get started with all these features. AutoDevOps may solve some of this, but GitLab has many powerful features wrapped up into it and some [teams] may just think that it is only a Git repo similar to GitHub."
+
+
+He states that there has been: "a sluggishness of others to adapt" and it's "a low-effort adaptation at that."
+
+### Goals
+* To save time. One of the reasons Matthieu moved his company to GitLab was to reduce the effort it took him to manage and configure multiple tools, thus saving him time. He has to balance his day job in addition to managing the company's GitLab installation and onboarding new teams to GitLab.
+* To use a platform which is easy to manage. Matthieu isn't a Systems Administrator, and when updating GitLab, creating backups, etc. He would prefer to work within GitLab's UI. Explanations / guided instructions when configuring settings in GitLab's interface would really help Matthieu. He needs reassurance that what he is about to change is
+
+1. the right setting
+2. will provide him with the desired result he wants.
+
+* Matthieu needs to educate his colleagues about GitLab. Matthieu's colleagues won't adopt GitLab as they're unaware of its capabilities and the positive impact it could have on their work. Matthieu needs support in getting this message across to them.
+
+---
## James Mackey
-- Medium to large size organisations using CE or EE
-- Small organisations using EE
+- Medium to large size organizations using CE or EE
+- Small organizations using EE
-<img src="img/james-mackey.png" width="300px">
+![james-mackey.png](img/james-mackey.png)
### Demographics
-- **Age**<br>36 years old
-- **Location**<br>US
-- **Education**<br>Masters degree in Computer Science
-- **Occupation**<br>Full-stack web developer
-- **Programming experience**<br>Over 10 years
-- **Frequently used programming languages**<br>JavaScript, SQL, Node.js, Java, PHP, Python
-- **Hobbies / interests**<br>DevOps, open source, web development, science, automation and electronics.
+**Age**
+
+36 years old
+
+**Location**
+
+US
+
+**Education**
+
+Masters degree in Computer Science
+
+**Occupation**
+
+Full-stack web developer
+
+**Programming experience**
+
+Over 10 years
+
+**Frequently used programming languages**
+
+JavaScript, SQL, Node.js, Java, PHP, Python
+
+**Hobbies / interests**
+
+DevOps, open source, web development, science, automation, and electronics.
### Motivations
-James works for a research company which currently hires around 800 staff. He began using GitLab.com back in 2013 for his own open source, hobby projects and loved “the simplicity of installation, administration and use”. After using GitLab for over a year, he began to wonder about using it at work. James explains:
+James works for a research company which currently hires around 800 staff. He began using GitLab.com back in 2013 for his own open source, hobby projects and loved "the simplicity of installation, administration and use". After using GitLab for over a year, he began to wonder about using it at work. James explains:
->
-“We first installed the CE edition...on a staging server for a PoC and asked a beta team to use it, specifically for the Merge Request features. Soon other teams began asking us to be beta users too, because the team that was already using GitLab was really enjoying it.”
->
+>"We first installed the CE edition...on a staging server for a PoC and asked a beta team to use it, specifically for the Merge Request features. Soon other teams began asking us to be beta users too because the team that was already using GitLab was really enjoying it."
-James and his colleagues also reviewed competitor products including GitHub Enterprise, but they found it “less innovative and with considerable costs...GitLab had the features we wanted at a much lower cost per head than GitHub”.
+
+James and his colleagues also reviewed competitor products including GitHub Enterprise, but they found it "less innovative and with considerable costs...GitLab had the features we wanted at a much lower cost per head than GitHub".
The company James works for provides employees with a discretionary budget to spend how they want on software, so James and his team decided to upgrade to EE.
-James feels partially responsible for his organisation’s decision to start using GitLab.
+James feels partially responsible for his organization's decision to start using GitLab.
+
+>"It's still up to the teams themselves [to decide] which tools to use. We just had a great experience moving our daily development to GitLab, so other teams have followed the path or are thinking about switching."
->
-“It's still up to the teams themselves [to decide] which tools to use. We just had a great experience moving our daily development to GitLab, so other teams have followed the path or are thinking about switching.”
->
### Frustrations
#### Third Party Integration
-Some of GitLab EE’s features are too basic, in particular, issues boards which do not have the level of reporting that James and his team need. Subsequently, they still need to use GitLab EE in conjunction with other tools, such as JIRA. Whilst James feels it isn’t essential for GitLab to meet all his needs (his company are happy for him to use, and pay for, multiple tools), he sometimes isn’t sure what is/isn’t possible with plugins and what level of custom development he and his team will need to do.
+Some of GitLab EE's features are too basic, in particular, issues boards which do not have the level of reporting that James and his team need. Subsequently, they still need to use GitLab EE in conjunction with other tools, such as JIRA. Whilst James feels it isn't essential for GitLab to meet all his needs (his company are happy for him to use, and pay for, multiple tools), he sometimes isn't sure what is/isn't possible with plugins and what level of custom development he and his team will need to do.
#### UX/UI
-James and his team use CI quite heavily for several projects. Whilst they’ve welcomed improvements to the builds and pipelines interface, they still have some difficulty following build process on the different tabs under Pipelines. Some confusion has arisen from not knowing where to find different pieces of information or how to get to the next stages logs from the current stage’s log output screen. They feel more intuitive linking and flow may alleviate the problem. Generally, they feel GitLab’s navigation needs to reviewed and optimised.
+James and his team use CI quite heavily for several projects. Whilst they've welcomed improvements to the builds and pipelines interface, they still have some difficulty following build process on the different tabs under Pipelines. Some confusion has arisen from not knowing where to find different pieces of information or how to get to the next stages logs from the current stage's log output screen. They feel more intuitive linking and flow may alleviate the problem. Generally, they feel GitLab's navigation needs to reviewed and optimized.
#### Permissions
->
-“There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for Gitlab CI/build pipelines.”
->
+>"There is no granular control over user or group permissions. The permissions for a project are too tightly coupled to the permissions for Gitlab CI/build pipelines."
+
### Goals
-* To be able to integrate third party tools easily with GitLab EE and to create custom integrations and patches where needed.
+* To be able to integrate third-party tools easily with GitLab EE and to create custom integrations and patches where needed.
* To use GitLab EE primarily for code hosting, merge requests, continuous integration and issue management. James and his team want to be able to understand and use these particular features easily.
* To able to share one instance of GitLab EE with multiple teams across the business. Advanced user management, the ability to separate permissions on different parts of the source code, etc are important to James.
@@ -130,36 +255,56 @@ James and his team use CI quite heavily for several projects. Whilst they’ve w
## Karolina Plaskaty
- Using GitLab.com for personal/hobby projects
- Would like to use GitLab at work
-- Working for a medium to large size organisation
+- Working within a medium to large size organization
-<img src="img/karolina-plaskaty.png" width="300px">
+![karolina-plaskaty.png](img/karolina-plaskaty.png)
### Demographics
-- **Age**<br>26 years old
-- **Location**<br>UK
-- **Education**<br>Self taught
-- **Occupation**<br>Junior web-developer
-- **Programming experience**<br>6 years
-- **Frequently used programming languages**<br>JavaScript and SQL
-- **Hobbies / interests**<br>Web development, mobile development, UX, open source, gaming and travel.
+**Age**
+
+26 years old
+
+**Location**
+
+UK
+
+**Education**
+
+Self taught
+
+**Occupation**
+
+Junior web-developer
+
+**Programming experience**
+
+6 years
+
+**Frequently used programming languages**
+
+JavaScript and SQL
+
+**Hobbies / interests**
+
+Web development, mobile development, UX, open source, gaming, and travel.
### Motivations
-Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab’s values and the fact that it isn’t a “behemoth of a company”. She explains that “displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose Gitlab personally and to recommend it at work.” She’s also an avid reader of GitLab’s blog.
+Karolina has been using GitLab.com for around a year. She roughly spends 8 hours every week programming, of that, 2 hours is spent contributing to open source projects. Karolina contributes to open source projects to gain programming experience and to give back to the community. She likes GitLab.com for its free private repositories and range of features which provide her with everything she needs for her personal projects. Karolina is also a massive fan of GitLab's values and the fact that it isn't a "behemoth of a company". She explains that "displaying every single thing (doc, culture, assumptions, development...) in the open gives me greater confidence to choose Gitlab personally and to recommend it at work." She's also an avid reader of GitLab's blog.
-Karolina works for a software development company which currently hires around 500 people. Karolina would love to use GitLab at work but the company has used GitHub Enterprise for a number of years. She describes management at her company as “old fashioned” and explains that it’s “less of a technical issue and more of a cultural issue” to convince upper management to move to GitLab. Karolina is also relatively new to the company so she’s apprehensive about pushing too hard to change version control platforms.
+Karolina works for a software development company which currently hires around 500 people. Karolina would love to use GitLab at work but the company has used GitHub Enterprise for a number of years. She describes management at her company as "old fashioned" and explains that it's "less of a technical issue and more of a cultural issue" to convince upper management to move to GitLab. Karolina is also relatively new to the company so she's apprehensive about pushing too hard to change version control platforms.
### Frustrations
#### Unable to use GitLab at work
-Karolina wants to use GitLab at work but isn’t sure how to approach the subject with management. In her current role, she doesn’t feel that she has the authority to request GitLab.
+Karolina wants to use GitLab at work but isn't sure how to approach the subject with management. In her current role, she doesn't feel that she has the authority to request GitLab.
#### Performance
-GitLab.com is frequently slow and unavailable. Karolina has also heard that GitLab is a “memory hog” which has deterred her from running GitLab on her own machine for just hobby / personal projects.
+GitLab.com is frequently slow and unavailable. Karolina has also heard that GitLab is a "memory hog" which has deterred her from running GitLab on her own machine for just hobby / personal projects.
#### UX/UI
-Karolina has an interest in UX and therefore has strong opinions about how GitLab should look and feel. She feels the interface is cluttered, “it has too many links/buttons” and the navigation “feels a bit weird sometimes. I get lost if I don’t pay attention.” As Karolina also enjoys contributing to open-source projects, it’s important to her that GitLab is well designed for public repositories, she doesn’t feel that GitLab currently achieves this.
+Karolina has an interest in UX and therefore has strong opinions about how GitLab should look and feel. She feels the interface is cluttered, "it has too many links/buttons" and the navigation "feels a bit weird sometimes. I get lost if I don't pay attention." As Karolina also enjoys contributing to open-source projects, it's important to her that GitLab is well designed for public repositories, she doesn't feel that GitLab currently achieves this.
### Goals
* To develop her programming experience and to learn from other developers.
* To contribute to both her own and other open source projects.
-* To use a fast and intuitive version control platform.
+* To use a fast and intuitive version control platform. \ No newline at end of file
diff --git a/doc/development/what_requires_downtime.md b/doc/development/what_requires_downtime.md
index 05e0a64af18..9d0c62ecc35 100644
--- a/doc/development/what_requires_downtime.md
+++ b/doc/development/what_requires_downtime.md
@@ -195,6 +195,63 @@ end
And that's it, we're done!
+## Changing Column Types For Large Tables
+
+While `change_column_type_concurrently` can be used for changing the type of a
+column without downtime it doesn't work very well for large tables. Because all
+of the work happens in sequence the migration can take a very long time to
+complete, preventing a deployment from proceeding.
+`change_column_type_concurrently` can also produce a lot of pressure on the
+database due to it rapidly updating many rows in sequence.
+
+To reduce database pressure you should instead use
+`change_column_type_using_background_migration` when migrating a column in a
+large table (e.g. `issues`). This method works similar to
+`change_column_type_concurrently` but uses background migration to spread the
+work / load over a longer time period, without slowing down deployments.
+
+Usage of this method is fairly simple:
+
+```ruby
+class ExampleMigration < ActiveRecord::Migration
+ include Gitlab::Database::MigrationHelpers
+
+ disable_ddl_transaction!
+
+ class Issue < ActiveRecord::Base
+ self.table_name = 'issues'
+
+ include EachBatch
+
+ def self.to_migrate
+ where('closed_at IS NOT NULL')
+ end
+ end
+
+ def up
+ change_column_type_using_background_migration(
+ Issue.to_migrate,
+ :closed_at,
+ :datetime_with_timezone
+ )
+ end
+
+ def down
+ change_column_type_using_background_migration(
+ Issue.to_migrate,
+ :closed_at,
+ :datetime
+ )
+ end
+end
+```
+
+This would change the type of `issues.closed_at` to `timestamp with time zone`.
+
+Keep in mind that the relation passed to
+`change_column_type_using_background_migration` _must_ include `EachBatch`,
+otherwise it will raise a `TypeError`.
+
## Adding Indexes
Adding indexes is an expensive process that blocks INSERT and UPDATE queries for
diff --git a/doc/development/writing_documentation.md b/doc/development/writing_documentation.md
index 43a79ffcaa5..133ac0234cf 100644
--- a/doc/development/writing_documentation.md
+++ b/doc/development/writing_documentation.md
@@ -15,7 +15,7 @@ request introducing these changes must be accompanied by the documentation
(either updating existing ones or creating new ones). This is also valid when
changes are introduced to the UI.
-The one resposible for writing the first piece of documentation is the developer who
+The one responsible for writing the first piece of documentation is the developer who
wrote the code. It's the job of the Product Manager to ensure all features are
shipped with its docs, whether is a small or big change. At the pace GitLab evolves,
this is the only way to keep the docs up-to-date. If you have any questions about it,
@@ -31,7 +31,7 @@ Every major feature (regardless if present in GitLab Community or Enterprise edi
should present, at the beginning of the document, two main sections: **overview** and
**use cases**. Every GitLab EE-only feature should also contain these sections.
-**Overview**: at the name suggests, the goal here is to provide an overview of the feature.
+**Overview**: as the name suggests, the goal here is to provide an overview of the feature.
Describe what is it, what it does, why it is important/cool/nice-to-have,
what problem it solves, and what you can do with this feature that you couldn't
do before.