diff options
Diffstat (limited to 'doc/development')
| -rw-r--r-- | doc/development/changelog.md | 109 | ||||
| -rw-r--r-- | doc/development/frontend.md | 49 | ||||
| -rw-r--r-- | doc/development/merge_request_performance_guidelines.md | 4 | ||||
| -rw-r--r-- | doc/development/performance.md | 1 | ||||
| -rw-r--r-- | doc/development/polling.md | 41 | ||||
| -rw-r--r-- | doc/development/profiling.md | 2 | ||||
| -rw-r--r-- | doc/development/query_recorder.md | 29 | ||||
| -rw-r--r-- | doc/development/testing.md | 4 |
8 files changed, 210 insertions, 29 deletions
diff --git a/doc/development/changelog.md b/doc/development/changelog.md index c71858c6a24..ce39a379a0e 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -1,7 +1,7 @@ -# Generate a changelog entry +# Changelog entries -This guide contains instructions for generating a changelog entry data file, as -well as information and history about our changelog process. +This guide contains instructions for when and how to generate a changelog entry +file, as well as information and history about our changelog process. ## Overview @@ -19,19 +19,71 @@ author: Ozzy Osbourne The `merge_request` value is a reference to a merge request that adds this entry, and the `author` key is used to give attribution to community -contributors. Both are optional. +contributors. **Both are optional**. Community contributors and core team members are encouraged to add their name to -the `author` field. GitLab team members should not. - -If you're working on the GitLab EE repository, the entry will be added to -`changelogs/unreleased-ee/` instead. +the `author` field. GitLab team members **should not**. [changelog.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG.md [unreleased]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/changelogs/ [YAML]: https://en.wikipedia.org/wiki/YAML -## Instructions +## What warrants a changelog entry? + +- Any user-facing change **should** have a changelog entry. Example: "GitLab now + uses system fonts for all text." +- A fix for a regression introduced and then fixed in the same release (i.e., + fixing a bug introduced during a monthly release candidate) **should not** + have a changelog entry. +- Any developer-facing change (e.g., refactoring, technical debt remediation, + test suite changes) **should not** have a changelog entry. Example: "Reduce + database records created during Cycle Analytics model spec." +- _Any_ contribution from a community member, no matter how small, **may** have + a changelog entry regardless of these guidelines if the contributor wants one. + Example: "Fixed a typo on the search results page. (Jane Smith)" + +## Writing good changelog entries + +A good changelog entry should be descriptive and concise. It should explain the +change to a reader who has _zero context_ about the change. If you have trouble +making it both concise and descriptive, err on the side of descriptive. + +- **Bad:** Go to a project order. +- **Good:** Show a user's starred projects at the top of the "Go to project" + dropdown. + +The first example provides no context of where the change was made, or why, or +how it benefits the user. + +- **Bad:** Copy [some text] to clipboard. +- **Good:** Update the "Copy to clipboard" tooltip to indicate what's being + copied. + +Again, the first example is too vague and provides no context. + +- **Bad:** Fixes and Improves CSS and HTML problems in mini pipeline graph and + builds dropdown. +- **Good:** Fix tooltips and hover states in mini pipeline graph and builds + dropdown. + +The first example is too focused on implementation details. The user doesn't +care that we changed CSS and HTML, they care about the _end result_ of those +changes. + +- **Bad:** Strip out `nil`s in the Array of Commit objects returned from + `find_commits_by_message_with_elastic` +- **Good:** Fix 500 errors caused by elasticsearch results referencing + garbage-collected commits + +The first example focuses on _how_ we fixed something, not on _what_ it fixes. +The rewritten version clearly describes the _end benefit_ to the user (fewer 500 +errors), and _when_ (searching commits with ElasticSearch). + +Use your best judgement and try to put yourself in the mindset of someone +reading the compiled changelog. Does this entry add value? Does it offer context +about _where_ and _why_ the change was made? + +## How to generate a changelog entry A `bin/changelog` script is available to generate the changelog entry file automatically. @@ -55,19 +107,28 @@ title: Hey DZ, I added a feature to GitLab! merge_request: author: ``` +If you're working on the GitLab EE repository, the entry will be added to +`changelogs/unreleased-ee/` instead. + +#### Arguments -### Arguments +| Argument | Shorthand | Purpose | +| ----------------- | --------- | --------------------------------------------- | +| [`--amend`] | | Amend the previous commit | +| [`--force`] | `-f` | Overwrite an existing entry | +| [`--merge-request`] | `-m` | Set merge request ID | +| [`--dry-run`] | `-n` | Don't actually write anything, just print | +| [`--git-username`] | `-u` | Use Git user.name configuration as the author | +| [`--help`] | `-h` | Print help message | -| Argument | Shorthand | Purpose | -| ----------------- | --------- | --------------------------------------------- | -| `--amend` | | Amend the previous commit | -| `--force` | `-f` | Overwrite an existing entry | -| `--merge-request` | `-m` | Merge Request ID | -| `--dry-run` | `-n` | Don't actually write anything, just print | -| `--git-username` | `-u` | Use Git user.name configuration as the author | -| `--help` | `-h` | Print help message | +[`--amend`]: #-amend +[`--force`]: #-force-or-f +[`--merge-request`]: #-merge-request-or-m +[`--dry-run`]: #-dry-run-or-n +[`--git-username`]: #-git-username-or-u +[`--help`]: #-help -#### `--amend` +##### `--amend` You can pass the **`--amend`** argument to automatically stage the generated file and amend it to the previous commit. @@ -88,7 +149,7 @@ merge_request: author: ``` -#### `--force` or `-f` +##### `--force` or `-f` Use **`--force`** or **`-f`** to overwrite an existing changelog entry if it already exists. @@ -105,7 +166,7 @@ merge_request: 1983 author: ``` -#### `--merge-request` or `-m` +##### `--merge-request` or `-m` Use the **`--merge-request`** or **`-m`** argument to provide the `merge_request` value: @@ -119,7 +180,7 @@ merge_request: 1983 author: ``` -#### `--dry-run` or `-n` +##### `--dry-run` or `-n` Use the **`--dry-run`** or **`-n`** argument to prevent actually writing or committing anything: @@ -135,7 +196,7 @@ author: $ ls changelogs/unreleased/ ``` -#### `--git-username` or `-u` +##### `--git-username` or `-u` Use the **`--git-username`** or **`-u`** argument to automatically fill in the `author` value with your configured Git `user.name` value: @@ -152,7 +213,7 @@ merge_request: author: Jane Doe ``` -## History and Reasoning +### History and Reasoning Our `CHANGELOG` file was previously updated manually by each contributor that felt their change warranted an entry. When two merge requests added their own diff --git a/doc/development/frontend.md b/doc/development/frontend.md index d646de7c54a..766b5f1f477 100644 --- a/doc/development/frontend.md +++ b/doc/development/frontend.md @@ -32,6 +32,21 @@ You can find the Frontend Architecture experts on the [team page][team-page]. You can find documentation about the desired architecture for a new feature built with Vue.js in [here][vue-section]. +### Realtime + +When writing code for realtime features we have to keep a couple of things in mind: +1. Do not overload the server with requests. +1. It should feel realtime. + +Thus, we must strike a balance between sending requests and the feeling of realtime. Use the following rules when creating realtime solutions. + +1. The server will tell you how much to poll by sending `X-Poll-Interval` in the header. Use that as your polling interval. This way it is easy for system administrators to change the polling rate. A `X-Poll-Interval: -1` means you should disable polling, and this must be implemented. +1. A response of `HTTP 429 Too Many Requests`, should disable polling as well. This must also be implemented. +1. Use a common library for polling. +1. Poll on active tabs only. Use a common library to find out which tab currently has eyes on it. Please use [Focus](https://gitlab.com/andrewn/focus). Specifically [Eyeballs Detector](https://gitlab.com/andrewn/focus/blob/master/lib/eyeballs-detector.js). +1. Use regular polling intervals, do not use backoff polling, or jitter, as the interval will be controlled by the server. +1. The backend code will most likely be using etags. You do not and should not check for status `304 Not Modified`. The browser will transform it for you. + ### Vue For more complex frontend features, we recommend using Vue.js. It shares @@ -66,6 +81,8 @@ Let's look into each of them: This is the index file of your new feature. This is where the root Vue instance of the new feature should be. +The Store and the Service should be imported and initialized in this file and provided as a prop to the main component. + Don't forget to follow [these steps.][page_specific_javascript] **A folder for Components** @@ -86,7 +103,7 @@ You can read more about components in Vue.js site, [Component System][component- **A folder for the Store** -The Store is a simple object that allows us to manage the state in a single +The Store is a class that allows us to manage the state in a single source of truth. The concept we are trying to follow is better explained by Vue documentation @@ -289,7 +306,7 @@ When exactly one object is needed for a given task, prefer to define it as a `class` rather than as an object literal. Prefer also to explicitly restrict instantiation, unless flexibility is important (e.g. for testing). -``` +```javascript // bad gl.MyThing = { @@ -332,6 +349,33 @@ gl.MyThing = MyThing; ``` +### Manipulating the DOM in a JS Class + +When writing a class that needs to manipulate the DOM guarantee a container option is provided. +This is useful when we need that class to be instantiated more than once in the same page. + +Bad: +```javascript +class Foo { + constructor() { + document.querySelector('.bar'); + } +} +new Foo(); +``` + +Good: +```javascript +class Foo { + constructor(opts) { + document.querySelector(`${opts.container} .bar`); + } +} + +new Foo({ container: '.my-element' }); +``` +You can find an example of the above in this [class][container-class-example]; + ## Supported browsers For our currently-supported browsers, see our [requirements][requirements]. @@ -457,3 +501,4 @@ Scenario: Developer can approve merge request [eslintrc]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.eslintrc [team-page]: https://about.gitlab.com/team [vue-section]: https://docs.gitlab.com/ce/development/frontend.html#how-to-build-a-new-feature-with-vue-js +[container-class-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/javascripts/mini_pipeline_graph_dropdown.js diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index 8232a0a113c..2b4126b43ef 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -68,7 +68,7 @@ end This will end up running one query for every object to update. This code can easily overload a database given enough rows to update or many instances of this code running in parallel. This particular problem is known as the -["N+1 query problem"](http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations). +["N+1 query problem"](http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations). You can write a test with [QueryRecoder](query_recorder.md) to detect this and prevent regressions. In this particular case the workaround is fairly easy: @@ -117,6 +117,8 @@ Post.all.includes(:author).each do |post| end ``` +Also consider using [QueryRecoder tests](query_recorder.md) to prevent a regression when eager loading. + ## Memory Usage **Summary:** merge requests **must not** increase memory usage unless absolutely diff --git a/doc/development/performance.md b/doc/development/performance.md index c1f129e576c..04419650b12 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -39,6 +39,7 @@ GitLab provides built-in tools to aid the process of improving performance: * [Sherlock](profiling.md#sherlock) * [GitLab Performance Monitoring](../administration/monitoring/performance/introduction.md) * [Request Profiling](../administration/monitoring/performance/request_profiling.md) +* [QueryRecoder](query_recorder.md) for preventing `N+1` regressions GitLab employees can use GitLab.com's performance monitoring systems located at <http://performance.gitlab.net>, this requires you to log in using your diff --git a/doc/development/polling.md b/doc/development/polling.md new file mode 100644 index 00000000000..a7f2962acf0 --- /dev/null +++ b/doc/development/polling.md @@ -0,0 +1,41 @@ +# Polling with ETag caching + +Polling for changes (repeatedly asking server if there are any new changes) +introduces high load on a GitLab instance, because it usually requires +executing at least a few SQL queries. This makes scaling large GitLab +instances (like GitLab.com) very difficult so we do not allow adding new +features that require polling and hit the database. + +Instead you should use polling mechanism with ETag caching in Redis. + +## How to use it + +1. Add the path of the endpoint which you want to poll to + `Gitlab::EtagCaching::Middleware`. +1. Implement cache invalidation for the path of your endpoint using + `Gitlab::EtagCaching::Store`. Whenever a resource changes you + have to invalidate the ETag for the path that depends on this + resource. +1. Check that the mechanism works: + - requests should return status code 304 + - there should be no SQL queries logged in `log/development.log` + +## How it works + +1. Whenever a resource changes we generate a random value and store it in + Redis. +1. When a client makes a request we set the `ETag` response header to the value + from Redis. +1. The client caches the response (client-side caching) and sends the ETag as + the `If-None-Match` header with every subsequent request for the same + resource. +1. If the `If-None-Match` header matches the current value in Redis we know + that the resource did not change so we can send 304 response immediately, + without querying the database at all. The client's browser will use the + cached response. +1. If the `If-None-Match` header does not match the current value in Redis + we have to generate a new response, because the resource changed. + +For more information see: +- [RFC 7232](https://tools.ietf.org/html/rfc7232) +- [ETag proposal](https://gitlab.com/gitlab-org/gitlab-ce/issues/26926) diff --git a/doc/development/profiling.md b/doc/development/profiling.md index e244ad4e881..933033a09e0 100644 --- a/doc/development/profiling.md +++ b/doc/development/profiling.md @@ -25,3 +25,5 @@ starting GitLab. For example: Bullet will log query problems to both the Rails log as well as the Chrome console. + +As a follow up to finding `N+1` queries with Bullet, consider writing a [QueryRecoder test](query_recorder.md) to prevent a regression. diff --git a/doc/development/query_recorder.md b/doc/development/query_recorder.md new file mode 100644 index 00000000000..e0127aaed4c --- /dev/null +++ b/doc/development/query_recorder.md @@ -0,0 +1,29 @@ +# QueryRecorder + +QueryRecorder is a tool for detecting the [N+1 queries problem](http://guides.rubyonrails.org/active_record_querying.html#eager-loading-associations) from tests. + +> Implemented in [spec/support/query_recorder.rb](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/support/query_recorder.rb) via [9c623e3e](https://gitlab.com/gitlab-org/gitlab-ce/commit/9c623e3e5d7434f2e30f7c389d13e5af4ede770a) + +As a rule, merge requests [should not increase query counts](merge_request_performance_guidelines.md#query-counts). If you find yourself adding something like `.includes(:author, :assignee)` to avoid having `N+1` queries, consider using QueryRecorder to enforce this with a test. Without this, a new feature which causes an additional model to be accessed will silently reintroduce the problem. + +## How it works + +This style of test works by counting the number of SQL queries executed by ActiveRecord. First a control count is taken, then you add new records to the database and rerun the count. If the number of queries has significantly increased then an `N+1` queries problem exists. + +```ruby +it "avoids N+1 database queries" do + control_count = ActiveRecord::QueryRecorder.new { visit_some_page }.count + create_list(:issue, 5) + expect { visit_some_page }.not_to exceed_query_limit(control_count) +end +``` + +As an example you might create 5 issues in between counts, which would cause the query count to increase by 5 if an N+1 problem exists. + +> **Note:** In some cases the query count might change slightly between runs for unrelated reasons. In this case you might need to test `exceed_query_limit(control_count + acceptable_change)`, but this should be avoided if possible. + +## See also + +- [Bullet](profiling.md#Bullet) For finding `N+1` query problems +- [Performance guidelines](performance.md) +- [Merge request performance guidelines](merge_request_performance_guidelines.md#query-counts)
\ No newline at end of file diff --git a/doc/development/testing.md b/doc/development/testing.md index 9b545d7f0f1..5ac7b8dadeb 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -35,8 +35,8 @@ GitLab uses [Karma] to run its [Jasmine] JavaScript specs. They can be run on the command line via `bundle exec karma`. - JavaScript tests live in `spec/javascripts/`, matching the folder structure of - `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js.es6` has a corresponding - `spec/javascripts/behaviors/autosize_spec.js.es6` file. + `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js` has a corresponding + `spec/javascripts/behaviors/autosize_spec.js` file. - Haml fixtures required for JavaScript tests live in `spec/javascripts/fixtures`. They should contain the bare minimum amount of markup necessary for the test. |