diff options
Diffstat (limited to 'doc/development')
| -rw-r--r-- | doc/development/README.md | 2 | ||||
| -rw-r--r-- | doc/development/changelog.md | 185 | ||||
| -rw-r--r-- | doc/development/post_deployment_migrations.md | 75 | ||||
| -rw-r--r-- | doc/development/testing.md | 36 |
4 files changed, 298 insertions, 0 deletions
diff --git a/doc/development/README.md b/doc/development/README.md index 14d6f08e43a..bf1f054b7d5 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -21,6 +21,7 @@ ## Process +- [Generate a changelog entry with `bin/changelog`](changelog.md) - [Code review guidelines](code_review.md) for reviewing code and having code reviewed. - [Merge request performance guidelines](merge_request_performance_guidelines.md) for ensuring merge requests do not negatively impact GitLab performance @@ -41,6 +42,7 @@ - [What requires downtime?](what_requires_downtime.md) - [Adding database indexes](adding_database_indexes.md) +- [Post Deployment Migrations](post_deployment_migrations.md) ## Compliance diff --git a/doc/development/changelog.md b/doc/development/changelog.md new file mode 100644 index 00000000000..6a97fae9cac --- /dev/null +++ b/doc/development/changelog.md @@ -0,0 +1,185 @@ +# Generate a changelog entry + +This guide contains instructions for generating a changelog entry data file, as +well as information and history about our changelog process. + +## Overview + +Each bullet point, or **entry**, in our [`CHANGELOG.md`][changelog.md] file is +generated from a single data file in the [`changelogs/unreleased/`][unreleased] +(or corresponding EE) folder. The file is expected to be a [YAML] file in the +following format: + +```yaml +--- +title: "Going through change[log]s" +merge_request: 1972 +author: Ozzy Osbourne +``` + +The `merge_request` value is a reference to a merge request that adds this +entry, and the `author` key is used to give attribution to community +contributors. Both are optional. + +Community contributors and core team members are encouraged to add their name to +the `author` field. GitLab team members should not. + +If you're working on the GitLab EE repository, the entry will be added to +`changelogs/unreleased-ee/` instead. + +[changelog.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG.md +[unreleased]: https://gitlab.com/gitlab-org/gitlab-ce/tree/master/changelogs/ +[YAML]: https://en.wikipedia.org/wiki/YAML + +## Instructions + +A `bin/changelog` script is available to generate the changelog entry file +automatically. + +Its simplest usage is to provide the value for `title`: + +```text +$ bin/changelog 'Hey DZ, I added a feature to GitLab!' +create changelogs/unreleased/my-feature.yml +--- +title: Hey DZ, I added a feature to GitLab! +merge_request: +author: +``` + +The entry filename is based on the name of the current Git branch. If you run +the command above on a branch called `feature/hey-dz`, it will generate a +`changelogs/unreleased/feature-hey-dz.yml` file. + +### Arguments + +| Argument | Shorthand | Purpose | +| ----------------- | --------- | --------------------------------------------- | +| `--amend` | | Amend the previous commit | +| `--force` | `-f` | Overwrite an existing entry | +| `--merge-request` | `-m` | Merge Request ID | +| `--dry-run` | `-n` | Don't actually write anything, just print | +| `--git-username` | `-u` | Use Git user.name configuration as the author | +| `--help` | `-h` | Print help message | + +#### `--amend` + +You can pass the **`--amend`** argument to automatically stage the generated +file and amend it to the previous commit. + +If you use **`--amend`** and don't provide a title, it will automatically use +the "subject" of the previous commit, which is the first line of the commit +message: + +```text +$ git show --oneline +ab88683 Added an awesome new feature to GitLab + +$ bin/changelog --amend +create changelogs/unreleased/feature-hey-dz.yml +--- +title: Added an awesome new feature to GitLab +merge_request: +author: +``` + +#### `--force` or `-f` + +Use **`--force`** or **`-f`** to overwrite an existing changelog entry if it +already exists. + +```text +$ bin/changelog 'Hey DZ, I added a feature to GitLab!' +error changelogs/unreleased/feature-hey-dz.yml already exists! Use `--force` to overwrite. + +$ bin/changelog 'Hey DZ, I added a feature to GitLab!' --force +create changelogs/unreleased/feature-hey-dz.yml +--- +title: Hey DZ, I added a feature to GitLab! +merge_request: 1983 +author: +``` + +#### `--merge-request` or `-m` + +Use the **`--merge-request`** or **`-m`** argument to provide the +`merge_request` value: + +```text +$ bin/changelog 'Hey DZ, I added a feature to GitLab!' -m 1983 +create changelogs/unreleased/feature-hey-dz.yml +--- +title: Hey DZ, I added a feature to GitLab! +merge_request: 1983 +author: +``` + +#### `--dry-run` or `-n` + +Use the **`--dry-run`** or **`-n`** argument to prevent actually writing or +committing anything: + +```text +$ bin/changelog --amend --dry-run +create changelogs/unreleased/feature-hey-dz.yml +--- +title: Added an awesome new feature to GitLab +merge_request: +author: + +$ ls changelogs/unreleased/ +``` + +#### `--git-username` or `-u` + +Use the **`--git-username`** or **`-u`** argument to automatically fill in the +`author` value with your configured Git `user.name` value: + +```text +$ git config user.name +Jane Doe + +$ bin/changelog --u 'Hey DZ, I added a feature to GitLab!' +create changelogs/unreleased/feature-hey-dz.yml +--- +title: Hey DZ, I added a feature to GitLab! +merge_request: +author: Jane Doe +``` + +## History and Reasoning + +Our `CHANGELOG` file was previously updated manually by each contributor that +felt their change warranted an entry. When two merge requests added their own +entries at the same spot in the list, it created a merge conflict in one as soon +as the other was merged. When we had dozens of merge requests fighting for the +same changelog entry location, this quickly became a major source of merge +conflicts and delays in development. + +This led us to a [boring solution] of "add your entry in a random location in +the list." This actually worked pretty well as we got further along in each +monthly release cycle, but at the start of a new cycle, when a new version +section was added and there were fewer places to "randomly" add an entry, the +conflicts became a problem again until we had a sufficient number of entries. + +On top of all this, it created an entirely different headache for [release managers] +when they cherry-picked a commit into a stable branch for a patch release. If +the commit included an entry in the `CHANGELOG`, it would include the entire +changelog for the latest version in `master`, so the release manager would have +to manually remove the later entries. They often would have had to do this +multiple times per patch release. This was compounded when we had to release +multiple patches at once due to a security issue. + +We needed to automate all of this manual work. So we [started brainstorming]. +After much discussion we settled on the current solution of one file per entry, +and then compiling the entries into the overall `CHANGELOG.md` file during the +[release process]. + +[boring solution]: https://about.gitlab.com/handbook/#boring-solutions +[release managers]: https://gitlab.com/gitlab-org/release-tools/blob/master/doc/release-manager.md +[started brainstorming]: https://gitlab.com/gitlab-org/gitlab-ce/issues/17826 +[release process]: https://gitlab.com/gitlab-org/release-tools + +--- + +[Return to Development documentation](README.md) diff --git a/doc/development/post_deployment_migrations.md b/doc/development/post_deployment_migrations.md new file mode 100644 index 00000000000..cfc91539bee --- /dev/null +++ b/doc/development/post_deployment_migrations.md @@ -0,0 +1,75 @@ +# Post Deployment Migrations + +Post deployment migrations are regular Rails migrations that can optionally be +executed after a deployment. By default these migrations are executed alongside +the other migrations. To skip these migrations you will have to set the +environment variable `SKIP_POST_DEPLOYMENT_MIGRATIONS` to a non-empty value +when running `rake db:migrate`. + +For example, this would run all migrations including any post deployment +migrations: + +```bash +bundle exec rake db:migrate +``` + +This however will skip post deployment migrations: + +```bash +SKIP_POST_DEPLOYMENT_MIGRATIONS=true bundle exec rake db:migrate +``` + +## Deployment Integration + +Say you're using Chef for deploying new versions of GitLab and you'd like to run +post deployment migrations after deploying a new version. Let's assume you +normally use the command `chef-client` to do so. To make use of this feature +you'd have to run this command as follows: + +```bash +SKIP_POST_DEPLOYMENT_MIGRATIONS=true sudo chef-client +``` + +Once all servers have been updated you can run `chef-client` again on a single +server _without_ the environment variable. + +The process is similar for other deployment techniques: first you would deploy +with the environment variable set, then you'll essentially re-deploy a single +server but with the variable _unset_. + +## Creating Migrations + +To create a post deployment migration you can use the following Rails generator: + +```bash +bundle exec rails g post_deployment_migration migration_name_here +``` + +This will generate the migration file in `db/post_migrate`. These migrations +behave exactly like regular Rails migrations. + +## Use Cases + +Post deployment migrations can be used to perform migrations that mutate state +that an existing version of GitLab depends on. For example, say you want to +remove a column from a table. This requires downtime as a GitLab instance +depends on this column being present while it's running. Normally you'd follow +these steps in such a case: + +1. Stop the GitLab instance +2. Run the migration removing the column +3. Start the GitLab instance again + +Using post deployment migrations we can instead follow these steps: + +1. Deploy a new version of GitLab while ignoring post deployment migrations +2. Re-run `rake db:migrate` but without the environment variable set + +Here we don't need any downtime as the migration takes place _after_ a new +version (which doesn't depend on the column anymore) has been deployed. + +Some other examples where these migrations are useful: + +* Cleaning up data generated due to a bug in GitLab +* Removing tables +* Migrating jobs from one Sidekiq queue to another diff --git a/doc/development/testing.md b/doc/development/testing.md index 8e91ac5e3ba..b0b26ccf57a 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -132,6 +132,42 @@ Adding new Spinach scenarios is acceptable _only if_ the new scenario requires no more than one new `step` definition. If more than that is required, the test should be re-implemented using RSpec instead. +## Testing Rake Tasks + +To make testing Rake tasks a little easier, there is a helper that can be included +in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use +`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures +a few other things to make testing Rake tasks easier. + +At a minimum, requiring the Rake helper will redirect `stdout`, include the +runtime task helpers, and include the `RakeHelpers` Spec support module. + +The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make +executing tasks simple. See `spec/support/rake_helpers.rb` for all available +methods. + +Example: + +```ruby +require 'rake_helper' + +describe 'gitlab:shell rake tasks' do + before do + Rake.application.rake_require 'tasks/gitlab/shell' + + stub_warn_user_is_not_gitlab + end + + describe 'install task' do + it 'invokes create_hooks task' do + expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke) + + run_rake_task('gitlab:shell:install') + end + end +end +``` + --- [Return to Development documentation](README.md) |