diff options
Diffstat (limited to 'doc/development/documentation')
3 files changed, 317 insertions, 22 deletions
diff --git a/doc/development/documentation/site_architecture/deployment.md b/doc/development/documentation/site_architecture/deployment.md new file mode 100644 index 00000000000..6b334dccd19 --- /dev/null +++ b/doc/development/documentation/site_architecture/deployment.md @@ -0,0 +1,16 @@ +# GitLab Docs deployment process + +The docs site is deployed to production with [GitLab Pages](../../../user/project/pages/index.md) +and previewed in merge requests with Review Apps. By default, we pull from the +master branch of [all the projects](index.md#repositories). + +The [scheduled pipelines](../../../user/project/pipelines/schedules.md) +are used to build the site once an hour. + +For more information, see [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/.gitlab-ci.yml). + +<-- TODO + +Document Rakefile and .gitlab-ci.yml. + +--> diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md index 1aef0ed855c..ded5cd0b40e 100644 --- a/doc/development/documentation/site_architecture/index.md +++ b/doc/development/documentation/site_architecture/index.md @@ -4,14 +4,16 @@ description: "Learn how GitLab's documentation website is architectured." # Documentation site architecture -Learn how we build and architecture [`gitlab-docs`](https://gitlab.com/gitlab-org/gitlab-docs) -and deploy it to <https://docs.gitlab.com>. +The [`gitlab-docs`](https://gitlab.com/gitlab-org/gitlab-docs) project hosts +the repository which is used to generate the GitLab documentation website and +is deployed to <https://docs.gitlab.com>. It uses the [Nanoc](http://nanoc.ws) +static site generator. -## Repository +## Architecture While the source of the documentation content is stored in GitLab's respective product -repositories, the source that is used to build the documentation site _from that content_ -is located at <https://gitlab.com/gitlab-org/gitlab-docs>. +[repositories](#repositories), the source that is used to build the documentation +site _from that content_ is located at <https://gitlab.com/gitlab-org/gitlab-docs>. The following diagram illustrates the relationship between the repositories from where content is sourced, the `gitlab-docs` project, and the published output. @@ -43,9 +45,6 @@ from where content is sourced, the `gitlab-docs` project, and the published outp G --> L ``` -See the [README there](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/README.md) -for detailed information. - ## Assets To provide an optimized site structure, design, and a search-engine friendly @@ -71,28 +70,133 @@ the GitLab Documentation website. To understand how the global nav (left sidebar) is built, please read through the [global navigation](global_nav.md) doc. -## Deployment - -The docs site is deployed to production with GitLab Pages, and previewed in -merge requests with Review Apps. +<!-- +## Helpers -The deployment aspects will be soon transferred from the [original document](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/README.md) -to this page. +TBA +--> -<!-- ## Repositories -TBA +You will not find any GitLab docs content in the `gitlab-docs` repository. +All documentation files are hosted in the respective repository of each +product, and all together are pulled to generate the docs website: -## Search engine +- [GitLab Enterprise Edition](https://gitlab.com/gitlab-org/gitlab-ee) +- [GitLab Community Edition](https://gitlab.com/gitlab-org/gitlab-ce) +- [Omnibus GitLab](https://gitlab.com/gitlab-org/omnibus-gitlab) +- [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) +- [GitLab Chart](https://gitlab.com/charts/gitlab) -TBA +NOTE: **Note:** +Although GitLab Community Edition is generated, it is hidden from the website +as it's a subset of the Enterprise Edition. We generate it for consistency, +until the [GitLab codebase is unified](https://gitlab.com/gitlab-org/gitlab-ee/issues/2952). + +## Using YAML data files + +The easiest way to achieve something similar to +[Jekyll's data files](https://jekyllrb.com/docs/datafiles/) in Nanoc is by +using the [`@items`](https://nanoc.ws/doc/reference/variables/#items-and-layouts) +variable. + +The data file must be placed inside the `content/` directory and then it can +be referenced in an ERB template. + +Suppose we have the `content/_data/versions.yaml` file with the content: + +```yaml +versions: +- 10.6 +- 10.5 +- 10.4 +``` + +We can then loop over the `versions` array with something like: + +```erb +<% @items['/_data/versions.yaml'][:versions].each do | version | %> + +<h3><%= version %></h3> + +<% end &> +``` + +Note that the data file must have the `yaml` extension (not `yml`) and that +we reference the array with a symbol (`:versions`). + +## Bumping versions of CSS and Javascript + +Whenever the custom CSS and Javascript files under `content/assets/` change, +make sure to bump their version in the frontmatter. This method guarantees that +your changes will take effect by clearing the cache of previous files. + +Always use Nanoc's way of including those files, do not hardcode them in the +layouts. For example use: + +```erb +<script async type="application/javascript" src="<%= @items['/assets/javascripts/badges.*'].path %>"></script> + +<link rel="stylesheet" href="<%= @items['/assets/stylesheets/toc.*'].path %>"> +``` + +The links pointing to the files should be similar to: + +```erb +<%= @items['/path/to/assets/file.*'].path %> +``` + +Nanoc will then build and render those links correctly according with what's +defined in [`Rules`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/Rules). + +## Linking to source files + +A helper called [`edit_on_gitlab`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/lib/helpers/edit_on_gitlab.rb) can be used +to link to a page's source file. We can link to both the simple editor and the +web IDE. Here's how you can use it in a Nanoc layout: + +- Default editor: `<a href="<%= edit_on_gitlab(@item, editor: :simple) %>">Simple editor</a>` +- Web IDE: `<a href="<%= edit_on_gitlab(@item, editor: :webide) %>">Web IDE</a>` + +If you don't specify `editor:`, the simple one is used by default. + +## Algolia search engine + +The docs site uses [Algolia docsearch](https://community.algolia.com/docsearch/) +for its search function. This is how it works: + +1. GitLab is a member of the [docsearch program](https://community.algolia.com/docsearch/#join-docsearch-program), + which is the free tier of [Algolia](https://www.algolia.com/). +1. Algolia hosts a [doscsearch config](https://github.com/algolia/docsearch-configs/blob/master/configs/gitlab.json) + for the GitLab docs site, and we've worked together to refine it. +1. That [config](https://community.algolia.com/docsearch/config-file.html) is + parsed by their [crawler](https://community.algolia.com/docsearch/crawler-overview.html) + every 24h and [stores](https://community.algolia.com/docsearch/inside-the-engine.html) + the [docsearch index](https://community.algolia.com/docsearch/how-do-we-build-an-index.html) + on [Algolia's servers](https://community.algolia.com/docsearch/faq.html#where-is-my-data-hosted%3F). +1. On the docs side, we use a [docsearch layout](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/layouts/docsearch.html) which + is present on pretty much every page except <https://docs.gitlab.com/search/>, + which uses its [own layout](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/layouts/instantsearch.html). In those layouts, + there's a javascript snippet which initiates docsearch by using an API key + and an index name (`gitlab`) that are needed for Algolia to show the results. + +NOTE: **For GitLab employees:** +The credentials to access the Algolia dashboard are stored in 1Password. If you +want to receive weekly reports of the search usage, search the Google doc with +title "Email, Slack, and GitLab Groups and Aliases", search for `docsearch`, +and add a comment with your email to be added to the alias that gets the weekly +reports. + +## Deployment + +Read about the [deployment process](deployment.md). ## Versions -TBA +The docs website supports versions. For more information, read about +[versions](versions.md). -## Helpers +## Review Apps for documentation merge requests -TBA ---> +If you are contributing to GitLab docs read how to [create a Review App with each +merge request](../index.md#previewing-the-changes-live). diff --git a/doc/development/documentation/site_architecture/versions.md b/doc/development/documentation/site_architecture/versions.md new file mode 100644 index 00000000000..b5e9764f99f --- /dev/null +++ b/doc/development/documentation/site_architecture/versions.md @@ -0,0 +1,175 @@ +# How the website versions are built + +This directory contains all needed Dockerfiles to build and deploy the +versioned website. It is heavily inspired by Docker's +[publish tools](https://github.com/docker/docker.github.io/tree/publish-tools). + +The following Dockerfiles are used. + +| Dockerfile | Docker image | Description | +| ---------- | ------------ | ----------- | +| [`Dockerfile.bootstrap`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.bootstrap) | `gitlab-docs:bootstrap` | Contains all the dependencies that are needed to build the website. If the gems are updated and `Gemfile{,.lock}` changes, the image must be rebuilt. | +| [`Dockerfile.builder.onbuild`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.builder.onbuild) | `gitlab-docs:builder-onbuild` | Base image to build the docs website. It uses `ONBUILD` to perform all steps and depends on `gitlab-docs:bootstrap`. | +| [`Dockerfile.nginx.onbuild`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.nginx.onbuild) | `gitlab-docs:nginx-onbuild` | Base image to use for building documentation archives. It uses `ONBUILD` to perform all required steps to copy the archive, and relies upon its parent `Dockerfile.builder.onbuild` that is invoked when building single documentation achives (see the `Dockerfile` of each branch. | +| [`Dockerfile.archives`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/dockerfiles/Dockerfile.archives) | `gitlab-docs:archives` | Contains all the versions of the website in one archive. It copies all generated HTML files from every version in one location. | + +## How to build the images + +You can build and tag all tooling images locally: + +1. Make sure you're on the `dockerfiles/` directory of the `gitlab-docs` repo. +1. Build the images: + + ```sh + docker build -t registry.gitlab.com/gitlab-org/gitlab-docs:bootstrap -f Dockerfile.bootstrap ../ + docker build -t registry.gitlab.com/gitlab-org/gitlab-docs:builder-onbuild -f Dockerfile.builder.onbuild ../ + docker build -t registry.gitlab.com/gitlab-org/gitlab-docs:nginx-onbuild -f Dockerfile.nginx.onbuild ../ + ``` + +For each image, there's a manual job under the `images` stage in +[`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/.gitlab-ci.yml) which can be invoked at will. + +## Release process + +When a new GitLab version is released on the 22nd, we need to create the respective +single Docker image, and update some files. + +### 1. Add the chart version + +Since the charts use a different version number than all the other GitLab +products, we need to add the new +[version mapping](https://docs.gitlab.com/charts/installation/version_mappings.html): + +1. Make sure you're on the root path of the `gitlab-docs` repo. +1. Open `content/_data/chart_versions.yaml` and add the new version using the + [version mapping](https://docs.gitlab.com/charts/installation/version_mappings.html). + Note that only the `major.minor` version is needed. +1. Create a new merge request and merge it. + +TIP: **Tip:** +It can be handy to create the future mappings since they are pretty much known. +In that case, when a new GitLab version is released, you don't have to repeat +this first step. + +### 2. Create an image for a single version + +1. Make sure you're on the root path of the `gitlab-docs` repo. +1. Run the raketask to create the single version: + + ``` + bundle exec rake "release:single[12.0]" + ``` + + A new `Dockerfile.12.0` should have been created. + +1. Test locally by building the image and running it: + + ``` + docker build -t docs:12.0 -f Dockerfile.12.0 . + docker run -it --rm -p 4000:4000 docs:12.0 + ``` + +1. Visit `http://localhost:4000/12.0/` and make sure everything works correctly. +1. Commit your changes and push the newly created branch, but **don't create a merge request**. + +Once you push, the `image:docker-singe` job will create a new Docker image +tagged with the branch name you created in the first step. + +### 3. Create the release merge request + +1. Make sure you're on the root path of the `gitlab-docs` repo. +1. Create a branch `release-X-Y`. +1. **Rotate the online and offline versions:** + At any given time, there are 4 browsable online versions: one pulled from + the upstream master branches and the three latest stable versions. + + Edit [`content/_data/versions.yaml`](https://gitlab.com/gitlab-org/gitlab-docs/blob/master/content/_data/versions.yaml) and rotate + the versions to reflect the new changes: + + - `online`: The 3 latest stable versions. + - `offline`: All the previous versions not available online. + +1. **Add the new offline version in the 404 page redirect script:** + + Since we're deprecating the oldest version each month, we need to redirect + those URLs in order not to create [404 entries](https://gitlab.com/gitlab-org/gitlab-docs/issues/221). + There's a temporary hack for now: + + 1. Edit `content/404.html`, making sure all offline versions under + `content/_data/versions.yaml` are in the javascript snippet at the end of + the document. + +1. **Update the `latest` and `archives` images:** + + The following two Dockerfiles need to be updated: + + 1. `Dockerfile.archives` - Add the latest version to the + list. + 1. `Dockerfile.master` - Rotate the versions (oldest + gets removed and latest is added at the end of the list). + +1. Commit and push to create the merge request using the "Release" template. + +### 4. Update the dropdown for all online versions + +The versions dropdown is in a way "hardcoded". When the site is built, it looks +at the contents of `content/_data/versions.yaml` and based on that, the dropdown +is populated. So, older branches will have different content, which means the +dropdown will be one release behind. Remember that the new changes of the dropdown +are included in the unmerged `release-X-Y` branch. + +The content of `content/_data/versions.yaml` needs to change for all online +versions. Here's a script to do this in one go: + +```sh +# Change this to reflect the latest stable version +CURRENT_VERSION="12.2" +CURRENT_BRANCH="12-2" + +# Change this to the latest 3 stable versions +for branch in 12.2 12.1 12.0 +do + git checkout ${branch} + git checkout release-${CURRENT_BRANCH} -- content/_data/versions.yaml + git commit -m "Update dropdown for ${branch} to ${CURRENT_VESION}" + git push origin ${branch} +done +``` + +Replace `CURRENT_VERSION` and `CURRENT_BRANCH` with the current values. + +### 5. Merge the release merge request + +After all the pipelines of the online versions succeed, it's time to merge the +release merge request. + +Once you merge, you may need to [run the scheduled pipeline](https://gitlab.com/gitlab-org/gitlab-docs/pipeline_schedules) +(press the play button), since this is built on a schedule, once an hour. + +Once the scheduled pipeline succeeds, the docs site will be deployed with all +new versions online. + +## Update an old image with new upstream content + +If there are upstream changes not included in the single Docker image, just +[rerun the pipeline](https://gitlab.com/gitlab-org/gitlab-docs/pipelines/new) +for the branch in question. + +## Porting new website changes to old versions + +CAUTION: **Warning:** +Porting changes to older branches can have unintended effects as we're constantly +changing the backend of the website. Use only when you know what you're doing +and make sure to test locally. + +The website will keep changing and being improved. In order to consolidate +those changes to the stable branches, we'd need to pick certain changes +from time to time. + +If this is not possible or there are many changes, merge master into them: + +```sh +git branch 12.0 +git fetch origin master +git merge origin/master +``` |