summaryrefslogtreecommitdiff
path: root/doc/user/project/merge_requests/code_quality.md
blob: ca15ec154fcd1569546e93321375e8b6f6012d15 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
---
stage: Verify
group: Testing
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
type: reference, howto
---

# Code Quality

> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3.
> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in 13.2.

Ensuring your project's code stays simple, readable and easy to contribute to can be problematic. With the help of [GitLab CI/CD](../../../ci/README.md), you can analyze your
source code quality using GitLab Code Quality.

Code Quality:

- Uses [Code Climate Engines](https://codeclimate.com), which are
  free and open source. Code Quality does not require a Code Climate
  subscription.
- Runs in [pipelines](../../../ci/pipelines/index.md) using a Docker image built in the
  [GitLab Code
  Quality](https://gitlab.com/gitlab-org/ci-cd/codequality) project using [default Code Climate configurations](https://gitlab.com/gitlab-org/ci-cd/codequality/-/tree/master/codeclimate_defaults).
- Can make use of a [template](#example-configuration).
- Is available with [Auto
  DevOps](../../../topics/autodevops/stages.md#auto-code-quality).
- Can be extended through [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) or a [custom tool](#implementing-a-custom-tool).

## Code Quality Widget

> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.3.
> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in 13.2.

Going a step further, GitLab can show the Code Quality report right
in the merge request widget area if a report from the target branch is available to compare to:

![Code Quality Widget](img/code_quality.png)

Watch a quick walkthrough of Code Quality in action:

<div class="video-fallback">
  See the video: <a href="https://www.youtube.com/watch?v=B32LxtJKo9M">Code Quality: Speed Run</a>.
</div>
<figure class="video-container">
  <iframe src="https://www.youtube.com/embed/B32LxtJKo9M" frameborder="0" allowfullscreen="true"> </iframe>
</figure>

NOTE:
For one customer, the auditor found that having Code Quality, SAST, and Container Scanning all automated in GitLab CI/CD was almost better than a manual review! [Read more](https://about.gitlab.com/customers/bi_worldwide/).

See also the Code Climate list of [Supported Languages for Maintainability](https://docs.codeclimate.com/docs/supported-languages-for-maintainability).

## Use cases

For instance, consider the following workflow:

1. Your backend team member starts a new implementation for making a certain
   feature in your app faster.
1. With Code Quality reports, they analyze how their implementation is impacting
   the code quality.
1. The metrics show that their code degrades the quality by 10 points.
1. You ask a co-worker to help them with this modification.
1. They both work on the changes until Code Quality report displays no
   degradations, only improvements.
1. You approve the merge request and authorize its deployment to staging.
1. Once verified, their changes are deployed to production.

## Example configuration

This example shows how to run Code Quality on your code by using GitLab CI/CD and Docker.
It requires GitLab 11.11 or later, and GitLab Runner 11.5 or later. If you are using
GitLab 11.4 or earlier, you can view the deprecated job definitions in the
[documentation archive](https://docs.gitlab.com/12.10/ee/user/project/merge_requests/code_quality.html#previous-job-definitions).

- Using shared runners, the job should be configured For the [Docker-in-Docker workflow](../../../ci/docker/using_docker_build.md#use-the-docker-executor-with-the-docker-image-docker-in-docker).
- Using private runners, there is an [alternative configuration](#set-up-a-private-runner-for-code-quality-without-docker-in-docker) recommended for running CodeQuality analysis more efficiently.

In either configuration, the runner mmust have enough disk space to handle generated Code Quality files. For example on the [GitLab project](https://gitlab.com/gitlab-org/gitlab) the files are approximately 7 GB.

Once you set up GitLab Runner, include the Code Quality template in your CI configuration:

```yaml
include:
  - template: Code-Quality.gitlab-ci.yml
```

The above example creates a `code_quality` job in your CI/CD pipeline which
scans your source code for code quality issues. The report is saved as a
[Code Quality report artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality)
that you can later download and analyze.

It's also possible to override the URL to the Code Quality image by
setting the `CODE_QUALITY_IMAGE` variable. This is particularly useful if you want
to lock in a specific version of Code Quality, or use a fork of it:

```yaml
include:
  - template: Code-Quality.gitlab-ci.yml

code_quality:
  variables:
    CODE_QUALITY_IMAGE: "registry.example.com/codequality-fork:latest"
```

In [GitLab 13.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/11100), you can override the [Code Quality environment variables](https://gitlab.com/gitlab-org/ci-cd/codequality#environment-variables):

```yaml
variables:
  TIMEOUT_SECONDS: 1

include:
  - template: Code-Quality.gitlab-ci.yml
```

By default, report artifacts are not downloadable. If you need them downloadable on the
job details page, you can add `gl-code-quality-report.json` to the artifact paths like so:

```yaml
include:
  - template: Code-Quality.gitlab-ci.yml

code_quality:
  artifacts:
    paths: [gl-code-quality-report.json]
```

The included `code_quality` job is running in the `test` stage, so it needs to be included in your CI configuration, like so:

```yaml
stages:
  - test
```

NOTE:
This information is automatically extracted and shown right in the merge request widget.

WARNING:
On self-managed instances, if a malicious actor compromises the Code Quality job
definition they could execute privileged Docker commands on the runner
host. Having proper access control policies mitigates this attack vector by
allowing access only to trusted actors.

### Set up a private runner for code quality without Docker-in-Docker

It's possible to configure your own runners and avoid Docker-in-Docker. You can use a
configuration that may greatly speed up job execution without requiring your runners
to operate in privileged mode.

This alternative configuration uses socket binding to share the Runner's Docker daemon
with the job environment. Be aware that this configuration [has significant considerations](../../../ci/docker/using_docker_build.md#use-docker-socket-binding)
to be consider, but may be preferable depending on your use case.

1. Register a new runner:

   ```shell
   $ gitlab-runner register --executor "docker" \
     --docker-image="docker:stable" \
     --url "https://gitlab.com/" \
     --description "cq-sans-dind" \
     --tag-list "cq-sans-dind" \
     --locked="false" \
     --access-level="not_protected" \
     --docker-volumes "/cache"\
     --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
     --registration-token="<project_token>" \
     --non-interactive
   ```

1. **Optional, but recommended:** Set the builds directory to `/tmp/builds`,
  so job artifacts are periodically purged from the runner host. If you skip
  this step, you must clean up the default builds directory (`/builds`) yourself.
  You can do this by adding the following two flags to `gitlab-runner register`
  in the previous step.

   ```shell
   --builds-dir /tmp/builds
   --docker-volumes /tmp/builds:/tmp/builds
   ```

   The resulting configuration:

   ```toml
   [[runners]]
     name = "cq-sans-dind"
     url = "https://gitlab.com/"
     token = "<project_token>"
     executor = "docker"
     builds_dir = "/tmp/builds"
     [runners.docker]
       tls_verify = false
       image = "docker:stable"
       privileged = false
       disable_entrypoint_overwrite = false
       oom_kill_disable = false
       disable_cache = false
       volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock", "/tmp/builds:/tmp/builds"]
       shm_size = 0
     [runners.cache]
       [runners.cache.s3]
       [runners.cache.gcs]
   ```

1. Apply two overrides to the `code_quality` job created by the template:

   ```yaml
   include:
     - template: Code-Quality.gitlab-ci.yml

   code_quality:
     services:            # Shut off Docker-in-Docker
     tags:
       - cq-sans-dind     # Set this job to only run on our new specialized runner
   ```

The end result is that:

- Privileged mode is not used.
- Docker-in-Docker is not used.
- Docker images, including all CodeClimate images, are cached, and not re-fetched for subsequent jobs.

With this configuration, the run time for a second pipeline is much shorter. For example
this [small change](https://gitlab.com/drewcimino/test-code-quality-template/-/merge_requests/4/diffs?commit_id=1e705607aef7236c1b20bb6f637965f3f3e53a46)
to an [open merge request](https://gitlab.com/drewcimino/test-code-quality-template/-/merge_requests/4/pipelines)
running Code Quality analysis ran significantly faster the second time:

![Code Quality sequential runs without DinD](img/code_quality_host_bound_sequential.png)

This configuration is not possible on `gitlab.com` shared runners. Shared runners
are configured with `privileged=true`, and they do not expose `docker.sock` into
the job container. As a result, socket binding cannot be used to make `docker` available
in the context of the job script.

[Docker-in-Docker](../../../ci/docker/using_docker_build.md#use-the-docker-executor-with-the-docker-image-docker-in-docker)
was chosen as an operational decision by the runner team, instead of exposing `docker.sock`.

### Disabling the code quality job

The `code_quality` job doesn't run if the `$CODE_QUALITY_DISABLED` environment
variable is present. Please refer to the environment variables [documentation](../../../ci/variables/README.md)
to learn more about how to define one.

To disable the `code_quality` job, add `CODE_QUALITY_DISABLED` as a custom environment
variable. This can be done:

- For the whole project, [in the project settings](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui)
  or [CI/CD configuration](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui).
- For a single pipeline run:

  1. Go to **CI/CD > Pipelines**
  1. Click **Run Pipeline**
  1. Add `CODE_QUALITY_DISABLED` as the variable key, with any value.

### Using with merge request pipelines

The configuration provided by the Code Quality template does not let the `code_quality` job
run on [pipelines for merge requests](../../../ci/merge_request_pipelines/index.md).

If pipelines for merge requests is enabled, the `code_quality:rules` must be redefined.

The template has these [`rules`](../../../ci/yaml/README.md#rules) for the `code quality` job:

```yaml
code_quality:
  rules:
    - if: '$CODE_QUALITY_DISABLED'
      when: never
    - if: '$CI_COMMIT_TAG || $CI_COMMIT_BRANCH'
```

If you are using merge request pipelines, your `rules` (or [`workflow: rules`](../../../ci/yaml/README.md#workflowrules))
might look like this example:

```yaml
job1:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' # Run job1 in merge request pipelines
    - if: '$CI_COMMIT_BRANCH == "master"'                # Run job1 in pipelines on the master branch (but not in other branch pipelines)
    - if: '$CI_COMMIT_TAG'                               # Run job1 in pipelines for tags
```

To make these work together, you need to overwrite the code quality `rules`
so that they match your current `rules`. From the example above, it could look like:

```yaml
include:
  - template: Code-Quality.gitlab-ci.yml

code_quality:
  rules:
    - if: '$CODE_QUALITY_DISABLED'
      when: never
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' # Run code quality job in merge request pipelines
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'      # Run code quality job in pipelines on the master branch (but not in other branch pipelines)
    - if: '$CI_COMMIT_TAG'                               # Run code quality job in pipelines for tags
```

## Configuring jobs using variables

The Code Quality job supports environment variables that users can set to
configure job execution at runtime.

For a list of available environment variables, see
[Environment variables](https://gitlab.com/gitlab-org/ci-cd/codequality#environment-variables).

## Implementing a custom tool

It's possible to have a custom tool provide Code Quality reports in GitLab. To
do this:

1. Define a job in your `.gitlab-ci.yml` file that generates the
   [Code Quality report
   artifact](../../../ci/pipelines/job_artifacts.md#artifactsreportscodequality).
1. Configure your tool to generate the Code Quality report artifact as a JSON
   file that implements a subset of the [Code Climate
   spec](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types).

The Code Quality report artifact JSON file must contain an array of objects
with the following properties:

| Name                   | Description                                                                            |
| ---------------------- | -------------------------------------------------------------------------------------- |
| `description`          | A description of the code quality violation.                                           |
| `fingerprint`          | A unique fingerprint to identify the code quality violation. For example, an MD5 hash. |
| `severity`             | A severity string (can be `info`, `minor`, `major`, `critical`, or `blocker`).                          |
| `location.path`        | The relative path to the file containing the code quality violation.                   |
| `location.lines.begin` | The line on which the code quality violation occurred.                                 |

Example:

```json
[
  {
    "description": "'unused' is assigned a value but never used.",
    "fingerprint": "7815696ecbf1c96e6894b779456d330e",
    "severity": "minor",
    "location": {
      "path": "lib/index.js",
      "lines": {
        "begin": 42
      }
    }
  }
]
```

NOTE:
Although the Code Climate spec supports more properties, those are ignored by
GitLab.

## Code Quality reports

After the Code Quality job completes:

- Potential changes to code quality are shown directly in the merge request.
  The Code Quality widget in the merge request compares the reports from the base and head of the branch,
  then lists any violations that are resolved or created when the branch is merged.
- The full JSON report is available as a
  [downloadable artifact](../../../ci/pipelines/job_artifacts.md#downloading-artifacts)
  for the `code_quality` job.
- The full list of code quality violations generated by a pipeline is shown in the
  Code Quality tab of the Pipeline Details page. **(STARTER)**

### Generating an HTML report

In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/ci-cd/codequality/-/issues/10),
it is possible to generate an HTML report file by setting the `REPORT_FORMAT`
variable to `html`. This is useful if you just want to view the report in a more
human-readable format or to publish this artifact on GitLab Pages for even
easier reviewing.

```yaml
include:
  - template: Code-Quality.gitlab-ci.yml

code_quality:
  variables:
    REPORT_FORMAT: html
  artifacts:
    paths: [gl-code-quality-report.html]
```

It's also possible to generate both JSON and HTML report files by defining
another job and using `extends: code_quality`:

```yaml
include:
  - template: Code-Quality.gitlab-ci.yml

code_quality_html:
  extends: code_quality
  variables:
    REPORT_FORMAT: html
  artifacts:
    paths: [gl-code-quality-report.html]
```

## Extending functionality

### Using Analysis Plugins

Should there be a need to extend the default functionality provided by Code Quality, as stated in [Code Quality](#code-quality), [Analysis Plugins](https://docs.codeclimate.com/docs/list-of-engines) are available.

For example, to use the [SonarJava analyzer](https://docs.codeclimate.com/docs/sonar-java),
add a file named `.codeclimate.yml` containing the [enablement code](https://docs.codeclimate.com/docs/sonar-java#enable-the-plugin)
for the plugin to the root of your repository:

```yaml
version: "2"
plugins:
  sonar-java:
    enabled: true
```

This adds SonarJava to the `plugins:` section of the [default `.codeclimate.yml`](https://gitlab.com/gitlab-org/ci-cd/codequality/-/blob/master/codeclimate_defaults/.codeclimate.yml)
included in your project.

Changes to the `plugins:` section do not affect the `exclude_patterns` section of the
default `.codeclimate.yml`. See the Code Climate documentation for
[excluding files and folders](https://docs.codeclimate.com/docs/excluding-files-and-folders)
for more details.

Here's [an example project](https://gitlab.com/jheimbuck_gl/jh_java_example_project) that uses Code Quality with a `.codeclimate.yml` file.

## Troubleshooting

### Changing the default configuration has no effect

A common issue is that the terms `Code Quality` (GitLab specific) and `Code Climate`
(Engine used by GitLab) are very similar. You must add a **`.codeclimate.yml`** file
to change the default configuration, **not** a `.codequality.yml` file. If you use
the wrong filename, the [default `.codeclimate.yml`](https://gitlab.com/gitlab-org/ci-cd/codequality/-/blob/master/codeclimate_defaults/.codeclimate.yml)
is still used.

### No Code Quality report is displayed in a Merge Request

This can be due to multiple reasons:

- You just added the Code Quality job in your `.gitlab-ci.yml`. The report does not
  have anything to compare to yet, so no information can be displayed. It only displays
  after future merge requests have something to compare to.
- Your pipeline is not set to run the code quality job on your default branch. If there is no report generated from the default branch, your MR branch reports will not have anything to compare to.
- If no [degradation or error is detected](https://docs.codeclimate.com/docs/maintainability#section-checks),
  nothing is displayed.
- The [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) CI/CD
  setting can cause the Code Quality artifact(s) to expire faster than desired.
- If you use the [`REPORT_STDOUT` environment variable](https://gitlab.com/gitlab-org/ci-cd/codequality#environment-variables), no report file is generated and nothing displays in the merge request.
- Large `codeclimate.json` files (esp. >10 MB) are [known to prevent the report from being displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/2737).
  As a work-around, try removing [properties](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types)
  that are [ignored by GitLab](#implementing-a-custom-tool). You can:
  - Configure the Code Quality tool to not output those types.
  - Use `sed`, `awk` or similar commands in the `.gitlab-ci.yml` script to
    edit the `codeclimate.json` before the job completes.

### Only a single Code Quality report is displayed, but more are defined

GitLab only uses the Code Quality artifact from the latest created job (with the largest job ID).
If multiple jobs in a pipeline generate a code quality artifact, those of earlier jobs are ignored.
To avoid confusion, configure only one job to generate a `codeclimate.json`.