summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorKeith Short <keithshort@chromium.org>2023-01-12 09:01:44 -0700
committerChromeos LUCI <chromeos-scoped@luci-project-accounts.iam.gserviceaccount.com>2023-01-13 22:48:14 +0000
commit0f52d7a9845beffe639aa7cefa195aaae274aad1 (patch)
treeb656cdd505df3ec03057d4c6baaf17000894e632
parent55198413f19b7bb43ed7ff9578e30c663ef4c664 (diff)
downloadchrome-ec-0f52d7a9845beffe639aa7cefa195aaae274aad1.tar.gz
docs: Document the code coverage test requirements
Document the code coverage test requirements. BUG=b:261987897 BRANCH=none TEST=View docs in preview mode Change-Id: I0bde6ed3f36f88f4e7d1102c29ea73d112b82e74 Signed-off-by: Keith Short <keithshort@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4166320 Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com> Reviewed-by: Jeremy Bettis <jbettis@chromium.org>
-rw-r--r--README.md7
-rw-r--r--docs/chromeos-ec-firmware-test-requirements.md121
-rw-r--r--docs/code_coverage.md8
-rw-r--r--docs/sitemap.md1
4 files changed, 136 insertions, 1 deletions
diff --git a/README.md b/README.md
index 80c131e7c9..8c04be20c3 100644
--- a/README.md
+++ b/README.md
@@ -175,7 +175,7 @@ Building directly from the EC repository:
```bash
cros_sdk
-cd ~/trunk/src/platform/ec
+cd ~/chromiumos/src/platform/ec
make -j BOARD=<boardname>
```
@@ -527,6 +527,9 @@ Please refer to existing commits (`git log`) to see the proper format for the
commit message. If you have configured git properly, running `git commit` with
the `-s` argument will add the Signed-off-by line for you.
+Note that code you submit must adhere to the [ChromeOS EC Firmware Test
+Requirements].
+
## Debugging
While adding `printf` statements can be handy, there are some other options for
@@ -677,3 +680,5 @@ cat /tmp/artifact_bundle_metadata
cat /tmp/metrics_build
ls -l /tmp/artifact_bundles/
```
+
+[ChromeOS EC Firmware Test Requirements]: ./docs/chromeos-ec-firmware-test-requirements.md
diff --git a/docs/chromeos-ec-firmware-test-requirements.md b/docs/chromeos-ec-firmware-test-requirements.md
new file mode 100644
index 0000000000..0a173769b3
--- /dev/null
+++ b/docs/chromeos-ec-firmware-test-requirements.md
@@ -0,0 +1,121 @@
+# ChromeOS EC Firmware Code Test Requirements
+
+[TOC]
+
+## Overview
+
+The ChromeOS EC firmware requires that all C source and header file code changes
+include tests covering at least 80% of any new or changed lines of code.
+
+The Gerrit commit-queue enforces the coverage requirement with the
+`firmware-zephyr-cov-cq` builder.
+
+The Google EC firmware team plans to gradually increase the code coverage
+requirement from 80% to 95% for new or changed lines of code.
+
+## Rationale
+
+Starting 2H’22, all new Chromebook projects use the Zephyr RTOS as the
+foundation for the ChromeOS EC firmware. As part of the switch to Zephyr RTOS,
+Google wrote unit and integration tests to verify functionality of the EC
+firmware. These tests run directly on your development system using the
+native_posix environment and do not require any Chromebook hardware.
+
+These tests improve the quality of the EC firmware, reducing the number of EC
+crashes and bugs found on shipping Chromebooks.
+
+The ChromeOS EC firmware requires that all projects based on the Zephyr RTOS
+achieve code coverage of 90% or better.
+
+A Gitlab project creates and publishes the code coverage reports for the
+Zephyr EC projects. The latest results are available on the Gitlab [Jobs page].
+Refer to the [Gitlab CI] documentation for implementation details.
+
+## Resources for writing tests
+
+* The [Zephyr Testing] documentation in the ChromeOS EC firmware provides the
+ most up-to-date information for writing tests specifically for the ChromeOS EC
+ firmware.
+
+* [Zephyr Test Framework] documentation in the Zephyr Project provides general
+ information about the test framework used by Zephyr RTOS.
+
+### Testing project specific code
+
+* The board
+
+### Generating coverage reports
+
+* [Zephyr ztest code coverage] provides instructions for generating code coverage
+reports locally on your machine.
+
+* [Code Coverage in CQ] provides details for viewing the coverage information
+ directly from Gerrit or by using Google's code search tool.
+
+### Bypassing the code coverage requirement
+
+In limited cases, you may amend your commit message to include the
+`LOW_COVERAGE_REASON` tag. This tag bypasses the code coverage requirement
+enforced by Gerrit. Simply add the tag followed by a description to justify
+bypassing code coverage.
+
+```
+LOW_COVERAGE_REASON=no emulator for the ANX7483 exists b/248086547
+```
+
+Permissible reasons for bypassing the code coverage requirements include:
+
+* Fixing a high-priority bug that blocks a release or some other milestone.
+
+* Fixing a bug in an existing driver that is currently untested.
+
+* Modifying on-chip EC peripheral drivers. See the [Known Issues](#known_issues)
+ section below for details.
+
+When bypassing code coverage, please open a bug to track the work required to
+create tests.
+
+Reviewers may reject your low coverage reason and request that you update or add
+tests for your change.
+
+## Exceptions to code coverage
+
+### Legacy EC code
+
+Zephyr EC projects share large portions of the legacy code found under the
+[`platform/ec`] repository. The code coverage requirements apply to the shared
+code, with exceptions for the following directories:
+
+* [`platform/ec/baseboard`]
+* [`platform/ec/board`]
+* [`platform/ec/chip`]
+
+### Known issues
+
+There are no tests for the chip specific code found under the
+[`platform/ec/zephyr/drivers`] directory. Due to include file conflicts, it is
+not currently known how to test these drivers.
+
+### False positives
+
+The coverage tool may falsely mark a line of code as executable but the line of
+code is not reachable. An example could be a switch statement, but the default
+label is not reachable. In these cases add a comment using exclusion markers,
+`LCOV_EXCL_*`, to the file. See the [geninfo manpage] for more information.
+
+Do not use exclusion markers to bypass executable code or to avoid writing tests
+for difficult to reach code.
+
+
+[`platform/ec/baseboard`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/baseboard
+[`platform/ec/board`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/board
+[`platform/ec/chip`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/chip
+[`platform/ec/zephyr/drivers`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/zephyr/drivers
+[Jobs page]: https://gitlab.com/zephyr-ec/ec/-/jobs/
+[Gitlab CI]: ./gitlab.md
+[Zephyr Testing]: ./zephyr/ztest.md
+[Zephyr Test Framework]: https://docs.zephyrproject.org/latest/develop/test/ztest.html
+[Zephyr ztest code coverage]: ./code_coverage.md#Zephyr-ztest-code-coverage
+[Code Coverage in CQ]: ./code_coverage.md#code-coverage-in-cq
+[`platform/ec`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/
+[geninfo manpage]: https://manpages.debian.org/unstable/lcov/geninfo.1.en.html
diff --git a/docs/code_coverage.md b/docs/code_coverage.md
index 0a5eeb3150..fd09585321 100644
--- a/docs/code_coverage.md
+++ b/docs/code_coverage.md
@@ -5,6 +5,12 @@ tests in the EC codebase.
[TOC]
+## Test Coverage Requirements
+
+All changes to the EC code base require you include tests covering at least 80%
+of any new or changed lines of code. Refer to the [ChromeOS EC Firmware Test
+Requirements] for details.
+
## Availability
Code coverage is only available for host-based unit tests, as opposed to manual
@@ -148,3 +154,5 @@ and click on the latest successful build.
![LUCI post-submit screenshot](images/postsubmit.png)
From there, it is exactly the same steps as above to get to the artifacts.
+
+[ChromeOS EC Firmware Test Requirements]: ./chromeos-ec-firmware-test-requirements.md
diff --git a/docs/sitemap.md b/docs/sitemap.md
index cac7d4d1d4..0bdc3ede12 100644
--- a/docs/sitemap.md
+++ b/docs/sitemap.md
@@ -38,6 +38,7 @@
* [Unit Tests](./unit_tests.md)
* [Zephyr Testing](./zephyr/ztest.md)
* [Code Coverage](./code_coverage.md)
+* [ChromeOS EC Firmware Test Requirements](./chromeos-ec-firmware-test-requirements.md)
## Updaters