diff options
author | Keith Short <keithshort@chromium.org> | 2023-01-12 09:01:44 -0700 |
---|---|---|
committer | Chromeos LUCI <chromeos-scoped@luci-project-accounts.iam.gserviceaccount.com> | 2023-01-13 22:48:14 +0000 |
commit | 0f52d7a9845beffe639aa7cefa195aaae274aad1 (patch) | |
tree | b656cdd505df3ec03057d4c6baaf17000894e632 | |
parent | 55198413f19b7bb43ed7ff9578e30c663ef4c664 (diff) | |
download | chrome-ec-0f52d7a9845beffe639aa7cefa195aaae274aad1.tar.gz |
docs: Document the code coverage test requirements
Document the code coverage test requirements.
BUG=b:261987897
BRANCH=none
TEST=View docs in preview mode
Change-Id: I0bde6ed3f36f88f4e7d1102c29ea73d112b82e74
Signed-off-by: Keith Short <keithshort@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4166320
Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com>
Reviewed-by: Jeremy Bettis <jbettis@chromium.org>
-rw-r--r-- | README.md | 7 | ||||
-rw-r--r-- | docs/chromeos-ec-firmware-test-requirements.md | 121 | ||||
-rw-r--r-- | docs/code_coverage.md | 8 | ||||
-rw-r--r-- | docs/sitemap.md | 1 |
4 files changed, 136 insertions, 1 deletions
@@ -175,7 +175,7 @@ Building directly from the EC repository: ```bash cros_sdk -cd ~/trunk/src/platform/ec +cd ~/chromiumos/src/platform/ec make -j BOARD=<boardname> ``` @@ -527,6 +527,9 @@ Please refer to existing commits (`git log`) to see the proper format for the commit message. If you have configured git properly, running `git commit` with the `-s` argument will add the Signed-off-by line for you. +Note that code you submit must adhere to the [ChromeOS EC Firmware Test +Requirements]. + ## Debugging While adding `printf` statements can be handy, there are some other options for @@ -677,3 +680,5 @@ cat /tmp/artifact_bundle_metadata cat /tmp/metrics_build ls -l /tmp/artifact_bundles/ ``` + +[ChromeOS EC Firmware Test Requirements]: ./docs/chromeos-ec-firmware-test-requirements.md diff --git a/docs/chromeos-ec-firmware-test-requirements.md b/docs/chromeos-ec-firmware-test-requirements.md new file mode 100644 index 0000000000..0a173769b3 --- /dev/null +++ b/docs/chromeos-ec-firmware-test-requirements.md @@ -0,0 +1,121 @@ +# ChromeOS EC Firmware Code Test Requirements + +[TOC] + +## Overview + +The ChromeOS EC firmware requires that all C source and header file code changes +include tests covering at least 80% of any new or changed lines of code. + +The Gerrit commit-queue enforces the coverage requirement with the +`firmware-zephyr-cov-cq` builder. + +The Google EC firmware team plans to gradually increase the code coverage +requirement from 80% to 95% for new or changed lines of code. + +## Rationale + +Starting 2H’22, all new Chromebook projects use the Zephyr RTOS as the +foundation for the ChromeOS EC firmware. As part of the switch to Zephyr RTOS, +Google wrote unit and integration tests to verify functionality of the EC +firmware. These tests run directly on your development system using the +native_posix environment and do not require any Chromebook hardware. + +These tests improve the quality of the EC firmware, reducing the number of EC +crashes and bugs found on shipping Chromebooks. + +The ChromeOS EC firmware requires that all projects based on the Zephyr RTOS +achieve code coverage of 90% or better. + +A Gitlab project creates and publishes the code coverage reports for the +Zephyr EC projects. The latest results are available on the Gitlab [Jobs page]. +Refer to the [Gitlab CI] documentation for implementation details. + +## Resources for writing tests + +* The [Zephyr Testing] documentation in the ChromeOS EC firmware provides the + most up-to-date information for writing tests specifically for the ChromeOS EC + firmware. + +* [Zephyr Test Framework] documentation in the Zephyr Project provides general + information about the test framework used by Zephyr RTOS. + +### Testing project specific code + +* The board + +### Generating coverage reports + +* [Zephyr ztest code coverage] provides instructions for generating code coverage +reports locally on your machine. + +* [Code Coverage in CQ] provides details for viewing the coverage information + directly from Gerrit or by using Google's code search tool. + +### Bypassing the code coverage requirement + +In limited cases, you may amend your commit message to include the +`LOW_COVERAGE_REASON` tag. This tag bypasses the code coverage requirement +enforced by Gerrit. Simply add the tag followed by a description to justify +bypassing code coverage. + +``` +LOW_COVERAGE_REASON=no emulator for the ANX7483 exists b/248086547 +``` + +Permissible reasons for bypassing the code coverage requirements include: + +* Fixing a high-priority bug that blocks a release or some other milestone. + +* Fixing a bug in an existing driver that is currently untested. + +* Modifying on-chip EC peripheral drivers. See the [Known Issues](#known_issues) + section below for details. + +When bypassing code coverage, please open a bug to track the work required to +create tests. + +Reviewers may reject your low coverage reason and request that you update or add +tests for your change. + +## Exceptions to code coverage + +### Legacy EC code + +Zephyr EC projects share large portions of the legacy code found under the +[`platform/ec`] repository. The code coverage requirements apply to the shared +code, with exceptions for the following directories: + +* [`platform/ec/baseboard`] +* [`platform/ec/board`] +* [`platform/ec/chip`] + +### Known issues + +There are no tests for the chip specific code found under the +[`platform/ec/zephyr/drivers`] directory. Due to include file conflicts, it is +not currently known how to test these drivers. + +### False positives + +The coverage tool may falsely mark a line of code as executable but the line of +code is not reachable. An example could be a switch statement, but the default +label is not reachable. In these cases add a comment using exclusion markers, +`LCOV_EXCL_*`, to the file. See the [geninfo manpage] for more information. + +Do not use exclusion markers to bypass executable code or to avoid writing tests +for difficult to reach code. + + +[`platform/ec/baseboard`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/baseboard +[`platform/ec/board`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/board +[`platform/ec/chip`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/chip +[`platform/ec/zephyr/drivers`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/zephyr/drivers +[Jobs page]: https://gitlab.com/zephyr-ec/ec/-/jobs/ +[Gitlab CI]: ./gitlab.md +[Zephyr Testing]: ./zephyr/ztest.md +[Zephyr Test Framework]: https://docs.zephyrproject.org/latest/develop/test/ztest.html +[Zephyr ztest code coverage]: ./code_coverage.md#Zephyr-ztest-code-coverage +[Code Coverage in CQ]: ./code_coverage.md#code-coverage-in-cq +[`platform/ec`]: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform/ec/ +[geninfo manpage]: https://manpages.debian.org/unstable/lcov/geninfo.1.en.html diff --git a/docs/code_coverage.md b/docs/code_coverage.md index 0a5eeb3150..fd09585321 100644 --- a/docs/code_coverage.md +++ b/docs/code_coverage.md @@ -5,6 +5,12 @@ tests in the EC codebase. [TOC] +## Test Coverage Requirements + +All changes to the EC code base require you include tests covering at least 80% +of any new or changed lines of code. Refer to the [ChromeOS EC Firmware Test +Requirements] for details. + ## Availability Code coverage is only available for host-based unit tests, as opposed to manual @@ -148,3 +154,5 @@ and click on the latest successful build. ![LUCI post-submit screenshot](images/postsubmit.png) From there, it is exactly the same steps as above to get to the artifacts. + +[ChromeOS EC Firmware Test Requirements]: ./chromeos-ec-firmware-test-requirements.md diff --git a/docs/sitemap.md b/docs/sitemap.md index cac7d4d1d4..0bdc3ede12 100644 --- a/docs/sitemap.md +++ b/docs/sitemap.md @@ -38,6 +38,7 @@ * [Unit Tests](./unit_tests.md) * [Zephyr Testing](./zephyr/ztest.md) * [Code Coverage](./code_coverage.md) +* [ChromeOS EC Firmware Test Requirements](./chromeos-ec-firmware-test-requirements.md) ## Updaters |