diff options
| author | Tom Hughes <tomhughes@chromium.org> | 2022-09-21 14:08:36 -0700 |
|---|---|---|
| committer | Tom Hughes <tomhughes@chromium.org> | 2022-09-22 12:59:38 -0700 |
| commit | c453fd704268ef72de871b0c5ac7a989de662334 (patch) | |
| tree | fcf6ce5810f9ff9e3c8cce434812dd75492269ed /docs/code_coverage.md | |
| parent | 6c1587ca70f558b4f96b3f0b18ad8b027d3ba99d (diff) | |
| parent | 28712dae9d7ed1e694f7622cc083afa71090d4d5 (diff) | |
| download | chrome-ec-firmware-fpmcu-dartmonkey-release.tar.gz | |
Merge remote-tracking branch cros/main into firmware-fpmcu-dartmonkey-releasefirmware-fpmcu-dartmonkey-release
Generated by: ./util/update_release_branch.py --board dartmonkey --relevant_paths_file
./util/fingerprint-relevant-paths.txt firmware-fpmcu-dartmonkey-release
Relevant changes:
git log --oneline 6c1587ca70..28712dae9d -- board/nocturne_fp
board/dartmonkey common/fpsensor docs/fingerprint driver/fingerprint
util/getversion.sh
ded9307b79 util/getversion.sh: Fix version when not in a git repo
956055e692 board: change Google USB vendor info
71b2ef709d Update license boilerplate text in source code files
33e11afda0 Revert "fpsensor: Build fpsensor source file with C++"
c8d0360723 fpsensor: Build fpsensor source file with C++
bc113abd53 fpsensor: Fix g++ compiler error
150a58a0dc fpsensor: Fix fp_set_sensor_mode return type
b33b5ce85b fpsensor: Remove nested designators for C++ compatibility
2e864b2539 tree-wide: const-ify argv for console commands
56d8b360f9 test: Add test for get ikm failure when seed not set
3a3d6c3690 test: Add test for fpsensor trivial key failure
233e6bbd08 fpsensor_crypto: Abstract calls to hmac_SHA256
0a041b285b docs/fingerprint: Typo correction
c03fab67e2 docs/fingerprint: Fix the path of fputils.py
0b5d4baf5a util/getversion.sh: Fix empty file list handling
6e128fe760 FPMCU dev board environment with Satlab
3eb29b6aa5 builtin: Move ssize_t to sys/types.h
345d62ebd1 docs/fingerprint: Update power numbers for latest dartmonkey release
c25ffdb316 common: Conditionally support printf %l and %i modifiers
9a3c514b45 test: Add a test to check if the debugger is connected
54e603413f Move standard library tests to their own file
43fa6b4bf8 docs/fingerprint: Update power numbers for latest bloonchipper release
25536f9a84 driver/fingerprint/fpc/bep/fpc_sensor_spi.c: Format with clang-format
4face99efd driver/fingerprint/fpc/libfp/fpc_sensor_pal.h: Format with clang-format
738de2b575 trng: Rename rand to trng_rand
14b8270edd docs/fingerprint: Update dragonclaw power numbers
0b268f93d1 driver/fingerprint/fpc/libfp/fpc_private.c: Format with clang-format
f80da163f2 driver/fingerprint/fpc/libfp/fpc_private.h: Format with clang-format
a0751778f4 board/nocturne_fp/ro_workarounds.c: Format with clang-format
5e9c85c9b1 driver/fingerprint/fpc/libfp/fpc_sensor_pal.c: Format with clang-format
c1f9dd3cf8 driver/fingerprint/fpc/libfp/fpc_bio_algorithm.h: Format with clang-format
eb1e1bed8d driver/fingerprint/fpc/libfp/fpc1145_private.h: Format with clang-format
6e7b611821 driver/fingerprint/fpc/bep/fpc_bio_algorithm.h: Format with clang-format
e0589cd5e2 driver/fingerprint/fpc/bep/fpc1035_private.h: Format with clang-format
58f0246dbe board/nocturne_fp/board_ro.c: Format with clang-format
7905e556a0 common/fpsensor/fpsensor_crypto.c: Format with clang-format
21289d170c driver/fingerprint/fpc/bep/fpc1025_private.h: Format with clang-format
98a20f937e common/fpsensor/fpsensor_state.c: Format with clang-format
a2d255d8af common/fpsensor/fpsensor.c: Format with clang-format
84e53a65da board/nocturne_fp/board.h: Format with clang-format
73055eeb3f driver/fingerprint/fpc/bep/fpc_private.c: Format with clang-format
0f7b5cb509 common/fpsensor/fpsensor_private.h: Format with clang-format
1ceade6e65 driver/fingerprint/fpc/bep/fpc_private.h: Format with clang-format
dca9d74321 Revert "trng: Rename rand to trng_rand"
a6b0b3554f trng: Rename rand to trng_rand
28d0b75b70 third_party/boringssl: Remove unused header
BRANCH=None
BUG=b:244387210 b:242720240 b:215613183 b:242720910 b:236386294
BUG=b:234181908 b:244781166 b:234781655 b:234143158 b:234181908
BUG=b:237344361 b:236025198 b:234181908 b:180945056 chromium:1098010
BUG=b:246424843 b:234181908 b:131913998
TEST=`make -j buildall`
TEST=./util/run_device_tests.py --board dartmonkey
Test "aes": PASSED
Test "cec": PASSED
Test "cortexm_fpu": PASSED
Test "crc": PASSED
Test "flash_physical": PASSED
Test "flash_write_protect": PASSED
Test "fpsensor_hw": PASSED
Test "fpsensor_spi_ro": PASSED
Test "fpsensor_spi_rw": PASSED
Test "fpsensor_uart_ro": PASSED
Test "fpsensor_uart_rw": PASSED
Test "mpu_ro": PASSED
Test "mpu_rw": PASSED
Test "mutex": PASSED
Test "pingpong": PASSED
Test "printf": PASSED
Test "queue": PASSED
Test "rollback_region0": PASSED
Test "rollback_region1": PASSED
Test "rollback_entropy": PASSED
Test "rtc": PASSED
Test "sha256": PASSED
Test "sha256_unrolled": PASSED
Test "static_if": PASSED
Test "stdlib": PASSED
Test "system_is_locked_wp_on": PASSED
Test "system_is_locked_wp_off": PASSED
Test "timer_dos": PASSED
Test "utils": PASSED
Test "utils_str": PASSED
Test "panic_data_dartmonkey_v2.0.2887": PASSED
Test "panic_data_nocturne_fp_v2.2.64": PASSED
Test "panic_data_nami_fp_v2.2.144": PASSED
Force-Relevant-Builds: all
Signed-off-by: Tom Hughes <tomhughes@chromium.org>
Change-Id: I2c312583a709fedae8fe11d92c22328c3b634bc7
Diffstat (limited to 'docs/code_coverage.md')
| -rw-r--r-- | docs/code_coverage.md | 103 |
1 files changed, 94 insertions, 9 deletions
diff --git a/docs/code_coverage.md b/docs/code_coverage.md index a15c61b358..7f0c544692 100644 --- a/docs/code_coverage.md +++ b/docs/code_coverage.md @@ -23,6 +23,29 @@ data into a code coverage report using the `lcov` and `genhtml` tools. The coverage report top-level page is `build/coverage/coverage_rpt/index.html`. +To get a report for one specific board's coverage run these commands: + +``` +BOARD=eldrid +make -j$(nproc) build/coverage/initial-${BOARD}.info test-coverage +# Merge board coverage and test coverage +lcov -o build/coverage/${BOARD}_merged.info --rc lcov_branch_coverage=1 \ + -a build/coverage/initial-${BOARD}.info -a build/coverage/lcov.info +# Filter out some unhelpful paths +lcov -o build/coverage/${BOARD}_filtered.info --rc lcov_branch_coverage=1 \ + -r build/coverage/${BOARD}_merged.info ${PWD}'/third_party/**' \ + ${PWD}'/build/**' '/usr/include/**' '/usr/lib/**' '${PWD}/test/**' \ + ${PWD}'/private/fingerprint/google-fpalg/mcutest/**' +# Restrict to only files used by the board +grep "SF:" "build/coverage/initial-${BOARD}.info" | sort -u | \ + sed -e 's|^SF:||' | xargs lcov --rc lcov_branch_coverage=1 \ + -o build/coverage/${BOARD}_final.info \ + -e build/coverage/${BOARD}_filtered.info +# Generate HTML +genhtml --branch-coverage -q -o build/coverage/${BOARD}_rpt \ + -t "${BOARD} coverage" -s build/coverage/${BOARD}_final.info +``` + ### Noise in the build output When building for code coverage, you may see multiple warnings of the form @@ -38,28 +61,90 @@ appear to be caused in part by using relative paths instead of absolute paths.) To build the Zephyr unit tests for code coverage run: -`zmake test --host-tests-only --coverage` -`genhtml -q -o build/zephyr/coverage_rpt/ build/zephyr/all_tests.info` +`./twister -v -i --coverage -p native_posix -p unit_testing` The coverage report top-level page is -`build/zephyr/coverage_rpt/index.html`. +`twister-out/coverage/index.html`. However you probably want to merge that with a single board's coverage report also, so that you can include code that is not part of any test as well. ``` zmake build --coverage herobrine -zmake test --host-tests-only --coverage +./twister -v -i --coverage -p native_posix -p unit_testing genhtml -q -s --branch-coverage -o build/zephyr/coverage_rpt/ \ - build/zephyr/all_tests.info build/zephyr/herobrine/output/zephyr.info + twister-out/coverage.info build/zephyr/herobrine/output/zephyr.info ``` The coverage report top-level page is `build/zephyr/coverage_rpt/index.html`. For coverage report for a single test you can run: -`zmake test --coverage <TESTNAME>` +`./twister -v -i --coverage -p native_posix -p unit_testing -s <pathToTest>/<testName>` + +Example of running test tasks.default from zephyr/test/tasks/testcase.yaml: +`./twister -v -i --coverage -p native_posix -p unit_testing -s zephyr/test/tasks/tasks.default` + +## Code Coverage in CQ + +There are several ways to see the code coverage without running the tests +locally, depending on what information you want to see. Many of the links +below are only available to Googlers or TVCs with google.com accounts. + +### Code search + +To see the coverage of each directory, visit +http://cs/chromeos_public/src/platform/ec/ and turn on the "Directory Coverage" +layer. The denominator for the percentage covered is not clear, so these +numbers are really only useful if you are looking in very general terms. I.e. +zephyr is covered better than common. Don't get too fixated on the specific +percent shown. The results are also the last 7 days of builds combined, so there +may be some odd results if the code has changed greatly in the last week. + + + +The coverage of files is much more useful. If you are about to write a test +and not sure what to focus on, you can look at the uncovered lines in code +search. Visit [a file](http://cs/chromeos_public/src/platform/ec/common/mkbp_event.c) +in code search and make sure the "File Coverage" layer is enabled. Lines that +are not covered by any test are in red, tested lines are in green, and uncolored +lines were not built at all in any board or test. + + + +### Presubmit + +Every gerrit cl, if you did a dry-run or full run of the CQ will have coverage +results. They are slightly difficult to get to, but are very useful. + +On the "Checks" tab, find the build "firmware-zephyr-cov-cq" and open it. + + + +On the LUCI page, expand the "test firmware" step and click on "response". That +will show you a very basic summary of the coverage. + + + +For a detailed report, you can download the coverage report. Expand "try to +upload artifacts", then "upload artifacts", and click on "gs upload dir". + + + +From there, click on the download icon for the html.tbz2 file, and untar it +locally. Open lcov_rpt/index.html to view your results. + + + +### Post-submit + +If you are interested in the state of the world, not a specific CL, and the +coverage info in Code Search is not sufficient, you can download the coverage +report from the post-submit CQ build. + +Visit https://ci.chromium.org/p/chromeos/builders/postsubmit/firmware-zephyr-cov-postsubmit +and click on the latest successful build. + + -Example: -`zmake test --coverage test-drivers` -`genhtml -q -o build/zephyr/test-drivers/output/coverage_rpt/ build/zephyr/test-drivers/output/zephyr.info` +From there, it is exactly the same steps as above to get to the artifacts. |