diff options
Diffstat (limited to 'third-party/benchmark/docs/AssemblyTests.md')
| -rw-r--r-- | third-party/benchmark/docs/AssemblyTests.md | 147 |
1 files changed, 0 insertions, 147 deletions
diff --git a/third-party/benchmark/docs/AssemblyTests.md b/third-party/benchmark/docs/AssemblyTests.md deleted file mode 100644 index 1fbdc269b53d..000000000000 --- a/third-party/benchmark/docs/AssemblyTests.md +++ /dev/null @@ -1,147 +0,0 @@ -# Assembly Tests - -The Benchmark library provides a number of functions whose primary -purpose in to affect assembly generation, including `DoNotOptimize` -and `ClobberMemory`. In addition there are other functions, -such as `KeepRunning`, for which generating good assembly is paramount. - -For these functions it's important to have tests that verify the -correctness and quality of the implementation. This requires testing -the code generated by the compiler. - -This document describes how the Benchmark library tests compiler output, -as well as how to properly write new tests. - - -## Anatomy of a Test - -Writing a test has two steps: - -* Write the code you want to generate assembly for. -* Add `// CHECK` lines to match against the verified assembly. - -Example: -```c++ - -// CHECK-LABEL: test_add: -extern "C" int test_add() { - extern int ExternInt; - return ExternInt + 1; - - // CHECK: movl ExternInt(%rip), %eax - // CHECK: addl %eax - // CHECK: ret -} - -``` - -#### LLVM Filecheck - -[LLVM's Filecheck](https://llvm.org/docs/CommandGuide/FileCheck.html) -is used to test the generated assembly against the `// CHECK` lines -specified in the tests source file. Please see the documentation -linked above for information on how to write `CHECK` directives. - -#### Tips and Tricks: - -* Tests should match the minimal amount of output required to establish -correctness. `CHECK` directives don't have to match on the exact next line -after the previous match, so tests should omit checks for unimportant -bits of assembly. ([`CHECK-NEXT`](https://llvm.org/docs/CommandGuide/FileCheck.html#the-check-next-directive) -can be used to ensure a match occurs exactly after the previous match). - -* The tests are compiled with `-O3 -g0`. So we're only testing the -optimized output. - -* The assembly output is further cleaned up using `tools/strip_asm.py`. -This removes comments, assembler directives, and unused labels before -the test is run. - -* The generated and stripped assembly file for a test is output under -`<build-directory>/test/<test-name>.s` - -* Filecheck supports using [`CHECK` prefixes](https://llvm.org/docs/CommandGuide/FileCheck.html#cmdoption-check-prefixes) -to specify lines that should only match in certain situations. -The Benchmark tests use `CHECK-CLANG` and `CHECK-GNU` for lines that -are only expected to match Clang or GCC's output respectively. Normal -`CHECK` lines match against all compilers. (Note: `CHECK-NOT` and -`CHECK-LABEL` are NOT prefixes. They are versions of non-prefixed -`CHECK` lines) - -* Use `extern "C"` to disable name mangling for specific functions. This -makes them easier to name in the `CHECK` lines. - - -## Problems Writing Portable Tests - -Writing tests which check the code generated by a compiler are -inherently non-portable. Different compilers and even different compiler -versions may generate entirely different code. The Benchmark tests -must tolerate this. - -LLVM Filecheck provides a number of mechanisms to help write -"more portable" tests; including [matching using regular expressions](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-pattern-matching-syntax), -allowing the creation of [named variables](https://llvm.org/docs/CommandGuide/FileCheck.html#filecheck-variables) -for later matching, and [checking non-sequential matches](https://llvm.org/docs/CommandGuide/FileCheck.html#the-check-dag-directive). - -#### Capturing Variables - -For example, say GCC stores a variable in a register but Clang stores -it in memory. To write a test that tolerates both cases we "capture" -the destination of the store, and then use the captured expression -to write the remainder of the test. - -```c++ -// CHECK-LABEL: test_div_no_op_into_shr: -extern "C" void test_div_no_op_into_shr(int value) { - int divisor = 2; - benchmark::DoNotOptimize(divisor); // hide the value from the optimizer - return value / divisor; - - // CHECK: movl $2, [[DEST:.*]] - // CHECK: idivl [[DEST]] - // CHECK: ret -} -``` - -#### Using Regular Expressions to Match Differing Output - -Often tests require testing assembly lines which may subtly differ -between compilers or compiler versions. A common example of this -is matching stack frame addresses. In this case regular expressions -can be used to match the differing bits of output. For example: - -```c++ -int ExternInt; -struct Point { int x, y, z; }; - -// CHECK-LABEL: test_store_point: -extern "C" void test_store_point() { - Point p{ExternInt, ExternInt, ExternInt}; - benchmark::DoNotOptimize(p); - - // CHECK: movl ExternInt(%rip), %eax - // CHECK: movl %eax, -{{[0-9]+}}(%rsp) - // CHECK: movl %eax, -{{[0-9]+}}(%rsp) - // CHECK: movl %eax, -{{[0-9]+}}(%rsp) - // CHECK: ret -} -``` - -## Current Requirements and Limitations - -The tests require Filecheck to be installed along the `PATH` of the -build machine. Otherwise the tests will be disabled. - -Additionally, as mentioned in the previous section, codegen tests are -inherently non-portable. Currently the tests are limited to: - -* x86_64 targets. -* Compiled with GCC or Clang - -Further work could be done, at least on a limited basis, to extend the -tests to other architectures and compilers (using `CHECK` prefixes). - -Furthermore, the tests fail for builds which specify additional flags -that modify code generation, including `--coverage` or `-fsanitize=`. - |