summaryrefslogtreecommitdiff
path: root/INTERPRETING.md
diff options
context:
space:
mode:
Diffstat (limited to 'INTERPRETING.md')
-rw-r--r--INTERPRETING.md191
1 files changed, 191 insertions, 0 deletions
diff --git a/INTERPRETING.md b/INTERPRETING.md
new file mode 100644
index 000000000..ed76fae8e
--- /dev/null
+++ b/INTERPRETING.md
@@ -0,0 +1,191 @@
+# Interpreting Test262 Tests
+
+All tests are declared as text files located within this project's `test`
+directory. In order to execute Test262 tests, runtimes must observe the
+following semantics.
+
+## Test Execution
+
+Test262 tests are only valid under the runtime environment conditions described
+here. Test environments may be further modified according to the metadata
+contained with each test--refer to the "Metadata" section for more details.
+
+### Realm Isolation
+
+Each test must be executed in a new [ECMAScript
+realm](https://tc39.github.io/ecma262/#sec-code-realms) dedicated to that test.
+Unless configured otherwise (via the `module` flag), source text must be
+interpreted as [global
+code](http://www.ecma-international.org/ecma-262/6.0/#sec-types-of-source-code).
+
+### Test262-Defined Bindings
+
+The contents of the following files must be evaluated in the test realm's
+global scope prior to test execution:
+
+1. `harness/assert.js`
+2. `harness/sta.js`
+
+### Host-Defined Functions
+
+The following values must be defined as writable, configurable, non-enumerable
+properties of the global scope prior to test execution.
+
+- **`print`** A function that exposes the string value of its first argument to
+ the test runner. This is used as a communication mechanism for asynchronous
+ tests (via the `async` flag, described below).
+
+### Strict Mode
+
+Unless configured otherwise (via the `noStrict`, `onlyStrict`, `module`, or
+`raw` flags), each test must be executed twice: once in ECMAScript's non-strict
+mode, and again in ECMAScript's strict mode. To run in strict mode, the test
+contents must be modified prior to execution--[a "use strict"
+directive](https://tc39.github.io/ecma262/#sec-directive-prologues-and-the-use-strict-directive)
+must be inserted as the initial character sequence of the file, followed by a
+semicolon (`;`) and newline character (`\n`):
+
+ "use strict";
+
+This must precede any additional text modifications described by test metadata.
+
+## Test Results
+
+By default, tests signal failure by generating an uncaught exception. If
+execution completes without generating an exception, the test must be
+interpreted as "passing." Any uncaught exception must be interpreted as test
+failure. These semantics may be modified by any test according to the metadata
+declared within the test itself (via the `negative` attribute and the `async`
+flag, described below).
+
+## Metadata
+
+Each test file may define metadata that describe additional requirements. This
+information is delimited by the token sequence `/*---` and `---*/` and is
+structured as [YAML](http://yaml.org/).
+
+### `negative`
+
+These tests are expected to generate an uncaught exception. The value of this
+attribute is the name of the constructor of the expected error. If a test
+configured with the `negative` attribute completes without throwing an
+exception, or if the name of the thrown exception's constructor does not match
+the specified constructor name, the test must be interpreted as "failing."
+
+*Example:*
+
+```js
+/*---
+negative: ReferenceError
+---*/
+unresolvable;
+```
+
+### `includes`
+
+One or more files whose content must be evaluated in the test realm's global
+scope prior to test execution. These files are located within the `harness/`
+directory of the Test262 project.
+
+*Example*
+
+```js
+/*---
+includes: [testBuildInObject.js]
+---*/
+testBuiltInObject(Number.prototype.toLocaleString, true, false, [], 0);
+```
+
+### `flags`
+
+The `flags` attribute is an optional value that specifies one or more of the
+following strings:
+
+- **`onlyStrict`** The test must be executed just once--in strict mode, only.
+ This must be accomplished using the transformation described in the section
+ titled "Strict Mode".
+
+ *Example*
+
+ ```js
+ /*---
+ flags: [onlyStrict]
+ ---*/
+ var thisVal = null;
+ [null].forEach(function() {
+ thisVal = this;
+ });
+ assert.sameValue(thisVal, undefined);
+ ```
+
+- **`noStrict`** The test must be executed just once--in non-strict mode, only.
+ In other words, the transformation described by the section titled "Strict
+ Mode" must **not** be applied to these tests.
+
+ *Example*
+
+ ```js
+ /*---
+ flags: [noStrict]
+ ---*/
+ var thisVal = null;
+ [null].forEach(function() {
+ thisVal = this;
+ });
+ assert.notSameValue(thisVal, undefined);
+ assert.sameValue(thisVal, this);
+ ```
+
+- **`module`** The test source code must be interpreted as [module
+ code](http://www.ecma-international.org/ecma-262/6.0/#sec-types-of-source-code).
+ In addition, this flag negates the default requirement to execute the test
+ both in strict mode and in non-strict mode. In other words, the
+ transformation described by the section titled "Strict Mode" must **not** be
+ applied to these tests.
+
+ *Example*
+
+ ```js
+ /*---
+ flags: [module]
+ ---*/
+ export default function* g() {}
+ ```
+
+- **`raw`** The test source code must not be modified in any way, and the test
+ must be executed just once (in non-strict mode, only).
+
+ *Example*
+
+ ```js
+ /*---
+ flags: [raw]
+ ---*/
+ 'use strict'
+ [0]
+ 's'.p = null;
+ ```
+
+- **`async`** The file `harness/donePrintHandle.js` must be evaluated in the
+ test realm's global scope prior to test execution. The test must not be
+ considered complete until the implementation-defined `print` function has
+ been invoked or some length of time has passed without any such invocation.
+ In the event of a passing test run, this function will be invoked with the
+ string `'Test262:AsyncTestComplete'`. If invoked with any other value, the
+ test must be interpreted as failed. The implementation is free to select an
+ appropriate length of time to wait before considering the test "timed out"
+ and failing.
+
+ *Example*
+
+ ```js
+ /*---
+ flags: [async]
+ ---*/
+ Promise.resolve()
+ .then(function() {
+ print('Test262:AsyncTestComplete');
+ }, function(reason) {
+ print('Error: ' + reason);
+ });
+ ```