Changes between Version 2 and Version 3 of RunWebKitTestsDesign

Timestamp:

Jul 19, 2007 2:36:54 PM (17 years ago)

Author:

Adam Roben

Comment:

--

RunWebKitTestsDesign

@@ -19 +19 @@
 Each platform configuration should define one variable, `$platformTestDirectory`, that specifies the most specific directory under `LayoutTests/platform/` that applies to that platform. For instance, on Mac OS X Leopard, `mac-leopard` should be specified.
 
-This name is broken into all possible hyphenated prefixes. For example, if a platform specifies `a-b-c` as its `$platformTestDirectory`, the directories in its ''platform hierarchy'' become (in order of most- to least-specific):
+This name is broken into all possible hyphenated prefixes. For example, if a platform specifies `a-b-c` as its `$platformTestDirectory`, the directories in its ''platform hierarchy'' become (in order of decreasing specificity):
 
  * `a-b-c`
@@ -26 +26 @@
 
 == Finding Tests ==
-`run-webkit-tests` should search recursively through all the subdirectories of `LayoutTests/` to find test files, just as it does currently. However, it should '''not''' search recursively beneath `platform/`. It should then search through each of the directories in the platform hierarchy in order of least- to most-specific to find additional platform-specific tests to run.
+`run-webkit-tests` should search recursively through all the subdirectories of `LayoutTests/` to find test files, just as it does currently. However, it should '''not''' search recursively beneath `platform/`. It should then search through each of the directories in the platform hierarchy in order of increasing specificity to find additional platform-specific tests to run.
+
+== Finding Expected Results ==
+When searching for the expected results for a specific test, `run-webkit-tests` should search through each directory in the platform hierarchy in order of decreasing specificity, then search in `LayoutTests/` if no platform-specific result is found. The first expected result found is the one used, regardless of whether it results in a success or failure for the test.
+
+== Generating New Results ==
+New platform-specific tests should have their results generated right next to the test. For example, a new test called `LayoutTests/platform/mac/leopard/fast/new-leopard-test.html` should have its results placed in `LayoutTests/platform/mac/leopard/fast/`.
+
+New cross-platform tests (those not in a subdirectory of `platform/`) should have their results generated in one of two places. For text-only tests, the results should be put next to the test itself. For render tree tests, the results should be put in the least-specific directory in the platform hierarchy (i.e., one of the top-level subdirectories of `platform/`). So, a new render tree test called `LayoutTests/fast/new-test.html` should have its results placed in `LayoutTests/platform/<name>/fast/`, where `<name>` is the least-specific platform name. (Note that this will mean that new render tree results will only be generated for one platform at a time.) If the user decides that the results are either more or less specific than the default location implies, the user should move the results manually.