Changes between Version 2 and Version 3 of RunWebKitTestsDesign


Ignore:
Timestamp:
Jul 19, 2007 2:36:54 PM (14 years ago)
Author:
Adam Roben
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • RunWebKitTestsDesign

    v2 v3  
    1919Each platform configuration should define one variable, `$platformTestDirectory`, that specifies the most specific directory under `LayoutTests/platform/` that applies to that platform. For instance, on Mac OS X Leopard, `mac-leopard` should be specified.
    2020
    21 This name is broken into all possible hyphenated prefixes. For example, if a platform specifies `a-b-c` as its `$platformTestDirectory`, the directories in its ''platform hierarchy'' become (in order of most- to least-specific):
     21This name is broken into all possible hyphenated prefixes. For example, if a platform specifies `a-b-c` as its `$platformTestDirectory`, the directories in its ''platform hierarchy'' become (in order of decreasing specificity):
    2222
    2323 * `a-b-c`
     
    2626
    2727== Finding Tests ==
    28 `run-webkit-tests` should search recursively through all the subdirectories of `LayoutTests/` to find test files, just as it does currently. However, it should '''not''' search recursively beneath `platform/`. It should then search through each of the directories in the platform hierarchy in order of least- to most-specific to find additional platform-specific tests to run.
     28`run-webkit-tests` should search recursively through all the subdirectories of `LayoutTests/` to find test files, just as it does currently. However, it should '''not''' search recursively beneath `platform/`. It should then search through each of the directories in the platform hierarchy in order of increasing specificity to find additional platform-specific tests to run.
     29
     30== Finding Expected Results ==
     31When searching for the expected results for a specific test, `run-webkit-tests` should search through each directory in the platform hierarchy in order of decreasing specificity, then search in `LayoutTests/` if no platform-specific result is found. The first expected result found is the one used, regardless of whether it results in a success or failure for the test.
     32
     33== Generating New Results ==
     34New platform-specific tests should have their results generated right next to the test. For example, a new test called `LayoutTests/platform/mac/leopard/fast/new-leopard-test.html` should have its results placed in `LayoutTests/platform/mac/leopard/fast/`.
     35
     36New cross-platform tests (those not in a subdirectory of `platform/`) should have their results generated in one of two places. For text-only tests, the results should be put next to the test itself. For render tree tests, the results should be put in the least-specific directory in the platform hierarchy (i.e., one of the top-level subdirectories of `platform/`). So, a new render tree test called `LayoutTests/fast/new-test.html` should have its results placed in `LayoutTests/platform/<name>/fast/`, where `<name>` is the least-specific platform name. (Note that this will mean that new render tree results will only be generated for one platform at a time.) If the user decides that the results are either more or less specific than the default location implies, the user should move the results manually.