= Planned Design For run-webkit-tests = == Directory Structure == * LayoutTests/ * ''cross-platform tests and results'' * platform/ * mac/ * ''Mac-specific tests, results, and Skipped file'' * mac-leopard/ * ''Leopard-specific tests, results, and Skipped file'' * qt/ * ''Qt-specific tests, results, and Skipped file'' * win/ * ''Windows-specific tests, results, and Skipped file'' The platform directories are named based on a hyphenated-prefix system. For example, `mac-leopard` is a subset of `mac`. == Configuration == Each platform configuration should define one variable, `$platformTestDirectory`, that specifies the most specific directory under `LayoutTests/platform/` that applies to that platform. For instance, on Mac OS X Leopard, `mac-leopard` should be specified. This name is broken into all possible hyphenated prefixes. For example, if a platform specifies `a-b-c` as its `$platformTestDirectory`, the directories in its ''platform hierarchy'' become (in order of decreasing specificity): * `a-b-c` * `a-b` * `a` == Finding Tests == `run-webkit-tests` should search recursively through all the subdirectories of `LayoutTests/` to find test files, just as it does currently. However, it should '''not''' search recursively beneath `platform/`. It should then search through each of the directories in the platform hierarchy in order of increasing specificity to find additional platform-specific tests to run. == Finding Expected Results == When searching for the expected results for a specific test, `run-webkit-tests` should search through each directory in the platform hierarchy in order of decreasing specificity, then search in `LayoutTests/` if no platform-specific result is found. The first expected result found is the one used, regardless of whether it results in a success or failure for the test. == Generating New Results == New platform-specific tests should have their results generated right next to the test. For example, a new test called `LayoutTests/platform/mac/leopard/fast/new-leopard-test.html` should have its results placed in `LayoutTests/platform/mac/leopard/fast/`. New cross-platform tests (those not in a subdirectory of `platform/`) should have their results generated in one of two places. For text-only tests, the results should be put next to the test itself. For render tree tests, the results should be put in the least-specific directory in the platform hierarchy (i.e., one of the top-level subdirectories of `platform/`). So, a new render tree test called `LayoutTests/fast/new-test.html` should have its results placed in `LayoutTests/platform//fast/`, where `` is the least-specific platform name. (Note that this will mean that new render tree results will only be generated for one platform at a time.) If the user decides that the results are either more or less specific than the default location implies, the user should move the results manually.