Version 3 (modified by 17 years ago) ( diff ) | ,
---|
Planned Design For run-webkit-tests
Directory Structure
- LayoutTests/
- cross-platform tests and results
- platform/
- mac/
- Mac-specific tests, results, and Skipped file
- mac-leopard/
- Leopard-specific tests, results, and Skipped file
- qt/
- Qt-specific tests, results, and Skipped file
- win/
- Windows-specific tests, results, and Skipped file
- mac/
The platform directories are named based on a hyphenated-prefix system. For example, mac-leopard
is a subset of mac
.
Configuration
Each platform configuration should define one variable, $platformTestDirectory
, that specifies the most specific directory under LayoutTests/platform/
that applies to that platform. For instance, on Mac OS X Leopard, mac-leopard
should be specified.
This name is broken into all possible hyphenated prefixes. For example, if a platform specifies a-b-c
as its $platformTestDirectory
, the directories in its platform hierarchy become (in order of decreasing specificity):
a-b-c
a-b
a
Finding Tests
run-webkit-tests
should search recursively through all the subdirectories of LayoutTests/
to find test files, just as it does currently. However, it should not search recursively beneath platform/
. It should then search through each of the directories in the platform hierarchy in order of increasing specificity to find additional platform-specific tests to run.
Finding Expected Results
When searching for the expected results for a specific test, run-webkit-tests
should search through each directory in the platform hierarchy in order of decreasing specificity, then search in LayoutTests/
if no platform-specific result is found. The first expected result found is the one used, regardless of whether it results in a success or failure for the test.
Generating New Results
New platform-specific tests should have their results generated right next to the test. For example, a new test called LayoutTests/platform/mac/leopard/fast/new-leopard-test.html
should have its results placed in LayoutTests/platform/mac/leopard/fast/
.
New cross-platform tests (those not in a subdirectory of platform/
) should have their results generated in one of two places. For text-only tests, the results should be put next to the test itself. For render tree tests, the results should be put in the least-specific directory in the platform hierarchy (i.e., one of the top-level subdirectories of platform/
). So, a new render tree test called LayoutTests/fast/new-test.html
should have its results placed in LayoutTests/platform/<name>/fast/
, where <name>
is the least-specific platform name. (Note that this will mean that new render tree results will only be generated for one platform at a time.) If the user decides that the results are either more or less specific than the default location implies, the user should move the results manually.