| | 1 | = Proposal for TestExpecations Syntax and Semantics = |
| | 2 | |
| | 3 | 1) TestExpecations files can exist at top level and per platform/ directory. |
| | 4 | |
| | 5 | 2) TestExpectations files are chained. The top level and applicable platform/ versions are concatenated in order, and the last expectation given for any particular test wins. |
| | 6 | |
| | 7 | 3) TestExpectations files can include expectations for subconfigurations of the relevant platform. Recognized tokens are: |
| | 8 | * All (both Debug and Release, and all subplatforms) |
| | 9 | * Debug |
| | 10 | * Release |
| | 11 | * platform names (as found in LayoutTests/platform/*) |
| | 12 | * subplatforms of the current platform as a suffix; for example, in the platform/chromium TestExpectations file, "mac" would indicate "chromium-mac". In the platform/mac TestExpectations file, "snow leopard" would indicate "mac-snowleopard". |
| | 13 | * platform names OR with each other but AND with Debug or Release or All |
| | 14 | |
| | 15 | 4) Scoping expectations to a subconfiguration is done by listing the configuration tokens as a header before some tests, in square brackets |
| | 16 | |
| | 17 | 5) An individual expectations line consists of a bug number in angle brackets, followed by tokens describing the expected result, followed by the path to the test. |
| | 18 | |
| | 19 | 6) Expectations covered by the same bug as the previous can use "^" as a shorthand to mean "same bug as above". |
| | 20 | |
| | 21 | Worked example, converting the first part of the Chromium expectations file: |
| | 22 | |
| | 23 | [Debug mac] |
| | 24 | <crbugs.com/24182> [Slow] fast/css/large-list-of-rules-crash.htm |
| | 25 | ^ [Slow] fast/forms/select-set-length-with-mutation-remove.html |
| | 26 | |
| | 27 | |
| | 28 | [Debug linux mac] |
| | 29 | <crbugs.com/24182> [Slow] fast/dom/Window/window-postmessage-clone-really-deep-array.html |
| | 30 | |
| | 31 | [All] |
| | 32 | <crbugs.com/24182> [Slow] fast/js/regexp-overflow.html |
| | 33 | |
| | 34 | [Debug] |
| | 35 | <crbugs.com/24182> [Slow] fast/js/toString-and-valueOf-override.html |
| | 36 | ^ [Slow] html5lib/webkit-resumer.html |
| | 37 | ^ [Slow] ietestcenter/Javascript/15.4.4.16-7-c-ii-2.html |
| | 38 | ^ [Slow] ietestcenter/Javascript/15.4.4.17-7-c-ii-2.html |
| | 39 | ^ [Slow] ietestcenter/Javascript/15.4.4.18-7-c-ii-1.html |
| | 40 | ^ [Slow] ietestcenter/Javascript/15.4.4.19-8-c-ii-1.html |
| | 41 | ^ [Slow] ietestcenter/Javascript/15.4.4.20-9-c-ii-1.html |
| | 42 | ^ [Slow] jquery/attributes.html |
| | 43 | ^ [Timeout] jquery/core.html |
| | 44 | ^ [Slow] jquery/data.html |
| | 45 | ^ [Slow] jquery/dimensions.html |
| | 46 | ^ [Timeout] jquery/event.html |
| | 47 | ^ [Timeout] jquery/manipulation.html |
| | 48 | ^ [Slow] jquery/offset.html |
| | 49 | ^ [Slow] http/tests/websocket/tests/hixie76/frame-lengths.html |
| | 50 | |
| | 51 | |
| | 52 | [Release win] |
| | 53 | <crbugs.com/24182> [Slow] http/tests/loading/onload-vs-immediate-refresh.pl |
| | 54 | |
| | 55 | [Debug win] |
| | 56 | <crbugs.com/24182> [Slow] http/tests/loading/redirect-methods.html |
| | 57 | ^ [Slow] http/tests/misc/uncacheable-script-repeated.html |
| | 58 | ^ [Slow] http/tests/websocket/tests/hixie76/simple-stress.html = PASS |
| | 59 | |
| | 60 | [win] |
| | 61 | <crbugs.com/24182> [Slow] http/tests/local/file-url-sent-as-referer.html = PASS |
| | 62 | ^ [Slow http/tests/xmlhttprequest/simple-cross-origin-progress-events.html = PASS |
| | 63 | ^ [Slow] http/tests/xmlhttprequest/supported-xml-content-types.html = PASS |
