Proposal for TestExpecations Syntax and Semantics

1) TestExpecations files can exist at top level and per platform/ directory.

2) TestExpectations files are chained. The top level and applicable platform/ versions are concatenated in order, and the last expectation given for any particular test wins.

3) TestExpectations files can include expectations for subconfigurations of the relevant platform. Recognized tokens are:

• All (both Debug and Release, and all subplatforms)
• Debug
• Release
• platform names (as found in LayoutTests/platform/*)
• subplatforms of the current platform as a suffix; for example, in the platform/chromium TestExpectations file, "mac" would indicate "chromium-mac". In the platform/mac TestExpectations file, "snow leopard" would indicate "mac-snowleopard".
• platform names OR with each other but AND with Debug or Release or All

4) Scoping expectations to a subconfiguration is done by listing the configuration tokens as a header before some tests, in square brackets

5) An individual expectations line consists of a bug number in angle brackets, followed by tokens describing the expected result, followed by the path to the test.

6) Expectations covered by the same bug as the previous can use "^" as a shorthand to mean "same bug as above".

Worked example, converting the first part of the Chromium expectations file:

[Debug mac]
<crbugs.com/24182> [Slow] fast/css/large-list-of-rules-crash.htm
^                  [Slow] fast/forms/select-set-length-with-mutation-remove.html


[Debug linux mac]
<crbugs.com/24182> [Slow] fast/dom/Window/window-postmessage-clone-really-deep-array.html

[All]
<crbugs.com/24182> [Slow] fast/js/regexp-overflow.html

[Debug]
<crbugs.com/24182> [Slow] fast/js/toString-and-valueOf-override.html
^                  [Slow] html5lib/webkit-resumer.html
^                  [Slow] ietestcenter/Javascript/15.4.4.16-7-c-ii-2.html
^                  [Slow] ietestcenter/Javascript/15.4.4.17-7-c-ii-2.html
^                  [Slow] ietestcenter/Javascript/15.4.4.18-7-c-ii-1.html
^                  [Slow] ietestcenter/Javascript/15.4.4.19-8-c-ii-1.html
^                  [Slow] ietestcenter/Javascript/15.4.4.20-9-c-ii-1.html
^                  [Slow] jquery/attributes.html
^                  [Timeout] jquery/core.html
^                  [Slow] jquery/data.html
^                  [Slow] jquery/dimensions.html
^                  [Timeout] jquery/event.html
^                  [Timeout] jquery/manipulation.html 
^                  [Slow] jquery/offset.html
^                  [Slow] http/tests/websocket/tests/hixie76/frame-lengths.html


[Release win]
<crbugs.com/24182> [Slow] http/tests/loading/onload-vs-immediate-refresh.pl

[Debug win]
<crbugs.com/24182> [Slow] http/tests/loading/redirect-methods.html
^                  [Slow] http/tests/misc/uncacheable-script-repeated.html
^                  [Slow] http/tests/websocket/tests/hixie76/simple-stress.html

[win]
<crbugs.com/24182> [Slow] http/tests/local/file-url-sent-as-referer.html
^                  [Slow] http/tests/xmlhttprequest/simple-cross-origin-progress-events.html
^                  [Slow] http/tests/xmlhttprequest/supported-xml-content-types.html