14 | | All layout tests have "expected results", which may be one of several forms. The test may produce a text file containing javascript log messages, or a text rendering of the Render Tree. It may also produce a screen capture of the rendered page as PNG files as well (if you are running with {{{--pixel-tests}}} enabled). For WebAudio tests, we can produce WAV files instead of either text or PNG files. For any of these types of tests, there are files checked into the LayoutTests directory named "-expected.{txt,png,wav}". In many (most?) cases, the output is expected to be generic and match on any webkit port. |
| 14 | All layout tests have "expected results", which may be one of several forms. The test may produce a text file containing javascript log messages, or a text rendering of the Render Tree. It may also produce a screen capture of the rendered page as PNG files as well (if you are running with {{{--pixel-tests}}} enabled). For WebAudio tests, we can produce WAV files instead of either text or PNG files. For any of these types of tests, there are files checked into the LayoutTests directory named "-expected.{txt,png,wav}". In many (most?) cases, the output is expected to be generic and match on any webkit port. Lastly, we also support the concept of "reference tests", which check that two pages are rendered identically (pixel-by-pixel). As long as the two tests' output match, the tests pass. For more on reference tests, see [wiki:RefTests]. |
23 | | In the latter case, this is dealt with differently on different ports. |
24 | | |
25 | | In all ports except for the Chromium ones, the convention is to check in the incorrect output as a platform-specific file, and then file a bug to track the incorrectness. For some tests, on some ports, the test is *never* expected to pass, in which case the test is added to the {{{Skipped}} files instead. We will also add tests to the Skipped files if it would affect the rest of the test run or cause NRWT itself to crash. |
26 | | |
27 | | In the Chromium ports, the convention is to add a line of text to the TestExpectations file (see below). |
28 | | |
29 | | Lastly, we also support the concept of "reference tests", which check that two pages are rendered identically (pixel-by-pixel). As long as the two tests' output match, the tests pass. For more on reference tests, see [wiki:RefTests]. |
| 23 | 1. Check in a new baseline as a platform-specific file and file a bug to track the incorrectness. Some types of failures (like crashes and timeouts) can't be handled this way, of course. |
| 24 | 2. Add an entry to the TestExpectations file (see below). |