= Writing Reftests =
Reftests is not supported yet in WebKit. Please see https://bugs.webkit.org/show_bug.cgi?id=36065 for the current status.
== Reftests in WebKit ==
Reftests use a HTML file ('-expected.html') to represent the expected rendering result instead of pre-generated expected files, such as '-expected.txt', '-expected.checksum' and '-expected.png'.
Let's see the following sample reftest.
* hello.html
{{{
reftest0001
Hello!
}}}
* hello-expected.html
{{{
reftest0001
Hello!
}}}
A new-run-webkit-tests recognizes this is a reftest and run the test as if this is a normal layout test.
You can think that a '-expected.html' file is used to produce '-expected.checksum' and '-expected.png' on the fly.
If there is a difference between the rendering results of two html files, the test fails.
Screenshot: Todo
There is yet another type of reftests, 'a mismatch reftest'.
Let's see the following sample 'mismatch' reftest.
* foo.html
{{{
reftest0001
Hello!
}}}
* foo-expected-mismatch.html
{{{
reftest0001
Hello!
}}}
In this case, the test fails if both produces the same results. The test passes if there is any differences between them.
Screenshot: Todo
== Naming convention ==
* (normal) reftests: {testname}.html and {testname}-expected.html
* 'mismatch' reftests: {testname}.html and {testname}-expected-mismatch.html
== FAQ ==
* Q. Can we use a platform-specific expected html file?
* A. Yes. Reftests try to obey the current layout test mechanism as possible as we can.
* Q. What happens if we 'rebase' for reftests?
* A. A 'Rebasing' doesn't make sense in reftests. A rebase line tool simply skips reftests.
* Q. If we have hello.html, hello-expected.html and hello-expected-mismatch.html filles, what happens? Does that run two reftests?
* A. No. That should be considerd a wrong configuration. Please don't use one test file for a (normal) reftest and a 'mismatch' reftest. Please choose the different test name for each test.
* Q. Can we use '-expected.html' and '-expected.{checksum,png}' at the same time for one test file?
* A. No.
* Q. What kind of failures reftests produce?
* A. If an image comparison failure happens in reftests, that is marked as a 'IMAGE' failure.
If either of two html files crashes, it is considered as 'CRASH'. So does 'TIMEOUT'.
== References ==
* https://developer.mozilla.org/en/Creating_reftest-based_unit_tests
* http://mxr.mozilla.org/mozilla-central/source/layout/tools/reftest/README.txt