| 29 | |
| 30 | Screenshot: Todo |
| 31 | |
| 32 | |
| 33 | There is yet another type of reftests, 'a mismatch reftest'. |
| 34 | |
| 35 | Let's see the following sample 'mismatch' reftest. |
| 36 | * foo.html |
| 37 | {{{ |
| 38 | <html><head><title>reftest0001</title> |
| 39 | <body><strong>Hello!</strong></body> |
| 40 | </html> |
| 41 | }}} |
| 42 | |
| 43 | * foo-expected-mismatch.html |
| 44 | {{{ |
| 45 | <html><head><title>reftest0001</title> |
| 46 | <body>Hello!</body> |
| 47 | </html> |
| 48 | }}} |
| 49 | |
| 50 | In this case, the test fails if both produces the same results. The test passes if there is any differences between them. |
| 51 | |
| 52 | |
| 53 | Screenshot: Todo |
| 54 | |
| 55 | |
| 56 | == Naming convention == |
| 57 | |
| 58 | * (normal) reftests: {testname}.html and {testname}-expected.html |
| 59 | * 'mismatch' reftests: {testname}.html and {testname}-expected-mismatch.html |
| 60 | |
| 61 | == FAQ == |
| 62 | |
| 63 | * Q. Can we use a platform-specific expected html file? |
| 64 | * A. Yes. Reftests try to obey the current layout test mechanism as possible as we can. |
| 65 | |
| 66 | * Q. What happens if we 'rebase' for reftests? |
| 67 | * A. A 'Rebasing' doesn't make sense in reftests. A rebase line tool simply skips reftests. |
| 68 | |
| 69 | * Q. If we have hello.html, hello-expected.html and hello-expected-mismatch.html filles, what happens? Does that run two reftests? |
| 70 | * A. No. That should be considerd a wrong configuration. Please don't use one test file for a (normal) reftest and a 'mismatch' reftest. Please choose the different test name for each test. |
| 71 | |
| 72 | * Q. Can we use '-expected.html' and '-expected.{checksum,png}' at the same time for one test file? |
| 73 | * A. No. |
| 74 | |
| 75 | * Q. What kind of failures reftests produce? |
| 76 | * A. If an image comparison failure happens in reftests, that is marked as a 'IMAGE' failure. |
| 77 | If either of two html files crashes, it is considered as 'CRASH'. So does 'TIMEOUT'. |
| 78 | |
| 79 | |
| 80 | |
| 81 | |
| 82 | == References == |
| 83 | |
| 84 | * https://developer.mozilla.org/en/Creating_reftest-based_unit_tests |
| 85 | * http://mxr.mozilla.org/mozilla-central/source/layout/tools/reftest/README.txt |