49 | | ''FIXME: Should have some tips here about how to make a test which is easy to know when it succeeds, things like a 100x100 green block, just printing SUCCESS or FAILURE, and using `dumpAsText()`.'' |
50 | | |
| 49 | A good reading on how to write awesome test cases is the [http://www.w3.org/Style/CSS/Test/guidelines.html CSS2.1 Test Case Authoring Guidelines]. Especially read the ''section: 4. Writing ideal tests'' |
| 50 | |
| 51 | Now in the context of WebKit, test should at least indicate: |
| 52 | * which bug it belongs to. That means the bug number or the bugzilla URL (better as you can paste it in a browser) but also the bug title to see what is tested. |
| 53 | * what is the condition for success or failure. |
| 54 | |
| 55 | Bad test: |
| 56 | |
| 57 | {{{ |
| 58 | <style> |
| 59 | .block { color: red; } |
| 60 | </style> |
| 61 | <p class="block"></p> |
| 62 | }}} |
| 63 | |
| 64 | This is bad because it misses all the WebKit information but also because it uses '''red''' when it means passing. Red should be reserved for failures. |
| 65 | |
| 66 | '''Note:''' This test is badly formatted and if this is not something your test needs, it is better to avoid it. |
| 67 | |
| 68 | Better test: |
| 69 | |
| 70 | {{{ |
| 71 | <!DOCTYPE html> |
| 72 | <html> |
| 73 | <head> |
| 74 | <style> |
| 75 | .block { color: green; } |
| 76 | </style> |
| 77 | </head> |
| 78 | <body> |
| 79 | <p> Bug <a href="http://webkit.org/b/XXXX">XXXX</a>: WebKit does not apply class-selector</p> |
| 80 | <p> For this test to pass, you should see a green rectangle below. </p> |
| 81 | <p class="block"></p> |
| 82 | </body> |
| 83 | </html> |
| 84 | }}} |
| 85 | |
| 86 | This test is a huge improvement but we can actually go one step further! (see the next section about writing pixel tests on that) |
| 93 | |
| 94 | === Do you really need a pixel test? === |
| 95 | |
| 96 | That should be your first question. Pixel tests are a burden on every port as any small chance in the engine could lead to your test needing a new pixel result. That does not mean that pixel tests are bad, they are invaluable for catching visual regressions but there are way to avoid dumping the pixel while checking the behavior you need! |
| 97 | |
| 98 | Let's take the example from the previous section and make it better. |
| 99 | |
| 100 | Even better test: |
| 101 | |
| 102 | {{{ |
| 103 | <!DOCTYPE html> |
| 104 | <html> |
| 105 | <head> |
| 106 | <style> |
| 107 | .block { color: green; } |
| 108 | </style> |
| 109 | <script> |
| 110 | if (window.layoutTestController) |
| 111 | layoutTestController.dumpAsText(); |
| 112 | |
| 113 | function log(message) |
| 114 | { |
| 115 | var console = document.getElementById('console'); |
| 116 | console.appendChild(document.createTextNode(message); |
| 117 | console.appendChild(document.createElement('br')); |
| 118 | } |
| 119 | |
| 120 | function checkBlockColor() |
| 121 | { |
| 122 | var block = document.getElementsByClassName('block')[0]; |
| 123 | var color = document.defaultView.getComputedStyle(block, null).getPropertyValue('color'); |
| 124 | if (color === 'rgb(0, 255, 0)') |
| 125 | log('PASSED'); |
| 126 | else |
| 127 | log('FAILED: color was not green but ' + color); |
| 128 | } |
| 129 | window.addEventListener('load', checkBlockColor, true); |
| 130 | </script> |
| 131 | </head> |
| 132 | <body> |
| 133 | <p> Bug <a href="http://webkit.org/b/XXXX">XXXX</a>: WebKit does not apply class-selector</p> |
| 134 | <p> For this test to pass, you should see a green rectangle below and / or you should see PASS below. </p> |
| 135 | <p id="console"></p> |
| 136 | <p class="block"></p> |
| 137 | </html> |
| 138 | }}} |
| 139 | |
| 140 | As you can see, we did not need the pixels to get the information. The idea is that in this case, we were testing CSS and not the painting part of WebKit so querying the property directly would give us the result we need without having to dump the pixels. |
| 141 | |
| 142 | === How to write portable pixel tests === |