Changes between Version 24 and Version 25 of Writing Layout Tests for DumpRenderTree


Ignore:
Timestamp:
Oct 27, 2011, 4:21:56 PM (13 years ago)
Author:
jchaffraix@webkit.org
Comment:

Splitting the existing content in a better structure, added some content about writing tests

Legend:

Unmodified
Added
Removed
Modified

  • Writing Layout Tests for DumpRenderTree

    v24 v25  
    22= Writing good test cases =
    33
    4 ''FIXME: Should have some tips here about how to make a test which is easy to know when it succeeds, things like a 100x100 green block, just printing SUCCESS or FAILURE, and using `dumpAsText()`.''
    5 
    6 ''FIXME: Should remind folks to write tests that work both in Safari and in DRT. The trick is to test for the existence of a special object like layoutTestController before using it. An example:''
     4== General tips ==
     5
     6A good test case should have the following properties:
     7
     8 * portable
     9 * fast
     10 * understandable & clear
     11
     12=== Portable ===
     13
     14A test should work in DumpRenderTree (of course!) '''and in a WebKit-base browser'''. This means that anything that is specific to DumpRenderTree should not make the test fail if it is not available.
     15
     16Bad test:
     17
    718{{{
     19<script>
     20layoutTestController.dumpAsText();
     21</script>
     22}}}
     23
     24This will throw an exception in any WebKit-based browser.
     25
     26Good test:
     27
     28{{{
     29<script>
    830if (window.layoutTestController)
    931    layoutTestController.dumpAsText()
     32</script>
    1033}}}
    1134
     35'''Note:''' You get more points if your test works in ANY browsers.
     36
     37=== fast ===
     38
     39WebKit has several thousands layout tests that are run in parallel. If your layout test take a lot time, it will slow down everyone's testing. Also DumpRenderTree has a default timer of several seconds before it considers that a test timed out. '''This is true even if you are using waitUntilDone / notifyDone.'''
     40
     41Here are some advices to avoid having a slow test case:
     42
     43 * Do not make unnecessary use of `window.setTimeout` and similar functions which create a lower bound on the time it takes the test to complete. For separating events in time, you can use `eventSender.leapForward`; for waiting on sub-resources, you can use load events (e.g. iframe.onload = doNextStep;)
     44
     45 * Do not make unnecessary use of external resources: use inline JavaScript and `style` elements instead of `link`s to external stylesheets. You can also use `data:` URLs sometimes for things like frames' `src` attribute. The exceptions are stylesheets and JavaScript libraries that are shared by multiple tests, and cases that test the loading of external resources.  (There are various data: url generators on the net.)
     46
     47=== Understandable & clear ===
     48
     49''FIXME: Should have some tips here about how to make a test which is easy to know when it succeeds, things like a 100x100 green block, just printing SUCCESS or FAILURE, and using `dumpAsText()`.''
     50
     51
     52One last comment (FIXME: move it to a better section):
     53
     54 * Tests should not access the Internet. Avoid `http:` URLs in `src` and `href` attributes, in CSS properties and in XMLHttpRequest. Testing WebKit's network layer should be done using the HTTP test facility, to be described below.
     55
     56
     57== Pixel test tips ==
     58
    1259 * Do not use fonts other than those bundled with Mac OS X (usually there's no need to specify font families in a test). The only exception to this is the ''Ahem'' font.
    1360
    14  * Tests should not access the Internet. Avoid `http:` URLs in `src` and `href` attributes, in CSS properties and in XMLHttpRequest. Testing WebKit's network layer should be done using the HTTP test facility, to be described below.
    15 
    16  * Do not make unnecessary use of `window.setTimeout` and similar functions which create a lower bound on the time it takes the test to complete. For separating events in time, you can use `eventSender.leapForward`; for waiting on sub-resources, you can use load events (e.g. iframe.onload = doNextStep;)
    17 
    18  * Do not make unnecessary use of external resources: use inline JavaScript and `style` elements instead of `link`s to external stylesheets. You can also use `data:` URLs sometimes for things like frames' `src` attribute. The exceptions are stylesheets and JavaScript libraries that are shared by multiple tests, and cases that test the loading of external resources.  (There are various data: url generators on the net.)
     61
    1962
    2063= Writing JavaScript-based DOM-only Test Cases =