Changeset 115340 in webkit


Ignore:
Timestamp:
Apr 26, 2012 12:21:22 PM (12 years ago)
Author:
robert@webkit.org
Message:

new-webkit-run-tests: handle ref tests from the CSS test suite
​https://bugs.webkit.org/show_bug.cgi?id=83048

Reviewed by Ryosuke Niwa.

Tools:

This allows new-run-webkit-tests to run reference tests created for the CSS test suite. It removes a big swathe
of pixel and text results that are no longer required now that the reference results are used and adds in the support
files from the CSS test suite that allow the ref tests to run.

  • Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:

(SingleTestRunner._run_reftest): Assert that the reference result exists. Previously the test would

run without it and report a pass!

  • Scripts/webkitpy/layout_tests/controllers/test_result_writer.py:

(copy_file): ensure the output directory exists, it may not if the reference test is

contained in a folder that contains nothing but reference tests.

  • Scripts/webkitpy/layout_tests/port/base.py:

(Port.available_reference_files): Add as a helper. Handle embedded relative paths, e.g: some/../path
(Port.reference_files): Only return reference results in the reftest list that actually exist on

disk. Also, don't ignore potential reference results on disk that are not in the reftest list, and
don't ignore them just because we have a reftest list for the test's directory.

(Port._real_tests): ignore anything that looks like a test in a 'support' directory, this semantic

is used by the CSS 2.1 test suite.

  • Scripts/webkitpy/layout_tests/port/base_unittest.py:

(PortTest.test_reference_files_when_reftestlist_and_reference_file_present): Ensure that a test with a reference result

on disk but no reference result listed in the directory's reftest list picks up its reference result.

  • Scripts/webkitpy/layout_tests/port/chromium_unittest.py:

(ChromiumDriverTest.test_strip_uri): Test that some/test.html and some/other/../test.html are treated

as the same file.

LayoutTests:

Add some of the reference infrastructure from the CSS 2.1 test suite.

  • css2.1/20110323/abspos-containing-block-initial-004-ref-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004-ref-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004a-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004a-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004b-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004b-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004c-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004c-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004d-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004d-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004e-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004e-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004f-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-004f-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005-ref-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005-ref-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005a-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005a-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005b-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005b-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005c-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005c-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005d-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-005d-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-007-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-007-ref-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009-ref-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009-ref-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009a-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009a-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009b-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009b-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009e-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009e-expected.txt: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009f-expected.png: Removed.
  • css2.1/20110323/abspos-containing-block-initial-009f-expected.txt: Removed.
  • css2.1/20110323/abspos-non-replaced-width-margin-000-expected.txt: Removed.
  • css2.1/20110323/abspos-replaced-width-margin-000-expected.txt: Removed. These results are superseded by the reference results.
  • css2.1/20110323/background-016.htm: Added to prove the CSS test suite reference results work.
  • css2.1/20110323/block-replaced-width-001.htm: Removed. This test does not pass its reference result. It regressed at some point.
  • css2.1/20110323/empty-inline-001.htm: The test has been updated from the CSS test suite.
  • css2.1/20110323/reftest.list: Added.
  • css2.1/reference/ref-filled-green-100px-square.htm: Added.
  • css2.1/reference/ref-filled-green-100px-square.xht: Added.
  • css2.1/reference/ref-if-there-is-no-red.htm: Added.
  • css2.1/reference/ref-if-there-is-no-red.xht: Added.
  • css2.1/reference/ref-nothing-below.htm: Added.
  • css2.1/reference/ref-nothing-below.xht: Added.
  • css2.1/reference/ref-this-text-should-be-green.htm: Added.
  • css2.1/reference/ref-this-text-should-be-green.xht: Added.

Ref test infrastructure from the CSS test suite.

  • platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/chromium-linux/css2.1/20110323/empty-inline-001-expected.png: Removed.
  • platform/chromium-mac-leopard/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
  • platform/chromium-mac-leopard/css2.1/20110323/empty-inline-001-expected.png: Removed.
  • platform/chromium-mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
  • platform/chromium-mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
  • platform/chromium-mac-snowleopard/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-mac-snowleopard/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-mac-snowleopard/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
  • platform/chromium-mac-snowleopard/css2.1/20110323/empty-inline-001-expected.png: Removed.
  • platform/chromium-mac/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
  • platform/chromium-mac/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
  • platform/chromium-mac/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-mac/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-mac/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
  • platform/chromium-mac/css2.1/20110323/empty-inline-001-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/empty-inline-001-expected.png: Removed.
  • platform/chromium-win/css2.1/20110323/empty-inline-001-expected.txt: Removed.
  • platform/chromium/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.png: Removed.
  • platform/chromium/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
  • platform/efl/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
  • platform/efl/css2.1/20110323/abspos-containing-block-initial-007-expected.txt: Removed.
  • platform/efl/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
  • platform/efl/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.txt: Removed.
  • platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.txt: Removed.
  • platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-expected.txt: Removed.
  • platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/efl/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
  • platform/efl/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
  • platform/gtk/css2.1/20110323/abspos-containing-block-initial-001-expected.txt: Removed.
  • platform/gtk/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.txt: Removed.
  • platform/gtk/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/gtk/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/gtk/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/gtk/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/gtk/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/gtk/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/gtk/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
  • platform/gtk/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
  • platform/gtk/css2.1/20110323/empty-inline-001-expected.txt: Removed.
  • platform/mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
  • platform/mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
  • platform/mac-snowleopard/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/mac/css2.1/20110323/abspos-containing-block-initial-001-expected.txt: Removed.
  • platform/mac/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.txt: Removed.
  • platform/mac/css2.1/20110323/abspos-containing-block-initial-007-expected.txt: Removed.
  • platform/mac/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.txt: Removed.
  • platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.txt: Removed.
  • platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-expected.txt: Removed.
  • platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/mac/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
  • platform/mac/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
  • platform/mac/css2.1/20110323/empty-inline-001-expected.png: Removed.
  • platform/mac/css2.1/20110323/empty-inline-001-expected.txt: Removed.
  • platform/qt/css2.1/20110323/abspos-containing-block-initial-001-expected.txt: Removed.
  • platform/qt/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.txt: Removed.
  • platform/qt/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
  • platform/qt/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/qt/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/qt/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
  • platform/qt/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
  • platform/qt/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
  • platform/qt/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
  • platform/qt/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
  • platform/qt/css2.1/20110323/empty-inline-001-expected.txt: Removed.

These results are superseded by the reference results.

Location:
trunk
Files:
11 added
130 deleted
8 edited

Legend:

Unmodified
Added
Removed

  • trunk/LayoutTests/ChangeLog

    r115335 r115340  
     12012-04-03  Robert Hogan  <robert@webkit.org>
     2
     3        new-webkit-run-tests: handle ref tests from the CSS test suite
     4        https://bugs.webkit.org/show_bug.cgi?id=83048
     5
     6        Reviewed by Ryosuke Niwa.
     7
     8        Add some of the reference infrastructure from the CSS 2.1 test suite.
     9
     10        * css2.1/20110323/abspos-containing-block-initial-004-ref-expected.png: Removed.
     11        * css2.1/20110323/abspos-containing-block-initial-004-ref-expected.txt: Removed.
     12        * css2.1/20110323/abspos-containing-block-initial-004a-expected.png: Removed.
     13        * css2.1/20110323/abspos-containing-block-initial-004a-expected.txt: Removed.
     14        * css2.1/20110323/abspos-containing-block-initial-004b-expected.png: Removed.
     15        * css2.1/20110323/abspos-containing-block-initial-004b-expected.txt: Removed.
     16        * css2.1/20110323/abspos-containing-block-initial-004c-expected.png: Removed.
     17        * css2.1/20110323/abspos-containing-block-initial-004c-expected.txt: Removed.
     18        * css2.1/20110323/abspos-containing-block-initial-004d-expected.png: Removed.
     19        * css2.1/20110323/abspos-containing-block-initial-004d-expected.txt: Removed.
     20        * css2.1/20110323/abspos-containing-block-initial-004e-expected.png: Removed.
     21        * css2.1/20110323/abspos-containing-block-initial-004e-expected.txt: Removed.
     22        * css2.1/20110323/abspos-containing-block-initial-004f-expected.png: Removed.
     23        * css2.1/20110323/abspos-containing-block-initial-004f-expected.txt: Removed.
     24        * css2.1/20110323/abspos-containing-block-initial-005-ref-expected.png: Removed.
     25        * css2.1/20110323/abspos-containing-block-initial-005-ref-expected.txt: Removed.
     26        * css2.1/20110323/abspos-containing-block-initial-005a-expected.png: Removed.
     27        * css2.1/20110323/abspos-containing-block-initial-005a-expected.txt: Removed.
     28        * css2.1/20110323/abspos-containing-block-initial-005b-expected.png: Removed.
     29        * css2.1/20110323/abspos-containing-block-initial-005b-expected.txt: Removed.
     30        * css2.1/20110323/abspos-containing-block-initial-005c-expected.png: Removed.
     31        * css2.1/20110323/abspos-containing-block-initial-005c-expected.txt: Removed.
     32        * css2.1/20110323/abspos-containing-block-initial-005d-expected.png: Removed.
     33        * css2.1/20110323/abspos-containing-block-initial-005d-expected.txt: Removed.
     34        * css2.1/20110323/abspos-containing-block-initial-007-expected.txt: Removed.
     35        * css2.1/20110323/abspos-containing-block-initial-007-ref-expected.txt: Removed.
     36        * css2.1/20110323/abspos-containing-block-initial-009-ref-expected.png: Removed.
     37        * css2.1/20110323/abspos-containing-block-initial-009-ref-expected.txt: Removed.
     38        * css2.1/20110323/abspos-containing-block-initial-009a-expected.png: Removed.
     39        * css2.1/20110323/abspos-containing-block-initial-009a-expected.txt: Removed.
     40        * css2.1/20110323/abspos-containing-block-initial-009b-expected.png: Removed.
     41        * css2.1/20110323/abspos-containing-block-initial-009b-expected.txt: Removed.
     42        * css2.1/20110323/abspos-containing-block-initial-009e-expected.png: Removed.
     43        * css2.1/20110323/abspos-containing-block-initial-009e-expected.txt: Removed.
     44        * css2.1/20110323/abspos-containing-block-initial-009f-expected.png: Removed.
     45        * css2.1/20110323/abspos-containing-block-initial-009f-expected.txt: Removed.
     46        * css2.1/20110323/abspos-non-replaced-width-margin-000-expected.txt: Removed.
     47        * css2.1/20110323/abspos-replaced-width-margin-000-expected.txt: Removed.
     48          These results are superseded by the reference results.
     49
     50        * css2.1/20110323/background-016.htm: Added to prove the CSS test suite reference results work.
     51
     52        * css2.1/20110323/block-replaced-width-001.htm: Removed. This test does not pass its reference result.
     53          It regressed at some point.
     54
     55        * css2.1/20110323/empty-inline-001.htm: The test has been updated from the CSS test suite.
     56
     57        * css2.1/20110323/reftest.list: Added.
     58        * css2.1/reference/ref-filled-green-100px-square.htm: Added.
     59        * css2.1/reference/ref-filled-green-100px-square.xht: Added.
     60        * css2.1/reference/ref-if-there-is-no-red.htm: Added.
     61        * css2.1/reference/ref-if-there-is-no-red.xht: Added.
     62        * css2.1/reference/ref-nothing-below.htm: Added.
     63        * css2.1/reference/ref-nothing-below.xht: Added.
     64        * css2.1/reference/ref-this-text-should-be-green.htm: Added.
     65        * css2.1/reference/ref-this-text-should-be-green.xht: Added.
     66
     67          Ref test infrastructure from the CSS test suite.
     68
     69        * platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
     70        * platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.png: Removed.
     71        * platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
     72        * platform/chromium-linux/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
     73        * platform/chromium-linux/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     74        * platform/chromium-linux/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
     75        * platform/chromium-linux/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     76        * platform/chromium-linux/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
     77        * platform/chromium-linux/css2.1/20110323/empty-inline-001-expected.png: Removed.
     78        * platform/chromium-mac-leopard/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
     79        * platform/chromium-mac-leopard/css2.1/20110323/empty-inline-001-expected.png: Removed.
     80        * platform/chromium-mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
     81        * platform/chromium-mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
     82        * platform/chromium-mac-snowleopard/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     83        * platform/chromium-mac-snowleopard/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     84        * platform/chromium-mac-snowleopard/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
     85        * platform/chromium-mac-snowleopard/css2.1/20110323/empty-inline-001-expected.png: Removed.
     86        * platform/chromium-mac/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
     87        * platform/chromium-mac/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
     88        * platform/chromium-mac/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     89        * platform/chromium-mac/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     90        * platform/chromium-mac/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
     91        * platform/chromium-mac/css2.1/20110323/empty-inline-001-expected.png: Removed.
     92        * platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
     93        * platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.png: Removed.
     94        * platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
     95        * platform/chromium-win/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
     96        * platform/chromium-win/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     97        * platform/chromium-win/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
     98        * platform/chromium-win/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     99        * platform/chromium-win/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
     100        * platform/chromium-win/css2.1/20110323/empty-inline-001-expected.png: Removed.
     101        * platform/chromium-win/css2.1/20110323/empty-inline-001-expected.txt: Removed.
     102        * platform/chromium/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.png: Removed.
     103        * platform/chromium/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
     104        * platform/efl/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
     105        * platform/efl/css2.1/20110323/abspos-containing-block-initial-007-expected.txt: Removed.
     106        * platform/efl/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.png: Removed.
     107        * platform/efl/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.txt: Removed.
     108        * platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     109        * platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.txt: Removed.
     110        * platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
     111        * platform/efl/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
     112        * platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     113        * platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-expected.txt: Removed.
     114        * platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
     115        * platform/efl/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
     116        * platform/efl/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
     117        * platform/efl/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
     118        * platform/gtk/css2.1/20110323/abspos-containing-block-initial-001-expected.txt: Removed.
     119        * platform/gtk/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.txt: Removed.
     120        * platform/gtk/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     121        * platform/gtk/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
     122        * platform/gtk/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
     123        * platform/gtk/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     124        * platform/gtk/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
     125        * platform/gtk/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
     126        * platform/gtk/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
     127        * platform/gtk/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
     128        * platform/gtk/css2.1/20110323/empty-inline-001-expected.txt: Removed.
     129        * platform/mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-001-expected.png: Removed.
     130        * platform/mac-snowleopard/css2.1/20110323/abspos-containing-block-initial-007-expected.png: Removed.
     131        * platform/mac-snowleopard/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     132        * platform/mac/css2.1/20110323/abspos-containing-block-initial-001-expected.txt: Removed.
     133        * platform/mac/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.txt: Removed.
     134        * platform/mac/css2.1/20110323/abspos-containing-block-initial-007-expected.txt: Removed.
     135        * platform/mac/css2.1/20110323/abspos-containing-block-initial-007-ref-expected.txt: Removed.
     136        * platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     137        * platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.txt: Removed.
     138        * platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
     139        * platform/mac/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
     140        * platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     141        * platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-expected.txt: Removed.
     142        * platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
     143        * platform/mac/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
     144        * platform/mac/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
     145        * platform/mac/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
     146        * platform/mac/css2.1/20110323/empty-inline-001-expected.png: Removed.
     147        * platform/mac/css2.1/20110323/empty-inline-001-expected.txt: Removed.
     148        * platform/qt/css2.1/20110323/abspos-containing-block-initial-001-expected.txt: Removed.
     149        * platform/qt/css2.1/20110323/abspos-containing-block-initial-001-ref-expected.txt: Removed.
     150        * platform/qt/css2.1/20110323/abspos-non-replaced-width-margin-000-expected.png: Removed.
     151        * platform/qt/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.png: Removed.
     152        * platform/qt/css2.1/20110323/abspos-non-replaced-width-margin-000-ref-expected.txt: Removed.
     153        * platform/qt/css2.1/20110323/abspos-replaced-width-margin-000-expected.png: Removed.
     154        * platform/qt/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.png: Removed.
     155        * platform/qt/css2.1/20110323/abspos-replaced-width-margin-000-ref-expected.txt: Removed.
     156        * platform/qt/css2.1/20110323/block-replaced-width-001-expected.png: Removed.
     157        * platform/qt/css2.1/20110323/block-replaced-width-001-expected.txt: Removed.
     158        * platform/qt/css2.1/20110323/empty-inline-001-expected.txt: Removed.
     159
     160          These results are superseded by the reference results.
     161
    11622012-04-26  Philip Rogers  <pdr@google.com>
    2163

  • trunk/LayoutTests/css2.1/20110323/empty-inline-001.htm

    r100949 r115340  
    55        <link rel="author" title="Microsoft" href="http://www.microsoft.com/">
    66        <link rel="help" href="http://www.w3.org/TR/CSS21/visuren.html#inline-formatting">
     7        <link rel="match" href="../reference/ref-if-there-is-no-red.htm">
     8
    79        <meta name="flags" content="">
    810        <meta name="assert" content="Empty inline elements create a zero-height line box.">
    … …  
    1921    </head>
    2022    <body>
    21         <p>Test passes if there is no red on this page.</p>
     23        <p>Test passes if there is <strong>no red</strong>.</p>
    2224        <div>
    2325            <span></span>

  • trunk/Tools/ChangeLog

    r115338 r115340  
     12012-04-08  Robert Hogan  <robert@webkit.org>
     2
     3        new-webkit-run-tests: handle ref tests from the CSS test suite
     4        https://bugs.webkit.org/show_bug.cgi?id=83048
     5
     6        Reviewed by Ryosuke Niwa.
     7
     8        This allows new-run-webkit-tests to run reference tests created for the CSS test suite. It removes a big swathe
     9        of pixel and text results that are no longer required now that the reference results are used and adds in the support
     10        files from the CSS test suite that allow the ref tests to run.
     11
     12        * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
     13        (SingleTestRunner._run_reftest): Assert that the reference result exists. Previously the test would
     14         run without it and report a pass!
     15        * Scripts/webkitpy/layout_tests/controllers/test_result_writer.py:
     16        (copy_file): ensure the output directory exists, it may not if the reference test is
     17          contained in a folder that contains nothing but reference tests.
     18        * Scripts/webkitpy/layout_tests/port/base.py:
     19        (Port.available_reference_files): Add as a helper. Handle embedded relative paths, e.g: some/../path
     20        (Port.reference_files): Only return reference results in the reftest list that actually exist on
     21          disk. Also, don't ignore potential reference results on disk that are not in the reftest list, and
     22          don't ignore them just because we have a reftest list for the test's directory.
     23        (Port._real_tests): ignore anything that looks like a test in a 'support' directory, this semantic
     24          is used by the CSS 2.1 test suite.
     25        * Scripts/webkitpy/layout_tests/port/base_unittest.py:
     26        (PortTest.test_reference_files_when_reftestlist_and_reference_file_present): Ensure that a test with a reference result
     27          on disk but no reference result listed in the directory's reftest list picks up its reference result.
     28        * Scripts/webkitpy/layout_tests/port/chromium_unittest.py:
     29        (ChromiumDriverTest.test_strip_uri): Test that some/test.html and some/other/../test.html are treated
     30         as the same file.
     31
    1322012-04-26  Carlos Garcia Campos  <cgarcia@igalia.com>
    233

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py

    r114788 r115340  
    288288        putAllMismatchBeforeMatch = sorted
    289289        for expectation, reference_filename in putAllMismatchBeforeMatch(self._reference_files):
     290            assert(self._filesystem.exists(reference_filename))
    290291            reference_test_name = self._port.relative_test_filename(reference_filename)
    291292            reference_output = self._driver.run_test(DriverInput(reference_test_name, self._timeout, test_output.image_hash, should_run_pixel_test=True))

  • trunk/Tools/Scripts/webkitpy/layout_tests/controllers/test_result_writer.py

    r111736 r115340  
    266266        dst_filepath = fs.join(self._root_output_dir, self._port.relative_test_filename(src_filepath))
    267267        self._make_output_directory()
     268        fs.maybe_make_directory(fs.dirname(dst_filepath))
    268269        fs.copyfile(src_filepath, dst_filepath)

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py

    r114577 r115340  
    472472        return parsed_list
    473473
     474    def available_reference_files(self, reftest_list, test_name):
     475        reference_files = []
     476        for expectation, ref_test in reftest_list.get(self.abspath_for_test(test_name), []):
     477            ref_test = self._filesystem.abspath(ref_test)
     478            if self._filesystem.exists(ref_test):
     479                reference_files.append((expectation, ref_test))
     480        return reference_files
     481
    474482    def reference_files(self, test_name):
    475483        """Return a list of expectation (== or !=) and filename pairs"""
    476484
    477485        reftest_list = self._get_reftest_list(test_name)
    478         if not reftest_list:
    479             reftest_list = []
    480             for expectation, prefix in (('==', ''), ('!=', '-mismatch')):
    481                 for extention in Port._supported_file_extensions:
    482                     path = self.expected_filename(test_name, prefix + extention)
    483                     if self._filesystem.exists(path):
    484                         reftest_list.append((expectation, path))
    485             return reftest_list
    486 
    487         return reftest_list.get(self._filesystem.join(self.layout_tests_dir(), test_name), [])
     486        reference_files = []
     487        if reftest_list:
     488            reference_files = self.available_reference_files(reftest_list, test_name)
     489
     490        for expectation, prefix in (('==', ''), ('!=', '-mismatch')):
     491            for extention in Port._supported_file_extensions:
     492                path = self.expected_filename(test_name, prefix + extention)
     493                if self._filesystem.exists(path):
     494                    reference_files.append((expectation, path))
     495        return reference_files
     496
    488497
    489498    def tests(self, paths):
    … …  
    493502    def _real_tests(self, paths):
    494503        # When collecting test cases, skip these directories
    495         skipped_directories = set(['.svn', '_svn', 'resources', 'script-tests', 'reference', 'reftest'])
     504        skipped_directories = set(['.svn', '_svn', 'resources', 'script-tests', 'reference', 'reftest', 'support'])
    496505        files = find_files.find(self._filesystem, self.layout_tests_dir(), paths, skipped_directories, Port._is_test_file)
    497506        return set([self.relative_test_filename(f) for f in files])

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/base_unittest.py

    r113927 r115340  
    353353        self.assertEqual(port.reference_files('passes/phpreftest.php'), [('!=', port.layout_tests_dir() + '/passes/phpreftest-expected-mismatch.svg')])
    354354
     355    def test_reference_files_when_reftestlist_and_reference_file_present(self):
     356        port = self.make_port(with_tests=True)
     357        port.host.filesystem.write_text_file(port.layout_tests_dir() + '/passes/test-4-expected.html', '')
     358        port.host.filesystem.files['passes/reftest.list'] = "\n".join(["== test.html test-ref.html",
     359        "",
     360        "# some comment",
     361        "!= test-2.html test-notref.html # more comments",
     362        "== test-3.html test-ref.html",
     363        "== test-3.html test-ref2.html",
     364        "!= test-3.html test-notref.html"])
     365        self.assertEqual(port.reference_files('passes/test-4.html'), [('==', port.layout_tests_dir() + '/passes/test-4-expected.html')])
     366
    355367    def test_operating_system(self):
    356368        self.assertEqual('mac', self.make_port().operating_system())

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/chromium_unittest.py

    r114788 r115340  
    9393        self._assert_write_command_and_read_line(expected_crash=True)
    9494
     95    def test_strip_uri(self):
     96        self.driver._proc = Mock()
     97        self.driver._proc.stdout = StringIO.StringIO("#URL:file:///some/test.html")
     98        self.driver.test_to_uri = lambda test: 'file:///some/other/../test.html'
     99        self.driver._port.driver_name = lambda: 'mockdriver'
     100        driver_output = self.driver.run_test(DriverInput(test_name='some/other/test.html', timeout=1, image_hash=None, is_reftest=False))
     101
    95102    def test_crash_log(self):
    96103        self.driver._proc = Mock()
Note: See TracChangeset for help on using the changeset viewer.