Changeset 139786 in webkit


Ignore:
Timestamp:
Jan 15, 2013 2:03:13 PM (11 years ago)
Author:
zandobersek@gmail.com
Message:

Tests with WontFix expectation are (indirectly) skipped
https://bugs.webkit.org/show_bug.cgi?id=105860

Reviewed by Dirk Pranke.

Only skip the WontFix expectation if there are no other layout test
expectations (like Failure, ImageOnlyFailure) listed for the test.
This makes it possible to run an expected failure for which there
is no reason or interest to fix. There are some ports that would like
to exercise this behavior as well as specific test cases that should use
such expectations.

  • Scripts/webkitpy/layout_tests/models/test_expectations.py:

(TestExpectationParser._tokenize_line): Only add the SKIP modifier for a
WONTFIX test if there are no expectations listed. The warning is adjusted
to reflect the new behavior.

  • Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py: Add

and adjust test cases for these changes.
(ExpectationSyntaxTests.test_wontfix):
(SemanticTests.test_skip_and_wontfix):

Location:
trunk/Tools
Files:
3 edited

Legend:

Unmodified
Added
Removed
  • trunk/Tools/ChangeLog

    r139779 r139786  
     12013-01-15  Zan Dobersek  <zdobersek@igalia.com>
     2
     3        Tests with WontFix expectation are (indirectly) skipped
     4        https://bugs.webkit.org/show_bug.cgi?id=105860
     5
     6        Reviewed by Dirk Pranke.
     7
     8        Only skip the WontFix expectation if there are no other layout test
     9        expectations (like Failure, ImageOnlyFailure) listed for the test.
     10        This makes it possible to run an expected failure for which there
     11        is no reason or interest to fix. There are some ports that would like
     12        to exercise this behavior as well as specific test cases that should use
     13        such expectations.
     14
     15        * Scripts/webkitpy/layout_tests/models/test_expectations.py:
     16        (TestExpectationParser._tokenize_line): Only add the SKIP modifier for a
     17        WONTFIX test if there are no expectations listed. The warning is adjusted
     18        to reflect the new behavior.
     19        * Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py: Add
     20        and adjust test cases for these changes.
     21        (ExpectationSyntaxTests.test_wontfix):
     22        (SemanticTests.test_skip_and_wontfix):
     23
    1242013-01-15  Jochen Eisinger  <jochen@chromium.org>
    225
  • trunk/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py

    r137343 r139786  
    347347                warnings.append('Missing a "]"')
    348348
    349         if 'WONTFIX' in modifiers and 'SKIP' not in modifiers:
     349        if 'WONTFIX' in modifiers and 'SKIP' not in modifiers and not expectations:
    350350            modifiers.append('SKIP')
    351351
    352352        if 'SKIP' in modifiers and expectations:
    353353            # FIXME: This is really a semantic warning and shouldn't be here. Remove when we drop the old syntax.
    354             warnings.append('A test marked Skip or WontFix must not have other expectations.')
     354            warnings.append('A test marked Skip must not have other expectations.')
    355355        elif not expectations:
    356356            if 'SKIP' not in modifiers and 'REBASELINE' not in modifiers and 'SLOW' not in modifiers:
  • trunk/Tools/Scripts/webkitpy/layout_tests/models/test_expectations_unittest.py

    r137343 r139786  
    357357    def test_wontfix(self):
    358358        self.assert_tokenize_exp('foo.html [ WontFix ]', modifiers=['WONTFIX', 'SKIP'], expectations=['PASS'])
     359        self.assert_tokenize_exp('foo.html [ WontFix ImageOnlyFailure ]', modifiers=['WONTFIX'], expectations=['IMAGE'])
     360        self.assert_tokenize_exp('foo.html [ WontFix Pass Failure ]', modifiers=['WONTFIX'], expectations=['PASS', 'FAIL'])
    359361
    360362    def test_blank_line(self):
     
    392394
    393395    def test_skip_and_wontfix(self):
    394         # Skip and WontFix are not allowed to have other expectations as well, because those
     396        # Skip is not allowed to have other expectations as well, because those
    395397        # expectations won't be exercised and may become stale .
    396398        self.parse_exp('failures/expected/text.html [ Failure Skip ]')
     
    398400
    399401        self.parse_exp('failures/expected/text.html [ Crash WontFix ]')
    400         self.assertTrue(self._exp.has_warnings())
     402        self.assertFalse(self._exp.has_warnings())
    401403
    402404        self.parse_exp('failures/expected/text.html [ Pass WontFix ]')
    403         self.assertTrue(self._exp.has_warnings())
     405        self.assertFalse(self._exp.has_warnings())
    404406
    405407    def test_slow_and_timeout(self):
Note: See TracChangeset for help on using the changeset viewer.