Changeset 65951 in webkit


Ignore:
Timestamp:
Aug 24, 2010 5:02:27 PM (14 years ago)
Author:
dpranke@chromium.org
Message:

2010-08-24 Dirk Pranke <​dpranke@chromium.org>

Reviewed by Eric Seidel.

new-run-webkit-tests: revise unit testing strategy for test failures

This change revamps the way we store test data for new-run-webkit-tests.
Previously we had a few copies of test files from the main test tree.
Now we have a bunch of completely fake test data and use the test
data to generate different kinds of test failures, getting
much better coverage.

​https://bugs.webkit.org/show_bug.cgi?id=44556

  • Scripts/webkitpy/layout_tests/data/failures/expected/crash.html: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.checksum: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.png: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image.html: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.checksum: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.png: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum.html: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/missing_image.html: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/missing_text.html: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/text-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/failures/expected/text.html: Added.
  • Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.checksum: Added.
  • Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.png: Added.
  • Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum.html: Added.
  • Scripts/webkitpy/layout_tests/data/image/canvas-bg.html: Removed.
  • Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.checksum: Removed.
  • Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png: Removed.
  • Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.txt: Removed.
  • Scripts/webkitpy/layout_tests/data/image/canvas-zoom.html: Removed.
  • Scripts/webkitpy/layout_tests/data/misc/crash-expected.txt: Removed.
  • Scripts/webkitpy/layout_tests/data/misc/crash.html: Removed.
  • Scripts/webkitpy/layout_tests/data/misc/missing-expectation.html: Removed.
  • Scripts/webkitpy/layout_tests/data/misc/passing-expected.txt: Removed.
  • Scripts/webkitpy/layout_tests/data/misc/passing.html: Removed.
  • Scripts/webkitpy/layout_tests/data/passes/image-expected.checksum: Added.
  • Scripts/webkitpy/layout_tests/data/passes/image-expected.png: Added.
  • Scripts/webkitpy/layout_tests/data/passes/image-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/passes/image.html: Added.
  • Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.checksum: Added.
  • Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.png: Added.
  • Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/passes/platform_image.html: Added.
  • Scripts/webkitpy/layout_tests/data/passes/text-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/passes/text.html: Added.
  • Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.checksum: Removed.
  • Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png: Removed.
  • Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.txt: Removed.
  • Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.checksum: Added.
  • Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.png: Added.
  • Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.txt: Added.
  • Scripts/webkitpy/layout_tests/data/platform/test/test_expectations.txt:
  • Scripts/webkitpy/layout_tests/data/text/article-element-expected.txt: Removed.
  • Scripts/webkitpy/layout_tests/data/text/article-element.html: Removed.
  • Scripts/webkitpy/layout_tests/layout_package/test_expectations_unittest.py:
  • Scripts/webkitpy/layout_tests/port/test.py:
  • Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py:
  • Scripts/webkitpy/style/checkers/test_expectations_unittest.py:
Location:
trunk/WebKitTools
Files:
35 added
15 deleted
6 edited

Legend:

Unmodified
Added
Removed

  • trunk/WebKitTools/ChangeLog

    r65949 r65951  
     12010-08-24  Dirk Pranke  <dpranke@chromium.org>
     2
     3        Reviewed by Eric Seidel.
     4
     5        new-run-webkit-tests: revise unit testing strategy for test failures
     6   
     7        This change revamps the way we store test data for new-run-webkit-tests.
     8        Previously we had a few copies of test files from the main test tree.
     9        Now we have a bunch of completely fake test data and use the test
     10        data to generate different kinds of test failures, getting
     11        much better coverage.
     12
     13        https://bugs.webkit.org/show_bug.cgi?id=44556
     14
     15        * Scripts/webkitpy/layout_tests/data/failures/expected/crash.html: Added.
     16        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.checksum: Added.
     17        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.png: Added.
     18        * Scripts/webkitpy/layout_tests/data/failures/expected/image-expected.txt: Added.
     19        * Scripts/webkitpy/layout_tests/data/failures/expected/image.html: Added.
     20        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.checksum: Added.
     21        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.png: Added.
     22        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum-expected.txt: Added.
     23        * Scripts/webkitpy/layout_tests/data/failures/expected/image_checksum.html: Added.
     24        * Scripts/webkitpy/layout_tests/data/failures/expected/missing_image.html: Added.
     25        * Scripts/webkitpy/layout_tests/data/failures/expected/missing_text.html: Added.
     26        * Scripts/webkitpy/layout_tests/data/failures/expected/text-expected.txt: Added.
     27        * Scripts/webkitpy/layout_tests/data/failures/expected/text.html: Added.
     28        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.checksum: Added.
     29        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.png: Added.
     30        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum-expected.txt: Added.
     31        * Scripts/webkitpy/layout_tests/data/failures/unexpected/text-image-checksum.html: Added.
     32        * Scripts/webkitpy/layout_tests/data/image/canvas-bg.html: Removed.
     33        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.checksum: Removed.
     34        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.png: Removed.
     35        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom-expected.txt: Removed.
     36        * Scripts/webkitpy/layout_tests/data/image/canvas-zoom.html: Removed.
     37        * Scripts/webkitpy/layout_tests/data/misc/crash-expected.txt: Removed.
     38        * Scripts/webkitpy/layout_tests/data/misc/crash.html: Removed.
     39        * Scripts/webkitpy/layout_tests/data/misc/missing-expectation.html: Removed.
     40        * Scripts/webkitpy/layout_tests/data/misc/passing-expected.txt: Removed.
     41        * Scripts/webkitpy/layout_tests/data/misc/passing.html: Removed.
     42        * Scripts/webkitpy/layout_tests/data/passes/image-expected.checksum: Added.
     43        * Scripts/webkitpy/layout_tests/data/passes/image-expected.png: Added.
     44        * Scripts/webkitpy/layout_tests/data/passes/image-expected.txt: Added.
     45        * Scripts/webkitpy/layout_tests/data/passes/image.html: Added.
     46        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.checksum: Added.
     47        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.png: Added.
     48        * Scripts/webkitpy/layout_tests/data/passes/platform_image-expected.txt: Added.
     49        * Scripts/webkitpy/layout_tests/data/passes/platform_image.html: Added.
     50        * Scripts/webkitpy/layout_tests/data/passes/text-expected.txt: Added.
     51        * Scripts/webkitpy/layout_tests/data/passes/text.html: Added.
     52        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.checksum: Removed.
     53        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.png: Removed.
     54        * Scripts/webkitpy/layout_tests/data/platform/test/image/canvas-bg-expected.txt: Removed.
     55        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.checksum: Added.
     56        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.png: Added.
     57        * Scripts/webkitpy/layout_tests/data/platform/test/passes/platform_image-expected.txt: Added.
     58        * Scripts/webkitpy/layout_tests/data/platform/test/test_expectations.txt:
     59        * Scripts/webkitpy/layout_tests/data/text/article-element-expected.txt: Removed.
     60        * Scripts/webkitpy/layout_tests/data/text/article-element.html: Removed.
     61        * Scripts/webkitpy/layout_tests/layout_package/test_expectations_unittest.py:
     62        * Scripts/webkitpy/layout_tests/port/test.py:
     63        * Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py:
     64        * Scripts/webkitpy/style/checkers/test_expectations_unittest.py:
     65
    1662010-08-24  Dirk Pranke  <dpranke@chromium.org>
    267

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/data/platform/test/test_expectations.txt

    r59725 r65951  
    1 WONTFIX : misc/missing-expectation.html = MISSING PASS
     1WONTFIX : failures/expected/text.html = TEXT
     2WONTFIX SKIP : failures/expected/crash.html = CRASH
     3WONTFIX : failures/expected/image_checksum.html = IMAGE
     4WONTFIX : failures/expected/missing_text.html = MISSING PASS
     5WONTFIX : failures/expected/missing_image.html = MISSING PASS

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/layout_package/test_expectations_unittest.py

    r59725 r65951  
    3434import unittest
    3535
    36 try:
    37    d = os.path.dirname(__file__)
    38 except NameError:
    39    d = os.path.dirname(sys.argv[0])
    40 
    41 sys.path.append(os.path.abspath(os.path.join(d, '..')))
    42 sys.path.append(os.path.abspath(os.path.join(d, '../../thirdparty')))
    43 
    44 import port
    45 from test_expectations import *
     36from webkitpy.layout_tests import port
     37from webkitpy.layout_tests.layout_package.test_expectations import *
    4638
    4739class FunctionsTest(unittest.TestCase):
    … …  
    9991
    10092    def get_basic_tests(self):
    101         return [self.get_test('text/article-element.html'),
    102                 self.get_test('image/canvas-bg.html'),
    103                 self.get_test('image/canvas-zoom.html'),
    104                 self.get_test('misc/crash.html'),
    105                 self.get_test('misc/passing.html')]
     93        return [self.get_test('failures/expected/text.html'),
     94                self.get_test('failures/expected/image_checksum.html'),
     95                self.get_test('failures/expected/crash.html'),
     96                self.get_test('failures/expected/missing_text.html'),
     97                self.get_test('passes/text.html')]
    10698
    10799    def get_basic_expectations(self):
    108100        return """
    109 BUG_TEST : text/article-element.html = TEXT
    110 BUG_TEST SKIP : misc/crash.html = CRASH
    111 BUG_TEST REBASELINE : misc/missing-expectation.html = MISSING
    112 BUG_TEST : image = IMAGE
     101BUG_TEST : failures/expected/text.html = TEXT
     102BUG_TEST SKIP : failures/expected/crash.html = CRASH
     103BUG_TEST REBASELINE : failure/expected/missing_image.html = MISSING
     104BUG_TEST : failures/expected/image_checksum.html = IMAGE
    113105"""
    114106
    … …  
    129121    def test_basic(self):
    130122        self.parse_exp(self.get_basic_expectations())
    131         self.assert_exp('text/article-element.html', TEXT)
    132         self.assert_exp('image/canvas-zoom.html', IMAGE)
    133         self.assert_exp('misc/passing.html', PASS)
     123        self.assert_exp('failures/expected/text.html', TEXT)
     124        self.assert_exp('failures/expected/image_checksum.html', IMAGE)
     125        self.assert_exp('passes/text.html', PASS)
    134126
    135127    def test_duplicates(self):
    136128        self.assertRaises(SyntaxError, self.parse_exp, """
    137 BUG_TEST : text/article-element.html = TEXT
    138 BUG_TEST : text/article-element.html = IMAGE""")
     129BUG_TEST : failures/expected/text.html = TEXT
     130BUG_TEST : failures/expected/text.html = IMAGE""")
    139131        self.assertRaises(SyntaxError, self.parse_exp,
    140132            self.get_basic_expectations(), """
    141 BUG_TEST : text/article-element.html = TEXT
    142 BUG_TEST : text/article-element.html = IMAGE""")
     133BUG_TEST : failures/expected/text.html = TEXT
     134BUG_TEST : failures/expected/text.html = IMAGE""")
    143135
    144136    def test_overrides(self):
    145137        self.parse_exp(self.get_basic_expectations(), """
    146 BUG_OVERRIDE : text/article-element.html = IMAGE""")
    147         self.assert_exp('text/article-element.html', IMAGE)
     138BUG_OVERRIDE : failures/expected/text.html = IMAGE""")
     139        self.assert_exp('failures/expected/text.html', IMAGE)
    148140
    149141    def test_matches_an_expected_result(self):
    … …  
    154146
    155147        self.parse_exp(self.get_basic_expectations())
    156         self.assertTrue(match('text/article-element.html', TEXT, True))
    157         self.assertTrue(match('text/article-element.html', TEXT, False))
    158         self.assertFalse(match('text/article-element.html', CRASH, True))
    159         self.assertFalse(match('text/article-element.html', CRASH, False))
    160 
    161         self.assertTrue(match('image/canvas-bg.html', IMAGE, True))
    162         self.assertTrue(match('image/canvas-bg.html', PASS, False))
    163 
    164         self.assertTrue(match('misc/crash.html', SKIP, False))
    165         self.assertTrue(match('misc/passing.html', PASS, False))
     148        self.assertTrue(match('failures/expected/text.html', TEXT, True))
     149        self.assertTrue(match('failures/expected/text.html', TEXT, False))
     150        self.assertFalse(match('failures/expected/text.html', CRASH, True))
     151        self.assertFalse(match('failures/expected/text.html', CRASH, False))
     152        self.assertTrue(match('failures/expected/image_checksum.html', IMAGE, True))
     153        self.assertTrue(match('failures/expected/image_checksum.html', PASS, False))
     154        self.assertTrue(match('failures/expected/crash.html', SKIP, False))
     155        self.assertTrue(match('passes/text.html', PASS, False))
    166156
    167157if __name__ == '__main__':

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/port/test.py

    r65572 r65951  
    4949    def baseline_path(self):
    5050        return os.path.join(self.layout_tests_dir(), 'platform',
    51                             self.name())
     51                            self.name() + self.version())
    5252
    5353    def baseline_search_path(self):
    … …  
    5757        return True
    5858
    59     def compare_text(self, expected_text, actual_text):
    60         return False
    61 
    6259    def diff_image(self, expected_filename, actual_filename,
    6360                   diff_filename=None, tolerance=0):
    64         return False
    65 
    66     def diff_text(self, expected_text, actual_text,
    67                   expected_filename, actual_filename):
    68         return ''
     61        with codecs.open(actual_filename, "r", "utf-8") as actual_fh:
     62            actual_contents = actual_fh.read()
     63        with codecs.open(expected_filename, "r", "utf-8") as expected_fh:
     64            expected_contents = expected_fh.read()
     65        diffed = actual_contents != expected_contents
     66        if diffed and diff_filename:
     67            with codecs.open(diff_filename, "w", "utf-8") as diff_fh:
     68                diff_fh.write("< %s\n---\n> %s\n" %
     69                              (expected_contents, actual_contents))
     70        return diffed
    6971
    7072    def layout_tests_dir(self):
    … …  
    8284            'webkitpy', 'layout_tests', 'data', 'platform', 'test',
    8385            'test_expectations.txt')
     86
     87    def _path_to_wdiff(self):
     88        return None
    8489
    8590    def results_directory(self):
    … …  
    128133        return test_platform_name
    129134
    130     def version():
    131         return ''
    132 
    133     def wdiff_text(self, expected_filename, actual_filename):
     135    def version(self):
    134136        return ''
    135137
    … …  
    151153
    152154    def run_test(self, uri, timeoutms, image_hash):
    153         if not self._image_written and self._port._options.pixel_tests:
    154             with open(self._image_path, "w") as f:
    155                 f.write("bad png file from TestDriver")
    156                 self._image_written = True
    157 
    158         # We special-case this because we can't fake an image hash for a
    159         # missing expectation.
    160         if uri.find('misc/missing-expectation') != -1:
    161             return (False, False, 'deadbeefdeadbeefdeadbeefdeadbeef', '', None)
    162         return (False, False, image_hash, '', None)
     155        basename = uri[(uri.rfind("/") + 1):uri.rfind(".html")]
     156
     157        error = ''
     158        checksum = None
     159        # There are four currently supported types of tests: text, image,
     160        # image hash (checksum), and stderr output. The fake output
     161        # is the basename of the file + "-" plus the type of test output
     162        # (or a blank string for stderr).
     163        #
     164        # If 'image' or 'check' appears in the basename, we assume this is
     165        # simulating a pixel test.
     166        #
     167        # If 'failures' appears in the URI, then we assume this test should
     168        # fail. Which type of failures are determined by which strings appear
     169        # in the basename of the test. For failures that produce outputs,
     170        # we change the fake output to basename + "_failed-".
     171        #
     172        # The fact that each test produces (more or less) unique output data
     173        # will allow us to see if any results get crossed by the rest of the
     174        # program.
     175        if 'failures' in uri:
     176            crash = 'crash' in basename
     177            timeout = 'timeout' in basename
     178            if 'error' in basename:
     179                error = basename + "_error\n"
     180            if 'text' in basename:
     181                output = basename + '_failed-txt\n'
     182            else:
     183                output = basename + '-txt\n'
     184            if self._port.options().pixel_tests:
     185                if ('image' in basename or 'check' in basename):
     186                    checksum = basename + "-checksum\n"
     187
     188                if 'image' in basename:
     189                    with open(self._image_path, "w") as f:
     190                        f.write(basename + "_failed-png\n")
     191                elif 'check' in basename:
     192                    with open(self._image_path, "w") as f:
     193                        f.write(basename + "-png\n")
     194                if 'checksum' in basename:
     195                    checksum = basename + "_failed-checksum\n"
     196        else:
     197            crash = False
     198            timeout = False
     199            output = basename + '-txt\n'
     200            if self._port.options().pixel_tests and (
     201                'image' in basename or 'check' in basename):
     202                checksum = basename + '-checksum\n'
     203                with open(self._image_path, "w") as f:
     204                    f.write(basename + "-png")
     205
     206        return (crash, timeout, checksum, output, error)
    163207
    164208    def start(self):

  • trunk/WebKitTools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py

    r65949 r65951  
    4949
    5050
    51 def passing_run(args, port_obj=None, record_results=False):
     51def passing_run(args, port_obj=None, record_results=False,
     52                tests_included=False):
    5253    args.extend(['--print', 'nothing'])
     54    if not tests_included:
     55        args.extend(['passes', 'failures/expected'])
    5356    if not record_results:
    5457        args.append('--no-record-results')
    … …  
    7679        self.assertTrue(passing_run(['--platform', 'test', '--run-singly']))
    7780        self.assertTrue(passing_run(['--platform', 'test',
    78                                      'text/article-element.html']))
     81                                     'passes/text.html'], tests_included=True))
    7982
    8083    def test_one_child_process(self):
    … …  
    123126        try:
    124127            # Test that we update expectations in place. If the expectation
    125             # is mssing, update the expected generic location.
     128            # is missing, update the expected generic location.
    126129            file_list = []
    127130            codecs.open = _mocked_open(original_open, file_list)
    128131            passing_run(['--platform', 'test', '--pixel-tests',
    129132                         '--reset-results',
    130                          'image/canvas-bg.html',
    131                          'image/canvas-zoom.html',
    132                          'misc/missing-expectation.html'])
    133             self.assertEqual(len(file_list), 9)
     133                         'passes/image.html',
     134                         'failures/expected/missing_image.html'],
     135                         tests_included=True)
     136            self.assertEqual(len(file_list), 6)
    134137            self.assertBaselines(file_list,
    135                 "data/image/canvas-zoom")
     138                "data/passes/image")
    136139            self.assertBaselines(file_list,
    137                 "data/platform/test/image/canvas-bg")
    138             self.assertBaselines(file_list,
    139                 "data/misc/missing-expectation")
     140                "data/failures/expected/missing_image")
    140141        finally:
    141142            codecs.open = original_open
    … …  
    151152            passing_run(['--platform', 'test', '--pixel-tests',
    152153                         '--new-baseline',
    153                          'image/canvas-zoom.html',
    154                          'image/canvas-bg.html',
    155                          'misc/missing-expectation.html'])
    156             self.assertEqual(len(file_list), 9)
     154                         'passes/image.html',
     155                         'failures/expected/missing_image.html'],
     156                        tests_included=True)
     157            self.assertEqual(len(file_list), 6)
    157158            self.assertBaselines(file_list,
    158                 "data/platform/test/image/canvas-zoom")
     159                "data/platform/test/passes/image")
    159160            self.assertBaselines(file_list,
    160                 "data/platform/test/image/canvas-bg")
    161             self.assertBaselines(file_list,
    162                 "data/platform/test/misc/missing-expectation")
     161                "data/platform/test/failures/expected/missing_image")
    163162        finally:
    164163            codecs.open = original_open

  • trunk/WebKitTools/Scripts/webkitpy/style/checkers/test_expectations_unittest.py

    r65308 r65951  
    6868        self._error_collector = ErrorCollector()
    6969        port_obj = port.get('test')
    70         self._test_file = os.path.join(port_obj.layout_tests_dir(), 'misc/passing.html')
     70        self._test_file = os.path.join(port_obj.layout_tests_dir(), 'passes/text.html')
    7171
    7272    def process_expectations(self, expectations, overrides=None):
    … …  
    8585    def test_valid_expectations(self):
    8686        self.assert_lines_lint(
    87             ["misc/passing.html = PASS"],
     87            ["passes/text.html = PASS"],
    8888            "")
    8989        self.assert_lines_lint(
    90             ["misc/passing.html = FAIL PASS"],
     90            ["passes/text.html = FAIL PASS"],
    9191            "")
    9292        self.assert_lines_lint(
    93             ["misc/passing.html = CRASH TIMEOUT FAIL PASS"],
     93            ["passes/text.html = CRASH TIMEOUT FAIL PASS"],
    9494            "")
    9595        self.assert_lines_lint(
    96             ["BUG1234 TEST : misc/passing.html = PASS FAIL"],
     96            ["BUG1234 TEST : passes/text.html = PASS FAIL"],
    9797            "")
    9898        self.assert_lines_lint(
    99             ["SKIP BUG1234 : misc/passing.html = TIMEOUT PASS"],
     99            ["SKIP BUG1234 : passes/text.html = TIMEOUT PASS"],
    100100            "")
    101101        self.assert_lines_lint(
    102             ["BUG1234 DEBUG : misc/passing.html = TIMEOUT PASS"],
     102            ["BUG1234 DEBUG : passes/text.html = TIMEOUT PASS"],
    103103            "")
    104104        self.assert_lines_lint(
    105             ["BUG1234 DEBUG SKIP : misc/passing.html = TIMEOUT PASS"],
     105            ["BUG1234 DEBUG SKIP : passes/text.html = TIMEOUT PASS"],
    106106            "")
    107107        self.assert_lines_lint(
    108             ["BUG1234 TEST DEBUG SKIP : misc/passing.html = TIMEOUT PASS"],
     108            ["BUG1234 TEST DEBUG SKIP : passes/text.html = TIMEOUT PASS"],
    109109            "")
    110110        self.assert_lines_lint(
    111             ["BUG1234 DEBUG TEST : misc/passing.html = TIMEOUT PASS"],
     111            ["BUG1234 DEBUG TEST : passes/text.html = TIMEOUT PASS"],
    112112            "")
    113113        self.assert_lines_lint(
    114             ["SLOW DEFER BUG1234 : misc/passing.html = PASS"],
     114            ["SLOW DEFER BUG1234 : passes/text.html = PASS"],
    115115            "")
    116116        self.assert_lines_lint(
    117             ["WONTFIX SKIP : misc/passing.html = TIMEOUT"],
     117            ["WONTFIX SKIP : passes/text.html = TIMEOUT"],
    118118            "")
    119119
    120120    def test_valid_modifiers(self):
    121121        self.assert_lines_lint(
    122             ["INVALID-MODIFIER : misc/passing.html = PASS"],
     122            ["INVALID-MODIFIER : passes/text.html = PASS"],
    123123            "Invalid modifier for test: invalid-modifier "
    124             "misc/passing.html  [test/expectations] [5]")
     124            "passes/text.html  [test/expectations] [5]")
    125125        self.assert_lines_lint(
    126             ["SKIP : misc/passing.html = PASS"],
     126            ["SKIP : passes/text.html = PASS"],
    127127            "Test lacks BUG modifier. "
    128             "misc/passing.html  [test/expectations] [2]")
     128            "passes/text.html  [test/expectations] [2]")
    129129        self.assert_lines_lint(
    130             ["WONTFIX DEFER : misc/passing.html = PASS"],
     130            ["WONTFIX DEFER : passes/text.html = PASS"],
    131131            "Test cannot be both DEFER and WONTFIX. "
    132             "misc/passing.html  [test/expectations] [5]")
     132            "passes/text.html  [test/expectations] [5]")
    133133
    134134    def test_expectation_errors(self):
    … …  
    137137            "Missing expectations. ['missing expectations']  [test/expectations] [5]")
    138138        self.assert_lines_lint(
    139             ["SLOW : misc/passing.html = TIMEOUT"],
     139            ["SLOW : passes/text.html = TIMEOUT"],
    140140            "A test can not be both slow and timeout. "
    141141            "If it times out indefinitely, then it should be just timeout. "
    142             "misc/passing.html  [test/expectations] [5]")
     142            "passes/text.html  [test/expectations] [5]")
    143143        self.assert_lines_lint(
    144144            ["does/not/exist.html = FAIL"],
    … …  
    147147    def test_parse_expectations(self):
    148148        self.assert_lines_lint(
    149             ["misc/passing.html = PASS"],
     149            ["passes/text.html = PASS"],
    150150            "")
    151151        self.assert_lines_lint(
    152             ["misc/passing.html = UNSUPPORTED"],
     152            ["passes/text.html = UNSUPPORTED"],
    153153            "Unsupported expectation: unsupported "
    154             "misc/passing.html  [test/expectations] [5]")
     154            "passes/text.html  [test/expectations] [5]")
    155155        self.assert_lines_lint(
    156             ["misc/passing.html = PASS UNSUPPORTED"],
     156            ["passes/text.html = PASS UNSUPPORTED"],
    157157            "Unsupported expectation: unsupported "
    158             "misc/passing.html  [test/expectations] [5]")
     158            "passes/text.html  [test/expectations] [5]")
    159159
    160160    def test_already_seen_test(self):
    161161        self.assert_lines_lint(
    162             ["misc/passing.html = PASS",
    163              "misc/passing.html = TIMEOUT"],
     162            ["passes/text.html = PASS",
     163             "passes/text.html = TIMEOUT"],
    164164            "Duplicate expectation. %s  [test/expectations] [5]" % self._test_file)
    165165
    166166    def test_tab(self):
    167167        self.assert_lines_lint(
    168             ["\tmisc/passing.html = PASS"],
     168            ["\tpasses/text.html = PASS"],
    169169            "Line contains tab character.  [whitespace/tab] [5]")
    170170
Note: See TracChangeset for help on using the changeset viewer.