Changeset 150955 in webkit


Ignore:
Timestamp:
May 30, 2013 3:43:06 AM (11 years ago)
Author:
rniwa@webkit.org
Message:

Web Inspector: tests in PerformanceTests/inspector/ are timing out
https://bugs.webkit.org/show_bug.cgi?id=77024

Reviewed by Andreas Kling.

PerformanceTests:

These tests have been disabled for ages and don't conform to the standard format.

  • Skipped:
  • inspector: Removed.
  • inspector/console-300-lines.html: Removed.
  • inspector/first-open-elements.html: Removed.
  • inspector/first-open-resources.html: Removed.
  • inspector/first-open-scripts.html.broken: Removed.
  • inspector/heap-snapshot-advanced.html: Removed.
  • inspector/heap-snapshot-performance-test.js: Removed.
  • inspector/heap-snapshot.html: Removed.
  • inspector/inspector-startup-time.html: Removed.
  • inspector/network-append-30-requests.html.broken: Removed.
  • inspector/performance-test.js: Removed.
  • inspector/show-panel.html.broken: Removed.

Tools:

Remove inspector performance tests since they have been disabled for ages,
and they don't use the standard parser-style performance output.

It's adding a lot of code complexity to our infrastructure.

  • Scripts/webkitpy/performance_tests/perftest.py:

(SingleProcessPerfTest.init):
(PerfTestFactory):

  • Scripts/webkitpy/performance_tests/perftest_unittest.py:

(TestPerfTestFactory.test_regular_test):

  • Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py:

(TestDriver.run_test):
(MainTest.run_test):
(MainTest._tests_for_runner):
(MainTest.test_run_test_set_kills_drt_per_run):
(MainTest._test_run_with_json_output):
(MainTest):
(MainTest.test_run_with_upload_json_should_generate_perf_webkit_json):

Location:
trunk
Files:
11 deleted
6 edited

Legend:

Unmodified
Added
Removed
  • trunk/PerformanceTests/ChangeLog

    r150781 r150955  
     12013-05-30  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        Web Inspector: tests in PerformanceTests/inspector/ are timing out
     4        https://bugs.webkit.org/show_bug.cgi?id=77024
     5
     6        Reviewed by Andreas Kling.
     7
     8        These tests have been disabled for ages and don't conform to the standard format.
     9
     10        * Skipped:
     11        * inspector: Removed.
     12        * inspector/console-300-lines.html: Removed.
     13        * inspector/first-open-elements.html: Removed.
     14        * inspector/first-open-resources.html: Removed.
     15        * inspector/first-open-scripts.html.broken: Removed.
     16        * inspector/heap-snapshot-advanced.html: Removed.
     17        * inspector/heap-snapshot-performance-test.js: Removed.
     18        * inspector/heap-snapshot.html: Removed.
     19        * inspector/inspector-startup-time.html: Removed.
     20        * inspector/network-append-30-requests.html.broken: Removed.
     21        * inspector/performance-test.js: Removed.
     22        * inspector/show-panel.html.broken: Removed.
     23
    1242013-05-27  Benjamin Poulain  <benjamin@webkit.org>
    225
  • trunk/PerformanceTests/Skipped

    r150341 r150955  
    6363Dromaeo/v8-richards.html
    6464
    65 # Bug 77024 - Web Inspector: tests in PerformanceTests/inspector/ are timing out
    66 inspector
    67 
    6865# Bug 100262 - REGRESSION(r131982): this test is crashing
    6966SVG/SvgNestedUse.html
  • trunk/Tools/ChangeLog

    r150951 r150955  
     12013-05-30  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        Web Inspector: tests in PerformanceTests/inspector/ are timing out
     4        https://bugs.webkit.org/show_bug.cgi?id=77024
     5
     6        Reviewed by Andreas Kling.
     7
     8        Remove inspector performance tests since they have been disabled for ages,
     9        and they don't use the standard parser-style performance output.
     10
     11        It's adding a lot of code complexity to our infrastructure.
     12
     13        * Scripts/webkitpy/performance_tests/perftest.py:
     14        (SingleProcessPerfTest.__init__):
     15        (PerfTestFactory):
     16        * Scripts/webkitpy/performance_tests/perftest_unittest.py:
     17        (TestPerfTestFactory.test_regular_test):
     18        * Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py:
     19        (TestDriver.run_test):
     20        (MainTest.run_test):
     21        (MainTest._tests_for_runner):
     22        (MainTest.test_run_test_set_kills_drt_per_run):
     23        (MainTest._test_run_with_json_output):
     24        (MainTest):
     25        (MainTest.test_run_with_upload_json_should_generate_perf_webkit_json):
     26
    1272013-05-30  Ryosuke Niwa  <rniwa@webkit.org>
    228
  • trunk/Tools/Scripts/webkitpy/performance_tests/perftest.py

    r148502 r150955  
    269269
    270270
    271 class ChromiumStylePerfTest(PerfTest):
    272     _chromium_style_result_regex = re.compile(r'^RESULT\s+(?P<name>[^=]+)\s*=\s+(?P<value>\d+(\.\d+)?)\s*(?P<unit>\w+)$')
    273 
    274     def __init__(self, port, test_name, test_path, test_runner_count=DEFAULT_TEST_RUNNER_COUNT):
    275         super(ChromiumStylePerfTest, self).__init__(port, test_name, test_path, test_runner_count)
    276 
    277     def run(self, time_out_ms):
    278         driver = self._create_driver()
    279         try:
    280             output = self.run_single(driver, self.test_path(), time_out_ms)
    281         finally:
    282             driver.stop()
    283 
    284         self._filter_output(output)
    285         if self.run_failed(output):
    286             return None
    287 
    288         return self.parse_and_log_output(output)
    289 
    290     def parse_and_log_output(self, output):
    291         test_failed = False
    292         results = {}
    293         for line in re.split('\n', output.text):
    294             resultLine = ChromiumStylePerfTest._chromium_style_result_regex.match(line)
    295             if resultLine:
    296                 # FIXME: Store the unit
    297                 results[resultLine.group('name').replace(' ', '')] = float(resultLine.group('value'))
    298                 _log.info(line)
    299             elif not len(line) == 0:
    300                 test_failed = True
    301                 _log.error(line)
    302         return results if results and not test_failed else None
    303 
    304 
    305271class ReplayServer(object):
    306272    def __init__(self, archive, record):
     
    457423    _pattern_map = [
    458424        (re.compile(r'^Dromaeo/'), SingleProcessPerfTest),
    459         (re.compile(r'^inspector/'), ChromiumStylePerfTest),
    460425        (re.compile(r'(.+)\.replay$'), ReplayPerfTest),
    461426    ]
  • trunk/Tools/Scripts/webkitpy/performance_tests/perftest_unittest.py

    r148502 r150955  
    3737from webkitpy.port.test import TestDriver
    3838from webkitpy.port.test import TestPort
    39 from webkitpy.performance_tests.perftest import ChromiumStylePerfTest
    4039from webkitpy.performance_tests.perftest import PerfTest
    4140from webkitpy.performance_tests.perftest import PerfTestMetric
     
    472471        test = PerfTestFactory.create_perf_test(MockPort(), 'some-dir/some-test', '/path/some-dir/some-test')
    473472        self.assertEqual(test.__class__, PerfTest)
    474 
    475     def test_inspector_test(self):
    476         test = PerfTestFactory.create_perf_test(MockPort(), 'inspector/some-test', '/path/inspector/some-test')
    477         self.assertEqual(test.__class__, ChromiumStylePerfTest)
  • trunk/Tools/Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py

    r148502 r150955  
    3939from webkitpy.port.driver import DriverOutput
    4040from webkitpy.port.test import TestPort
    41 from webkitpy.performance_tests.perftest import ChromiumStylePerfTest
    4241from webkitpy.performance_tests.perftest import PerfTest
    4342from webkitpy.performance_tests.perftestsrunner import PerfTestsRunner
    44 
    45 
    46 class InspectorPassTestData:
    47     text = 'RESULT group_name: test_name= 42 ms'
    48     output = """Running inspector/pass.html (2 of 2)
    49 RESULT group_name: test_name= 42 ms
    50 Finished: 0.1 s
    51 
    52 """
    5343
    5444
     
    117107"""
    118108
     109    results = {'url': 'http://trac.webkit.org/browser/trunk/PerformanceTests/Parser/some-parser.html',
     110        'metrics': {'Time': {'current': [[1080.0, 1120.0, 1095.0, 1101.0, 1104.0]] * 4}}}
     111
    119112
    120113class MemoryTestData:
     
    169162        crash = False
    170163        if driver_input.test_name.endswith('pass.html'):
    171             text = InspectorPassTestData.text
     164            text = SomeParserTestData.text
    172165        elif driver_input.test_name.endswith('timeout.html'):
    173166            timeout = True
     
    215208    def run_test(self, test_name):
    216209        runner, port = self.create_runner()
    217         tests = [ChromiumStylePerfTest(port, test_name, runner._host.filesystem.join('some-dir', test_name))]
     210        tests = [PerfTest(port, test_name, runner._host.filesystem.join('some-dir', test_name))]
    218211        return runner._run_tests_set(tests) == 0
    219212
     
    242235            path = filesystem.join(runner._base_path, test)
    243236            dirname = filesystem.dirname(path)
    244             if test.startswith('inspector/'):
    245                 tests.append(ChromiumStylePerfTest(runner._port, test, path))
    246             else:
    247                 tests.append(PerfTest(runner._port, test, path))
     237            tests.append(PerfTest(runner._port, test, path))
    248238        return tests
    249 
    250     def test_run_test_set(self):
    251         runner, port = self.create_runner()
    252         tests = self._tests_for_runner(runner, ['inspector/pass.html', 'inspector/silent.html', 'inspector/failed.html',
    253             'inspector/tonguey.html', 'inspector/timeout.html', 'inspector/crash.html'])
    254         output = OutputCapture()
    255         output.capture_output()
    256         try:
    257             unexpected_result_count = runner._run_tests_set(tests)
    258         finally:
    259             stdout, stderr, log = output.restore_output()
    260         self.assertEqual(unexpected_result_count, len(tests) - 1)
    261         self.assertTrue('\nRESULT group_name: test_name= 42 ms\n' in log)
    262239
    263240    def test_run_test_set_kills_drt_per_run(self):
     
    274251        unexpected_result_count = runner._run_tests_set(tests)
    275252
    276         self.assertEqual(TestDriverWithStopCount.stop_count, 6)
     253        self.assertEqual(TestDriverWithStopCount.stop_count, 9)
    277254
    278255    def test_run_test_set_for_parser_tests(self):
     
    307284
    308285    def _test_run_with_json_output(self, runner, filesystem, upload_succeeds=False, results_shown=True, expected_exit_code=0, repeat=1, compare_logs=True):
    309         filesystem.write_text_file(runner._base_path + '/inspector/pass.html', 'some content')
     286        filesystem.write_text_file(runner._base_path + '/Parser/some-parser.html', 'some content')
    310287        filesystem.write_text_file(runner._base_path + '/Bindings/event-target-wrapper.html', 'some content')
    311288
     
    334311            for i in xrange(repeat):
    335312                runs = ' (Run %d of %d)' % (i + 1, repeat) if repeat > 1 else ''
    336                 expected_logs += 'Running 2 tests%s\n' % runs + EventTargetWrapperTestData.output + InspectorPassTestData.output
     313                expected_logs += 'Running 2 tests%s\n' % runs + EventTargetWrapperTestData.output + SomeParserTestData.output
    337314            if results_shown:
    338315                expected_logs += 'MOCK: user.open_url: file://...\n'
     
    346323        "Bindings":
    347324            {"url": "http://trac.webkit.org/browser/trunk/PerformanceTests/Bindings",
    348             "tests": {"event-target-wrapper": EventTargetWrapperTestData.results}}}
     325            "tests": {"event-target-wrapper": EventTargetWrapperTestData.results}},
     326        "Parser":
     327            {"url": "http://trac.webkit.org/browser/trunk/PerformanceTests/Parser",
     328            "tests": {"some-parser": SomeParserTestData.results}}}
    349329
    350330    def test_run_with_json_output(self):
     
    539519        self.assertEqual(output['builderKey'], 'value1')
    540520        self.assertEqual(output['revisions'], {'WebKit': {'revision': '5678', 'timestamp': '2013-02-01 08:48:05 +0000'}})
    541         self.assertEqual(output['tests'].keys(), ['Bindings'])
     521        self.assertEqual(output['tests'].keys(), ['Bindings', 'Parser'])
    542522        self.assertEqual(sorted(output['tests']['Bindings'].keys()), ['tests', 'url'])
    543523        self.assertEqual(output['tests']['Bindings']['url'], 'http://trac.webkit.org/browser/trunk/PerformanceTests/Bindings')
Note: See TracChangeset for help on using the changeset viewer.