Changeset 130366 in webkit


Ignore:
Timestamp:
Oct 3, 2012 11:29:45 PM (12 years ago)
Author:
pdr@google.com
Message:

Force GC between PageLoad tests.
​https://bugs.webkit.org/show_bug.cgi?id=98203

Reviewed by Ryosuke Niwa.

Previously, our PageLoad PerfTests had multi-modal distributions,
typically with a small cluster at 1-2x the median. This turned out
to be caused by not garbage collecting between tests!

This patch adds a new file, force-gc.html, and loads this file between
PageLoad tests to force a GC. I manually verified that this cleans up
our perf test outliers.

PerformanceTests:

  • resources/force-gc.html: Added.

Tools:

  • Scripts/webkitpy/performance_tests/perftest.py:

(PageLoadingPerfTest.init):
(PageLoadingPerfTest):
(PageLoadingPerfTest.run_single):

This function now loads two pages: one to force a gc and
then the test to run.

  • Scripts/webkitpy/performance_tests/perftest_unittest.py:

Modified several existing tests to show that the force-gc file
is loaded.

(MockPort):
(MockPort.init):
(MockPort.perf_tests_dir):
(TestPageLoadingPerfTest.MockDriver.init):
(TestPageLoadingPerfTest.MockDriver.run_test):
(TestPageLoadingPerfTest.test_run):
(TestPageLoadingPerfTest.test_run_with_bad_output):
(TestReplayPerfTest.ReplayTestPort):
(TestReplayPerfTest.ReplayTestPort.init):
(TestReplayPerfTest.test_run_single.run_test):
(TestReplayPerfTest.test_run_single):
(TestReplayPerfTest.test_run_single_fails_when_output_has_error):
(TestPerfTestFactory.test_regular_test):
(TestPerfTestFactory.test_inspector_test):
(TestPerfTestFactory.test_page_loading_test):

Location:
trunk
Files:
1 added
4 edited

Legend:

Unmodified
Added
Removed

  • trunk/PerformanceTests/ChangeLog

    r130341 r130366  
     12012-10-03  Philip Rogers  <pdr@google.com>
     2
     3        Force GC between PageLoad tests.
     4        https://bugs.webkit.org/show_bug.cgi?id=98203
     5
     6        Reviewed by Ryosuke Niwa.
     7
     8        Previously, our PageLoad PerfTests had multi-modal distributions,
     9        typically with a small cluster at 1-2x the median. This turned out
     10        to be caused by not garbage collecting between tests!
     11
     12        This patch adds a new file, force-gc.html, and loads this file between
     13        PageLoad tests to force a GC. I manually verified that this cleans up
     14        our perf test outliers.
     15
     16        * resources/force-gc.html: Added.
     17
    1182012-10-03  Julien Chaffraix  <jchaffraix@webkit.org>
    219

  • trunk/Tools/ChangeLog

    r130363 r130366  
     12012-10-03  Philip Rogers  <pdr@google.com>
     2
     3        Force GC between PageLoad tests.
     4        https://bugs.webkit.org/show_bug.cgi?id=98203
     5
     6        Reviewed by Ryosuke Niwa.
     7
     8        Previously, our PageLoad PerfTests had multi-modal distributions,
     9        typically with a small cluster at 1-2x the median. This turned out
     10        to be caused by not garbage collecting between tests!
     11
     12        This patch adds a new file, force-gc.html, and loads this file between
     13        PageLoad tests to force a GC. I manually verified that this cleans up
     14        our perf test outliers.
     15
     16        * Scripts/webkitpy/performance_tests/perftest.py:
     17        (PageLoadingPerfTest.__init__):
     18        (PageLoadingPerfTest):
     19        (PageLoadingPerfTest.run_single):
     20
     21            This function now loads two pages: one to force a gc and
     22            then the test to run.
     23
     24        * Scripts/webkitpy/performance_tests/perftest_unittest.py:
     25
     26            Modified several existing tests to show that the force-gc file
     27            is loaded.
     28
     29        (MockPort):
     30        (MockPort.__init__):
     31        (MockPort.perf_tests_dir):
     32        (TestPageLoadingPerfTest.MockDriver.__init__):
     33        (TestPageLoadingPerfTest.MockDriver.run_test):
     34        (TestPageLoadingPerfTest.test_run):
     35        (TestPageLoadingPerfTest.test_run_with_bad_output):
     36        (TestReplayPerfTest.ReplayTestPort):
     37        (TestReplayPerfTest.ReplayTestPort.__init__):
     38        (TestReplayPerfTest.test_run_single.run_test):
     39        (TestReplayPerfTest.test_run_single):
     40        (TestReplayPerfTest.test_run_single_fails_when_output_has_error):
     41        (TestPerfTestFactory.test_regular_test):
     42        (TestPerfTestFactory.test_inspector_test):
     43        (TestPerfTestFactory.test_page_loading_test):
     44
    1452012-10-03  Christophe Dumez  <christophe.dumez@intel.com>
    246

  • trunk/Tools/Scripts/webkitpy/performance_tests/perftest.py

    r130135 r130366  
    203203
    204204class PageLoadingPerfTest(PerfTest):
     205    _FORCE_GC_FILE = 'resources/force-gc.html'
     206
    205207    def __init__(self, port, test_name, path_or_url):
    206208        super(PageLoadingPerfTest, self).__init__(port, test_name, path_or_url)
     209        self.force_gc_test = self._port.host.filesystem.join(self._port.perf_tests_dir(), self._FORCE_GC_FILE)
     210
     211    def run_single(self, driver, path_or_url, time_out_ms, should_run_pixel_test=False):
     212        # Force GC to prevent pageload noise. See https://bugs.webkit.org/show_bug.cgi?id=98203
     213        super(PageLoadingPerfTest, self).run_single(driver, self.force_gc_test, time_out_ms, False)
     214        return super(PageLoadingPerfTest, self).run_single(driver, path_or_url, time_out_ms, should_run_pixel_test)
    207215
    208216    def run(self, driver, time_out_ms):

  • trunk/Tools/Scripts/webkitpy/performance_tests/perftest_unittest.py

    r130135 r130366  
    4444
    4545
     46class MockPort(TestPort):
     47    def __init__(self, custom_run_test=None):
     48        super(MockPort, self).__init__(host=MockHost(), custom_run_test=custom_run_test)
     49
    4650class MainTest(unittest.TestCase):
    4751    def test_parse_output(self):
    … …  
    99103class TestPageLoadingPerfTest(unittest.TestCase):
    100104    class MockDriver(object):
    101         def __init__(self, values):
     105        def __init__(self, values, test):
    102106            self._values = values
    103107            self._index = 0
     108            self._test = test
    104109
    105110        def run_test(self, input, stop_when_done):
     111            if input.test_name == self._test.force_gc_test:
     112                return
    106113            value = self._values[self._index]
    107114            self._index += 1
    … …  
    112119
    113120    def test_run(self):
    114         test = PageLoadingPerfTest(None, 'some-test', '/path/some-dir/some-test')
    115         driver = TestPageLoadingPerfTest.MockDriver(range(1, 21))
     121        port = MockPort()
     122        test = PageLoadingPerfTest(port, 'some-test', '/path/some-dir/some-test')
     123        driver = TestPageLoadingPerfTest.MockDriver(range(1, 21), test)
    116124        output_capture = OutputCapture()
    117125        output_capture.capture_output()
    … …  
    130138        output_capture.capture_output()
    131139        try:
    132             test = PageLoadingPerfTest(None, 'some-test', '/path/some-dir/some-test')
    133             driver = TestPageLoadingPerfTest.MockDriver([1, 2, 3, 4, 5, 6, 7, 'some error', 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20])
     140            port = MockPort()
     141            test = PageLoadingPerfTest(port, 'some-test', '/path/some-dir/some-test')
     142            driver = TestPageLoadingPerfTest.MockDriver([1, 2, 3, 4, 5, 6, 7, 'some error', 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20], test)
    134143            self.assertEqual(test.run(driver, None), None)
    135144        finally:
    … …  
    142151class TestReplayPerfTest(unittest.TestCase):
    143152
    144     class ReplayTestPort(TestPort):
     153    class ReplayTestPort(MockPort):
    145154        def __init__(self, custom_run_test=None):
    146155
    … …  
    150159
    151160            self._custom_driver_class = ReplayTestDriver
    152             super(self.__class__, self).__init__(host=MockHost())
     161            super(self.__class__, self).__init__()
    153162
    154163        def _driver_class(self):
    … …  
    180189
    181190        def run_test(test_input, stop_when_done):
     191            if test_input.test_name == test.force_gc_test:
     192                loaded_pages.append(test_input)
     193                return
    182194            if test_input.test_name != "about:blank":
    183195                self.assertEqual(test_input.test_name, 'http://some-test/')
    … …  
    197209            actual_stdout, actual_stderr, actual_logs = output_capture.restore_output()
    198210
    199         self.assertEqual(len(loaded_pages), 1)
    200         self.assertEqual(loaded_pages[0].test_name, 'http://some-test/')
     211        self.assertEqual(len(loaded_pages), 2)
     212        self.assertEqual(loaded_pages[0].test_name, test.force_gc_test)
     213        self.assertEqual(loaded_pages[1].test_name, 'http://some-test/')
    201214        self.assertEqual(actual_stdout, '')
    202215        self.assertEqual(actual_stderr, '')
    … …  
    263276            actual_stdout, actual_stderr, actual_logs = output_capture.restore_output()
    264277
    265         self.assertEqual(len(loaded_pages), 1)
    266         self.assertEqual(loaded_pages[0].test_name, 'http://some-test/')
     278        self.assertEqual(len(loaded_pages), 2)
     279        self.assertEqual(loaded_pages[0].test_name, test.force_gc_test)
     280        self.assertEqual(loaded_pages[1].test_name, 'http://some-test/')
    267281        self.assertEqual(actual_stdout, '')
    268282        self.assertEqual(actual_stderr, '')
    … …  
    317331class TestPerfTestFactory(unittest.TestCase):
    318332    def test_regular_test(self):
    319         test = PerfTestFactory.create_perf_test(None, 'some-dir/some-test', '/path/some-dir/some-test')
     333        test = PerfTestFactory.create_perf_test(MockPort(), 'some-dir/some-test', '/path/some-dir/some-test')
    320334        self.assertEqual(test.__class__, PerfTest)
    321335
    322336    def test_inspector_test(self):
    323         test = PerfTestFactory.create_perf_test(None, 'inspector/some-test', '/path/inspector/some-test')
     337        test = PerfTestFactory.create_perf_test(MockPort(), 'inspector/some-test', '/path/inspector/some-test')
    324338        self.assertEqual(test.__class__, ChromiumStylePerfTest)
    325339
    326340    def test_page_loading_test(self):
    327         test = PerfTestFactory.create_perf_test(None, 'PageLoad/some-test', '/path/PageLoad/some-test')
     341        test = PerfTestFactory.create_perf_test(MockPort(), 'PageLoad/some-test', '/path/PageLoad/some-test')
    328342        self.assertEqual(test.__class__, PageLoadingPerfTest)
    329343
Note: See TracChangeset for help on using the changeset viewer.