Changeset 55243 in webkit

Timestamp:

Feb 25, 2010 10:16:20 AM (14 years ago)

Author:

abarth@webkit.org

Message:

2010-02-25 Adam Barth <​abarth@webkit.org>

Reviewed by David Levin.

EWS leaks memory slowly
​https://bugs.webkit.org/show_bug.cgi?id=35395

The EWS bots leak memory very slowly. If you run them for about a
month, each one will take up around 1 GB of virutal memory. If you run
several of them on one machine, you'll eventually exhaust all available
memory and grind the bots to a halt.

This patch introduces a --exit-after-iteration option to the queues so
that we run them for a finite amount of time. Once they exit and
restart, they'll reclaim the leaked memory. I'm not sure how many
iterations I'll end up running them for. I'll need to sort that out
operationally, but my initial guess is around 1000.

• Scripts/webkitpy/commands/queues.py:
• Scripts/webkitpy/commands/queues_unittest.py:

Location:

trunk/WebKitTools

Files:

• ChangeLog (modified) (1 diff)
• Scripts/webkitpy/commands/queues.py (modified) (2 diffs)
• Scripts/webkitpy/commands/queues_unittest.py (modified) (2 diffs)

trunk/WebKitTools/ChangeLog

@@ +1 @@
+2010-02-25  Adam Barth  <abarth@webkit.org>
+
+        Reviewed by David Levin.
+
+        EWS leaks memory slowly
+        https://bugs.webkit.org/show_bug.cgi?id=35395
+
+        The EWS bots leak memory very slowly.  If you run them for about a
+        month, each one will take up around 1 GB of virutal memory.  If you run
+        several of them on one machine, you'll eventually exhaust all available
+        memory and grind the bots to a halt.
+
+        This patch introduces a --exit-after-iteration option to the queues so
+        that we run them for a finite amount of time.  Once they exit and
+        restart, they'll reclaim the leaked memory.  I'm not sure how many
+        iterations I'll end up running them for.  I'll need to sort that out
+        operationally, but my initial guess is around 1000.
+
+        * Scripts/webkitpy/commands/queues.py:
+        * Scripts/webkitpy/commands/queues_unittest.py:
+
 2010-02-25  Jarkko Sakkinen  <jarkko.sakkinen@tieto.com>
 

trunk/WebKitTools/Scripts/webkitpy/commands/queues.py

@@ -58 +58 @@
         options_list = (options or []) + [
             make_option("--no-confirm", action="store_false", dest="confirm", default=True, help="Do not ask the user for confirmation before running the queue.  Dangerous!"),
+            make_option("--exit-after-iteration", action="store", type="int", dest="iterations", default=None, help="Stop running the queue after iterating this number of times."),
         ]
         Command.__init__(self, "Run the %s" % self.name, options=options_list)
+        self._iteration_count = 0
 
     def _cc_watchers(self, bug_id):
@@ -96 +98 @@
 
     def should_continue_work_queue(self):
-        return True
+        self._iteration_count += 1
+        return not self.options.iterations or self._iteration_count <= self.options.iterations
 
     def next_work_item(self):

trunk/WebKitTools/Scripts/webkitpy/commands/queues_unittest.py

@@ -32 +32 @@
 from webkitpy.commands.queues import *
 from webkitpy.commands.queuestest import QueuesTest
+from webkitpy.mock import Mock
 from webkitpy.mock_bugzillatool import MockBugzillaTool
 from webkitpy.outputcapture import OutputCapture
@@ -65 +66 @@
         self._assert_run_webkit_patch([1])
         self._assert_run_webkit_patch(["one", 2])
+
+    def test_iteration_count(self):
+        queue = TestQueue()
+        queue.options = Mock()
+        queue.options.iterations = 3
+        self.assertTrue(queue.should_continue_work_queue())
+        self.assertTrue(queue.should_continue_work_queue())
+        self.assertTrue(queue.should_continue_work_queue())
+        self.assertFalse(queue.should_continue_work_queue())
+
+    def test_no_iteration_count(self):
+        queue = TestQueue()
+        queue.options = Mock()
+        self.assertTrue(queue.should_continue_work_queue())
+        self.assertTrue(queue.should_continue_work_queue())
+        self.assertTrue(queue.should_continue_work_queue())
+        self.assertTrue(queue.should_continue_work_queue())