Changeset 129555 in webkit


Ignore:
Timestamp:
Sep 25, 2012 2:22:08 PM (12 years ago)
Author:
rniwa@webkit.org
Message:

Perf-o-matic should store "values" and support array'ed input
​https://bugs.webkit.org/show_bug.cgi?id=97601

Reviewed by Dirk Pranke.

Support new JSON format where the outermost structure is an array instead of a dictionary and results may
contain "values". This change will let us remove some code from run-perf-tests.

Old: {"webkit-revision": 123456, "results": {"test": {"avg": 123}}
New: [{"webkit-revision": 123456, "results": {"test": {"avg": 123, values: [122, 123, 124]}}}]

  • Websites/webkit-perf.appspot.com/app.yaml: Incremented the version number.
  • Websites/webkit-perf.appspot.com/models.py:

(TestResult): Added values property.
(TestResult.get_or_insert_from_parsed_json): Pass in "values" to the constructor if the value is present.
(ReportLog.get_value): Use the first item in the array if self._parsed uses the new format.
(ReportLog.results_are_well_formed): Verifies that items in "values" are floats convertible. Also verify that
if the JSON uses new format, there is exactly one set of results. In theory, we could support multiple results
but we don't do that now for its complexity.

  • Websites/webkit-perf.appspot.com/models_unittest.py:

(TestResultTests.test_get_or_insert_stat_value): Make sure values is present and is an empty list.
(TestResultTests.test_get_or_insert_stat_value_with_values): Added.
(ReportLogTests.test_results_are_well_formed):
(ReportLogTests.test_chromium_revision): Renamed from chromium_revision so that it actually runs.
(ReportLogTests.test_results_in_array):

Location:
trunk
Files:
4 edited

Legend:

Unmodified
Added
Removed

  • trunk/ChangeLog

    r129472 r129555  
     12012-09-25  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        Perf-o-matic should store "values" and support array'ed input
     4        https://bugs.webkit.org/show_bug.cgi?id=97601
     5
     6        Reviewed by Dirk Pranke.
     7
     8        Support new JSON format where the outermost structure is an array instead of a dictionary and results may
     9        contain "values". This change will let us remove some code from run-perf-tests.
     10
     11        Old: {"webkit-revision": 123456, "results": {"test": {"avg": 123}}
     12        New: [{"webkit-revision": 123456, "results": {"test": {"avg": 123, values: [122, 123, 124]}}}]
     13
     14        * Websites/webkit-perf.appspot.com/app.yaml: Incremented the version number.
     15        * Websites/webkit-perf.appspot.com/models.py:
     16        (TestResult): Added values property.
     17        (TestResult.get_or_insert_from_parsed_json): Pass in "values" to the constructor if the value is present.
     18        (ReportLog.get_value): Use the first item in the array if self._parsed uses the new format.
     19        (ReportLog.results_are_well_formed): Verifies that items in "values" are floats convertible. Also verify that
     20        if the JSON uses new format, there is exactly one set of results. In theory, we could support multiple results
     21        but we don't do that now for its complexity.
     22        * Websites/webkit-perf.appspot.com/models_unittest.py:
     23        (TestResultTests.test_get_or_insert_stat_value): Make sure values is present and is an empty list.
     24        (TestResultTests.test_get_or_insert_stat_value_with_values): Added.
     25        (ReportLogTests.test_results_are_well_formed):
     26        (ReportLogTests.test_chromium_revision): Renamed from chromium_revision so that it actually runs.
     27        (ReportLogTests.test_results_in_array):
     28
    1292012-09-25  Laszlo Gombos  <l.gombos@samsung.com>
    230

  • trunk/Websites/webkit-perf.appspot.com/app.yaml

    r123938 r129555  
    11application: webkit-perf
    2 version: 19
     2version: 20
    33runtime: python27
    44api_version: 1

  • trunk/Websites/webkit-perf.appspot.com/models.py

    r124184 r129555  
    203203    valueMin = db.FloatProperty()
    204204    valueMax = db.FloatProperty()
     205    values = db.ListProperty(float)
    205206
    206207    @staticmethod
    … …  
    223224        return cls.get_or_insert(key_name, name=test_name, build=build, value=float(result['avg']),
    224225            valueMedian=_float_or_none(result, 'median'), valueStdev=_float_or_none(result, 'stdev'),
    225             valueMin=_float_or_none(result, 'min'), valueMax=_float_or_none(result, 'max'))
     226            valueMin=_float_or_none(result, 'min'), valueMax=_float_or_none(result, 'max'),
     227            values=result.get('values', []))
    226228
    227229    def replace_to_change_test_name(self, new_name):
    … …  
    248250
    249251    def get_value(self, keyName):
    250         if not self._parsed_payload():
    251             return None
    252         return self._parsed.get(keyName)
     252        parsed = self._parsed_payload()
     253        if not parsed:
     254            return None
     255        if isinstance(parsed, list):
     256            parsed = parsed[0]
     257        return parsed.get(keyName)
    253258
    254259    def results(self):
    … …  
    266271                return False
    267272
     273        if isinstance(self._parsed_payload(), list) and len(self._parsed_payload()) != 1:
     274            return False
     275
    268276        if not isinstance(self.results(), dict):
    269277            return False
    … …  
    272280            if isinstance(testResult, dict):
    273281                for key, value in testResult.iteritems():
    274                     if key != "unit" and not _is_float_convertible(value):
     282                    if key == "values":
     283                        if not isinstance(value, list):
     284                            return False
     285                        for item in value:
     286                            if not _is_float_convertible(item):
     287                                return False
     288                    elif key != "unit" and not _is_float_convertible(value):
    275289                        return False
    276290                if 'avg' not in testResult:

  • trunk/Websites/webkit-perf.appspot.com/models_unittest.py

    r123938 r129555  
    351351        self.assertEqual(result.valueMin, 30.5)
    352352        self.assertEqual(result.valueMax, 45)
     353        self.assertEqual(result.values, [])
     354
     355    def test_get_or_insert_stat_value_with_values(self):
     356        branch, platform, builder = _create_some_builder()
     357        build = _create_build(branch, platform, builder)
     358        result = TestResult.get_or_insert_from_parsed_json('some-test', build,
     359            {"avg": 40, "median": "40.1", "stdev": 3.25, "min": 30.5, "max": 45, "values": [1.0, 2.0, 3.0]})
     360        self.assertEqual(result.values, [1.0, 2.0, 3.0])
    353361
    354362    def test_replace_to_change_test_name(self):
    … …  
    452460        assert_results_are_well_formed('{"results": {"test": {"avg": 456, "median": 789}}}', True)
    453461        assert_results_are_well_formed('{"results": {"test": {"avg": 456, "unit": "bytes"}}}', True)
     462        assert_results_are_well_formed('{"results": {"test": {"avg": 456, "unit": "bytes", "values": [1.0, 2.0, 3.0]}}}', True)
     463
     464        assert_results_are_well_formed('[]', False)
     465        assert_results_are_well_formed('[{"results": {"test": 123}}]', True)
     466        assert_results_are_well_formed('[{"results": {"test": 123}}, {"results": {"test": 123}}]', False)
     467        assert_results_are_well_formed('[{"results": {"test": {"avg": 456, "unit": "bytes", "values": [1.0, 2.0, 3.0]}}}]', True)
    454468
    455469    def test_builder(self):
    … …  
    501515        self.assertEqual(log.webkit_revision(), 123)
    502516
    503     def chromium_revision(self):
     517    def test_chromium_revision(self):
    504518        log = self._create_log_with_payload('{"chromium-revision": 123}')
    505         self.assertEqual(log.webkit_revision(), 123)
     519        self.assertEqual(log.chromium_revision(), 123)
    506520
    507521        log = self._create_log_with_payload('{"key": "value"}')
    508         self.assertEqual(log.webkit_revision(), None)
     522        self.assertEqual(log.chromium_revision(), None)
     523
     524    def test_results_in_array(self):
     525        platform = Platform.create_if_possible("some-platform", "Some Platform")
     526        log = self._create_log_with_payload('[{"platform": "some-platform", "build-number": 123, "webkit-revision": 456}]')
     527        self.assertEqual(log.platform().key(), platform.key())
     528        self.assertEqual(log.build_number(), 123)
     529        self.assertEqual(log.webkit_revision(), 456)
    509530
    510531
Note: See TracChangeset for help on using the changeset viewer.