Changeset 108270 in webkit


Ignore:
Timestamp:
Feb 20, 2012 4:25:47 PM (12 years ago)
Author:
rniwa@webkit.org
Message:

Move more logic from handler classes to model classes and add unit tests
https://bugs.webkit.org/show_bug.cgi?id=78989

Reviewed by Hajime Morita.

Extracted various functions from CreateHandler, ReportHanlder, and RunsHanlder to model classes
in order to unit-test them, added DataStoreTestsBase to reduce the code duplication in tests,
and added a whole bunch of unit tests in models_unittest.py.

  • Websites/webkit-perf.appspot.com/create_handler.py:

(CreateHandler._create_branch):
(CreateHandler._create_platform):

  • Websites/webkit-perf.appspot.com/models.py:

(_create_if_possible):
(_create_if_possible.execute):
(Branch):
(Branch.create_if_possible):
(Platform):
(Platform.create_if_possible):
(Build):
(Build.get_or_insert_from_log):
(Test):
(Test.update_or_insert):
(Test.update_or_insert.execute):
(TestResult):
(TestResult.get_or_insert_from_parsed_json):
(TestResult.get_or_insert_from_parsed_json._float_or_none):
(TestResult.generate_runs):

  • Websites/webkit-perf.appspot.com/models_unittest.py:

(DataStoreTestsBase):
(DataStoreTestsBase.assertThereIsNoInstanceOf):
(DataStoreTestsBase.assertOnlyInstance):
(DataStoreTestsBase.assertEqualUnorderedList):
(HelperTests):
(HelperTests.test_create_in_transaction_with_numeric_id_holder):
(HelperTests.test_failing_in_create_in_transaction_with_numeric_id_holder):
(HelperTests.test_raising_in_create_in_transaction_with_numeric_id_holder):
(HelperTests.test_delete_model_with_numeric_id_holder):
(BranchTests):
(BranchTests.test_create_if_possible):
(PlatformTests):
(PlatformTests.test_create_if_possible):
(BuilderTests):
(_create_some_builder):
(BuildTests):
(BuildTests.test_get_or_insert_from_log):
(TestModelTests):
(TestModelTests.test_update_or_insert):
(TestModelTests.test_update_or_insert_to_update):
(TestResultTests):
(TestResultTests._create_build):
(TestResultTests.test_get_or_insert_value):
(TestResultTests.test_get_or_insert_stat_value):
(TestResultTests._create_results):
(TestResultTests.test_generate_runs):
(ReportLogTests):
(ReportLogTests.test_branch):
(ReportLogTests.test_platform):
(PersistentCacheTests):
(PersistentCacheTests.setUp):
(PersistentCacheTests.test_set):

  • Websites/webkit-perf.appspot.com/report_process_handler.py:

(ReportProcessHandler.post):

  • Websites/webkit-perf.appspot.com/runs_handler.py:

(RunsHandler.get):
(RunsHandler.post):

Location:
trunk
Files:
6 edited

Legend:

Unmodified
Added
Removed
  • trunk/ChangeLog

    r108258 r108270  
     12012-02-19  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        Move more logic from handler classes to model classes and add unit tests
     4        https://bugs.webkit.org/show_bug.cgi?id=78989
     5
     6        Reviewed by Hajime Morita.
     7
     8        Extracted various functions from CreateHandler, ReportHanlder, and RunsHanlder to model classes
     9        in order to unit-test them, added DataStoreTestsBase to reduce the code duplication in tests,
     10        and added a whole bunch of unit tests in models_unittest.py.
     11
     12        * Websites/webkit-perf.appspot.com/create_handler.py:
     13        (CreateHandler._create_branch):
     14        (CreateHandler._create_platform):
     15        * Websites/webkit-perf.appspot.com/models.py:
     16        (_create_if_possible):
     17        (_create_if_possible.execute):
     18        (Branch):
     19        (Branch.create_if_possible):
     20        (Platform):
     21        (Platform.create_if_possible):
     22        (Build):
     23        (Build.get_or_insert_from_log):
     24        (Test):
     25        (Test.update_or_insert):
     26        (Test.update_or_insert.execute):
     27        (TestResult):
     28        (TestResult.get_or_insert_from_parsed_json):
     29        (TestResult.get_or_insert_from_parsed_json._float_or_none):
     30        (TestResult.generate_runs):
     31        * Websites/webkit-perf.appspot.com/models_unittest.py:
     32        (DataStoreTestsBase):
     33        (DataStoreTestsBase.assertThereIsNoInstanceOf):
     34        (DataStoreTestsBase.assertOnlyInstance):
     35        (DataStoreTestsBase.assertEqualUnorderedList):
     36        (HelperTests):
     37        (HelperTests.test_create_in_transaction_with_numeric_id_holder):
     38        (HelperTests.test_failing_in_create_in_transaction_with_numeric_id_holder):
     39        (HelperTests.test_raising_in_create_in_transaction_with_numeric_id_holder):
     40        (HelperTests.test_delete_model_with_numeric_id_holder):
     41        (BranchTests):
     42        (BranchTests.test_create_if_possible):
     43        (PlatformTests):
     44        (PlatformTests.test_create_if_possible):
     45        (BuilderTests):
     46        (_create_some_builder):
     47        (BuildTests):
     48        (BuildTests.test_get_or_insert_from_log):
     49        (TestModelTests):
     50        (TestModelTests.test_update_or_insert):
     51        (TestModelTests.test_update_or_insert_to_update):
     52        (TestResultTests):
     53        (TestResultTests._create_build):
     54        (TestResultTests.test_get_or_insert_value):
     55        (TestResultTests.test_get_or_insert_stat_value):
     56        (TestResultTests._create_results):
     57        (TestResultTests.test_generate_runs):
     58        (ReportLogTests):
     59        (ReportLogTests.test_branch):
     60        (ReportLogTests.test_platform):
     61        (PersistentCacheTests):
     62        (PersistentCacheTests.setUp):
     63        (PersistentCacheTests.test_set):
     64        * Websites/webkit-perf.appspot.com/report_process_handler.py:
     65        (ReportProcessHandler.post):
     66        * Websites/webkit-perf.appspot.com/runs_handler.py:
     67        (RunsHandler.get):
     68        (RunsHandler.post):
     69
    1702012-02-20  Patrick Gansterer  <paroga@webkit.org>
    271
  • trunk/Websites/webkit-perf.appspot.com/create_handler.py

    r108094 r108270  
    8686        if not key or not name:
    8787            return 'Invalid key or name'
    88 
    89         error = [None]
    90 
    91         def execute(id):
    92             if Branch.get_by_key_name(key):
    93                 error[0] = 'Branch "%s" already exists' % key
    94                 return
    95             branch = Branch(id=id, name=name, key_name=key)
    96             branch.put()
    97             return branch
    98 
    99         create_in_transaction_with_numeric_id_holder(execute)
    100         return error[0]
     88        return None if Branch.create_if_possible(key, name) else 'Branch "%s" already exists' % key
    10189
    10290    def _create_platform(self, key, name):
    10391        if not key or not name:
    10492            return 'Invalid key name'
    105 
    106         error = [None]
    107 
    108         def execute(id):
    109             if Platform.get_by_key_name(key):
    110                 error[0] = 'Platform "%s" already exists' % key
    111                 return
    112             platform = Platform(id=id, name=name, key_name=key)
    113             platform.put()
    114             return platform
    115 
    116         create_in_transaction_with_numeric_id_holder(execute)
    117         return error[0]
     93        return None if Platform.create_if_possible(key, name) else 'Platform "%s" already exists' % key
  • trunk/Websites/webkit-perf.appspot.com/models.py

    r108094 r108270  
    3535from google.appengine.ext import db
    3636from google.appengine.api import memcache
     37from time import mktime
    3738
    3839
     
    6970
    7071
     72def _create_if_possible(model, key, name):
     73
     74    def execute(id):
     75        if model.get_by_key_name(key):
     76            return None
     77        branch = model(id=id, name=name, key_name=key)
     78        branch.put()
     79        return branch
     80
     81    return create_in_transaction_with_numeric_id_holder(execute)
     82
     83
    7184class Branch(db.Model):
    7285    id = db.IntegerProperty(required=True)
    7386    name = db.StringProperty(required=True)
    7487
     88    @staticmethod
     89    def create_if_possible(key, name):
     90        return _create_if_possible(Branch, key, name)
     91
    7592
    7693class Platform(db.Model):
    7794    id = db.IntegerProperty(required=True)
    7895    name = db.StringProperty(required=True)
     96
     97    @staticmethod
     98    def create_if_possible(key, name):
     99        return _create_if_possible(Platform, key, name)
    79100
    80101
     
    108129    timestamp = db.DateTimeProperty(required=True)
    109130
     131    @staticmethod
     132    def get_or_insert_from_log(log):
     133        builder = log.builder()
     134        key_name = builder.name + ':' + str(int(mktime(log.timestamp().timetuple())))
     135
     136        return Build.get_or_insert(key_name, branch=log.branch(), platform=log.platform(), builder=builder,
     137            buildNumber=log.build_number(), timestamp=log.timestamp(),
     138            revision=log.webkit_revision(), chromiumRevision=log.chromium_revision())
     139
    110140
    111141# Used to generate TestMap in the manifest efficiently
     
    119149    def cache_key(test_id, branch_id, platform_id):
    120150        return 'runs:%d,%d,%d' % (test_id, branch_id, platform_id)
     151
     152    @staticmethod
     153    def update_or_insert(test_name, branch, platform):
     154        existing_test = [None]
     155
     156        def execute(id):
     157            test = Test.get_by_key_name(test_name)
     158            if test:
     159                if branch.key() not in test.branches:
     160                    test.branches.append(branch.key())
     161                if platform.key() not in test.platforms:
     162                    test.platforms.append(platform.key())
     163                existing_test[0] = test
     164                return None
     165
     166            test = Test(id=id, name=test_name, key_name=test_name, branches=[branch.key()], platforms=[platform.key()])
     167            test.put()
     168            return test
     169
     170        return create_in_transaction_with_numeric_id_holder(execute) or existing_test[0]
    121171
    122172
     
    133183    def key_name(build, test_name):
    134184        return build.key().name() + ':' + test_name
     185
     186    @classmethod
     187    def get_or_insert_from_parsed_json(cls, test_name, build, result):
     188        key_name = cls.key_name(build, test_name)
     189
     190        def _float_or_none(dictionary, key):
     191            value = dictionary.get(key)
     192            if value:
     193                return float(value)
     194            return None
     195
     196        if not isinstance(result, dict):
     197            return cls.get_or_insert(key_name, name=test_name, build=build, value=float(result))
     198
     199        return cls.get_or_insert(key_name, name=test_name, build=build, value=float(result['avg']),
     200            valueMedian=_float_or_none(result, 'median'), valueStdev=_float_or_none(result, 'stdev'),
     201            valueMin=_float_or_none(result, 'min'), valueMax=_float_or_none(result, 'max'))
     202
     203    @staticmethod
     204    def generate_runs(branch, platform, test_name):
     205        builds = Build.all()
     206        builds.filter('branch =', branch)
     207        builds.filter('platform =', platform)
     208
     209        for build in builds:
     210            results = TestResult.all()
     211            results.filter('name =', test_name)
     212            results.filter('build =', build)
     213            for result in results:
     214                yield build, result
     215        raise StopIteration
    135216
    136217
  • trunk/Websites/webkit-perf.appspot.com/models_unittest.py

    r108094 r108270  
    3434from google.appengine.api import memcache
    3535from google.appengine.ext import testbed
    36 
    37 
    38 class HelperTests(unittest.TestCase):
     36from time import mktime
     37
     38
     39class DataStoreTestsBase(unittest.TestCase):
    3940    def setUp(self):
    4041        self.testbed = testbed.Testbed()
     
    4546        self.testbed.deactivate()
    4647
     48    def assertThereIsNoInstanceOf(self, model):
     49        self.assertEqual(len(model.all().fetch(5)), 0)
     50
     51    def assertOnlyInstance(self, only_instasnce):
     52        self.assertEqual(len(only_instasnce.__class__.all().fetch(5)), 1)
     53        self.assertTrue(only_instasnce.__class__.get(only_instasnce.key()))
     54
     55    def assertEqualUnorderedList(self, list1, list2):
     56        self.assertEqual(set(list1), set(list2))
     57
     58
     59class HelperTests(DataStoreTestsBase):
    4760    def _assert_there_is_exactly_one_id_holder_and_matches(self, id):
    4861        id_holders = models.NumericIdHolder.all().fetch(5)
     
    5669            return models.Branch(id=id, name='some branch', key_name='some-branch').put()
    5770
    58         self.assertEqual(len(models.Branch.all().fetch(5)), 0)
    59         self.assertEqual(len(models.NumericIdHolder.all().fetch(5)), 0)
     71        self.assertThereIsNoInstanceOf(models.Branch)
     72        self.assertThereIsNoInstanceOf(models.NumericIdHolder)
    6073
    6174        self.assertTrue(models.create_in_transaction_with_numeric_id_holder(execute))
     
    7386            return None
    7487
    75         self.assertEqual(len(models.Branch.all().fetch(5)), 0)
    76         self.assertEqual(len(models.NumericIdHolder.all().fetch(5)), 0)
     88        self.assertThereIsNoInstanceOf(models.Branch)
     89        self.assertThereIsNoInstanceOf(models.NumericIdHolder)
    7790
    7891        self.assertFalse(models.create_in_transaction_with_numeric_id_holder(execute))
    7992
    80         self.assertEqual(len(models.Branch.all().fetch(5)), 0)
    81         self.assertEqual(len(models.NumericIdHolder.all().fetch(5)), 0)
     93        self.assertThereIsNoInstanceOf(models.Branch)
     94        self.assertThereIsNoInstanceOf(models.NumericIdHolder)
    8295
    8396    def test_raising_in_create_in_transaction_with_numeric_id_holder(self):
     
    87100            return None
    88101
    89         self.assertEqual(len(models.Branch.all().fetch(5)), 0)
    90         self.assertEqual(len(models.NumericIdHolder.all().fetch(5)), 0)
     102        self.assertThereIsNoInstanceOf(models.Branch)
     103        self.assertThereIsNoInstanceOf(models.NumericIdHolder)
    91104
    92105        self.assertRaises(TypeError, models.create_in_transaction_with_numeric_id_holder, (execute))
    93106
    94         self.assertEqual(len(models.Branch.all().fetch(5)), 0)
    95         self.assertEqual(len(models.NumericIdHolder.all().fetch(5)), 0)
     107        self.assertThereIsNoInstanceOf(models.Branch)
     108        self.assertThereIsNoInstanceOf(models.NumericIdHolder)
    96109
    97110    def test_delete_model_with_numeric_id_holder(self):
     
    101114
    102115        branch = models.Branch.get(models.create_in_transaction_with_numeric_id_holder(execute))
    103         self.assertEqual(len(models.NumericIdHolder.all().fetch(5)), 1)
     116        self.assertOnlyInstance(branch)
    104117
    105118        models.delete_model_with_numeric_id_holder(branch)
    106119
    107         self.assertEqual(len(models.Branch.all().fetch(5)), 0)
    108         self.assertEqual(len(models.NumericIdHolder.all().fetch(5)), 0)
     120        self.assertThereIsNoInstanceOf(models.Branch)
     121        self.assertThereIsNoInstanceOf(models.NumericIdHolder)
    109122
    110123    def test_model_from_numeric_id(self):
     
    121134
    122135
    123 class BuilderTests(unittest.TestCase):
    124     def setUp(self):
    125         self.testbed = testbed.Testbed()
    126         self.testbed.activate()
    127         self.testbed.init_datastore_v3_stub()
    128 
    129     def tearDown(self):
    130         self.testbed.deactivate()
    131 
     136class BranchTests(DataStoreTestsBase):
     137    def test_create_if_possible(self):
     138        self.assertThereIsNoInstanceOf(models.Branch)
     139
     140        branch = models.Branch.create_if_possible('some-branch', 'some branch')
     141        self.assertTrue(branch)
     142        self.assertTrue(branch.key().name(), 'some-branch')
     143        self.assertTrue(branch.name, 'some branch')
     144        self.assertOnlyInstance(branch)
     145
     146        self.assertFalse(models.Branch.create_if_possible('some-branch', 'some other branch'))
     147        self.assertTrue(branch.name, 'some branch')
     148        self.assertOnlyInstance(branch)
     149
     150
     151class PlatformTests(DataStoreTestsBase):
     152    def test_create_if_possible(self):
     153        self.assertThereIsNoInstanceOf(models.Platform)
     154
     155        platform = models.Platform.create_if_possible('some-platform', 'some platform')
     156        self.assertTrue(platform)
     157        self.assertTrue(platform.key().name(), 'some-platform')
     158        self.assertTrue(platform.name, 'some platform')
     159        self.assertOnlyInstance(platform)
     160
     161        self.assertFalse(models.Platform.create_if_possible('some-platform', 'some other platform'))
     162        self.assertTrue(platform.name, 'some platform')
     163        self.assertOnlyInstance(platform)
     164
     165
     166class BuilderTests(DataStoreTestsBase):
    132167    def test_create(self):
    133168        builder_key = models.Builder.create('some builder', 'some password')
     
    159194
    160195
    161 class ReportLog(unittest.TestCase):
    162     def setUp(self):
    163         self.testbed = testbed.Testbed()
    164         self.testbed.activate()
    165         self.testbed.init_datastore_v3_stub()
    166 
    167     def tearDown(self):
    168         self.testbed.deactivate()
    169 
     196def _create_some_builder():
     197    branch = models.Branch.create_if_possible('some-branch', 'Some Branch')
     198    platform = models.Platform.create_if_possible('some-platform', 'Some Platform')
     199    builder_key = models.Builder.create('some-builder', 'Some Builder')
     200    return branch, platform, models.Builder.get(builder_key)
     201
     202
     203class BuildTests(DataStoreTestsBase):
     204    def test_get_or_insert_from_log(self):
     205        branch, platform, builder = _create_some_builder()
     206
     207        timestamp = datetime.now().replace(microsecond=0)
     208        log = models.ReportLog(timestamp=timestamp, headers='some headers',
     209            payload='{"branch": "some-branch", "platform": "some-platform", "builder-name": "some-builder",' +
     210                '"build-number": 123, "webkit-revision": 456, "timestamp": %d}' % int(mktime(timestamp.timetuple())))
     211
     212        self.assertThereIsNoInstanceOf(models.Build)
     213
     214        build = models.Build.get_or_insert_from_log(log)
     215        self.assertTrue(build)
     216        self.assertEqual(build.branch.key(), branch.key())
     217        self.assertEqual(build.platform.key(), platform.key())
     218        self.assertEqual(build.builder.key(), builder.key())
     219        self.assertEqual(build.buildNumber, 123)
     220        self.assertEqual(build.revision, 456)
     221        self.assertEqual(build.chromiumRevision, None)
     222        self.assertEqual(build.timestamp, timestamp)
     223
     224        self.assertOnlyInstance(build)
     225
     226
     227class TestModelTests(DataStoreTestsBase):
     228    def test_update_or_insert(self):
     229        branch = models.Branch.create_if_possible('some-branch', 'Some Branch')
     230        platform = models.Platform.create_if_possible('some-platform', 'Some Platform')
     231
     232        self.assertThereIsNoInstanceOf(models.Test)
     233
     234        test = models.Test.update_or_insert('some-test', branch, platform)
     235        self.assertTrue(test)
     236        self.assertEqual(test.branches, [branch.key()])
     237        self.assertEqual(test.platforms, [platform.key()])
     238        self.assertOnlyInstance(test)
     239
     240    def test_update_or_insert_to_update(self):
     241        branch = models.Branch.create_if_possible('some-branch', 'Some Branch')
     242        platform = models.Platform.create_if_possible('some-platform', 'Some Platform')
     243        test = models.Test.update_or_insert('some-test', branch, platform)
     244        self.assertOnlyInstance(test)
     245
     246        other_branch = models.Branch.create_if_possible('other-branch', 'Other Branch')
     247        other_platform = models.Platform.create_if_possible('other-platform', 'Other Platform')
     248        test = models.Test.update_or_insert('some-test', other_branch, other_platform)
     249        self.assertOnlyInstance(test)
     250        self.assertEqualUnorderedList(test.branches, [branch.key(), other_branch.key()])
     251        self.assertEqualUnorderedList(test.platforms, [platform.key(), other_platform.key()])
     252
     253
     254class TestResultTests(DataStoreTestsBase):
     255    def _create_build(self):
     256        branch, platform, builder = _create_some_builder()
     257        build_key = models.Build(key_name='some-build', branch=branch, platform=platform, builder=builder,
     258            buildNumber=1, revision=100, timestamp=datetime.now()).put()
     259        return models.Build.get(build_key)
     260
     261    def test_get_or_insert_value(self):
     262        build = self._create_build()
     263        self.assertThereIsNoInstanceOf(models.TestResult)
     264        result = models.TestResult.get_or_insert_from_parsed_json('some-test', build, 50)
     265        self.assertOnlyInstance(result)
     266        self.assertEqual(result.name, 'some-test')
     267        self.assertEqual(result.build.key(), build.key())
     268        self.assertEqual(result.value, 50.0)
     269        self.assertEqual(result.valueMedian, None)
     270        self.assertEqual(result.valueStdev, None)
     271        self.assertEqual(result.valueMin, None)
     272        self.assertEqual(result.valueMax, None)
     273
     274    def test_get_or_insert_stat_value(self):
     275        build = self._create_build()
     276        self.assertThereIsNoInstanceOf(models.TestResult)
     277        result = models.TestResult.get_or_insert_from_parsed_json('some-test', build,
     278            {"avg": 40, "median": "40.1", "stdev": 3.25, "min": 30.5, "max": 45})
     279        self.assertOnlyInstance(result)
     280        self.assertEqual(result.name, 'some-test')
     281        self.assertEqual(result.build.key(), build.key())
     282        self.assertEqual(result.value, 40.0)
     283        self.assertEqual(result.valueMedian, 40.1)
     284        self.assertEqual(result.valueStdev, 3.25)
     285        self.assertEqual(result.valueMin, 30.5)
     286        self.assertEqual(result.valueMax, 45)
     287
     288    def _create_results(self, test_name, values):
     289        branch, platform, builder = _create_some_builder()
     290        results = []
     291        for i, value in enumerate(values):
     292            build = models.Build(branch=branch, platform=platform, builder=builder,
     293                buildNumber=i, revision=100 + i, timestamp=datetime.now())
     294            build.put()
     295            result = models.TestResult(name=test_name, build=build, value=value)
     296            result.put()
     297            results.append(result)
     298        return branch, platform, results
     299
     300    def test_generate_runs(self):
     301        branch, platform, results = self._create_results('some-test', [50.0, 51.0, 52.0, 49.0, 48.0])
     302        last_i = 0
     303        for i, (build, result) in enumerate(models.TestResult.generate_runs(branch, platform, "some-test")):
     304            self.assertEqual(build.buildNumber, i)
     305            self.assertEqual(build.revision, 100 + i)
     306            self.assertEqual(result.name, 'some-test')
     307            self.assertEqual(result.value, results[i].value)
     308            last_i = i
     309        self.assertTrue(last_i + 1, len(results))
     310
     311
     312class ReportLogTests(DataStoreTestsBase):
    170313    def _create_log_with_payload(self, payload):
    171314        return models.ReportLog(timestamp=datetime.now(), headers='some headers', payload=payload)
     
    207350        self.assertEqual(log.builder().key(), builder_key)
    208351
    209     # FIXME test_branch and test_platform
     352    def test_branch(self):
     353        log = self._create_log_with_payload('{"key": "value"}')
     354        self.assertEqual(log.branch(), None)
     355
     356        log = self._create_log_with_payload('{"branch": "some-branch"}')
     357        self.assertEqual(log.branch(), None)
     358
     359        branch = models.Branch.create_if_possible("some-branch", "Some Branch")
     360        log = self._create_log_with_payload('{"branch": "some-branch"}')
     361        self.assertEqual(log.branch().key(), branch.key())
     362
     363    def test_platform(self):
     364        log = self._create_log_with_payload('{"key": "value"}')
     365        self.assertEqual(log.platform(), None)
     366
     367        log = self._create_log_with_payload('{"platform": "some-platform"}')
     368        self.assertEqual(log.platform(), None)
     369
     370        platform = models.Platform.create_if_possible("some-platform", "Some Platform")
     371        log = self._create_log_with_payload('{"platform": "some-platform"}')
     372        self.assertEqual(log.platform().key(), platform.key())
    210373
    211374    def test_build_number(self):
     
    231394
    232395
    233 class PersistentCacheTests(unittest.TestCase):
     396class PersistentCacheTests(DataStoreTestsBase):
    234397    def setUp(self):
    235398        self.testbed = testbed.Testbed()
     
    238401        self.testbed.init_memcache_stub()
    239402
    240     def tearDown(self):
    241         self.testbed.deactivate()
    242 
    243403    def _assert_persistent_cache(self, name, value):
    244404        self.assertEqual(models.PersistentCache.get_by_key_name(name).value, value)
     
    246406
    247407    def test_set(self):
    248         self.assertEqual(len(models.PersistentCache.all().fetch(5)), 0)
     408        self.assertThereIsNoInstanceOf(models.PersistentCache)
    249409
    250410        models.PersistentCache.set_cache('some-cache', 'some data')
  • trunk/Websites/webkit-perf.appspot.com/report_process_handler.py

    r107535 r108270  
    4040from models import Test
    4141from models import TestResult
    42 from models import create_in_transaction_with_numeric_id_holder
    4342
    4443
     
    5655        branch = log.branch()
    5756        platform = log.platform()
    58         build = self._create_build_if_possible(log, branch, platform)
     57        build = Build.get_or_insert_from_log(log)
    5958
    6059        for test_name, result in log.results().iteritems():
    61             test = self._add_test_if_needed(test_name, branch, platform)
    62             self._add_test_result_if_needed(test_name, build, result)
     60            test = Test.update_or_insert(test_name, branch, platform)
     61            TestResult.get_or_insert_from_parsed_json(test_name, build, result)
    6362            schedule_runs_update(test.id, branch.id, platform.id)
    6463
     
    7170
    7271        self.response.out.write('OK')
    73 
    74     def _create_build_if_possible(self, log, branch, platform):
    75         builder = log.builder()
    76         key_name = builder.name + ':' + str(int(time.mktime(log.timestamp().timetuple())))
    77 
    78         return Build.get_or_insert(key_name, branch=branch, platform=platform, builder=builder, buildNumber=log.build_number(),
    79             timestamp=log.timestamp(), revision=log.webkit_revision(), chromiumRevision=log.chromium_revision())
    80 
    81     def _add_test_if_needed(self, test_name, branch, platform):
    82 
    83         def execute(id):
    84             test = Test.get_by_key_name(test_name)
    85             returnValue = None
    86             if not test:
    87                 test = Test(id=id, name=test_name, key_name=test_name)
    88                 returnValue = test
    89             if branch.key() not in test.branches:
    90                 test.branches.append(branch.key())
    91             if platform.key() not in test.platforms:
    92                 test.platforms.append(platform.key())
    93             test.put()
    94             return returnValue
    95         return create_in_transaction_with_numeric_id_holder(execute) or Test.get_by_key_name(test_name)
    96 
    97     def _add_test_result_if_needed(self, test_name, build, result):
    98         key_name = TestResult.key_name(build, test_name)
    99 
    100         def _float_or_none(dictionary, key):
    101             value = dictionary.get(key)
    102             if value:
    103                 return float(value)
    104             return None
    105 
    106         if not isinstance(result, dict):
    107             return TestResult.get_or_insert(key_name, name=test_name, build=build, value=float(result))
    108 
    109         return TestResult.get_or_insert(key_name, name=test_name, build=build, value=float(result['avg']),
    110             valueMedian=_float_or_none(result, 'median'), valueStdev=_float_or_none(result, 'stdev'),
    111             valueMin=_float_or_none(result, 'min'), valueMax=_float_or_none(result, 'max'))
  • trunk/Websites/webkit-perf.appspot.com/runs_handler.py

    r108189 r108270  
    6262        # days = self.request.get('days', 365)
    6363
    64         builds = Build.all()
    65         builds.filter('branch =', model_from_numeric_id(branch_id, Branch))
    66         builds.filter('platform =', model_from_numeric_id(platform_id, Platform))
     64        branch = model_from_numeric_id(branch_id, Branch)
     65        platform = model_from_numeric_id(platform_id, Platform)
     66        test = model_from_numeric_id(test_id, Test)
     67        assert branch
     68        assert platform
     69        assert test
    6770
    68         test = model_from_numeric_id(test_id, Test)
    69         test_name = test.name if test else None
    7071        test_runs = []
    7172        averages = {}
     
    7374        timestamps = []
    7475
    75         for build in builds:
    76             results = TestResult.all()
    77             results.filter('name =', test_name)
    78             results.filter('build =', build)
    79             for result in results:
    80                 builderId = build.builder.key().id()
    81                 posixTimestamp = mktime(build.timestamp.timetuple())
    82                 statistics = None
    83                 supplementary_revisions = None
    84                 if result.valueStdev != None and result.valueMin != None and result.valueMax != None:
    85                     statistics = {'stdev': result.valueStdev, 'min': result.valueMin, 'max': result.valueMax}
    86                 if build.chromiumRevision != None:
    87                     supplementary_revisions = {'Chromium': build.chromiumRevision}
     76        for build, result in TestResult.generate_runs(branch, platform, test.name):
     77            builderId = build.builder.key().id()
     78            posixTimestamp = mktime(build.timestamp.timetuple())
     79            statistics = None
     80            supplementary_revisions = None
     81            if result.valueStdev != None and result.valueMin != None and result.valueMax != None:
     82                statistics = {'stdev': result.valueStdev, 'min': result.valueMin, 'max': result.valueMax}
     83            if build.chromiumRevision != None:
     84                supplementary_revisions = {'Chromium': build.chromiumRevision}
    8885
    89                 test_runs.append([result.key().id(),
    90                     [build.key().id(), build.buildNumber, build.revision, supplementary_revisions],
    91                     posixTimestamp, result.value, 0,  # runNumber
    92                     [],  # annotations
    93                     builderId, statistics])
     86            test_runs.append([result.key().id(),
     87                [build.key().id(), build.buildNumber, build.revision, supplementary_revisions],
     88                posixTimestamp, result.value, 0,  # runNumber
     89                [],  # annotations
     90                builderId, statistics])
    9491
    95                 # FIXME: Calculate the average; in practice, we wouldn't have more than one value for a given revision
    96                 averages[build.revision] = result.value
    97                 values.append(result.value)
    98                 timestamps.append(posixTimestamp)
     92            # FIXME: Calculate the average; in practice, we wouldn't have more than one value for a given revision
     93            averages[build.revision] = result.value
     94            values.append(result.value)
     95            timestamps.append(posixTimestamp)
    9996
    10097        result = json.dumps({
Note: See TracChangeset for help on using the changeset viewer.