Changeset 107386 in webkit


Ignore:
Timestamp:
Feb 10, 2012 1:56:08 AM (12 years ago)
Author:
rniwa@webkit.org
Message:

Perf-o-matic should process reports in background
​https://bugs.webkit.org/show_bug.cgi?id=78309

Reviewed by Hajime Morita.

Split the logic to create Build, Test, and TestResult objects from ReportHandler into ReportProcessHandler.
ReportHandler now merely creates ReportLog and schedules a task to process it.

Also added ReportLogHandler to manage stale ReportLogs.

  • Websites/webkit-perf.appspot.com/app.yaml:
  • Websites/webkit-perf.appspot.com/controller.py:

(schedule_manifest_update):
(schedule_dashboard_update):
(schedule_runs_update):
(CachedRunsHandler.get):
(schedule_report_process):

  • Websites/webkit-perf.appspot.com/main.py:
  • Websites/webkit-perf.appspot.com/merge_tests.html: Renamed from Websites/webkit-perf.appspot.com/merge_tests.yaml.
  • Websites/webkit-perf.appspot.com/models.py:

(ReportLog):
(ReportLog._parsed_payload):
(ReportLog.get_value):
(ReportLog.results):
(ReportLog.builder):
(ReportLog.branch):
(ReportLog.platform):
(ReportLog.build_number):
(ReportLog.webkit_revision):
(ReportLog.chromium_revision):
(ReportLog._model_by_key_name_in_payload):
(ReportLog._integer_in_payload):
(ReportLog.timestamp):

  • Websites/webkit-perf.appspot.com/report_handler.py:

(ReportHandler.post):
(ReportHandler._output):
(ReportHandler._results_are_valid):
(ReportHandler._results_are_valid._is_float_convertible):
(ReportHandler):

  • Websites/webkit-perf.appspot.com/report_logs.html: Added.
  • Websites/webkit-perf.appspot.com/report_logs_handler.py: Added.

(ReportLogsHandler):
(ReportLogsHandler.get):
(ReportLogsHandler.post):
(ReportLogsHandler._error):

  • Websites/webkit-perf.appspot.com/report_process_handler.py: Copied from Websites/webkit-perf.appspot.com/report_handler.py.

(ReportProcessHandler):
(ReportProcessHandler.post):
(ReportProcessHandler._create_build_if_possible):
(ReportProcessHandler._create_build_if_possible.execute):
(ReportProcessHandler._add_test_if_needed):

Location:
trunk
Files:
2 added
6 edited
1 copied
1 moved

Legend:

Unmodified
Added
Removed

  • trunk/ChangeLog

    r107337 r107386  
     12012-02-10  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        Perf-o-matic should process reports in background
     4        https://bugs.webkit.org/show_bug.cgi?id=78309
     5
     6        Reviewed by Hajime Morita.
     7
     8        Split the logic to create Build, Test, and TestResult objects from ReportHandler into ReportProcessHandler.
     9        ReportHandler now merely creates ReportLog and schedules a task to process it.
     10
     11        Also added ReportLogHandler to manage stale ReportLogs.
     12
     13        * Websites/webkit-perf.appspot.com/app.yaml:
     14        * Websites/webkit-perf.appspot.com/controller.py:
     15        (schedule_manifest_update):
     16        (schedule_dashboard_update):
     17        (schedule_runs_update):
     18        (CachedRunsHandler.get):
     19        (schedule_report_process):
     20        * Websites/webkit-perf.appspot.com/main.py:
     21        * Websites/webkit-perf.appspot.com/merge_tests.html: Renamed from Websites/webkit-perf.appspot.com/merge_tests.yaml.
     22        * Websites/webkit-perf.appspot.com/models.py:
     23        (ReportLog):
     24        (ReportLog._parsed_payload):
     25        (ReportLog.get_value):
     26        (ReportLog.results):
     27        (ReportLog.builder):
     28        (ReportLog.branch):
     29        (ReportLog.platform):
     30        (ReportLog.build_number):
     31        (ReportLog.webkit_revision):
     32        (ReportLog.chromium_revision):
     33        (ReportLog._model_by_key_name_in_payload):
     34        (ReportLog._integer_in_payload):
     35        (ReportLog.timestamp):
     36        * Websites/webkit-perf.appspot.com/report_handler.py:
     37        (ReportHandler.post):
     38        (ReportHandler._output):
     39        (ReportHandler._results_are_valid):
     40        (ReportHandler._results_are_valid._is_float_convertible):
     41        (ReportHandler):
     42        * Websites/webkit-perf.appspot.com/report_logs.html: Added.
     43        * Websites/webkit-perf.appspot.com/report_logs_handler.py: Added.
     44        (ReportLogsHandler):
     45        (ReportLogsHandler.get):
     46        (ReportLogsHandler.post):
     47        (ReportLogsHandler._error):
     48        * Websites/webkit-perf.appspot.com/report_process_handler.py: Copied from Websites/webkit-perf.appspot.com/report_handler.py.
     49        (ReportProcessHandler):
     50        (ReportProcessHandler.post):
     51        (ReportProcessHandler._create_build_if_possible):
     52        (ReportProcessHandler._create_build_if_possible.execute):
     53        (ReportProcessHandler._add_test_if_needed):
     54
    1552012-02-09  Ryosuke Niwa  <rniwa@webkit.org>
    256

  • trunk/Websites/webkit-perf.appspot.com/app.yaml

    r107274 r107386  
    11application: webkit-perf
    2 version: 12
     2version: 13
    33runtime: python27
    44api_version: 1

  • trunk/Websites/webkit-perf.appspot.com/controller.py

    r107337 r107386  
    6565
    6666def schedule_manifest_update():
    67     taskqueue.add(url='/api/test/update')
     67    taskqueue.add(url='/api/test/update', name='manifest_update')
    6868
    6969
    … …  
    8383
    8484def schedule_dashboard_update():
    85     taskqueue.add(url='/api/test/dashboard/update')
     85    taskqueue.add(url='/api/test/dashboard/update', name='dashboard_update')
    8686
    8787
    … …  
    101101
    102102def schedule_runs_update(test_id, branch_id, platform_id):
    103     taskqueue.add(url='/api/test/runs/update', params={'id': test_id, 'branchid': branch_id, 'platformid': platform_id})
     103    taskqueue.add(url='/api/test/runs/update', name='runs_update_%d_%d_%d' % (test_id, branch_id, platform_id),
     104        params={'id': test_id, 'branchid': branch_id, 'platformid': platform_id})
    104105
    105106
    … …  
    123124        else:
    124125            schedule_runs_update(test_id, branch_id, platform_id)
     126
     127
     128def schedule_report_process(log):
     129    taskqueue.add(url='/api/test/report/process', params={'id': log.key().id()})

  • trunk/Websites/webkit-perf.appspot.com/main.py

    r107274 r107386  
    2929from report_handler import ReportHandler
    3030from report_handler import AdminReportHandler
     31from report_process_handler import ReportProcessHandler
     32from report_logs_handler import ReportLogsHandler
    3133from runs_handler import RunsHandler
    3234from merge_tests_handler import MergeTestsHandler
    … …  
    3537    ('/admin/report/?', AdminReportHandler),
    3638    ('/admin/merge-tests/?', MergeTestsHandler),
     39    ('/admin/report-logs/?', ReportLogsHandler),
    3740    ('/admin/create/(.*)', CreateHandler),
    3841    ('/api/test/?', CachedManifestHandler),
    3942    ('/api/test/update', ManifestHandler),
    4043    ('/api/test/report/?', ReportHandler),
     44    ('/api/test/report/process', ReportProcessHandler),
    4145    ('/api/test/runs/?', CachedRunsHandler),
    4246    ('/api/test/runs/update', RunsHandler),

  • trunk/Websites/webkit-perf.appspot.com/models.py

    r107337 r107386  
    2929
    3030import hashlib
     31import json
    3132import re
    3233
     34from datetime import datetime
    3335from google.appengine.ext import db
    3436
    … …  
    122124    headers = db.TextProperty()
    123125    payload = db.TextProperty()
     126    commit = db.BooleanProperty()
     127
     128    def _parsed_payload(self):
     129        if self.__dict__.get('_parsed') == None:
     130            try:
     131                self._parsed = json.loads(self.payload)
     132            except ValueError:
     133                self._parsed = False
     134        return self._parsed
     135
     136    def get_value(self, keyName):
     137        if not self._parsed_payload():
     138            return None
     139        return self._parsed.get(keyName, '')
     140
     141    def results(self):
     142        return self.get_value('results')
     143
     144    def builder(self):
     145        return self._model_by_key_name_in_payload(Builder, 'builder-name')
     146
     147    def branch(self):
     148        return self._model_by_key_name_in_payload(Branch, 'branch')
     149
     150    def platform(self):
     151        return self._model_by_key_name_in_payload(Platform, 'platform')
     152
     153    def build_number(self):
     154        return self._integer_in_payload('build-number')
     155
     156    def webkit_revision(self):
     157        return self._integer_in_payload('webkit-revision')
     158
     159    def chromium_revision(self):
     160        return self._integer_in_payload('chromium-revision')
     161
     162    def _model_by_key_name_in_payload(self, model, keyName):
     163        key = self.get_value(keyName)
     164        if not key:
     165            return None
     166        return model.get_by_key_name(key)
     167
     168    def _integer_in_payload(self, keyName):
     169        try:
     170            return int(self.get_value(keyName))
     171        except ValueError:
     172            return None
     173
     174    def timestamp(self):
     175        try:
     176            return datetime.fromtimestamp(self._integer_in_payload('timestamp'))
     177        except TypeError:
     178            return None
     179        except ValueError:
     180            return None
    124181
    125182

  • trunk/Websites/webkit-perf.appspot.com/report_handler.py

    r107274 r107386  
    3333import json
    3434import re
    35 import time
    3635from datetime import datetime
    3736
    38 from controller import schedule_runs_update
    39 from controller import schedule_dashboard_update
    40 from controller import schedule_manifest_update
    41 from models import Builder
    42 from models import Branch
    43 from models import Build
    44 from models import NumericIdHolder
    45 from models import Platform
    4637from models import ReportLog
    47 from models import Test
    48 from models import TestResult
    49 from models import create_in_transaction_with_numeric_id_holder
     38from controller import schedule_report_process
    5039
    5140
    … …  
    6150        log.put()
    6251
     52        self._encountered_error = False
     53
    6354        try:
    64             self._body = json.loads(self.request.body)
     55            parsedPayload = json.loads(self.request.body)
     56            password = parsedPayload.get('password', '')
    6557        except ValueError:
    6658            return self._output('Failed to parse the payload as a json. Report key: %d' % log.key().id())
    6759
    68         builder = self._model_by_key_name_in_body_or_error(Builder, 'builder-name')
    69         branch = self._model_by_key_name_in_body_or_error(Branch, 'branch')
    70         platform = self._model_by_key_name_in_body_or_error(Platform, 'platform')
    71         build_number = self._integer_in_body('build-number')
    72         timestamp = self._timestamp_in_body()
    73         revision = self._integer_in_body('webkit-revision')
    74         chromium_revision = self._integer_in_body('webkit-revision') if 'chromium-revision' in self._body else None
     60        builder = log.builder()
     61        builder != None or self._output('No builder named "%s"' % log.get_value('builder-name'))
     62        log.branch() != None or self._output('No branch named "%s"' % log.get_value('branch'))
     63        log.platform() != None or self._output('No platform named "%s"' % log.get_value('platform'))
     64        log.build_number() != None or self._output('Invalid build number "%s"' % log.get_value('build-number'))
     65        log.timestamp() != None or self._output('Invalid timestamp "%s"' % log.get_value('timestamp'))
     66        log.webkit_revision() != None or self._output('Invalid webkit revision "%s"' % log.get_value('webkit-revision'))
    7567
    7668        failed = False
    77         if builder and not (self.bypass_authentication() or builder.authenticate(self._body.get('password', ''))):
     69        if builder and not (self.bypass_authentication() or builder.authenticate(password)):
    7870            self._output('Authentication failed')
    79             failed = True
    8071
    81         if not self._results_are_valid():
     72        if not self._results_are_valid(log):
    8273            self._output("The payload doesn't contain results or results are malformed")
    83             failed = True
    8474
    85         if not (builder and branch and platform and build_number and revision and timestamp) or failed:
     75        if self._encountered_error:
    8676            return
    8777
    88         build = self._create_build_if_possible(builder, build_number, branch, platform, timestamp, revision, chromium_revision)
    89         if not build:
    90             return
    91 
    92         def _float_or_none(dictionary, key):
    93             value = dictionary.get(key)
    94             if value:
    95                 return float(value)
    96             return None
    97 
    98         for test_name, result in self._body['results'].iteritems():
    99             test = self._add_test_if_needed(test_name, branch, platform)
    100             schedule_runs_update(test.id, branch.id, platform.id)
    101             if isinstance(result, dict):
    102                 TestResult(name=test_name, build=build, value=float(result['avg']), valueMedian=_float_or_none(result, 'median'),
    103                     valueStdev=_float_or_none(result, 'stdev'), valueMin=_float_or_none(result, 'min'), valueMax=_float_or_none(result, 'max')).put()
    104             else:
    105                 TestResult(name=test_name, build=build, value=float(result)).put()
    106 
    107         log = ReportLog.get(log.key())
    108         log.delete()
    109 
    110         # We need to update dashboard and manifest because they are affected by the existance of test results
    111         schedule_dashboard_update()
    112         schedule_manifest_update()
    113 
    114         return self._output('OK')
    115 
    116     def _model_by_key_name_in_body_or_error(self, model, keyName):
    117         key = self._body.get(keyName, '')
    118         instance = key and model.get_by_key_name(key)
    119         if not instance:
    120             self._output('There are no %s named "%s"' % (model.__name__.lower(), key))
    121         return instance
    122 
    123     def _integer_in_body(self, key):
    124         value = self._body.get(key, '')
    125         try:
    126             return int(value)
    127         except:
    128             return self._output('Invalid %s: "%s"' % (key.replace('-', ' '), value))
    129 
    130     def _timestamp_in_body(self):
    131         value = self._body.get('timestamp', '')
    132         try:
    133             return datetime.fromtimestamp(int(value))
    134         except:
    135             return self._output('Failed to parse the timestamp: %s' % value)
     78        log.commit = True
     79        log.put()
     80        schedule_report_process(log)
     81        self._output("OK")
    13682
    13783    def _output(self, message):
     84        self._encountered_error = True
    13885        self.response.out.write(message + '\n')
    13986
    … …  
    14188        return False
    14289
    143     def _results_are_valid(self):
     90    def _results_are_valid(self, log):
    14491
    14592        def _is_float_convertible(value):
    … …  
    14996            except TypeError:
    15097                return False
     98            except ValueError:
     99                return False
    151100
    152         if 'results' not in self._body or not isinstance(self._body['results'], dict):
     101        if not isinstance(log.results(), dict):
    153102            return False
    154103
    155         for testResult in self._body['results'].values():
     104        for testResult in log.results().values():
    156105            if isinstance(testResult, dict):
    157106                for value in testResult.values():
    … …  
    166115        return True
    167116
    168     def _create_build_if_possible(self, builder, build_number, branch, platform, timestamp, revision, chromium_revision):
    169         key_name = builder.name + ':' + str(int(time.mktime(timestamp.timetuple())))
    170 
    171         def execute():
    172             build = Build.get_by_key_name(key_name)
    173             if build:
    174                 return self._output('The build at %s already exists for %s' % (str(timestamp), builder.name))
    175 
    176             return Build(branch=branch, platform=platform, builder=builder, buildNumber=build_number,
    177                 timestamp=timestamp, revision=revision, chromiumRevision=chromium_revision, key_name=key_name).put()
    178         return db.run_in_transaction(execute)
    179 
    180     def _add_test_if_needed(self, test_name, branch, platform):
    181 
    182         def execute(id):
    183             test = Test.get_by_key_name(test_name)
    184             returnValue = None
    185             if not test:
    186                 test = Test(id=id, name=test_name, key_name=test_name)
    187                 returnValue = test
    188             if branch.key() not in test.branches:
    189                 test.branches.append(branch.key())
    190             if platform.key() not in test.platforms:
    191                 test.platforms.append(platform.key())
    192             test.put()
    193             return returnValue
    194         return create_in_transaction_with_numeric_id_holder(execute) or Test.get_by_key_name(test_name)
    195 
    196117
    197118class AdminReportHandler(ReportHandler):

  • trunk/Websites/webkit-perf.appspot.com/report_process_handler.py

    r107331 r107386  
    3131from google.appengine.ext import db
    3232
    33 import json
    34 import re
    3533import time
    36 from datetime import datetime
    3734
    3835from controller import schedule_runs_update
    3936from controller import schedule_dashboard_update
    4037from controller import schedule_manifest_update
    41 from models import Builder
    42 from models import Branch
    4338from models import Build
    44 from models import NumericIdHolder
    45 from models import Platform
    4639from models import ReportLog
    4740from models import Test
    … …  
    5043
    5144
    52 class ReportHandler(webapp2.RequestHandler):
     45class ReportProcessHandler(webapp2.RequestHandler):
    5346    def post(self):
    5447        self.response.headers['Content-Type'] = 'text/plain; charset=utf-8'
    5548
    56         headers = "\n".join([key + ': ' + value for key, value in self.request.headers.items()])
     49        log_id = int(self.request.get('id', 0))
    5750
    58         # Do as best as we can to remove the password
    59         request_body_without_password = re.sub(r'"password"\s*:\s*".+?",', '', self.request.body)
    60         log = ReportLog(timestamp=datetime.now(), headers=headers, payload=request_body_without_password)
    61         log.put()
    62 
    63         try:
    64             self._body = json.loads(self.request.body)
    65         except ValueError:
    66             return self._output('Failed to parse the payload as a json. Report key: %d' % log.key().id())
    67 
    68         builder = self._model_by_key_name_in_body_or_error(Builder, 'builder-name')
    69         branch = self._model_by_key_name_in_body_or_error(Branch, 'branch')
    70         platform = self._model_by_key_name_in_body_or_error(Platform, 'platform')
    71         build_number = self._integer_in_body('build-number')
    72         timestamp = self._timestamp_in_body()
    73         revision = self._integer_in_body('webkit-revision')
    74         chromium_revision = self._integer_in_body('webkit-revision') if 'chromium-revision' in self._body else None
    75 
    76         failed = False
    77         if builder and not (self.bypass_authentication() or builder.authenticate(self._body.get('password', ''))):
    78             self._output('Authentication failed')
    79             failed = True
    80 
    81         if not self._results_are_valid():
    82             self._output("The payload doesn't contain results or results are malformed")
    83             failed = True
    84 
    85         if not (builder and branch and platform and build_number and revision and timestamp) or failed:
    86             return
    87 
    88         build = self._create_build_if_possible(builder, build_number, branch, platform, timestamp, revision, chromium_revision)
    89         if not build:
     51        log = ReportLog.get_by_id(log_id)
     52        if not log or not log.commit:
     53            self.response.out.write("Not processed")
    9054            return
    9155
    … …  
    9660            return None
    9761
    98         for test_name, result in self._body['results'].iteritems():
     62        branch = log.branch()
     63        platform = log.platform()
     64        build = self._create_build_if_possible(log, branch, platform)
     65
     66        for test_name, result in log.results().iteritems():
    9967            test = self._add_test_if_needed(test_name, branch, platform)
    10068            schedule_runs_update(test.id, branch.id, platform.id)
    … …  
    11280        schedule_manifest_update()
    11381
    114         return self._output('OK')
     82        self.response.out.write('OK')
    11583
    116     def _model_by_key_name_in_body_or_error(self, model, keyName):
    117         key = self._body.get(keyName, '')
    118         instance = key and model.get_by_key_name(key)
    119         if not instance:
    120             self._output('There are no %s named "%s"' % (model.__name__.lower(), key))
    121         return instance
    122 
    123     def _integer_in_body(self, key):
    124         value = self._body.get(key, '')
    125         try:
    126             return int(value)
    127         except:
    128             return self._output('Invalid %s: "%s"' % (key.replace('-', ' '), value))
    129 
    130     def _timestamp_in_body(self):
    131         value = self._body.get('timestamp', '')
    132         try:
    133             return datetime.fromtimestamp(int(value))
    134         except:
    135             return self._output('Failed to parse the timestamp: %s' % value)
    136 
    137     def _output(self, message):
    138         self.response.out.write(message + '\n')
    139 
    140     def bypass_authentication(self):
    141         return False
    142 
    143     def _results_are_valid(self):
    144 
    145         def _is_float_convertible(value):
    146             try:
    147                 float(value)
    148                 return True
    149             except TypeError:
    150                 return False
    151 
    152         if 'results' not in self._body or not isinstance(self._body['results'], dict):
    153             return False
    154 
    155         for testResult in self._body['results'].values():
    156             if isinstance(testResult, dict):
    157                 for value in testResult.values():
    158                     if not _is_float_convertible(value):
    159                         return False
    160                 if 'avg' not in testResult:
    161                     return False
    162                 continue
    163             if not _is_float_convertible(testResult):
    164                 return False
    165 
    166         return True
    167 
    168     def _create_build_if_possible(self, builder, build_number, branch, platform, timestamp, revision, chromium_revision):
    169         key_name = builder.name + ':' + str(int(time.mktime(timestamp.timetuple())))
     84    def _create_build_if_possible(self, log, branch, platform):
     85        builder = log.builder()
     86        key_name = builder.name + ':' + str(int(time.mktime(log.timestamp().timetuple())))
    17087
    17188        def execute():
    17289            build = Build.get_by_key_name(key_name)
    17390            if build:
    174                 return self._output('The build at %s already exists for %s' % (str(timestamp), builder.name))
     91                return build
    17592
    176             return Build(branch=branch, platform=platform, builder=builder, buildNumber=build_number,
    177                 timestamp=timestamp, revision=revision, chromiumRevision=chromium_revision, key_name=key_name).put()
     93            return Build(branch=branch, platform=platform, builder=builder, buildNumber=log.build_number(),
     94                timestamp=log.timestamp(), revision=log.webkit_revision(), chromiumRevision=log.chromium_revision(),
     95                key_name=key_name).put()
    17896        return db.run_in_transaction(execute)
    17997
    … …  
    193111            return returnValue
    194112        return create_in_transaction_with_numeric_id_holder(execute) or Test.get_by_key_name(test_name)
    195 
    196 
    197 class AdminReportHandler(ReportHandler):
    198     def bypass_authentication(self):
    199         return True
Note: See TracChangeset for help on using the changeset viewer.