Changeset 180959 in webkit


Ignore:
Timestamp:
Mar 3, 2015 3:22:59 PM (9 years ago)
Author:
ap@apple.com
Message:

build.webkit.org/dashboard: Don't repeatedly handle each test type
https://bugs.webkit.org/show_bug.cgi?id=142211

Reviewed by Tim Horton and Matt Hanson.

  • BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/Buildbot.js:

(Buildbot.prototype.javascriptTestResultsURLForIteration): Deleted.
(Buildbot.prototype.apiTestResultsURLForIteration): Deleted.
(Buildbot.prototype.platformAPITestResultsURLForIteration): Deleted.
(Buildbot.prototype.webkitpyTestResultsURLForIteration): Deleted.
(Buildbot.prototype.webkitperlTestResultsURLForIteration): Deleted.
(Buildbot.prototype.bindingsTestResultsURLForIteration): Deleted.
Removed functions that build a link to test step results. The buildbot provides
these links in JSON.

  • BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotIteration.js:

(BuildbotIteration): Put failing tests into an array, instead of named variables.
(BuildbotIteration.ProductiveSteps): Removed step names that are not used on build.webkit.org.
We can easily add them to the map as needed.
(BuildbotIteration.TestSteps): Added a list of test steps to be displayed by test queues.
(BuildbotIteration.prototype._parseData): Moved code for parsing step results away
to BuildbotTestResults class. We used to parse here, build an intermediate data structure,
and then build a BuildbotTestResults object, which was strange.
(BuildbotIteration.prototype.loadLayoutTestResults): Ditto.

  • BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotQueueView.js:

Corrected an unrelated assertion that was buggy, and kept firing.

  • BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotTestResults.js:

(BuildbotTestResults):
(BuildbotTestResults.prototype._parseResults.resultSummarizer):
(BuildbotTestResults.prototype._parseResults):
(BuildbotTestResults.prototype.addFullLayoutTestResults):
Moved the code for parsing JSON results for a single step here.

  • BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotTesterQueueView.js:

(BuildbotTesterQueueView.prototype._testStepFailureDescription):
(BuildbotTesterQueueView.prototype._testStepFailureDescriptionWithCount):
(BuildbotTesterQueueView.prototype._presentPopoverForGenericTestFailures):
(BuildbotTesterQueueView.prototype.update.appendBuilderQueueStatus): Deleted.
(BuildbotTesterQueueView.prototype.update): Deleted.
(BuildbotTesterQueueView.prototype._presentPopoverForMultipleFailureKinds): Deleted.
Updated for the new data structures. One behavior change is that we now display individual
counts when multiple test kinds fail, e.g. "1 javascript test failure, 83+ layout
test failures, 3 platform api test failures".

Location:
trunk/Tools
Files:
6 edited

Legend:

Unmodified
Added
Removed
  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/Buildbot.js

    r163213 r180959  
    114114    },
    115115
    116     javascriptTestResultsURLForIteration: function(iteration)
    117     {
    118         return this.baseURL + "builders/" + encodeURIComponent(iteration.queue.id) + "/builds/" + iteration.id + "/steps/jscore-test/logs/stdio";
    119     },
    120 
    121     apiTestResultsURLForIteration: function(iteration)
    122     {
    123         return this.baseURL + "builders/" + encodeURIComponent(iteration.queue.id) + "/builds/" + iteration.id + "/steps/run-api-tests/logs/stdio";
    124     },
    125 
    126     platformAPITestResultsURLForIteration: function(iteration)
    127     {
    128         return this.baseURL + "builders/" + encodeURIComponent(iteration.queue.id) + "/builds/" + iteration.id + "/steps/API%20tests/logs/stdio";
    129     },
    130 
    131     webkitpyTestResultsURLForIteration: function(iteration)
    132     {
    133         return this.baseURL + "builders/" + encodeURIComponent(iteration.queue.id) + "/builds/" + iteration.id + "/steps/webkitpy-test/logs/stdio";
    134     },
    135 
    136     webkitperlTestResultsURLForIteration: function(iteration)
    137     {
    138         return this.baseURL + "builders/" + encodeURIComponent(iteration.queue.id) + "/builds/" + iteration.id + "/steps/webkitperl-test/logs/stdio";
    139     },
    140 
    141     bindingsTestResultsURLForIteration: function(iteration)
    142     {
    143         return this.baseURL + "builders/" + encodeURIComponent(iteration.queue.id) + "/builds/" + iteration.id + "/steps/bindings-generation-tests/logs/stdio";
    144     },
    145 
    146116    layoutTestResultsURLForIteration: function(iteration)
    147117    {
  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotIteration.js

    r180920 r180959  
    4747    this.internalRevision = null;
    4848
    49     this.layoutTestResults = null;
    50     this.javascriptTestResults = null;
    51     this.apiTestResults = null;
    52     this.platformAPITestResults = null;
    53     this.pythonTestResults = null;
    54     this.perlTestResults = null;
    55     this.bindingTestResults = null;
     49    this.layoutTestResults = null; // Layout test results can be needed even if all tests passed, e.g. for the leaks bot.
     50
     51    this.failedTestSteps = [];
    5652
    5753    this._finished = finished;
     
    7167BuildbotIteration.ProductiveSteps = {
    7268    "compile-webkit": 1,
    73     "build archive": 1,
    7469    "build ASan archive": 1,
    7570    "Build" : 1,
     
    8176    "webkitperl-test": 1,
    8277    "bindings-generation-tests": 1,
    83     "run Membuster OS Memory": 1,
    84     "run scrollperf": 1,
    85     "run PLT3": 1,
    8678    "perf-test": 1
     79};
     80
     81BuildbotIteration.TestSteps = {
     82    "layout-test": "layout test",
     83    "jscore-test": "javascript test",
     84    "run-api-tests": "api test",
     85    "API tests": "platform api test",
     86    "webkitpy-test": "webkitpy test",
     87    "webkitperl-test": "webkitperl test",
     88    "bindings-generation-tests": "bindings tests",
    8789};
    8890
     
    186188    _parseData: function(data)
    187189    {
    188         function collectTestResults(data, stepName)
    189         {
    190             var testStep = data.steps.findFirst(function(step) { return step.name === stepName; });
    191             if (!testStep)
    192                 return null;
    193 
    194             var testResults = {};
    195 
    196             if (!testStep.isFinished) {
    197                 // The step never even ran, or hasn't finished running.
    198                 testResults.finished = false;
    199                 return testResults;
    200             }
    201 
    202             testResults.finished = true;
    203 
    204             if (!testStep.results || testStep.results[0] === BuildbotIteration.SUCCESS) {
    205                 // All tests passed.
    206                 testResults.allPassed = true;
    207                 return testResults;
    208             }
    209 
    210             if (/Exiting early/.test(testStep.results[1][0]))
    211                 testResults.tooManyFailures = true;
    212 
    213             function resultSummarizer(matchString, sum, outputLine)
    214             {
    215                 var match = /^(\d+)\s/.exec(outputLine);
    216                 if (!match)
    217                     return sum;
    218                 if (!outputLine.contains(matchString))
    219                     return sum;
    220                 if (!sum || sum === -1)
    221                     sum = 0;
    222                 return sum + parseInt(match[1], 10);
    223             }
    224 
    225             testResults.failureCount = testStep.results[1].reduce(resultSummarizer.bind(null, "fail"), undefined);
    226             testResults.flakeyCount = testStep.results[1].reduce(resultSummarizer.bind(null, "flake"), undefined);
    227             testResults.totalLeakCount = testStep.results[1].reduce(resultSummarizer.bind(null, "total leak"), undefined);
    228             testResults.uniqueLeakCount = testStep.results[1].reduce(resultSummarizer.bind(null, "unique leak"), undefined);
    229             testResults.newPassesCount = testStep.results[1].reduce(resultSummarizer.bind(null, "new pass"), undefined);
    230             testResults.missingCount = testStep.results[1].reduce(resultSummarizer.bind(null, "missing"), undefined);
    231             testResults.crashCount = testStep.results[1].reduce(resultSummarizer.bind(null, "crash"), undefined);
    232 
    233             if (!testResults.failureCount && !testResults.flakyCount && !testResults.totalLeakCount && !testResults.uniqueLeakCount && !testResults.newPassesCount && !testResults.missingCount) {
    234                 // This step exited with a non-zero exit status, but we didn't find any output about the number of failed tests.
    235                 // Something must have gone wrong (e.g., timed out and was killed by buildbot).
    236                 testResults.errorOccurred = true;
    237             }
    238 
    239             return testResults;
    240         }
    241 
    242190        console.assert(!this.id || this.id === data.number);
    243191        this.id = data.number;
     
    287235        this.endTime = new Date(data.times[1] * 1000);
    288236
    289         var layoutTestResults = collectTestResults.call(this, data, "layout-test");
    290         this.layoutTestResults = layoutTestResults ? new BuildbotTestResults(this, layoutTestResults) : null;
    291 
    292         var javascriptTestResults = collectTestResults.call(this, data, "jscore-test");
    293         this.javascriptTestResults = javascriptTestResults ? new BuildbotTestResults(this, javascriptTestResults) : null;
    294 
    295         var apiTestResults = collectTestResults.call(this, data, "run-api-tests");
    296         this.apiTestResults = apiTestResults ? new BuildbotTestResults(this, apiTestResults) : null;
    297 
    298         var platformAPITestResults = collectTestResults.call(this, data, "API tests");
    299         this.platformAPITestResults = platformAPITestResults ? new BuildbotTestResults(this, platformAPITestResults) : null;
    300 
    301         var pythonTestResults = collectTestResults.call(this, data, "webkitpy-test");
    302         this.pythonTestResults = pythonTestResults ? new BuildbotTestResults(this, pythonTestResults) : null;
    303 
    304         var perlTestResults = collectTestResults.call(this, data, "webkitperl-test");
    305         this.perlTestResults = perlTestResults ? new BuildbotTestResults(this, perlTestResults) : null;
    306 
    307         var bindingTestResults = collectTestResults.call(this, data, "bindings-generation-tests");
    308         this.bindingTestResults = bindingTestResults ? new BuildbotTestResults(this, bindingTestResults) : null;
     237        this.failedTestSteps = [];
     238        data.steps.forEach(function(step) {
     239            if (!step.isFinished || !(step.name in BuildbotIteration.TestSteps))
     240                return;
     241            var results = new BuildbotTestResults(step);
     242            if (step.name === "layout-test")
     243                this.layoutTestResults = results;
     244            if (results.allPassed)
     245                return;
     246            this.failedTestSteps.push(results);
     247        }, this);
    309248
    310249        var masterShellCommandStep = data.steps.findFirst(function(step) { return step.name === "MasterShellCommand"; });
     
    392331            return;
    393332
    394         function collectResults(subtree, predicate)
    395         {
    396             // Results object is a trie:
    397             // directory
    398             //   subdirectory
    399             //     test1.html
    400             //       expected:"PASS"
    401             //       actual: "IMAGE"
    402             //       report: "REGRESSION"
    403             //     test2.html
    404             //       expected:"FAIL"
    405             //       actual:"TEXT"
    406 
    407             var result = [];
    408             for (var key in subtree) {
    409                 var value = subtree[key];
    410                 console.assert(typeof value === "object");
    411                 var isIndividualTest = value.hasOwnProperty("actual") && value.hasOwnProperty("expected");
    412                 if (isIndividualTest) {
    413                     // Possible values for actual and expected keys: PASS, FAIL, AUDIO, IMAGE, TEXT, IMAGE+TEXT, TIMEOUT, CRASH, MISSING.
    414                     // Both actual and expected can be space separated lists. Actual contains two values when retrying a failed test
    415                     // gives a different result (retrying may be disabled in tester configuration).
    416                     // Possible values for report key (when present): REGRESSION, MISSING, FLAKY.
    417 
    418                     if (predicate(value)) {
    419                         var item = {path: key};
    420 
    421                         // FIXME (bug 127186): Crash log URL will be incorrect if crash only happened on retry (e.g. "TEXT CRASH").
    422                         // It should point to retries subdirectory, but the information about which attempt failed gets lost here.
    423                         if (value.actual.contains("CRASH"))
    424                             item.crash = true;
    425                         if (value.actual.contains("TIMEOUT"))
    426                             item.timeout = true;
    427 
    428                         // FIXME (bug 127186): Similarly, we don't have a good way to present results for something like "TIMEOUT TEXT",
    429                         // not even UI wise. For now, only show a diff link if the first attempt has the diff.
    430                         if (value.actual.split(" ")[0].contains("TEXT"))
    431                             item.has_diff = true;
    432 
    433                         // FIXME (bug 127186): It is particularly unfortunate for image diffs, because we currently only check image results
    434                         // on retry (except for reftests), so many times, you will see images on buidbot page, but not on the dashboard.
    435                         // FIXME: Find a way to display expected mismatch reftest failures.
    436                         if (value.actual.split(" ")[0].contains("IMAGE") && value.reftest_type != "!=")
    437                             item.has_image_diff = true;
    438 
    439                         if (value.has_stderr)
    440                             item.has_stderr = true;
    441 
    442                         result.push(item);
    443                     }
    444 
    445                 } else {
    446                     var nestedTests = collectResults(value, predicate);
    447                     for (var i = 0, end = nestedTests.length; i < end; ++i)
    448                         nestedTests[i].path = key + "/" + nestedTests[i].path;
    449                     result = result.concat(nestedTests);
    450                 }
    451             }
    452 
    453             return result;
    454         }
    455 
    456333        JSON.load(this.queue.buildbot.layoutTestFullResultsURLForIteration(this), function(data) {
    457334            this.queue.buildbot.isAuthenticated = true;
    458             this.hasPrettyPatch = data.has_pretty_patch;
    459 
    460             this.layoutTestResults.regressions = collectResults(data.tests, function(info) { return info["report"] === "REGRESSION" });
    461             console.assert(data.num_regressions === this.layoutTestResults.regressions.length);
    462 
    463             this.layoutTestResults.flakyTests = collectResults(data.tests, function(info) { return info["report"] === "FLAKY" });
    464             console.assert(data.num_flaky === this.layoutTestResults.flakyTests.length);
    465 
    466             this.layoutTestResults.testsWithMissingResults = collectResults(data.tests, function(info) { return info["report"] === "MISSING" });
    467             // data.num_missing is not always equal to the size of testsWithMissingResults array,
    468             // because buildbot counts regressions that had missing pixel results on retry (e.g. "TEXT MISSING").
    469             console.assert(data.num_missing >= this.layoutTestResults.testsWithMissingResults.length);
    470 
     335
     336            this.layoutTestResults.addFullLayoutTestResults(data);
    471337            callback();
    472338        }.bind(this),
  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotQueueView.js

    r180524 r180959  
    145145        }
    146146
    147         console.assert(trac.oldestRecordedRevisionNumber >= firstRevisionNumber);
     147        console.assert(trac.oldestRecordedRevisionNumber <= firstRevisionNumber);
    148148
    149149        // FIXME: To be 100% correct, we should also filter out changes that are ignored by
  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotTestResults.js

    r179650 r180959  
    2424 */
    2525
    26 BuildbotTestResults = function(buildbotIteration, info)
     26BuildbotTestResults = function(testStep)
    2727{
    2828    BaseObject.call(this);
    2929
    30     this.finished = info.finished || false;
    31 
    32     this.allPassed = info.allPassed || false;
    33     this.errorOccurred = info.errorOccurred || false;
    34     this.tooManyFailures = info.tooManyFailures || false;
    35 
    36     this.failureCount = info.failureCount || 0;
    37     this.flakeyCount = info.flakeyCount || 0;
    38     this.totalLeakCount = info.totalLeakCount || 0;
    39     this.uniqueLeakCount = info.uniqueLeakCount || 0;
    40     this.newPassesCount = info.newPassesCount || 0;
    41     this.missingCount = info.missingCount || 0;
    42     this.crashCount = info.crashCount || 0;
     30    this._parseResults(testStep);
    4331};
    4432
     
    4735BuildbotTestResults.prototype = {
    4836    constructor: BuildbotTestResults,
    49     __proto__: BaseObject.prototype
     37    __proto__: BaseObject.prototype,
     38
     39    _parseResults: function(testStep)
     40    {
     41        this.name = testStep.name;
     42        this.URL = testStep.logs[0][1];
     43
     44        this.allPassed = false;
     45        this.errorOccurred = false;
     46        this.tooManyFailures = false;
     47
     48        this.failureCount = 0;
     49        this.flakeyCount = 0;
     50        this.totalLeakCount = 0;
     51        this.uniqueLeakCount = 0;
     52        this.newPassesCount = 0;
     53        this.missingCount = 0;
     54        this.crashCount = 0;
     55
     56        if (!testStep.isFinished) {
     57            // The step never even ran, or hasn't finished running.
     58            this.finished = false;
     59            return;
     60        }
     61
     62        this.finished = true;
     63
     64        if (!testStep.results || testStep.results[0] === BuildbotIteration.SUCCESS || testStep.results[0] === BuildbotIteration.WARNINGS) {
     65            // All tests passed.
     66            this.allPassed = true;
     67            return;
     68        }
     69
     70        if (/Exiting early/.test(testStep.results[1][0]))
     71            this.tooManyFailures = true;
     72
     73        function resultSummarizer(matchString, sum, outputLine)
     74        {
     75            var match = /^(\d+)\s/.exec(outputLine);
     76            if (!match)
     77                return sum;
     78            if (!outputLine.contains(matchString))
     79                return sum;
     80            if (!sum || sum === -1)
     81                sum = 0;
     82            return sum + parseInt(match[1], 10);
     83        }
     84
     85        this.failureCount = testStep.results[1].reduce(resultSummarizer.bind(null, "fail"), undefined);
     86        this.flakeyCount = testStep.results[1].reduce(resultSummarizer.bind(null, "flake"), undefined);
     87        this.totalLeakCount = testStep.results[1].reduce(resultSummarizer.bind(null, "total leak"), undefined);
     88        this.uniqueLeakCount = testStep.results[1].reduce(resultSummarizer.bind(null, "unique leak"), undefined);
     89        this.newPassesCount = testStep.results[1].reduce(resultSummarizer.bind(null, "new pass"), undefined);
     90        this.missingCount = testStep.results[1].reduce(resultSummarizer.bind(null, "missing"), undefined);
     91        this.crashCount = testStep.results[1].reduce(resultSummarizer.bind(null, "crash"), undefined);
     92
     93        if (!this.failureCount && !this.flakyCount && !this.totalLeakCount && !this.uniqueLeakCount && !this.newPassesCount && !this.missingCount) {
     94            // This step exited with a non-zero exit status, but we didn't find any output about the number of failed tests.
     95            // Something must have gone wrong (e.g., timed out and was killed by buildbot).
     96            this.errorOccurred = true;
     97        }
     98    },
     99
     100    addFullLayoutTestResults: function(data)
     101    {
     102        console.assert(this.name === "layout-test");
     103
     104        function collectResults(subtree, predicate)
     105        {
     106            // Results object is a trie:
     107            // directory
     108            //   subdirectory
     109            //     test1.html
     110            //       expected:"PASS"
     111            //       actual: "IMAGE"
     112            //       report: "REGRESSION"
     113            //     test2.html
     114            //       expected:"FAIL"
     115            //       actual:"TEXT"
     116
     117            var result = [];
     118            for (var key in subtree) {
     119                var value = subtree[key];
     120                console.assert(typeof value === "object");
     121                var isIndividualTest = value.hasOwnProperty("actual") && value.hasOwnProperty("expected");
     122                if (isIndividualTest) {
     123                    // Possible values for actual and expected keys: PASS, FAIL, AUDIO, IMAGE, TEXT, IMAGE+TEXT, TIMEOUT, CRASH, MISSING.
     124                    // Both actual and expected can be space separated lists. Actual contains two values when retrying a failed test
     125                    // gives a different result (retrying may be disabled in tester configuration).
     126                    // Possible values for report key (when present): REGRESSION, MISSING, FLAKY.
     127
     128                    if (predicate(value)) {
     129                        var item = {path: key};
     130
     131                        // FIXME (bug 127186): Crash log URL will be incorrect if crash only happened on retry (e.g. "TEXT CRASH").
     132                        // It should point to retries subdirectory, but the information about which attempt failed gets lost here.
     133                        if (value.actual.contains("CRASH"))
     134                            item.crash = true;
     135                        if (value.actual.contains("TIMEOUT"))
     136                            item.timeout = true;
     137
     138                        // FIXME (bug 127186): Similarly, we don't have a good way to present results for something like "TIMEOUT TEXT",
     139                        // not even UI wise. For now, only show a diff link if the first attempt has the diff.
     140                        if (value.actual.split(" ")[0].contains("TEXT"))
     141                            item.has_diff = true;
     142
     143                        // FIXME (bug 127186): It is particularly unfortunate for image diffs, because we currently only check image results
     144                        // on retry (except for reftests), so many times, you will see images on buildbot page, but not on the dashboard.
     145                        // FIXME: Find a way to display expected mismatch reftest failures.
     146                        if (value.actual.split(" ")[0].contains("IMAGE") && value.reftest_type != "!=")
     147                            item.has_image_diff = true;
     148
     149                        if (value.has_stderr)
     150                            item.has_stderr = true;
     151
     152                        result.push(item);
     153                    }
     154
     155                } else {
     156                    var nestedTests = collectResults(value, predicate);
     157                    for (var i = 0, end = nestedTests.length; i < end; ++i)
     158                        nestedTests[i].path = key + "/" + nestedTests[i].path;
     159                    result = result.concat(nestedTests);
     160                }
     161            }
     162
     163            return result;
     164        }
     165
     166        this.hasPrettyPatch = data.has_pretty_patch;
     167
     168        this.regressions = collectResults(data.tests, function(info) { return info["report"] === "REGRESSION" });
     169        console.assert(data.num_regressions === this.regressions.length);
     170
     171        this.flakyTests = collectResults(data.tests, function(info) { return info["report"] === "FLAKY" });
     172        console.assert(data.num_flaky === this.flakyTests.length);
     173
     174        this.testsWithMissingResults = collectResults(data.tests, function(info) { return info["report"] === "MISSING" });
     175        // data.num_missing is not always equal to the size of testsWithMissingResults array,
     176        // because buildbot counts regressions that had missing pixel results on retry (e.g. "TEXT MISSING").
     177        console.assert(data.num_missing >= this.testsWithMissingResults.length);
     178    },
    50179};
  • trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotTesterQueueView.js

    r180525 r180959  
    6565                var messageElement = this.revisionContentForIteration(iteration, (iteration.productive && willHaveAnotherStatusLine) ? iteration.previousProductiveIteration : null);
    6666
    67                 var layoutTestResults = iteration.layoutTestResults || {failureCount: 0};
    68                 var javascriptTestResults = iteration.javascriptTestResults || {failureCount: 0};
    69                 var apiTestResults = iteration.apiTestResults || {failureCount: 0};
    70                 var platformAPITestResults = iteration.platformAPITestResults || {failureCount: 0};
    71                 var pythonTestResults = iteration.pythonTestResults || {failureCount: 0};
    72                 var perlTestResults = iteration.perlTestResults || {errorOccurred: false};
    73                 var bindingTestResults = iteration.bindingTestResults || {errorOccurred: false};
    74 
    7567                if (iteration.successful) {
    7668                    var url = iteration.queue.buildbot.buildPageURLForIteration(iteration);
     
    8072                    var url = iteration.queue.buildbot.buildPageURLForIteration(iteration);
    8173                    var status = new StatusLineView(messageElement, StatusLineView.Status.Danger, iteration.text, undefined, url);
    82                 } else if (queue.crashesOnly && !layoutTestResults.crashCount) {
    83                     var url = iteration.queue.buildbot.buildPageURLForIteration(iteration);
    84                     var status = new StatusLineView(messageElement, StatusLineView.Status.Good, "no crashes found", undefined, url);
    85                     limit = 0;
    86                 } else if (queue.crashesOnly && layoutTestResults.crashCount) {
    87                     var url = iteration.queue.buildbot.layoutTestResultsURLForIteration(iteration);
    88                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, layoutTestResults.crashCount === 1 ? "crash found" : "crashes found", layoutTestResults.crashCount, url);
    89                     new PopoverTracker(status.statusBubbleElement, this._presentPopoverForLayoutTestRegressions.bind(this), iteration);
    90                 } else if (!layoutTestResults.failureCount && !javascriptTestResults.failureCount && !apiTestResults.failureCount && !platformAPITestResults.failureCount && !pythonTestResults.failureCount && !perlTestResults.errorOccurred && !bindingTestResults.errorOccurred) {
     74                } else if (iteration.failedTestSteps.length === 0) {
    9175                    // Something wrong happened, but it was not a test failure.
    9276                    var url = iteration.queue.buildbot.buildPageURLForIteration(iteration);
    9377                    var status = new StatusLineView(messageElement, StatusLineView.Status.Danger, iteration.text, undefined, url);
    94                 } else if (layoutTestResults.failureCount && !javascriptTestResults.failureCount && !apiTestResults.failureCount && !platformAPITestResults.failureCount && !pythonTestResults.failureCount && !perlTestResults.errorOccurred && !bindingTestResults.errorOccurred) {
    95                     var url = iteration.queue.buildbot.layoutTestResultsURLForIteration(iteration);
    96                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, layoutTestResults.failureCount === 1 ? "layout test failure" : "layout test failures", layoutTestResults.tooManyFailures ? layoutTestResults.failureCount + "\uff0b" : layoutTestResults.failureCount, url);
    97                     new PopoverTracker(status.statusBubbleElement, this._presentPopoverForLayoutTestRegressions.bind(this), iteration);
    98                 } else if (!layoutTestResults.failureCount && javascriptTestResults.failureCount && !apiTestResults.failureCount && !platformAPITestResults.failureCount && !pythonTestResults.failureCount && !perlTestResults.errorOccurred && !bindingTestResults.errorOccurred) {
    99                     var url = iteration.queue.buildbot.javascriptTestResultsURLForIteration(iteration);
    100                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, javascriptTestResults.failureCount === 1 ? "javascript test failure" : "javascript test failures", javascriptTestResults.failureCount, url);
    101                 } else if (!layoutTestResults.failureCount && !javascriptTestResults.failureCount && apiTestResults.failureCount && !platformAPITestResults.failureCount && !pythonTestResults.failureCount && !perlTestResults.errorOccurred && !bindingTestResults.errorOccurred) {
    102                     var url = iteration.queue.buildbot.apiTestResultsURLForIteration(iteration);
    103                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, apiTestResults.failureCount === 1 ? "api test failure" : "api test failures", apiTestResults.failureCount, url);
    104                 } else if (!layoutTestResults.failureCount && !javascriptTestResults.failureCount && !apiTestResults.failureCount && platformAPITestResults.failureCount && !pythonTestResults.failureCount && !perlTestResults.errorOccurred && !bindingTestResults.errorOccurred) {
    105                     var url = iteration.queue.buildbot.platformAPITestResultsURLForIteration(iteration);
    106                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, platformAPITestResults.failureCount === 1 ? "platform api test failure" : "api test failures", platformAPITestResults.failureCount, url);
    107                 } else if (!layoutTestResults.failureCount && !javascriptTestResults.failureCount && !apiTestResults.failureCount && !platformAPITestResults.failureCount && pythonTestResults.failureCount && !perlTestResults.errorOccurred && !bindingTestResults.errorOccurred) {
    108                     var url = iteration.queue.buildbot.webkitpyTestResultsURLForIteration(iteration);
    109                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, pythonTestResults.failureCount === 1 ? "webkitpy test failure" : "webkitpy test failures", pythonTestResults.failureCount, url);
    110                 } else if (!layoutTestResults.failureCount && !javascriptTestResults.failureCount && !apiTestResults.failureCount && !platformAPITestResults.failureCount && !pythonTestResults.failureCount && perlTestResults.errorOccurred && !bindingTestResults.errorOccurred) {
    111                     var url = iteration.queue.buildbot.webkitperlTestResultsURLForIteration(iteration);
    112                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, "webkitperl test failed", undefined, url);
    113                 } else if (!layoutTestResults.failureCount && !javascriptTestResults.failureCount && !apiTestResults.failureCount && !platformAPITestResults.failureCount && !pythonTestResults.failureCount && !perlTestResults.errorOccurred && bindingTestResults.errorOccurred) {
    114                     var url = iteration.queue.buildbot.bindingsTestResultsURLForIteration(iteration);
    115                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, "bindings tests failed", undefined, url);
     78                } else if (queue.crashesOnly) {
     79                    // A crashes-only queue is a queue where we are only interested in crashes, e.g. a GuardMalloc or an ASan one.
     80                    // Currently, only layout tests are supported in such.
     81                    var layoutTestResults = iteration.layoutTestResults;
     82                    if (!layoutTestResults.crashCount) {
     83                        var url = iteration.queue.buildbot.buildPageURLForIteration(iteration);
     84                        var status = new StatusLineView(messageElement, StatusLineView.Status.Good, "no crashes found", undefined, url);
     85                        limit = 0;
     86                    } else {
     87                        var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, layoutTestResults.crashCount === 1 ? "crash found" : "crashes found", layoutTestResults.crashCount, iteration.queue.buildbot.layoutTestResultsURLForIteration(iteration));
     88                        new PopoverTracker(status.statusBubbleElement, this._presentPopoverForLayoutTestRegressions.bind(this), iteration);
     89                    }
     90                } else if (iteration.failedTestSteps.length === 1) {
     91                    var failedStep = iteration.failedTestSteps[0];
     92                    if (failedStep.name === "layout-test") {
     93                        var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, this._testStepFailureDescription(failedStep), failedStep.tooManyFailures ? failedStep.failureCount + "\uff0b" : failedStep.failureCount, iteration.queue.buildbot.layoutTestResultsURLForIteration(iteration));
     94                        new PopoverTracker(status.statusBubbleElement, this._presentPopoverForLayoutTestRegressions.bind(this), iteration);
     95                    } else {
     96                        var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, this._testStepFailureDescription(failedStep), failedStep.failureCount ? failedStep.failureCount : undefined, failedStep.URL);
     97                        new PopoverTracker(status.statusBubbleElement, this._presentPopoverForGenericTestFailures.bind(this), iteration);
     98                    }
    11699                } else {
    117100                    var url = iteration.queue.buildbot.buildPageURLForIteration(iteration);
    118                     var totalFailures = layoutTestResults.failureCount + javascriptTestResults.failureCount + apiTestResults.failureCount + platformAPITestResults.failureCount + pythonTestResults.failureCount + perlTestResults.errorOccurred + bindingTestResults.errorOccurred;
    119                     var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, totalFailures === 1 ? "test failure" : "test failures", totalFailures, url);
    120                     new PopoverTracker(status.statusBubbleElement, this._presentPopoverForMultipleFailureKinds.bind(this), iteration);
     101                    var failureDescriptions = iteration.failedTestSteps.map(function(failedStep) { return this._testStepFailureDescriptionWithCount(failedStep) }, this);
     102                    var status = new StatusLineView(messageElement, StatusLineView.Status.Bad, failureDescriptions.join(", "), undefined, url);
     103                    new PopoverTracker(status.statusBubbleElement, this._presentPopoverForGenericTestFailures.bind(this), iteration);
    121104                }
    122105
     
    133116        this.appendBuildStyle.call(this, this.releaseQueues, "Release", appendBuilderQueueStatus);
    134117        this.appendBuildStyle.call(this, this.debugQueues, "Debug", appendBuilderQueueStatus);
     118    },
     119
     120    _testStepFailureDescription: function(failedStep)
     121    {
     122        if (!failedStep.failureCount)
     123            return BuildbotIteration.TestSteps[failedStep.name] + " failed";
     124        if (failedStep.failureCount === 1)
     125            return BuildbotIteration.TestSteps[failedStep.name] + " failure";
     126        return BuildbotIteration.TestSteps[failedStep.name] + " failures";
     127    },
     128
     129    _testStepFailureDescriptionWithCount: function(failedStep)
     130    {
     131        if (!failedStep.failureCount)
     132            return this._testStepFailureDescription(failedStep);
     133        if (failedStep.tooManyFailures) {
     134            // E.g. "50+ layout test failures", preventing line breaks around the "+".
     135            return failedStep.failureCount + "\ufeff\uff0b\u00a0" + this._testStepFailureDescription(failedStep);
     136        }
     137        // E.g. "1 layout test failure", preventing line break after the number.
     138        return failedStep.failureCount + "\u00a0" + this._testStepFailureDescription(failedStep);
    135139    },
    136140
     
    264268    },
    265269
    266     _presentPopoverForMultipleFailureKinds: function(element, popover, iteration)
     270    _presentPopoverForGenericTestFailures: function(element, popover, iteration)
    267271    {
    268272        function addResultKind(message, url) {
     
    275279        }
    276280
    277         var layoutTestResults = iteration.layoutTestResults || {failureCount: 0};
    278         var javascriptTestResults = iteration.javascriptTestResults || {failureCount: 0};
    279         var apiTestResults = iteration.apiTestResults || {failureCount: 0};
    280         var platformAPITestResults = iteration.platformAPITestResults || {failureCount: 0};
    281         var pythonTestResults = iteration.pythonTestResults || {failureCount: 0};
    282         var perlTestResults = iteration.perlTestResults || {errorOccurred: false};
    283         var bindingTestResults = iteration.bindingTestResults || {errorOccurred: false};
    284 
    285281        var content = document.createElement("div");
    286282        content.className = "test-results-popover";
     
    289285        this._addDividerToPopover(content);
    290286
    291         if (layoutTestResults.failureCount) {
    292             var message = (layoutTestResults.tooManyFailures ? layoutTestResults.failureCount + "\uff0b" : layoutTestResults.failureCount) + "\u00a0" +
    293                 (layoutTestResults.failureCount === 1 ? "layout test failure" : "layout test failures");
    294             addResultKind(message, iteration.queue.buildbot.layoutTestResultsURLForIteration(iteration));
    295         }
    296 
    297         if (javascriptTestResults.failureCount) {
    298             var message = javascriptTestResults.failureCount + "\u00a0" + (javascriptTestResults.failureCount === 1 ? "javascript test failure" : "javascript test failures");
    299             addResultKind(message, iteration.queue.buildbot.javascriptTestResultsURLForIteration(iteration));
    300         }
    301 
    302         if (apiTestResults.failureCount) {
    303             var message = apiTestResults.failureCount + "\u00a0" + (apiTestResults.failureCount === 1 ? "api test failure" : "api test failures");
    304             addResultKind(message, iteration.queue.buildbot.apiTestResultsURLForIteration(iteration));
    305         }
    306 
    307         if (platformAPITestResults.failureCount) {
    308             var message = platformAPITestResults.failureCount + "\u00a0" + (platformAPITestResults.failureCount === 1 ? "platform api test failure" : "platform api test failures");
    309             addResultKind(message, iteration.queue.buildbot.platformAPITestResultsURLForIteration(iteration));
    310         }
    311 
    312         if (pythonTestResults.failureCount) {
    313             var message = pythonTestResults.failureCount + "\u00a0" + (pythonTestResults.failureCount === 1 ? "webkitpy test failure" : "webkitpy test failures");
    314             addResultKind(message, iteration.queue.buildbot.webkitpyTestResultsURLForIteration(iteration));
    315         }
    316 
    317         if (perlTestResults.errorOccurred)
    318             addResultKind("webkitperl tests failed", iteration.queue.buildbot.webkitperlTestResultsURLForIteration(iteration));
    319 
    320         if (bindingTestResults.errorOccurred)
    321             addResultKind("bindings tests failed", iteration.queue.buildbot.bindingsTestResultsURLForIteration(iteration));
     287        iteration.failedTestSteps.forEach(function(failedStep) {
     288            if (failedStep.name === "layout-test")
     289                addResultKind(this._testStepFailureDescriptionWithCount(failedStep), iteration.queue.buildbot.layoutTestResultsURLForIteration(iteration));
     290            else
     291                addResultKind(this._testStepFailureDescriptionWithCount(failedStep), failedStep.URL);
     292        }, this);
    322293
    323294        var rect = Dashboard.Rect.rectFromClientRect(element.getBoundingClientRect());
  • trunk/Tools/ChangeLog

    r180945 r180959  
     12015-03-03  Alexey Proskuryakov  <ap@apple.com>
     2
     3        build.webkit.org/dashboard: Don't repeatedly handle each test type
     4        https://bugs.webkit.org/show_bug.cgi?id=142211
     5
     6        Reviewed by Tim Horton and Matt Hanson.
     7
     8        * BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/Buildbot.js:
     9        (Buildbot.prototype.javascriptTestResultsURLForIteration): Deleted.
     10        (Buildbot.prototype.apiTestResultsURLForIteration): Deleted.
     11        (Buildbot.prototype.platformAPITestResultsURLForIteration): Deleted.
     12        (Buildbot.prototype.webkitpyTestResultsURLForIteration): Deleted.
     13        (Buildbot.prototype.webkitperlTestResultsURLForIteration): Deleted.
     14        (Buildbot.prototype.bindingsTestResultsURLForIteration): Deleted.
     15        Removed functions that build a link to test step results. The buildbot provides
     16        these links in JSON.
     17
     18        * BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotIteration.js:
     19        (BuildbotIteration): Put failing tests into an array, instead of named variables.
     20        (BuildbotIteration.ProductiveSteps): Removed step names that are not used on build.webkit.org.
     21        We can easily add them to the map as needed.
     22        (BuildbotIteration.TestSteps): Added a list of test steps to be displayed by test queues.
     23        (BuildbotIteration.prototype._parseData): Moved code for parsing step results away
     24        to BuildbotTestResults class. We used to parse here, build an intermediate data structure,
     25        and then build a BuildbotTestResults object, which was strange.
     26        (BuildbotIteration.prototype.loadLayoutTestResults): Ditto.
     27
     28        * BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotQueueView.js:
     29        Corrected an unrelated assertion that was buggy, and kept firing.
     30
     31        * BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotTestResults.js:
     32        (BuildbotTestResults):
     33        (BuildbotTestResults.prototype._parseResults.resultSummarizer):
     34        (BuildbotTestResults.prototype._parseResults):
     35        (BuildbotTestResults.prototype.addFullLayoutTestResults):
     36        Moved the code for parsing JSON results for a single step here.
     37
     38        * BuildSlaveSupport/build.webkit.org-config/public_html/dashboard/Scripts/BuildbotTesterQueueView.js:
     39        (BuildbotTesterQueueView.prototype._testStepFailureDescription):
     40        (BuildbotTesterQueueView.prototype._testStepFailureDescriptionWithCount):
     41        (BuildbotTesterQueueView.prototype._presentPopoverForGenericTestFailures):
     42        (BuildbotTesterQueueView.prototype.update.appendBuilderQueueStatus): Deleted.
     43        (BuildbotTesterQueueView.prototype.update): Deleted.
     44        (BuildbotTesterQueueView.prototype._presentPopoverForMultipleFailureKinds): Deleted.
     45        Updated for the new data structures. One behavior change is that we now display individual
     46        counts when multiple test kinds fail, e.g. "1 javascript test failure, 83+ layout
     47        test failures, 3 platform api test failures".
     48
    1492015-03-03  Matthew Mirman  <mmirman@apple.com>
    250
Note: See TracChangeset for help on using the changeset viewer.