Changeset 196794 in webkit


Ignore:
Timestamp:
Feb 18, 2016 8:18:27 PM (8 years ago)
Author:
rniwa@webkit.org
Message:

Perf dashboard should let user cancel pending A/B testing and hide failed ones
​https://bugs.webkit.org/show_bug.cgi?id=154433

Reviewed by Chris Dumez.

Added a button to hide a test group in the details view (the bottom table) in the analysis task page, and
"Show hidden tests" link to show the hidden test groups on demand. When a test group is hidden, all pending
requests in the group will also be canceled since a common scenario of using this feature is that the user
had triggered an useless A/B testing; e.g. all builds will fail, wrong, etc... We can revisit and add the
capability to just cancel the pending requests and leaving the group visible later if necessary.

Run ALTER TYPE build_request_status_type ADD VALUE 'canceled'; to add the new type.

  • init-database.sql: Added testgroup_hidden column to analysis_test_groups table and added 'canceled'

as a value to build_request_status_type table.

  • public/api/test-groups.php:

(format_test_group): Added 'hidden' field in the JSON result.

  • public/privileged-api/update-test-group.php:

(main): Added the support for updating testgroup_hidden column. When this column is set to true, also
cancel all pending build requests (by setting its request_status to 'canceled' which will be ignore by
the syncing script).

  • public/v3/components/test-group-results-table.js:

(TestGroupResultsTable.prototype.setTestGroup): Reset _renderedTestGroup here so that the next call to
render() will update the table; e.g. when build requests' status change from 'Pending' to 'Canceled'.

  • public/v3/models/build-request.js:

(BuildRequest.prototype.hasCompleted): A build request is considered complete/finished if it's canceled.
(BuildRequest.prototype.hasPending): Added.
(BuildRequest.prototype.statusLabel): Handle 'canceled' status.

  • public/v3/models/test-group.js:

(TestGroup):
(TestGroup.prototype.updateSingleton): Added to update 'hidden' field.
(TestGroup.prototype.isHidden): Added.
(TestGroup.prototype.hasPending): Added.
(TestGroup.prototype.hasPending): Added.
(TestGroup.prototype.updateHiddenFlag): Added. Uses the privileged API to update testgroup_hidden column.
The JSON API also updates the status of the 'pending' build requests in the group to 'canceled'.

  • public/v3/pages/analysis-task-page.js:

(AnalysisTaskPage): Added _showHiddenTestGroups and _filteredTestGroups as instance variables.
(AnalysisTaskPage.prototype._didFetchTestGroups):
(AnalysisTaskPage.prototype._showAllTestGroups): Added.
(AnalysisTaskPage.prototype._didUpdateTestGroupHiddenState): Extracted from _didFetchTestGroups.
(AnalysisTaskPage.prototype._renderTestGroupList): Use the filtered list of test groups to show the list
of test groups. When all test groups are shown, we would first show the hidden ones after the regular ones.
(AnalysisTaskPage.prototype._createTestGroupListItem): Extracted from _renderTestGroupList.
(AnalysisTaskPage.prototype._renderTestGroupDetails): Update the text inside the button to hide the test
group. Also show a warning text that the pending requests will be canceled if there are any.
(AnalysisTaskPage.prototype._hideCurrentTestGroup): Added.
(AnalysisTaskPage.cssTemplate): Updated the style.

Location:
trunk/Websites/perf.webkit.org
Files:
8 edited

Legend:

Unmodified
Added
Removed

  • trunk/Websites/perf.webkit.org/ChangeLog

    r196792 r196794  
     12016-02-18  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        Perf dashboard should let user cancel pending A/B testing and hide failed ones
     4        https://bugs.webkit.org/show_bug.cgi?id=154433
     5
     6        Reviewed by Chris Dumez.
     7
     8        Added a button to hide a test group in the details view (the bottom table) in the analysis task page, and
     9        "Show hidden tests" link to show the hidden test groups on demand. When a test group is hidden, all pending
     10        requests in the group will also be canceled since a common scenario of using this feature is that the user
     11        had triggered an useless A/B testing; e.g. all builds will fail, wrong, etc... We can revisit and add the
     12        capability to just cancel the pending requests and leaving the group visible later if necessary.
     13
     14        Run `ALTER TYPE build_request_status_type ADD VALUE 'canceled';` to add the new type.
     15
     16        * init-database.sql: Added testgroup_hidden column to analysis_test_groups table and added 'canceled'
     17        as a value to build_request_status_type table.
     18        * public/api/test-groups.php:
     19        (format_test_group): Added 'hidden' field in the JSON result.
     20        * public/privileged-api/update-test-group.php:
     21        (main): Added the support for updating testgroup_hidden column. When this column is set to true, also
     22        cancel all pending build requests (by setting its request_status to 'canceled' which will be ignore by
     23        the syncing script).
     24        * public/v3/components/test-group-results-table.js:
     25        (TestGroupResultsTable.prototype.setTestGroup): Reset _renderedTestGroup here so that the next call to
     26        render() will update the table; e.g. when build requests' status change from 'Pending' to 'Canceled'.
     27        * public/v3/models/build-request.js:
     28        (BuildRequest.prototype.hasCompleted): A build request is considered complete/finished if it's canceled.
     29        (BuildRequest.prototype.hasPending): Added.
     30        (BuildRequest.prototype.statusLabel): Handle 'canceled' status.
     31        * public/v3/models/test-group.js:
     32        (TestGroup):
     33        (TestGroup.prototype.updateSingleton): Added to update 'hidden' field.
     34        (TestGroup.prototype.isHidden): Added.
     35        (TestGroup.prototype.hasPending): Added.
     36        (TestGroup.prototype.hasPending): Added.
     37        (TestGroup.prototype.updateHiddenFlag): Added. Uses the privileged API to update testgroup_hidden column.
     38        The JSON API also updates the status of the 'pending' build requests in the group to 'canceled'.
     39        * public/v3/pages/analysis-task-page.js:
     40        (AnalysisTaskPage): Added _showHiddenTestGroups and _filteredTestGroups as instance variables.
     41        (AnalysisTaskPage.prototype._didFetchTestGroups):
     42        (AnalysisTaskPage.prototype._showAllTestGroups): Added.
     43        (AnalysisTaskPage.prototype._didUpdateTestGroupHiddenState): Extracted from _didFetchTestGroups.
     44        (AnalysisTaskPage.prototype._renderTestGroupList): Use the filtered list of test groups to show the list
     45        of test groups. When all test groups are shown, we would first show the hidden ones after the regular ones.
     46        (AnalysisTaskPage.prototype._createTestGroupListItem): Extracted from _renderTestGroupList.
     47        (AnalysisTaskPage.prototype._renderTestGroupDetails): Update the text inside the button to hide the test
     48        group. Also show a warning text that the pending requests will be canceled if there are any.
     49        (AnalysisTaskPage.prototype._hideCurrentTestGroup): Added.
     50        (AnalysisTaskPage.cssTemplate): Updated the style.
     51
    1522016-02-18  Ryosuke Niwa  <rniwa@webkit.org>
    253

  • trunk/Websites/perf.webkit.org/init-database.sql

    r194611 r196794  
    226226CREATE TABLE analysis_test_groups (
    227227    testgroup_id serial PRIMARY KEY,
    228     testgroup_task integer REFERENCES analysis_tasks NOT NULL,
     228    testgroup_task integer NOT NULL REFERENCES analysis_tasks ON DELETE CASCADE,
    229229    testgroup_name varchar(256),
    230230    testgroup_author varchar(256),
    231231    testgroup_created_at timestamp NOT NULL DEFAULT (CURRENT_TIMESTAMP AT TIME ZONE 'UTC'),
     232    testgroup_hidden boolean NOT NULL DEFAULT FALSE,
    232233    CONSTRAINT testgroup_name_must_be_unique_for_each_task UNIQUE(testgroup_task, testgroup_name));
    233234CREATE INDEX testgroup_task_index ON analysis_test_groups(testgroup_task);
    … …  
    240241    root_commit integer REFERENCES commits NOT NULL);
    241242
    242 CREATE TYPE build_request_status_type as ENUM ('pending', 'scheduled', 'running', 'failed', 'completed');
     243CREATE TYPE build_request_status_type as ENUM ('pending', 'scheduled', 'running', 'failed', 'completed', 'canceled');
    243244CREATE TABLE build_requests (
    244245    request_id serial PRIMARY KEY,
    … …  
    246247    request_platform integer REFERENCES platforms NOT NULL,
    247248    request_test integer REFERENCES tests NOT NULL,
    248     request_group integer REFERENCES analysis_test_groups NOT NULL,
     249    request_group integer NOT NULL REFERENCES analysis_test_groups ON DELETE CASCADE,
    249250    request_order integer NOT NULL,
    250251    request_root_set integer REFERENCES root_sets NOT NULL,

  • trunk/Websites/perf.webkit.org/public/api/test-groups.php

    r185852 r196794  
    6666        'author' => $group_row['testgroup_author'],
    6767        'createdAt' => strtotime($group_row['testgroup_created_at']) * 1000,
     68        'hidden' => Database::is_true($group_row['testgroup_hidden']),
    6869        'buildRequests' => array(),
    6970        'rootSets' => array(),

  • trunk/Websites/perf.webkit.org/public/privileged-api/update-test-group.php

    r196521 r196794  
    1515        $values['name'] = $data['name'];
    1616
     17    if (array_key_exists('hidden', $data))
     18        $values['hidden'] = Database::to_database_boolean($data['hidden']);
     19
    1720    if (!$values)
    1821        exit_with_error('NothingToUpdate');
    … …  
    2629    }
    2730
     31    if (array_get($data, 'hidden')) {
     32        $db->query_and_get_affected_rows('UPDATE build_requests SET request_status = $1
     33            WHERE request_group = $2 AND request_status = $3', array('canceled', $test_group_id, 'pending'));
     34    }
     35
    2836    $db->commit_transaction();
    2937

  • trunk/Websites/perf.webkit.org/public/v3/components/test-group-results-table.js

    r196787 r196794  
    99
    1010    didUpdateResults() { this._renderedTestGroup = null; }
    11     setTestGroup(testGroup) { this._testGroup = testGroup; }
     11    setTestGroup(testGroup)
     12    {
     13        this._testGroup = testGroup;
     14        this._renderedTestGroup = null;
     15    }
    1216
    1317    heading()

  • trunk/Websites/perf.webkit.org/public/v3/models/build-request.js

    r196521 r196794  
    3131    rootSet() { return this._rootSet; }
    3232
    33     hasCompleted() { return this._status == 'failed' || this._status == 'completed'; }
     33    hasCompleted() { return this._status == 'failed' || this._status == 'completed' || this._status == 'canceled'; }
    3434    hasStarted() { return this._status != 'pending'; }
     35    hasPending() { return this._status == 'pending'; }
    3536    statusLabel()
    3637    {
    … …  
    4647        case 'completed':
    4748            return 'Completed';
     49        case 'canceled':
     50            return 'Canceled';
    4851        }
    4952    }

  • trunk/Websites/perf.webkit.org/public/v3/models/test-group.js

    r196521 r196794  
    88        this._authorName = object.author;
    99        this._createdAt = new Date(object.createdAt);
     10        this._isHidden = object.hidden;
    1011        this._buildRequests = [];
    1112        this._requestsAreInOrder = false;
    … …  
    1819    }
    1920
     21    updateSingleton(object)
     22    {
     23        super.updateSingleton(object);
     24
     25        console.assert(this._taskId == object.task);
     26        console.assert(+this._createdAt == +object.createdAt);
     27        console.assert(this._platform == object.platform);
     28
     29        this._isHidden = object.hidden;
     30    }
     31
    2032    createdAt() { return this._createdAt; }
     33    isHidden() { return this._isHidden; }
    2134    buildRequests() { return this._buildRequests; }
    2235    addBuildRequest(request)
    … …  
    97110    }
    98111
     112    hasPending()
     113    {
     114        return this._buildRequests.some(function (request) { return request.hasPending(); });
     115    }
     116
    99117    compareTestResults(rootSetA, rootSetB)
    100118    {
    … …  
    166184    }
    167185
     186    updateHiddenFlag(hidden)
     187    {
     188        var self = this;
     189        var id = this.id();
     190        return PrivilegedAPI.sendRequest('update-test-group', {
     191            group: id,
     192            hidden: !!hidden,
     193        }).then(function (data) {
     194            return TestGroup.cachedFetch(`../api/test-groups/${id}`, {}, true)
     195                .then(TestGroup._createModelsFromFetchedTestGroups.bind(TestGroup));
     196        });
     197    }
     198
    168199    static createAndRefetchTestGroups(task, name, repetitionCount, rootSets)
    169200    {

  • trunk/Websites/perf.webkit.org/public/v3/pages/analysis-task-page.js

    r196768 r196794  
    4545        this._errorMessage = null;
    4646        this._currentTestGroup = null;
     47        this._filteredTestGroups = null;
     48        this._showHiddenTestGroups = false;
    4749
    4850        this._chartPane = this.content().querySelector('analysis-task-chart-pane').component();
    … …  
    7072        this._newTestGroupFormForViewer.setStartCallback(this._createNewTestGroupFromViewer.bind(this));
    7173
    72         this._retryForm = this.content().querySelector('.test-group-retry-form').firstChild.component();
     74        this._retryForm = this.content().querySelector('.test-group-retry-form test-group-form').component();
    7375        this._retryForm.setStartCallback(this._retryCurrentTestGroup.bind(this));
     76        this._hideButton = this.content().querySelector('.test-group-hide-button');
     77        this._hideButton.onclick = this._hideCurrentTestGroup.bind(this);
    7478    }
    7579
    … …  
    158162    {
    159163        this._testGroups = testGroups.sort(function (a, b) { return +a.createdAt() - b.createdAt(); });
    160         this._currentTestGroup = testGroups.length ? testGroups[testGroups.length - 1] : null;
    161 
    162         this._analysisResultsViewer.setTestGroups(testGroups);
    163         this._testGroupResultsTable.setTestGroup(this._currentTestGroup);
     164        this._didUpdateTestGroupHiddenState();
    164165        this._assignTestResultsIfPossible();
    165166        this.render();
     167    }
     168
     169    _showAllTestGroups()
     170    {
     171        this._showHiddenTestGroups = true;
     172        this._didUpdateTestGroupHiddenState();
     173        this.render();
     174    }
     175
     176    _didUpdateTestGroupHiddenState()
     177    {
     178        this._renderedCurrentTestGroup = null;
     179        this._renderedTestGroups = null;
     180        if (!this._showHiddenTestGroups)
     181            this._filteredTestGroups = this._testGroups.filter(function (group) { return !group.isHidden(); });
     182        else
     183            this._filteredTestGroups = this._testGroups;
     184        this._currentTestGroup = this._filteredTestGroups ? this._filteredTestGroups[this._filteredTestGroups.length - 1] : null;
     185        this._analysisResultsViewer.setTestGroups(this._filteredTestGroups);
     186        this._testGroupResultsTable.setTestGroup(this._currentTestGroup);
    166187    }
    167188
    … …  
    278299            this._testGroupLabelMap.clear();
    279300
    280             var self = this;
    281             var updateTestGroupName = this._updateTestGroupName.bind(this);
    282             var showTestGroup = this._showTestGroup.bind(this);
    283 
    284             this.renderReplace(this.content().querySelector('.test-group-list'),
    285                 this._testGroups.map(function (group) {
    286                     var text = new EditableText(group.label());
    287                     text.setStartedEditingCallback(function () { return text.render(); });
    288                     text.setUpdateCallback(function () { return updateTestGroupName(group); });
    289 
    290                     self._testGroupLabelMap.set(group, text);
    291                     return element('li', {class: 'test-group-list-' + group.id()},
    292                         link(text, group.label(), function () { showTestGroup(group); }));
    293                 }).reverse());
     301            var unhiddenTestGroups = this._filteredTestGroups.filter(function (group) { return !group.isHidden(); });
     302            var hiddenTestGroups = this._filteredTestGroups.filter(function (group) { return group.isHidden(); });
     303
     304            var listItems = [];
     305            for (var group of hiddenTestGroups)
     306                listItems.unshift(this._createTestGroupListItem(group));
     307            for (var group of unhiddenTestGroups)
     308                listItems.unshift(this._createTestGroupListItem(group));
     309
     310            if (this._testGroups.length != this._filteredTestGroups.length) {
     311                listItems.push(element('li', {class: 'test-group-list-show-all'},
     312                    link('Show hidden tests', this._showAllTestGroups.bind(this))));
     313            }
     314
     315            this.renderReplace(this.content().querySelector('.test-group-list'), listItems);
    294316
    295317            this._renderedCurrentTestGroup = null;
    … …  
    297319
    298320        if (this._testGroups) {
    299             for (var testGroup of this._testGroups) {
     321            for (var testGroup of this._filteredTestGroups) {
    300322                var label = this._testGroupLabelMap.get(testGroup);
    301323                label.setText(testGroup.label());
    … …  
    303325            }
    304326        }
     327    }
     328
     329    _createTestGroupListItem(group)
     330    {
     331        var text = new EditableText(group.label());
     332        text.setStartedEditingCallback(function () { return text.render(); });
     333        text.setUpdateCallback(this._updateTestGroupName.bind(this, group));
     334
     335        this._testGroupLabelMap.set(group, text);
     336        return ComponentBase.createElement('li', {class: 'test-group-list-' + group.id()},
     337            ComponentBase.createLink(text, group.label(), this._showTestGroup.bind(this, group)));
    305338    }
    306339
    … …  
    333366            this._retryForm.element().style.display = this._currentTestGroup ? null : 'none';
    334367
     368            this.content().querySelector('.test-group-hide-button').textContent
     369                = this._currentTestGroup && this._currentTestGroup.isHidden() ? 'Unhide' : 'Hide';
     370
     371            this.content().querySelector('.pending-request-cancel-warning').style.display
     372                = this._currentTestGroup && this._currentTestGroup.hasPending() ? null : 'none';
     373
    335374            this._renderedCurrentTestGroup = this._currentTestGroup;
    336375        }
    … …  
    374413        }, function (error) {
    375414            self.render();
    376             alert('Failed to update the name: ' + error);
     415            alert('Failed to hide the test name: ' + error);
     416        });
     417    }
     418
     419    _hideCurrentTestGroup()
     420    {
     421        var self = this;
     422        console.assert(this._currentTestGroup);
     423        return this._currentTestGroup.updateHiddenFlag(!this._currentTestGroup.isHidden()).then(function () {
     424            self._didUpdateTestGroupHiddenState();
     425            self.render();
     426        }, function (error) {
     427            self._mayHaveMutatedTestGroupHiddenState();
     428            self.render();
     429            alert('Failed to update the group: ' + error);
    377430        });
    378431    }
    … …  
    570623                        <test-group-results-table></test-group-results-table>
    571624                        <div class="test-group-retry-form"><test-group-form></test-group-form></div>
     625                        <button class="test-group-hide-button">Hide</button>
     626                        <span class="pending-request-cancel-warning">(cancels pending requests)</span>
    572627                    </div>
    573628                </section>
    … …  
    688743            .test-group-retry-form {
    689744                padding: 0;
     745                margin: 0.5rem;
     746            }
     747
     748            .test-group-hide-button {
    690749                margin: 0.5rem;
    691750            }
    … …  
    698757                border-right: solid 1px #ccc;
    699758                white-space: nowrap;
     759                min-width: 8rem;
    700760            }
    701761
    … …  
    708768            .test-group-list > li {
    709769                display: block;
     770                font-size: 0.9rem;
    710771            }
    711772
    … …  
    714775                color: inherit;
    715776                text-decoration: none;
    716                 font-size: 0.9rem;
    717777                margin: 0;
    718778                padding: 0.2rem;
    719779            }
     780           
     781            .test-group-list > li.test-group-list-show-all {
     782                font-size: 0.8rem;
     783                margin-top: 0.5rem;
     784                padding-right: 1rem;
     785                text-align: center;
     786                color: #999;
     787            }
     788
     789            .test-group-list > li.test-group-list-show-all:not(.selected) a:hover {
     790                background: inherit;
     791            }
    720792
    721793            .test-group-list > li.selected > a {
Note: See TracChangeset for help on using the changeset viewer.