Changeset 199123 in webkit


Ignore:
Timestamp:
Apr 6, 2016 4:19:29 PM (8 years ago)
Author:
rniwa@webkit.org
Message:

New buildbot syncing scripts that supports multiple builders and slaves
​https://bugs.webkit.org/show_bug.cgi?id=156269

Reviewed by Chris Dumez.

Add sync-buildbot.js that supports scheduling A/B testing jobs on multiple builders and slaves.
The old python script (sync-with-buildbot.py) could only support a single builder and slave
for each platform, test pair.

The main logic is implemented in BuildbotTriggerable.syncOnce. Various helper methods are added
throughout the codebase and tests have been refactored.

BuildbotSyncer has been updated to support multiple platform, test pairs. It's now responsible
for syncing everything on each builder (on a buildbot).

Added more unit tests for BuildbotSyncer and server tests for BuildbotTriggerable, and refactored
test helpers and mocks as needed.

  • public/v3/models/build-request.js:

(BuildRequest.prototype.status): Added.
(BuildRequest.prototype.isScheduled): Added.

  • public/v3/models/metric.js:

(Metric.prototype.fullName): Added.

  • public/v3/models/platform.js:

(Platform): Added the map based on platform name.
(Platform.findByName): Added.

  • public/v3/models/test.js:

(Test.topLevelTests):
(Test.findByPath): Added. Finds a test based on an array of test names; e.g. ['A', 'B'] would
find the test whose name is "B" which has a parent test named "A".
(Test.prototype.fullName): Added.

  • server-tests/api-build-requests-tests.js:

(addMockData): Moved to resources/mock-data.js.
(addAnotherMockTestGroup): Ditto.

  • server-tests/resources/mock-data.js: Added.

(MockData.resetV3Models): Added.
(MockData.addMockData): Moved from api-build-requests-tests.js.
(MockData.addAnotherMockTestGroup): Ditto.
(MockData.mockTestSyncConfigWithSingleBuilder): Added.
(MockData.mockTestSyncConfigWithTwoBuilders): Added.
(MockData.pendingBuild): Added.
(MockData.runningBuild): Added.
(MockData.finishedBuild): Added.

  • server-tests/resources/test-server.js:

(TestServer):
(TestServer.prototype.remoteAPI):
(TestServer.prototype._ensureTestDatabase): Don't fail even if the test database doesn't exit.
(TestServer.prototype._startApache): Create a RemoteAPI instance to access the test sever.
(TestServer.prototype._waitForPid): Increase the timeout.
(TestServer.prototype.inject): Replace global.RemoteAPI during the test and restore it afterwards.

  • server-tests/tools-buildbot-triggerable-tests.js: Added. Tests BuildbotTriggerable.syncOnce.

(MockLogger): Added.
(MockLogger.prototype.log): Added.
(MockLogger.prototype.error): Added.

  • tools/detect-changes.js:

(parseArgument): Moved to js/parse-arguments.js.

  • tools/js/buildbot-syncer.js:

(BuildbotBuildEntry):
(BuildbotBuildEntry.prototype.syncer): Added.
(BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): Added. Returns a new status
for a build request (of the matching build request ID) if it needs to be updated in the server.
(BuildbotSyncer): This class
(BuildbotSyncer.prototype.addTestConfiguration): Added.
(BuildbotSyncer.prototype.testConfigurations): Returns the list of test configurations.
(BuildbotSyncer.prototype.matchesConfiguration): Returns true iff the request can be scheduled on
this builder.
(BuildbotSyncer.prototype.scheduleRequest): Added. Schedules a new job on buildbot for a request.
(BuildbotSyncer.prototype.scheduleFirstRequestInGroupIfAvailable): Added. Schedules a new job for
the specified build request on the first slave that's available.
(BuildbotSyncer.prototype.pullBuildbot): Return a list of BuildbotBuildEntry instead of an object.
Also store it on an instance variable so that scheduleFirstRequestInGroupIfAvailable could use it.
(BuildbotSyncer.prototype._pullRecentBuilds):
(BuildbotSyncer.prototype.pathForPendingBuildsJSON): Renamed from urlForPendingBuildsJSON and now
only returns the path instead of the full URL since RemoteAPI takes a path, not full URL.
(BuildbotSyncer.prototype.pathForBuildJSON): Ditto from pathForBuildJSON.
(BuildbotSyncer.prototype.pathForForceBuild): Added.
(BuildbotSyncer.prototype.url): Use RemoteAPI's url method instead of manually constructing URL.
(BuildbotSyncer.prototype.urlForBuildNumber): Ditto.
(BuildbotSyncer.prototype._propertiesForBuildRequest): Now that each syncer can have multiple test
configurations associated with it, find the one matching for this request.
(BuildbotSyncer._loadConfig): Create a syncer per builder and add all test configurations to it.
(BuildbotSyncer._validateAndMergeConfig): Added the support for 'SlaveList', which is a list of
slave names present on this builder.

  • tools/js/buildbot-triggerable.js: Added.

(BuildbotTriggerable): Added.
(BuildbotTriggerable.prototype.name): Added.
(BuildbotTriggerable.prototype.syncOnce): Added. The main logic for the syncing script. It pulls
existing build requests from the perf dashboard, pulls buildbot for pending and running/completed
builds on each builder (represented by each syncer), schedules build requests on buildbot if there
is any builder/slave available, and updates the status of build requests in the database.
(BuildbotTriggerable.prototype._validateRequests): Added.
(BuildbotTriggerable.prototype._pullBuildbotOnAllSyncers): Added.
(BuildbotTriggerable.prototype._scheduleNextRequestInGroupIfSlaveIsAvailable): Added.
(BuildbotTriggerable._testGroupMapForBuildRequests): Added.

  • tools/js/database.js:
  • tools/js/parse-arguments.js: Added. Extracted out of tools/detect-changes.js.

(parseArguments):

  • tools/js/remote.js:

(RemoteAPI): Now optionally takes the server configuration.
(RemoteAPI.prototype.url): Added.
(RemoteAPI.prototype.getJSON): Removed the code for specifying request content.
(RemoteAPI.prototype.getJSONWithStatus): Ditto.
(RemoteAPI.prototype.postJSON): Added.
(RemoteAPI.prototype.postFormUrlencodedData): Added.
(RemoteAPI.prototype.sendHttpRequest): Fixed the code to specify auth.

  • tools/js/v3-models.js: Don't include RemoteAPI here as they require a configuration for each host.
  • tools/sync-buildbot.js: Added.

(main): Added. Parse the arguments and start the loop.
(syncLoop): Added.

  • unit-tests/buildbot-syncer-tests.js: Added tests for pullBuildbot, scheduleRequest, as well as

scheduleFirstRequestInGroupIfAvailable. Refactored helper functions as needed.
(sampleiOSConfig):
(smallConfiguration): Added.
(smallPendingBuild): Added.
(smallInProgressBuild): Added.
(smallFinishedBuild): Added.
(createSampleBuildRequest): Create a unique build request for each platform.
(samplePendingBuild): Optionally specify build time and slave name.
(sampleInProgressBuild): Optionally specify slave name.
(sampleFinishedBuild): Ditto.

  • unit-tests/resources/mock-remote-api.js:

(assert.notReached.assert.notReached):
(MockRemoteAPI.url): Added.
(MockRemoteAPI.postFormUrlencodedData): Added.
(MockRemoteAPI._addRequest): Extracted from getJSONWithStatus.
(MockRemoteAPI.waitForRequest): Extracted from inject. For tools-buildbot-triggerable-tests.js, we
need to instantiate a RemoteAPI for buildbot without replacing global.RemoteAPI.
(MockRemoteAPI.inject):
(MockRemoteAPI.reset): Added.

Location:
trunk/Websites/perf.webkit.org
Files:
5 added
15 edited

Legend:

Unmodified
Added
Removed

  • trunk/Websites/perf.webkit.org/ChangeLog

    r198923 r199123  
     12016-04-05  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        New buildbot syncing scripts that supports multiple builders and slaves
     4        https://bugs.webkit.org/show_bug.cgi?id=156269
     5
     6        Reviewed by Chris Dumez.
     7
     8        Add sync-buildbot.js that supports scheduling A/B testing jobs on multiple builders and slaves.
     9        The old python script (sync-with-buildbot.py) could only support a single builder and slave
     10        for each platform, test pair.
     11
     12        The main logic is implemented in BuildbotTriggerable.syncOnce. Various helper methods are added
     13        throughout the codebase and tests have been refactored.
     14
     15        BuildbotSyncer has been updated to support multiple platform, test pairs. It's now responsible
     16        for syncing everything on each builder (on a buildbot).
     17
     18        Added more unit tests for BuildbotSyncer and server tests for BuildbotTriggerable, and refactored
     19        test helpers and mocks as needed.
     20
     21        * public/v3/models/build-request.js:
     22        (BuildRequest.prototype.status): Added.
     23        (BuildRequest.prototype.isScheduled): Added.
     24        * public/v3/models/metric.js:
     25        (Metric.prototype.fullName): Added.
     26        * public/v3/models/platform.js:
     27        (Platform): Added the map based on platform name.
     28        (Platform.findByName): Added.
     29        * public/v3/models/test.js:
     30        (Test.topLevelTests):
     31        (Test.findByPath): Added. Finds a test based on an array of test names; e.g. ['A', 'B'] would
     32        find the test whose name is "B" which has a parent test named "A".
     33        (Test.prototype.fullName): Added.
     34        * server-tests/api-build-requests-tests.js:
     35        (addMockData): Moved to resources/mock-data.js.
     36        (addAnotherMockTestGroup): Ditto.
     37        * server-tests/resources/mock-data.js: Added.
     38        (MockData.resetV3Models): Added.
     39        (MockData.addMockData): Moved from api-build-requests-tests.js.
     40        (MockData.addAnotherMockTestGroup): Ditto.
     41        (MockData.mockTestSyncConfigWithSingleBuilder): Added.
     42        (MockData.mockTestSyncConfigWithTwoBuilders): Added.
     43        (MockData.pendingBuild): Added.
     44        (MockData.runningBuild): Added.
     45        (MockData.finishedBuild): Added.
     46        * server-tests/resources/test-server.js:
     47        (TestServer):
     48        (TestServer.prototype.remoteAPI):
     49        (TestServer.prototype._ensureTestDatabase): Don't fail even if the test database doesn't exit.
     50        (TestServer.prototype._startApache): Create a RemoteAPI instance to access the test sever.
     51        (TestServer.prototype._waitForPid): Increase the timeout.
     52        (TestServer.prototype.inject): Replace global.RemoteAPI during the test and restore it afterwards.
     53        * server-tests/tools-buildbot-triggerable-tests.js: Added. Tests BuildbotTriggerable.syncOnce.
     54        (MockLogger): Added.
     55        (MockLogger.prototype.log): Added.
     56        (MockLogger.prototype.error): Added.
     57        * tools/detect-changes.js:
     58        (parseArgument): Moved to js/parse-arguments.js.
     59        * tools/js/buildbot-syncer.js:
     60        (BuildbotBuildEntry):
     61        (BuildbotBuildEntry.prototype.syncer): Added.
     62        (BuildbotBuildEntry.prototype.buildRequestStatusIfUpdateIsNeeded): Added. Returns a new status
     63        for a build request (of the matching build request ID) if it needs to be updated in the server.
     64        (BuildbotSyncer): This class
     65        (BuildbotSyncer.prototype.addTestConfiguration): Added.
     66        (BuildbotSyncer.prototype.testConfigurations): Returns the list of test configurations.
     67        (BuildbotSyncer.prototype.matchesConfiguration): Returns true iff the request can be scheduled on
     68        this builder.
     69        (BuildbotSyncer.prototype.scheduleRequest): Added. Schedules a new job on buildbot for a request.
     70        (BuildbotSyncer.prototype.scheduleFirstRequestInGroupIfAvailable): Added. Schedules a new job for
     71        the specified build request on the first slave that's available.
     72        (BuildbotSyncer.prototype.pullBuildbot): Return a list of BuildbotBuildEntry instead of an object.
     73        Also store it on an instance variable so that scheduleFirstRequestInGroupIfAvailable could use it.
     74        (BuildbotSyncer.prototype._pullRecentBuilds):
     75        (BuildbotSyncer.prototype.pathForPendingBuildsJSON): Renamed from urlForPendingBuildsJSON and now
     76        only returns the path instead of the full URL since RemoteAPI takes a path, not full URL.
     77        (BuildbotSyncer.prototype.pathForBuildJSON): Ditto from pathForBuildJSON.
     78        (BuildbotSyncer.prototype.pathForForceBuild): Added.
     79        (BuildbotSyncer.prototype.url): Use RemoteAPI's url method instead of manually constructing URL.
     80        (BuildbotSyncer.prototype.urlForBuildNumber): Ditto.
     81        (BuildbotSyncer.prototype._propertiesForBuildRequest): Now that each syncer can have multiple test
     82        configurations associated with it, find the one matching for this request.
     83        (BuildbotSyncer._loadConfig): Create a syncer per builder and add all test configurations to it.
     84        (BuildbotSyncer._validateAndMergeConfig): Added the support for 'SlaveList', which is a list of
     85        slave names present on this builder.
     86        * tools/js/buildbot-triggerable.js: Added.
     87        (BuildbotTriggerable): Added.
     88        (BuildbotTriggerable.prototype.name): Added.
     89        (BuildbotTriggerable.prototype.syncOnce): Added. The main logic for the syncing script. It pulls
     90        existing build requests from the perf dashboard, pulls buildbot for pending and running/completed
     91        builds on each builder (represented by each syncer), schedules build requests on buildbot if there
     92        is any builder/slave available, and updates the status of build requests in the database.
     93        (BuildbotTriggerable.prototype._validateRequests): Added.
     94        (BuildbotTriggerable.prototype._pullBuildbotOnAllSyncers): Added.
     95        (BuildbotTriggerable.prototype._scheduleNextRequestInGroupIfSlaveIsAvailable): Added.
     96        (BuildbotTriggerable._testGroupMapForBuildRequests): Added.
     97        * tools/js/database.js:
     98        * tools/js/parse-arguments.js: Added. Extracted out of tools/detect-changes.js.
     99        (parseArguments):
     100        * tools/js/remote.js:
     101        (RemoteAPI): Now optionally takes the server configuration.
     102        (RemoteAPI.prototype.url): Added.
     103        (RemoteAPI.prototype.getJSON): Removed the code for specifying request content.
     104        (RemoteAPI.prototype.getJSONWithStatus): Ditto.
     105        (RemoteAPI.prototype.postJSON): Added.
     106        (RemoteAPI.prototype.postFormUrlencodedData): Added.
     107        (RemoteAPI.prototype.sendHttpRequest): Fixed the code to specify auth.
     108        * tools/js/v3-models.js: Don't include RemoteAPI here as they require a configuration for each host.
     109        * tools/sync-buildbot.js: Added.
     110        (main): Added. Parse the arguments and start the loop.
     111        (syncLoop): Added.
     112        * unit-tests/buildbot-syncer-tests.js: Added tests for pullBuildbot, scheduleRequest, as well as
     113        scheduleFirstRequestInGroupIfAvailable. Refactored helper functions as needed.
     114        (sampleiOSConfig):
     115        (smallConfiguration): Added.
     116        (smallPendingBuild): Added.
     117        (smallInProgressBuild): Added.
     118        (smallFinishedBuild): Added.
     119        (createSampleBuildRequest): Create a unique build request for each platform.
     120        (samplePendingBuild): Optionally specify build time and slave name.
     121        (sampleInProgressBuild): Optionally specify slave name.
     122        (sampleFinishedBuild): Ditto.
     123        * unit-tests/resources/mock-remote-api.js:
     124        (assert.notReached.assert.notReached):
     125        (MockRemoteAPI.url): Added.
     126        (MockRemoteAPI.postFormUrlencodedData): Added.
     127        (MockRemoteAPI._addRequest): Extracted from getJSONWithStatus.
     128        (MockRemoteAPI.waitForRequest): Extracted from inject. For tools-buildbot-triggerable-tests.js, we
     129        need to instantiate a RemoteAPI for buildbot without replacing global.RemoteAPI.
     130        (MockRemoteAPI.inject):
     131        (MockRemoteAPI.reset): Added.
     132
    11332016-03-30  Ryosuke Niwa  <rniwa@webkit.org>
    2134

  • trunk/Websites/perf.webkit.org/public/v3/models/build-request.js

    r198882 r199123  
    4141    rootSet() { return this._rootSet; }
    4242
     43    status() { return this._status; }
    4344    hasFinished() { return this._status == 'failed' || this._status == 'completed' || this._status == 'canceled'; }
    4445    hasStarted() { return this._status != 'pending'; }
     46    isScheduled() { return this._status == 'scheduled'; }
    4547    isPending() { return this._status == 'pending'; }
    4648    statusLabel()

  • trunk/Websites/perf.webkit.org/public/v3/models/metric.js

    r198479 r199123  
    3434    path() { return this._test.path().concat([this]); }
    3535
    36     fullName()
    37     {
    38         return this._test.path().map(function (test) { return test.label(); }).join(' \u220B ') + ' : ' + this.label();
    39     }
     36    fullName() { return this._test.fullName() + ' : ' + this.label(); }
    4037
    4138    label()

  • trunk/Websites/perf.webkit.org/public/v3/models/platform.js

    r198691 r199123  
    99        this._containingTests = null;
    1010
     11        this.ensureNamedStaticMap('name')[object.name] = this;
     12
    1113        for (var metric of this._metrics)
    1214            metric.addPlatform(this);
     15    }
     16
     17    static findByName(name)
     18    {
     19        var map = this.namedStaticMap('name');
     20        return map ? map[name] : null;
    1321    }
    1422

  • trunk/Websites/perf.webkit.org/public/v3/models/test.js

    r198923 r199123  
    2323    static topLevelTests() { return this.sortByName(this.listForStaticMap('topLevelTests')); }
    2424
     25    static findByPath(path)
     26    {
     27        var matchingTest = null;
     28        var testList = this.topLevelTests();
     29        for (var part of path) {
     30            matchingTest = null;
     31            for (var test of testList) {
     32                if (part == test.name()) {
     33                    matchingTest = test;
     34                    break;
     35                }
     36            }
     37            if (!matchingTest)
     38                return null;
     39            testList = matchingTest.childTests();
     40        }
     41        return matchingTest;
     42    }
     43
    2544    parentTest() { return Test.findById(this._parentId); }
    2645
    … …  
    3554        return path;
    3655    }
     56
     57    fullName() { return this.path().map(function (test) { return test.label(); }).join(' \u220B '); }
    3758
    3859    onlyContainsSingleMetric() { return !this.childTests().length && this._metrics.length == 1; }

  • trunk/Websites/perf.webkit.org/server-tests/api-build-requests-tests.js

    r198882 r199123  
    33let assert = require('assert');
    44
    5 require('../tools/js/v3-models.js');
    6 
     5let MockData = require('./resources/mock-data.js');
    76let TestServer = require('./resources/test-server.js');
    87
    … …  
    1211
    1312    beforeEach(function () {
    14         AnalysisTask._fetchAllPromise = null;
    15         AnalysisTask.clearStaticMap();
    16         BuildRequest.clearStaticMap();
    17         CommitLog.clearStaticMap();
    18         Metric.clearStaticMap();
    19         Platform.clearStaticMap();
    20         Repository.clearStaticMap();
    21         RootSet.clearStaticMap();
    22         Test.clearStaticMap();
    23         TestGroup.clearStaticMap();
    24     })
     13        MockData.resetV3Models();
     14    });
    2515
    2616    it('should return "TriggerableNotFound" when the database is empty', function (done) {
    … …  
    4636    });
    4737
    48     function addMockData(db, statusList)
    49     {
    50         if (!statusList)
    51             statusList = ['pending', 'pending', 'pending', 'pending'];
    52         return Promise.all([
    53             db.insert('build_triggerables', {id: 1, name: 'build-webkit'}),
    54             db.insert('repositories', {id: 9, name: 'OS X'}),
    55             db.insert('repositories', {id: 11, name: 'WebKit'}),
    56             db.insert('commits', {id: 87832, repository: 9, revision: '10.11 15A284'}),
    57             db.insert('commits', {id: 93116, repository: 11, revision: '191622', time: (new Date(1445945816878)).toISOString()}),
    58             db.insert('commits', {id: 96336, repository: 11, revision: '192736', time: (new Date(1448225325650)).toISOString()}),
    59             db.insert('platforms', {id: 65, name: 'some platform'}),
    60             db.insert('tests', {id: 200, name: 'some test'}),
    61             db.insert('test_metrics', {id: 300, test: 200, name: 'some metric'}),
    62             db.insert('test_configurations', {id: 301, metric: 300, platform: 65, type: 'current'}),
    63             db.insert('root_sets', {id: 401}),
    64             db.insert('roots', {set: 401, commit: 87832}),
    65             db.insert('roots', {set: 401, commit: 93116}),
    66             db.insert('root_sets', {id: 402}),
    67             db.insert('roots', {set: 402, commit: 87832}),
    68             db.insert('roots', {set: 402, commit: 96336}),
    69             db.insert('analysis_tasks', {id: 500, platform: 65, metric: 300, name: 'some task'}),
    70             db.insert('analysis_test_groups', {id: 600, task: 500, name: 'some test group'}),
    71             db.insert('build_requests', {id: 700, status: statusList[0], triggerable: 1, platform: 65, test: 200, group: 600, order: 0, root_set: 401}),
    72             db.insert('build_requests', {id: 701, status: statusList[1], triggerable: 1, platform: 65, test: 200, group: 600, order: 1, root_set: 402}),
    73             db.insert('build_requests', {id: 702, status: statusList[2], triggerable: 1, platform: 65, test: 200, group: 600, order: 2, root_set: 401}),
    74             db.insert('build_requests', {id: 703, status: statusList[3], triggerable: 1, platform: 65, test: 200, group: 600, order: 3, root_set: 402}),
    75         ]);
    76     }
    77 
    78     function addAnotherMockTestGroup(db, statusList)
    79     {
    80         if (!statusList)
    81             statusList = ['pending', 'pending', 'pending', 'pending'];
    82         return Promise.all([
    83             db.insert('analysis_test_groups', {id: 599, task: 500, name: 'another test group'}),
    84             db.insert('build_requests', {id: 713, status: statusList[3], triggerable: 1, platform: 65, test: 200, group: 599, order: 3, root_set: 402}),
    85             db.insert('build_requests', {id: 710, status: statusList[0], triggerable: 1, platform: 65, test: 200, group: 599, order: 0, root_set: 401}),
    86             db.insert('build_requests', {id: 712, status: statusList[2], triggerable: 1, platform: 65, test: 200, group: 599, order: 2, root_set: 401}),
    87             db.insert('build_requests', {id: 711, status: statusList[1], triggerable: 1, platform: 65, test: 200, group: 599, order: 1, root_set: 402}),
    88         ]);
    89     }
    90 
    9138    it('should return build requets associated with a given triggerable with appropriate roots and rootSets', function (done) {
    9239        let db = TestServer.database();
    9340        db.connect().then(function () {
    94             return addMockData(db);
     41            return MockData.addMockData(db);
    9542        }).then(function () {
    9643            return TestServer.remoteAPI().getJSONWithStatus('/api/build-requests/build-webkit');
    … …  
    15097        let db = TestServer.database();
    15198        db.connect().then(function () {
    152             return addMockData(db);
     99            return MockData.addMockData(db);
    153100        }).then(function () {
    154101            return TestServer.remoteAPI().getJSONWithStatus('/api/build-requests/build-webkit?useLegacyIdResolution=true');
    … …  
    208155        let db = TestServer.database();
    209156        db.connect().then(function () {
    210             return addMockData(db);
     157            return MockData.addMockData(db);
    211158        }).then(function () {
    212159            return Manifest.fetch();
    … …  
    303250        let db = TestServer.database();
    304251        db.connect().then(function () {
    305             return addMockData(db, ['completed', 'completed', 'completed', 'completed']);
     252            return MockData.addMockData(db, ['completed', 'completed', 'completed', 'completed']);
    306253        }).then(function () {
    307254            return Manifest.fetch();
    … …  
    317264        let db = TestServer.database();
    318265        db.connect().then(function () {
    319             return addMockData(db, ['failed', 'failed', 'canceled', 'canceled']);
     266            return MockData.addMockData(db, ['failed', 'failed', 'canceled', 'canceled']);
    320267        }).then(function () {
    321268            return Manifest.fetch();
    … …  
    331278        let db = TestServer.database();
    332279        db.connect().then(function () {
    333             return addMockData(db, ['completed', 'completed', 'scheduled', 'pending']);
     280            return MockData.addMockData(db, ['completed', 'completed', 'scheduled', 'pending']);
    334281        }).then(function () {
    335282            return Manifest.fetch();
    … …  
    357304        let db = TestServer.database();
    358305        db.connect().then(function () {
    359             return addMockData(db, ['completed', 'completed', 'completed', 'running']);
     306            return MockData.addMockData(db, ['completed', 'completed', 'completed', 'running']);
    360307        }).then(function () {
    361308            return Manifest.fetch();
    … …  
    383330        let db = TestServer.database();
    384331        db.connect().then(function () {
    385             return Promise.all([addMockData(db), addAnotherMockTestGroup(db)])
     332            return Promise.all([MockData.addMockData(db), MockData.addAnotherMockTestGroup(db)])
    386333        }).then(function () {
    387334            return Manifest.fetch();

  • trunk/Websites/perf.webkit.org/server-tests/resources/test-server.js

    r198691 r199123  
    2727        this._databasePort = Config.value('database.port');
    2828        this._database = null;
     29
     30        this._remote = null
    2931    }
    3032
    … …  
    4951    remoteAPI()
    5052    {
    51         assert(this._server);
    52         RemoteAPI.configure(this._server);
    53         return RemoteAPI;
     53        assert(this._remote);
     54        return this._remote;
    5455    }
    5556
    … …  
    113114    _ensureTestDatabase()
    114115    {
    115         this._executePgsqlCommand('dropdb');
     116        try {
     117            this._executePgsqlCommand('dropdb');
     118        } catch (error) { }
    116119        this._executePgsqlCommand('createdb');
    117120        this._executePgsqlCommand('psql', ['--command', `grant all privileges on database "${this._databaseName}" to "${this._databaseUser}";`]);
    … …  
    182185        this._pidWaitStart = Date.now();
    183186        this._pidFile = pidFile;
     187
     188        this._remote = new RemoteAPI(this._server);
     189
    184190        return new Promise(this._waitForPid.bind(this, true));
    185191    }
    … …  
    203209    {
    204210        if (fs.existsSync(this._pidFile) != shouldExist) {
    205             if (Date.now() - this._pidWaitStart > 5000)
     211            if (Date.now() - this._pidWaitStart > 8000)
    206212                reject();
    207213            else
    … …  
    216222        let self = this;
    217223        before(function () {
    218             this.timeout(5000);
     224            this.timeout(10000);
    219225            return self.start();
    220226        });
     227
     228        let originalRemote;
    221229
    222230        beforeEach(function () {
    … …  
    224232            self.initDatabase();
    225233            self.cleanDataDirectory();
     234            originalRemote = global.RemoteAPI;
     235            global.RemoteAPI = self._remote;
    226236        });
    227237
    228238        after(function () {
    229             this.timeout(5000);
     239            this.timeout(10000);
     240            global.RemoteAPI = originalRemote;
    230241            return self.stop();
    231242        });

  • trunk/Websites/perf.webkit.org/tools/detect-changes.js

    r196663 r199123  
    88var Statistics = require('../public/shared/statistics.js');
    99var StatisticsStrategies = require('../public/v2/statistics-strategies.js');
     10var parseArguments = require('./js/parse-arguments.js').parseArguments;
    1011
    1112// FIXME: We shouldn't use a global variable like this.
    … …  
    1314function main(argv)
    1415{
    15     var options = parseArgument(argv, [
     16    var options = parseArguments(argv, [
    1617        {name: '--server-config-json', required: true},
    1718        {name: '--change-detection-config-json', required: true},
    … …  
    2526
    2627    fetchManifestAndAnalyzeData(options['--server-config-json']);
    27 }
    28 
    29 function parseArgument(argv, acceptedOptions) {
    30     var args = argv.slice(2);
    31     var options = {}
    32     for (var i = 0; i < args.length; i += 2) {
    33         var current = args[i];
    34         var next = args[i + 1];
    35         for (var option of acceptedOptions) {
    36             if (current == option['name']) {
    37                 options[option['name']] = next;
    38                 next = null;
    39                 break;
    40             }
    41         }
    42         if (next) {
    43             console.error('Invalid argument:', current);
    44             return null;
    45         }
    46     }
    47     for (var option of acceptedOptions) {
    48         var name = option['name'];
    49         if (option['required'] && !(name in options)) {
    50             console.log('Required argument', name, 'is missing');
    51             return null;
    52         }
    53         var value = options[name] || option['default'];
    54         var converter = option['type'];
    55         options[name] = converter ? converter(value) : value;
    56     }
    57     return options;
    5828}
    5929

  • trunk/Websites/perf.webkit.org/tools/js/buildbot-syncer.js

    r198824 r199123  
    2727    }
    2828
     29    syncer() { return this._syncer; }
    2930    buildNumber() { return this._buildNumber; }
    3031    slaveName() { return this._slaveName; }
    … …  
    3435    hasFinished() { return !this.isPending() && !this.isInProgress(); }
    3536    url() { return this.isPending() ? this._syncer.url() : this._syncer.urlForBuildNumber(this._buildNumber); }
     37
     38    buildRequestStatusIfUpdateIsNeeded(request)
     39    {
     40        assert.equal(request.id(), this._buildRequestId);
     41        if (!request)
     42            return null;
     43        if (this.isPending()) {
     44            if (request.isPending())
     45                return 'scheduled';
     46        } else if (this.isInProgress()) {
     47            if (!request.hasStarted())
     48                return 'running';
     49        } else if (this.hasFinished()) {
     50            if (!request.hasFinished())
     51                return 'failedIfNotCompleted';
     52        }
     53        return null;
     54    }
    3655}
    3756
     57
    3858class BuildbotSyncer {
    3959
    40     constructor(url, object)
    41     {
    42         this._url = url;
     60    constructor(remote, object)
     61    {
     62        this._remote = remote;
     63        this._testConfigurations = [];
    4364        this._builderName = object.builder;
    44         this._platformName = object.platform;
    45         this._testPath = object.test;
    46         this._propertiesTemplate = object.properties;
    4765        this._slavePropertyName = object.slaveArgument;
     66        this._slaveList = object.slaveList;
    4867        this._buildRequestPropertyName = object.buildRequestArgument;
    49     }
    50 
    51     testPath() { return this._testPath }
     68        this._entryList = null;
     69    }
     70
    5271    builderName() { return this._builderName; }
    53     platformName() { return this._platformName; }
     72
     73    addTestConfiguration(test, platform, propertiesTemplate)
     74    {
     75        assert(test instanceof Test);
     76        assert(platform instanceof Platform);
     77        this._testConfigurations.push({test: test, platform: platform, propertiesTemplate: propertiesTemplate});
     78    }
     79    testConfigurations() { return this._testConfigurations; }
     80
     81    matchesConfiguration(request)
     82    {
     83        for (let config of this._testConfigurations) {
     84            if (config.platform == request.platform() && config.test == request.test())
     85                return true;
     86        }
     87        return false;
     88    }
     89
     90    scheduleRequest(newRequest, slaveName)
     91    {
     92        let properties = this._propertiesForBuildRequest(newRequest);
     93
     94        assert.equal(!this._slavePropertyName, !slaveName);
     95        if (this._slavePropertyName)
     96            properties[this._slavePropertyName] = slaveName;
     97
     98        return this._remote.postFormUrlencodedData(this.pathForForceBuild(), properties);
     99    }
     100
     101    scheduleFirstRequestInGroupIfAvailable(newRequest)
     102    {
     103        assert(newRequest instanceof BuildRequest);
     104
     105        if (!this.matchesConfiguration(newRequest))
     106            return null;
     107
     108        let hasPendingBuildsWithoutSlaveNameSpecified = false;
     109        let usedSlaves = new Set;
     110        for (let entry of this._entryList) {
     111            if (entry.isPending()) {
     112                if (!entry.slaveName())
     113                    hasPendingBuildsWithoutSlaveNameSpecified = true;
     114                usedSlaves.add(entry.slaveName());
     115            }
     116        }
     117
     118        if (!this._slaveList || hasPendingBuildsWithoutSlaveNameSpecified) {
     119            if (usedSlaves.size)
     120                return null;
     121            return this.scheduleRequest(newRequest, null);
     122        }
     123
     124        for (let slaveName of this._slaveList) {
     125            if (!usedSlaves.has(slaveName))
     126                return this.scheduleRequest(newRequest, slaveName);
     127        }
     128
     129        return null;
     130    }
    54131
    55132    pullBuildbot(count)
    56133    {
    57134        let self = this;
    58         return RemoteAPI.getJSON(this.urlForPendingBuildsJSON()).then(function (content) {
     135        return this._remote.getJSON(this.pathForPendingBuildsJSON()).then(function (content) {
    59136            let pendingEntries = content.map(function (entry) { return new BuildbotBuildEntry(self, entry); });
    60 
    61137            return self._pullRecentBuilds(count).then(function (entries) {
    62138                let entryByRequest = {};
    … …  
    68144                    entryByRequest[entry.buildRequestId()] = entry;
    69145
    70                 return entryByRequest;
     146                let entryList = [];
     147                for (let id in entryByRequest)
     148                    entryList.push(entryByRequest[id]);
     149
     150                self._entryList = entryList;
     151
     152                return entryList;
    71153            });
    72154        });
    … …  
    83165
    84166        let self = this;
    85         return RemoteAPI.getJSON(this.urlForBuildJSON(selectedBuilds)).then(function (content) {
    86             let entries = [];
     167        return this._remote.getJSON(this.pathForBuildJSON(selectedBuilds)).then(function (content) {
     168            var entries = [];
    87169            for (let index of selectedBuilds) {
    88170                let entry = content[index];
    … …  
    94176    }
    95177
    96     urlForPendingBuildsJSON() { return `${this._url}/json/builders/${this._builderName}/pendingBuilds`; }
    97     urlForBuildJSON(selectedBuilds)
    98     {
    99         return `${this._url}/json/builders/${this._builderName}/builds/?`
     178    pathForPendingBuildsJSON() { return `/json/builders/${this._builderName}/pendingBuilds`; }
     179    pathForBuildJSON(selectedBuilds)
     180    {
     181        return `/json/builders/${this._builderName}/builds/?`
    100182            + selectedBuilds.map(function (number) { return 'select=' + number; }).join('&');
    101183    }
    102 
    103     url() { return `${this._url}/builders/${this._builderName}/`; }
    104     urlForBuildNumber(number) { return `${this._url}/builders/${this._builderName}/builds/${number}`; }
     184    pathForForceBuild() { return `/builders/${this._builderName}/force`; }
     185
     186    url() { return this._remote.url(`/builders/${this._builderName}/`); }
     187    urlForBuildNumber(number) { return this._remote.url(`/builders/${this._builderName}/builds/${number}`); }
    105188
    106189    _propertiesForBuildRequest(buildRequest)
    107190    {
    108         console.assert(buildRequest instanceof BuildRequest);
     191        assert(buildRequest instanceof BuildRequest);
    109192
    110193        let rootSet = buildRequest.rootSet();
    111         console.assert(rootSet instanceof RootSet);
     194        assert(rootSet instanceof RootSet);
    112195
    113196        let repositoryByName = {};
    … …  
    115198            repositoryByName[repository.name()] = repository;
    116199
     200        let propertiesTemplate = null;
     201        for (let config of this._testConfigurations) {
     202            if (config.platform == buildRequest.platform() && config.test == buildRequest.test())
     203                propertiesTemplate = config.propertiesTemplate;
     204        }
     205        assert(propertiesTemplate);
     206
    117207        let properties = {};
    118         for (let key in this._propertiesTemplate) {
    119             let value = this._propertiesTemplate[key];
     208        for (let key in propertiesTemplate) {
     209            let value = propertiesTemplate[key];
    120210            if (typeof(value) != 'object')
    121211                properties[key] = value;
    … …  
    153243    }
    154244
    155     static _loadConfig(url, config)
     245    static _loadConfig(remote, config)
    156246    {
    157247        let shared = config['shared'] || {};
    … …  
    159249        let builders = config['builders'] || {};
    160250
    161         let syncers = [];
     251        let syncerByBuilder = new Map;
    162252        for (let entry of config['configurations']) {
    163253            let newConfig = {};
    … …  
    181271            assert('properties' in newConfig, 'configuration must specify arguments to post on a builder');
    182272            assert('buildRequestArgument' in newConfig, 'configuration must specify buildRequestArgument');
    183             syncers.push(new BuildbotSyncer(url, newConfig));
    184         }
    185 
    186         return syncers;
     273
     274            let test = Test.findByPath(newConfig.test);
     275            assert(test, `${newConfig.test} is not a valid test path`);
     276
     277            let platform = Platform.findByName(newConfig.platform);
     278            assert(platform, `${newConfig.platform} is not a valid platform name`);
     279
     280            let syncer = syncerByBuilder.get(newConfig.builder);
     281            if (!syncer) {
     282                syncer = new BuildbotSyncer(remote, newConfig);
     283                syncerByBuilder.set(newConfig.builder, syncer);
     284            }
     285            syncer.addTestConfiguration(test, platform, newConfig.properties);
     286        }
     287
     288        return Array.from(syncerByBuilder.values());
    187289    }
    188290
    … …  
    203305                config[name] = value.slice();
    204306                break;
     307            case 'slaveList':
     308                assert(value instanceof Array, 'slaveList should be an array');
     309                assert(value.every(function (part) { return typeof part == 'string'; }), 'slaveList should be an array of strings');
     310                config[name] = value;
     311                break;
    205312            case 'type': // fallthrough
    206313            case 'builder': // fallthrough

  • trunk/Websites/perf.webkit.org/tools/js/database.js

    r198691 r199123  
    8686    'build_triggerables': 'triggerable',
    8787    'build_requests': 'request',
     88    'build_slaves': 'slave',
    8889    'builders': 'builder',
    8990    'commits': 'commit',

  • trunk/Websites/perf.webkit.org/tools/js/remote.js

    r198824 r199123  
    44let http = require('http');
    55let https = require('https');
     6let querystring = require('querystring');
    67
    7 let RemoteAPI = new (class RemoteAPI {
    8     constructor()
     8class RemoteAPI {
     9    constructor(server)
    910    {
    10         this._server = {
    11             scheme: 'http',
    12             host: 'localhost',
    13         }
     11        this._server = null;
     12        if (server)
     13            this.configure(server);
     14    }
     15
     16    url(path)
     17    {
     18        let scheme = this._server.scheme;
     19        let port = this._server.port;
     20        let portSuffix = (scheme == 'http' && port == 80) || (scheme == 'https' && port == 443) ? '' : `:${port}`;
     21        if (path.charAt(0) != '/')
     22            path = '/' + path;
     23        return `${scheme}://${this._server.host}portSuffix${path}`;
    1424    }
    1525
    … …  
    2333    }
    2434
    25     getJSON(path, data)
     35    getJSON(path)
    2636    {
    27         let contentType = null;
    28         if (data) {
    29             contentType = 'application/json';
    30             data = JSON.stringify(data);
    31         }
    32         return this.sendHttpRequest(path, 'GET', contentType, data).then(function (result) {
     37        return this.sendHttpRequest(path, 'GET', null, null).then(function (result) {
    3338            return JSON.parse(result.responseText);
    3439        });
    3540    }
    3641
    37     getJSONWithStatus(path, data)
     42    getJSONWithStatus(path)
    3843    {
    39         return this.getJSON(path, data).then(function (result) {
     44        return this.getJSON(path).then(function (result) {
    4045            if (result['status'] != 'OK')
    4146                return Promise.reject(result);
    4247            return result;
     48        });
     49    }
     50
     51    postJSON(path, data)
     52    {
     53        const contentType = 'application/json';
     54        const payload = JSON.stringify(data);
     55        return this.sendHttpRequest(path, 'POST', 'application/json', payload).then(function (result) {
     56            return JSON.parse(result.responseText);
     57        });
     58    }
     59
     60    postFormUrlencodedData(path, data)
     61    {
     62        const contentType = 'application/x-www-form-urlencoded';
     63        const payload = querystring.stringify(data);
     64        return this.sendHttpRequest(path, 'POST', contentType, payload).then(function (result) {
     65            return result.responseText;
    4366        });
    4467    }
    … …  
    5174                hostname: server.host,
    5275                port: server.port || 80,
    53                 auth: server.auth,
     76                auth: server.auth ? server.auth.username + ':' + server.auth.password : null,
    5477                method: method,
    5578                path: path,
    … …  
    7497        });
    7598    }
    76 })
     99};
    77100
    78101if (typeof module != 'undefined')

  • trunk/Websites/perf.webkit.org/tools/js/v3-models.js

    r198824 r199123  
    3030importFromV3('instrumentation.js', 'Instrumentation');
    3131
    32 // RemoteAPI has a different implementation in node since XHR isn't available.
    33 global.RemoteAPI = require('./remote.js').RemoteAPI;
    34 
    3532global.Statistics = require('../../public/shared/statistics.js');

  • trunk/Websites/perf.webkit.org/unit-tests/buildbot-syncer-tests.js

    r198882 r199123  
    3131                'arguments': {'test_name': 'jetstream'}
    3232            },
    33             "dromaeo-dom": {
    34                 "test": ["Dromaeo", "DOM Core Tests"],
    35                 "arguments": {"tests": "dromaeo-dom"}
     33            'dromaeo-dom': {
     34                'test': ['Dromaeo', 'DOM Core Tests'],
     35                'arguments': {'tests': 'dromaeo-dom'}
    3636            },
    3737        },
    … …  
    3939            'iPhone-bench': {
    4040                'builder': 'ABTest-iPhone-RunBenchmark-Tests',
    41                 'arguments': { 'forcescheduler': 'ABTest-iPhone-RunBenchmark-Tests-ForceScheduler' }
     41                'arguments': { 'forcescheduler': 'ABTest-iPhone-RunBenchmark-Tests-ForceScheduler' },
     42                'slaveList': ['ABTest-iPhone-0'],
    4243            },
    4344            'iPad-bench': {
    4445                'builder': 'ABTest-iPad-RunBenchmark-Tests',
    45                 'arguments': { 'forcescheduler': 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler' }
     46                'arguments': { 'forcescheduler': 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler' },
     47                'slaveList': ['ABTest-iPad-0'],
    4648            }
    4749        },
    … …  
    7274};
    7375
     76function smallConfiguration()
     77{
     78    return {
     79        'builder': 'some builder',
     80        'platform': 'Some platform',
     81        'test': ['Some test'],
     82        'arguments': {},
     83        'buildRequestArgument': 'id'};
     84}
     85
     86function smallPendingBuild()
     87{
     88    return {
     89        'builderName': 'some builder',
     90        'builds': [],
     91        'properties': [],
     92        'source': {
     93            'branch': '',
     94            'changes': [],
     95            'codebase': 'WebKit',
     96            'hasPatch': false,
     97            'project': '',
     98            'repository': '',
     99            'revision': ''
     100        },
     101    };
     102}
     103
     104function smallInProgressBuild()
     105{
     106    return {
     107        'builderName': 'some builder',
     108        'builds': [],
     109        'properties': [],
     110        'currentStep': { },
     111        'eta': 123,
     112        'number': 456,
     113        'source': {
     114            'branch': '',
     115            'changes': [],
     116            'codebase': 'WebKit',
     117            'hasPatch': false,
     118            'project': '',
     119            'repository': '',
     120            'revision': ''
     121        },
     122    };
     123}
     124
     125function smallFinishedBuild()
     126{
     127    return {
     128        'builderName': 'some builder',
     129        'builds': [],
     130        'properties': [],
     131        'currentStep': null,
     132        'eta': null,
     133        'number': 789,
     134        'source': {
     135            'branch': '',
     136            'changes': [],
     137            'codebase': 'WebKit',
     138            'hasPatch': false,
     139            'project': '',
     140            'repository': '',
     141            'revision': ''
     142        },
     143        'times': [0, 1],
     144    };
     145}
     146
    74147function createSampleBuildRequest(platform, test)
    75148{
    … …  
    83156    ]});
    84157
    85     let request = BuildRequest.ensureSingleton('16733', {'rootSet': rootSet, 'status': 'pending', 'platform': platform, 'test': test});
     158    let request = BuildRequest.ensureSingleton('16733-' + platform.id(), {'rootSet': rootSet, 'status': 'pending', 'platform': platform, 'test': test});
    86159    return request;
    87160}
    88161
    89 function samplePendingBuild(buildRequestId)
     162function samplePendingBuild(buildRequestId, buildTime, slaveName)
    90163{
    91164    return {
    … …  
    103176                'Force Build Form'
    104177            ],
     178            ['slavename', slaveName, ''],
    105179            ['scheduler', 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler', 'Scheduler']
    106180        ],
    … …  
    114188            'revision': ''
    115189        },
    116         'submittedAt': 1458704983
     190        'submittedAt': buildTime || 1458704983
    117191    };
    118192}
    119193
    120 function sampleInProgressBuild()
     194function sampleInProgressBuild(slaveName)
    121195{
    122196    return {
    … …  
    150224            ['roots_dict', JSON.stringify(sampleRootSetData), 'Force Build Form'],
    151225            ['scheduler', 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler', 'Scheduler'],
    152             ['slavename', 'ABTest-iPad-0', 'BuildSlave'],
     226            ['slavename', slaveName || 'ABTest-iPad-0', 'BuildSlave'],
    153227        ],
    154228        'reason': 'A build was forced by \'<unknown>\': force build',
    … …  
    208282}
    209283
    210 function sampleFinishedBuild(buildRequestId)
     284function sampleFinishedBuild(buildRequestId, slaveName)
    211285{
    212286    return {
    … …  
    226300            ['roots_dict', JSON.stringify(sampleRootSetData), 'Force Build Form'],
    227301            ['scheduler', 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler', 'Scheduler'],
    228             ['slavename', 'ABTest-iPad-0', 'BuildSlave'],
     302            ['slavename', slaveName || 'ABTest-iPad-0', 'BuildSlave'],
    229303        ],
    230304        'reason': 'A build was forced by \'<unknown>\': force build',
    … …  
    286360describe('BuildbotSyncer', function () {
    287361    MockModels.inject();
    288     let requests = MockRemoteAPI.inject();
     362    let requests = MockRemoteAPI.inject('http://build.webkit.org');
    289363
    290364    describe('_loadConfig', function () {
    291365
    292         function smallConfiguration()
    293         {
    294             return {
    295                 'builder': 'some builder',
    296                 'platform': 'some platform',
    297                 'test': ['some test'],
    298                 'arguments': {},
    299                 'buildRequestArgument': 'id'};
    300         }
    301 
    302366        it('should create BuildbotSyncer objects for a configuration that specify all required options', function () {
    303             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [smallConfiguration()]});
     367            let syncers = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]});
    304368            assert.equal(syncers.length, 1);
    305369        });
    … …  
    309373                let config = smallConfiguration();
    310374                delete config['builder'];
    311                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     375                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    312376            }, 'builder should be a required option');
    313377            assert.throws(function () {
    314378                let config = smallConfiguration();
    315379                delete config['platform'];
    316                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     380                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    317381            }, 'platform should be a required option');
    318382            assert.throws(function () {
    319383                let config = smallConfiguration();
    320384                delete config['test'];
    321                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     385                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    322386            }, 'test should be a required option');
    323387            assert.throws(function () {
    324388                let config = smallConfiguration();
    325389                delete config['arguments'];
    326                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     390                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    327391            });
    328392            assert.throws(function () {
    329393                let config = smallConfiguration();
    330394                delete config['buildRequestArgument'];
    331                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     395                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    332396            });
    333397        });
    … …  
    337401                let config = smallConfiguration();
    338402                config.test = 'some test';
    339                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     403                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    340404            });
    341405            assert.throws(function () {
    342406                let config = smallConfiguration();
    343407                config.test = [1];
    344                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     408                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    345409            });
    346410        });
    … …  
    350414                let config = smallConfiguration();
    351415                config.arguments = 'hello';
    352                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     416                BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [config]});
    353417            });
    354418        });
    … …  
    358422                let config = smallConfiguration();
    359423                config.arguments = {'some': {'root': 'some root', 'rootsExcluding': ['other root']}};
    360                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     424                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
    361425            });
    362426            assert.throws(function () {
    363427                let config = smallConfiguration();
    364428                config.arguments = {'some': {'otherKey': 'some root'}};
    365                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     429                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
    366430            });
    367431            assert.throws(function () {
    368432                let config = smallConfiguration();
    369433                config.arguments = {'some': {'root': ['a', 'b']}};
    370                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     434                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
    371435            });
    372436            assert.throws(function () {
    373437                let config = smallConfiguration();
    374438                config.arguments = {'some': {'root': 1}};
    375                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     439                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
    376440            });
    377441            assert.throws(function () {
    378442                let config = smallConfiguration();
    379443                config.arguments = {'some': {'rootsExcluding': 'a'}};
    380                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     444                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
    381445            });
    382446            assert.throws(function () {
    383447                let config = smallConfiguration();
    384448                config.arguments = {'some': {'rootsExcluding': [1]}};
    385                 BuildbotSyncer._loadConfig('http://build.webkit.org/', {'configurations': [config]});
     449                BuildbotSyncer._loadConfig(RemoteAPI, {'configurations': [config]});
    386450            });
    387451        });
    388452
    389453        it('should create BuildbotSyncer objects for valid configurations', function () {
    390             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
    391             assert.equal(syncers.length, 5);
     454            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
     455            assert.equal(syncers.length, 2);
    392456            assert.ok(syncers[0] instanceof BuildbotSyncer);
    393457            assert.ok(syncers[1] instanceof BuildbotSyncer);
    394             assert.ok(syncers[2] instanceof BuildbotSyncer);
    395             assert.ok(syncers[3] instanceof BuildbotSyncer);
    396             assert.ok(syncers[4] instanceof BuildbotSyncer);
    397458        });
    398459
    399460        it('should parse builder names correctly', function () {
    400             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
     461            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
    401462            assert.equal(syncers[0].builderName(), 'ABTest-iPhone-RunBenchmark-Tests');
    402             assert.equal(syncers[1].builderName(), 'ABTest-iPhone-RunBenchmark-Tests');
    403             assert.equal(syncers[2].builderName(), 'ABTest-iPhone-RunBenchmark-Tests');
    404             assert.equal(syncers[3].builderName(), 'ABTest-iPad-RunBenchmark-Tests');
    405             assert.equal(syncers[4].builderName(), 'ABTest-iPad-RunBenchmark-Tests');
    406         });
    407 
    408         it('should parse platform names correctly', function () {
    409             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
    410             assert.equal(syncers[0].platformName(), 'iPhone');
    411             assert.equal(syncers[1].platformName(), 'iPhone');
    412             assert.equal(syncers[2].platformName(), 'iPhone');
    413             assert.equal(syncers[3].platformName(), 'iPad');
    414             assert.equal(syncers[4].platformName(), 'iPad');
    415         });
    416 
    417         it('should parse test names correctly', function () {
    418             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
    419             assert.deepEqual(syncers[0].testPath(), ['Speedometer']);
    420             assert.deepEqual(syncers[1].testPath(), ['JetStream']);
    421             assert.deepEqual(syncers[2].testPath(), ['Dromaeo', 'DOM Core Tests']);
    422             assert.deepEqual(syncers[3].testPath(), ['Speedometer']);
    423             assert.deepEqual(syncers[4].testPath(), ['JetStream']);
     463            assert.equal(syncers[1].builderName(), 'ABTest-iPad-RunBenchmark-Tests');
     464        });
     465
     466        it('should parse test configurations correctly', function () {
     467            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
     468
     469            let configurations = syncers[0].testConfigurations();
     470            assert.equal(configurations.length, 3);
     471            assert.equal(configurations[0].platform, MockModels.iphone);
     472            assert.equal(configurations[0].test, MockModels.speedometer);
     473            assert.equal(configurations[1].platform, MockModels.iphone);
     474            assert.equal(configurations[1].test, MockModels.jetstream);
     475            assert.equal(configurations[2].platform, MockModels.iphone);
     476            assert.equal(configurations[2].test, MockModels.domcore);
     477
     478            configurations = syncers[1].testConfigurations();
     479            assert.equal(configurations.length, 2);
     480            assert.equal(configurations[0].platform, MockModels.ipad);
     481            assert.equal(configurations[0].test, MockModels.speedometer);
     482            assert.equal(configurations[1].platform, MockModels.ipad);
     483            assert.equal(configurations[1].test, MockModels.jetstream);
    424484        });
    425485    });
    … …  
    427487    describe('_propertiesForBuildRequest', function () {
    428488        it('should include all properties specified in a given configuration', function () {
    429             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
     489            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
    430490            let properties = syncers[0]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
    431491            assert.deepEqual(Object.keys(properties), ['desired_image', 'roots_dict', 'test_name', 'forcescheduler', 'build_request_id']);
    … …  
    433493
    434494        it('should preserve non-parametric property values', function () {
    435             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
     495            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
    436496            let properties = syncers[0]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
    437497            assert.equal(properties['test_name'], 'speedometer');
    438498            assert.equal(properties['forcescheduler'], 'ABTest-iPhone-RunBenchmark-Tests-ForceScheduler');
    439499
    440             properties = syncers[1]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
     500            properties = syncers[1]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.ipad, MockModels.jetstream));
    441501            assert.equal(properties['test_name'], 'jetstream');
    442             assert.equal(properties['forcescheduler'], 'ABTest-iPhone-RunBenchmark-Tests-ForceScheduler');
     502            assert.equal(properties['forcescheduler'], 'ABTest-iPad-RunBenchmark-Tests-ForceScheduler');
    443503        });
    444504
    445505        it('should resolve "root"', function () {
    446             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
     506            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
    447507            let properties = syncers[0]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
    448508            assert.equal(properties['desired_image'], '13A452');
    … …  
    450510
    451511        it('should resolve "rootsExcluding"', function () {
    452             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
     512            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
    453513            let properties = syncers[0]._propertiesForBuildRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
    454514            assert.equal(properties['roots_dict'], JSON.stringify(sampleRootSetData));
    … …  
    456516
    457517        it('should set the property for the build request id', function () {
    458             let syncers = BuildbotSyncer._loadConfig('http://build.webkit.org/', sampleiOSConfig());
     518            let syncers = BuildbotSyncer._loadConfig(RemoteAPI, sampleiOSConfig());
    459519            let request = createSampleBuildRequest(MockModels.iphone, MockModels.speedometer);
    460520            let properties = syncers[0]._propertiesForBuildRequest(request);
    … …  
    465525    describe('pullBuildbot', function () {
    466526        it('should fetch pending builds from the right URL', function () {
    467             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     527            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    468528            assert.equal(syncer.builderName(), 'ABTest-iPad-RunBenchmark-Tests');
    469             let expectedURL = 'http://build.webkit.org/json/builders/ABTest-iPad-RunBenchmark-Tests/pendingBuilds';
    470             assert.equal(syncer.urlForPendingBuildsJSON(), expectedURL);
     529            let expectedURL = '/json/builders/ABTest-iPad-RunBenchmark-Tests/pendingBuilds';
     530            assert.equal(syncer.pathForPendingBuildsJSON(), expectedURL);
    471531            syncer.pullBuildbot();
    472532            assert.equal(requests.length, 1);
    … …  
    475535
    476536        it('should fetch recent builds once pending builds have been fetched', function (done) {
    477             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     537            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    478538            assert.equal(syncer.builderName(), 'ABTest-iPad-RunBenchmark-Tests');
    479539
    480540            syncer.pullBuildbot(1);
    481541            assert.equal(requests.length, 1);
    482             assert.equal(requests[0].url, 'http://build.webkit.org/json/builders/ABTest-iPad-RunBenchmark-Tests/pendingBuilds');
     542            assert.equal(requests[0].url, '/json/builders/ABTest-iPad-RunBenchmark-Tests/pendingBuilds');
    483543            requests[0].resolve([]);
    484544            Promise.resolve().then(function () {
    485545                assert.equal(requests.length, 2);
    486                 assert.equal(requests[1].url, 'http://build.webkit.org/json/builders/ABTest-iPad-RunBenchmark-Tests/builds/?select=-1');
     546                assert.equal(requests[1].url, '/json/builders/ABTest-iPad-RunBenchmark-Tests/builds/?select=-1');
    487547                done();
    488548            }).catch(done);
    … …  
    490550
    491551        it('should fetch the right number of recent builds', function (done) {
    492             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     552            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    493553
    494554            syncer.pullBuildbot(3);
    495555            assert.equal(requests.length, 1);
    496             assert.equal(requests[0].url, 'http://build.webkit.org/json/builders/ABTest-iPad-RunBenchmark-Tests/pendingBuilds');
     556            assert.equal(requests[0].url, '/json/builders/ABTest-iPad-RunBenchmark-Tests/pendingBuilds');
    497557            requests[0].resolve([]);
    498558            Promise.resolve().then(function () {
    499559                assert.equal(requests.length, 2);
    500                 assert.equal(requests[1].url, 'http://build.webkit.org/json/builders/ABTest-iPad-RunBenchmark-Tests/builds/?select=-1&select=-2&select=-3');
     560                assert.equal(requests[1].url, '/json/builders/ABTest-iPad-RunBenchmark-Tests/builds/?select=-1&select=-2&select=-3');
    501561                done();
    502562            }).catch(done);
    … …  
    504564
    505565        it('should create BuildbotBuildEntry for pending builds', function (done) {
    506             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     566            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    507567            let promise = syncer.pullBuildbot();
    508568            requests[0].resolve([samplePendingBuild()]);
    509569            promise.then(function (entries) {
    510                 assert.deepEqual(Object.keys(entries), ['16733']);
    511                 let entry = entries['16733'];
     570                assert.equal(entries.length, 1);
     571                let entry = entries[0];
    512572                assert.ok(entry instanceof BuildbotBuildEntry);
    513573                assert.ok(!entry.buildNumber());
    … …  
    523583
    524584        it('should create BuildbotBuildEntry for in-progress builds', function (done) {
    525             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     585            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    526586
    527587            let promise = syncer.pullBuildbot(1);
    … …  
    534594
    535595            promise.then(function (entries) {
    536                 assert.deepEqual(Object.keys(entries), ['16733']);
    537                 let entry = entries['16733'];
     596                assert.equal(entries.length, 1);
     597                let entry = entries[0];
    538598                assert.ok(entry instanceof BuildbotBuildEntry);
    539599                assert.equal(entry.buildNumber(), 614);
    … …  
    549609
    550610        it('should create BuildbotBuildEntry for finished builds', function (done) {
    551             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     611            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    552612
    553613            let promise = syncer.pullBuildbot(1);
    … …  
    560620
    561621            promise.then(function (entries) {
    562                 assert.deepEqual(Object.keys(entries), ['18935']);
    563                 let entry = entries['18935'];
     622                assert.deepEqual(entries.length, 1);
     623                let entry = entries[0];
    564624                assert.ok(entry instanceof BuildbotBuildEntry);
    565625                assert.equal(entry.buildNumber(), 1755);
    … …  
    575635
    576636        it('should create BuildbotBuildEntry for mixed pending, in-progress, finished, and missing builds', function (done) {
    577             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     637            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    578638
    579639            let promise = syncer.pullBuildbot(5);
    580640            assert.equal(requests.length, 1);
    581641
    582             requests[0].resolve([samplePendingBuild(123, 456)]);
     642            requests[0].resolve([samplePendingBuild(123)]);
    583643
    584644            Promise.resolve().then(function () {
    … …  
    588648
    589649            promise.then(function (entries) {
    590                 assert.deepEqual(Object.keys(entries), ['123', '16733', '18935']);
    591 
    592                 let entry = entries['123'];
     650                assert.deepEqual(entries.length, 3);
     651
     652                let entry = entries[0];
    593653                assert.ok(entry instanceof BuildbotBuildEntry);
    594654                assert.equal(entry.buildNumber(), null);
    … …  
    600660                assert.equal(entry.url(), 'http://build.webkit.org/builders/ABTest-iPad-RunBenchmark-Tests/');
    601661
    602                 entry = entries['16733'];
     662                entry = entries[1];
    603663                assert.ok(entry instanceof BuildbotBuildEntry);
    604664                assert.equal(entry.buildNumber(), 614);
    … …  
    610670                assert.equal(entry.url(), 'http://build.webkit.org/builders/ABTest-iPad-RunBenchmark-Tests/builds/614');
    611671
    612                 entry = entries['18935'];
     672                entry = entries[2];
    613673                assert.ok(entry instanceof BuildbotBuildEntry);
    614674                assert.equal(entry.buildNumber(), 1755);
    … …  
    624684        });
    625685
    626         it('should override BuildbotBuildEntry for pending builds by in-progress builds', function (done) {
    627             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     686        it('should sort BuildbotBuildEntry by order', function (done) {
     687            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    628688
    629689            let promise = syncer.pullBuildbot(5);
    630690            assert.equal(requests.length, 1);
    631691
    632             requests[0].resolve([samplePendingBuild()]);
     692            requests[0].resolve([samplePendingBuild(456, 2), samplePendingBuild(123, 1)]);
    633693
    634694            Promise.resolve().then(function () {
    635695                assert.equal(requests.length, 2);
    636                 requests[1].resolve({[-1]: sampleInProgressBuild()});
     696                requests[1].resolve({[-3]: sampleFinishedBuild(), [-1]: {'error': 'Not available'}, [-2]: sampleInProgressBuild()});
    637697            }).catch(done);
    638698
    639699            promise.then(function (entries) {
    640                 assert.deepEqual(Object.keys(entries), ['16733']);
    641 
    642                 let entry = entries['16733'];
     700                assert.deepEqual(entries.length, 4);
     701
     702                let entry = entries[0];
     703                assert.ok(entry instanceof BuildbotBuildEntry);
     704                assert.equal(entry.buildNumber(), null);
     705                assert.equal(entry.slaveName(), null);
     706                assert.equal(entry.buildRequestId(), 123);
     707                assert.ok(entry.isPending());
     708                assert.ok(!entry.isInProgress());
     709                assert.ok(!entry.hasFinished());
     710                assert.equal(entry.url(), 'http://build.webkit.org/builders/ABTest-iPad-RunBenchmark-Tests/');
     711
     712                entry = entries[1];
     713                assert.ok(entry instanceof BuildbotBuildEntry);
     714                assert.equal(entry.buildNumber(), null);
     715                assert.equal(entry.slaveName(), null);
     716                assert.equal(entry.buildRequestId(), 456);
     717                assert.ok(entry.isPending());
     718                assert.ok(!entry.isInProgress());
     719                assert.ok(!entry.hasFinished());
     720                assert.equal(entry.url(), 'http://build.webkit.org/builders/ABTest-iPad-RunBenchmark-Tests/');
     721
     722                entry = entries[2];
    643723                assert.ok(entry instanceof BuildbotBuildEntry);
    644724                assert.equal(entry.buildNumber(), 614);
    … …  
    650730                assert.equal(entry.url(), 'http://build.webkit.org/builders/ABTest-iPad-RunBenchmark-Tests/builds/614');
    651731
     732                entry = entries[3];
     733                assert.ok(entry instanceof BuildbotBuildEntry);
     734                assert.equal(entry.buildNumber(), 1755);
     735                assert.equal(entry.slaveName(), 'ABTest-iPad-0');
     736                assert.equal(entry.buildRequestId(), 18935);
     737                assert.ok(!entry.isPending());
     738                assert.ok(!entry.isInProgress());
     739                assert.ok(entry.hasFinished());
     740                assert.equal(entry.url(), 'http://build.webkit.org/builders/ABTest-iPad-RunBenchmark-Tests/builds/1755');
     741
     742                done();
     743            }).catch(done);
     744        });
     745
     746        it('should override BuildbotBuildEntry for pending builds by in-progress builds', function (done) {
     747            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
     748
     749            let promise = syncer.pullBuildbot(5);
     750            assert.equal(requests.length, 1);
     751
     752            requests[0].resolve([samplePendingBuild()]);
     753
     754            Promise.resolve().then(function () {
     755                assert.equal(requests.length, 2);
     756                requests[1].resolve({[-1]: sampleInProgressBuild()});
     757            }).catch(done);
     758
     759            promise.then(function (entries) {
     760                assert.equal(entries.length, 1);
     761
     762                let entry = entries[0];
     763                assert.ok(entry instanceof BuildbotBuildEntry);
     764                assert.equal(entry.buildNumber(), 614);
     765                assert.equal(entry.slaveName(), 'ABTest-iPad-0');
     766                assert.equal(entry.buildRequestId(), 16733);
     767                assert.ok(!entry.isPending());
     768                assert.ok(entry.isInProgress());
     769                assert.ok(!entry.hasFinished());
     770                assert.equal(entry.url(), 'http://build.webkit.org/builders/ABTest-iPad-RunBenchmark-Tests/builds/614');
     771
    652772                done();
    653773            }).catch(done);
    … …  
    655775
    656776        it('should override BuildbotBuildEntry for pending builds by finished builds', function (done) {
    657             let syncer = BuildbotSyncer._loadConfig('http://build.webkit.org', sampleiOSConfig())[3];
     777            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
    658778
    659779            let promise = syncer.pullBuildbot(5);
    … …  
    668788
    669789            promise.then(function (entries) {
    670                 assert.deepEqual(Object.keys(entries), ['16733']);
    671 
    672                 let entry = entries['16733'];
     790                assert.equal(entries.length, 1);
     791
     792                let entry = entries[0];
    673793                assert.ok(entry instanceof BuildbotBuildEntry);
    674794                assert.equal(entry.buildNumber(), 1755);
    … …  
    683803            }).catch(done);
    684804        });
     805    });
     806
     807    describe('scheduleRequest', function () {
     808        it('should schedule a build request on a specified slave', function (done) {
     809            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[0];
     810
     811            syncer.scheduleRequest(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer), 'some-slave');
     812            Promise.resolve().then(function () {
     813                assert.equal(requests.length, 1);
     814                assert.equal(requests[0].url, '/builders/ABTest-iPhone-RunBenchmark-Tests/force');
     815                assert.equal(requests[0].method, 'POST');
     816                assert.deepEqual(requests[0].data, {
     817                    'build_request_id': '16733-' + MockModels.iphone.id(),
     818                    'desired_image': '13A452',
     819                    'forcescheduler': 'ABTest-iPhone-RunBenchmark-Tests-ForceScheduler',
     820                    'roots_dict': '{"WebKit":{"id":"111127","time":1456955807334,"repository":"WebKit","revision":"197463"},'
     821                        + '"Shared":{"id":"111237","time":1456931874000,"repository":"Shared","revision":"80229"}}',
     822                    'slavename': 'some-slave',
     823                    'test_name': 'speedometer'
     824                });
     825                done();
     826            }).catch(done);
     827        });
     828    });
     829
     830    describe('scheduleFirstRequestInGroupIfAvailable', function () {
     831
     832        function pullBuildbotWithAssertion(syncer, pendingBuilds, inProgressAndFinishedBuilds)
     833        {
     834            let promise = syncer.pullBuildbot(5);
     835            assert.equal(requests.length, 1);
     836            requests[0].resolve(pendingBuilds);
     837            return Promise.resolve().then(function () {
     838                assert.equal(requests.length, 2);
     839                requests[1].resolve(inProgressAndFinishedBuilds);
     840                requests.length = 0;
     841            }).then(function () {
     842                return promise;
     843            });
     844        }
     845
     846        it('should schedule a build if builder has no builds if slaveList is not specified', function (done) {
     847            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]})[0];
     848
     849            pullBuildbotWithAssertion(syncer, [], {}).then(function () {
     850                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.somePlatform, MockModels.someTest));
     851            }).then(function () {
     852                assert.equal(requests.length, 1);
     853                assert.equal(requests[0].url, '/builders/some builder/force');
     854                assert.equal(requests[0].method, 'POST');
     855                assert.deepEqual(requests[0].data, {id: '16733-' + MockModels.somePlatform.id()});
     856                done();
     857            }).catch(done);
     858        });
     859
     860        it('should schedule a build if builder only has finished builds if slaveList is not specified', function (done) {
     861            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]})[0];
     862
     863            pullBuildbotWithAssertion(syncer, [], {[-1]: smallFinishedBuild()}).then(function () {
     864                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.somePlatform, MockModels.someTest));
     865            }).then(function () {
     866                assert.equal(requests.length, 1);
     867                assert.equal(requests[0].url, '/builders/some builder/force');
     868                assert.equal(requests[0].method, 'POST');
     869                assert.deepEqual(requests[0].data, {id: '16733-' + MockModels.somePlatform.id()});
     870                done();
     871            }).catch(done);
     872        });
     873
     874        it('should not schedule a build if builder has a pending build if slaveList is not specified', function (done) {
     875            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, {'configurations': [smallConfiguration()]})[0];
     876
     877            pullBuildbotWithAssertion(syncer, [smallPendingBuild()], {}).then(function () {
     878                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.somePlatform, MockModels.someTest));
     879            }).then(function () {
     880                assert.equal(requests.length, 0);
     881                done();
     882            }).catch(done);
     883        });
     884
     885        it('should schedule a build if builder does not have pending or completed builds on the matching slave', function (done) {
     886            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[0];
     887
     888            pullBuildbotWithAssertion(syncer, [], {}).then(function () {
     889                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.iphone, MockModels.speedometer));
     890            }).then(function () {
     891                assert.equal(requests.length, 1);
     892                assert.equal(requests[0].url, '/builders/ABTest-iPhone-RunBenchmark-Tests/force');
     893                assert.equal(requests[0].method, 'POST');
     894                done();
     895            }).catch(done);
     896        });
     897
     898        it('should schedule a build if builder only has finished builds on the matching slave', function (done) {
     899            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
     900
     901            pullBuildbotWithAssertion(syncer, [], {[-1]: sampleFinishedBuild()}).then(function () {
     902                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.ipad, MockModels.speedometer));
     903            }).then(function () {
     904                assert.equal(requests.length, 1);
     905                assert.equal(requests[0].url, '/builders/ABTest-iPad-RunBenchmark-Tests/force');
     906                assert.equal(requests[0].method, 'POST');
     907                done();
     908            }).catch(done);
     909        });
     910
     911        it('should not schedule a build if builder has a pending build on the maching slave', function (done) {
     912            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
     913
     914            pullBuildbotWithAssertion(syncer, [samplePendingBuild()], {}).then(function () {
     915                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.ipad, MockModels.speedometer));
     916            }).then(function () {
     917                assert.equal(requests.length, 0);
     918                done();
     919            }).catch(done);
     920        });
     921
     922        it('should schedule a build if builder only has a pending build on a non-maching slave', function (done) {
     923            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
     924
     925            pullBuildbotWithAssertion(syncer, [samplePendingBuild(1, 1, 'another-slave')], {}).then(function () {
     926                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.ipad, MockModels.speedometer));
     927            }).then(function () {
     928                assert.equal(requests.length, 1);
     929                done();
     930            }).catch(done);
     931        });
     932
     933        it('should schedule a build if builder only has an in-progress build on the matching slave', function (done) {
     934            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
     935
     936            pullBuildbotWithAssertion(syncer, [], {[-1]: sampleInProgressBuild()}).then(function () {
     937                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.ipad, MockModels.speedometer));
     938            }).then(function () {
     939                assert.equal(requests.length, 1);
     940                done();
     941            }).catch(done);
     942        });
     943
     944        it('should schedule a build if builder has an in-progress build on another slave', function (done) {
     945            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[1];
     946
     947            pullBuildbotWithAssertion(syncer, [], {[-1]: sampleInProgressBuild('other-slave')}).then(function () {
     948                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.ipad, MockModels.speedometer));
     949            }).then(function () {
     950                assert.equal(requests.length, 1);
     951                done();
     952            }).catch(done);
     953        });
     954
     955        it('should not schedule a build if the request does not match any configuration', function (done) {
     956            let syncer = BuildbotSyncer._loadConfig(MockRemoteAPI, sampleiOSConfig())[0];
     957
     958            pullBuildbotWithAssertion(syncer, [], {}).then(function () {
     959                syncer.scheduleFirstRequestInGroupIfAvailable(createSampleBuildRequest(MockModels.ipad, MockModels.speedometer));
     960            }).then(function () {
     961                assert.equal(requests.length, 0);
     962                done();
     963            }).catch(done);
     964        });
    685965
    686966    });

  • trunk/Websites/perf.webkit.org/unit-tests/resources/mock-remote-api.js

    r198826 r199123  
    55
    66var MockRemoteAPI = {
     7    url: function (path)
     8    {
     9        return `${this.urlPrefix}${path}`;
     10    },
    711    getJSON: function (url)
    812    {
    … …  
    1115    getJSONWithStatus: function (url)
    1216    {
     17        return this._addRequest(url, 'GET', null);
     18    },
     19    postFormUrlencodedData: function (url, data)
     20    {
     21        return this._addRequest(url, 'POST', data);
     22    },
     23    _addRequest: function (url, method, data)
     24    {
    1325        var request = {
    1426            url: url,
     27            method: method,
     28            data: data,
    1529            promise: null,
    1630            resolve: null,
    … …  
    2337        });
    2438
     39        if (this._waitingPromise) {
     40            this._waitingPromiseResolver();
     41            this._waitingPromise = null;
     42            this._waitingPromiseResolver = null;
     43        }
     44
    2545        MockRemoteAPI.requests.push(request);
    2646        return request.promise;
    2747    },
    28     inject: function ()
     48    waitForRequest()
     49    {
     50        if (!this._waitingPromise) {
     51            this._waitingPromise = new Promise(function (resolve, reject) {
     52                MockRemoteAPI._waitingPromiseResolver = resolve;
     53            });
     54        }
     55        return this._waitingPromise;
     56    },
     57    inject: function (urlPrefix)
    2958    {
    3059        var originalRemoteAPI = global.RemoteAPI;
    3160
    3261        beforeEach(function () {
    33             MockRemoteAPI.requests.length = 0;
     62            MockRemoteAPI.reset(urlPrefix);
    3463            originalRemoteAPI = global.RemoteAPI;
    3564            global.RemoteAPI = MockRemoteAPI;
    … …  
    4170
    4271        return MockRemoteAPI.requests;
    43     }
     72    },
     73    reset: function (urlPrefix)
     74    {
     75        if (urlPrefix)
     76            MockRemoteAPI.urlPrefix = urlPrefix;
     77        MockRemoteAPI.requests.length = 0;
     78    },
     79    requests: [],
     80    _waitingPromise: null,
     81    _waitingPromiseResolver: null,
     82    urlPrefix: 'http://mockhost',
    4483};
    45 MockRemoteAPI.requests = [];
    4684
    4785if (typeof module != 'undefined')

  • trunk/Websites/perf.webkit.org/unit-tests/resources/mock-v3-models.js

    r198826 r199123  
    88            CommitLog.clearStaticMap();
    99            Metric.clearStaticMap();
     10            Platform.clearStaticMap();
     11            Repository.clearStaticMap();
    1012            RootSet.clearStaticMap();
    1113            Test.clearStaticMap();
Note: See TracChangeset for help on using the changeset viewer.