Changeset 168732 in webkit


Ignore:
Timestamp:
May 13, 2014 2:08:10 PM (10 years ago)
Author:
rniwa@webkit.org
Message:

DYEBench should run 20 iterations in browser
​https://bugs.webkit.org/show_bug.cgi?id=132795

Reviewed by Gavin Barraclough.

Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
a more stable time and account for differences in the runtime environment, particularly,
ASLR (Address Space Layout Randomization).

While we can't account for the latter effect when the benchmark is ran inside a browser,
we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.

While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
it's MUCH better than the current sample size of 5.

  • DoYouEvenBench/Full.html:

(benchmarkClient.iterationCount): Set the default iteration count to 20.
(startTest): Use benchmarkClient.iterationCount as the iteration count.

  • DoYouEvenBench/resources/benchmark-report.js:

(benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
inside run-perf-tests.

Location:
trunk/PerformanceTests
Files:
3 edited

Legend:

Unmodified
Added
Removed

  • trunk/PerformanceTests/ChangeLog

    r168564 r168732  
     12014-05-13  Ryosuke Niwa  <rniwa@webkit.org>
     2
     3        DYEBench should run 20 iterations in browser
     4        https://bugs.webkit.org/show_bug.cgi?id=132795
     5
     6        Reviewed by Gavin Barraclough.
     7
     8        Right now, run-perf-tests runs BYEBench 4 times with different instances of DRT/WTR to get
     9        a more stable time and account for differences in the runtime environment, particularly,
     10        ASLR (Address Space Layout Randomization).
     11
     12        While we can't account for the latter effect when the benchmark is ran inside a browser,
     13        we can at least run 20 iterations to get a more stable (but perhaps biased by ALSR) number.
     14
     15        While the sample size of 20 is statistically unsound to compute the arithmetic mean from,
     16        it's MUCH better than the current sample size of 5.
     17
     18        * DoYouEvenBench/Full.html:
     19        (benchmarkClient.iterationCount): Set the default iteration count to 20.
     20        (startTest): Use benchmarkClient.iterationCount as the iteration count.
     21        * DoYouEvenBench/resources/benchmark-report.js:
     22        (benchmarkClient.iterationCount): Continue to use the old iteration count of 5 when ran
     23        inside run-perf-tests.
     24
    1252014-05-09  Ryosuke Niwa  <rniwa@webkit.org>
    226

  • trunk/PerformanceTests/DoYouEvenBench/Full.html

    r168564 r168732  
    22<html>
    33<head>
    4 <title>DoYouEvenBench v0.10</title>
     4<title>DoYouEvenBench v0.11</title>
    55<style type="text/css">
    66caption { margin: 0; padding: 0; font-family: sans-serif; font-size: 1em; font-weight: bold; white-space: nowrap; }
    … …  
    4444
    4545    window.benchmarkClient = {
     46        iterationCount: 20,
    4647        willRunTest: function () {
    4748            if (!progress) {
    … …  
    8081
    8182function startTest() {
    82     var iterationCount = 5;
    83     benchmarkClient.testsCount = iterationCount * Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
     83    var totalSubtestCount = Suites.reduce(function (testsCount, suite) { return testsCount + suite.tests.length; }, 0);
     84    benchmarkClient.testsCount = benchmarkClient.iterationCount * totalSubtestCount;
    8485    var runner = new BenchmarkRunner(Suites, benchmarkClient);
    85     runner.runMultipleIterations(iterationCount);
     86    runner.runMultipleIterations(benchmarkClient.iterationCount);
    8687}
    8788

  • trunk/PerformanceTests/DoYouEvenBench/resources/benchmark-report.js

    r162058 r168732  
    2020
    2121    window.benchmarkClient = {
     22        iterationCount: 5, // Use 4 different instances of DRT/WTR to run 5 iterations.
    2223        willStartFirstIteration: function (iterationCount) {
    2324            createTest = function (name, aggregator, isLastTest) {
Note: See TracChangeset for help on using the changeset viewer.