Changeset 52057 in webkit


Ignore:
Timestamp:
Dec 12, 2009 6:18:53 PM (14 years ago)
Author:
mjs@apple.com
Message:

2009-12-12 Maciej Stachowiak <​mjs@apple.com>

Reviewed by Oliver Hunt.

Give command-line sunspider the ability to handle multiple suites and versions
​https://bugs.webkit.org/show_bug.cgi?id=32477

  • Scripts/run-sunspider: Updated for changes to command-line parameters.
  • Scripts/sunspider-compare-results: ditto

2009-12-12 Maciej Stachowiak <​mjs@apple.com>

Reviewed by Oliver Hunt.

Give command-line sunspider the ability to handle multiple suites and versions
​https://bugs.webkit.org/show_bug.cgi?id=32477


Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
enumerating the tests. Current suites are:


sunspider-0.9 - riginal SunSpider content
sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
ubench - "ubench" microbenchmark suite
v8-v4 - version 4 of the v8 benchmark
parse-only - some parser tests


Sunspider can now be run with the --suite= parameter to select a
suite. The default is --suite=sunspider-0.9.

  • .: Modified property svn:ignore to ignore new results directories.
  • make-hosted: Generate based on sunspider-0.9 suite.
  • resources/TEMPLATE.html: Fix an HTML compliance problem.
  • resources/sunspider-standalone-driver.js: Become aware of suite names.
  • sunspider: Modified to handle multiple suites as described above.
  • sunspider-compare-results: ditto
  • tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
  • tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
  • tests/LIST-UBENCH: Removed.
  • tests/LIST-V8: Removed.
  • tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
  • tests/3d-morph.js: ditto
  • tests/3d-raytrace.js: ditto
  • tests/access-binary-trees.js: ditto
  • tests/access-fannkuch.js: ditto
  • tests/access-nbody.js: ditto
  • tests/access-nsieve.js: ditto
  • tests/bitops-3bit-bits-in-byte.js: ditto
  • tests/bitops-bits-in-byte.js: ditto
  • tests/bitops-bitwise-and.js: ditto
  • tests/bitops-nsieve-bits.js: ditto
  • tests/controlflow-recursive.js: ditto
  • tests/crypto-aes.js: ditto
  • tests/crypto-md5.js: ditto
  • tests/crypto-sha1.js: ditto
  • tests/date-format-tofte.js: ditto
  • tests/date-format-xparb.js: ditto
  • tests/math-cordic.js: ditto
  • tests/math-partial-sums.js: ditto
  • tests/math-spectral-norm.js: ditto
  • tests/regexp-dna.js: ditto
  • tests/string-base64.js: ditto
  • tests/string-fasta.js: ditto
  • tests/string-tagcloud.js: ditto
  • tests/string-unpack-code.js: ditto
  • tests/string-validate-input.js: ditto
  • tests/sunspider-0.9/: Added
  • tests/sunspider-0.9/*.js: Moved from one level up
  • tests/sunspider-0.9.1: Added.
  • tests/sunspider-0.9.1/*.js: Moved from one level up.
  • tests/v8-crypto.js: Moved to v8-v4/ directory.
  • tests/v8-deltablue.js: ditto
  • tests/v8-earley-boyer.js: ditto
  • tests/v8-raytrace.js: ditto
  • tests/v8-regexp.js: ditto
  • tests/v8-richards.js: ditto
  • tests/v8-splay.js: ditto
  • tests/v8-v4: Added.
Location:
trunk
Files:
3 added
10 edited
27 copied
37 moved

Legend:

Unmodified
Added
Removed

  • trunk/SunSpider

    • Property svn:ignore

      •  

        old new  
         1parse-only-results
         2sunspider-0.9-results
         3sunspider-0.9.1-results
         4ubench-results
         5v8-v4-results
        16tmp
        27*.mshark

  • trunk/SunSpider/ChangeLog

    r48923 r52057  
     12009-12-12  Maciej Stachowiak  <mjs@apple.com>
     2
     3        Reviewed by Oliver Hunt.
     4
     5        Give command-line sunspider the ability to handle multiple suites and versions
     6        https://bugs.webkit.org/show_bug.cgi?id=32477
     7       
     8        Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
     9        enumerating the tests. Current suites are:
     10       
     11        sunspider-0.9 - riginal SunSpider content
     12        sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
     13        ubench - "ubench" microbenchmark suite
     14        v8-v4 - version 4 of the v8 benchmark
     15        parse-only - some parser tests
     16       
     17        Sunspider can now be run with the --suite= parameter to select a
     18        suite. The default is --suite=sunspider-0.9.
     19
     20        * .: Modified property svn:ignore to ignore new results directories.
     21        * make-hosted: Generate based on sunspider-0.9 suite.
     22        * resources/TEMPLATE.html: Fix an HTML compliance problem.
     23        * resources/sunspider-standalone-driver.js: Become aware of suite names.
     24        * sunspider: Modified to handle multiple suites as described above.
     25        * sunspider-compare-results: ditto
     26        * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
     27        * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
     28        * tests/LIST-UBENCH: Removed.
     29        * tests/LIST-V8: Removed.
     30        * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
     31        * tests/3d-morph.js: ditto
     32        * tests/3d-raytrace.js: ditto
     33        * tests/access-binary-trees.js: ditto
     34        * tests/access-fannkuch.js: ditto
     35        * tests/access-nbody.js: ditto
     36        * tests/access-nsieve.js: ditto
     37        * tests/bitops-3bit-bits-in-byte.js: ditto
     38        * tests/bitops-bits-in-byte.js: ditto
     39        * tests/bitops-bitwise-and.js: ditto
     40        * tests/bitops-nsieve-bits.js: ditto
     41        * tests/controlflow-recursive.js: ditto
     42        * tests/crypto-aes.js: ditto
     43        * tests/crypto-md5.js: ditto
     44        * tests/crypto-sha1.js: ditto
     45        * tests/date-format-tofte.js: ditto
     46        * tests/date-format-xparb.js: ditto
     47        * tests/math-cordic.js: ditto
     48        * tests/math-partial-sums.js: ditto
     49        * tests/math-spectral-norm.js: ditto
     50        * tests/regexp-dna.js: ditto
     51        * tests/string-base64.js: ditto
     52        * tests/string-fasta.js: ditto
     53        * tests/string-tagcloud.js: ditto
     54        * tests/string-unpack-code.js: ditto
     55        * tests/string-validate-input.js: ditto
     56        * tests/sunspider-0.9/: Added
     57        * tests/sunspider-0.9/*.js: Moved from one level up
     58        * tests/sunspider-0.9.1: Added.
     59        * tests/sunspider-0.9.1/*.js: Moved from one level up.
     60        * tests/v8-crypto.js: Moved to v8-v4/ directory.
     61        * tests/v8-deltablue.js: ditto
     62        * tests/v8-earley-boyer.js: ditto
     63        * tests/v8-raytrace.js: ditto
     64        * tests/v8-regexp.js: ditto
     65        * tests/v8-richards.js: ditto
     66        * tests/v8-splay.js: ditto
     67        * tests/v8-v4: Added.
     68
    1692009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
    270

  • trunk/SunSpider/make-hosted

    r27411 r52057  
    3434my %uniqueCategories = ();
    3535
    36 open TESTLIST, "<./tests/LIST";
     36open TESTLIST, "<./tests/sunspider-0.9/LIST";
    3737while (<TESTLIST>) {
    3838    chomp;
    … …  
    5151    my $name = "${test}";
    5252
    53     open SCRIPT, "<tests/${test}.js";
     53    open SCRIPT, "<tests/sunspider-0.9/${test}.js";
    5454    my $script = do { local $/; <SCRIPT> };
    5555    close SCRIPT;

  • trunk/SunSpider/resources/TEMPLATE.html

    r28678 r52057  
    2727
    2828<title>SunSpider @NAME@</title>
    29 <link rel="stylesheet" href="sunspider.css"></link>
     29<link rel="stylesheet" href="sunspider.css">
    3030</head>
    3131

  • trunk/SunSpider/resources/sunspider-standalone-driver.js

    r44870 r52057  
    3131
    3232for (var j = 0; j < tests.length; j++) {
    33     var testName = "tests/" + tests[j] + ".js";
     33    var testName = "tests/" + suiteName + "/" + tests[j] + ".js";
    3434    var startTime = new Date;
    3535    if (testName.indexOf('parse-only') >= 0)

  • trunk/SunSpider/sunspider

    r48923 r52057  
    3939my $ubench = 0;
    4040my $v8suite = 0;
    41 my $parseonly = 0;
     41my $suite = "";
     42my $parseOnly = 0;
    4243my $jsShellPath;
    4344my $jsShellArgs = "";
    … …  
    5859  --shark20         Like --shark, but with a 20 microsecond sampling interval
    5960  --shark-cache     Like --shark, but performs a L2 cache-miss sample instead of time sample
    60   --ubench          Use microbenchmark suite instead of regular tests (to check for core execution regressions)
    61   --v8-suite        Use the V8 benchmark suite
    62   --parse-only      Use the parse-only benchmark suite
     61  --suite           Select a specific benchmark suite. The default is sunspider-0.9
     62  --ubench          Use microbenchmark suite instead of regular tests. Same as --suite=ubench
     63  --v8-suite        Use the V8 benchmark suite. Same as --suite=v8-v4
     64  --parse-only      Use the parse-only benchmark suite. Same as --suite=parse-only
    6365EOF
    6466
    … …  
    7072           'shark20' => \$runShark20,
    7173           'shark-cache' => \$runSharkCache,
     74           'suite=s' => \$suite,
    7275           'ubench' => \$ubench,
    7376           'v8-suite' => \$v8suite,
    74            'parse-only' => \$parseonly,
     77           'parse-only' => \$parseOnly,
    7578           'tests=s' => \$testsPattern,
    7679           'help' => \$showHelp);
    7780
    78 my $resultDirectory = "sunspider-results";
    79 $resultDirectory = "ubench-results" if ($ubench);
    80 $resultDirectory = "v8-results" if ($v8suite);
    81 $resultDirectory = "parse-only-results" if ($parseonly);
     81
     82$suite = "ubench" if ($ubench);
     83$suite = "v8-v4" if ($v8suite);
     84$suite = "parse-only" if ($parseOnly);
     85$suite = "sunspider-0.9" if (!$suite);
     86
     87my $resultDirectory = "${suite}-results";
    8288
    8389$runShark = 1 if $runSharkCache;
    … …  
    124130sub loadTestsList()
    125131{
    126     my $testlist = "LIST";
    127     $testlist = "LIST-UBENCH" if ($ubench);
    128     $testlist = "LIST-V8" if ($v8suite);
    129     $testlist = "LIST-PARSE-ONLY" if ($parseonly);
    130 
    131     open TESTLIST, "<", "tests/${testlist}" or die "Can't find ./tests/${testlist}";
     132    open TESTLIST, "<", "tests/${suite}/LIST" or die "Can't find ./tests/${suite}/LIST";
    132133    while (<TESTLIST>) {
    133134        chomp;
    … …  
    151152sub writePrefixFile()
    152153{
    153     my $prefix = "var tests = [ " . join(", ", map { '"' . $_ . '"' } @tests) . " ];\n";
     154    my $prefix = "var suiteName = " . '"' . $suite . '"' . ";\n";
     155    $prefix .= "var tests = [ " . join(", ", map { '"' . $_ . '"' } @tests) . " ];\n";
    154156    $prefix .= "var categories = [ " . join(", ", map { '"' . $_ . '"' } @categories) . " ];\n";
    155157

  • trunk/SunSpider/sunspider-compare-results

    r44870 r52057  
    3030my $showHelp = 0;
    3131my $jsShellPath;
     32my $suite = "";
    3233my $ubench = 0;
    3334my $v8suite = 0;
    34 my $parseonly = 0;
     35my $parseOnly = 0;
    3536
    3637my $programName = basename($0);
    … …  
    3940  --help        Show this help message
    4041  --shell       Path to javascript shell
    41   --ubench      Compare microbenchmark results
    42   --v8-suite    Compare the V8 benchmark results
     42  --suite           Select a specific benchmark suite. The default is sunspider-0.9
     43  --ubench          Use microbenchmark suite instead of regular tests. Same as --suite=ubench
     44  --v8-suite        Use the V8 benchmark suite. Same as --suite=v8-v4
    4345  --parse-only  Compare the parse-only benchmark results
    4446EOF
    4547
    4648GetOptions('shell=s' => \$jsShellPath,
     49           'suite=s' => \$suite,
    4750           'ubench' => \$ubench,
    4851           'v8-suite' => \$v8suite,
    49            'parse-only' => \$parseonly,
     52           'parse-only' => \$parseOnly,
    5053           'help' => \$showHelp);
    5154
    52 my $resultDirectory = "sunspider-results";
    53 $resultDirectory = "ubench-results" if ($ubench);
    54 $resultDirectory = "v8-results" if ($v8suite);
    55 $resultDirectory = "parse-only-results" if ($parseonly);
     55$suite = "ubench" if ($ubench);
     56$suite = "v8-v4" if ($v8suite);
     57$suite = "parse-only" if ($parseOnly);
     58$suite = "sunspider-0.9" if (!$suite);
     59
     60my $resultDirectory = "${suite}-results";
    5661
    5762if ((scalar @ARGV != 0 && scalar @ARGV != 2) || !$jsShellPath || $showHelp) {

  • trunk/SunSpider/tests/sunspider-0.9/regexp-dna.js

    r52056 r52057  
    17091709
    17101710for(k in subs)
    1711  dnaInput = dnaInput.replace(k, subs[k]) // FIXME: Would like this to be a global substitution in a future version of SunSpider.
     1711    dnaInput = dnaInput.replace(k, subs[k], "g")
    17121712 // search string, replacement string, flags

  • trunk/SunSpider/tests/ubench/LIST

    r52056 r52057  
    1 ubench/function-closure
    2 ubench/function-empty
    3 ubench/function-correct-args
    4 ubench/function-excess-args
    5 ubench/function-missing-args
    6 ubench/function-sum
    7 ubench/loop-empty-resolve
    8 ubench/loop-empty
    9 ubench/loop-sum
     1function-closure
     2function-empty
     3function-correct-args
     4function-excess-args
     5function-missing-args
     6function-sum
     7loop-empty-resolve
     8loop-empty
     9loop-sum

  • trunk/WebKitTools/ChangeLog

    r52053 r52057  
     12009-12-12  Maciej Stachowiak  <mjs@apple.com>
     2
     3        Reviewed by Oliver Hunt.
     4
     5        Give command-line sunspider the ability to handle multiple suites and versions
     6        https://bugs.webkit.org/show_bug.cgi?id=32477
     7
     8        * Scripts/run-sunspider: Updated for changes to command-line parameters.
     9        * Scripts/sunspider-compare-results: ditto
     10
    1112009-12-12  Maciej Stachowiak  <mjs@apple.com>
    212

  • trunk/WebKitTools/Scripts/run-sunspider

    r49476 r52057  
    4242my $runShark20 = 0;
    4343my $runSharkCache = 0;
     44my $suite = "";
    4445my $ubench = 0;
    4546my $v8 = 0;
    … …  
    6061  --shark20         Like --shark, but with a 20 microsecond sampling interval
    6162  --shark-cache     Like --shark, but performs a L2 cache-miss sample instead of time sample
    62   --ubench          Use microbenchmark suite instead of regular tests (to check for core execution regressions)
    63   --v8              Use the V8 benchmark suite.
    64   --parse-only      Use the parse-only benchmark suite
     63  --suite           Select a specific benchmark suite. The default is sunspider-0.9.1
     64  --ubench          Use microbenchmark suite instead of regular tests. Same as --suite=ubench
     65  --v8-suite        Use the V8 benchmark suite. Same as --suite=v8-v4
     66  --parse-only      Use the parse-only benchmark suite. Same as --suite=parse-only
    6567EOF
    6668
    … …  
    7173           'shark20' => \$runShark20,
    7274           'shark-cache' => \$runSharkCache,
     75           'suite=s' => \$suite,
    7376           'ubench' => \$ubench,
    7477           'v8' => \$v8,
    … …  
    118121push @args, "--shark20" if $runShark20;
    119122push @args, "--shark-cache" if $runSharkCache;
     123push @args, "--suite=${suite}" if $suite;
    120124push @args, "--ubench" if $ubench;
    121125push @args, "--v8" if $v8;

  • trunk/WebKitTools/Scripts/sunspider-compare-results

    r52053 r52057  
    4040my $root;
    4141my $showHelp = 0;
     42my $suite = "";
    4243my $ubench = 0;
    4344my $v8 = 0;
    … …  
    4849  --help        Show this help message
    4950  --root        Path to root tools build
    50   --ubench      Compare microbenchmark results
    51   --v8          Compare the V8 benchmark results
     51  --suite           Select a specific benchmark suite. The default is sunspider-0.9.1
     52  --ubench          Use microbenchmark suite instead of regular tests. Same as --suite=ubench
     53  --v8-suite        Use the V8 benchmark suite. Same as --suite=v8-v4
     54  --parse-only      Use the parse-only benchmark suite. Same as --suite=parse-only
    5255EOF
    5356
    5457GetOptions('root=s' => sub { my ($argName, $value); setConfigurationProductDir(Cwd::abs_path($value)); },
     58           'suite=s' => \$suite,
    5559           'ubench' => \$ubench,
    5660           'v8' => \$v8,
     61           'parse-only' => \$parseonly,
    5762           'help' => \$showHelp);
    5863
    … …  
    120125my @args = ("--shell", $jscPath);
    121126# This code could be removed if we chose to pass extra args to sunspider instead of Xcode
     127push @args, "--suite=${suite}" if $suite;
    122128push @args, "--ubench" if $ubench;
    123129push @args, "--v8" if $v8;
     130push @args, "--parse-only" if $parseonly;
    124131
    125132exec currentPerlPath(), "./sunspider-compare-results", @args, @ARGV;
Note: See TracChangeset for help on using the changeset viewer.