Changeset 160016 in webkit


Ignore:
Timestamp:
Dec 3, 2013 10:51:12 AM (10 years ago)
Author:
mhahnenberg@apple.com
Message:

run-jsc-stress-tests only supports host environments that have make installed
​https://bugs.webkit.org/show_bug.cgi?id=124550

Reviewed by Darin Adler.

This might not be the case for all hosts, so this patch implements an alternate "backend"
for run-jsc-stress-tests to use normal shell commands rather than Makefiles. To remain at
least somewhat competitive with the make-based test runner, the shell backend uses subshells
run in the background to allow tests to run in parallel. Since the concurrency primitives
in shell scripting are rather coarse, the overhead of this parallelism is higher than that
of the make-based runner.

  • Scripts/jsc-stress-test-helpers/shell-runner.sh: Added. This is the runner that is copied into

the bundle and controls all of the parallel aspects of the shell-based test runner.

  • Scripts/run-jsc-stress-tests:
Location:
trunk/Tools
Files:
1 added
2 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r160006 r160016  
     12013-12-03  Mark Hahnenberg  <mhahnenberg@apple.com>
     2
     3        run-jsc-stress-tests only supports host environments that have make installed
     4        https://bugs.webkit.org/show_bug.cgi?id=124550
     5
     6        Reviewed by Darin Adler.
     7
     8        This might not be the case for all hosts, so this patch implements an alternate "backend"
     9        for run-jsc-stress-tests to use normal shell commands rather than Makefiles. To remain at
     10        least somewhat competitive with the make-based test runner, the shell backend uses subshells
     11        run in the background to allow tests to run in parallel. Since the concurrency primitives
     12        in shell scripting are rather coarse, the overhead of this parallelism is higher than that
     13        of the make-based runner.
     14
     15        * Scripts/jsc-stress-test-helpers/shell-runner.sh: Added. This is the runner that is copied into
     16        the bundle and controls all of the parallel aspects of the shell-based test runner.
     17        * Scripts/run-jsc-stress-tests:
     18
    1192013-12-03  Tamas Gergely  <tgergely.u-szeged@partner.samsung.com>
    220

  • trunk/Tools/Scripts/run-jsc-stress-tests

    r159990 r160016  
    6969
    7070def mysys(*cmd)
    71     printCommandArray(*cmd)
     71    printCommandArray(*cmd) if $verbosity >= 1
    7272    raise "Command failed: #{$?.inspect}" unless system(*cmd)
    7373end
    … …  
    9393$tarball = false
    9494$copyVM = false
     95$testRunnerType = :make
    9596
    9697def usage
    … …  
    103104    puts "--run-bundle                Runs a bundle previously created by run-jsc-stress-tests."
    104105    puts "--tarball                   Creates a tarball of the final bundle."
     106    puts "--shell-runner              Uses the shell-based test runner instead of the default make-based runner."
     107    puts "                            In general the shell runner is slower than the make runner."
    105108    puts "--help               (-h)   Print this message."
    106109    exit 1
    … …  
    114117               ['--tarball', GetoptLong::NO_ARGUMENT],
    115118               ['--force-vm-copy', GetoptLong::NO_ARGUMENT],
     119               ['--shell-runner', GetoptLong::NO_ARGUMENT],
    116120               ['--verbose', '-v', GetoptLong::NO_ARGUMENT]).each {
    117121    | opt, arg |
    … …  
    134138    when '--force-vm-copy'
    135139        $copyVM = true
     140    when '--shell-runner'
     141        $testRunnerType = :shell
    136142    end
    137143}
    … …  
    382388   
    383389    def reproScriptCommand
    384         # We have to find our way back to the .parallel directory since that's where all of the relative
     390        # We have to find our way back to the .runner directory since that's where all of the relative
    385391        # paths assume they start out from.
    386392        script = "CURRENT_DIR=\"$( cd \"$( dirname \"${BASH_SOURCE[0]}\" )\" && pwd )\"\n"
    … …  
    390396            script += "cd ..\n"
    391397        }
    392         script += "cd .parallel\n"
     398        script += "cd .runner\n"
    393399
    394400        script += "export DYLD_FRAMEWORK_PATH=$(cd ../#{$frameworkPath.dirname}; pwd)\n"
    … …  
    917923end
    918924
    919 def prepareParallelTestRunner
     925def prepareTestRunner
     926    raise if $bundle
     927
     928    $runlist.each_with_index {
     929        | plan, index |
     930        plan.index = index
     931    }
     932
     933    Dir.mkdir($runnerDir) unless $runnerDir.directory?
     934    toDelete = []
     935    Dir.foreach($runnerDir) {
     936        | filename |
     937        if filename =~ /^test_/
     938            toDelete << filename
     939        end
     940    }
     941   
     942    toDelete.each {
     943        | filename |
     944        File.unlink($runnerDir + filename)
     945    }
     946
     947    $runlist.each {
     948        | plan |
     949        plan.writeRunScript($runnerDir + "test_script_#{plan.index}")
     950    }
     951
     952    case $testRunnerType
     953    when :make
     954        prepareMakeTestRunner
     955    when :shell
     956        prepareShellTestRunner
     957    else
     958        raise "Unknown test runner type: #{$testRunnerType.to_s}"
     959    end
     960end
     961
     962def prepareShellTestRunner
     963    FileUtils.cp SCRIPTS_PATH + "jsc-stress-test-helpers" + "shell-runner.sh", $runnerDir + "runscript"
     964end
     965
     966def prepareMakeTestRunner
    920967    # The goals of our parallel test runner are scalability and simplicity. The
    921968    # simplicity part is particularly important. We don't want to have to have
    … …  
    923970    #
    924971    # As such, we just pass off all of the hard work to 'make'. This creates a
    925     # dummy directory ("$outputDir/.parallel") in which we create a dummy
     972    # dummy directory ("$outputDir/.runner") in which we create a dummy
    926973    # Makefile. The Makefile has an 'all' rule that depends on all of the tests.
    927974    # That is, for each test we know we will run, there is a rule in the
    … …  
    946993    #
    947994    # In the end, this script collects all of the failures by searching for
    948     # files in the .parallel directory whose name matches /^test_fail_/, where
     995    # files in the .runner directory whose name matches /^test_fail_/, where
    949996    # the thing after the 'fail_' is the test index. Those are the files that
    950997    # would be created by the test scripts if they detect failure. We're
    … …  
    952999    # Even if two tests fail at the same time, since they're touching different
    9531000    # files we won't miss any failures.
    954    
    9551001    runIndices = []
    956     $runlist.each_with_index {
    957         | plan, index |
    958         runIndices << index
    959         plan.index = index
    960     }
    961    
    962     Dir.mkdir($parallelDir) unless $parallelDir.directory?
    963     toDelete = []
    964     Dir.foreach($parallelDir) {
    965         | filename |
    966         if filename =~ /^test_/
    967             toDelete << filename
    968         end
    969     }
    970    
    971     toDelete.each {
    972         | filename |
    973         File.unlink($parallelDir + filename)
    974     }
    975 
    9761002    $runlist.each {
    9771003        | plan |
    978         plan.writeRunScript($parallelDir + "test_script_#{plan.index}")
    979     }
    980    
    981     File.open($parallelDir + "Makefile", "w") {
     1004        runIndices << plan.index
     1005    }
     1006   
     1007    File.open($runnerDir + "Makefile", "w") {
    9821008        | outp |
    9831009        outp.puts("all: " + runIndices.map{|v| "test_done_#{v}"}.join(' '))
    … …  
    10051031puts
    10061032
    1007 def cleanParallelDirectory
     1033def cleanRunnerDirectory
    10081034    raise unless $bundle
    1009     Dir.foreach($parallelDir) {
     1035    Dir.foreach($runnerDir) {
    10101036        | filename |
    10111037        next unless filename =~ /^test_fail/
    1012         FileUtils.rm_f $parallelDir + filename
    1013     }
    1014 end
    1015 
    1016 def runParallelTestRunner
    1017     Dir.chdir($parallelDir) {
     1038        FileUtils.rm_f $runnerDir + filename
     1039    }
     1040end
     1041
     1042def runTestRunner
     1043    case $testRunnerType
     1044    when :shell
     1045        runShellTestRunner
     1046    when :make
     1047        runMakeTestRunner
     1048    else
     1049        raise "Unknown test runner type: #{$testRunnerType.to_s}"
     1050    end
     1051end
     1052
     1053def runShellTestRunner
     1054    Dir.chdir($runnerDir) {
     1055        mysys("sh", "runscript")
     1056    }
     1057end
     1058
     1059def runMakeTestRunner
     1060    Dir.chdir($runnerDir) {
    10181061        # -1 for the Makefile, and -2 for '..' and '.'
    10191062        numberOfTests = Dir.entries(".").count - 3
    … …  
    10841127    raise if $bundle
    10851128
    1086     Dir.foreach($parallelDir) {
     1129    Dir.foreach($runnerDir) {
    10871130        | filename |
    10881131        next unless filename =~ /test_fail_/
    … …  
    11061149
    11071150$outputDir = $outputDir.realpath
    1108 $parallelDir = $outputDir + ".parallel"
     1151$runnerDir = $outputDir + ".runner"
    11091152
    11101153prepareBundle unless $bundle
    1111 prepareParallelTestRunner unless $bundle
    1112 cleanParallelDirectory if $bundle
     1154
     1155puts " "
     1156
     1157prepareTestRunner unless $bundle
     1158cleanRunnerDirectory if $bundle
    11131159cleanOldResults if $bundle
    1114 runParallelTestRunner
     1160runTestRunner
    11151161cleanEmptyResultFiles
    11161162detectFailures unless $bundle
Note: See TracChangeset for help on using the changeset viewer.