Changeset 230998 in webkit


Ignore:
Timestamp:
Apr 25, 2018 9:35:34 AM (6 years ago)
Author:
Jonathan Bedard
Message:

API test harness should be Python
https://bugs.webkit.org/show_bug.cgi?id=181043
<rdar://problem/36164410>

Reviewed by David Kilzer.

Run API tests from a Python script rather than Perl. The Python API tests have
the same structure as the layout tests.

  • Scripts/run-api-tests: Changed from Perl to Python.
  • Scripts/webkitpy/api_tests: Added.
  • Scripts/webkitpy/api_tests/init.py: Added.
  • Scripts/webkitpy/api_tests/manager.py: Added.

(Manager):
(Manager.init):
(Manager._test_list_from_output): Construct a list of tests from the output of
TestWTF and TestWebKitAPI.
(Manager._find_test_subset): Given a list of all possible tests and command line
arguments, return a list of tests which match the provided arguments.
(Manager._collect_tests): Run TestWTF and TestWebKitAPI with --gtest_list_tests
to determine which tests will be run.
(Manager._print_test_result): Formatted print of a test with it's output.
(Manager._print_tests_result_with_status): Print all tests and output for tests
which ran with a specified exit status.
(Manager.run): Collect all tests, run them and then print results from the run.

  • Scripts/webkitpy/api_tests/run_api_tests.py: Added.

(main): Parse arguments, construct a port, early exit for illegal ports and
then run tests.
(run): Setup logging and printing before using the Manager to run tests.
(parse_args): Use argparse to define all options used by run-api-tests.

  • Scripts/webkitpy/api_tests/runner.py: Added.

(Runner):
(Runner.init):
(Runner._shard_tests): Split tests so that each suite runs together.
(Runner.command_for_port): Run a command on the device specified by the given
port object. Once <https://bugs.webkit.org/show_bug.cgi?id=175204> is completed,
this function would be entirely replaced by the server_process factory owned
by the port object.
(Runner.run): Run all shards in the message_pool.
(Runner.handle): Handle reports from child processes.
(Runner.result_map_by_status): Return a mapping of tests to output for tests
which exit with a specific status.
(_Worker):
(_Worker.init):
(_Worker._filter_noisy_output): Filter out objc warnings since these polite the log.
(_Worker._run_single_test): Runs a single test in a single child process.
(_Worker._run_shard_with_binary): Attempts to run a group of tests in the same
child process. If this technique fails, the remaining tests are run singly.
(_Worker.handle): Run the specified shard.

  • Scripts/webkitpy/port/base.py:

(Port):
(Port.check_api_test_build): Check if TestWTF and TestWebKitAPI are built.
(Port.environment_for_api_tests): Return the environment needed to run
the TestWebKitAPI binary.
(Port. path_to_api_test_binaries): Return a list of the path to all binaries
used when running API tests.
(Port._build_api_tests): Build TestWTF and TestWebKitAPI if required.

  • Scripts/webkitpy/port/mac.py:

(MacPort.environment_for_api_tests): Enable GuardMalloc for API tests.

  • Scripts/webkitpy/port/server_process.py:

(ServerProcess.pop_all_buffered_stdout):

Location:
trunk/Tools
Files:
5 added
6 edited

Legend:

Unmodified
Added
Removed
  • trunk/Tools/ChangeLog

    r230993 r230998  
     12018-04-25  Jonathan Bedard  <jbedard@apple.com>
     2
     3        API test harness should be Python
     4        https://bugs.webkit.org/show_bug.cgi?id=181043
     5        <rdar://problem/36164410>
     6
     7        Reviewed by David Kilzer.
     8
     9        Run API tests from a Python script rather than Perl. The Python API tests have
     10        the same structure as the layout tests.
     11
     12        * Scripts/run-api-tests: Changed from Perl to Python.
     13        * Scripts/webkitpy/api_tests: Added.
     14        * Scripts/webkitpy/api_tests/__init__.py: Added.
     15        * Scripts/webkitpy/api_tests/manager.py: Added.
     16        (Manager):
     17        (Manager.__init__):
     18        (Manager._test_list_from_output): Construct a list of tests from the output of
     19        TestWTF and TestWebKitAPI.
     20        (Manager._find_test_subset): Given a list of all possible tests and command line
     21        arguments, return a list of tests which match the provided arguments.
     22        (Manager._collect_tests): Run TestWTF and TestWebKitAPI with --gtest_list_tests
     23        to determine which tests will be run.
     24        (Manager._print_test_result): Formatted print of a test with it's output.
     25        (Manager._print_tests_result_with_status): Print all tests and output for tests
     26        which ran with a specified exit status.
     27        (Manager.run): Collect all tests, run them and then print results from the run.
     28        * Scripts/webkitpy/api_tests/run_api_tests.py: Added.
     29        (main): Parse arguments, construct a port, early exit for illegal ports and
     30        then run tests.
     31        (run): Setup logging and printing before using the Manager to run tests.
     32        (parse_args): Use argparse to define all options used by run-api-tests.
     33        * Scripts/webkitpy/api_tests/runner.py: Added.
     34        (Runner):
     35        (Runner.__init__):
     36        (Runner._shard_tests): Split tests so that each suite runs together.
     37        (Runner.command_for_port): Run a command on the device specified by the given
     38        port object. Once <https://bugs.webkit.org/show_bug.cgi?id=175204> is completed,
     39        this function would be entirely replaced by the server_process factory owned
     40        by the port object.
     41        (Runner.run): Run all shards in the message_pool.
     42        (Runner.handle): Handle reports from child processes.
     43        (Runner.result_map_by_status): Return a mapping of tests to output for tests
     44        which exit with a specific status.
     45        (_Worker):
     46        (_Worker.__init__):
     47        (_Worker._filter_noisy_output): Filter out objc warnings since these polite the log.
     48        (_Worker._run_single_test): Runs a single test in a single child process.
     49        (_Worker._run_shard_with_binary): Attempts to run a group of tests in the same
     50        child process. If this technique fails, the remaining tests are run singly.
     51        (_Worker.handle): Run the specified shard.
     52        * Scripts/webkitpy/port/base.py:
     53        (Port):
     54        (Port.check_api_test_build): Check if TestWTF and TestWebKitAPI are built.
     55        (Port.environment_for_api_tests): Return the environment needed to run
     56        the TestWebKitAPI binary.
     57        (Port. path_to_api_test_binaries): Return a list of the path to all binaries
     58        used when running API tests.
     59        (Port._build_api_tests): Build TestWTF and TestWebKitAPI if required.
     60        * Scripts/webkitpy/port/mac.py:
     61        (MacPort.environment_for_api_tests): Enable GuardMalloc for API tests.
     62        * Scripts/webkitpy/port/server_process.py:
     63        (ServerProcess.pop_all_buffered_stdout):
     64
    1652018-04-25  Carlos Garcia Campos  <cgarcia@igalia.com>
    266
  • trunk/Tools/Scripts/run-api-tests

    r226395 r230998  
    1 #!/usr/bin/env perl
     1#!/usr/bin/env python
    22
    3 # Copyright (C) 2010-2012, 2014-2015 Apple Inc. All rights reserved.
     3# Copyright (C) 2018 Apple Inc. All rights reserved.
    44#
    55# Redistribution and use in source and binary forms, with or without
    66# modification, are permitted provided that the following conditions
    77# are met:
    8 # 1. Redistributions of source code must retain the above copyright
     8# 1.  Redistributions of source code must retain the above copyright
    99#    notice, this list of conditions and the following disclaimer.
    10 # 2. Redistributions in binary form must reproduce the above copyright
     10# 2.  Redistributions in binary form must reproduce the above copyright
    1111#    notice, this list of conditions and the following disclaimer in the
    1212#    documentation and/or other materials provided with the distribution.
    1313#
    14 # THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
    15 # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
    16 # THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
    17 # PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
    18 # BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
    19 # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
    20 # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
    21 # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
    22 # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
    23 # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
    24 # THE POSSIBILITY OF SUCH DAMAGE.
     14# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND ANY
     15# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
     16# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
     17# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR ANY
     18# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
     19# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
     20# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
     21# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
     22# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
     23# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    2524
    26 use strict;
    27 use warnings;
     25"""Wrapper around webkitpy/api_tests/run_api_tests.py"""
     26from webkitpy.common import multiprocessing_bootstrap
    2827
    29 use File::Basename;
    30 use File::Spec;
    31 use FindBin;
    32 use Getopt::Long qw(:config pass_through);
    33 use IPC::Open3;
    34 use JSON::PP;
    35 use lib $FindBin::Bin;
    36 use sigtrap qw(die normal-signals);
    37 use webkitdirs;
    38 use VCSUtils;
    39 
    40 sub buildTestTool();
    41 sub dumpTestsBySuite(\@);
    42 sub listAllTests();
    43 sub runTest($$);
    44 sub runTestsBySuite(\@);
    45 sub prepareEnvironmentForRunningTestTool();
    46 sub archCommandLineArgumentsForRestrictedEnvironmentVariables();
    47 sub testToolPaths();
    48 sub writeJSONDataIfApplicable();
    49 
    50 # Defined in VCSUtils.
    51 sub possiblyColored($$);
    52 
    53 # Timeout for individual test, in sec
    54 my $timeout = 30;
    55 
    56 my $showHelp = 0;
    57 my $verbose = 0;
    58 my $showLeaks = 0;
    59 my $dumpTests = 0;
    60 my $disableTimeout = 0;
    61 my $build = 1;
    62 my $root;
    63 my $buildDefault = $build ? "build" : "do not build";
    64 my @testsFailed;
    65 my @testsTimedOut;
    66 my $wtfOnly = 0;
    67 my %testToToolMap;
    68 my %jsonData = ();
    69 my $jsonFileName;
    70 
    71 
    72 my $programName = basename($0);
    73 my $usage = <<EOF;
    74 Usage: $programName [options] [suite or test prefixes]
    75   --help                Show this help message
    76   -v|--verbose          Verbose output
    77   -d|--dump-tests       Dump the names of testcases without running them
    78   --[no-]build          Build (or do not build) unit tests prior to running (default: $buildDefault)
    79   --json-output=        Create a file at the specified path, listing test failures and timeouts in JSON format.
    80   --root=               Path to the pre-built root containing TestWebKitAPI
    81   --show-leaks          Show leaks in the output
    82   --no-timeout          Disable test timeouts
    83   --wtf-only            Only build and run TestWTF
    84 
    85 Platform options:
    86   --ios-simulator       Run tests in the iOS Simulator
    87   --simulator           DEPRECATED alias of --ios-simulator
    88 
    89 @{[ sharedCommandLineOptionsUsage(indent => 2, switchWidth => 21) ]}
    90 Examples
    91 
    92 The following command will run a single test:
    93     $programName WebKit.AboutBlank
    94 
    95 The following command will run all tests in suites that begin with 'WebKit':
    96     $programName WebKit
    97 
    98 EOF
    99 
    100 my $getOptionsResult = GetOptions(
    101     sharedCommandLineOptions(),
    102     'help' => \$showHelp,
    103     'verbose|v' => \$verbose,
    104     'show-leaks' => \$showLeaks,
    105     'no-timeout' => \$disableTimeout,
    106     'json-output=s' => \$jsonFileName,
    107     'dump|d' => \$dumpTests,
    108     'build!' => \$build,
    109     'root=s' => \$root,
    110     'wtf-only' => \$wtfOnly,
    111 );
    112 
    113 if (!$getOptionsResult || $showHelp) {
    114    print STDERR $usage;
    115    exit 1;
    116 }
    117 
    118 setConfiguration();
    119 
    120 setConfigurationProductDir(Cwd::abs_path($root)) if (defined($root));
    121 
    122 if (defined($jsonFileName)) {
    123     $jsonFileName = File::Spec->rel2abs($jsonFileName);
    124 }
    125 
    126 buildTestTool() if $build && !defined($root);
    127 setPathForRunningWebKitApp(\%ENV);
    128 
    129 my $simulatorDevice;
    130 if (willUseIOSSimulatorSDK()) {
    131     $simulatorDevice = findOrCreateSimulatorForIOSDevice(SIMULATOR_DEVICE_SUFFIX_FOR_WEBKIT_DEVELOPMENT);
    132     relaunchIOSSimulator($simulatorDevice);
    133 }
    134 my @testsToRun = listAllTests();
    135 
    136 @testsToRun = grep { my $test = $_; grep { $test =~ m/^\Q$_\E/ } @ARGV; } @testsToRun if @ARGV;
    137 
    138 if ($dumpTests) {
    139     dumpTestsBySuite(@testsToRun);
    140     exit 0;
    141 }
    142 
    143 END { shutDownIOSSimulatorDevice($simulatorDevice) if $simulatorDevice; }
    144 
    145 exit runTestsBySuite(@testsToRun);
    146 
    147 sub isSupportedPlatform()
    148 {
    149     return isAppleCocoaWebKit() || isAppleWinWebKit();
    150 }
    151 
    152 sub dumpTestsBySuite(\@)
    153 {
    154     my ($tests) = @_;
    155     print "Dumping test cases\n";
    156     print "------------------\n";
    157     my $lastSuite = "";
    158     for my $suiteAndTest (sort @$tests) {
    159         my ($suite, $test) = split(/\./, $suiteAndTest);
    160         if ($lastSuite ne $suite) {
    161             $lastSuite = $suite;
    162             print "$suite:\n";
    163         }
    164         print "   $test\n";
    165     }
    166     print "------------------\n";
    167 }
    168 
    169 sub runTestsBySuite(\@)
    170 {
    171     my ($tests) = @_;
    172     for my $suiteAndTest (sort @$tests) {
    173         my ($suite, $test) = split(/\./, $suiteAndTest);
    174         runTest($suite, $test);
    175     }
    176 
    177     if (@testsFailed) {
    178         print "\nTests that failed:\n";
    179         for my $test (@testsFailed) {
    180             print "  $test\n";
    181         }
    182     }
    183     if (@testsTimedOut) {
    184         print "\nTests that timed out:\n";
    185         for my $test (@testsTimedOut) {
    186             print "  $test\n";
    187         }
    188     }
    189 
    190     if (defined($jsonFileName)) {
    191         $jsonData{'failures'} = \@testsFailed;
    192         $jsonData{'timeouts'} = \@testsTimedOut;
    193     }
    194 
    195     writeJSONDataIfApplicable();
    196 
    197     return @testsFailed > 0 || @testsTimedOut > 0;
    198 }
    199 
    200 sub runTest($$)
    201 {
    202     my ($suite, $testName) = @_;
    203     my $test = $suite . "." . $testName;
    204 
    205     my $gtestArg = "--gtest_filter=" . $test;
    206 
    207     my $result = 0;
    208     my $timedOut = 0;
    209 
    210     die "run-api-tests is not supported on this platform.\n" unless isSupportedPlatform();
    211 
    212     local %ENV = %ENV;
    213     prepareEnvironmentForRunningTestTool();
    214 
    215     local *DEVNULL;
    216     my ($childIn, $childOut, $childErr);
    217     if ($verbose || $showLeaks) {
    218         $childErr = 0;
    219     } else {
    220         open(DEVNULL, ">", File::Spec->devnull()) or die "Failed to open /dev/null";
    221         $childErr = ">&DEVNULL";
    222     }
    223 
    224     my $pid;
    225     my @commonArguments = ($testToToolMap{$test}, $gtestArg, @ARGV);
    226     if (willUseIOSSimulatorSDK()) {
    227         $pid = open3($childIn, $childOut, $childErr, qw(xcrun --sdk iphonesimulator simctl spawn), $simulatorDevice->{UDID}, @commonArguments) or die "Failed to run test: $test.";
    228     } elsif (isAppleCocoaWebKit() && architecture()) {
    229         $pid = open3($childIn, $childOut, $childErr, "arch", "-" . architecture(), archCommandLineArgumentsForRestrictedEnvironmentVariables(), @commonArguments) or die "Failed to run test: $test.";
    230     } else {
    231         $pid = open3($childIn, $childOut, $childErr, @commonArguments) or die "Failed to run test: $test.";
    232     }
    233 
    234     eval {
    235         if ($disableTimeout) {
    236             waitpid($pid, 0);   
    237         } else {
    238             local $SIG{ALRM} = sub { die "alarm\n" };
    239             alarm $timeout;
    240             waitpid($pid, 0);
    241             alarm 0;
    242         }
    243         $result = $?;
    244     };
    245     if ($@) {
    246         die unless $@ eq "alarm\n";
    247         kill SIGTERM, $pid or kill SIGKILL, $pid;
    248         $timedOut = 1;
    249     }
    250 
    251     my @testOutput = <$childOut>;
    252     @testOutput = grep { !/^LEAK:/ } @testOutput unless $showLeaks;
    253     map { s/\*\*PASS\*\*/possiblyColored("bold green", "PASS")/eg } @testOutput;
    254     map { s/\*\*FAIL\*\*/possiblyColored("bold red", "FAIL")/eg } @testOutput;
    255 
    256     if ($result) {
    257         push @testsFailed, $test;
    258         if (!$timedOut && index("@testOutput", $test) == -1) {
    259             print STDOUT possiblyColored("bold red", "UNEXPECTEDLY EXITED"), " $test\n";
    260         }
    261     } elsif ($timedOut) {
    262         push @testsTimedOut, $test;
    263         print STDOUT possiblyColored("bold yellow", "TIMEOUT"), " $test\n";
    264     }
    265 
    266     print STDOUT @testOutput;
    267 
    268     close($childIn);
    269     close($childOut);
    270     close($childErr) unless ($verbose || $showLeaks);
    271     close(DEVNULL) unless ($verbose || $showLeaks);
    272 
    273     if ($timedOut || $result) {
    274         return $timedOut || $result;
    275     }
    276 
    277     return 0;
    278 }
    279 
    280 sub listAllTests()
    281 {
    282     my @toolOutput;
    283     my $timedOut;
    284 
    285     die "run-api-tests is not supported on this platform.\n" unless isSupportedPlatform();
    286 
    287     prepareEnvironmentForRunningTestTool();
    288 
    289     local *DEVNULL;
    290     my ($childIn, $childOut, $childErr);
    291     if ($verbose) {
    292         $childErr = ">&STDERR";
    293     } else {
    294         open(DEVNULL, ">", File::Spec->devnull()) or die "Failed to open /dev/null";
    295         $childErr = ">&DEVNULL";
    296     }
    297 
    298     my @tests = ();
    299     foreach (testToolPaths()) {
    300         my $pid;
    301         my $testTool = $_;
    302         my @commonArguments = ($testTool, "--gtest_list_tests");
    303         if (isIOSWebKit()) {
    304             $pid = open3($childIn, $childOut, $childErr, qw(xcrun --sdk iphonesimulator simctl spawn), $simulatorDevice->{UDID}, @commonArguments) or die "Failed to build list of tests!";
    305         } elsif (isAppleCocoaWebKit() && architecture()) {
    306             $pid = open3($childIn, $childOut, $childErr, "arch", "-" . architecture(), archCommandLineArgumentsForRestrictedEnvironmentVariables(), @commonArguments) or die "Failed to build list of tests!";
    307         } else {
    308             $pid = open3($childIn, $childOut, $childErr, @commonArguments) or die "Failed to build list of tests!";
    309         }
    310 
    311         close($childIn);
    312         @toolOutput = <$childOut>;
    313         close($childOut);
    314         close($childErr);
    315 
    316         waitpid($pid, 0);
    317         my $result = $?;
    318 
    319         if ($result) {
    320             print STDERR "Failed to build list of tests!--\n";
    321             exit exitStatus($result);
    322         }
    323 
    324         my $suite;
    325         for my $line (@toolOutput) {
    326            $line =~ s/[\r\n]*$//;
    327            if ($line =~ m/\.$/) {
    328               $suite = $line; # "SuiteName."
    329            } else {
    330               $line =~ s/^\s*//; # "TestName"
    331               my $fullName = $suite . $line; # "SuiteName.TestName";
    332               push @tests, $fullName;
    333               $testToToolMap{$fullName} = $testTool;
    334             }
    335         }
    336     }
    337 
    338     close(DEVNULL) unless ($verbose);
    339 
    340     return @tests;
    341 }
    342 
    343 sub buildTestTool()
    344 {
    345     my $originalCwd = getcwd();
    346 
    347     chdirWebKit();
    348 
    349     my $buildTestTool = "build-api-tests";
    350     print STDERR "Running $buildTestTool\n";
    351 
    352     local *DEVNULL;
    353     my ($childIn, $childOut, $childErr);
    354     if ($verbose) {
    355         # When not quiet, let the child use our stdout/stderr.
    356         $childOut = ">&STDOUT";
    357         $childErr = ">&STDERR";
    358     } else {
    359         open(DEVNULL, ">", File::Spec->devnull()) or die "Failed to open /dev/null";
    360         $childOut = ">&DEVNULL";
    361         $childErr = ">&DEVNULL";
    362     }
    363 
    364     my @args = argumentsForConfiguration();
    365     if ($wtfOnly) {
    366         push @args, "--wtf-only";
    367     }
    368 
    369     my $pathToBuildTestTool = File::Spec->catfile("Tools", "Scripts", $buildTestTool);
    370     my $buildProcess = open3($childIn, $childOut, $childErr, "perl", $pathToBuildTestTool, @args) or die "Failed to run " . $buildTestTool;
    371 
    372     close($childIn);
    373     close($childOut);
    374     close($childErr);
    375     close(DEVNULL) unless ($verbose);
    376 
    377     waitpid($buildProcess, 0);
    378     my $buildResult = $?;
    379 
    380     if ($buildResult) {
    381         print STDERR "Compiling TestWebKitAPI failed!\n";
    382         exit exitStatus($buildResult);
    383     }
    384 
    385     chdir $originalCwd;
    386 }
    387 
    388 sub prepareEnvironmentForRunningTestTool()
    389 {
    390     return unless isAppleCocoaWebKit();
    391 
    392     if (willUseIOSSimulatorSDK()) {
    393         my %simulatorENV;
    394         {
    395             local %ENV;
    396             setupIOSWebKitEnvironment(productDir());
    397             %simulatorENV = %ENV;
    398         }
    399         # Prefix the environment variables with SIMCTL_CHILD_ per `xcrun simctl help launch`.
    400         foreach my $key (keys %simulatorENV) {
    401             $ENV{"SIMCTL_CHILD_$key"} = $simulatorENV{$key};
    402         }
    403         return;
    404     }
    405     setupMacWebKitEnvironment(productDir());
    406 }
    407 
    408 sub testToolPaths()
    409 {
    410     if (!isAppleWinWebKit()) {
    411         my @toolPaths = ();
    412         if (!$wtfOnly) {
    413             push @toolPaths, File::Spec->catfile(productDir(), "TestWebKitAPI");
    414         }
    415         push @toolPaths, File::Spec->catfile(productDir(), "TestWTF");
    416         return @toolPaths;
    417     }
    418 
    419     my $binDir = isWin64() ? "bin64" : "bin32";
    420     my $pathWTF = File::Spec->catfile(productDir(), $binDir, "TestWTF");
    421     my $pathWebCore = File::Spec->catfile(productDir(), $binDir, "TestWebCore");
    422     my $pathWebKit = File::Spec->catfile(productDir(), $binDir, "TestWebKitLegacy");
    423 
    424     my $suffix;
    425     if (configuration() eq "Debug_All") {
    426         $suffix = "_debug";
    427     } else {
    428         $suffix = "";
    429     }
    430     return ("$pathWTF$suffix.exe", "$pathWebCore$suffix.exe", "$pathWebKit$suffix.exe");
    431 }
    432 
    433 sub writeJSONDataIfApplicable()
    434 {
    435     if (defined($jsonFileName)) {
    436         open(my $fileHandler, ">", $jsonFileName) or die;
    437         print $fileHandler "${\encode_json(\%jsonData)}\n";
    438         close($fileHandler);
    439     }
    440 }
     28multiprocessing_bootstrap.run('webkitpy', 'api_tests', 'run_api_tests.py')
  • trunk/Tools/Scripts/webkitpy/port/base.py

    r230601 r230998  
    241241        return True
    242242
     243    def check_api_test_build(self):
     244        if not self._root_was_set and self.get_option('build') and not self._build_api_tests():
     245            return False
     246        if self.get_option('install') and not self._check_port_build():
     247            return False
     248
     249        for binary in self.path_to_api_test_binaries():
     250            if not self._filesystem.exists(binary):
     251                _log.error('{} was not found at {}'.format(os.path.basename(binary), binary))
     252                return False
     253        return True
     254
     255    def environment_for_api_tests(self):
     256        build_root_path = str(self._build_path())
     257        return {
     258            'DYLD_LIBRARY_PATH': build_root_path,
     259            '__XPC_DYLD_LIBRARY_PATH': build_root_path,
     260            'DYLD_FRAMEWORK_PATH': build_root_path,
     261            '__XPC_DYLD_FRAMEWORK_PATH': build_root_path,
     262        }
     263
    243264    def _check_driver(self):
    244265        driver_path = self._path_to_driver()
     
    13371358        return self._build_path('ImageDiff')
    13381359
     1360    def path_to_api_test_binaries(self):
     1361        binary_names = ['TestWTF']
     1362        if self.host.platform.is_win():
     1363            binary_names += ['TestWebCore', 'TestWebKitLegacy']
     1364        else:
     1365            binary_names += ['TestWebKitAPI']
     1366        binary_paths = [self._build_path(binary_name) for binary_name in binary_names]
     1367        if self.host.platform.is_win():
     1368            binary_paths = [os.path.splitext(binary_path)[0] + '.exe' for binary_path in binary_paths]
     1369        return binary_paths
     1370
    13391371    def _path_to_lighttpd(self):
    13401372        """Returns the path to the LigHTTPd binary.
     
    14451477            if self.get_option('webkit_test_runner'):
    14461478                self._run_script("build-webkittestrunner", args=self._build_driver_flags(), env=env)
     1479        except ScriptError as e:
     1480            _log.error(e.message_with_output(output_limit=None))
     1481            return False
     1482        return True
     1483
     1484    def _build_api_tests(self):
     1485        environment = self.host.copy_current_environment().to_dictionary()
     1486        try:
     1487            self._run_script('build-api-tests', args=self._build_driver_flags(), env=environment)
    14471488        except ScriptError as e:
    14481489            _log.error(e.message_with_output(output_limit=None))
  • trunk/Tools/Scripts/webkitpy/port/ios_simulator.py

    r229379 r230998  
    109109        SimulatedDeviceManager.tear_down(self.host)
    110110
     111    def environment_for_api_tests(self):
     112        no_prefix = super(IOSSimulatorPort, self).environment_for_api_tests()
     113        result = {}
     114        SIMCTL_ENV_PREFIX = 'SIMCTL_CHILD_'
     115        for value in no_prefix:
     116            if not value.startswith(SIMCTL_ENV_PREFIX):
     117                result[SIMCTL_ENV_PREFIX + value] = no_prefix[value]
     118            else:
     119                result[value] = no_prefix[value]
     120        return result
     121
    111122    def setup_environ_for_server(self, server_name=None):
    112123        _log.debug("setup_environ_for_server")
  • trunk/Tools/Scripts/webkitpy/port/mac.py

    r229955 r230998  
    136136        return config_map
    137137
     138    def environment_for_api_tests(self):
     139        result = super(MacPort, self).environment_for_api_tests()
     140        if self.get_option('guard_malloc'):
     141            result['DYLD_INSERT_LIBRARIES'] = '/usr/lib/libgmalloc.dylib'
     142            result['__XPC_DYLD_INSERT_LIBRARIES'] = '/usr/lib/libgmalloc.dylib'
     143        return result
     144
    138145    def setup_environ_for_server(self, server_name=None):
    139146        env = super(MacPort, self).setup_environ_for_server(server_name)
  • trunk/Tools/Scripts/webkitpy/port/server_process.py

    r225733 r230998  
    187187        return None
    188188
     189    def pop_all_buffered_stdout(self):
     190        return self._pop_output_bytes(len(self._output))
     191
    189192    def pop_all_buffered_stderr(self):
    190193        return self._pop_error_bytes(len(self._error))
Note: See TracChangeset for help on using the changeset viewer.