⚠ Archived content — this site is no longer maintained.   Current WebKit documentation is at docs.webkit.org.

Changeset 99377 in webkit


Ignore:
Timestamp:
Nov 6, 2011, 11:48:20 AM (15 years ago)
Author:
eric@webkit.org
Message:

Clarify how the symbol and runtime-feature based test exclusion works and cleanup the code a bit
​https://bugs.webkit.org/show_bug.cgi?id=66078

Reviewed by Adam Barth.

The runtime feature detection was fixed to work in bug 64472.
In this bug I moved the symbol-based feature detection from
popen() to Executive.run_command and cleaned up the callers
and unittests to make sure that we're correctly parsing the
nm output correctly. The old code happened to work even though
the runtime-features path was using "str in list" and the
symbol features path was using "str in str" and it just happened
to do what we wanted to. Now runtime features and symbol feature
blacklists are computed separately (and with better documentation).

This system remains confusing, partially because these are black-lists
which are amended to whatever static blacklist may exist for the
port as part of a Skipped list file.

For example, notice how the runtime feature list only has directory
blacklists for a couple features. If all features are off,
how do we skip enough tests with only 2 entries in the blacklist map?
The answer is that Windows is the only port to use runtime feature
detection, and the win/Skipped file turns off all the other features
statically (like mathml, mhtml, wss, etc.) where as some other ports (like AppleMac)
which use symbol-based feature detection turn of mathml, wcss, etc
using the blacklists found in _missing_symbol_to_skipped_tests.

I also noticed a couple places where we still referenced xhtmlmp
even though support for such has been removed from WebKit. Removed those.

This should result in no functional change.

  • Scripts/webkitpy/layout_tests/port/gtk.py:
    • Use self._filesystem instead of os.path
  • Scripts/webkitpy/layout_tests/port/webkit.py:
  • Scripts/webkitpy/layout_tests/port/webkit_unittest.py:
Location:
trunk/Tools
Files:
6 edited

Legend:

Unmodified
Added
Removed

  • trunk/Tools/ChangeLog

    r99376 r99377  
     12011-11-06  Eric Seidel  <eric@webkit.org>
     2
     3        Clarify how the symbol and runtime-feature based test exclusion works and cleanup the code a bit
     4        https://bugs.webkit.org/show_bug.cgi?id=66078
     5
     6        Reviewed by Adam Barth.
     7
     8        The runtime feature detection was fixed to work in bug 64472.
     9        In this bug I moved the symbol-based feature detection from
     10        popen() to Executive.run_command and cleaned up the callers
     11        and unittests to make sure that we're correctly parsing the
     12        nm output correctly.  The old code happened to work even though
     13        the runtime-features path was using "str in list" and the
     14        symbol features path was using "str in str" and it just happened
     15        to do what we wanted to.  Now runtime features and symbol feature
     16        blacklists are computed separately (and with better documentation).
     17
     18        This system remains confusing, partially because these are black-lists
     19        which are amended to whatever static blacklist may exist for the
     20        port as part of a Skipped list file.
     21
     22        For example, notice how the runtime feature list only has directory
     23        blacklists for a couple features.  If all features are off,
     24        how do we skip enough tests with only 2 entries in the blacklist map?
     25        The answer is that Windows is the only port to use runtime feature
     26        detection, and the win/Skipped file turns off all the other features
     27        statically (like mathml, mhtml, wss, etc.) where as some other ports (like AppleMac)
     28        which use symbol-based feature detection turn of mathml, wcss, etc
     29        using the blacklists found in _missing_symbol_to_skipped_tests.
     30
     31        I also noticed a couple places where we still referenced xhtmlmp
     32        even though support for such has been removed from WebKit.  Removed those.
     33
     34        This should result in no functional change.
     35
     36        * Scripts/webkitpy/layout_tests/port/gtk.py:
     37         - Use self._filesystem instead of os.path
     38        * Scripts/webkitpy/layout_tests/port/webkit.py:
     39        * Scripts/webkitpy/layout_tests/port/webkit_unittest.py:
     40
    1412011-11-06  Dan Bernstein  <mitz@apple.com>
    242

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/gtk.py

    r98195 r99377  
    118118        for library in gtk_library_names:
    119119            full_library = self._build_path(".libs", library)
    120             if os.path.isfile(full_library):
     120            if self._filesystem.isfile(full_library):
    121121                return full_library
    122122        return None

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/test_files.py

    r90543 r99377  
    4444
    4545# When collecting test cases, we include any file with these extensions.
    46 _supported_file_extensions = set(['.html', '.shtml', '.xml', '.xhtml', '.xhtmlmp', '.pl',
     46_supported_file_extensions = set(['.html', '.shtml', '.xml', '.xhtml', '.pl',
    4747                                  '.htm', '.php', '.svg', '.mht'])
    4848# When collecting test cases, skip these directories

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit.py

    r98833 r99377  
    249249            output = self._executive.run_command(supported_features_command, error_handler=Executive.ignore_error)
    250250        except OSError, e:
    251             _log.warn("Exception runnig driver: %s, %s.  Driver must be built before calling WebKitPort.test_expectations()." % (supported_features_command, e))
    252             return []
     251            _log.warn("Exception running driver: %s, %s.  Driver must be built before calling WebKitPort.test_expectations()." % (supported_features_command, e))
     252            return None
    253253
    254254        # Note: win/DumpRenderTree.cpp does not print a leading space before the features_string.
    255255        match_object = re.match("SupportedFeatures:\s*(?P<features_string>.*)\s*", output)
    256256        if not match_object:
    257             return []
     257            return None
    258258        return match_object.group('features_string').split(' ')
    259259
    260     def _supported_symbol_list(self):
    261         """Return the supported symbols of WebCore."""
     260    def _webcore_symbols_string(self):
    262261        webcore_library_path = self._path_to_webcore_library()
    263262        if not webcore_library_path:
    264             return []
    265         symbol_list = ' '.join(os.popen("nm " + webcore_library_path).readlines())
    266         return symbol_list
    267 
    268     def _directories_for_features(self):
    269         """Return the supported feature dictionary. The keys are the
    270         features and the values are the directories in lists."""
    271         directories_for_features = {
     263            return None
     264        try:
     265            return self._executive.run_command('nm', webcore_library_path, error_handler=Executive.ignore_error)
     266        except OSError, e:
     267            _log.warn("Failed to run nm: %s.  Can't determine WebCore supported features." % e)
     268        return None
     269
     270    # Ports which use run-time feature detection should define this method and return
     271    # a dictionary mapping from Feature Names to skipped directoires.  NRWT will
     272    # run DumpRenderTree --print-supported-features and parse the output.
     273    # If the Feature Names are not found in the output, the corresponding directories
     274    # will be skipped.
     275    def _missing_feature_to_skipped_tests(self):
     276        """Return the supported feature dictionary. Keys are feature names and values
     277        are the lists of directories to skip if the feature name is not matched."""
     278        # FIXME: This list matches WebKitWin and should be moved onto the Win port.
     279        return {
    272280            "Accelerated Compositing": ["compositing"],
    273281            "3D Rendering": ["animations/3d", "transforms/3d"],
    274282        }
    275         return directories_for_features
    276 
    277     def _directories_for_symbols(self):
    278         """Return the supported feature dictionary. The keys are the
    279         symbols and the values are the directories in lists."""
    280         directories_for_symbol = {
     283
     284    # Ports which use compile-time feature detection should define this method and return
     285    # a dictionary mapping from symbol substrings to possibly disabled test directories.
     286    # When the symbol substrings are not matched, the directories will be skipped.
     287    # If ports don't ever enable certain features, then those directories can just be
     288    # in the Skipped list instead of compile-time-checked here.
     289    def _missing_symbol_to_skipped_tests(self):
     290        """Return the supported feature dictionary. The keys are symbol-substrings
     291        and the values are the lists of directories to skip if that symbol is missing."""
     292        return {
    281293            "MathMLElement": ["mathml"],
    282294            "GraphicsLayer": ["compositing"],
    … …  
    285297            "MHTMLArchive": ["mhtml"],
    286298        }
    287         return directories_for_symbol
    288299
    289300    def _skipped_tests_for_unsupported_features(self):
    290         """Return the directories of unsupported tests. Search for the
    291         symbols in the symbol_list, if found add the corresponding
    292         directories to the skipped directory list."""
    293         feature_list = self._runtime_feature_list()
    294         directories = self._directories_for_features()
    295 
    296         # if DRT feature detection not supported
    297         if not feature_list:
    298             feature_list = self._supported_symbol_list()
    299             directories = self._directories_for_symbols()
    300 
    301         if not feature_list:
    302             return []
    303 
    304         skipped_directories = [directories[feature]
    305                               for feature in directories.keys()
    306                               if feature not in feature_list]
    307         return reduce(operator.add, skipped_directories)
     301        # If the port supports runtime feature detection, disable any tests
     302        # for features missing from the runtime feature list.
     303        supported_feature_list = self._runtime_feature_list()
     304        # If _runtime_feature_list returns a non-None value, then prefer
     305        # runtime feature detection over static feature detection.
     306        if supported_feature_list is not None:
     307            return reduce(operator.add, [directories for feature, directories in self._missing_feature_to_skipped_tests().items() if feature not in supported_feature_list])
     308
     309        # Runtime feature detection not supported, fallback to static dectection:
     310        # Disable any tests for symbols missing from the webcore symbol string.
     311        webcore_symbols_string = self._webcore_symbols_string()
     312        if webcore_symbols_string is not None:
     313            return reduce(operator.add, [directories for symbol_substring, directories in self._missing_symbol_to_skipped_tests().items() if symbol_substring not in webcore_symbols_string], [])
     314        # Failed to get any runtime or symbol information, don't skip any tests.
     315        return []
    308316
    309317    def _tests_from_skipped_file_contents(self, skipped_file_contents):

  • trunk/Tools/Scripts/webkitpy/layout_tests/port/webkit_unittest.py

    r99140 r99377  
    4444    port_name = "testwebkitport"
    4545
    46     def __init__(self, symbol_list=None, feature_list=None,
     46    def __init__(self, symbols_string=None, feature_list=None,
    4747                 expectations_file=None, skips_file=None,
    4848                 executive=None, filesystem=None, user=None,
    4949                 **kwargs):
    50         self.symbol_list = symbol_list
    51         self.feature_list = feature_list
     50        self.symbols_string = symbols_string  # Passing "" disables all staticly-detectable features.
     51        self.feature_list = feature_list  # Passing [] disables all runtime-detectable features.
    5252        executive = executive or MockExecutive(should_log=False)
    5353        filesystem = filesystem or MockFileSystem()
    … …  
    6161        return self.feature_list
    6262
    63     def _supported_symbol_list(self):
    64         return self.symbol_list
     63    def _webcore_symbols_string(self):
     64        return self.symbols_string
    6565
    6666    def _tests_for_other_platforms(self):
    … …  
    100100
    101101    def test_skipped_directories_for_symbols(self):
    102         supported_symbols = ["GraphicsLayer", "WebCoreHas3DRendering", "fooSymbol"]
    103         expected_directories = set(["mathml", "fast/canvas/webgl", "compositing/webgl", "http/tests/canvas/webgl", "mhtml"])
    104         result_directories = set(TestWebKitPort(supported_symbols, None)._skipped_tests_for_unsupported_features())
     102        # This first test confirms that the commonly found symbols result in the expected skipped directories.
     103        symbols_string = " ".join(["GraphicsLayer", "WebCoreHas3DRendering", "isXHTMLMPDocument", "fooSymbol"])
     104        expected_directories = set([
     105            "mathml",  # Requires MathMLElement
     106            "fast/canvas/webgl",  # Requires WebGLShader
     107            "compositing/webgl",  # Requires WebGLShader
     108            "http/tests/canvas/webgl",  # Requires WebGLShader
     109            "mhtml",  # Requires MHTMLArchive
     110        ])
     111
     112        result_directories = set(TestWebKitPort(symbols_string, None)._skipped_tests_for_unsupported_features())
     113        self.assertEqual(result_directories, expected_directories)
     114
     115        # Test that the nm string parsing actually works:
     116        symbols_string = """
     117000000000124f498 s __ZZN7WebCore13GraphicsLayer12replaceChildEPS0_S1_E19__PRETTY_FUNCTION__
     118000000000124f500 s __ZZN7WebCore13GraphicsLayer13addChildAboveEPS0_S1_E19__PRETTY_FUNCTION__
     119000000000124f670 s __ZZN7WebCore13GraphicsLayer13addChildBelowEPS0_S1_E19__PRETTY_FUNCTION__
     120"""
     121        # Note 'compositing' is not in the list of skipped directories (hence the parsing of GraphicsLayer worked):
     122        expected_directories = set(['mathml', 'transforms/3d', 'compositing/webgl', 'fast/canvas/webgl', 'animations/3d', 'mhtml', 'http/tests/canvas/webgl'])
     123        result_directories = set(TestWebKitPort(symbols_string, None)._skipped_tests_for_unsupported_features())
    105124        self.assertEqual(result_directories, expected_directories)
    106125
    … …  
    108127        port = WebKitPort(executive=MockExecutive())
    109128        port._executive.run_command = lambda command, cwd=None, error_handler=None: "Nonsense"
    110         self.assertEquals(port._runtime_feature_list(), [])
     129        # runtime_features_list returns None when its results are meaningless (it couldn't run DRT or parse the output, etc.)
     130        self.assertEquals(port._runtime_feature_list(), None)
    111131        port._executive.run_command = lambda command, cwd=None, error_handler=None: "SupportedFeatures:foo bar"
    112132        self.assertEquals(port._runtime_feature_list(), ['foo', 'bar'])
    … …  
    119139
    120140    def test_skipped_layout_tests(self):
    121         self.assertEqual(TestWebKitPort(None, None).skipped_layout_tests(), set(["media"]))
     141        self.assertEqual(TestWebKitPort(None, None).skipped_layout_tests(), set(['media']))
    122142
    123143    def test_skipped_file_search_paths(self):

  • trunk/Tools/Scripts/webkitpy/layout_tests/servers/lighttpd.conf

    r89801 r99377  
    2222  ".htm"          =>      "text/html",
    2323  ".xhtml"        =>      "application/xhtml+xml",
    24   ".xhtmlmp"      =>      "application/vnd.wap.xhtml+xml",
    2524  ".js"           =>      "application/x-javascript",
    2625  ".log"          =>      "text/plain",
Note: See TracChangeset for help on using the changeset viewer.