Changes between Version 3 and Version 4 of LayoutTestResultFallbackOrder


Ignore:
Timestamp:
Jul 7, 2011 4:35:17 PM (9 years ago)
Author:
dpranke@chromium.org
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • LayoutTestResultFallbackOrder

    v3 v4  
    1 == Why layout tests search for results the way they do ==
    2 
    3 Here are the results fallback order for chromium-mac (as of April, 2011):
    4 
    5 {{{
    6  'leopard': ['chromium-mac-leopard', 'chromium-mac-snowleopard', 'chromium-mac', 'chromium',
    7              'mac-leopard', 'mac-snowleopard', 'mac'],
    8  'snowleopard': ['chromium-mac-snowleopard', 'chromium-mac', 'chromium',
    9                  'mac-snowleopard', 'mac'],
    10 }}}
    11 
    12 A common question is why chromium-mac-snowleopard and mac-snowleopard are in the fallback list for leopard.
    13 
    14 Here's the logic behind the layout test paths:
    15 
    16 Assume you have three tests called foo, bar, and baz and suppose we're
    17 a couple of years ago, when Leopard was the most recent version of the
    18 Mac. The generic version of a platform always contains the "future" or
    19 "latest" version of the results.
    20 
    21 
    22 {{{
    23 LayoutTests/platform/chromium-mac contains:
    24   foo-expected.txt  r1.1
    25   bar-expected.txt r.1.1
    26   baz-expected.txt r.1.1
    27 }}}
    28 
    29 Now Snow Leopard comes out, and foo produces different results. You
    30 want SL to find the new file, and the other two to find the old file.
    31 We put the new file in the "future" version, and move the older file
    32 into the directory matching the last platform that passed it:
    33 
    34 {{{
    35 platform/chromium-mac:
    36   foo-expected.txt r1.2
    37   bar-expected.txt r1.1
    38   baz-expected.txt r1.1
    39 platform/chromium-mac-leopard:
    40   foo-expected.txt r1.1
    41 }}}
    42 
    43 SL only looks in the -mac dir, and Leopard looks in -mac-leopard, then -mac.
    44 Now Lion comes out, and bar changes, and foo changes again. We can
    45 leave the -leopard directory alone, and create a new -snowleopard
    46 directory, and only have to move the two failing tests there:
    47 
    48 {{{
    49 platform/chromium-mac:
    50   foo-expected.txt r1.3
    51   bar-expected.txt r1.2
    52   baz-expected.txt r1.1
    53 platform/chromium-mac-snowleopard:
    54   foo-expected.txt r1.2
    55   bar-expected.txt r1.1
    56 platform/chromium-mac-leopard:
    57   foo-expected.txt r1.1
    58 }}}
    59 
    60 Lion only looks in -mac, SL looks in -mac-snowleopard, then -mac, and
    61 Leopard looks in -leopard, then -snowleopard, then -mac.
    62 If Leopard didn't look in the snowleopard directory, then we would
    63 have to copy versions of files more places every time a new release
    64 came out. This approach minimizes the amount of file shuffling we have
    65 to do, and keeps most of the files in platform/chromium-mac. But, it
    66 means that version X needs to look in every newer version in order to
    67 find the right files.
    68 
    69 Another way to think of it is that directory X-1 contains all and only
    70 the files that were different between versions X-1 and X, and then
    71 realize that this is, in a sense, associative (but not commutative).
    72 So, if you want to find the differences between Leopard (Lion-2) and
    73 Lion, you can look for delta(Lion-2, Lion-1) then delta(Lion-1, Lion)
    74 (but you can't reverse the order).
    75 
    76 Also, in Chromium we try to match the upstream baselines where
    77 possible (but we prioritize Chromium results over upstream results),
    78 so we look in both the chromium version-specific path and then the
    79 upstream version-specific path.
     1This page is obsolete. See [wiki:LayoutTestSearchPath] instead.