Changes between Initial Version and Version 1 of LayoutTestResultFallbackOrder

Apr 13, 2011 10:05:23 AM (11 years ago)



  • LayoutTestResultFallbackOrder

    v1 v1  
     1Why Layout Tests search for results the way they do
     3Here are the results fallback order for chromium-mac (as of April, 2011):
     6 'leopard': ['chromium-mac-leopard', 'chromium-mac-snowleopard', 'chromium-mac', 'chromium',
     7             'mac-leopard', 'mac-snowleopard', 'mac'],
     8 'snowleopard': ['chromium-mac-snowleopard', 'chromium-mac', 'chromium',
     9                 'mac-snowleopard', 'mac'],
     12A common question is why chromium-mac-snowleopard and mac-snowleopard are in the fallback list for leopard.
     14Here's the logic behind the layout test paths:
     16Assume you have three tests called foo, bar, and baz and suppose we're
     17a couple of years ago, when Leopard was the most recent version of the
     18Mac. The generic version of a platform always contains the "future" or
     19"latest" version of the results.
     21LayoutTests/platform/chromium-mac contains:
     22  foo-expected.txt  r1.1
     23  bar-expected.txt r.1.1
     24  baz-expected.txt r.1.1
     26Now Snow Leopard comes out, and foo produces different results. You
     27want SL to find the new file, and the other two to find the old file.
     28We put the new file in the "future" version, and move the older file
     29into the directory matching the last platform that passed it:
     32  foo-expected.txt r1.2
     33  bar-expected.txt r1.1
     34  baz-expected.txt r1.1
     36  foo-expected.txt r1.1
     38SL only looks in the -mac dir, and Leopard looks in -mac-leopard, then -mac.
     39Now Lion comes out, and bar changes, and foo changes again. We can
     40leave the -leopard directory alone, and create a new -snowleopard
     41directory, and only have to move the two failing tests there:
     44  foo-expected.txt r1.3
     45  bar-expected.txt r1.2
     46  baz-expected.txt r1.1
     48  foo-expected.txt r1.2
     49  bar-expected.txt r1.1
     51  foo-expected.txt r1.1
     53Lion only looks in -mac, SL looks in -mac-snowleopard, then -mac, and
     54Leopard looks in -leopard, then -snowleopard, then -mac.
     55If Leopard didn't look in the snowleopard directory, then we would
     56have to copy versions of files more places every time a new release
     57came out. This approach minimizes the amount of file shuffling we have
     58to do, and keeps most of the files in platform/chromium-mac. But, it
     59means that version X needs to look in every newer version in order to
     60find the right files.
     62Another way to think of it is that directory X-1 contains all and only
     63the files that were different between versions X-1 and X, and then
     64realize that this is, in a sense, associative (but not commutative).
     65So, if you want to find the differences between Leopard (Lion-2) and
     66Lion, you can look for delta(Lion-2, Lion-1) then delta(Lion-1, Lion)
     67(but you can't reverse the order).
     69Also, in Chromium we try to match the upstream baselines where
     70possible (but we prioritize Chromium results over upstream results),
     71so we look in both the chromium version-specific path and then the
     72upstream version-specific path.