Changes between Initial Version and Version 1 of LayoutTestResultFallbackOrder

Timestamp:

Apr 13, 2011 10:05:23 AM (13 years ago)

Author:

tony@chromium.org

Comment:

from ​http://groups.google.com/a/chromium.org/group/chromium-dev/browse_thread/thread/52a01dd2c7380457/a6f57f5b912b9cc8?lnk=raot&pli=1

LayoutTestResultFallbackOrder

@@ +1 @@
+Why Layout Tests search for results the way they do
+
+Here are the results fallback order for chromium-mac (as of April, 2011):
+
+{{{
+ 'leopard': ['chromium-mac-leopard', 'chromium-mac-snowleopard', 'chromium-mac', 'chromium', 
+             'mac-leopard', 'mac-snowleopard', 'mac'], 
+ 'snowleopard': ['chromium-mac-snowleopard', 'chromium-mac', 'chromium', 
+                 'mac-snowleopard', 'mac'], 
+}}}
+
+A common question is why chromium-mac-snowleopard and mac-snowleopard are in the fallback list for leopard.
+
+Here's the logic behind the layout test paths: 
+
+Assume you have three tests called foo, bar, and baz and suppose we're 
+a couple of years ago, when Leopard was the most recent version of the 
+Mac. The generic version of a platform always contains the "future" or 
+"latest" version of the results. 
+
+LayoutTests/platform/chromium-mac contains: 
+  foo-expected.txt  r1.1 
+  bar-expected.txt r.1.1 
+  baz-expected.txt r.1.1 
+
+Now Snow Leopard comes out, and foo produces different results. You 
+want SL to find the new file, and the other two to find the old file. 
+We put the new file in the "future" version, and move the older file 
+into the directory matching the last platform that passed it: 
+
+platform/chromium-mac: 
+  foo-expected.txt r1.2 
+  bar-expected.txt r1.1 
+  baz-expected.txt r1.1 
+platform/chromium-mac-leopard: 
+  foo-expected.txt r1.1 
+
+SL only looks in the -mac dir, and Leopard looks in -mac-leopard, then -mac. 
+Now Lion comes out, and bar changes, and foo changes again. We can 
+leave the -leopard directory alone, and create a new -snowleopard 
+directory, and only have to move the two failing tests there: 
+
+platform/chromium-mac: 
+  foo-expected.txt r1.3 
+  bar-expected.txt r1.2 
+  baz-expected.txt r1.1 
+platform/chromium-mac-snowleopard: 
+  foo-expected.txt r1.2 
+  bar-expected.txt r1.1 
+platform/chromium-mac-leopard: 
+  foo-expected.txt r1.1 
+
+Lion only looks in -mac, SL looks in -mac-snowleopard, then -mac, and 
+Leopard looks in -leopard, then -snowleopard, then -mac. 
+If Leopard didn't look in the snowleopard directory, then we would 
+have to copy versions of files more places every time a new release 
+came out. This approach minimizes the amount of file shuffling we have 
+to do, and keeps most of the files in platform/chromium-mac. But, it 
+means that version X needs to look in every newer version in order to 
+find the right files. 
+
+Another way to think of it is that directory X-1 contains all and only 
+the files that were different between versions X-1 and X, and then 
+realize that this is, in a sense, associative (but not commutative). 
+So, if you want to find the differences between Leopard (Lion-2) and 
+Lion, you can look for delta(Lion-2, Lion-1) then delta(Lion-1, Lion) 
+(but you can't reverse the order). 
+
+Also, in Chromium we try to match the upstream baselines where 
+possible (but we prioritize Chromium results over upstream results), 
+so we look in both the chromium version-specific path and then the 
+upstream version-specific path.