| 1 | Why Layout Tests search for results the way they do |
| 2 | |
| 3 | Here are the results fallback order for chromium-mac (as of April, 2011): |
| 4 | |
| 5 | {{{ |
| 6 | 'leopard': ['chromium-mac-leopard', 'chromium-mac-snowleopard', 'chromium-mac', 'chromium', |
| 7 | 'mac-leopard', 'mac-snowleopard', 'mac'], |
| 8 | 'snowleopard': ['chromium-mac-snowleopard', 'chromium-mac', 'chromium', |
| 9 | 'mac-snowleopard', 'mac'], |
| 10 | }}} |
| 11 | |
| 12 | A common question is why chromium-mac-snowleopard and mac-snowleopard are in the fallback list for leopard. |
| 13 | |
| 14 | Here's the logic behind the layout test paths: |
| 15 | |
| 16 | Assume you have three tests called foo, bar, and baz and suppose we're |
| 17 | a couple of years ago, when Leopard was the most recent version of the |
| 18 | Mac. The generic version of a platform always contains the "future" or |
| 19 | "latest" version of the results. |
| 20 | |
| 21 | LayoutTests/platform/chromium-mac contains: |
| 22 | foo-expected.txt r1.1 |
| 23 | bar-expected.txt r.1.1 |
| 24 | baz-expected.txt r.1.1 |
| 25 | |
| 26 | Now Snow Leopard comes out, and foo produces different results. You |
| 27 | want SL to find the new file, and the other two to find the old file. |
| 28 | We put the new file in the "future" version, and move the older file |
| 29 | into the directory matching the last platform that passed it: |
| 30 | |
| 31 | platform/chromium-mac: |
| 32 | foo-expected.txt r1.2 |
| 33 | bar-expected.txt r1.1 |
| 34 | baz-expected.txt r1.1 |
| 35 | platform/chromium-mac-leopard: |
| 36 | foo-expected.txt r1.1 |
| 37 | |
| 38 | SL only looks in the -mac dir, and Leopard looks in -mac-leopard, then -mac. |
| 39 | Now Lion comes out, and bar changes, and foo changes again. We can |
| 40 | leave the -leopard directory alone, and create a new -snowleopard |
| 41 | directory, and only have to move the two failing tests there: |
| 42 | |
| 43 | platform/chromium-mac: |
| 44 | foo-expected.txt r1.3 |
| 45 | bar-expected.txt r1.2 |
| 46 | baz-expected.txt r1.1 |
| 47 | platform/chromium-mac-snowleopard: |
| 48 | foo-expected.txt r1.2 |
| 49 | bar-expected.txt r1.1 |
| 50 | platform/chromium-mac-leopard: |
| 51 | foo-expected.txt r1.1 |
| 52 | |
| 53 | Lion only looks in -mac, SL looks in -mac-snowleopard, then -mac, and |
| 54 | Leopard looks in -leopard, then -snowleopard, then -mac. |
| 55 | If Leopard didn't look in the snowleopard directory, then we would |
| 56 | have to copy versions of files more places every time a new release |
| 57 | came out. This approach minimizes the amount of file shuffling we have |
| 58 | to do, and keeps most of the files in platform/chromium-mac. But, it |
| 59 | means that version X needs to look in every newer version in order to |
| 60 | find the right files. |
| 61 | |
| 62 | Another way to think of it is that directory X-1 contains all and only |
| 63 | the files that were different between versions X-1 and X, and then |
| 64 | realize that this is, in a sense, associative (but not commutative). |
| 65 | So, if you want to find the differences between Leopard (Lion-2) and |
| 66 | Lion, you can look for delta(Lion-2, Lion-1) then delta(Lion-1, Lion) |
| 67 | (but you can't reverse the order). |
| 68 | |
| 69 | Also, in Chromium we try to match the upstream baselines where |
| 70 | possible (but we prioritize Chromium results over upstream results), |
| 71 | so we look in both the chromium version-specific path and then the |
| 72 | upstream version-specific path. |