Scope

Easy import (and re-import) of available test suites on the web, in particular W3C CSS and WPT test-suites.

Importing new tests

1. Update LayoutTests/imported/w3c/resources/ImportExpectations to activate the import of selected tests
2. Run Tools/Scripts/import-w3c-tests as is, or Tools/Scripts/import-w3c-tests -t web-platform-tests/XXselected-testsXX to have a faster import
3. Run rwt (Tools/Scripts/run-webkit-tests) to generate the new expectations on LayoutTests/imported/w3c/web-platform-tests/XXselected-testsXX
4. Update if needed the list of imported files that should be marked as [ Skip ] in LayoutTests/TestExpectations
   • Tools/Scripts/import-w3c-tests will display in the console this to-be-skipped list of files

The import-w3c-tests is doing the following things:

1. Clone the test repositories (as defined in LayoutTests/imported/w3c/resources/TestRepositories)
2. Checkout a specific revision
   • It is important to keep the same revision for consistency
3. Import tests
   • Only reference and JS tests should be imported at the moment

Manual edits of tests are highly discouraged

• It might be hard to keep those edits when rebasing the tests
• If required, document these clearly in the imported tests themselves
• Upstream those changes to the original test repository if possible

Rebasing tests

The process consists in updating the revision and reimporting all tests. It is not yet automated. A preliminary patch is available at ​https://bugs.webkit.org/show_bug.cgi?id=134767

1. Check whether WebKit testharness.js is up-to-date with the latest one used in the W3C repository
   • If not update WebKit testharness.js accordingly
2. Update the repository revision in LayoutTests/imported/w3c/resources/TestRepositories
3. Delete all *html|xhtml|xhtm|js|svg* files in the corresponding LayoutTests/imported/w3c sub-folder
4. Run Tools/Scripts/import-w3c-tests
5. Check for any manual edit that should be kept
6. Regenerate expected files using rwt
7. Remove all dangling expected.txt files and empty folders
   • Some bots have difficulties with removed/added empty files.

Laundry list

• Handle WPT test repository recursive submodules
  • Automated generation of LayoutTests/imported/w3c/resources/web-platform-tests-modules.json
• Generate WPT MANIFEST.json and use it to know whether a file is a test or a resource file
• Check whether a WPT resource file is already skipped in LayoutTests/TestExpectations
• Finalise rebasing test script
• Update import script to handle non-W3C test repositories
  • Blink tests
  • WHATWG streams test suite
• Move already imported test suites to LayoutTests/imported/w3c/web-platform-tests
  • LayoutTests/imported/w3c/canvas, LayoutTests/imported/w3c/html-templates, LayoutTests/imported/w3c/indexeddb
  • LayoutTests/imported/blink/imported/csswg-test/css-writing-modes-3, LayoutTests/imported/blink/imported/web-platform-tests