Version 3 (modified by 9 years ago) ( diff ) | ,
---|
Scope
Easy import (and re-import) of available test suites on the web, in particular W3C CSS and WPT test-suites.
Importing new tests
- Update LayoutTests/imported/w3c/resources/ImportExpectations to activate the import of selected tests
- Run "Tools/Scripts/import-w3c-tests" as is, or "Tools/Scripts/import-w3c-tests -t web-platform-tests/XXselected-testsXX" to have a faster import
- Run rwt to generate the new expectations on LayoutTests/imported/w3c/web-platform-tests/XXselected-testsXX
- Update if needed the list of imported files that should be marked as [ Skip ] in LayoutTests/TestExpectations
- Tools/Scripts/import-w3c-tests will display in the console this to-be-skipped list of files
The import-w3c-tests is doing the following things:
- Clone the test repositories (as defined in LayoutTests/imported/w3c/resources/TestRepositories)
- Checkout a specified revision
- It is important to keep the same revision for consistency
- Import tests
- Only reference and JS tests should be imported at the moment
Manual edits of tests are highly discouraged
- It might be hard to keep those edits when rebasing the tests
- If required, document these clearly in the imported tests themselves
- Upstream those changes to the original test repository if possible
Rebasing tests
The process consists in updating the revision and reimporting all tests. It is not yet automated. A preliminary patch is available at https://bugs.webkit.org/show_bug.cgi?id=134767
- Check whether WebKit testharness.js is up-to-date with the latest one used in the W3C repository
- If not update WebKit testharness.js accordingly
- Update the repository revision in LayoutTests/imported/w3c/resources/TestRepositories
- Delete all html|xhtml|xhtm|js|svg files in the corresponding LayoutTests/imported/w3C sub-folder
- Run Tools/Scripts/import-w3c-tests
- Check for any manual edit that should be kept
- Regenerate expected files using rwt
- Remove all dangling expected.txt files and empty folders
- Some bots have difficulties with removed/added empty files.
Laundry list
- Handle WPT test repository recursive submodules
- Automated generation of LayoutTests/imported/w3c/resources/web-platform-tests-modules.json
- Generate WPT MANIFEST.json and use it to know whether a file is a test or a resource file
- Check whether a WPT resource file is already skipped in LayoutTests/TestExpectations
- Finalise rebasing test script
- Update import script to handle non-W3C test repositories
- Blink tests
- WHATWG streams test suite
- Move already imported test suites to LayoutTests/imported/w3c/web-platform-tests
- LayoutTests/imported/w3c/canvas, LayoutTests/imported/w3c/html-templates, LayoutTests/imported/w3c/indexeddb
- LayoutTests/imported/blink/imported/csswg-test/css-writing-modes-3, LayoutTests/imported/blink/imported/web-platform-tests
Note:
See TracWiki
for help on using the wiki.