|Version 5 (modified by firstname.lastname@example.org, 11 months ago) (diff)|
Selective Regression Tests in Early Warning System
Regression test selection based on procedure level code coverage of the changes will be applied to EWS. See the description of the selection method, meta-bug and patches at SelectiveTestBuildBot.
The test system is modified so that it executes both the full and the selective test on the patch. First the test selection is performed based on the changes in the patch and using the coverage database, and if it returns a non-trivial selection (not 0 and not all the tests), then the full test is executed on the same patch. The system returns always with a failure so that we can examine the build logs to verify the correctness.
Here will be some example, where the early warning system caught a bug with test selection:
- At revision 115655 the patch committed for the bug 85217 failed on the fast/flexbox/flex-hang.html test with unexpected text diff mismatch. For this result we had to run only 1036 test case.
with unexpected text diff mismatch. For this result we had to run only 361 test case.