| | 45 | |
| | 46 | == Proposal for release testing == |
| | 47 | |
| | 48 | The idea for a new release would be that it has no regressions over the previous release. This should mean: |
| | 49 | |
| | 50 | * All LayoutTests that passed on the old version should pass on the release candidate. |
| | 51 | |
| | 52 | This can be realized by either storing the list of tests ran on the old release somewhere or using the LayoutTests/ directory of the old release. This has the possible problem that DRT was changed in a incompatible way or the WebKit behavior was changed and now a test would fail. Alternatively one could compare the list of tests that were ran and for every removed test it may not be in the Skipped list. |
| | 53 | |
| | 54 | * It should pass all manual tests. |
| | 55 | This can be realized by creating a "meta" manual page linking to all manual tests we want to run before the release. |
| | 56 | |
| | 57 | * It should not consume more memory on the existing test cases and benchmarks. |
| | 58 | This can be realized by using libmemusage.so and use QtLauncher on a list of URLs and use mirrored content (from the benchmarking). Afterwards one can manually compare the memory usage. |
| | 59 | |
| | 60 | As part of the release the tester(s) should write a simple protocol. It should include the LayoutTests ran, the manual tests ran and their results, the result of libmemusage.so and should be stored on a publicly available place for future reference. |
| | 61 | |
