| 5 | | There are four main tasks for the gardener: |
| | 8 | * Get Ubuntu 11.10, Install it. Install this metapackage: [https://launchpad.net/~loki-inf/+archive/sedkit] |
| | 9 | {{{ |
| | 10 | * sudo add-apt-repository ppa:loki-inf/sedkit |
| | 11 | * sudo apt-get update |
| | 12 | * sudo apt-get install sedkit-env-qtwebkit |
| | 13 | }}} |
| | 14 | * Build the actual Qt 5 for the WebKit trunk with this builder script: |
| | 15 | {{{ |
| | 16 | * git clone git://github.com/ossy-szeged/qt5-tools.git |
| | 17 | * ./build-qt5.sh |
| | 18 | * If there is Qt5 update, then the only thing left is to use git pull and run the script again. |
| | 19 | }}} |
| | 20 | * Download WebKit, configure git svn, build WebKit. |
| | 21 | {{{ |
| | 22 | * git clone git://git.webkit.org/WebKit.git WebKit |
| | 23 | * cd WebKit |
| | 24 | * git svn init --prefix=origin/ -T trunk http://svn.webkit.org/repository/webkit |
| | 25 | * git config --replace svn-remote.svn.fetch trunk:refs/remotes/origin/master |
| | 26 | * git svn fetch |
| 12 | | Let's look at each one in detail... |
| | 32 | = How to gardening = |
| | 33 | * Starting point is the waterfall: ( [http://build.webkit.sed.hu/waterfall], [http://build.webkit.org/waterfall] ) |
| | 34 | * If You can see red bots... |
| | 35 | * Determine which changeset caused the fail |
| | 36 | * Waterfall is your friend ;) |
| | 37 | * http://build.webkit.org/waterfall |
| | 38 | * http://build.webkit.sed.hu/waterfall |
| | 39 | * TestFailures tool is your friend too |
| | 40 | * http://build.webkit.org/TestFailures/ (It works only with WebKit based browsers!) |
| | 41 | * http://build.webkit.sed.hu/TestFailures/ (It works only with WebKit based browsers!) |
| | 42 | * Show more tests on waterfall: Click on a builder and then ( Show: default 25 50 100 200 ) |
| | 43 | (example: http://build.webkit.org/builders/Qt%20Linux%20Release?numbuilds=200) |
| | 44 | * Click on "Build #", you can see the blamelist |
| | 45 | * If the blamelist is too big, check more builders |
| | 46 | * If the blamelist is still too big, use git bisect locally to find the culprit |
| | 47 | * If it is a simple buildfix, or Qt specific rebaseline, do it yourself immediately. |
| | 48 | * If the bug is platform independent (fail on Apple, Chromium, GTK or EFL too) and bigger than a simple fix, |
| | 49 | try to ping the author and the reviewer and/or comment the original bug. If they don't answer in 15-20 minutes: |
| | 50 | * rollout with sheriffbot: |
| | 51 | * On #webkit: sherrifbot: rollout 12345 It broke the build on SL, GTK, Qt |
| | 52 | * Land the rollout patch manually or mark it with cq+ to land it by commit queue. |
| | 53 | * file a new bug report about the regression and put the failing tests to the Skipped list (qt, qt-5.0-wk1 or qt-5.0-wk2) See: Create new bug report |
| | 54 | * You should check whether the test fails on wk1, wk2 or both. |
| | 55 | * You should check if it's a 32-bit or a 64-bit bug (or it happens on both systems) - JSC patches sometime break only 32 or 64 bit platforms |
| | 56 | * [http://build.webkit.org/builders/Qt%20Linux%20Release Qt 32 bit release bot] |
| | 57 | * [http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Debug Qt 32 bit debug bot] |
| | 58 | * [http://build.webkit.sed.hu/builders/x86-64%20Linux%20Qt%20Release Qt 64 bit release bot] |
| | 59 | * [http://build.webkit.sed.hu/builders/x86-64%20Linux%20Qt%20Debug Qt 64 bit debug bot] |
| | 60 | * You should check if it's a release or a debug bug (or it happens on both systems) - ASSERTS are checked in debug mode only |
| 21 | | Freenode IRC channels: |
| | 74 | * Create bug report step by step: |
| | 75 | [https://bugs.webkit.org/] [[BR]] |
| | 76 | 1. Enter a new bug report |
| | 77 | 2. WebKit |
| | 78 | 3. Component: Tools / Test |
| | 79 | 4. CC fields: You should send this report to the author of the patch, the reviewer, or perhaps some other gardeners. Use cc input field completion tool on BugZilla to find a nickname by e-mail address or real name (It only works with WebKit based browsers!) |
| | 80 | 5. Summary: A short description about the fail. For example: |
| | 81 | {{{ |
| | 82 | [Qt] REGRESSION (r53450): fast/forms/textarea-scrollbar-height.html fails. |
| | 83 | [Qt][WK2] fast/forms/range/slider-mouse-events.html and fast/forms/range/slider-onchange-event.html fail after r124625. |
| | 84 | [Qt] new test editing/spelling/spelling-unified-emulation.html is failing. |
| | 85 | }}} |
| | 86 | 6. Description: Detailed description of the fail by giving the test results! (Diff). |
| | 87 | 7. Blocks: You should add to the blocks list the correct meta bug number AND the original bug: |
| | 88 | {{{ |
| | 89 | * newly added failing test: 87008 |
| | 90 | * REGRESSION: 79666 |
| | 91 | * crash/timeout: 79668 |
| | 92 | * API-WK1: 38654 |
| | 93 | * API-WK2: 70236 |
| 28 | | These are currently the QtWebKit skip lists (black lists for LayoutTests): |
| | 108 | You can easily decide if a test only needs rebaseline. You can compare the result with results on other platforms. |
| | 109 | * "run-webkit-tests --qt --compare-port port name-of-the-failing test" |
| | 110 | * "webkit-patch rebaseline" if you are sure if the actual result on the bot is correct (for text only tests!) |
| | 111 | * "webkit-patch garden-o-matic" |
| | 112 | * [WK1]"Tools/Scripts/run-webkit-tests --qt --new-baseline --additional-platform-directory= "platform absolute name" -p --test-list= List" Text + PNG |
| | 113 | * [WK2]"Tools/Scripts/run-webkit-tests --qt -2 --new-baseline --additional-platform-directory=/home/kadam/webkit/WebkitGardener/LayoutTests/platform/qt -p --test-list= List" Text + PNG |
| 30 | | * LayoutTests/platform/qt/Skipped |
| 31 | | * LayoutTests/platform/qt-4.8/Skipped |
| 32 | | * LayoutTests/platform/qt-arm/Skipped |
| 33 | | * LayoutTests/platform/qt-linux/Skipped |
| 34 | | * LayoutTests/platform/qt-mac/Skipped |
| 35 | | * LayoutTests/platform/qt-win/Skipped |
| 36 | | * LayoutTests/platform/qt-wk2/Skipped |
| | 115 | * After the rebaseline You should check if the new results they are correct. ([#testing Testing]) |
| | 116 | * [http://trac.webkit.org/wiki/Rebaseline Learn more] |
| | 117 | {{{ |
| | 118 | #!html |
| | 119 | <a name="skip"></a> |
| | 120 | }}} |
| | 121 | = Skip = |
| 38 | | The "qt" list (LayoutTests/platform/qt/Skipped) is common and applies to all platforms. The other lists are for failures that are specific to that platform. |
| | 123 | * ''' First of all, the most important thing: don't skip thoughtlessly! ''' |
| | 124 | * If the bug is Qt specific and you can't fix it immediately, file a bug [Create bug report] about it, and put the failing tests to the Skipped list. |
| | 125 | * The Skipped list has a regular structure. Try to put the test into the appropriate group. For example: Failing fast tests, Failing editing/selection tests etc. |
| | 126 | * When skipping a test, put it on the appropriate skipped list. You should clarify if it's just Qt, or WebKit1 or WebKit2 specific, and use the Skipped lit accordingly. |
| | 127 | * For example: |
| | 128 | * If the test is Qt specific and fails on both (WK1 and WK2) bots, then use LayoutTests/platform/qt/Skipped. |
| | 129 | * If the test is Qt specific and fails only on WK1 or WK2, then use LayoutTests/platform/qt-5.0-wk1/Skipped or LayoutTests/platform/qt-5.0-wk2/Skipped. |
| | 130 | * If it fails also on other platforms and the problem occurs only on WebKit2, then you should use LayoutTests/platform/wk2. |
| | 131 | * Bug report has to be filled accordingly and You should write it the original bug. |
| | 132 | |
| | 133 | * '''Principle''': If anybody write comment to the bug report later, then try to fix the bug about this suggestion or find anybody else to fix it. (We can't expect if an Apple, Google, ... developer fix Qt related bugs, but usually they can give us useful advices how can we fix the bug. |
| | 134 | * Some examples for skipping:[[BR]] |
| 58 | | The Qt API tests live in Source/WebKit/qt/tests/ and are part of the bot rotation. Although a failing API test currently does not make the bot flip red, it's important that we maintain the tests regardless. |
| | 166 | == Minimal build == |
| | 167 | * [http://build.webkit.org/builders/Qt%20Linux%20Release%20minimal Qt Linux Release minimal bot] is to test if every feature is correctly guarded by #ifdef-s. |
| | 168 | * Examples for --minimal buildfix: |
| | 169 | * [http://trac.webkit.org/changeset/127371/trunk/Source/WebCore/css/StyleBuilder.cpp] |
| | 170 | * [http://trac.webkit.org/changeset/126299#file1] |
| | 171 | * [http://trac.webkit.org/changeset/123796] |
| | 172 | * [https://trac.webkit.org/changeset/118335] |
| | 173 | * [https://trac.webkit.org/changeset/93955] |
| | 174 | {{{ |
| | 175 | #!html |
| | 176 | <a name="testing"></a> |
| | 177 | }}} |
| | 178 | = Testing = |
| 60 | | The procedure is very similar to the LayoutTests, though we should have slightly higher tolerance for non-Qt WebKit developers inadvertently breaking our API tests. (This because our API takes some "artistic liberties" with regard to layering violations in more than a few places.) |
| 61 | | |
| 62 | | Skip the failing test (or the failing sub-portion of the test if you are able to) by adding a [http://doc.qt.nokia.com/4.7/qtest.html#QSKIP] QSKIP() statement. Open a bug about the failure, "Qt, QtTriaged, Regression", CC'ing the appropriate people, and adding a comment with a reference to the bug# along with the QSKIP(). |
| | 180 | * You can run tests with the following commands: |
| | 181 | {{{ |
| | 182 | * Run JavaScriptCore tests: Tools/Scripts/run-javascriptcore-tests |
| | 183 | * Run Layout tests: |
| | 184 | * Tools/Scripts/run-webkit-tests [-2] |
| | 185 | * xvfb-run -a --server-args="-screen 0 1024x768x24" Tools/Scripts/run-webkit-tests [-2] --no-show-results (bots use this command!) |
| | 186 | * Tools/Scripts/new-run-webkit-tests --qt [-2] name-of-the-tests |
| | 187 | * Run SunSpider performance tests: Tools/Script/run-sunspider |
| | 188 | * Run python unit tests: Tools/Scripts/test-webkitpy |
| | 189 | * Run perl unit tests: Tools/Scripts/test-webkitperl |
| | 190 | * Run Qt WebKit1 API tests: python ./Tools/Scripts/run-qtwebkit-tests --output-file=qt-unit-tests.html --do-not-open-results --timeout=120 WebKitBuild/Release/Source/WebKit/qt/tests/ |
| | 191 | * Run Qt WebKit2 API tests: python ./Tools/Scripts/run-qtwebkit-tests --output-file=qt-unit-tests.html --do-not-open-results --timeout=120 WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/ |
| | 192 | }}} |
