wiki:QtWebKitGardening

Version 11 (modified by Csaba Osztrogonác, 12 years ago) ( diff )

--

Goals

  • Trunk must build on all of our platform
  • Don't bring regressions into the tree
  • Buildbots should be green almost at all times

First steps before gardening

  • Get Ubuntu 11.10 (or 12.04 or 12.10) and install it. Install this metapackage: ​https://launchpad.net/~u-szeged/+archive/sedkit 
     * sudo apt-get install python-software-properties
 * sudo add-apt-repository ppa:u-szeged/sedkit
 * sudo apt-get update
 * sudo apt-get install sedkit-env-qtwebkit
  • Build the actual Qt 5 for the WebKit trunk with this builder script: 
     * git clone git://github.com/ossy-szeged/qt5-tools.git
 * ./build-qt5.sh 
 * If there is Qt5 update, then the only thing left is to use git pull and run the script again.

 note:
 * build-qt5.sh uses 30 threads by default (-j option) 
 * Qt is installed into /usr/local/Trolltech (normal user doesn't have write permission for it by default
  • Download testfonts, set environment variables (note: the latest suffix of QTDIR is QT_WEEKLY_REV in ​build-qt5-env) 
     * git clone git://gitorious.org/qtwebkit/testfonts.git
 * export WEBKIT_TESTFONTS=/home/webkit/testfonts
 * export TZ=America/Los_Angeles ( to make all JSC tests pass :) )
 * export QTDIR=/usr/local/Trolltech/Qt5/Qt-5.0.0-beta1
 * export PATH=/usr/local/Trolltech/Qt5/Qt-5.0.0-beta1/bin:$PATH
  • Download WebKit, configure git svn, build WebKit. 
     * git clone git://git.webkit.org/WebKit.git WebKit
 * cd WebKit
 * git svn init --prefix=origin/ -T trunk http://svn.webkit.org/repository/webkit
 * git config --replace svn-remote.svn.fetch trunk:refs/remotes/origin/master
 * git svn fetch

 * to build: Tools/Scripts/build-webkit (Set MAKEFLAGS=-j5)
  • You are ready for Gardening. ;)

How to gardening

(example: ​http://build.webkit.org/builders/Qt%20Linux%20Release?numbuilds=200)

  • Click on "Build #", you can see the blamelist
  • If the blamelist is too big, check more builders
  • If the blamelist is still too big, use git bisect locally to find the culprit
  • If it is a simple buildfix, or Qt specific rebaseline, do it yourself immediately.
  • If the bug is platform independent (fail on Apple, Chromium, GTK or EFL too) and bigger than a simple fix, try to ping the author and the reviewer and/or comment the original bug. If they don't answer in 15-20 minutes:
    • rollout with sheriffbot:
      • On #webkit: sherrifbot: rollout 12345 It broke the build on SL, GTK, Qt
      • Land the rollout patch manually or mark it with cq+ to land it by commit queue.
    • or file a new bug report about the regression and put the failing tests to the Skipped list (qt, qt-5.0-wk1 or qt-5.0-wk2) See: Create new bug report
  • You should check whether the test fails on wk1, wk2 or both.
  • You should check if it's a 32-bit or a 64-bit bug (or it happens on both systems) - JSC patches sometime break only 32 or 64 bit platforms
  • You should check if it's a release or a debug bug (or it happens on both systems) - ASSERTS are checked in debug mode only

Tips and tricks

  • Use cc input field completion tool on BugZilla to find a nickname from e-mail address or real name (It works only with WebKit based browsers!)
  • Use "old-run-webkit-tests --platform mac -p name-of-the-failing test" to determine if the tests needs only rebaseline or not (If you are sure if the Mac result is correct.)
  • Use "webkit-patch rebaseline" if you are sure if the actual result is correct on the bot (for text only tests!)
  • Use "old-run-webkit-tests --platform qt -p --reset-results --add-platform-exception name-of-the-failing test" to create new results

Create new bug report

  • There's no need to create a bug report:
    • If there is already a related bug report about the test or about problems covering the tests (for example: unimplemented features)
  • In any other case, it is recommended to create a bug report, in which you inform the author and the reviewer about the current error. The developer can inform you whether this result is the (new) expected result (rebaseline) or it is an error indeed (Skip).
  • Create bug report step by step: ​https://bugs.webkit.org/
    1. Enter a new bug report
    2. WebKit
    3. Component: Tools / Test
    4. CC fields: You should send this report to the author of the patch, the reviewer, or perhaps some other gardeners. Use cc input field completion tool on BugZilla to find a nickname by e-mail address or real name (It only works with WebKit based browsers!)
    5. Summary: A short description about the fail. For example: 
      [Qt] REGRESSION (r53450): fast/forms/textarea-scrollbar-height.html fails.
[Qt][WK2] fast/forms/range/slider-mouse-events.html and fast/forms/range/slider-onchange-event.html fail after r124625.
[Qt] new test editing/spelling/spelling-unified-emulation.html is failing.
    6. Description: Detailed description of the fail by giving the test results! (Diff).
    7. Blocks: You should add to the blocks list the correct meta bug number AND the original bug: 
         * newly added failing test: 87008
   * REGRESSION: 79666
   * crash/timeout: 79668
   * API-WK1: 38654
   * API-WK2: 70236

Rebaseline

  • For example for rebaseline:
    • Changes caused by testfonts update.

  • Modify test results. Should update test expectation.

You can easily decide if a test only needs rebaseline. You can compare the result with results on other platforms.

  • "run-webkit-tests --qt --compare-port port name-of-the-failing test"
  • "webkit-patch rebaseline" if you are sure if the actual result on the bot is correct (for text only tests!)
  • "webkit-patch garden-o-matic"
  • [WK1]"Tools/Scripts/run-webkit-tests --qt --new-baseline --additional-platform-directory= "platform absolute name" -p --test-list= List" Text + PNG
  • [WK2]"Tools/Scripts/run-webkit-tests --qt -2 --new-baseline --additional-platform-directory=/home/kadam/webkit/WebkitGardener/LayoutTests/platform/qt -p --test-list= List" Text + PNG

Skip

  • First of all, the most important thing: don't skip thoughtlessly!
  • If the bug is Qt specific and you can't fix it immediately, file a bug [Create bug report] about it, and put the failing tests to the Skipped list.
  • The Skipped list has a regular structure. Try to put the test into the appropriate group. For example: Failing fast tests, Failing editing/selection tests etc.
  • When skipping a test, put it on the appropriate skipped list. You should clarify if it's just Qt, or WebKit1 or WebKit2 specific, and use the Skipped lit accordingly.
  • For example:
    • If the test is Qt specific and fails on both (WK1 and WK2) bots, then use LayoutTests/platform/qt/Skipped.
    • If the test is Qt specific and fails only on WK1 or WK2, then use LayoutTests/platform/qt-5.0-wk1/Skipped or LayoutTests/platform/qt-5.0-wk2/Skipped.
    • If it fails also on other platforms and the problem occurs only on WebKit2, then you should use LayoutTests/platform/wk2.
  • Bug report has to be filled accordingly and You should write it the original bug.

  • Principle: If anybody write comment to the bug report later, then try to fix the bug about this suggestion or find anybody else to fix it. (We can't expect if an Apple, Google, ... developer fix Qt related bugs, but usually they can give us useful advices how can we fix the bug.
  • Some examples for skipping:
  # [Qt] REGRESSION(r113520): fast/transforms/scrollIntoView-transformed.html fails
  # https://bugs.webkit.org/show_bug.cgi?id=83419
  fast/transforms/scrollIntoView-transformed.html

  # [Qt] New fast/forms/number/number-validation-message.html introduced in r121019 fails.
  # https://bugs.webkit.org/show_bug.cgi?id=89760
  fast/forms/number/number-validation-message.html
  • You need to pay attention to disabled and unimplemented features.
  • Missing things on the Skipped list can be found at "Missing features in our DumpRenderTree implementation" and "Disabled features" sections.

  • Some for example:
  • ENABLE(SHADOW_DOM) is disabled.

  • ENABLE(MUTATION_OBSERVERS) is disabled.

  • Drag and Drop Support in DRT:
    • DumpRenderTree needs eventSender.beginDragWithFiles() implementation

Buildfix

Minimal build

Testing

  • You can run tests with the following commands: 
     * Run JavaScriptCore tests: Tools/Scripts/run-javascriptcore-tests 
 * Run Layout tests:
   * Tools/Scripts/run-webkit-tests [-2]
   * xvfb-run -a --server-args="-screen 0 1024x768x24" Tools/Scripts/run-webkit-tests [-2] --no-show-results (bots use this command!)
   * Tools/Scripts/new-run-webkit-tests --qt [-2] name-of-the-tests
 * Run SunSpider performance tests: Tools/Script/run-sunspider
 * Run python unit tests: Tools/Scripts/test-webkitpy
 * Run perl unit tests: Tools/Scripts/test-webkitperl
 * Run Qt WebKit1 API tests: python ./Tools/Scripts/run-qtwebkit-tests --output-file=qt-unit-tests.html --do-not-open-results --timeout=120 WebKitBuild/Release/Source/WebKit/qt/tests/
 * Run Qt WebKit2 API tests: python ./Tools/Scripts/run-qtwebkit-tests --output-file=qt-unit-tests.html --do-not-open-results --timeout=120 WebKitBuild/Release/Source/WebKit2/UIProcess/API/qt/tests/

Community support

Help the others to fix Qt related build issues:

(FIXME: These searches aren't perfect, because they don't know if the build failure is still valid. They simple
search bugs modified in the last 7 days and with "did not pass on qt-ews/qt-wk2-ews" comment sometime in the past.)

Attachments (5)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.