wiki:WebKitGTK/StartHacking

Version 47 (modified by Michael Catanzaro, 10 years ago) ( diff )

Don't recommend old unsupported distro releases

  1. Start hacking in WebKitGTK+
    1. HW conditionants
    2. Choosing the working distribution
    3. Getting the source
    4. Compiling
      1. Compilation issues
    5. Rebuilding
      1. Keeping up to date
    6. Documentation
      1. API
      2. Misc
    7. Hacking guide
      1. Development branches and dependencies
    8. Testing
    9. Debugging
    10. Continuous integration
    11. Contributing patches
    12. Other tips and tricks
      1. Nice working prompt
      2. Applications using WebKitGTK+
      3. Using a chroot as working environment
      4. Persistent GSettings
      5. Working behind a proxy
      6. Extending jhbuild with custom modules or newer versions

Start hacking in WebKitGTK+

HW conditionants

WebKit is such a bahamut that the first thing to take into account is the location we want to use as our working place.

Basically, we will be able to compile in two flavors: Release and Debug. As its name says, the debug compilation mode will keep the debugging symbols. This is handy for debugging but will take longer time in the compilation, will use much more space and will need much more RAM. In addition, the performance may be affected when trying to open a debugging session with GDB.

  • A local copy of the Git repository will take ~7Gb.
  • As the GTK port uses for its build jhbuild, the dependencies needed will take ~3Gb.
  • The objects of a Release build will take another additional ~1Gb.
  • The objects of a Debug build will take another several additional ~10Gb.

In addition, there may be other factors to get into account:

  • You may want to speed up your compilation through the usage of ccache and, therefore, reserve another ~8Gb (with few builds my cache is taking ~3Gb).
  • You may want to combine WebKitGTK development with Epiphany development, the reference application using WebKitGTK and the official GNOME web browser. This will take another additional ~4Gbs.
  • You may want to do your hacking in a chroot environment so it doesn't interfere with your production distribution. A basic Ubuntu Quantal Quetzal x64 in which to work takes to me ~4Gb.

With all this, I would reccommend to start working in a hard drive partition in which we have at least a minimum of 50Gb of free space.

In addition, WebKit needs tons of RAM memory and CPU power. If you don't want to get bored waiting for your compilations to end you will need a powerful machine with several cores and Gbs or RAM. Not only that, nowadays hackers usually carry out their job in laptops. As time goes by they become more powerful but still several WebKit hackers get the support of desktop computers to speed up their compilations with the help of distributed compiling tools like distcc.

My computer is a ​Lenovo Thinkpad X220 with a SSD hard drive, 8Gb RAM and a dual-core Intel® Core™ i7-2620M (2.7GHz, 4MB L3 cache). A clean Release build takes ~1h.

You will have to take all of this into consideration before starting to work in WebKit.

Choosing the working distribution

Development branches of WebKitGTK require quite an up to date distribution. In the end, this is development so you may want to have your working tools as updated as possible. Recommended distributions include Debian Testing or Sid, or the latest release of Ubuntu or Fedora.

If you happen to have an older distribution as your daily production environment you may be interested on using a chroot environment in which to install one of these distributions.

Getting the source

This is already explained in the BuildingGtk section and the instructions to use Git in UsingGitWithWebKit but, trying to give straight commands, you most probably would like to use Git as your revision control tool. WebKit actually uses SVN for its official revision control tool but tons of developers use Git and its usage is well supported. 

$ git clone git://git.webkit.org/WebKit.git WebKit

First thing you would like to do is installing the base dependencies: 

$ Tools/gtk/install-dependencies

As you may also want to keep track of the official SVN repository, let's do (this example shows just my very personal preferences): 

$ Tools/Scripts/webkit-patch setup-git-clone
Do you want to auto-color status, diff, and branch? (y/n) [Y/n]: y
Do you use a rebase-based workflow? (y=yes; n=no, I use a merge-based workflow) [Y/n]: y
Do you want to append the git branch name to every build? (e.g. WebKitBuild/mybranch/; y/n) [Y/n]: n
Done
$ git svn fetch

or, manually: 

$ cd WebKit
$ git svn init --prefix=origin/ -T trunk http://svn.webkit.org/repository/webkit
$ git config --replace svn-remote.svn.fetch trunk:refs/remotes/origin/master
$ git svn fetch

Done! You are ready to start using the code.

Compiling

Before even starting to think in compiling we may want to be sure that we won't have to repeat the same huge effort shortly after. Hence, we would like to put into place some build speeding techniques.

Summarizing, you may want to use gold as your linker.

If you are working isolated with a single computer, just start using ccache.

If you are in a network in which other computers are available to help on compiling, just start using distcc or some other distributed compiling software like icecc.

After you have your settings ready, you can just follow the instructions in BuildingGtk. Basically, you will compile and install the needed dependencies with jhbuild and will compile WebKitGTK with the default options.

There are a couple of things still to be considered:

  • Release or Debug? The default compilation will do a Release build. You may think that it may be worthy to actually do a Debug build since you are planning to start hacking and make use of gdb. However, a Debug build will take longer, more hard drive space and more RAM. In addition, running a debug build in gdb may be a pain. Because of this, most of the WebKit hackers choose to just use a Release build and make an extensive usage of printf as debugging tool.
    • In any case, don't worry too much since a Release and a Debug build can co-exist as WebKit store the resulting objects in different directories.
  • Also, there is the possibility to tweak a Release build to keep certain debugging helpers and yet not being a full Debug build. For example, you may want to disable the compilation of unneeded pieces for if you are not working on them. For example, you can pass: 
    $ Tools/Scripts/build-webkit --gtk
    • You may also want to turn some debugging flags in a Release build. For example, you can turn on the logging support setting the C macro LOG_DISABLED=0. 
      $ export CPPFLAGS="-DLOG_DISABLED=0"
$ ./Tools/Scripts/build-webkit --gtk
  • You may also want to pass extra parameters to the cmake command. This can be done with the --cmakeargs argument: 
    $ ./Tools/Scripts/build-webkit --release --gtk --cmakeargs="-GNinja -j8"
  • You should check all the possible flags to pass to the building script: 
    $ Tools/Scripts/build-webkit --help

Compilation issues

If you have strange issues while compiling you can try the following (in no particular order):

  • If you are using ccache, clean it: 
    $ ccache -C
  • If you may be using the gold compiler, change ld in order to use ld.bfd instead of ld.gold (see instructions).
  • Use a jhbuild shell and compile with -j1. Example with problems compiling cairo: 
    $ Tools/jhbuild/jhbuild-wrapper --gtk shell
$ cd WebKitBuild/Dependencies/Source/cairo-1.12.8/
$ ../../Root/bin/make -j1
$ ../../Root/bin/make install

That instructions assume that the configure step was successful in the first place. If you need to repeat all the module build, it's better to do it directly using jhbuild. Never invoke ./configure manually without parameters if you don't know what you're doing: 

$ Tools/jhbuild/jhbuild-wrapper --gtk buildone -f cairo

Rebuilding

As you code new features and fix bugs you will need to rebuild the sources or dependencies. You have to be careful if you don't want to waste time. You don't want compiled code to be swiped by mistake given the time that it takes to build it.

The build result is located in the WebKitBuild directory. Release builds will be place under WebKitBuild/Release while Debug builds will be placed under WebKitBuil/Debug.

As you should know by now, building dependencies will typically be gotten with the help of jhbuild. The default command for this, as explained in BuildingGtk is: 

$ Tools/Scripts/update-webkitgtk-libs

This command will clean current dependencies tree. Therefore, if you only want to perform an action on a certain module it would be better to do it directly using jhbuild. For example: 

$ Tools/jhbuild/jhbuild-wrapper --gtk buildone -f cairo

The result of building the dependencies will be placed under WebKitBuild/Dependencies.

Similarly, if we have peformed changes in the code, in BuildingGtk we have seen that the script to perform a build is: 

$ Tools/Scripts/build-webkit --gtk

Now, if we want just to manuall build our changes or only certain targets, we can get into the jhbuild environment with 

$ Tools/jhbuild/jhbuild-wrapper --gtk shell

And manually invoke make: 

$ cd WebKitBuild/Release
$ make

Or, if we just want to rebuild MiniBrowser: 

$ cd WebKitBuild/Release
$ make bin/MiniBrowser

Keeping up to date

WebKit development happens at a high speed. Most of the changes won't have a real impact on the feature or bug you are actually fixing while usually would mean rebuilding huge parts of the code. Therefore, it is better not to rebase master to our working branch too often.

Usually, you would want to rebase your work only when you have completed the task, but be also aware of parallel work that may affect your code. Plan ahead when you'll have a free slot of time in which to rebase and compile again as it will probably take several minutes.

Documentation

API

WebKit external APIs are documented by each port. For examples, this is latest ​stable API for WebKit2GTK.

WebKit internal APIs are not really documented. If you want to know how something works it is better that you just go to the code.

Misc

There is no proper WebKit tutorial available as WebKit gets updated almost every day and it is very hard to maintain any such tutorial. What we have is the tech talks, blog posts, etc written by WebKit experts. You can find a compilation of these articles here: ​http://arunpatole.com/blog/2011/webkit-documentation/

webkit.org also has a ​technical articles section.

Hacking guide

This is a Hacker's guide to WebKit/GTK+.

Development branches and dependencies

Stable Release follows the current WebKitGtk+ stable and unstable branches.

Stable and unstable releases are announced in the ​mailing list. Current exact versions can be checked at ​http://webkitgtk.org/ or ​http://webkitgtk.org/releases/

Software dependencies for every version are listed in the release announcement. You can look for the a mail with subject [webkit-gtk] WebKitGTK+ <version> released! to check them.

Testing

WebKit provides a really complete set of tets to check the code and your changes. You can get more information about them in WebKitGtkLayoutTests.

In addition, you may want to check your changes in an actual application. WebKit provides a dummy browser in which to do so. GTK+ port actually has MiniBrowser.

  • To run MiniBrowser: 
    $ Tools/Scripts/run-launcher --gtk

Of course, this can be done also manually from the jhbuild shell: 

$ Tools/jhbuild/jhbuild-wrapper --gtk shell
$ WebKitBuild/Release/bin/MiniBrowser &

Debugging

Here you have some debugging tips and also some other tips for tracking memory errors.

Continuous integration

WebKit continuous integration happens in ​http://build.webkit.org/. There are a number of gardening guidelines to help keeping our build bots green.

Contributing patches

This notes are taking into account that the WebKit repository is managed with Git and that git-svn is tracking the official SVN repository:

Steps to upload a patch to ​WebKit bugzilla.

  • You should be in a branch where you only have the changes of your patch compared to master/trunk.
  • Generate the ChangeLog files of your patch (this will generate the ChangeLog from the current diff not the committed patches): 
    $ Tools/Scripts/prepare-ChangeLog
    • Options:
      • -b #bug-number#: This will automatically fills the first 2 lines of the ChangeLog file.
      • -g #commit-identifier#: This allow you to generate the ChangeLog for an already locally committed patch.
      • -o: It automatically opens the editor with the new ChangeLog entries.
  • Fill the ChangeLog files modified in the previous patch.
    • The first line should be the bug/patch title (already done if you used -b #bug-number# option).
    • Next line is the link to the bug (already done if you used -b #bug-number# option).
    • Then you should leave the Reviewed by NOBODY (OOPS!). that will be automatically filled by commit-queue when the patch lands. If it's a gardening patch (e.g. skip a LayoutTest already passing) you can change that by something like Unreviewed gardening. as these patches don't need to be reviwed and request to someone else to do the cq+.
  • Next you should add a description about your changes. And add comments in each methods you have modified if there's anything important to explain.
  • Upload your patch: 
    $ Tools/Scripts/webkit-patch upload
    • Options:
      • -g #commit-identifier#: This allow you to upload an already locally committed patch.
      • --request-commit: To add cq? flag to your patch. While you're not committer you cannot do cq+ so if you've added the flag cq? he'll cq+ the patch when he r+ it.
      • --comment: If you want to add a comment about the patch when it's uploaded to the bugzilla.
      • --suggest-reviewers: Suggests you reviewers to be included in CC (WARNING: it only works if you have a git-svn repository properly configured (see these instructions)
    • webkit-patch upload will pass the style checker and you should fix any issue before uploading it.
    • It also shows you the diff of your patch that is going to be uploaded in order to check that everything is right.
    • Finally it requests you your credentials for ​https://bugs.webkit.org.
  • Then you could check in the bugzilla if the ​Early Warning System (EWS) detect any issue in your patch. If it doesn't apply in trunk or any port is broken because of your patch you should review your patch and upload it again.
  • If EWSs are green you could add some people to CC in order to review your patch (maybe you already done it with --suggest-reviewers option), be sure that the patch is marked for review r?.

Other tips and tricks

Nice working prompt

Configure prompt to know that you're in a jhbuild shell and in a Git repository. Different versions:

  • With parse_git_branch: 
    $ cat ~/.bashrc
...
if [ x$UNDER_JHBUILD != x ]; then
   PS1="\u@\h[JHBUILD]:\w\$(parse_git_branch)\$ "
else
   PS1="${debian_chroot:+($debian_chroot)}\u@\h:\w\$(parse_git_branch)\$ "
fi
...
  • Without parse_git_branch: 
    $ cat ~/.bashrc
...
PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w$(__git_ps1 " (\[\033[31m\]%s\[\033[00m\])")\$ '

if [ $UNDER_JHBUILD ]; then
    PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h[JHBUILD]:\w$(__git_ps1 " (\[\033[31m\]%s\[\033[00m\])")\$ '
fi
...

If something doesn't work, check that you have the bash-completion package installed.

Applications using WebKitGTK+

This is a list of applications using WebKitGTK+. Specially interesting is the reference web browser ​Epiphany, which is GNOME's official web browser.

If you want to compile Epiphany against your compiled WebKit follow the instructions at ​https://wiki.gnome.org/Apps/Epiphany/Development#Getting_the_code_and_building

Using a chroot as working environment

These are some tips regarding the usage of a chroot as working environment.

Persistent GSettings

Sometimes you may get a warning from the GTK+ applications about GSettings like this: 

GLib-GIO-Message: Using the 'memory' GSettings backend.  Your settings will not be saved or shared with other applications.

When working inside the jhbuild environment, you may need to run a session D-Bus daemon in it and make sure that the dconf module is built: 

# Make sure dconf is built and enter the jhbuild environment
./Tools/jhbuild/jhbuild-wrapper --gtk build dconf
./Tools/jhbuild/jhbuild-wrapper --gtk shell
# Launch a D-Bus instance inside the environment
eval "$(dbus-launch --sh-syntax)"
export GSETTINGS_BACKEND=dconf
#
# ...do some work testing/work now...
#
# Kill the D-Bus daemon inside jhbuild environment before exiting
kill $DBUS_SESSION_BUS_PID
exit

Working behind a proxy

Sometimes you are in a resricted environment and you need your WebKit development environment to work properly behind a proxy, so things like run-launcher --gtk http://www.webkit.org will actually work, instead of getting stuck trying to load stuff using the wrong "no proxy" settings.

In order to do that, you would need to build dconf inside the jhbuild environment, so glib-networking will be able read your configuration through GSettings properly, instead of just reading the default values from the GSettings memory backend that would be in use otherwise.

So, to do that, just do the following: 

./Tools/jhbuild/jhbuild-wrapper --gtk build dconf

After that, every single thing you do inside the jhbuild (e.g. using the run-launcher script) should properly pick up your proxy settings and work.

Extending jhbuild with custom modules or newer versions

The last two sections refer to building the optional module dconf inside the jhbuild to solve some specific and well known issues, but if you still need to build more modules inside that jhbuild, or just newer versions of what already is available in WebKitGTK+'s development environment, then check Extending the WebKitGTK+ JHBuild for specific instructions on how to do that.

Note: See TracWiki for help on using the wiki.