Start hacking in WebKitGTK+
HW conditionants
WebKit is such a bahamut that the first thing to take into account is the location we want to use as our working place.
Basically, we will be able to compile in two flavors: Release and Debug. As its name says, the debug compilation mode will keep the debugging symbols. This is handy for debugging but will take longer time in the compilation, will use much more space and will need much more RAM. In addition, the performance may be affected when trying to open a debugging session with GDB.
- A local copy of the Git repository will take ~7Gb.
- As the GTK port uses for its build jhbuild, the dependencies needed will take ~3Gb.
- The objects of a Release build will take another additional ~1Gb.
- The objects of a Debug build will take another several additional ~10Gb.
In addition, there may be other factors to get into account:
- You may want to speed up your compilation through the usage of ccache and, therefore, reserve another ~8Gb (with few builds my cache is taking ~3Gb).
- You may want to combine WebKitGTK development with Epiphany development, the reference application using WebKitGTK and the official GNOME web browser. This will take another additional ~4Gbs.
- You may want to do your hacking in a chroot environment so it doesn't interfere with your production distribution. A basic Ubuntu Quantal Quetzal x64 in which to work takes to me ~4Gb.
With all this, I would reccommend to start working in a hard drive partition in which we have at least a minimum of 50Gb of free space.
In addition, WebKit needs tons of RAM memory and CPU power. If you don't want to get bored waiting for your compilations to end you will need a powerful machine with several cores and Gbs or RAM. Not only that, nowadays hackers usually carry out their job in laptops. As time goes by they become more powerful but still several WebKit hackers get the support of desktop computers to speed up their compilations with the help of distributed compiling tools like distcc.
My computer is a Lenovo Thinkpad X220 with a SSD hard drive, 8Gb RAM and a dual-core Intel® Core™ i7-2620M (2.7GHz, 4MB L3 cache). A clean Release build takes ~1h.
You will have to take all of this into consideration before starting to work in WebKit.
Choosing the working distribution
Development branches of WebKitGTK require quite an up to date distribution. In the end, this is development so you may want to have your working tools as updated as possible. Recommended distributions include Debian Testing or Sid, or the latest release of Ubuntu or Fedora.
If you happen to have an older distribution as your daily production environment you may be interested on using a chroot environment in which to install one of these distributions.
Getting the source
This is already explained in the BuildingGtk section and the instructions to use Git in UsingGitWithWebKit but, trying to give straight commands, you most probably would like to use Git as your revision control tool. WebKit actually uses SVN for its official revision control tool but tons of developers use Git and its usage is well supported.
$ git clone git://git.webkit.org/WebKit.git WebKit
First thing you would like to do is installing the base dependencies:
$ Tools/gtk/install-dependencies
As you may also want to keep track of the official SVN repository, let's do (this example shows just my very personal preferences):
$ Tools/Scripts/webkit-patch setup-git-clone Do you want to auto-color status, diff, and branch? (y/n) [Y/n]: y Do you use a rebase-based workflow? (y=yes; n=no, I use a merge-based workflow) [Y/n]: y Do you want to append the git branch name to every build? (e.g. WebKitBuild/mybranch/; y/n) [Y/n]: n Done $ git svn fetch
or, manually:
$ cd WebKit $ git svn init --prefix=origin/ -T trunk http://svn.webkit.org/repository/webkit $ git config --replace svn-remote.svn.fetch trunk:refs/remotes/origin/master $ git svn fetch
Done! You are ready to start using the code.
Compiling
Before even starting to think in compiling we may want to be sure that we won't have to repeat the same huge effort shortly after. Hence, we would like to put into place some build speeding techniques.
Summarizing, you may want to use gold as your linker.
If you are working isolated with a single computer, just start using ccache.
If you are in a network in which other computers are available to help on compiling, just start using distcc, icecc or some other distributed compiling software.
After you have your settings ready, you can just follow the instructions in BuildingGtk. Basically, you will compile and install the needed dependencies with jhbuild and will compile WebKitGTK with the default options.
There are a couple of things still to be considered:
- Release or Debug? The default compilation will do a Release build. You may think that it may be worthy to actually do a Debug build since you are planning to start hacking and make use of gdb. However, a Debug build will take longer, more hard drive space and more RAM. In addition, running a debug build in gdb may be a pain. Because of this, most of the WebKit hackers choose to just use a Release build and make an extensive usage of printf as debugging tool.
- In any case, don't worry too much since a Release and a Debug build can co-exist as WebKit store the resulting objects in different directories.
- Also, there is the possibility to tweak a Release build to keep certain debugging helpers and yet not being a full Debug build. For example, you may want to disable the compilation of unneeded pieces for if you are not working on them. For example, you can pass:
$ Tools/Scripts/build-webkit --gtk
- You may also want to turn some debugging flags in a Release build. For example, you can turn on the logging support setting the C macro LOG_DISABLED=0.
$ export CPPFLAGS="-DLOG_DISABLED=0" $ ./Tools/Scripts/build-webkit --gtk
- You may also want to turn some debugging flags in a Release build. For example, you can turn on the logging support setting the C macro LOG_DISABLED=0.
- You may also want to pass extra parameters to the cmake command. This can be done with the --cmakeargs argument:
$ ./Tools/Scripts/build-webkit --release --gtk --cmakeargs="-GNinja -j8"
- You should check all the possible flags to pass to the building script:
$ Tools/Scripts/build-webkit --help
Compilation issues
If you have strange issues while compiling you can try the following (in no particular order):
- If you are using
ccache, clean it:$ ccache -C
- If you may be using the
goldcompiler, changeldin order to useld.bfdinstead ofld.gold(see instructions).
- Use a
jhbuildshell and compile with-j1. Example with problems compilingcairo:$ Tools/jhbuild/jhbuild-wrapper --gtk shell $ cd WebKitBuild/Dependencies/Source/cairo-1.12.8/ $ ../../Root/bin/make -j1 $ ../../Root/bin/make install
That instructions assume that the configure step was successful in the first place. If you need to repeat all the module build, it's better to do it directly using jhbuild. Never invoke ./configure manually without parameters if you don't know what you're doing:
$ Tools/jhbuild/jhbuild-wrapper --gtk buildone -f cairo
Rebuilding
As you code new features and fix bugs you will need to rebuild the sources or dependencies. You have to be careful if you don't want to waste time. You don't want compiled code to be swiped by mistake given the time that it takes to build it.
The build result is located in the WebKitBuild directory. Release builds will be place under WebKitBuild/Release while Debug builds will be placed under WebKitBuil/Debug.
As you should know by now, building dependencies will typically be gotten with the help of jhbuild. The default command for this, as explained in BuildingGtk is:
$ Tools/Scripts/update-webkitgtk-libs
This command will clean current dependencies tree. Therefore, if you only want to perform an action on a certain module it would be better to do it directly using jhbuild. For example:
$ Tools/jhbuild/jhbuild-wrapper --gtk buildone -f cairo
The result of building the dependencies will be placed under WebKitBuild/Dependencies.
Similarly, if we have peformed changes in the code, in BuildingGtk we have seen that the script to perform a build is:
$ Tools/Scripts/build-webkit --gtk
Now, if we want just to manuall build our changes or only certain targets, we can get into the jhbuild environment with
$ Tools/jhbuild/jhbuild-wrapper --gtk shell
And manually invoke make:
$ cd WebKitBuild/Release
$ make
Or, if we just want to rebuild MiniBrowser:
$ cd WebKitBuild/Release
$ make bin/MiniBrowser
Keeping up to date
WebKit development happens at a high speed. Most of the changes won't have a real impact on the feature or bug you are actually fixing while usually would mean rebuilding huge parts of the code. Therefore, it is better not to rebase master to our working branch too often.
Usually, you would want to rebase your work only when you have completed the task, but be also aware of parallel work that may affect your code. Plan ahead when you'll have a free slot of time in which to rebase and compile again as it will probably take several minutes.
Documentation
API
WebKit external APIs are documented by each port. For examples, this is latest stable API for WebKit2GTK.
WebKit internal APIs are not really documented. If you want to know how something works it is better that you just go to the code.
Misc
There is no proper WebKit tutorial available as WebKit gets updated almost every day and it is very hard to maintain any such tutorial. What we have is the tech talks, blog posts, etc written by WebKit experts. You can find a compilation of these articles here: http://arunpatole.com/blog/2011/webkit-documentation/
webkit.org also has a technical articles section.
Hacking guide
This is a Hacker's guide to WebKit/GTK+.
Development branches and dependencies
WebKitGTKStableReleases explains the policy about stable releases.
Stable and unstable releases are announced in the mailing list. Current exact versions can be checked at http://webkitgtk.org/ or http://webkitgtk.org/releases/
Software dependencies for every version are listed in the release announcement. You can look for the a mail with subject [webkit-gtk] WebKitGTK+ <version> released! to check them.
Testing
WebKit provides a really complete set of tets to check the code and your changes. You can get more information about them in WebKitGtkLayoutTests.
In addition, you may want to check your changes in an actual application. WebKit provides a dummy browser in which to do so. GTK+ port actually has MiniBrowser.
- To run MiniBrowser:
$ Tools/Scripts/run-minibrowser --gtk
Of course, this can be done also manually from the jhbuild shell:
$ Tools/jhbuild/jhbuild-wrapper --gtk shell
$ WebKitBuild/Release/bin/MiniBrowser &
Debugging
Here you have some debugging tips and also some other tips for tracking memory errors.
Continuous integration
WebKit continuous integration happens in http://build.webkit.org/. There are a number of gardening guidelines to help keeping our build bots green.
Contributing patches
This notes are taking into account that the WebKit repository is managed with Git and that git-svn is tracking the official SVN repository:
- Create an account in WebKit bugzilla.
- Every patch goes in a different bug, so if it doesn't have bug yet report it :-)
- WARNING: You need to install Subversion in your system (having
git-svninstalled will help too. Follow these instructions. - There're more detailed info in http://www.webkit.org/coding/contributing.html
Steps to upload a patch to WebKit bugzilla.
- You should be in a branch where you only have the changes of your patch compared to master/trunk.
- Generate the
ChangeLogfiles of your patch (this will generate theChangeLogfrom the current diff not the committed patches):$ Tools/Scripts/prepare-ChangeLog
- Options:
-b #bug-number#: This will automatically fills the first 2 lines of theChangeLogfile.-g #commit-identifier#: This allow you to generate theChangeLogfor an already locally committed patch.-o: It automatically opens the editor with the newChangeLogentries.
- Options:
- Fill the
ChangeLogfiles modified in the previous patch.- The first line should be the bug/patch title (already done if you used
-b #bug-number#option). - Next line is the link to the bug (already done if you used
-b #bug-number#option). - Then you should leave the
Reviewed by NOBODY (OOPS!).that will be automatically filled by commit-queue when the patch lands. If it's a gardening patch (e.g. skip a LayoutTest already passing) you can change that by something likeUnreviewed gardening.as these patches don't need to be reviwed and request to someone else to do thecq+.
- The first line should be the bug/patch title (already done if you used
- Next you should add a description about your changes. And add comments in each methods you have modified if there's anything important to explain.
- Upload your patch:
$ Tools/Scripts/webkit-patch upload
- Options:
-g #commit-identifier#: This allow you to upload an already locally committed patch.--request-commit: To addcq?flag to your patch. While you're not committer you cannot docq+so if you've added the flagcq?he'llcq+the patch when her+it.--comment: If you want to add a comment about the patch when it's uploaded to the bugzilla.--suggest-reviewers: Suggests you reviewers to be included inCC(WARNING: it only works if you have agit-svnrepository properly configured (see these instructions)
webkit-patch uploadwill pass the style checker and you should fix any issue before uploading it.- It also shows you the diff of your patch that is going to be uploaded in order to check that everything is right.
- Finally it requests you your credentials for https://bugs.webkit.org.
- Options:
- Then you could check in the bugzilla if the Early Warning System (EWS) detect any issue in your patch. If it doesn't apply in trunk or any port is broken because of your patch you should review your patch and upload it again.
- If EWSs are green you could add some people to
CCin order to review your patch (maybe you already done it with--suggest-reviewersoption), be sure that the patch is marked for reviewr?.
Other tips and tricks
Nice working prompt
Configure prompt to know that you're in a jhbuild shell and in a Git repository. Different versions:
- With
parse_git_branch:$ cat ~/.bashrc ... if [ x$UNDER_JHBUILD != x ]; then PS1="\u@\h[JHBUILD]:\w\$(parse_git_branch)\$ " else PS1="${debian_chroot:+($debian_chroot)}\u@\h:\w\$(parse_git_branch)\$ " fi ...
- Without
parse_git_branch:$ cat ~/.bashrc ... PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w$(__git_ps1 " (\[\033[31m\]%s\[\033[00m\])")\$ ' if [ $UNDER_JHBUILD ]; then PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h[JHBUILD]:\w$(__git_ps1 " (\[\033[31m\]%s\[\033[00m\])")\$ ' fi ...
If something doesn't work, check that you have the bash-completion package installed.
Applications using WebKitGTK+
This is a list of applications using WebKitGTK+. Specially interesting is the reference web browser Epiphany, which is GNOME's official web browser.
If you want to compile Epiphany against your compiled WebKit follow the instructions at https://wiki.gnome.org/Apps/Epiphany/Development#Getting_the_code_and_building
Using a chroot as working environment
These are some tips regarding the usage of a chroot as working environment.
Persistent GSettings
Sometimes you may get a warning from the GTK+ applications about GSettings like this:
GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications.
When working inside the jhbuild environment, you may need to run a session D-Bus daemon in it and make sure that the dconf module is built:
# Make sure dconf is built and enter the jhbuild environment ./Tools/jhbuild/jhbuild-wrapper --gtk build dconf ./Tools/jhbuild/jhbuild-wrapper --gtk shell # Launch a D-Bus instance inside the environment eval "$(dbus-launch --sh-syntax)" export GSETTINGS_BACKEND=dconf # # ...do some work testing/work now... # # Kill the D-Bus daemon inside jhbuild environment before exiting kill $DBUS_SESSION_BUS_PID exit
Working behind a proxy
If you just need to use a proxy in MiniBrowser (eg: for testing purposes, bandwidth shaping, etc.) set the right environment variables before launching MiniBrowser. For example:
export http_proxy="http://localhost:3128" export https_proxy="http://localhost:3128"
But sometimes you are in a resricted environment and you need your whole WebKit development environment to work properly behind a proxy, so things like run-minibrowser --gtk http://www.webkit.org will actually work, instead of getting stuck trying to load stuff using the wrong "no proxy" settings.
In order to do that, you would need to build dconf inside the jhbuild environment, so glib-networking will be able read your configuration through GSettings properly, instead of just reading the default values from the GSettings memory backend that would be in use otherwise.
So, to do that, just do the following:
./Tools/jhbuild/jhbuild-wrapper --gtk build dconf
After that, every single thing you do inside the jhbuild (e.g. using the run-minibrowser script) should properly pick up your proxy settings and work.
Extending jhbuild with custom modules or newer versions
The last two sections refer to building the optional module dconf inside the jhbuild to solve some specific and well known issues, but if you still need to build more modules inside that jhbuild, or just newer versions of what already is available in WebKitGTK+'s development environment, then check Extending the WebKitGTK+ JHBuild for specific instructions on how to do that.
