Tips and Tricks for using Git with WebKit
- Proposed workflow for using GitHub for WebKit development
- How to setup an always up-to-date git-svn mirror of QtWebKit
- Proposal for moving WebKit development to Git
You can download Git binaries directly from the official site! Or:
- Update your Git install: http://code.google.com/p/git-osx-installer/
- Update your SVN install (for git-svn bindings): http://www.open.collab.net/downloads/community/
- Mac port:
sudo port install git-core +svn+bash_completion
sudo apt-get install git-core git-svn
Install git for Cygwin using the Cygwin installer.
Cygwin's git is functionally complete but has performance issues. For better performance, you can use msysgit (note that you still need Cygwin's git to work with some of WebKit's scripts):
- Download the latest msysgit full installer. You want the file with the label "Full installer for official Git", like Git-N.N.N.N-*.exe, not the mysysGit-* files.
- When installing msysgit, tell the installer to:
- Use Git Bash only
- Checkout as-is, commit as-is
- Choose Start > Programs > Git > Git Bash
- Create a file named .bashrc in your Git Bash home directory that contains the following:
cd /c/cygwin/home/<username> export HOME=$(pwd)
- Relaunch Git Bash. The shell should be using your Cygwin home directory as its home directory.
Whenever you want to run a git command manually, use Git Bash. You can also run prepare-ChangeLog from Git Bash.
To checkout WebKit using git:
git clone git://git.webkit.org/WebKit.git WebKit
Run the following command to automatically configure the local clone to follow subversion:
Or follow the instructions below.
If you want to be able to commit changes to the Subversion repository, or just want to check out branches that aren't contained in WebKit.git, you will need track the Subversion repository. To do that, inform git-svn of the location of the WebKit SVN repository, and update the branch that git-svn uses to track the Subversion repository so that it will re-use the history that we've already cloned from git.webkit.org rather than trying to fetch it all from Subversion:
cd WebKit git svn init --prefix=origin/ -T trunk http://svn.webkit.org/repository/webkit git config --replace svn-remote.svn.fetch trunk:refs/remotes/origin/master
This will add the following section to your .git/config:
[svn-remote "svn"] url = http://svn.webkit.org/repository/webkit fetch = trunk:refs/remotes/origin/master
You can then run the following command to have git-svn rebuild its metadata from your local repository, and to pull any recent commits from the WebKit SVN repository.
git svn fetch
If you're not tracking the Subversion repository the following command will fetch new commits from git.webkit.org:
You can then merge or rebase your local branches with origin/master to pick up the new commits.
If you are tracking the Subversion repository, this command will fetch information about new commits from Subversion, reset your local branch to match Subversion exactly, and then apply your local commits on top:
git svn rebase
Use the -l|--local flag to tell git svn rebase to look for new commits in the local repository. This makes it possible to pull from git.webkit.org and then update the svn branches quickly:
git svn rebase -l
If you'd like to fetch new commits from the Subversion repository without moving your local branch, you can use the following command:
git svn fetch
Note that 'git svn fetch' will be way faster if done after "git fetch" or "git pull", since it'll realize it already has all the revisions locally.
Commit manually through git-svn directly
If you have been granted commit access to WebKit's SVN repository it is possible to work entirely with git and to commit through git-svn, however using webkit-patch land (see next section) is encouraged since it deals with changelogs, commit logs and bugzilla for you.
After you have configured your working copy to track the Subversion repository you can:
- Create a tot_staging branch or whatever name you choose
- Apply a patch, cherry-pick a commit, or even merge a branch if it has been reviewed
- Run git svn rebase and fix any ChangeLog conflicts that might result
- Ensure the git log entry for your local commit contains an accurate copy of all the Changelog entries for your commit.
- And then when everything is ready--
git svn dcommitSince "git svn dcommit" creates a revision in the subversion repository for each local commit, you may need to squash (i.e. combine) commits to ensure that your commit to the WebKit repository will create just one revision. You can do this, for example, by using--
git rebase -i HEAD~nwhere n is the number of commits you want to see in the interactive editor. You can use git commit -a --amend for example to amend an existing local commit and avoid creating additional commits that you may need to squash later on.
As you may have guessed from step 4 above, "git svn dcommit" does not use the "commit-log-editor" setting to create a commit message to store in the remote Subversion repository. Instead it simply uses the commit message already associated to the local commit, so you need to ensure that it is an accurate copy of all your commit's Changelog entries. This is somewhat different from committing with Subversion, where "svn commit" does intervene with "commit-log-editor" to create a commit message for the remote repository.
WebKit Script support for Git
- Important git config settings for preparing ChangeLogs.
git config --global user.name "Foo Bar" git config --global user.email "firstname.lastname@example.org"
webkit-patch and check-webkit-style commands all work with git. By default all webkit-patch commands will treat all changes in your branch as a single commit (i.e. all local commits + working copy changes get uploaded/committed as a single commit). To operate on a specific commit, use --git-commit=commitish or "-g commitish". commitish can be a single commit (e.g. HEAD~1), a commit range (e.g. HEAD~3..HEAD~1, operates on HEAD~2 and HEAD~1 as a single commit) or the working copy (i.e., HEAD..).
The various scripts in Tools/Scripts have been made to work pretty well with Git. Here are some of the specific things you can do with them:
- Telling Git to use resolve-ChangeLogs automatically as a merge-driver for ChangeLogs if you use a rebase-based workflow
git config merge.changelog.driver "perl Tools/Scripts/resolve-ChangeLogs --fix-merged --merge-driver %O %A %B"
If you have a merge-based workflow, use this instead or your ChangeLog entries will merge to the middle of the file instead of the beginning
git config merge.changelog.driver "perl Tools/Scripts/resolve-ChangeLogs --fix-merged --merge-driver %O %B %A"
- Teaching Git to produce better diffs of ObjC files
git config diff.objcpp.xfuncname "^[-+@a-zA-Z_].*$" git config diff.objcppheader.xfuncname "^[@a-zA-Z_].*$"
- Telling the various scripts to append the git branch name to every build. This is especially useful so you don't clobber your previous branch's build when you switch branches
git config core.webKitBranchBuild (true|false) //the default is off
- Overriding the core.webKitBranchBuild setting for a specific branch
git config branch.$branchName.webKitBranchBuild (true|false)
The following are only needed if you don't use webkit-patch.
- Using prepare-Changelog with git
Tools/Scripts/prepare-ChangeLog --git-commit=$committish --git-reviewer="Foo Reviewer"
- Using resolve-ChangeLog with git. Assuming you got a conflict merging a ChangeLog file, this tool will reapply the patch using patch --fuzz=3 so that your change lands at the top of the ChangeLog file. If the patch was successfully applied, git-add is run on the ChangeLog file. Note that this tool does not change the date of the ChangeLog entry (unlike svn-apply).
Tools/Scripts/resolve-ChangeLogs path/to/ChangeLog [path/to/ChangeLog ...]
- Using commit-log-editor with git will automatically insert the ChangeLog entry as your commit message
git config core.editor "perl Tools/Scripts/commit-log-editor"If you want to make sure log gets regenerated from ChangeLog entry each time you modify an already existing commit, use --regenerate-log:
git config core.editor "perl Tools/Scripts/commit-log-editor --regenerate-log"
- If you do not manually generate a ChangeLog entry and you have staged changes in your working tree, commit-log-editor will automatically generate a commit message in the WebKit ChangeLog entry format when you do 'git commit'. You can control this behaviour with the git configuration option webkitGenerateCommitMessage on a global or per-branch basis.
git config core.webkitGenerateCommitMessage (true|false) //the default is true git config branch.$branchName.webkitGenerateCommitMessage (true|false)
Misc. Tips and Tricks
- You can setup Git shell completion and branch name in your bash prompt. In your <path-to-git-source>/contrib/completion directory you will find a 'git-completion.bash' file. The command "git --exec-path" may help you determine your path to git. Follow the instructions in that file to enable shell completion. Here is a nice bash prompt for instance
PS1='\[\033[41;1;37m\]\u@\h:\[\033[40;1;33m\]\W$(__git_ps1 " (%s)")>\[\033[0m\] '
- You can set up multiple working directories to work on more than one branch at a time. In your /path/to/git/source/contrib/workdir directory you will find a 'git-new-workdir' script that can create new working directories. The usage is
./git-new-workdir <repository> <new_workdir> [<branch>]
- Colorize various git commands
git config --global color.status auto git config --global color.diff auto git config --global color.branch auto
- If you're using git-send-bugzilla or webkit-patch you may also want git to remember your bugzilla credentials:
git config --global bugzilla.username "email@example.com" git config --global bugzilla.password "yourpassword"
- If you're committing binary files such as pixel results, you should set the svn:mime-type property on those files. You can have SVN do this automatically and git will respect those settings. Put the following in your ~/.subversion/config file:
[miscellany] enable-auto-props = yes # Add an additonal line for each type of binary file you wish to commit. [auto-props] *.jpg = svn:mime-type=image/jpeg *.pdf = svn:mime-type=application/pdf *.png = svn:mime-type=image/png *.webp = svn:mime-type=image/webp
- If there are file system crawlers on your system that might be touching your working tree (as seems to happen when Xcode is running), you should tell git to ignore st_ctime information when checking if paths in the working tree have changed:
git config --global core.trustctime false
- If you have several bugs on the go at once and have a separate branch for each, it can be a pain to switch between branches because it will invariably clobber your build. To avoid rebuilding the world every time you switch branches use the following simple trick. Let's assume you need to switch to the branch that contains your work for bug 7000 (in a branch called '7000').
git checkout -b 7000-2 git cherry-pick refs/heads/7000
- You can setup your git repository to ignore the same files that are ignored in the tracked Subversion repository with: (this will take some time)
git svn show-ignore >> .git/info/exclude
If you are not familiar with Git and wish to see a sample workflow for making modifications, see the web inspector tutorial.
Maintaining Good Performance
git-gc can be used to make your git repository take up less space and increase git's performance. Some git commands will run a "lite" version of git-gc automatically, but you may want to run it manually from time to time. Running it is as simple as:
It is a good idea to repack occasionally, like so (be warned that this can take multiple hours!):
# The following git-config commands are only needed on Windows AFAIK git config pack.threads 1 git config pack.windowMemory 256m # The following commands achieve excellent compression without wasting effort and preserve low-latency reads git config --local pack.compression 9 # If this is the first repack since setting pack.compression, add -F to the command below. If repack doesn't recognise -F, just call it without it. git repack -adf --window=65 --depth=3 # To avoid repacking ancient history, keep the resulting pack touch $(ls .git/objects/pack/pack-*.idx|sed s/idx/keep/)
Breaking Up Large Patches
Let's say you've made a few commits on your master branch while fixing a bug, and now you want to break up those changes into logical patches to post for review:
trunk \/ T - A - B - C - D ^ master ^ HEAD
What I sometimes do in this case is make a new branch based on trunk (let's name the branch "prepare" in this example) and check it out:
git branch trunk prepare git checkout prepare
(Or you can do this in a single command: git checkout -b trunk prepare.)
trunk \/ T - A - B - C - D ^ ^ prepare master ^ HEAD
Then I'll selectively apply changes from master to my working tree and index using "git checkout -p". Just like "git add -p" lets you interactively choose chunks of a file to add to the index, "git checkout -p" lets you interactively choose chunks of a file from another commit to apply to your working tree and index.
So, perhaps file1 has some changes in commit C that I'd like to put into my first patch:
git checkout -p master~1 file1
("master~1" means "one commit before master".)
I'd do this for as many changes/files as I think seem appropriate for the first patch, then write a ChangeLog and upload the patch to Bugzilla:
webkit-patch upload -g HEAD....
(The "-g HEAD...." flag (note the 4 dots!) tells webkit-patch only to consider the changes you have in your working tree and index. This is an important flag to pass if you have multiple commits on your current branch that haven't been pushed back to Subversion yet.)
Then I'd commit the current patch:
git add file1 ChangeLog git commit
Then I'd repeat as needed until all the changes from master are represented in my prepare branch and uploaded to Bugzilla. At this point I'd probably delete my master branch and rename prepare to master:
git branch -D master # Deletes master git branch -m master # Renames current branch to master
(Renaming the branch to master doesn't really make any difference to git, but I find it easier to keep track of things this way.)
Once all the changes have been reviewed, I'd use the Tools/Scripts/git-add-reviewer script to add the right reviewer names to all the commits on my current branch that are newer than trunk:
git add-reviewer -i trunk
Then I'd rebase my patches on top of trunk and commit them back to Subversion:
git svn rebase git svn dcommit
And finally mark the bugs fixed in Bugzilla:
webkit-patch mark-bug-fixed 12345 webkit-patch mark-bug-fixed 12346 webkit-patch mark-bug-fixed 12347
Copying file aka svn copy
The Git does automatic tracking of copied files like what "svn copy" does. Although, on default, git checks modified source file and new file. There are options which we should tell git to check unmodified source files, -C and --find-copies-harder.
Here is sample workflow how to use -C and --find-copies-harder with "git svn dcommit":
- Copy files
- Update ChangeLog
- git commit and put contents of ChangeLog into git commit log. It will be svn commit message
- git svn dcommit -n -C50 --find-copies-harder Dry run
Output may be
Committing to http://svn.webkit.org/repository/webkit/trunk... diff-tree b6f4e217870317390ebcd2f4fe7410b8b5220b96~1 b6f4e217870317390ebcd2f4fe7410b8b5220b96
- git diff-tree --patch -C50 --find-copies-harder b6f4e217870317390ebcd2f4fe7410b8b5220b96~1 b6f4e217870317390ebcd2f4fe7410b8b5220b96
- Output should be "copy from" and "copy to"
- git svn dcommit -C50 --find-copies-harder
Output should be looks like:
Committing to http://svn.webkit.org/repository/webkit/trunk... C Source/WebCore/Foo.cpp C Source/WebCore/ChangeLog Committed r123456 M Source/WebCore/ChangeLog