wiki:UsingGitWithWebKit

Version 93 (modified by Adrian Perez de Castro, 7 years ago) ( diff )

Add note about how to fetch branches using git-svn

  1. Tips and Tricks for using Git with WebKit
    1. Install
      1. Mac users
      2. ​Debian users
      3. Windows users
    2. Checkout
    3. Updating
    4. Commit manually through git-svn directly
    5. WebKit Script support for Git
    6. Misc. Tips and Tricks
    7. Ignores
    8. Workflow
    9. Maintaining Good Performance
    10. Breaking Up Large Patches
    11. Copying file aka svn copy
    12. Checking out Subversion branches

Tips and Tricks for using Git with WebKit

See also:

Install

You can download Git binaries directly from the ​official site! Or:

Mac users

​Debian users

   sudo apt-get install git-core git-svn

Windows users

Install git for Cygwin using the Cygwin installer.

Cygwin's git is functionally complete but has performance issues. For better performance, you can use ​msysgit (note that you still need Cygwin's git to work with some of WebKit's scripts):

  • ​Download the latest msysgit full installer. You want the file with the label "Full installer for official Git", like Git-N.N.N.N-*.exe, not the mysysGit-* files.
  • When installing msysgit, tell the installer to:
    • Use Git Bash only
    • Checkout as-is, commit as-is
  • Choose Start > Programs > Git > Git Bash
  • Create a file named .bashrc in your Git Bash home directory that contains the following: 
    cd /c/cygwin/home/<username>
export HOME=$(pwd)
  • Relaunch Git Bash. The shell should be using your Cygwin home directory as its home directory.

Whenever you want to run a git command manually, use Git Bash. You can also run prepare-ChangeLog from Git Bash.

Checkout

To checkout WebKit using git: 

   git clone git://git.webkit.org/WebKit.git WebKit

or 

   git clone https://git.webkit.org/git/WebKit.git WebKit

Run the following command to automatically configure the local clone to follow subversion: 

   Tools/Scripts/webkit-patch setup-git-clone

Or follow the instructions below.

If you want to be able to commit changes to the Subversion repository, or just want to check out branches that aren't contained in WebKit.git, you will need track the Subversion repository. To do that, inform git-svn of the location of the WebKit SVN repository, and update the branch that git-svn uses to track the Subversion repository so that it will re-use the history that we've already cloned from git.webkit.org rather than trying to fetch it all from Subversion: 

   cd WebKit
   git svn init --prefix=origin/ -T trunk http://svn.webkit.org/repository/webkit
   git config --replace svn-remote.svn.fetch trunk:refs/remotes/origin/master

This will add the following section to your .git/config: 

[svn-remote "svn"]
        url = http://svn.webkit.org/repository/webkit
	fetch = trunk:refs/remotes/origin/master

You can then run the following command to have git-svn rebuild its metadata from your local repository, and to pull any recent commits from the WebKit SVN repository. 

   git svn fetch

Updating

If you're not tracking the Subversion repository the following command will fetch new commits from git.webkit.org: 

    git fetch

You can then merge or rebase your local branches with origin/master to pick up the new commits.

If you are tracking the Subversion repository, this command will fetch information about new commits from Subversion, reset your local branch to match Subversion exactly, and then apply your local commits on top: 

   git svn rebase

Use the -l|--local flag to tell git svn rebase to look for new commits in the local repository. This makes it possible to pull from git.webkit.org and then update the svn branches quickly: 

   git svn rebase -l

If you'd like to fetch new commits from the Subversion repository without moving your local branch, you can use the following command: 

   git svn fetch

Note that 'git svn fetch' will be way faster if done after "git fetch" or "git pull", since it'll realize it already has all the revisions locally.

Commit manually through git-svn directly

If you have been granted commit access to WebKit's SVN repository it is possible to work entirely with git and to commit through git-svn, however using webkit-patch land (see next section) is encouraged since it deals with changelogs, commit logs and bugzilla for you.

After you have configured your working copy to track the Subversion repository you can:

  1. Create a tot_staging branch or whatever name you choose
  2. Apply a patch, cherry-pick a commit, or even merge a branch if it has been reviewed
  3. Run git svn rebase and fix any ChangeLog conflicts that might result
  4. Ensure the git log entry for your local commit contains an accurate copy of all the Changelog entries for your commit.
  5. And then when everything is ready-- 
    git svn dcommit
    Since "git svn dcommit" creates a revision in the subversion repository for each local commit, you may need to squash (i.e. combine) commits to ensure that your commit to the WebKit repository will create just one revision. You can do this, for example, by using-- 
    git rebase -i HEAD~n
    where n is the number of commits you want to see in the interactive editor. You can use git commit -a --amend for example to amend an existing local commit and avoid creating additional commits that you may need to squash later on.

As you may have guessed from step 4 above, "git svn dcommit" does not use the "commit-log-editor" setting to create a commit message to store in the remote Subversion repository. Instead it simply uses the commit message already associated to the local commit, so you need to ensure that it is an accurate copy of all your commit's Changelog entries. This is somewhat different from committing with Subversion, where "svn commit" does intervene with "commit-log-editor" to create a commit message for the remote repository.

WebKit Script support for Git

  • Important git config settings for preparing ChangeLogs. 
    git config --global user.name "Foo Bar"
git config --global user.email "foo@webkit.org"

webkit-patch and check-webkit-style commands all work with git. By default all webkit-patch commands will treat all changes in your branch as a single commit (i.e. all local commits + working copy changes get uploaded/committed as a single commit). To operate on a specific commit, use --git-commit=commitish or "-g commitish". commitish can be a single commit (e.g. HEAD~1), a commit range (e.g. HEAD~3..HEAD~1, operates on HEAD~2 and HEAD~1 as a single commit) or the working copy (i.e., HEAD..).

The various scripts in Tools/Scripts have been made to work pretty well with Git. Here are some of the specific things you can do with them:

  • Telling Git to use resolve-ChangeLogs automatically as a merge-driver for ChangeLogs if you use a rebase-based workflow 
      git config merge.changelog.driver "perl Tools/Scripts/resolve-ChangeLogs --fix-merged --merge-driver %O %A %B"

If you have a merge-based workflow, use this instead or your ChangeLog entries will merge to the middle of the file instead of the beginning 

  git config merge.changelog.driver "perl Tools/Scripts/resolve-ChangeLogs --fix-merged --merge-driver %O %B %A"
  • Teaching Git to produce better diffs of ObjC files 
    git config diff.objcpp.xfuncname "^[-+@a-zA-Z_].*$"
git config diff.objcppheader.xfuncname "^[@a-zA-Z_].*$"
  • Telling the various scripts to append the git branch name to every build. This is especially useful so you don't clobber your previous branch's build when you switch branches 
      git config core.webKitBranchBuild (true|false) //the default is off
  • Overriding the core.webKitBranchBuild setting for a specific branch 
      git config branch.$branchName.webKitBranchBuild (true|false)

The following are only needed if you don't use webkit-patch.

  • Using prepare-Changelog with git 
      Tools/Scripts/prepare-ChangeLog --git-commit=$committish --git-reviewer="Foo Reviewer"
  • Using resolve-ChangeLog with git. Assuming you got a conflict merging a ChangeLog file, this tool will reapply the patch using patch --fuzz=3 so that your change lands at the top of the ChangeLog file. If the patch was successfully applied, git-add is run on the ChangeLog file. Note that this tool does not change the date of the ChangeLog entry (unlike svn-apply). 
      Tools/Scripts/resolve-ChangeLogs path/to/ChangeLog [path/to/ChangeLog ...]
  • Using commit-log-editor with git will automatically insert the ChangeLog entry as your commit message 
      git config core.editor "perl Tools/Scripts/commit-log-editor"
    If you want to make sure log gets regenerated from ChangeLog entry each time you modify an already existing commit, use --regenerate-log: 
      git config core.editor "perl Tools/Scripts/commit-log-editor --regenerate-log"
  • If you do not manually generate a ChangeLog entry and you have staged changes in your working tree, commit-log-editor will automatically generate a commit message in the WebKit ChangeLog entry format when you do 'git commit'. You can control this behaviour with the git configuration option webkitGenerateCommitMessage on a global or per-branch basis. 
      git config core.webkitGenerateCommitMessage (true|false) //the default is true
  git config branch.$branchName.webkitGenerateCommitMessage (true|false)

Misc. Tips and Tricks

  • You can setup Git shell completion and branch name in your bash prompt. In your <path-to-git-source>/contrib/completion directory you will find a 'git-completion.bash' file. The command "git --exec-path" may help you determine your path to git. Follow the instructions in that file to enable shell completion. Here is a nice bash prompt for instance 
    PS1='\[\033[41;1;37m\]\u@\h:\[\033[40;1;33m\]\W$(__git_ps1 " (%s)")>\[\033[0m\] '
  • You can set up multiple working directories to work on more than one branch at a time. In your /path/to/git/source/contrib/workdir directory you will find a 'git-new-workdir' script that can create new working directories. The usage is 
    ./git-new-workdir <repository> <new_workdir> [<branch>]
  • Colorize various git commands 
    git config --global color.status auto
git config --global color.diff auto
git config --global color.branch auto
  • If you're using git-send-bugzilla or webkit-patch you may also want git to remember your bugzilla credentials: 
    git config --global bugzilla.username "name@example.com"
git config --global bugzilla.password "yourpassword"
  • If you're committing binary files such as pixel results, you should set the svn:mime-type property on those files. You can have SVN do this automatically and git will respect those settings. Put the following in your ~/.subversion/config file: 
    [miscellany]
enable-auto-props = yes

# Add an additonal line for each type of binary file you wish to commit.
[auto-props]
*.jpg = svn:mime-type=image/jpeg
*.pdf = svn:mime-type=application/pdf
*.png = svn:mime-type=image/png
*.webp = svn:mime-type=image/webp
  • If there are file system crawlers on your system that might be touching your working tree (as seems to happen when Xcode is running), you should tell git to ignore st_ctime information when checking if paths in the working tree have changed: 
    git config --global core.trustctime false
  • If you have several bugs on the go at once and have a separate branch for each, it can be a pain to switch between branches because it will invariably clobber your build. To avoid rebuilding the world every time you switch branches use the following simple trick. Let's assume you need to switch to the branch that contains your work for bug 7000 (in a branch called '7000'). 
    git checkout -b 7000-2
git cherry-pick refs/heads/7000

Ignores

  • You can setup your git repository to ignore the same files that are ignored in the tracked Subversion repository with: (this will take some time) 
    git svn show-ignore >> .git/info/exclude

Workflow

If you are not familiar with Git and wish to see a sample workflow for making modifications, see the web inspector tutorial.

Maintaining Good Performance

git-gc can be used to make your git repository take up less space and increase git's performance. Some git commands will run a "lite" version of git-gc automatically, but you may want to run it manually from time to time. Running it is as simple as: 

git gc

It is a good idea to repack occasionally, like so (be warned that this can take multiple hours!): 

# The following git-config commands are only needed on Windows AFAIK
git config pack.threads 1
git config pack.windowMemory 256m

# The following commands achieve excellent compression without wasting effort and preserve low-latency reads
 git config --local pack.compression 9
 # If this is the first repack since setting pack.compression, add -F to the command below. If repack doesn't recognise -F, just call it without it.
 git repack -adf --window=65 --depth=3

# To avoid repacking ancient history, keep the resulting pack
touch $(ls .git/objects/pack/pack-*.idx|sed s/idx/keep/)

Breaking Up Large Patches

Let's say you've made a few commits on your master branch while fixing a bug, and now you want to break up those changes into logical patches to post for review: 

trunk
 \/
  T - A - B - C - D
                  ^
                master
                  ^
                 HEAD

What I sometimes do in this case is make a new branch based on trunk (let's name the branch "prepare" in this example) and check it out: 

git branch trunk prepare
git checkout prepare

(Or you can do this in a single command: git checkout -b trunk prepare.) 

trunk
 \/
  T - A - B - C - D
  ^               ^
prepare          master
  ^
 HEAD

Then I'll selectively apply changes from master to my working tree and index using "git checkout -p". Just like "git add -p" lets you interactively choose chunks of a file to add to the index, "git checkout -p" lets you interactively choose chunks of a file from another commit to apply to your working tree and index.

So, perhaps file1 has some changes in commit C that I'd like to put into my first patch: 

git checkout -p master~1 file1

("master~1" means "one commit before master".)

I'd do this for as many changes/files as I think seem appropriate for the first patch, then write a ChangeLog and upload the patch to Bugzilla: 

webkit-patch upload -g HEAD....

(The "-g HEAD...." flag (note the 4 dots!) tells webkit-patch only to consider the changes you have in your working tree and index. This is an important flag to pass if you have multiple commits on your current branch that haven't been pushed back to Subversion yet.)

Then I'd commit the current patch: 

git add file1 ChangeLog
git commit

Then I'd repeat as needed until all the changes from master are represented in my prepare branch and uploaded to Bugzilla. At this point I'd probably delete my master branch and rename prepare to master: 

git branch -D master # Deletes master
git branch -m master # Renames current branch to master

(Renaming the branch to master doesn't really make any difference to git, but I find it easier to keep track of things this way.)

Once all the changes have been reviewed, I'd use the Tools/Scripts/git-add-reviewer script to add the right reviewer names to all the commits on my current branch that are newer than trunk: 

git add-reviewer -i trunk

Then I'd rebase my patches on top of trunk and commit them back to Subversion: 

git svn rebase
git svn dcommit

And finally mark the bugs fixed in Bugzilla: 

webkit-patch mark-bug-fixed 12345
webkit-patch mark-bug-fixed 12346
webkit-patch mark-bug-fixed 12347

Copying file aka svn copy

The Git does automatic tracking of copied files like what "svn copy" does. Although, on default, git checks modified source file and new file. There are options which we should tell git to check unmodified source files, -C and --find-copies-harder.

Here is sample workflow how to use -C and --find-copies-harder with "git svn dcommit":

  • Copy files
  • Update ChangeLog
  • git commit and put contents of ChangeLog into git commit log. It will be svn commit message
  • git svn dcommit -n -C50 --find-copies-harder Dry run Output may be 
    Committing to http://svn.webkit.org/repository/webkit/trunk...
diff-tree b6f4e217870317390ebcd2f4fe7410b8b5220b96~1 b6f4e217870317390ebcd2f4fe7410b8b5220b96
  • git diff-tree --patch -C50 --find-copies-harder b6f4e217870317390ebcd2f4fe7410b8b5220b96~1 b6f4e217870317390ebcd2f4fe7410b8b5220b96
    • Output should be "copy from" and "copy to"
  • git svn dcommit -C50 --find-copies-harder Output should be looks like: 
    Committing to http://svn.webkit.org/repository/webkit/trunk...
        C Source/WebCore/Foo.cpp
        C Source/WebCore/ChangeLog
Committed r123456
        M Source/WebCore/ChangeLog

See also ​WK-89155(failure example) and ​r120567 (successful example)

Checking out Subversion branches

You can edit .git/config to add a branches setting for the Subversion remote. For example, the following will make all the WebKitGTK+ release branches available as origin/webkitgtk-releases/*: 

[svn-remote "svn"]
        url = http://svn.webkit.org/repository/webkit
        fetch = trunk:refs/remotes/origin/master
        branches = releases/WebKitGTK/*:refs/remotes/origin/webkitgtk-releases/*

After editing the file, run git svn fetch. It may take a long time for it to complete depending on how many branches are locally missing for the updated configuration. The example above which fetches all the WebKitGTK+ release branches can take up to 10 hours to complete, so you may want to ​limit the amount of branches fetched.

Once the fetch has finished, you can checkout branches normally with Git, e.g. git checkout origin/webkitgtk-releases/webkit-2.16.6.

Note: See TracWiki for help on using the wiki.