Since I like to eat, and hence get paid so that I can afford food, I have to work on things that take longer than a week.  But luckily, git can help get a diff for review board based only on a string in the commit comments.

This came about when one of my co-workers posted a bash scripts someone had written that would take a git diff and make it into a svn-like diff.  And that worked great.  My first attempt was to modify it to do multiple files at once, but it ended up making a diff for each revision for each file (if you need that functionality, you can get that here).

So I pulled out python and worked on a new script, which is at the bottom of this post, or you can get it here.

Once you download the file, you run it like this from your git repository:

git-create-review.py (some grep string) > diff.output

The script will take the argument as a string to search for in the commit comments. This is thanks to the git log –grep=foo command available in git. It outputs directly to the terminal, which you can see I’m redirecting into diff.output.  Once you have the file, you can upload to Review Board as you normally would.

I did it by using a search string because we use JIRA to track our work in svn.  We have pre-commit hooks to ensure that we put a JIRA number on each commit.  This helps out git as it has something to search through to get the change list.

Known issues:

First: It will get all the changes between the first and last revision that had the search string.

Example:

I change File.java under JIRA-123 (now at r1).

Another change to File.java under JIRA-321 (now at r2).

I change the file again under JIRA-123 (now r3).

This script will take all changes between 1 and 3, even if I don’t want the JIRA-321 changes to show up.

Second: it doesn’t have any context around removed/moved files. It still should pick up a renamed file as just a completely new file, but it doesn’t have any context of renaming.  But I believe that’s the case with svn anyway.

Third:  I am by no means a good python programmer, and might not know all the conventions that people normally follow. The code below may cause eye hemorrhaging, face hemorrhaging…pretty much any of your organs could be hemorrhaged by looking at the code. CONSIDER THIS YOUR ONLY WARNING.

But I did some testing with newly created files, and removed files, and binary files, but I’m sure I missed some crazy edge cases.  Let me know if you hit any snags.

The script follows:

That person may be tremendously happy…that is, until, you try to “git svn fetch” to get new revisions and branched and what have you, and you get an error message like this:

Last fetched revision of (some branch or tag) was r3421, but we are about to fetch: r3421!

In an attempt to fix it, you may find this blog post, saying to remove the .ref files from your svn directory, which also proves fruitless.

The fix: modify the .git/svn/.metadata file so that the branches-maxRev and tags-maxRev equals the latest commit revision in svn.  So it should look like this after you are done:

; This file is used internally by git-svn
; You should not have to edit it
[svn-remote "svn"]
 reposRoot = (svn server name)
 uuid = c08781d2-f03e-0410-8c7c-e884ea3e41f3
 branches-maxRev = 17224
 tags-maxRev = 17224

Where 17224 was our latest revision.

I got the idea from git-svn’s webpage, where it states:

If the subset of branches or tags is changed after fetching, then .git/svn/.metadata must be manually edited to remove (or reset) branches-maxRev and/or tags-maxRev as appropriate.

I must caveat this with the fact that I have no idea why this works. Git svn is still black magic to me, and making this work is akin to me throwing muskrat bones onto a flaming pyre, and drawing conclusions from the charred remains.  Maybe someone with more git knowledge could drop it off here?

So you want to work with multiple svn branches (not to be confused with git branches). What you have to do is clone the svn repository in a different way, like this:

$ git svn clone https://somesvnrepo/svn/project -T trunk -b branches -t tags

This should work in most cases. The arguments to the -T, -b, and -t switches are the paths that the trunk, branches, and tags are in respectively. Typically, your branches and tags should be what I put above, but just in case it is different in your case, there you go.

Working with your shiny repo

After a short time in which humans achieve interstellar travel, git finishes cloning, and you have access to the entire svn repository. Now, let’s say you wanted to start working on trunk. Like before, you should checkout a branch, but this time it’s going to use trunk as it’s parent:

$ git checkout -b BranchOfTrunk trunk

But Lo! You have the entire repo, so you can checkout that branch from whatever branch you have in subversion.

$ git checkout -b BranchOfSubversionBranch subversion-branch

Now, you can switch between BranchOfTrunk and BranchOfSubversionBranch, and you’ll get all the code associated with trunk and subversion-branch respectively. No more checkout of a branch ever again, wasting precious keystrokes you could use to post to Facebook.

If a new svn branch is created on the server, make sure to run the following command to see it:

$ git svn fetch
. (updating blah blah)
.
.
$ git branch -a

The last command will show you all the branches, local (your own branches) and remote (every dumb svn branch), for this repository.

NOTE: git svn fetch and git svn rebase are almost exactly the same. fetch will update the entire git repository with anything not received from svn. rebase is concerned only with the parent of the branch you have checked out. If you recall last time, git svn rebase was equivalent to svn update. Now you can work with the same workflow as I mentioned before, working, committing locally, rebasing, etc.

Merging from subversion branch to trunk? BALDERDASH!

Now, let’s say that you want to commit a series of changes into a subversion branch, but you also want to commit the same changes in trunk also. What I’m going to show you is the workflow that I go through most of the time. I think it’s the most effective way to accomplish it in git.

First, you can run a fake dcommit.

$ git svn dcommit --dry-run
Committing to https://svnserver/svn/project/branches/subversion-branch ...
diff-tree b64fdc9250bbe04b3246214ade509e0b0d9d912d~1 b64fdc9250bbe04b3246214ade509e0b0d9d912d

The “–dry-run” is key here, for two reasons. First, you’ll be able to see what branch you are actually committing to in subversion. Not a bad idea, since you’re working in a git repo that has all your subversion branches.

Second, you’ll see a diff tree with some of the commit hashes, one corresponding to each time you did a commit locally, in order. You will want to keep the values of the hash around, because next we are going to “cherry-pick” them into trunk:

$ git checkout BranchOfTrunk
Switched to branch "BranchOfTrunk"
$ git cherry-pick b64fdc9
Finished one cherry-pick.
Created commit e4a7971:  Your commit comments here
 12 files changed, 58 insertions(+), 66 deletions(-)

You’ll notice I only used the first 7 digits of the hash (b64fdc9) to do the cherry-pick. That’s all you need to use…git will pick out the revision from just those seven characters. What this does is pulls the change from the other branch into this branch. If you have more than one commit, you should do this multiple times, in the order of the commits. But then you have the things you did on the branch ready to be checked into trunk!

This is the way I usually work. You may decide that doing a subversion merge is easier for what you are trying to do (I sometimes do also). I’d also like to know if there is an easier way to do this. This doesn’t seem the easiest way, but was the easiest way I could find so far. And I googled for like…20 minutes.

Hopefully this helps…someone. One person, at least.

But! Not all is lost.  You can use git with Subversion.  This can give you some of the benefits of using a DVCS without having to change everything you do.  I believe you can do this also with Mercurial…I went with git because I had read somewhere it had the best support to working with Subversion. AND RANDOM INTERNET BLOGS ARE ALWAYS RIGHT.

First of all, I will say that I used this as a guide when I was working.  It’s a great start to doing this stuff. I’ll also assume you can find and install git on your own, since you’re all-grow’d up now.

Step 1: Creating a local repo

Ok so let’s get started.  The first step is to get the subversion library.  That webpage I gave you gives you four different ways of doing it, but I decided to go with the fourth option, which is cloning the entire Subversion repository.  You can do that by doing:

git svn clone https://svnserver/project/trunk

Now, you should know that this could take a while.  You might as well go and eat dinner for the next three days while this completes.  Just kidding, this took me about an hour or so while I was working remotely over a VPN, so it’s not so bad.  Our project has about 8000 revisions, and about 230 MBs of files.

So you’re probably wondering “But now if I have the whole history, isn’t it going to take like…a bazillion bytes on my disk?”  To that I say “Non”…it uses compression, and my git repo ended up being about 1 MB bigger than my subversion one.

Step 2: Create a branch

From there, you can start working in git!  The first thing you’ll probably want to do is create a branch:

git branch someNeatoBranch

To change to that branch:

git checkout someNeatoBranch

Now you are working on this particular branch. Note you can do this in one step by doing:

git checkout -b someNeatoBranch

which will create it and put you on it to work.

Step 3: Commit locally

Now you can edit the files to your heart’s content.  And commit locally as often as you like, by doing

git commit -a

You can commit individual files by specifying them, but -a just grabs them all.

Step 4: Commit back to Subversion

Once you’re ready to commit back into the main Subversion repo, first you need to pull all the changes from there, to make sure there are no conflicts. You can do this by using this command:

git svn rebase

This will pull down the latest code and apply your code on top of it, merging if necessary.  Then to commit back to Subversion, you use:

git svn dcommit

And you will notice that your files are committed back to the repo without any more intervention from you!  But where are your comments?  Well, all the comments from the commits you did locally will be used to commit to Subversion.  It will also do that many commits.  So if you did 5 different commits locally, it will turn into 5 different Subversion commits.  As long as your team members don’t worry about the revision number spirialing up towards the heavens, it should be ok.

Step 5: Brunch!

This is a pretty simple workflow to get you started on using git with Subversion, but it’s not even half the story.  There’s so much more powerful stuff with the branching aspect, and I even haven’t tried it yet to share work with my coworkers before commiting to svn (I’m kinda like a git island at the moment).  But try it out, and then check out some of the resources below that helped me get started.

Also, you might want to check how my co-worker James one-upped me.  He was working on some highly experimental code that he wanted to share with a teammate, and created a true git repo in approximately 8 seconds, from existing code.

Resources

• An introduction to git-svn for Subversion/SVK users and deserters
• Git – SVN Crash Course