Relocating Subversion Externals
November 10th, 2007
So I recently bought a shiny new 24″ iMac. One of the first things I tried to do was to move my Subversion repository onto it. In Tiger I never did get around to running it in Apache, but this time around I was determined. As it happens, it was incredibly easy.
It wasn’t long before I hit my first snag – Externals Definitions. From the Subversion book:
An externals definition is a mapping of a local directory to the URL—and possibly a particular revision—of a versioned resource.
Commonly, an externals definition will be used to ensure that your project always pulls the most recent stable version of a third-party library. Less commonly though, and in my case, they are used for dependency management. For example, I have the TV Forecast project and the TV Countdown project. The common code between these two projects is in the TV project. So basically, my Subversion repository looks like this:
/TV Forecast/trunk/ /TV Countdown/trunk/ /TV/trunk/
The svn:externals property is applied to a directory. Given that my Repository is located at file:///Repository, the value of the svn:externals property applied to the /TV Forecast/trunk and /TV Countdown/trunk directories is:
TV file:///Repository/TV/trunk/
Therefore, when the trunk of TV Forecast or TV Countdown is checked out, it will check out the trunk of the TV project into a TV directory. I’ve found externals definitions to be very useful when managing releases as releasing is as simple as:
svn export 'file:///Repository/TV Forecast/trunk' 'TV Forecast'
That is, I don’t need to go around fetching dependencies.
Unfortunately, there is an enormous downside: externals definitions don’t relocate easily. Now that my iMac is hosting my Subversion Repository from Apache, the new Repository location is http://Peanut/svn. If I try to check out the head of TV Forecast, Subversion will bail telling me that the external cannot be found. Sure, I could just modify the property, but this will only affect the repository from the current revision onwards; It won’t help me to check out prior revisions. So what I’m going to need is a way to retrofit the new Repository location into all existing values of the svn:externals property. The solution? Hack the repository!
First things first, the repository needs to be serialized. Subversion allows for an entire repository to be serialized to and from a ‘dump file’ using two commands on svnadmin; dump and load. For example:
svnadmin dump OLD_REPO > dumpfileold.data Magic svnadmin load NEW_REPO < dumpfilenew.data
To begin, I found SvnDumpTool. This is a package of Python scripts designed to simplify hacking at dump files. It came very close to doing what I required, but not quite. The transform-prop command allows you specify a regular expression and a replacement string. This would almost do it, except that the svn:externals property can specify an arbitrary number of local directory/URL pairs. What I really needed was a simple ‘Find & Replace’ that operated within property values – so I added it.
This package contains the 0.4.0 release of SvnDumpTool patched to include the ‘replace-prop’ command. So in my case, as I’m trying to migrate my externals definitions from file:///Repository to http://Peanut.local/svn, I use the command:
svndumptool.py
replace-prop
svn:externals
file:///Repository
http://Peanut.local/svn
dumpfileold.data
dumpfilenew.data
You should then verify the changes with:
svndumptool.py diff dumpfileold.data dumpfilenew.data
Good luck.
So Perforce isn’t so bad then… Actually looking around for a food VCS that i can use throught the web and a proxy…
I love changing the past…
Cool… Thanks. If your patch works, I’ll be happy. I’m changing an svn:// to a slightly different http://. Same problem.
Looks like this feature is in 0.5.0… thanks for bringing it to my attention.
I had no trouble switching svn:externals definitions from “svn+ssh://” to “svn://”
Maybe this is a unique side effect of not changing it to http. More likely, it’s a side affect of having more current software, SVN 1.5.
Perforce isn’t so bad. But it’s not free, either.