Subversion - Accessing servers

From LXF Wiki

Table of contents

Accessing Subversion servers as a client

(Original version written by Graham Morrison for Linux Format magazine issue 69.)


After covering Subversion servers in the previous tutorial, we stray on to the more familiar territory of Subversion clients - from where you can actually get some development work done.


Hopefully, after the previous Subversion tutorial, you were able to finally rid yourself of any pesky CVS servers you may have had languishing in some back room. In case you need reminding, the first tutorial covered setting up a Subversion repository, importing an existing project from another version control system, and administering the server once it was up and running. It basically covered everything you needed to move on to the next generation of versioning control software (now that there's no longer a free version of BitKeeper!).

This month is going to deal with a far more likely scenario for most users, and that's using Subversion as a client. The client is the part that accesses the server, and handles all the complicated details like which versions to chose and how to rectify any version conflicts. The client is the interface between your local work space copy of a project, the copies held by the other developers, and the reference copy held on the server.

Using the Subversion client is also a tempting way to download the very latest software fixes, such as security updates or cutting edge features. You need to be careful though, as the latest Subversion download is often in a constant state of flux. Any single day could see several changes, and using an ad hoc download for system critical work is unwise but sometimes unavoidable. It's a little like reading a book before the author has finished writing it.

This month's tutorial will focus on using the client as part of the development cycle, undoubtedly where Subversion gets the most use. With the same examples from last month, the four sections of this tutorial will cover downloading, modifying, uploading and analysing your work, which should be everything you need for making a real contribution to your favourite project.


Part 1 - Downloading a project from a Subversion server

The first thing to do when you need to work on a project hosted on a Subversion server is download your own working copy. We covered this last month, and all you need to remember is that we used the 'checkout' or co command:

$ svn co file:///usr/share/subres
A  subres/helloworld.cpp
A  subres/Makefile
Checked out revision 1

The path to your repository usually takes the form of 'svn://' which is an external connection to a repository located somewhere in the ether of the Internet. But you can also reference a local repository, as we created last month, using 'file://'. Working on a local repository makes it much easier to experiment with some of the concepts we'll cover, especially those that involve updating data. Remote repositories usually make it a little difficult to get write access, with good reason, and it's worth setting up a local repository to experiment with.

If you've still got what you think is a Subversion working copy from last month, you can check using the info command:

$ cd subres; svn info
Path: .
URL: file:///usr/share/subres
Repository UUID: 6319a556-cbf4-0310-b561-de3fb53aee82
Revision: 1
Node Kind: directory
Schedule: normal
Last Changed Author: graham
Last Changed Rev: 1
Last Changed Date: 2005-04-15 16:33:44 +0100 (Fri, 15 Apr 2005)

If you've been wondering how Subversion manages the differences between your working copy and those held on the repository, then you'll be pleased to know that the answer is contained within the hidden .svn directory - the administrative directory. This directory contains all the information Subversion needs to manage the differences between the two source trees, such as the file types and their modification dates. Nearly every change you make is reflected in the administration directory, and editing it by hand could only lead to disaster!


Getting to the repository

There are several ways of accessing a repository, depending on the protocol specified as the first part of the URL.

  • file:// Used for direct access to files stored on your file system
  • http:// This uses the WebDAV protocol configured Apache server
  • https:// As with the WWW, adds SSL encryption to a HTTP connection
  • svn:// Subversions own custom protocol supplied by a server running svnserve
  • svn+ssh:// The Subversion protocol augmented by an SSH tunnel for security
69_tut_subversion_03.png-thumb.png (http://www.linuxformat.co.uk/images/wiki/69_tut_subversion_03.png)
JSVN is a Java Subversion client that can help you manage your local Subversion working copy. From the user-interface you can view any changes that have been made, as well as check the local differences against the Subversion server.


Part 2 - Modifying files

When you've checked out your own working copy of a project, you can use the 'status' command to check for any modifications against the repository. If there are no modified files, there will be no output, but if there are, then they will be listed, along with a 'M' in the first column. We've edited the simple helloworld.cpp example from our repository (it contains only a single line), and performing a status command should reveal its modified state:

$ svn status
M      helloworld.cpp

In the above example, the helloworld.cpp file has been modified in the local working copy. You can check the differences between your local copy and that held on the repository using svn diff, which works in exactly the same way as the well-known 'diff' command, only bridging the two source tree locations.

69_tut_subversion_01.png-thumb.png (http://www.linuxformat.co.uk/images/wiki/69_tut_subversion_01.png)
This diagram illustrates what typically happens when there's a conflict in the merge process.

  • a - Both developer 1 and developer 2 have exactly the same version as the file in the Subversion repository
  • b - Developer 2 modifies their local copy of the file
  • c - Developer 2 commits the change back to the repository, while developer 1 makes their own changes to the same part of the file
  • d - Developer 1 is unable to commit their changes because there's a conflict with what Developer 2 has already committed
  • e - Developer one needs to update their working copy with Developer 2's changes, before resolving the conflict and making another commit to the repository.


$ svn diff
Index: helloworld.cpp
===========================
--- helloworld.cpp      (revision 1)
+++ helloworld.cpp      (working copy)
@@ -5,6 +5,7 @@
 int main(int argc, char *argv[])
 {
   cout << "Hello World!" << endl;
+  cout << "This is an addition." << endl;
   return(0);
 }

You can see that the line containing "This is an addition" has been flagged as different, denoted by the '+' symbol in the first column. As the file has been changed we need to copy our modified file to the repository so that other developers will be able to benefit from our addition. As with CVS, this is achieved using the 'commit' command, followed by entering a comment to describe the modification that's being committed. With this kind of development model, it's usually far better to make as many small committals as you can, using a descriptive comment, so that all changes can be easily traced when something inevitably goes wrong.

$ svn commit
Sending        helloworld.cpp
Transmitting file data .
Committed revision 2.

For any other developers working on the same repository to be able to take advantage of your changes, they would need to update their own working copy using the 'update' command. Their source tree should then be the same as yours.

$ svn update
U  helloworld.cpp
Updated to revision 2.


A Real Example

Kopete is the instant messenger that's installed by default as part of KDE. The trouble is that it often needs to be updated. Several of the messaging protocols Kopete uses to communicate with the server and other clients are 'reverse engineered' from a closed proprietary one. This is the only way to provide compatibility with networks such as AOL or MSN Messenger, but the protocols often mysteriously change without any notice. The Kopete developers typically analyse the flow of network traffic between the client and the server, and provide their own updates. These are always available first from Kopete's Subversion repository, as it takes a considerable amount of time to organise more convenient packages.

The first step to getting the latest version is to connect to KDE's anonymous Subversion server, accessible by all who are brave enough by typing the following:

$ svn co -N svn://anonsvn.kde.org/home/kde/trunk/KDE/kdenetwork
...
Checked out revision 416028.

Kopete is part of KDE's network package, which means it can be found under the kdenetwork directory. The '-N' flag, part of the above command, is used to download a single directory rather than let Subversion recursively download all other subdirectories. As a result, the above command downloaded only the files contained within the kdenetwork directory, so the next step is to get Kopete itself:

$ cd kdenetwork
$ svn update kopete
A  kopete
...
Updated to revision 416064.

Using the svn update command downloads any changes between your local copy and that on the repository. As we don't currently have a local copy of the Kopete directory, the result of running update is to download the whole directory. Each file or directory is displayed on the standard output as it is downloaded, preceded by an 'A' to show that the file has been added (other possibilities are U for updated, or D for deleted, among others). The final message is always the revision number of the version you've just retrieved.

As software repositories seldom contain the scripts necessary for building a project, the next step is to download the necessary scripts from the kde-common directory on the repository. We can then execute the commands you need to actually compile your local copy of Kopete.

$ svn co svn://anonsvn.kde.org/home/kde/trunk/KDE/kde-common/admin
$ make -f Makefile.cvs
$ ./configure --prefix=/usr
$ make
$ su -c "make install"


Part 3 - Merging changes

It can often be the case that two people are working on the same piece of code, at the same time. This is obviously going to cause a problem when Subversion attempts an update. If there are two developers who are working on the same source file, and the second attempts to commit their changes after the first has already done so, Subversion would complain using a rather cryptic message:

$ svn commit
Sending        helloworld.cpp
svn: Commit failed (details follow):
svn: Out of date: 'helloworld.cpp' in transaction '4'
svn: Your commit message was left in a temporary file:
svn:    '/home/graham/build/svn2/subres/svn-commit.tmp'

The previous example failed because the source file we were trying to update had already been updated by someone else. As you can see in the output, Subversion has made a copy of the commit message describing the change, so at least you don't need to type your detailed explanation of your changes again. Performing a 'diff' on the repository will show where the conflicting code has changed:

$ svn diff
Index: helloworld.cpp
===================================================================
--- helloworld.cpp      (revision 2)
+++ helloworld.cpp      (working copy)
@@ -5,7 +5,7 @@
 int main(int argc, char *argv[])
 {
   cout << "Hello World!" << endl;
-  cout << "This is an addition." << endl;
+  cout << "Another change in the output." << endl;
   return(0);
 }

The conflict is with the second 'cout' line, which was changed by the first developer causing a conflict for the second developer. Resolving the issue is accomplished in two stages, the first of which is to perform 'svn update' on the second developer's working copy. To make thinks a little easier, you can see the comments for each revision using the log command, and the duplicated files are given an extension that refers to the revision they're taken from. For example:

$  ls
helloworld.cpp       helloworld.cpp.r2  Makefile
helloworld.cpp.mine  helloworld.cpp.r3  svn-commit.tmp

helloworld.cpp will contain both the repository version and the contested modification, while helloworld.cpp.r2 is the previous version before any updates, and helloworld.cpp.r3 is the version currently held in the repository (because the repository is currently on its third revision). Subversion won't allow you to successfully execute the commit until the conflict has been manually resolved.

You can choose to copy one of the temporary files over the helloworld.cpp, effectively making the local version the equivalent of either a pre or post update working copy (helloworld.cpp.r2 or helloworld.cpp.r3 in our example). You could also use the 'svn revert' command to roll-back your local copy to a previous version, or you could get your hands dirty and resolve the conflict manually.

To continue using our example, the contents of the helloworld.cpp file currently look like this:

#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
  cout << "Hello World!" << endl;
<<<<<<< .mine
  cout << "Second modified addition." << endl;
=======
  cout << "First modified addition." << endl;
>>>>>>> .r3
  return(0);
}

The less-than, greater-than and equals symbols are all conflict markers. The text between the less-than and the equals symbols is the local modification that's causing the problem, which is why it's labelled 'mine'. The section after the 'equals' symbol is the current version held on the repository (at revision number 3 according to the label). We need to edit the offending code to try and accommodate both changes, perhaps by changing the text to something like "Both modified additions:

  cout << "Hello World!" << endl;
  cout << "Both modified additions." << endl;
  return(0);
<pre>

The next stage is to let Subversion know we've resolved the conflict, using the sensibly named 'resolved' command directly followed by the troublesome file. This should also remove both of the temporary files, hopefully leaving your working copy in a healthy enough state for a commit.

<pre>
$ svn resolved helloworld.cpp
Resolved conflicted state of 'helloworld.cpp'

$ svn commit
Sending        helloworld.cpp
Transmitting file data .
Committed revision 4.

$ svn update
At revision 4.


Part 4 - Further information

There are many different ways of getting status information from the repository, not least of which is the aforementioned 'status' command. When there's been a conflict though, one of the more useful is the 'log' command, which basically lists the comments associated with the latest commits. For example, the log for our helloworld.cpp example looks like this:

$ svn log
------------------------------------------------------------------------
r4 | graham1 | 2005-05-23 14:49:53 +0100 (Mon, 23 May 2005) | 2 lines

Resolved conflict by incorporating both changes.

------------------------------------------------------------------------
r3 | graham2 | 2005-05-23 12:39:56 +0100 (Mon, 23 May 2005) | 2 lines

Changed the text in the second line of output.

------------------------------------------------------------------------
r2 | graham1 | 2005-05-23 12:19:36 +0100 (Mon, 23 May 2005) | 2 lines

The addition of a second line of output.

------------------------------------------------------------------------
r1 | graham1 | 2005-04-15 16:33:44 +0100 (Fri, 15 Apr 2005) | 1 line

Initial project import
------------------------------------------------------------------------

You can see why having a descriptive comment for each commit is so important; it enables you to see at a glance exactly what might have gone wrong from the log. You can also trace who may have been responsible for any breakages, or who you may need to get in touch with to resolve any conflict. Each entry from the log starts with a revision number, followed by the developer, the time of the commit, and their comment for the update.

Another command that's helpful for resolving conflicts is the 'cat' command. Just like its shell counterpart that output the contents of a file, 'svn cat' displays the contents of the file in its current state on the Subversion server, so you can easily check any differences between your local version and the remote one. The clever part is that you can specify a specific revision as well, so you can look at previous versions and how the code may have changed.

$ svn cat --revision 2 helloworld.cpp
#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
  cout << "Hello World!" << endl;
  cout << "This is an addition." << endl;
  return(0);
}

In the above example, we were able to query the Subversion server for the second revision made to the repository for the helloworld.cpp file before all the conflict problems. You can also use the same 'revision' argument with the list command, to list the files contained within the repository for each revision, but the best use of 'revision' is with the diff command. Using revision with diff enables you to compare changes online from one revision to another. We could, for example, see what's changed between the first and fourth revision of the helloworld.cpp file:

$ svn diff --revision 1:4 helloworld.cpp
Index: helloworld.cpp
========================
--- helloworld.cpp      (revision 1)
+++ helloworld.cpp      (revision 4)
@@ -5,6 +5,7 @@
 int main(int argc, char *argv[])
 {
   cout << "Hello World!" << endl;
+  cout << "Both modified additions." << endl;
   return(0);
 }

The above output shows that the difference between revision 1 of helloworld.cpp and revision 4 of helloworld.cpp is the addition of the "Both modified additions" line, added in the previous stage of this tutorial. As the output of the svn diff is the same as the Shell command of the same name, the above command could be used to generate a input compatible with applications like KDE's Kompare, for graphically comparing the differences between the two revisions:

$ svn diff --revision 1:4 helloworld.cpp |kompare -o -
69_tut_subversion_02.png-thumb.png (http://www.linuxformat.co.uk/images/wiki/69_tut_subversion_02.png)
Nothing compares to KDE's difference viewer, imaginatively called Kompare.

As you you've made it through this month's Subversion tutorial, you should now feel confident enough to work on a project hosted on a Subversion server. KDE, for example, has recently migrated from CVS to Subversion, and it makes a great way of getting into the development cycle, even if you can't make your own changes until you've proven yourself. Next tutorial is the last in this short series on Subversion, and we'll be looking at some of the more technical aspects to using Subversion, such as branches and tags.


Patch work

A patch is a list of differences between two different versions of the same project and they're useful for two reasons. Firstly, they let you work on a project you don't have write access to by simply emailing your changes to a developer, and secondly, they let you send your changes to someone who can test them for you without taking the major step of committing them to the repository.

There's nothing intrinsically technical about a patch, it's basically the output of the 'diff' command which can compare two different files. The clever part is a program called 'patch', originally written by Larry 'I wrote Perl' Wall, that merges the differences from the patch file into the destination code. You can create the patch file from Subversion by piping the 'diff' output to a file:

$ svn diff > patch_file

You can then send the resulting patch file to the developer you wish to integrate or test your changes. Many developers get dozens of these patches daily, all of a varying quality, but that shouldn't put you off! The developer can then apply your changes to their local working copy using the patch command from the same location the patch_file was created:

$ patch -p0  < patchfile


Quick tip

It's easy enough to see the differences between two files when there are only a couple, but changes can often be far more numerous and complex. In these cases, you can make your life a little easier by using a graphical difference viewer. KDiff3 is one where you can see the differences across several files at once, and even supports three-way merging. KDE comes with its own viewer called 'Kompare' that opens automatically when it detects a patch file. It makes things much clearer by showing the original file on the left, and highlighting exactly where changes have been made against the new version of the same file on the right.