Tutorials

svnserve HowTo

How To Install an SVN Server the Easy Way on your Mac

This tutorial loosely follows the instructions given in this blog.

Ever wondered how to get a svn server running on your Mac? Here is how:

On MacOS X 10.5 and later a svn server is already installed. To work with svn you have to prepare a so called source code repository. You can do so as follows:

Running svnserve

  • Create a new user subversion and log in as this user (standard user, no admin user).
  • Create a Repositories folder in the home of the subversion user.
  • Change the permissions of the folder: chmod o-rwx Repositories
  • Start the server by typing /usr/bin/svnserve --inetd --root=/Users/subversion/Repositories & in the unix shell and let the server run in the background.

Creating a new repository

  • Now as the user subversion use the svnadmin tool to create a repository, for example:
    • svnadmin create /Users/subversion/Repositories/myrepo
  • Once the server is running, you can log in as a different user and manage the versions of your source code with the svn tool. The most common operations are “svn checkout”, “svn add”, “svn update” and “svn commit” commands. For more information how to actually work with svn see the svn handbook.

Testing

  • A final simple test to see if it worked is to checkout your repository:
    • If you have a Mac that is reachable in your local network as http://mac.local, the address for the new repository above would be svn://mac.local/myrepo. We checkout this repository in a local directory using the unix shell:
    • svn checkout svn://mac.local/myrepo
      • This creates a local copy of the latest version of your repository.
      • Of course the local copy is empty initially. See below how to populate your repository with something meaningful.

Auto-launching svnserve

If you did’t get an error message, we have a working svn server, but we are not yet finished. The main part covered by this tutorial, is to get the server running automatically when a svn request comes in and to provide password access for each served repository.

So how do we start the server automatically? We replace the manual startup of the server on the unix shell by preparing a so called plist file for the MacOS X launch daemon:

We edit a new file org.tigris.subversion.svnserve.plist in /Library/LaunchDaemons and put the following text in it: 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Debug</key>
        <false/>
        <key>GroupName</key>
        <string>staff</string>
        <key>Label</key>
        <string>org.tigris.subversion.svnserve</string>
        <key>OnDemand</key>
        <true/>
        <key>Program</key>
        <string>/usr/bin/svnserve</string>
        <key>ProgramArguments</key>
        <array>
                <string>svnserve</string>
                <string>--inetd</string>
                <string>--root=/Users/subversion/Repositories</string>
        </array>
        <key>ServiceDescription</key>
        <string>SVN Version Control System</string>
        <key>Sockets</key>
        <dict>
                <key>Listeners</key>
                <array>
                <dict>
                        <key>SockFamily</key>
                        <string>IPv4</string>
                        <key>SockServiceName</key>
                        <string>svn</string>
                        <key>SockType</key>
                        <string>stream</string>
                </dict>
                <dict>
                        <key>SockFamily</key>
                        <string>IPv6</string>
                        <key>SockServiceName</key>
                        <string>svn</string>
                        <key>SockType</key>
                        <string>stream</string>
                </dict>
                </array>
        </dict>
        <key>Umask</key>
        <integer>2</integer>
        <key>UserName</key>
        <string>subversion</string>
        <key>inetdCompatibility</key>
        <dict>
                <key>Wait</key>
                <false/>
        </dict>
</dict>
</plist>

This plist file describes that a svn:// request will launch the /usr/bin/svnserve program under the permissions of the user subversion and that repositories are managed in the Repositories folder of this user.

We now start the svn server:

sudo launchctl load /Library/LaunchDaemons/org.tigris.subversion.svnserve.plist

The mapping of a service to a TCP port is described in /etc/services. The lines 

svn             3690/udp    # Subversion
svn             3690/tcp    # Subversion

denote that a tcp (or udp) request on port 3690 will be served by the “svn” service, for which svnserve is launched.

Access Control

We’ve installed a server and made sure it launches, however, we still have to do some last set-up work on the server. For security reasons, the server is locked down. Nobody can write to it. So what you have to do is go into your repository folder and edit the conf/svnserve.conf inside it that svnadmin helpfully created for you.

It says the default access permissions are auth-access = write. This tells the server that write access will be granted to everyone who provides a valid user name and password, and anon-access = read, which means that anyone can read the files even without a user name and password. You might want to change that first line to anon-access = none, if you want only people who have a valid user name and password to see your code in the repository.

Next, remove the comment sign “#” and the space at the start of the line password-db = passwd. This will make svnserve look for a file passwd right next to svnserve.conf as its source of user names and passwords. It contains two sample entries that will seem familiar if you’ve read The Subversion Book.

Simply replace those entries with username = password pairs of your own and do not forget to restart the server.

SVN usage example

You can test your permission setup with a “hello svn” message:

  • After checking out a repo in a local directory, create the C file hello.c in that local copy:
#include <stdio.h>
int main()
{
   printf("hello svn\n");
   return(0);
}
[Get Code]
  • Then add this file to version control:
    • svn add hello.c
  • And commit this local change to your repository:
    • svn commit
    • Note: For each commit you need to provide a comment like “initial commit”, the so called commit log.
  • The above commit should have failed, because you did not provide a user name and password!
  • Next try:
    • svn --username="username" --password="password" commit
  • That should have worked!
  • Now modify your file:
    • E.g. replace the “hello svn” phrase with “hello svn, you work so sweet!”.
  • And show the changes with:
    • svn diff
  • Show the modified files with:
    • svn stat
  • If those changes are ok, then commit them to the repository:
    • svn commit
    • Note: svn stores your password so that you do not need to retype every time you commit.
    • Comment: When developing source code, it is mandatory to review your changes before actually committing them. Other developers will just beat the hell out of you, if you broke the build!
    • Each change will result in a new version with increasing revision number.
  • You are now at revision #2.
  • To move to the svn head (the latest development revision) use the svn update command:
    • svn up
  • You can see the log of your code development:
    • svn log
  • Of course you should do something meaningful with your code, compile and execute it:
    • gcc hello.c -o hello && ./hello

That’s it. You are using (sub)version control!

WebSVN

Finally, we add a web front end to the subversion repositories. There are thousands of alternatives (like ViewCV and indefero), but for this tutorial we use WebSVN, because it is a snap to install:

  • Download WebSVN from http://www.websvn.info.
  • Unpack the archive to the html directory of your Apache web server.
  • Rename the package to websvn.
  • Copy the websvn/include/distconfig.php file to websvn/include/config.php and make the following modification:
    • Uncomment the line reading config→parentPath to point to your repositories, that is:
      • $config->parentPath('/Users/subversion/Repositories/');
  • Now open http://mac.local/websvn in your browser and you will see all your repositories in the web front end. Ain’t that sweet and cool.

To be on the safe side, we also should disable the websvn contents from being crawled. We do so by copying the following “robots.txt” into the html directory of the web server: 

User-agent: *
Disallow: /websvn

Automation

If you need to add repositories frequently, here is a script that automates that task. It creates a new repository for a given user name and sets a random password: 

#!/bin/tcsh -f

set user = $1
if ($user == "") then
   echo please specify a svn user
   exit 1
endif

set repo = ~subversion/Repositories

echo creating a svn repository for the user $user
svnadmin create $repo/$user

echo enabling the password-db
echo "[general]" > $repo/$user/conf/svnserve.conf
echo password-db = passwd >> $repo/$user/conf/svnserve.conf

echo restricting access to authorized users
echo anon-access = none >> $repo/$user/conf/svnserve.conf

set password = `date | /sbin/md5 | head -c8`
echo setting the password to $password
echo $user = $password >> $repo/$user/conf/passwd

echo created a svn repository to be checked out via the following command line:
echo svn co --username=$user --password=$password svn://$HOST/$user

It you have a valid email address for the particular user, then you can also email the password automatically: 

echo sending the password as email
sudo postfix start
echo "the password of your svn account $user is $password" | mail -s "Your SVN account@$HOST" user@address

The latter requires postfix to be setup. As a minimum postfix configuration, the relayhost needs to be set in /etc/postfix/main.cf (append “relayhost = …”). How to setup postfix with gmail see here.

Here is a more elaborate script adds a new user to a repository and emails the password to a given email address: 

#!/bin/tcsh -f

set name = $1
if ($name == "") then
   echo "please specify a svn repo name"
   echo "and specify a new user name (or email address) as optional second argument"
   echo "and specify a repository prefix as optional third argument"
   exit 1
endif

set email = $2
set prefix = $3

set repo = ~subversion/Repositories

if ($prefix != "") then
   set repo = $repo/$prefix
   if (! -e $repo) mkdir $repo
endif

set masteruser = "..."
set masterpassword = "..."
set masteremail = "svn@master.org"

set user = $name
if ($email != "") set user = $email
if ($email:s/@// == $email) set email = ""

if ($email != "") then
   set username = `echo $user | sed "s/@.*//g"`
   set userhost = `echo $user | sed "s/.*@//g"`
   set user = $username@`echo $userhost | /sbin/md5 | head -c4`
endif

set user = $user:s/@/_at_/
set usertime = `date "+%Y%m%d%H%M%S"`

set exist = 0
if (-e $repo/$name) set exist = 1

if ($exist != 0 && $user == $name) then
   echo the repository does already exist
   exit 1
endif

if ($exist == 0) then
   echo creating the svn repository "$name"
   svnadmin create $repo/$name

   echo enabling the password-db
   echo "[general]" > $repo/$name/conf/svnserve.conf
   echo password-db = passwd >> $repo/$name/conf/svnserve.conf

   echo restricting write access to authorized users
   echo anon-access = read >> $repo/$name/conf/svnserve.conf
   echo auth-access = write >> $repo/$name/conf/svnserve.conf

   # echo setting the master password for "$masteruser" to "$masterpassword"
   echo setting the master password
   echo $masteruser = $masterpassword >> $repo/$name/conf/passwd
endif

set password = `date | /sbin/md5 | head -c8`
echo setting the password for "$user" to "$password"
echo $user = $password >> $repo/$name/conf/passwd

set url = svn://$HOST
if ("$prefix" != "") set url = $url/$prefix
set url = $url/$name

echo setup the repository to be checked out via the following command line:
echo svn co --username=$user --password=$password $url

if ($email != "") then
   echo sending the password as email to $email

   set message = /var/tmp/svnmessage$user$usertime.txt

   echo "A subversion account has been created for you @ $HOST." >> $message
   echo "The password of your SVN account $user is $password ." >> $message
   echo "You can check out the repository "$name" via the following unix command:" >> $message
   echo "   svn co --username=$user --password=$password $url" >> $message

   cat $message | mail -s "Your SVN account @ $HOST" $email
   cat $message | mail -s "SVN account @ $HOST" $masteremail

   rm -f $message
endif

Example usage: 

./svnaddrepo.sh repo user@gmail.com
./svnaddrepo.sh repo anotheruser@gmail.com
...

An example email from the SVN server to the added user: 

A subversion account has been created for you @ schorsch.efi.fh-nuernberg.de.
The password of your SVN account xxxxxxxxxx@th-nuernberg.de is yyyyyyyy .
You can check out the repository zzzzzzzz via the following unix command:
  svn co --username=xxxxxxxxxx@th-nuernberg.de --password=yyyyyyyy svn://schorsch.efi.fh-nuernberg.de/zzzzzzzz

Cleaning Up

After the svn server has been running for a while, a couple of incidents can happen eventually:

First, the tmp folder may be filling up. To ensure that the disk is not running out of space, we should clean the tmp folder regularly.

To do so, we delete files in /var/tmp that are older than a week, if the modification time is older than 7 days: 

sudo find /var/tmp \( -mtime +7 -and -not -type d \) -exec rm {} \;

To let the server do this regularly we add another entry in /etc/crontab 

#min	hour	mday	month	wday	user	command
0	2	*	*	*	root	/etc/cleanup.sh

and provide a shell script in /etc/cleanup.sh that does the cleaning: 

#!/bin/tcsh -f

find /var/tmp \( -mtime +7 -and -not -type d \) -exec rm {} \;

Sometimes Apache may be responding very sluggishly. A reason can be that a httpd process is hogging the cpu due to a hung websvn script. To terminate those hung processes, we restart apache gracefully by adding the following line to the above cron job: 

/usr/sbin/apachectl -k graceful # httpd.conf: GracefulShutDownTimeout 1000

To ensure that hung processes get terminated eventually we need to set the GracefulShutDownTimeout in /etc/apache2/httpd.conf (the default value is zero, which means that hung processes do not experience a timeout): 

GracefulShutDownTimeout 1000

Also, under extremely heavy load by hundreds of clients, hung processes can accumulate so that the system almost grinds to a halt. This can happen especially when people are using aggressive download helpers for downloading large files. To prevent that, we need to limit the number of simultaneous httpd processes in /etc/apache2/httpd.conf (the Mac Mini’s default value of 1024 is far too large): 

MinSpareServers 8
MaxSpareServers 32
StartServers 1
MaxKeepAliveRequests 500
KeepAliveTimeout 5
MaxClients 512
ListenBackLog 256

But we also need to ensure that eventually all requests get served by our reduced set of httpd processes in a timely manner. If there are many large files being downloaded simultaneously, we can run out of serving httpd processes, so that another incoming request just sits around until another download finshes. This can take a while. As a consequence, the download of huge files needs to be limited. We are using the mod_bw module of Apache 2.0 for that purpose.

First we need to load the module by adding the following line to /etc/apache2/http.conf of our Mac Mini (for other Unix flavors this may differ): 

LoadModule bw_module libexec/apache2/mod_bw.so

Then, we can limit the access of large files by adding the following configuration to the virtual host definition (e.g. /etc/apache2/sites/0000_any_80_.conf): 

        # Limit band width for large files
        BandwidthModule On
        ForceBandWidthModule On
        BandWidth all 1000000
        LargeFileLimit .gz 1000 500000
        LargeFileLimit .zip 1000 500000
        LargeFileLimit .tif 1000 500000

If we would like to exclude a particular client ip or an entire subnet from a limitation, we can add a “MinBandWidth” directive to the configuration. For example, we can add the following line, which increases download speed for the subnet 141.75.0.0/16 to a minimum of 2MB/s while all others are served with a minimum of 50kb/s only (the order of directives is important, because the first item takes precedence): 

        MinBandWidth 141.75.0.0/16 2000000
        MinBandWidth all 200000

The limitation parameters are dependent on the band width of the actual internet connection of the server and cannot be generalized. More information about the limiting capabilities of the mod_bw module can be found here.

Now our server will not go down under heavily load and should be happily serving SVN, html and large file content for a long long time :)

To monitor that everything is fine, log on your server with “ssh” and have a look at “top”. The httpd processes should show up but neither the cpu load of the entire system nor the load of a single httpd process, that is serving plain http requests or downloads, should be more than a couple of percent.

To see what content is actually being served have a look at the access log: 

tail -f /var/log/apache2/access_log

Or have a look at the status page of the server:

http://localhost/server-status

You can also use CGI scripting to monitor your server more conveniently, as described in the CGI scripting tutorial. Here is the uptimee of my SVN server:

If anything goes wrong, you can still restart your server via ssh on the command line: 

sudo /usr/sbin/apachectl -k restart

Sometimes it is necessary to block a whole subnet due to malicious requests (in my experience mostly from china and russia/ukraine). For example, if we would like to block the entire chinese subnet 211.136.0.0, we add the following rule (number 100) to the firewall configuration: 

sudo ipfw add 100 deny all from 211.136.0.0/16 to any

To show the active rule set: 

sudo ipfw list

To delete the rule again: 

sudo ipfw del 100

To make the rules permanent, we need to add them to /etc/ipfilter/ipfw.conf: 

...
...
...
add 100 deny all from 211.136.0.0/16 to any

Unfortunately, the permanent rules do only show up after a reboot.

Options: