Create SVN (Subversion) web statistics on Debian Lenny Linux with mpy-svn-stats and svnstats.sh

Monday, 5th July 2010

I’ve recently desired to have a visualized statistics on how many commits, imports, people who commit into subversion’s repositories,graphs showing up the most active comitters, commits into the all subversion repositories grouped by month, week etc.
This kind of valuable information can give you insight, on a projects code life cycle. It can also help you to find out who takes most active participation into a certain project code development etc. and therefore could play a vital role in finding out about the work efficiency within your IT company or IT department.

There are plenty of softwares that can generate you some shiny statistics on how often and by whom are commits into your repositories as well as general statistics concerning your repositories accessibility.

Some of the projects suggested by most Linux users online, who had to resolve the same task e.g. (find some decent software to generate them good statistics on the svn use.) are:

1. statsvn

Here is a description on what statsvn is directly taken from its website:

StatSVN retrieves information from a Subversion repository and generates various tables and charts describing the project development, e.g.

StatSVN looks really promising, however what I find personally repulsive about it is that it depends on a Sun Java virtual machine
I have a bad taste for third party software that depends on java and therefore the software uses an XML dump generated from svn log –xml -v path/to/repos > svn-logfile.xml after which it’s necessary to pass the generated svn-logfile.xml file to statsvn, for instance:

statsvn [options] svn-logfile.xml path/to/repos

though a debian of statsvn is available and packaged for Debian in /usr/share/doc/statsvn/README.Debian we read:

Notes to Debian users:

* the jtreemap-based report has been disabled as jtreemap is currently
not packaged for Debian, and Debian cannot ship the applet without
its sources (not included in statsvn’s sources).

— Vincent Fourmond <fourmond@debian.org>, Tue, 4 Mar 2008 21:14:14 +0100

What I understood from statsvn documentation is that jtreemap is absolutely necessary in order to have a running statsvn, regardless if you have or you don’t have a java vm installed.

To take a general idea on what kind of Repo Roadmap does svnstat generates with jtreemap check out the following link

Since jtreemap is not available prepackaged for Debian I decided not to use svnstats though it looked quite superb.

Some further research on the kind of softwares available online able to generate me some statistics from cvs or subversion source repositories led me to,

2. svnplot

svnplot stroke me with it’s perfect looking many graphics generated on the Lines of Code commited, contribution of different authors to the repository, File count, avarage commit file sizes, common activity, author commit activity etc. etc.

I think tt’s worthy to check out some example statistics about a sample repository statistics generated by svnplot to get a better idea what to expect if you choose to install it on your server.

Even though svnplot looked also promising It wasn’t actually my choice because I think it’s not really mature enough as a software, the second reason which hold me back from installing it on my debian server was that I find it too much as a work in progress still.

Since neither svnstast nor svnplot didn’t well match my expectation and lacked a debian package I finally choose:

3. mpy svn stats as a solution to generate and graph information about svn usage

There are few reasons I finally took svn-mpy-stats to be the solution of choice.

1. It is available as a package in Debian Linux and easily installable via apt-get
2. It is written in Python and therefore doesn’t require a java virtual machine or some extra cosmetics to make it work3. It’s really simple and straight forward to configure and already tested and reported that it works well in Debian GNU/Linux

So here is the few simple steps I took to install mpy-svn-stats on Debian Lenny (in Debian Sid / Squeeze I suppose the procedure would be analogous.

– Install mpy-svn-stats via apt-get or aptitude

debian-server:~# apt-get install mpy-svn-stats

Run it for your svn repository with a command like:

debian-server:~# mkdir /var/www/svnstats
/usr/bin/mpy-svn-stats -o /var/www/svnstats/ file:///var/svn-repos/repository_name

In the above command substitute /var/www/svnstats/ and /var/svn-repos/repository_name with a directory of choice where you like to generate the html statistics for the svn usage as well as the full path and name of your repository.

Of course it’s a good idea to make mpy-svn-stats run periodically with for instance crontab or at or any other unix task cheduler available for your Linux system.

So far so good. You have probably already noticed that it’s rather inconvenient because you have to execute mpy-svn-stats command to each of your svn repositories individually.
This is absolute madness if your company is creating new svn source repository projects often, like let’s say everyday, because you will have to generate statistics for each of the repositories either manually or add new repositories manually to a script which will be later invoked by a crontab rule.

To get around this constrain, I’ve come up with a tiny shell script svnstats.sh which takes care for everything on it’s own.

It automatically will loop in your main subversion repositories directory through all the sub-repositories and generate individual html statistics in a separate automatically created directory by the script.

So to make your life easier and automate the process of generating stats with mpy-svn-stats consider downloading svnstats.sh and installing it as a separate rule like so:

debian-server:~# crontab -u root -e

Include therein the following:

# generate svn statistics everyday in 05:20 a.m.20 5 * * * /usr/sbin/svnstats.sh >/dev/null >>&1

Now everyday at 05:20 your mpy-svn-stats will generate a nice graphs and statistics for your subversion repository usage in /var/www/svstats, if you consider generating the data into a different location consider editting the head of mpy-svn-stats svnstats.sh script and change according to your likings.

Now let’s create an Alias in Apache to enable the (mpy-svn-stats generated by svnstats.sh) to be visualized via web:

– Edit VirtualHost configuration file of choice and put there, something like:

Alias /svnstats/ /var/www/svnstats/

Lastly it might be a good idea to use htaccess to protect your url with a password, afterwards you can enjoy your mpy svn statistics.

Share this on:

Download PDFDownload PDF

Tags: , , , , , , , , , ,

2 Responses to “Create SVN (Subversion) web statistics on Debian Lenny Linux with mpy-svn-stats and svnstats.sh”

  1. Benoit says:
    Firefox 3.6.6 Firefox 3.6.6 Windows XP Windows XP
    Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.2.6) Gecko/20100625 Firefox/3.6.6 ( .NET CLR 3.5.30729; .NET4.0E)

    Hi

    Thanks for looking at StatSVN. Please note that StatSVN is not directly depend on JTreemap, it runs without it. One page requires the jar file for the JTreeMap Applet.

    I was under the impression that JTreeMap had been packaged for Debian, it is one of our open source project on SourceForge, the code and jars are available like any other project. http://jtreemap.sf.net

    Having said that, I will have a look at finding a way to disable the report if required. This is likely to require a change in StatCVS, would you be willing to give it a go?

    regards

    Benoit
    @benoitx

    View CommentView Comment
    • admin says:
      Epiphany 2.29.92 Epiphany 2.29.92 Debian GNU/Linux x64 Debian GNU/Linux x64
      Mozilla/5.0 (X11; U; Linux x86_64; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Safari/531.2+ Debian/squeeze/sid () Epiphany/2.29.92

      Hi Benoit,
      It’s actually my pleasure to look at StatSVN. Also thanks for shedding some light informing that StatSVN is not actually dependant on jtreemap. Unfortunately in Debian testing (Squeeze / Sid) as this is the release of Debian I use on my workstation jtreemap deb package is not available.
      Not a problem to give it a go 🙂

      regards,
      Georgi

      View CommentView Comment

Leave a Reply

CommentLuv badge