REST in JasperReports Server - simple SVN versioning and continuous integration

Goals

Using REST to design a simple continuous integration with SVN, Python 2.7 and JasperReports Server gives some obvious benefits:

1. Versioning of reports with easy roll-back procedure
2. Continuous integration
3. Management of development and production environments

Disclaimer

Sample code is not production ready. It has numerous issues and should be used only as a starting point or as demonstration means.

Documentation

1. JasperReports Server Web Services Guide - see Docs.
2. Python 2.7 official documentation.
3. Requests library for Python
4. Apache Subversion (SVN) documentation 
5. pysvn documentation

Short Description

There are various hooks available in Subversion. We are mostly interested in post-commit hooks. Using knowledge of previous article about uploading reports (see uploading reports with python 2.7 and requests ), we can see that continuous integration in its simple form should be easy:

1. Commit to SVN
2. On the server side, run post-commit hook
3. Post-commit hook calls update on local version of SVN repo via pysvn
4. Executes PUT request on new reports
5. If reports have been updated, we need to run POST request to update them

Details

Using pysvn to create or update working copy:

#!/usr/bin/env python
import pysvn
import logging
import argparse

#see attachment and comments for details
from report_upload import report_upload

def svn_checkout(svn_url, svn_path):
  """
  Used to checkout or update local copy
  svn_url - repo url, could be svn://, file://, http://
  svn_path - local path of the working copy, i.e. /home/user/jasper_repo/
  """
  client = pysvn.Client()
  if not os.path.exists(svn_path):
    client.checkout(svn_url, svn_path)
  else:
    try:
      client.update(svn_path)
    except Exception as e:
      print e
      raise 

def report_log(rev, svn_path):
  """
  Using revision and local working copy to generate changed files
  rev is revision number in, for ex., 3
  svn_path  local path of the working copy, i.e. /home/user/jasper_repo/
  """
  revision = pysvn.Revision(pysvn.opt_revision_kind.number, rev)
  client = pysvn.Client()

  # getting svn log for revision number with changed paths
  log = client.log(svn_path, discover_changed_paths=True, revision_end=revision)
  
  # getting list of files for HEAD
  head_files = client.list(svn_path, recurse=True)
  files = []
  for i in head_files:
    # using pysvn.node_kind.file to separate actual files
    if i[0].kind == pysvn.node_kind.file:
      files.append(unicode(i[0].repos_path[1:]))

  diff_paths = []
  # separating changed paths in log  to a list
  for i in log:
    paths = i['changed_paths']
    for j in paths:
      if j['action'] in ['A', 'M']:
        diff_paths.append(unicode(j['path']))

  # returning only intersection with file list and changed files list
  diff_files = set(diff_paths).intersection(files)
  return list(diff_files)

if __name__ == '__main__':
  # logging in a very simple form
  logging.basicConfig(filename='error.log',
                      level=logging.WARNING,
                      format='%(asctime)s %(message)s')

  # initializing argparse. It is useful if we need to run the hook manually
  parser = argparse.ArgumentParser(description="SVN uploader on post-commit")
  parser.add_argument("-R","--rev", help="Revision", required=True)
  parser.add_argument("-f","--svn_url", help="SVN url", required=True)
  parser.add_argument("-p","--svn_path", help="local path to init")
  args = parser.parse_args()
  
  # checking out or updating local svn repository
  try:
    svn_checkout(args.svn_url, args.svn_path)
  except Exception as e:
    logging.error('Exception in svn_checkout(): %s' % e)

  # getting file list for report upload
  diff_files = report_log(args.rev, args.svn_path)
 
  for i in diff_files:
    report_upload(repo_path=i,
                  report_jrxml=args.svn_path + str(i),
                  report_datasource="/datasources/repositoryDS",
                  jasper_url='http://localhost:8080/jasperserver-pro',
                  user='jasperadmin',
                  password='jasperadmin')
  
  # done!

You can use report_upload from uploading reports with python 2.7 and requests.

The most notable changes are:

• jrxml file is loaded in a variable
• if PUT does not work we attempt POST. A correct way would be to test path for existence and select a correct method depending on the request.

Add a post-commit hook to svn hooks. For example:

svn_upload.py -R $2 -f file:///opt/svn-repo -p /opt/upload/jasper-working-copy

See subversion documentation and sample hook scripts (hooks/post-commit.tmpl) for details.

On windows you will have to use cmd or bat scripts. For example:

set TXT=%1
set REV=%2
echo "updating %TXT%" >> commit.log
D:\source\jasper\trunk\rests\update_svn.py -R %REV% -f file:///D:/source/test_repo -p d:/source/svn_test>>commit.log

Sample code

See attached zip archive for sample code.

Current issues

1. No production optimization
2. Does not handle PUT and POST differences correctly - jsut attempts one after another/D
3. Does not handle locks. It should be locking files if development is done by more than one person to avoid conflicts.
4. No actual error handling. If your postcommit failed - it has failed, and you will never know it.
5. No deleting of deleted files.
6. Works only with one datasource. No attachments or Input Controls.
7. Directories are also not created- you will have to follow existing repo path in your SVN repository or force other paths.
8. Logging should be more elaborate.

svn_update.zip