ska/kat spin presentation software engineering (!?) 2007-01-17 robert crida
Post on 27-Dec-2015
213 Views
Preview:
TRANSCRIPT
SKA/KAT SPIN Presentation
Software Engineering (!?)
2007-01-17
Robert Crida
Overview
Software Configuration Management (SCM) Automated Build System External Dependencies Design Study – Antenna Control Software Module
(ACSM)
SCM Challenge
KAT is expected to have a life span in excess of 20 years (project will outlive the people)
Maintenance is inevitable during that period Continuous development is also likely Importance over that period of being able to freeze
build environment What about hardware failure that far into the
future?
SCM
Use subversion – potentially multiple repositories (collaborations, different subsections)
Store multiple projects in a tree Standard svn hierarchy for each project
trunk (primary development) tags (releases) branches (qa, users, etc)
Try to include all dependencies as projects within the hierarchy
Subversion Layout Tools (used by the build process)
doxygen tags
doxygen-1.4.7 doxygen-1.5.1
etc (typically only store tags for each tool) Code (ultimately linked into products)
ThirdPartyLibraries (typically only tags) InternalProjectsHierarchy
Each layer can only depend on other projects in the same or lower layers, ideally wanted to only depend on projects in lower levels but deemed too difficult for users, instead have to work out build order within each layer.
Build system based on distutils Actually we use from recursivebuild import setup recursivebuild.setup delegates to setuptools.setup and
provides a convenient place to centrally add new commands (eg doc, pylint)
Use setuptools because of egg support Allows multiple builds of the same library to be present
simultaneously from pkg_resources import require require(“numpy==1.0”) import numpy
Most distutils projects easily converted by changing import of setup function
Python projects
Build System
Each directory of the hierarchy has a script to build its children script determines order passes arguments from parent arguments modeled after distutils.setup
Leaves are the projects themselves, must provide script understanding build targets Python projects use setup.py Other projects provide eg build.py to wrap make or
scons etc
Build system contd 1
All scripts must understand standard build targets install (more like build)
puts python eggs into suitable directory should compile and link other libs but not deploy
test (runs unit tests on the project) doc (runs doxygen on project) pylint (runs pylint on project – rename to lint!)
Build system contd 2
Build controller machine stores svn repository mirror plus overall build script
Build environment to be configured on a virtual machine, currently requires only gcc and python
Virtual machine is cloned prior to each build svn repository mirror is updated and copied to the
build vm Multiple build steps are executed remotely on the
vm by the controller
Build system contd 3
This allows multiple vm's with different compiler versions and even OS versions
Each can be built using (probably) the same control script
Testing
Integrate unit testing based on xUnit into the auto-build
Investigating Fitnesse based testing for automated ATP type testing
Looked at web testing technologies such as Twill and selenium to assist with automated web app testing
Dependencies
Python projects can depend on other python projects using the egg mechanism, hence can just find suitable egg in standard PYTHONPATH
Depend on other projects by specifying paths within the hierarchy multiple versions of a lib can coexist requires extensive include and lib paths use $PROJECT_ROOT environment variable instead of
relative paths due to svn moves (different nesting levels)
External Dependencies Try to minimize external dependencies
all need to go into SCM increases code-base (potential maintenance problem) traded off against leveraging existing code
AIPS++ many dependencies hard to build – difficult to configure tools constantly changing
Use each one's own build system must configure appropriate include and lib paths
External Python Dependencies
Eggs help a lot – in theory Despite distutils, many python packages have non-
standard builds which makes life difficult Haven't yet found a python project which defaults
to using setuptools
top related