1

Research Computing @ UQResearch Computing @ UQ

5 May 20145 May 2014 Issue 3Issue 3

The Research Computing Centre supports collaboration to facilitate discoveriesin science and engineering, humanities, and social sciences, through advancedcomputation, data analysis and other digital research tools.

Message from the Director

Welcome to the May RCC newsletter. It’s hard tobelieve we are almost half way through 2014!

In this newsletter we report on a number of activitiesaround the University. By the time you read this wewill have completed deployment of the second stageof the QRIScloud infrastructure I mentioned in thelast newsletter, out at the Polaris Data Centre inSpringfield. This time, we will give you a fewdetails about this facility. RCC is also well advancedin the next stage of procurement for data storage,and this will boost the cloud storage from its currentPB to some 10s of PB.

Our first QURPA international intern, Brian Song,reports back from his trip to our partner, theUniversity of California, San Diego. Brian tells us alittle about his trip. We are happy to promote theWinter School in Mathematical and ComputationalBiology to be held at UQ in July, and give you somedetails about this exciting event.

We also discuss an enhancement to the UQ LibraryeSpace repository that allows it to store computercode as well as data. This advance will make itpossible to publish code, and if appropriate, link thisto specific publications. This is a great advance forresearchers involved in computational science.

Speaking of which, don’t forget the InternationalConference in Computational Science will be heldin Cairns in June. It represents a great opportunity tohear some excellent international speakers, as wellas meet up with colleagues.

Last newsletter I mentioned the Genomics VirtualLaboratory (GVL), a NeCTAR funded virtual lab.

2

In this edition we give you a little moreinformation. We also give you some detail of theAuScope Inversion Lab which is using the NationalComputing Infrastructure for Large-ScaleGeophysical Inversions.

Don’t forget that we are always happy to see you atour HPC training sessions to help you get the mostout of the HPC infrastructure, both at the Universityand nationally. Finally, we have the first of whatwill become a rolling set of introductions tomembers of the RCC team. David Green, DeputyDirector, kicks off with a candid presentation.

Yours Sincerely,

David Abramson Director

UQ Chooses PolarisUQ has chosen Polaris to host the new QRISCloudinfrastructure discussed in the last newsletter. ThePolaris Data Centre offers a very high level ofprofessional management. It is regarded as one of

Australia’s most modern, highly secure, purpose-built, Tier 3+ data centres. The facility is unique indelivering a 99.99% availability.

Housed in Springfield, Polaris provides a 14,000m2

purpose-built, Tier 3+, high security facilitydelivering nearly 7000 m2 of raised floor over threelevels. It uses biometric man-traps on main entriesto all raised floor areas, with full digital CCTVsurveillance and logging, and provides a scalable,redundant power and cooling capacity. Polaris hasbeen reviewed and independently audited in regardsto the security and availability requirements of amajor banking / financial institution, an AustralianState Government, and other multinationalorganisations.

Whilst the initial deployment of equipment will beQRIScloud, RCC expects to place furtherequipment, including the new FlashLite DataIntensive computer out there later this year.Collocating these high-end resources will providesignificant performance for research computing atUQ.

First QURPAStudentReports backBrian Song (Bachelorof Engineering(Software), and now anMPhil candidate andmember of theComplex and Intelligent Systems group supervisedby Professor Janet Wiles was recently deployed tothe University of California, San Diego (UCSD) forfour weeks sponsored by the QueenslandUndergraduate Research Projects Abroad (QURPA)

program.

One of the mainpurposes of thistrip was to attendthe TemporalDynamics ofLearningConference, all-hands meeting at

3

UCSD. Despite the official title, TDLC attracted avast array of topics to be presented including gazetracking, social interaction, robotics, neuralnetworks, dolphin vocalisation and studies inautistic children. Brian’s research, ComputationTime Scales in iRat-Rat Social Interactions, waspresented at the TDLC poster session. (The iRatwas originally built by UQ to be used in rat-robotexperimentsconducted atUCSD). TDLC wasonly 3 days,so for theremainder ofthe trip,Brian spentmost of histime assisting in iRat-Rat experiments in ProfessorAndrea Chiba’s Rat Lab (in UCSD’s CognitiveScience department). The overall purpose of therat-robot experiments was to see if they could elicitor verify altruistic motives made by (real) rats. Brian also assisted in a neural recording experimentin Professor Doug Nitz’s lab where they raced theiRat and a rat down a straight 2m track. Thepurpose of this experiment was to observe theneural activity of mirror place cells in the rat as itraces against the iRat.

Throughout the trip, Brian also met with keyresearchers at CatIT2 (Qualcomm Institute), theSalk Institute and Brain Corporation. CalIT2 is asignificant partner for the Research ComputingCentre.

What is MURPA/QURPA?

MURPA (Monash Undergraduate Research ProjectAbroad) was launched at Monash University in2008. QURPA is the local extension of thisprogram. The program supports a unique summerresearch placement for final year undergraduatestudents studying computer science, softwareengineering, or technically oriented IT units, in aleading research group overseas. It not onlyprovides a research experience at the undergraduatelevel, but does that in an international context.

Students attend advanced seminars, simultaneouslytransmitted to Monash and UQ in semester 2.Interns then undertake an 8-week summer semesterinternational research project at the prestigiousUniversity of California, San Diego (UCSD) andthe University of Warwick, UK. This year for thefirst time, students were placed at the Institute forInfocomm Research (I2R) in Singapore.

This annual program exposes final yearundergraduate students to an international researchexperience within a leading research laboratory.

Winter School attracts WhiteHouse Champion of ScienceKeynoteHarvard Professor John Quackenbush will deliver akeynote address on “Taming the Big Data Dragon”at the Winter School in Mathematical andComputational Biologyto be held at UQ inJuly. His address willcover the datachallenges facing thebiological andcomputationalbiological sciences.Professor Quackenbushwas recently honouredat a White Houseceremony as one of the Obama Administration'sChampions of Science for his contributions toOpen Science.

Professor Quackenbush’s address will be one ofseveral sessions on Big Data at the Winter School,organised by RCC Director Dr David Abramsonand Dr Nick Hamilton, a recent co-appointmentbetween the Institute for Molecular Bioscience andthe Queensland Cyber Infrastructure Foundation.

The Winter School, now in its tenth year, isdesigned to introduce mathematical andcomputational biology and bioinformatics toadvanced undergraduate and postgraduate students,postdoctoral researchers and others working in thefields of mathematics, statistics, computer science,information technology, complex systems analysis,

4

engineering.

The School has been noted for the "compellingorators" it attracts and this year is no different withsix international, 14 national and 15 Queenslandfield-leading researchers presenting. Highlightsinclude sessions on Next Generation Sequencingand Bioinformatics, Modelling from High-Throughput Data, Statistical Applications andMolecular Phylogenetics.

The Winter School will be held at UQ from 7 to 11July 2014. Early Bird registration is ridiculouslycheap at only $205 for students and includeslunches and morning teas across the week.

International Conference onComputational Science (ICCS)10-12 June 2014 - Cairns, Australia

"Big Data meets Computational Science"

The International Conference on ComputationalScience is an annual conference that bringstogether researchers from various applicationareas who are pioneering computationalmethods in sciences such as physics, chemistry,life sciences, and engineering, as well as in artsand humanitarian fields, along with researchersand scientists from mathematics and computerscience as basic computing disciplines. Thismixture of computational researchers is able todiscuss problems and solutions in their areas,identify new issues, and to shape futuredirections for research.

For more information see:http://iccs2014.ivec.org/

ICCS is well known for its excellent line up ofkeynote speakers. The keynotes for 2014 are:

Professor Vassil Alexandrov, ICREA ResearchProfessor in Computational Science, BarcelonaSupercomputing Centre, Spain

Professor Dr Luis Bettencourt, Santa Fe

Institute, New Mexico, USA

Professor Professor Peter T. Cummings,Department of Chemical and BiomolecularEngineering, Vanderbilt University, USA

Professor John Mattick, Garvan Institute ofMedical Research, Sydney, Australia

Professor Bob Pressey, Australian ResearchCouncil Centre of Excellence for Coral ReefStudies, James Cook University, Australia

Professor Mark Ragan, Institute for MolecularBioscience, The University of Queensland,Australia

Research code record type to befeatured in UQ eSpaceScientists who write research code and softwareengineers will soon be able to record their work inthe UQ eSpacerepository.

Until now, codehas not beenrecognised as adistinct researchoutput type in UQeSpace. In thepast, anyonewanting to describe research code had to make dowith record types such as Data Set or CreativeWork, neither of which include the fields needed todescribe software adequately.

Some researchers felt this was unfair, as writingand developing code is an important part of theirresearch. In many cases, research would be neitherrepeatable nor understandable without access to theaccompanying code base. In other cases, the codeis the research.

Having a record in the UQ eSpace repository helpsmake research code discoverable.

As one earth sciences researcher put it: ‘Theexistence of releases as individual entities isimportant in cases like ours where funding iscontingent on regular releases’.

5

It also helps ‘legitimise’ the code by providing itscontext. The same researcher went on to say: ‘Whenpeople are deciding whether to use software, they maywish to know how long the programs have been indevelopment, and whether [the code] is still in activedevelopment’.

UQ eSpace staff are now developing a metadatarecord for research code/software. The record typewas designed by library staff in conjunction with staffof the Queensland Cyber Infrastructure Foundationand the Research Computing Centre, after QCIF staffwere lobbied on the issue by a number of researchers.

The new record type will allow depositors to describeresearch code quite finely. Fields include title anddescription, version, the research project to which itbelongs (if any), the programming languages anddistribution file formats, and the required softwareand hardware platforms. Licensing and accessconditions can also be recorded. It will be possible tolink records to earlier or later versions of the samecode.

UQ eSpace staff are still deciding whether to host thecode itself. Records can be linked to code lodged inexternal repositories such as GitHub or SourceForge.Related materials such as documentation orREADME files can certainly be deposited, as multipledatastreams attached to the code record.

The UQ eSpace repository already provides apermanent identifier for records. Staff are currentlyinvestigating minting DOIs, in which case researchersmight soon be able to get a DOI for their code.

The Genomics Virtual LaboratoryThrough the Genomics Virtual Laboratory (GVL),researchers can instantiate, manage and tailor theirown GenomicsAnalysis platform onthe nationally fundedAustralian ResearchCloud (NeCTAR)using nationalgenomics collections(RDSI)

GVL provides Researchers and Developers with anumber of resources:

Galaxy: perform reproducible analyses via a friendlyuser interfaceCommand Line Notebook: perform reproducibleanalyses with full command line access and aWindows interfaceTutorials: learn bioinformatics analysis techniquesCloudMan: manage your cluster on the cloud,dynamically add and remove nodesand several platforms:

Personal: Launch your own instance with your defaultAustralian Research Cloud allocationServer: Launch your own analysis cluster, tailored toyour needsManaged: Use the GVL managed resources

GVL’s resource aimed at biologists, Galaxy, is anopen, web-based platform for accessible,reproducible, and transparent computationalbiomedical research.

Accessible: Userswithout programmingexperience can easilyspecify parameters andrun tools andworkflows.Reproducible: Galaxycaptures information sothat any user can repeatand understand acomplete computationalanalysis.Transparent: Users share and publish analyses via theweb and create Pages, interactive, web-baseddocuments that describe a complete analysis

6

For researchers with more IT expertise, GVL’sCommand Line Notebook is a command lineUbuntu server with:

iPython Notebook, a web-based interactivecomputational environment where you cancombine code execution, text, mathematics, plotsand rich media into a single document:Xfce Desktop Environment, a lightweight desktopenvironment for UNIX-like operating systems.RStudio, a powerful and productive user interfacefor R.GVL Tutorials are a set of self-paced instructionsallowing you to learn analysis techniques on ourGVL Galaxy Tutorial server (galaxy-tut.genome.edu.au). You can further exploreanalysis techniques by using the GVL protocoldocuments.

The GVL also hosts a mirror of the UCSC GenomeBrowser on the Australian Research Cloud and aresearch-scale Galaxy instance.

For more information about the GVL and itsfacilities, contact Ron Horst

HPC Training

The RCC hosts regular training sessions for its newusers with monthly introductory "hands on"sessions in a computer-equipped teaching space.Details of sessions are available on the RCCwebsite.

After attending one of these hands on sessions,users are invited to attend a follow up session inwhich more individualised assistance can beprovided. We can revisit how to use the RCCfacilities in greater detail or explore how to bestutilise the facilities to solve your research problem.Booking in for training is easy- just email us!

AuScope Inversion Lab is usingthe National ComputingInfrastructure for Large-ScaleGeophysical Inversions

Geophysical observations are widely used in

geological sciences, exploration and, more recently,environmental monitoring in areas such as coalseam gas operations. Through the process ofinversion, geophysical data collected above or nearthe surface are extended into three-dimensionalinformation about the distribution of physical rockproperties in the subsurface, which is shown inIllustration 1. Inversion is a computationallyintensive task, especially when combining data setswith different physics in joint inversions. As anexample of this, the inversion of a magneticanomaly data set on a provincial scale at aresolution of 250m will require a grid size of over300 million cells. As it is infeasible to complete aninversion of this size on a desktop computer,compute clusters need to be used. Thesesupercomputers provide a powerful computationalenvironment by linking weaker computationalnodes through a fast interconnect. In order to makeefficient use of these systems, scalable softwarewith suitable mathematical methods and datastructures need to be used.

Under the Australian Geophysical ObservationSystem (AGOS) and with additional fundingthrough the AuScope CRIS and NCRIS2initiatives, The University of Queensland hasdeveloped a new scalable software “Downunder”for the inversion of gravity and magnetic anomalydata on compute clusters. The numerical methodsuse state of the art optimization algorithms and thefinite element technology, and are implemented ina way which is optimal for running on computeclusters with multi-core processors. The softwarehas now been ported to the newest NationalComputing Infrastructure (NCI) supercomputer andhas been scaled up to 8192 CPU cores for a jointinversion of synthetic gravity and magnetic data.The lower portion of Illustration 1 is thesusceptibility distribution obtained from theinversion of magnetic anomaly data (top right) forthe Clarence-

6

7

Moreton Basin in South East Queensland andnorthern New South Wales. The magnetic data at aresolution of 160m have been obtained from theNational Geophysical Airborne Data Base via theNeCTAR funded Virtual Geophysics Laboratory(VGL, see http://vgl.auscope.org.au). VGL is acollaborative effort of Geoscience Australia,CSIRO and NCI. The inversion grid was composedof 140 million cells distributed over 512 cores (16cores on 32 nodes) and completed in under 80minutes on the NCI facility. In comparison, theinversion of the same data at an 80m resolutionused 697 million cells on 8192 cores (512 nodes)and completed in 70 minutes.

It is a key capability of the AGOS inversionsoftware to allow users with different level of skillsto apply and extend the software to their particularproblems. Using python as a programminglanguage, the program supports users of varyingskill levels for both programming and mathematics.The numerical package escript is the computationalback end for the inversion process. It provides ageneric computational environment for solving

partial differential equations which are at the coreof geophysical inversions. Its numerical solver canrun in parallel on supercomputers and also offersthe ability to work on unstructured 2D and 3Dmeshes, which enables the inversion of data withspatially variable resolution. The escript software ismaintained and distributed through AuScope CRISand NCRIS2 funding and can be downloaded. Formore information contact Lutz Gross, School ofEarth Sciences.

Meet RCC: David Green, DeputyDirectorDr David Green majored in physics and appliedmathematics at UQ,thankfully after thedemise of flared jeansand platform shoes.His first real sciencejob was inenvironmentalmodelling for the QldElectricityCommission.

David completed aPhD in the School of Physics at the University ofSydney on the properties of electrochromic thinfilm window coatings that control solar heat gain. Even if we could have solved all the materialsscience challenges, people still did not like thecolour! He worked as an academic in AppliedPhysics and Computer Systems departments atUTS for almost fifteen years. His commitment tofostering student learning was recognised invarious forums. He career-changed into researchcomputing support roles at Griffith University inthe mid-2000s. David joined The University ofQueensland as HPC Manager within ITS in 2007.

David has long been using computers to "figurestuff out" and enjoys working with researchers tohelp them to use computers to "figure stuff out". Hehas been Deputy Director of the RCC since itsestablishment in 2011.

7