VO services with JPEG2000 client-server visualisation:

AstronomyData Services at Pawsey Supercomputing Centre.

Slava Kitaeff & Dan Marrable

Western Australia - ASKAP/MWA/SKA

2 2013 Harley Wood Winter School,

Pawsey Supercomputing Centre

3

Pawseyisthegovernment-supportedhigh-performancecompu7ngna7onalfacility(Perth,WesternAustralia)thatsupportsresearchersinWesternAustraliaandacrossAustraliathroughprovidingtheinfrastructureforthecomputa7onalresearchworkflows.Thisincludes•  supercomputers•  cloudcompu7ng•  datastorage•  visualisa7on

HPC@Pawsey

4

•  48bladesx4nodesx2CPUs(IntelXeonE5-2690V3“Haswell”2.6GHz)x12-cores=35,712cores

•  1.1PetaFLOP•  Interconnect-CrayAries•  Localstorage–3PBCraySonexion1600

Lustreappliance

Magnus

#41inTOP500(November2014)

HPC@Pawsey

5

•  118computeblades,eachofwhichhasfournodes

•  Eachnodesupportstwo,10-coreIntelXeonE5-2960V2“IvyBridge”processorsopera_ngat3.00GHz

•  Totalof9,440cores•  ~200TeraFLOPSofcomputepower.•  Interconnect-CrayAries

Galaxy

Zeus&Zythos

•  39nodesinvariousconfigura_ons•  Zythosisthelargestnode:SGIUV2000

systemwith6TBsharedmemory,264IntelXeonprocessorcoresand4NVIDIAK20GPUs.

NeCTAR l  NeCTAR (National eResearch

Collaboration Tools and Research)

l  NeCTAR is an Australian Government project to build infrastructure specifically for the needs of Australian researchers

l  NecTAR is a $47 million dollar, Australian Government project conducted as part the Super Science initiative and financed by the Education Investment Fund

l  NeCTAR has built: l  New virtual laboratories l  A research cloud l  eResearch tools l  Hosting services

RDS/RDSI

Australian National Data Storage

InPerth~4Petabytesdiskstorage(GPFS),plus>35PerabyteDMFTapestorage

Astronomy Data Services (ADS) on RDS (ADS@RDS)

Project Objectives: •  To develop a framework for publishing

astronomy data on RDS

•  To identify the data discovery and data retrieval protocols

•  Build a working prototype

8

Astronomy Data Services at Pawsey

9

BIG motivation to look for alternatives in visualisation

10

SKA1datasizes/volumes

Howtoexplore/visualisethis?

Initial “wish list”

•  Multiple resolutions without penalties •  Lossles & lossy compression •  Alternatives to cutout •  Good support for metadata

11

•  Lossless and lossy compression in a single compression architecture

•  Multiple resolution/fidelity representation out of a single file without increased file size penalties.

•  Superior compression efficiency and performance •  Low computational requirements. •  Progressive transmission/recovery by fidelity or resolution. •  Spatial random access to image regions at varying degrees of

granularity. •  Adaptive transmission (transfer only what’s needed) •  Support of volumetric images. •  Extensive metadata support and handling. •  Interactivity in networked applications using JPIP protocol.

JPEG2000 is not “just another photographic image format”.

It’s not actually just a format.

•  Adaptive transmission (transfer only what’s needed)

•  Different parts of the same image can be stored using different quality (precincts not tiles).

•  Support of volumetric image cubes. •  Extensive metadata support and handling.

•  Interactivity in networked applications using JPIP protocol.

13

SkuareView

•  Skuareview encoders/decoders (FITS<->JPEG2000) https://github.com/ICRAR/SkuareView •  Skuareview JPIP server https://github.com/ICRAR/SkuareView-Server •  Using KDU library (needs a license, but it’s free

for research and academia)

14

JPIP-enabled Aladin

15

Skuareview JPIP image viewer

16

Conclusion

If you want to see how JPIP client-server visualisation works in live demo, see me during the morning tea break at ASVO desk.

17

18

-end-

19

-backupslides-

JPEG200 encoding

20

Lossless compression Featured Image

•  FITS – 16.97MB •  JPEG2000 – 1.68MB •  Ratio 1:10

Spectral line observations (RRL) with MWA

22

Lossless compression Sparsely populated image

Ratio 1:5

•  FITS - 6.9 MB; •  JPEG2000 - 2.3 MB •  Ratio 1:3

Lossy compression with controlled degradation and adaptive quality

- >40 images tested

- Most could be compressed lossely to least 1:20 ratio showing no visually noticeable degradation

Lossy(targetPSNR=44.5dB),ra7o1:20,Original

•  1:100s ratio can be achieved with adaptive quality

Image compression/interrogation

25

OriginalFITS~500MBRa7o1:14Encodedlosslessly~35MBCompressionRa7oTostartviewingfromservertheen7reimageatlowresolu7onviaJPIP~8.3MBStartviewing607mesfaster!

AladinwithJPEG2000&JPIP

Dimensions8192x8192Pixels~67.1millionResolu7on28.8arcsec

Controlledadap7velossyencodingactslikeanoisefilter.10xfasterthanDuchamp’sAtros,6%moreaccurate arXiv:1403.2801 arXiv:1401.7433Students:J.T.Maroreckii,A.Cannon,S.Peters