Docker & radio astronomy
1
Who Am I
Gijs Molenaar
Scientific Software Engineer in NL
PhD student in SA
MSc Artificial Intelligence
2
@gijzelaerr http://pythonic.nl
3
4
5
6
7
8
The problem
9
Scientist Q wants to crunchData X with
software Y onPlatform Z
10
Scientist Q
Smart person
Self educated on the field of software engineer
Gets the job done
Good at duct tape programming
11
Data X
Really big data
Datasets of peta, exa bytes
LOFAR - 50 GB/s
SKA - 10 times global Internet traffic
12
Software Y
13
Complex math heavy software
Written during PhD of scientist Q
Often bad coding practices
(Unit) tests?
Version control?
‘Runs on my computer’
Platform Z
14
Solution until now
15
Solution until now
16
Compile everything by hand
Still common practice (!)
17
18
Canonical launchpad
19
$ docker run skasa/papino ipython notebook
21
Rodrigues - telescope simulator
22
23
24
25
26
27
science vs scalable reliable web apps
Much less users
Much more data
High availability less important
Security less important
Performance very important
Durability very important
28
Lessons learned
Don’t build virtual machine images
use small minimal containers
use exec to examine problems inside container
29
Problems to solve / future work
GPU acceleration
Adaptation
‘Mixing’ containers (nixos?)
Container size (South Africa)
Container aware programming language / library?
30
