sustainable scientific software development · continuous integration tools regularly run tests for...
TRANSCRIPT
![Page 1: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/1.jpg)
SustainableScientificSoftwareDevelopment
Europython2017
AliceHarpole
![Page 2: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/2.jpg)
MotivationImodel'explosionsinspace'or:theeffectsofincludinggeneralrelativityinmodelsofTypeIX-rayburstsinneutronstaroceans
![Page 3: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/3.jpg)
![Page 4: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/4.jpg)
MotivationFedupofreadingaboutexcitingcodes,onlytofindthey'renotopensourcetheyhavenexttonodocumentationquestionableapproachestotesting
Thisisnotgoodscience!
![Page 5: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/5.jpg)
OverviewWhatissoftwaresustainability(andwhyshouldIcare)?WhyscientificsoftwareisdifferentScientificsoftwaredevelopmentworkflowVersioncontrolTestingContinuousintegration&codecoverageDocumentationDistribution
Conclusions
![Page 6: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/6.jpg)
Whatissoftwaresustainability(andwhyshouldIcare)?Willmycodestillworkin5/10/20years'time?Canitbefound?Canitberun?
Ifnot,harmsfuturescientificprogress
![Page 7: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/7.jpg)
Whatmakesscientificsoftwaredifferent?
Builttoinvestigatecomplex,unknownphenomenaOftendevelopedoverlongperiodsoftimeCaninvolvelotsofcollaborationBuiltbyscientists,notsoftwareengineers
TurbulencemodelledbyDedalus
![Page 8: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/8.jpg)
TheScientificMethodInexperimentalscience,resultsarenottrustedunlessfollowscientificmethod:
testingofapparatusdocumentationofmethod
Demonstrateexperiment'sresultsareaccurate,reproducibleandreliable
![Page 9: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/9.jpg)
TheScientificMethodIncomputationalscience,wearedoingexperimentswiththecomputerasourapparatusWeshouldalsofollowscientificmethodandnottrustresultsfromcodeswithoutpropertestingordocumentation
![Page 12: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/12.jpg)
DevelopmentworkflowGoal:implementsustainablepracticesthroughoutdevelopmentFortunately,therearelotsoftoolsthatwillhelpusautomatethings!
![Page 13: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/13.jpg)
VersioncontrolKeepsalogofallchangestocodeComputationalscienceversionofalabbook
![Page 14: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/14.jpg)
AlexanderGrahamBell'slabbook-Wikimedia
![Page 15: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/15.jpg)
![Page 16: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/16.jpg)
VersioncontrolAidscollaboration-nooverwritingeachother'schangesCanhackwithoutfear-developonabranch,sonodangerofirreversiblybreakingeverything
![Page 17: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/17.jpg)
TestingShouldnottrustresultsunlessapparatus&method(i.e.thesoftware)thatproducedthemhasbeendemonstratedtoworkanylimitations(e.g.numericalerror,algorithmchoice)areunderstoodandquantified
![Page 18: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/18.jpg)
![Page 19: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/19.jpg)
![Page 20: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/20.jpg)
TestingScientificcodescanbehardtotestastheyareoftencomplexinvestigateunknowns
Doesnotmeanweshouldgiveup!
![Page 21: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/21.jpg)
Testing:Step1BreakitdownwithunittestsCan'ttrustthesumifthepartsdon'tworkMakestestingcomplexcodesmoremanageableMakesurethesecoverentireparameterspaceandcheckcodebreakswhenitshould
![Page 22: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/22.jpg)
import unittest
def squared(x): return x*x
class test_units(unittest.TestCase):
def test_squared(self): self.assertTrue(squared(-5) == 25) self.assertTrue(squared(1e5) == 1e10) self.assertRaises(TypeError, squared, "A string")
![Page 23: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/23.jpg)
Testing:Step2BuilditbackupwithintegrationtestsNeedtocheckallpartsworktogetherCangetmoredifficulthere
![Page 24: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/24.jpg)
Testing:Step3MonitordevelopmentwithregressiontestsCheckversionsagainsteachotherPerformanceshouldimprove(oratleastnotgetworse)Bonus!Helpsenforcebackwardscompatibilityforusers
![Page 25: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/25.jpg)
Science-specificissuesUnknownbehaviourUsecontrols-simpleinputdatawithknownsolution
Randomnessisolaterandompartstestaverages,checklimits,conservationofphysicalquantities
![Page 26: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/26.jpg)
data = rand(80,80) # declare some random data
def func(a): # function to apply to data return a**2 * numpy.sin(a)
output = func(data) # calculate & plot some function of random dataplt.imshow(output); plt.colorbar(); plt.show()
![Page 27: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/27.jpg)
Input is , so output must be 0 ≤ x ≤ 1
0 ≤ f (x) ≤ sin(1) ≃ 0.841
= f (x) dx ≃ 0.223f (x)⎯ ⎯⎯⎯⎯⎯⎯⎯⎯
∫1
0
def test_limits(a): if numpy.all(a >= 0.) and numpy.all(a <= 0.842): return True return False
def test_average(a): if numpy.isclose(numpy.average(a), 0.223, rtol=5.e-2): return True return False
if test_limits(output): print('Function output within correct limits') else: print('Function output is not within correct limits') if test_average(output): print('Function output has correct average') else: print('Function output does not have correct average')
Function output within correct limits Function output has correct average
![Page 28: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/28.jpg)
Science-specificissuesSimulations
convergencetests-doesaccuracyofsolutionimprovewithorderofalgorithmused?ifnot,algorithmmaynotbeimplementedcorrectly
Numericalerrorusenumpy.isclose&numpy.allclose
![Page 29: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/29.jpg)
# use trapezium rule to find integral of sin x between 0,1 hs = numpy.array([1. / (4. * 2.**n) for n in range(8)]) errors = numpy.zeros_like(hs)
for i, h in enumerate(hs): xs = numpy.arange(0., 1.+h, h) ys = numpy.sin(xs)
# use trapezium rule to approximate integral of sin(x) integral_approx = sum((xs[1:] - xs[:-1]) * 0.5 * (ys[1:] + ys[:-1])) errors[i] = -numpy.cos(1) + numpy.cos(0) - integral_approx
plt.loglog(hs, errors, 'x', label='Error') plt.plot(hs, 0.1*hs**2, label=r'$h^2$') plt.xlabel(r'$h$'); plt.ylabel('error')
![Page 30: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/30.jpg)
Continuousintegration&codecoverage
Continuousintegrationtoolsregularlyruntestsforyou&reportbackresults
&Findoutwhenbugsoccurmuchsooner-mucheasiertofix!Danger:outdatedtestsalmostasuselessasnotestsIftestsonlycover20%ofcode,whyshouldyoutrusttheother80%?
Codecoverage!
TravisCI CircleCI
Codecov
![Page 31: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/31.jpg)
![Page 32: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/32.jpg)
![Page 33: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/33.jpg)
DocumentationIdeal:someoneelseinyourfieldshouldbeabletosetupanduseyourcodewithoutextrahelpfromyouIncludecomprehensiveinstallationinstructionsDocumentthecodeitself(sensiblefunction&variablenames,comments)Userguidewithexamplestodemonstrateusagejupyternotebooksgreatforthis
Automatewith ,hostatSphinx ReadtheDocs
![Page 34: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/34.jpg)
![Page 35: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/35.jpg)
![Page 36: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/36.jpg)
DistributionMakeitfindableOpensource!(wherepossible)DOIe.g.from
Reproducibleresultsrequireareproducibleruntimeenvironmentpackagecodeine.g.dockercontainer,condaenvironment,PyPI
Installationshouldbeaspainlessaspossiblemakefiles,trytolimitrelianceonnon-opensourcelibraries/material
zenodo
![Page 37: Sustainable Scientific Software Development · Continuous integration tools regularly run tests for you & report back results & Find out when bugs occur much sooner - much easier](https://reader035.vdocuments.us/reader035/viewer/2022070107/6021c2ee27f60a0e3311eeb5/html5/thumbnails/37.jpg)
ConclusionsWeneedtofuture-proofoursoftwareApplythescientificmethodtosoftwaredevelopmentOnlytrustresultsfromcodesthatare
reproducible(opensource!)testeddocumented
CheckouttheSSIwebsite formorewww.software.ac.uk