pycon apac 2016 keynote
TRANSCRIPT
![Page 1: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/1.jpg)
SaturdayMorningKeynoteWesMcKinney@wesmckinn
PyConAPAC2016(Seoul)
![Page 2: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/2.jpg)
Me
DataPad
ApacheArrow
Featheribis
![Page 3: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/3.jpg)
Inprocess:PythonforDataAnalysis:2ndEdi:onComing2017(inEnglishJ)
![Page 4: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/4.jpg)
Q:Whatbringsyouhere?
![Page 5: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/5.jpg)
Oursharedvalues
![Page 6: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/6.jpg)
PrideinsoMwarecraMsmanship
![Page 7: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/7.jpg)
Mystory
• AccidentalsoMwaredeveloper• 2007:Myfirstjob(financialresearchanalyst)
• IstartedwriPngPythonlibrariestodomyownworkbeQer
• SoonIwashelpingmycolleaguesworkbeQer,too
![Page 8: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/8.jpg)
Tools
![Page 9: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/9.jpg)
Tools
![Page 10: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/10.jpg)
Empathythefeelingthatyouunderstandandshareanotherperson'sexperiencesandemoPons:theabilitytosharesomeoneelse'sfeelings
Source:Merriam-Webster'sLearner'sDicPonary
![Page 11: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/11.jpg)
Opensourceiswonderful…
![Page 12: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/12.jpg)
Opensourceiswonderful…butitcanalsobefrustraPng
![Page 13: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/13.jpg)
Sustainableopensource
• Howtokeepcontributorsfromdrowning/burningout?
• Howtofundthework?
• Howtoprotectandservethecommunity?
![Page 14: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/14.jpg)
TheGrind
![Page 15: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/15.jpg)
“Thegrindisanendlessstreamofbugreports,requests,demands,quesPons,andoccasionalinquisiPons.” DHH,CreatorofRubyonRails
![Page 16: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/16.jpg)
pandas,theopensourceproject
• PartsofcodedatebacktoApril2008• Over600uniquecontributorsonGitHub• AcPveprojectmaintainersrangefrom4-7people
• >6900ClosedIssues• >5100PullRequests
![Page 17: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/17.jpg)
pandasatendof2012
![Page 18: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/18.jpg)
April7,2014
![Page 19: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/19.jpg)
"Somemightarguethat[Heartbleed]istheworst
vulnerabilityfound(atleastintermsofitspotenPalimpact)
sincecommercialtrafficbegantoflowontheInternet."
JosephSteinberg,Forbescybersecuritycolumnist
![Page 20: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/20.jpg)
“Thereshouldbeatleast…[6]fullPmeOpenSSLteammembers,notjustone,abletoconcentrate…withouthavingtohustlecommercialwork.Ifyou’rea…inaposiPontodosomethingaboutit,giveitsomethought.Please.I’mgemngoldandwearyandI’dliketorePresomeday.”SteveMarquess,OpenSSLteam
![Page 21: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/21.jpg)
ByNadiaEghbal,supportedbytheFordFoundaPon
Formoreonthis
![Page 22: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/22.jpg)
“TheCathedralandtheBazaar”
![Page 23: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/23.jpg)
Python’snormalizaPoninindustry
• Pythonhasbecomealeadinglanguageinsteadofsomething“experimental”or“risky”
• ManybusinessesfoundedonthegrowthofthePythonuserbase
• SeePaulGraham’s2004essay“ThePythonParadox”—howthingshavechanged!
![Page 24: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/24.jpg)
Governance“theprocessesofinteracPonanddecision-makingamongtheactorsinvolvedinacollecPveproblem…”
M.HuMy(viaWikipedia)
![Page 25: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/25.jpg)
OpennessandTransparency
![Page 26: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/26.jpg)
Consensus
![Page 27: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/27.jpg)
Someexamplegovernancedocuments
• NumPy(seethedocs)
• IPython/Jupytergovernance– github.com/jupyter/governance
• pandas– github.com/pydata/pandas-governance– ModeledaMerJupytergovernance
![Page 28: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/28.jpg)
hQp://numfocus.org
hQp://apache.org
![Page 29: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/29.jpg)
![Page 30: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/30.jpg)
conda-forge
• Community-curatedcondapackagechannel(hostedonanaconda.org)
• Reproduciblebuildinfrastructure(Docker+CircleCI+TravisCI+Appveyor)
• AutomatedGitHubhelpertools
conda config --add channels conda-forge
![Page 31: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/31.jpg)
Whatisnextforpandas?
• pandas1.0– Astable,maintenance-onlyrelease
• Beginning“pandas2.0”– PlanningsignificantrefactoringontheinternalsofSeries,DataFrame
![Page 32: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/32.jpg)
Whypandas2.0?
• Somechangesdifficult/impossibletodoinanincrementalway
• pandas’srelaPonshipwiththeecosystemhasevolvedoverthelast5years
• Makepandas
– Fasteranduselessmemory– Fixlong-standinglimitaPons/inconsistencies– Easierinteroperability/extensibility
![Page 33: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/33.jpg)
ApacheArrow
hQp://arrow.apache.org
![Page 34: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/34.jpg)
HighPerformanceSharing&InterchangeToday With Arrow
• Each system has its own internal memory format
• 70-80% CPU wasted on serialization and deserialization
• Similar functionality implemented in multiple projects
• All systems utilize the same memory format
• No overhead for cross-system communication
• Projects can share functionality (eg, Parquet-to-Arrow reader)
![Page 35: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/35.jpg)
FeatherFileFormatforPythonandR
• Problem:fast,language-agnosPcbinarydataframefileformat
• ByWesMcKinney(Python)andHadleyWickham(R)
• ReadspeedsclosetodiskIOperformance
• LeveragesApacheArrow
![Page 36: PyCon APAC 2016 Keynote](https://reader034.vdocuments.us/reader034/viewer/2022050613/587add141a28ab542b8b59f9/html5/thumbnails/36.jpg)
Thankyou
@wesmckinnhQp://wesmckinney.com
pandassprintonMonday!