![Page 1: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/1.jpg)
Python and pandas, oh my:a rich ecosystem of open source
tools for petrophysical and geological assessment
Dan HallauThe Discovery Group, Inc.Denver, CO
![Page 2: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/2.jpg)
Disclaimer / Notes
• I am not a computer scientist!
• Just an enthusiast who has learned the python programming language in his spare time and now uses it every day.
11/3/2016 www.discovery-group.com 2
![Page 3: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/3.jpg)
Outline
• What is python?• What can you do with python in
geoscience/petrophysics?
11/3/2016 3www.discovery-group.com
![Page 4: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/4.jpg)
Python
• Interpreted programming language
11/3/2016 www.discovery-group.com 4
![Page 5: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/5.jpg)
Python
• Interpreted programming language
11/3/2016 www.discovery-group.com 5
![Page 6: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/6.jpg)
Python
• Interpreted programming language
11/3/2016 www.discovery-group.com 6
![Page 7: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/7.jpg)
• “Batteries included”– Lots of
excellent tools in the standard library
11/3/2016 www.discovery-group.com 7
Python cont’d
![Page 8: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/8.jpg)
• “Batteries included”– Lots of
excellent tools in the standard library
11/3/2016 www.discovery-group.com 8
Python cont’d
![Page 9: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/9.jpg)
• “Batteries included”– Lots of
excellent tools in the standard library
11/3/2016 www.discovery-group.com 9
Python cont’d
![Page 10: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/10.jpg)
• “Batteries included”– Lots of
excellent tools in the standard library
11/3/2016 www.discovery-group.com 10
Python cont’d
![Page 11: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/11.jpg)
• Open source– You are free to
study the source code, modify it, distribute it
11/3/2016 www.discovery-group.com 11
Python cont’d
![Page 12: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/12.jpg)
• Cross-platform
11/3/2016 www.discovery-group.com 12
Python cont’d
![Page 13: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/13.jpg)
11/3/2016 www.discovery-group.com 13
Python cont’d
• Also an excellent ecosystem!• Dead simple to install external
libraries$ pip install pandas
$ python>>> import pandas>>>
![Page 14: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/14.jpg)
11/3/2016 www.discovery-group.com 14
Python cont’d
• Also an excellent ecosystem!• Dead simple to install external
libraries$ pip install pandas
$ python>>> import pandas>>>
https://xkcd.com/353/https://xkcd.com/353/https://xkcd.com/353/
![Page 15: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/15.jpg)
Example project and workflow
• Lance Formation– Stack of undifferentiated fluvial sand
bodies• Notoriously difficult to correlate
• Wanted to compare sandstone and reservoir quality off- vs. on-structure
• I won’t be showing results from the study so much as I’ll be showing the python tools we used
11/3/2016 www.discovery-group.com 15
![Page 16: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/16.jpg)
Tools/libraries used in this example workflow
11/3/2016 www.discovery-group.com 16
pandas.pydata.org
numpy.org
github.com/kinverarity1/lasio
jupyter.org
matplotlib.org
![Page 17: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/17.jpg)
Jupyter notebook
11/3/2016 www.discovery-group.com 17
![Page 18: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/18.jpg)
Jupyter notebook
11/3/2016 www.discovery-group.com 18
![Page 19: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/19.jpg)
Jupyter notebook
11/3/2016 www.discovery-group.com 19
![Page 20: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/20.jpg)
Workflow
• Pick tops, export LAS data• Read all LAS files, concatenate into a
single dataframe• Compute basic petrophysics• Create zones• Run statistics on zones, groupby
various selectors• Create visualizations11/3/2016 www.discovery-group.com 20
Petra
Python
![Page 21: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/21.jpg)
Loading LAS files
11/3/2016 www.discovery-group.com 21
![Page 22: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/22.jpg)
Create zones
www.discovery-group.com 22
![Page 23: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/23.jpg)
Design petrophysical functions
11/3/2016 www.discovery-group.com 23
![Page 24: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/24.jpg)
functions
11/3/2016 www.discovery-group.com 24
![Page 25: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/25.jpg)
describe
11/3/2016 www.discovery-group.com 25
![Page 26: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/26.jpg)
groupbys
11/3/2016 www.discovery-group.com 26
![Page 27: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/27.jpg)
groupbys cont’d
11/3/2016 www.discovery-group.com 27
![Page 28: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/28.jpg)
Plotting
11/3/2016 www.discovery-group.com 28
![Page 29: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/29.jpg)
More plots
11/3/2016 www.discovery-group.com 29
![Page 30: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/30.jpg)
More plots
11/3/2016 www.discovery-group.com 30
![Page 31: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/31.jpg)
Ternary plots
11/3/2016 www.discovery-group.com 31
github.com/marcharper/python-ternary
• You don’t need an expensive specialized software package to get excellent, publication-quality results
![Page 32: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/32.jpg)
Ternary plot example
11/3/2016 www.discovery-group.com 32
![Page 33: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/33.jpg)
Machine learning• Domain of computer science / statistics
where computers “learn” using large datasets
• e.g. exploratory data analysis through hierarchical clustering. Plot data with respect to various selectors, cluster the data based on similarities.
• Could then create a classifier. If log data is being used:
11/3/2016 www.discovery-group.com 33
![Page 34: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/34.jpg)
Machine learning
11/3/2016 www.discovery-group.com 34
scikit-learn.org/stable/
caffe.berkeleyvision.org/
tensorflow.org/versions/r0.10/api_docs/python/index.html
![Page 35: Python and pandas, oh my: a rich ecosystem of open source ... · Python and pandas, oh my: a rich ecosystem of open source tools for petrophysical and geological assessment Dan Hallau](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e2212740120d4654c62d15f/html5/thumbnails/35.jpg)
Other python uses
• Endless possibilities– Automation–Web-scraping– File system manipulations• Multi-rename files• Create archives
– Sending emails, text messages
11/3/2016 www.discovery-group.com 35