duncan irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as...
TRANSCRIPT
![Page 1: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/1.jpg)
1
Duncan Irving Teradata (Oil & Gas Consulting Practice EMEA/APAC)
![Page 2: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/2.jpg)
2
Data Science in E&P: “and”, not “or”
Dr Duncan Irving
![Page 3: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/3.jpg)
3
Agenda
• Data Management: New v. Old
• Where Data Science fits in
• Data Science in Upstream:
6 case studies
• What we have learnt from our recent data
science projects in terms of data management
Getting started with Data Science
![Page 4: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/4.jpg)
4
Big Data: disruption and innovation
How we understand and interact with each
other
How organisations understand and interact
with us
How we interact with technology and services
How we exploit knowledge…
at scale and pace
![Page 5: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/5.jpg)
5
Business
value
Technology
investment
Disruption is upon us – how do we exploit all the new data?
infinite sensors
unlimited connectivity
scalable storage
cheap processing
high-end visualisation
massive data volumes
scale and diversity of data types
data transfer across domains
operational time imperative
scientific
technical
business
more barrels
optimise operations
The intelligent field
![Page 6: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/6.jpg)
6
Data-driven analytical architectures
Analytics at scale
Designed to: • work at scale
• provision agility
• provide multi-directional flow
• use “exhaust” data
![Page 7: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/7.jpg)
7
Heavy industry analytical architectures
In E&P: • Raw data is too siloed • Sensors are for operational
control, not business value • Acquisition is deliberate and
often costly
No Analytics at scale
![Page 8: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/8.jpg)
8
Our workflows haven’t really changed much since the first data started coming back to shore with the oil…
But…
![Page 9: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/9.jpg)
9
It comes from
Fleets:
from lots and lots of
similar things
Systems: across the
same big “thing”
Collectors: “big
models” or
monitoring
It can contain…
Outliers: Which of my
things are behaving
differently?
Emergent behaviour:
Is my system
changing to a new
state?
Events: are there
hidden signals?
It has impact
“Fleet-wide” 24/7 for
holistic management
High-level KPIs at
business units and
facilities level
Performed at sub-
second level and data
kept for decades
“New data” comes in three flavours
![Page 10: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/10.jpg)
10
Our data managers are highly skilled “librarians”
• curate measurement data
• Ad hoc management of interp
• “work to spec”
…but want to deploy their domain expertise much more!
…but that looks a lot like the old data!
Yes, but the KPIs are different • Business related
• Business budgets, not IT (Low Capex / spend
from Opex)
• Show business value – early, and continuously
![Page 11: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/11.jpg)
11
Our different tribes speak different languages
Business
Technical Scientific
FORTRAN
SQL
Matlab
Unstructured Multi-
structured Structured
JAVa VB
excel
![Page 12: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/12.jpg)
12
How do other industries deal with this?
Business
Technical Scientific
Unstructured Multi-
structured Structured
SQL
Matlab excel
Python
R
JAVa VB
![Page 13: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/13.jpg)
13
Shine a light – see what’s in your data
![Page 14: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/14.jpg)
14
So how do I get started?
Source: xkcd.com
“With a small project, an open mind
and a big vision”
![Page 15: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/15.jpg)
15
We’ve heard about the data. So where’s the science?
Google flu trends
let us down
We’re still not really sure
what use Geostatistics is You’re going to
revolutionize E&P with a
scripting language, some
stats packages and some
random data?
…and where’s your
data governance?
![Page 16: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/16.jpg)
16
• 6 week MSc project at University of Manchester with New Zealand public data
• 3 weeks spent on data prep and engineering
Case Study #1 Basin-scale prospectivity analytics
Pragmatic data model from:
• LAS files
• Well headers
• Mud logs
• Well summary
• Completion Report
• A well constrained vocabulary
was fundamental to enabling
numerical analysis
Formation Name Member Name New Formation
Name
New Member Name
Moki Formation Moki Moki Formation Moki A Sandstone
Moki A Moki B Sandstone
Moki A Sandstone
Moki A SS
Moki B
Moki B Sandstone
Moki B Sandstone interval
Moki B Equivalent
Moki Equivalent
![Page 17: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/17.jpg)
17
Workflow to classify interbedded sandstone/mudstone and sandstone/siltstone facies:
base
dataset
analytical
dataset nPath
Rebuild and
pivot
Apply time
warping locate
Uw
i
De
pth
Cu
rve
na
me
cu
rve
va
l
ue
Uw
i
De
pth
GR
cu
rve
va
lu
e
pa
rtition
ID
create
multiple
windows
(paths)
along
datasets
Flip paths
of points
into
separate
key-value
pairs and
rebuild as
table
Create template
and map onto
partitions in pre-
built table then
calculate
goodness of fit
using DTW
Take top
centile of
matches and
validate
Uw
i
De
pth
GR
cu
rve
va
lu
e
pa
rtition
ID
…
• A much clearer, simpler reservoir model with 62 members in 17 formations
• An open-ended model to incorporate other data (e.g. production histories)
• Ask any question of the data with spatial, chronological and logical relationships – at scale
• Identified overlooked pay features (hot shales) and re-classified others (interbedded facies)
Case Study #1 Basin-scale prospectivity analytics
![Page 18: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/18.jpg)
18
Case Study #2 Drilling and Well analytics: Planning
• How will oil and gas operators ensure safe, accurate, efficient and economical D&W operations?
• CGG has access to geology, petrophysics, wells, and drilling data
• Teradata provides analytical platform to run complex data analyses
Source: CGG-Teradata
• Data analytics across Drilling & Wells
is not typically performed due to
silos and limitations of existing
solutions
• Modern D&W activities already
generate a large number of
parameters and will generate even
more in the near future
• We can identify trends, patterns, and risks in D&W domains and suggest optimal parameters for D&W
planning and operations
Formations
Rate of
Penetration
Weight on bit
Borehole
caliper
![Page 19: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/19.jpg)
19
• look for patterns to that will inform better operational decisions: increase drilling efficiency to avoid catastrophic bit damage
• An 8-week Data Science study across scientific and operational datasets identified $17M of savings in drilling practice
• surface and downhole
• metadata relating to well
and drill string
• bit damage severity and
profile
• well position and trajectory
• petrophysical information
• “It’s just hard formation – that’s the way it is”.
Unpredictable and repeated failures occur. Some single-trip sections achieved, but success/failure
criteria not understood
Case Study #3 Drilling and Wells analytics: Operations
![Page 20: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/20.jpg)
20
• Find combinations of a wide range of drilling parameters likely to avoid bit failure and model alarms to ensure efficient drilling
• Create rules for best practice during operations based on ever-growing knowledge base
• Consistently drill horizontal section in a single trip in hard formations
Case Study #3 Drilling and Wells analytics: Operations
![Page 21: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/21.jpg)
21
Case Study #4 4D seismic effects
• See PNEC2014 or EAGE papers!
Time shift Pressure
difference
Repeatability
(NRMS)
![Page 22: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/22.jpg)
22 © 2014 Teradata
Case Study #5 4D Seismic acquisition analytics
• Navigation, gun array, Met/Ocean and seismic trace data from 4D surveying
• How can data be integrated for analysis and possible operationalization?
• What value in the answers?
• One-off insight or should it be
operationalised?
• What is there of value in the multitude of file
formats?
• What are the analytical questions?
• What approaches?
• Lots of science v. lots of stats!
![Page 23: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/23.jpg)
23
• 2 hrs of data from seafloor array containing seismic event from casing collapse
• What are data management considerations for these analytical workloads
© 2014 Teradata
Internal 48 hr Hackathon to test Teradata capabilities with passive seismic reservoir monitoring data
Case Study #6 Passive Seismic Monitoring analytics
• Could borehole collapse event be
identified using simple statistical filters?
• What are the analytical components and
strategies?
• Can the event be located in time and
space?
• Can we define signatures for
operationalization?
• Can precursors be extracted to provision
early warning?
![Page 24: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/24.jpg)
24 © 2014 Teradata
What should a data science team look like?
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Data
Engineering
SQL
Domain
Expertise
Visualisation
Math & Stats
Platform
• No such thing as a perfect data
scientist
• For deployment you need
platform expertise
• You need outstanding data
management and data
engineering skills (and culture)
![Page 25: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/25.jpg)
25
• Loading into granular form
• Single view of data for whole team (cloud, or on premise)
• No up-front modelling
• Clear documentation and audit trail
• Keep loaders in a repository so they can be reused –not bound to application import functionality
• Data Lineage – reproducibility
• Data Quality – profiling what numerical values make sense?
Data Management Learnings
© 2014 Teradata
![Page 26: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/26.jpg)
26
• How should data be stored?
– Granular
– Profiles of activity – e.g. regular frequency profile instead of storing complete time series
– Profiles of valuable patterns
• Use a scalable platform (MPP)
• Use a language that is as universal as possible e.g. python
– Data Analysis – sciPy, NumPy require scientific and numerical prowess
– APIs into other domains e.g. HPC, filesystem, visualisation
Data Engineering Learnings
© 2014 Teradata
![Page 27: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/27.jpg)
27
• Keep data online and accessible – one-off studies may lead to a more operationalised event processing usage
• Profile incoming data regularly (e.g. production time series every few minutes across a reservoir) – keep profiles as descriptions of system states
• Store well-understood patterns of behaviour for repeatable mining (i.e. where have I seen this before?)
• Document activity continuously – people and skills are fluid through the life time of data. What has worked, what hasn’t worked, what approaches were considered but never picked up?
Data Mining Learnings
© 2014 Teradata
![Page 28: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/28.jpg)
28
Focus for
data integration
Focus for Data Science
Value creation from data
Gather data (old and new)
Discover Value (One-off)
Operationalize Value (Continuous)
Take Action (realize value)
![Page 29: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/29.jpg)
29
• Domain understanding is vital
• Have a well-scoped value proposition
• Work in agile mode with regular, well-managed sprints (no fixed agenda, no free-for-all)
• Have good visualisations
• How will you deploy and operationalise your insights?
Business Impact Learnings
![Page 30: Duncan Irving4afd6f873f5f9269a871... · minutes across a reservoir) – keep profiles as descriptions of system states •Store well-understood patterns of behaviour for repeatable](https://reader034.vdocuments.us/reader034/viewer/2022042314/5f02c7ed7e708231d405fa51/html5/thumbnails/30.jpg)
30
The magic ingredients
© 2014 Teradata
Scientific
understanding
Statistical
framework Analytical
engine
Domain
insight