role of a dba · any data, streaming spark/cloud data mesh schema on write schema on read big data...
TRANSCRIPT
![Page 1: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/1.jpg)
Role of a DBA
IN3020 and IN4020
Trond BrennaPrincipal Solution EngineerOracle NorwayMay 04, 2020
![Page 2: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/2.jpg)
Safe harbor statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation.
2 Copyright © 2020, Oracle and/or its affiliates |
![Page 3: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/3.jpg)
Agenda
1 My Background
2 Traditional Role of a DBA
3 Changes in Data Management
4 Future DBA challenges and role
5 Summary
3 Copyright © 2020, Oracle and/or its affiliates |
![Page 4: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/4.jpg)
Trond Brenna
Principal Solution Engineer
Oracle Norway4th May 2020
4 Copyright © 2020, Oracle and/or its affiliates |
![Page 5: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/5.jpg)
My Background
Experience• Ergo Enet 96-01: Consultant
- Support, Training and Technical Advisor• Oracle Norway 01-05: Senior Instructor
- DBA admin, DBA Backup & Recovery and Database Tuning- SQL Basic
• Bluegarden 05-11: DBA/Team Manager DBA- Operations - Developing Database Platform- Lead DBA
• Oracle Norway 11-dd: Principal Solution Engineer- Database Solution Architect- IaaS/PaaS Architect- Strategic Customer work- Speaker on internal and external conferences
5 Copyright © 2020, Oracle and/or its affiliates |
![Page 6: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/6.jpg)
Traditional Role of a DBA
6 Copyright © 2020, Oracle and/or its affiliates |
![Page 7: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/7.jpg)
Traditional Role of a DBAWhere did they come from?
7 Copyright © 2020, Oracle and/or its affiliates |
Charateristics• IT education
• Basic database understanding• Data Modelling
• Job experience• The SysAdmin way• The Developer way
![Page 8: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/8.jpg)
Traditional Role of a DBAThe SysAdmins
8 Copyright © 2020, Oracle and/or its affiliates |
• Linux/Unix admins• DataCenter/Operating System consultants
• Storage• Server
• OS scripting
This was the guys (guessing about 99% guys) working on the floor
![Page 9: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/9.jpg)
Traditional Role of a DBAThe Developers
9 Copyright © 2020, Oracle and/or its affiliates |
• SQL Background• Database Procedure Language (i.e. PL/SQL)• Performance focus
• Indexing• Application Data Model responsibilty/focus• Worked between DBA and Developers
• App patching
Worked as DevOps before this term was «invented»
![Page 10: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/10.jpg)
Traditional Role of a DBAWhat did they do?
10 Copyright © 2020, Oracle and/or its affiliates |
• Creating databases • Scripts• Wizard
• Monitoring • SQL -> Tuning• Storage• Memory• Locks• Backup configure/status
• Updating Test/Dev/QA/Staging/Pre-prod databases• Upgrades and patches
• Reasonable level
Keeping the system up and running with an acceptable performance
![Page 11: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/11.jpg)
Traditional Role of a DBAWhat did they not have time to do?
11 Copyright © 2020, Oracle and/or its affiliates |
• Planning for the future• Get involved in Business related projects• Evaluate Database-platform security level
• Security patches• Patching• Compliance
• Focus on Quality of Service
Too little time to focus on getting value of the business data
![Page 12: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/12.jpg)
Traditional Role of a DBA
Command Line Interface (CLI)• Require deep knowledge of DB
system• Good for scripting• Some automation
DBA/Developer Tools• Combination of SQL and GUI• Access to objects• Drag-and-drop and
simplification of tasks
Graphical User Interface (GUI)• Monitoring/Alerts• Dashbords• Jobs/Scheduler• Task Wizards• Advisors
12 Copyright © 2020, Oracle and/or its affiliates |
Tools
![Page 13: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/13.jpg)
Changes in Data Management
13 Copyright © 2020, Oracle and/or its affiliates |
![Page 14: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/14.jpg)
Changes in Data ManagementTraditional Data Management
• Realational Databases (RDBMS)• Online Transaction Processing (OLTP)• Descision Support System/Datawarehouse
(DSS/DW)
Multimodel Databases• Spatial and Graph• JSON• XML• Data Mining (R)• Machine Learning
14 Copyright © 2020, Oracle and/or its affiliates |
![Page 15: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/15.jpg)
15
Generational Change of Data Platform Technologies
Relational
Structured Data
BIG DATA DATA LAKE
Multi-structured data
HadoopNoSQL
Any Data, Streaming
Spark/Cloud
DATA MESH
Schema on ReadSchema on Write Big Data 2.0 –in-memory streaming
User/Domain FocusDistributedDocker/KubernetesMicroServicesDevOps
DATSIS. discoverableaddressable trustworthy self-describing, inter-operablesecurePolyglot
Multi-model
![Page 16: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/16.jpg)
Future DBA challenges and role
16 Copyright © 2020, Oracle and/or its affiliates |
![Page 17: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/17.jpg)
Future DBA challenges and roleImpact of Cloud -> Operational
Infrastructure-as-Service• Infrastructure created• Do-it-Yourself install DB-
system• DBA customer
responsibility • Full access to OS
Platform-as-a-Service Automated• Infrastructure created
and DB system installed
• Customer give input to DB configuration
• Tooling and API available
• Customer have responsibility to most of tasks
• Often full access to OS
Platform-as-a-Service Managed• DB and OS Managed
by Cloud vendor• Customer responsible
to data quality• API available
Platform-as-a-Service Autonomous• DB and OS Managed
by Cloud vendor• Customer responsible
to data quality• API available• AI and ML used for
monitoring and performance
17 Copyright © 2020, Oracle and/or its affiliates |
Level of Customer MgmtMore Less
![Page 18: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/18.jpg)
Future DBA challenges and roleWill DBAs be redundant?
18 Copyright © 2020, Oracle and/or its affiliates |
• Maria Colgan• Master Product Manager• Oracle Corp
![Page 19: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/19.jpg)
InnovationMaintenance
Today More Time is Spent on Maintenance vs Innovation
![Page 20: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/20.jpg)
20
Today More Time is Spent on Maintenance vs Innovation
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
Maintenance
72% of IT Budget is spent on Generic Maintenance
Tasks vs Innovation- ComputerWorld
Cost and Complexity Reliability
91% Experience Unplanned Data Center Outages
- Healthcare IT News
Database downtime costs $7,900 / minute
- DB Maestro
¾ Cost of Database Managementspent on labor
- IDC
91%72% 75%
![Page 21: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/21.jpg)
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
What Autonomous Database Means for DBAs
21
Value Scale
Innovation
Maintenance
•Tasks Specific to the Business– Architecture, planning, data modeling– Data security and data lifecycle management– Application-related tuning– End-to-End service level management
•Tactical Operations– Configuration and tuning of systems, network, storage– Database provisioning, patching– Database backups, H/A, disaster recovery– Database optimization
![Page 22: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/22.jpg)
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
What Autonomous Database Means for DBAs
•Tasks Specific to the Business– Architecture, planning, data modeling– Data security and data lifecycle management– Application-related tuning– End-to-End service level management
•Tactical Operations– Configuration and tuning of systems, network, storage– Database provisioning, patching– Database backups, H/A, disaster recovery– Database optimization
22
Removes tactical drudgery, more time to innovate Value Scale
Innovation
Maintenance
![Page 23: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/23.jpg)
InnovationMaintenance
Autonomous Database Enables More Innovation
![Page 24: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/24.jpg)
Future DBA challenges and roleWhat does the experts say?
24 Copyright © 2020, Oracle and/or its affiliates |
• Three experienced DBA/Architects about the DBA future
![Page 25: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/25.jpg)
“….DBAs need to stop thinking like maintenance workers and engineers who keep things operational, and more like data scientists. Penny Avril, Oracle's VP of database project management even goes so far as to tell DBAs to drop the "B" and think of themselves more as "DAs," or data administrators.”
"A data administrator isn't just keeping data in a database, but understands the importance of that data to key business stakeholders and in driving the business forward.”
Brandon Vigliarolo quote Penny Avril VP Oracle PMTechRepublic.com 11th April 2019
25 Copyright © 2020, Oracle and/or its affiliates |
![Page 26: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/26.jpg)
“The first point I always like to make to IT professionals is that the most important thing you can do in your career is to not chase the new, shiny technology………..However, it is equally important to have a grounding in your main focus area before trying to expand into new technologies.”
“Having strong DBA skills like data security, data protection and performance tuning will make you marketable for the foreseeable future. Nearly all of these skills will translate to the cloud, even if you are using a Database as a Service offering like Azure SQL Database or Amazon RDS.”
Joey D’AntoniRedmond – The Independent Voice of the Microsoft IT Community – 30th May 2018
26 Copyright © 2020, Oracle and/or its affiliates |
![Page 27: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/27.jpg)
“ ……listed below are three of my beliefs in regards to the future of the DBA job:
1. The DBA job as we know it today, will change, it will evolve into something else.2. You better start learning new skills today, so you can evolve with the DBA job3. If you have not automated repetitive task until now, then now is the time to do it.”
Diana RobeteFounder of DBA Paradise – 23rd October 2019
27 Copyright © 2020, Oracle and/or its affiliates |
![Page 28: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/28.jpg)
Future DBA challenges and roleMore focus on business related data management
28 Copyright © 2020, Oracle and/or its affiliates |
• «Data is the new oil»• Data Quality• Analytics/Business Intelligence (BI)• Data Science, Big Data, Artifical Intelligence (AI)
![Page 29: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/29.jpg)
Future DBA challenges and roleNew tasks for DBA
29 Copyright © 2020, Oracle and/or its affiliates |
• Architect• New projects• Developing the data platform
• Closer to Business• Make sure updated data is available for analyst
• Analytics• Support• Involved
• Data platform advisor• Where
• Hybrid• Cloud• On premis
• Choose right data platform• RDBMS• NoSQL• File based (Hadoop)• Distributed
![Page 30: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/30.jpg)
Future DBA challenges and roleNew tasks for DBA
30 Copyright © 2020, Oracle and/or its affiliates |
• Developing Data Platform• Security• High Availability• Disaster Recovery• Change Management
• Growt• Consolidation
![Page 31: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/31.jpg)
Summary
31 Copyright © 2020, Oracle and/or its affiliates
![Page 32: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/32.jpg)
SummaryNew role of DBA
32 Copyright © 2020, Oracle and/or its affiliates |
• DBA role will change• Data management changes• Cloud impact
• DBA should • Become more Data Architets• Get closer to the Business
• Always useful knowlegde• Security• Automation
![Page 33: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/33.jpg)
SummaryReferences
33 Copyright © 2020, Oracle and/or its affiliates |
• https://www.techrepublic.com/article/will-automated-databases-kill-the-dba-position/• https://redmondmag.com/articles/2018/05/30/dba-in-the-cloud-era.aspx• http://dbaparadise.com/2019/10/my-three-beliefs-about-the-future-of-the-dba-job/• DataMesh: https://trustgrid.io/what-is-a-data-mesh/• Data Lake: https://en.wikipedia.org/wiki/Data_lake• Big Data: https://en.wikipedia.org/wiki/Big_data
![Page 35: Role of a DBA · Any Data, Streaming Spark/Cloud DATA MESH Schema on Write Schema on Read Big Data 2.0 – in-memory streaming User/Domain Focus Distributed Docker/Kubernetes MicroServices](https://reader034.vdocuments.us/reader034/viewer/2022042219/5ec5627b9c571c0d232230b8/html5/thumbnails/35.jpg)