Introduction toData Warehousing

Athens IT Professionals Meetup2017-01-10

What is a “Data Warehouse”

Why do I need one?

Combines data from separate systems for a cross-system analysis

Business Intelligence Dashboards

Compare historic information over time

Can be used as an “archive” (although there are less costly solutions for this)

Makes data available to users who might not have access or be able to access it natively

“Analysts” can explore data instead of “programmers”

(SQL knowledge versus API/Programming knowledge)

Use Cases

Reporting

Analytics

Dashboards

Alerts

How much work is it?

Hopefully this talk is a good start

Expect to spend a month playing with things and learning before you can do anything useful.

Tools are probably going to cost several hundred dollars/month minimum

Confused?Lots of products do several pieces of the puzzle

Components of the Stack

Raw Data Sources

Data Extraction / Transformation / Load

Data Warehouse (Database)

Visualization / Reporting

ETL has changed over time as data warehouse software has gotten more powerful

Data Sources

Text, CSV Files

SQL Servers

Log files

Any Application that stores data

Via Export, API, Web scrape, etc

Getting Data to the Warehouse

Direct load from CSV

Segment.io

Flydata.com

Zappier

Stitchdata

Custom integrations

Google “import X to data warehouse”

Data Warehouses

AWS Redshift

Microsoft Azure

IBM

Astera

Terradata

Google Bigquery

Hadoop

Many others

Data Visualization and Reporting

SQL Tools:SQL Workbench

Data Grip

Most general-purpose SQL tools

Visualization Tools

Periscope

Looker

AWS Quicksite

Tableau

Dashboard Tools

Hundreds of options

Thank you

Questions?