How Will Applications Drive Future Data-Intensive Systems?

Data-Intensive Computing Workshop

Applications Break-Out Session

Some Driving Applications

• Google-style Search• Social Networking

(Facebook/Twitter)• Data warehouse mining• Biomedical• Sensor networks (e.g.,

video, radar)• Cosmology

• Astro• Climate• Fusion• Machine translation• National security• Disaster preparedness• Financial analytics• GIS

Many Domains benefit from Data-Intensive Computing

Common Application Structures

• Big Data

• Big Data

DerivedData

Query

background live

Anticipated vs. ad hoc analysis/queries

DerivedData

Query

Application Trends: Scale

E.g., Climate Change Studies need:• 5 orders of magnitude data scale• 5 orders of magnitude speed scale (including algorithmic

improvements)

But More than That…

Application Trends: Features

• SW as service, pervasive mobile clients

• P2P interaction• Built-in verifiability/

provenance of answers• Too much raw data; must

decide what (derived) data to retain

• Dealing with privacy controls, role-based authentication

• Multi-resolution, Multi-D visualization (multi-sensory presentation) at scale

• Queries expressed using multimedia

• Heterogeneity, Cross data sources

• Increased value of data=>increased demand for data security/integrity

Big Data Challenges:Around the Corner for All of Us

Reducing App Development Time

Key issues:

• Effective workflow tools: need for convergence to open, standard tools (Multi-user: Tasks are collaborative)

• Effective big data libraries & frameworks

• Avoid recoding when scale changes

• Use familiar APIs (C.S. stuff just works)

Some Lessons Learned• Curriculum mismatch between domain scientists and

computer science courses

• Hard to determine the resource needs of an app a priori

• Cross-disciplinary work is challenging

– More cross-disciplinary possibilities in sharing Big Data

• Typically not a big data cliff: can make do with less data, but improve with more data

– Although some apps need min data size to be useful

– Meet needs of those already feeling the pinch vs. Trying to leap ahead

• Economics: data is free, networking is free

– Payment may not be money: what demand of users