measuring the growth of work as input size n increases, how well does our automated system work?...

25
Measuring the Growth of Work As input size N increases, how well does our automated system work? Depends on what you want to do! Use algorithmic complexity theory: Use measure big o: O(N) which means worst case Important for Search engines Databases Social networks Crime/terrorism Sub-linear O(Log N) Linear O(N) Nearly linear O(N Log N) Quadratic O(N 2 ) Exponential O(2 N ) O(N!) O(N N ) Performance classes Polynomial Death to scaling

Upload: rosalyn-cornelia-gibson

Post on 16-Jan-2016

216 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Measuring the Growth of WorkAs input size N increases, how well does our automated system work?

– Depends on what you want to do!Use algorithmic complexity theory:

– Use measure big o: O(N) which means worst case

Important for– Search engines– Databases– Social networks– Crime/terrorism

Sub-linear O(Log N)

Linear O(N)

Nearly linear O(N Log N)

Quadratic O(N2)

Exponential O(2N)

O(N!)

O(NN)

Performance classesPolynomial

Death toscaling

Page 2: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

How Much InformationReaches Americans Consumers?

Roger Bohn - [email protected]

HMI? Project -- Global Information Industry Center (GIIC)

UC San Diego

November 3, 2009

Page 3: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

HMI? 2009 Scope

Consumers - Information used by Americans 1960 to 2008

Enterprise - Information processed by servers worldwide in 2008

Special studies, such as

Scientific data at MIT - giic.ucsd.edu

Health-care data

Enterprise storage growth

Page 4: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

HMI? Web site papers

Page 5: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Defining “information”

Dozens of definitions (more than one per person!)

Information to consumers ≃ Data that is directly for people

Information supplied versus consumed

Whether they pay attention or not (multiple streams count fully)

Multiple metrics for measuring an information object

Bytes: emphasizes moving visual imagery InfoC

Words: ignores pictures InfoW

Page 6: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

How many bytes in a 20 minute TV show?

Raw bytes = Screen size in pixels x bytes per pixel x frames per sec.

1920 x 1080 x 3 x 30 = 186 Megabytes per second = 1.5 Gbps

TV compressed down to 1 to 2% e.g. 12 Mbps

Infocompressed counts “bytes through the cable”

Examples: SDTV ~ 4 Mbps = 2 GB per hour

Book = 12 bytes per word x 4 words per second = 400 bps

Page 7: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Methodology

Page 8: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

How do we consume information?

Source: HMI? 2009

InfocInfow

Page 9: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Information consumed by “average American” per day

InfoC in GB/day InfoW in words/day

TV (incl. DVR, Internet, mobile) 12.0 44,342

Radio 0.1 8,315

Phone 0.01 5,269

Print 0.01 8,659

Computer 0.08 27,122

Computer games 18.5 2,459

Movies 3.3 198

Recorded music 0.08 1,112

Total 34.0 97,476

Page 10: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Information consumed by “average American” per day

InfoC in GB/day

TV (incl. DVR, Internet, mobile) 12.0

Radio 0.1

Phone 0.01

Print 0.01

Computer 0.08

Computer games 18.5

Movies 3.3

Recorded music 0.08

Total 34.0

Page 11: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Computer games ubiquitous

35 M consoles sold in 2008; 10M Nintendo DS + 10 M Wiil

70% of online adults play computer games

7 of top 10 iPhone apps are games (by revenue)!

“Casual gaming” has removed gender bias

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Page 12: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Sub-categories: computer games

# of Users (millions)

Average use/month Total InfoH % of Total InfoH % of Total InfoC

PC (high performance) 21 86.9 hours 1,4051,405 1.71% 38.28%

PC (standard) 124 18.1 194 2.12% 5.29%

Consoles 89 30.3 368 2.55% 10.02%

Handheld Devices 129 12.6 24 1.54% 0.64%

Total 1991 7.92% 54.23%

Full calculations go deeper: types of users, types of games, compression factors, others

Page 13: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

How push so many bytes?

Page 14: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Detail, fluid animation, lighting, good AI.This is from Xbox version - PC graphics better

Page 15: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Slices: How is info delivered?Airwaves includes TV broadcast, radio, mobile; Wires includes Cable TV, Internet; Physical = Print + Local computer

Page 16: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Other use Slices

“Interactive” information = telephone, computer games

now 55% of InfoC, 36% of InfoW

Traditional media = Cable TV, broadcast TV, radio, movies, print

Now now 36% of InfoC, 56% of InfoW

Define groupings of media types and information measures

Page 17: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

What about Information Growth?

As costs fall, use increases.

Hypothesis:

~ 35% per year growth in InfoC

Inco

rrect

!

Page 18: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

slow Growth

1960 to 1980 1980 to 2008

INFOHours 3.9% 2.6%

INFOWords 3.7% 3.0%

INFOC (bytes) 2.9% 5.4%5.4%

US Population 1.1% 1.0%

GDP $/capita 3.6% 2.9%

Source: HMI? 2009

Page 19: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Decomposing growth 1960-1980

InfoC = (# Users) x (Average InfoH per user) x (Average bpsC)x 3600/8

∆InfoC = ∆Population + ∆(InfoH per capita) + ∆bpsC

»Where ∆Y = (dY/dtime)/ Y = % rate of change of Y

5.4% = 1.0% + 1.6% + 2.8%

= Population growth + Hours/person growth + InfoC intensity growth

Why did average bits per second only grow at 2.8% per year?

Page 20: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Why such slow growth?

TV penetration ~97% in 1980

Hours per day: 7.4 in 1980 11.7 in 2008

Intensity: 2.9 Mbps in 1980

6.5 Mbps in 2008

TV in 1980: 4 Mbps SDTV

TV in 2008: 4 Mbps SDTV

12 Mbps HDTV

< 1 Mbps Internet

Page 21: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

what’s happened to print media?

Print media = Magazines + books + newspapers

Print = fastest way to deliver words

Pool: 240 words per minute

24% of Iwords in 1960;

9% in 2008

Web browsing another 28%

Tele

vision

19601980

2008

Prin

t

Computer e.g. Web browse

Telephone has grown in Sharew

Source: HMI? 2009

InfoW distribution across media

Page 22: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Roll your own projections

What happens if cable TV increases bandwidth per channel? TV usage migrates to Hulu?

What if DVR users store 1% of what they watch, at high fidelity, for 2 weeks?

Quality versus quantity trends

Page 23: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Rise of the Internet

Page 24: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Variable bit rates between and within categories

TV (cable, DVD, etc.) 3,000 (DVR) to 11,000 (DVD) to

Print .018 (color magazine) to .001 (straight text)

Radio .019 to .096

Phone .064 (landline) to .010 (cellular)

Computer non-game 100 (average Internet broadband)

Computer game 23,000 (console) to 2000

Page 25: Measuring the Growth of Work As input size N increases, how well does our automated system work? –Depends on what you want to do! Use algorithmic complexity

Two Categories of Algorithms

2 4 8 16 32 64 128 256 512 1024Size of Input (N)

1035

1030

1025

1020

1015

trillionbillionmillion100010010

N

N5

2NNN

Unreasonable

Don’t Care!

Reasonable

Ru

nti

me

sec

Lifetime of the universe 1010 years = 1017 sec