data organization
DESCRIPTION
Data Organization. Problem. Huge amounts of information How do I find Information that I know I want Information related to what I want How do I understand Particular pieces of information The whole collection of information. Limitations. Screen space Network bandwidth - PowerPoint PPT PresentationTRANSCRIPT
Data Organization
Problem
• Huge amounts of information
• How do I find– Information that I know I want– Information related to what I want
• How do I understand– Particular pieces of information– The whole collection of information
Limitations
• Screen space
• Network bandwidth– Bandwidth - how much information can be
transmitted per second
• Human attention
Kinds of things to organize
• Menu items– MS Word - about 150 menu items
• Text– Pages in a book - 500– Documents on the WWW - gazillions
• Images– All of the pictures created in a commercial
advertising company
Kinds of things to organize
• Sounds– Sound tracks to all TV and Radio news broadcasts
• Video– A complete collection of classic movies
• Structured information (records)– People– Cars– Students– Electronic appliance parts
A question of scale
• 10 things
• 100 things - menu
• 1,000 things - files on your computer
• 10,000 things - students at a university
• 1,000,000 things - books in a library
• gazillion things - WWW pages
Three ways to find things
• Lists – arrays
• Trees – organize in to categories
• Search – describe what you want and have the computer
find it
Finding things in lists
• How long will it take to find “Ron Dallin” in the Provo/Orem phone book?
• How long will it take to find “764-0588” in the Provo/Orem phone book?
Binary search - for “Goodrich”0 Anderson Lars 920-32311 Bilinski Phrank 123-31232 Clark Terry 764-23543 Garcia Jose 238-12344 Goodrich Carrie 824-92945 Gump George 232-03126 Jones Phred 723-92327 Randolph Jane 823-12428 Schwartz Gertrude 923-31349 Spimoni Maria 921-329410 Watt Harold 130-2313
Binary search - for “Goodrich”0 Anderson Lars 920-32311 Bilinski Phrank 123-31232 Clark Terry 764-23543 Garcia Jose 238-12344 Goodrich Carrie 824-92945 Gump George 232-03126 Jones Phred 723-92327 Randolph Jane 823-12428 Schwartz Gertrude 923-31349 Spimoni Maria 921-329410 Watt Harold 130-2313
Lower = 0Upper = 10
Guess = (0+10)/2 = 5
Binary search - for “Goodrich”0 Anderson Lars 920-32311 Bilinski Phrank 123-31232 Clark Terry 764-23543 Garcia Jose 238-12344 Goodrich Carrie 824-92945 Gump George 232-03126 Jones Phred 723-92327 Randolph Jane 823-12428 Schwartz Gertrude 923-31349 Spimoni Maria 921-329410 Watt Harold 130-2313
Lower = 0Upper = 5
Guess = (0+5)/2 = 2
Binary search - for “Goodrich”0 Anderson Lars 920-32311 Bilinski Phrank 123-31232 Clark Terry 764-23543 Garcia Jose 238-12344 Goodrich Carrie 824-92945 Gump George 232-03126 Jones Phred 723-92327 Randolph Jane 823-12428 Schwartz Gertrude 923-31349 Spimoni Maria 921-329410 Watt Harold 130-2313
Lower = 2Upper = 5
Guess = (2+5)/2 = 3
Binary search - for “Goodrich”0 Anderson Lars 920-32311 Bilinski Phrank 123-31232 Clark Terry 764-23543 Garcia Jose 238-12344 Goodrich Carrie 824-92945 Gump George 232-03126 Jones Phred 723-92327 Randolph Jane 823-12428 Schwartz Gertrude 923-31349 Spimoni Maria 921-329410 Watt Harold 130-2313
Lower = 3Upper = 5
Guess = (3+5)/2 = 4
Binary search
• If there are 64 things in a list, how many times can you divide that list in half?– 32, 16, 8, 4, 2, 1
• 6 times
Binary search
• If there are 1024 things in a list, how many times can you divide that list in half?– 512, 256, 128, 64, 32, 16, 8, 4, 2, 1
• 10 times
Binary search
• If the size of the list doubles, how many more steps are required in a binary search?
1
Binary search
• If there are N items in a list then binary search takes
• log2(N) steps
Binary search
• Estimating log2(N)– Count the number of digits and multiply by 2.5
• 1000– 4*2.5 = 10 steps
• 1,000,000– 7*2.5 = 17-18 steps
• 1,000,000,000– 10*2.5= 25 steps
Provo/Orem phone book
• How long to find “Ron Dallin”– 400,000 in Utah county
– Log2(400,000) approx 6*2.5 = 15 steps
How to find a phone number
0 Anderson Lars 920-32311 Bilinski Phrank 123-31232 Clark Terry 764-23543 Garcia Jose 238-12344 Goodrich Carrie 824-92945 Gump George 232-03126 Jones Phred 723-92327 Randolph Jane 823-12428 Schwartz Gertrude 923-31349 Spimoni Maria 921-329410 Watt Harold 130-2313
• 920-3231– 1 step
• 130-2313– 11 steps
• Average?– 5 steps
• Average N?– N/2
Provo/Orem phone book
• How many steps to find a phone number?– 400,000/2 = 200,000 average
• How can we improve this?
Sort the phone book by phone number
• What if I want to search on both name and number?
0 Bilinski Phrank 123-31231 Watt Harold 130-23132 Gump George 232-03123 Garcia Jose 238-12344 Jones Phred 723-92325 Clark Terry 764-23546 Randolph Jane 823-12427 Goodrich Carrie 824-92948 Anderson Lars 920-32319 Spimoni Maria 921-329410 Schwartz Gertrude 923-3134
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Anderson
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Anderson, Bilinski
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Anderson, Bilinski, Clark
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Anderson, Bilinski, Clark, Garcia
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
123-3123
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
123-3123, 130-2313
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
123-3123, 130-2313, 232-0312
Using an Index0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
123-3123, 130-2313, 232-0312, 238-1234
Search for Goodrich
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
Lower = 0Upper = 10
Guess = 5
lower
Search for Goodrich
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
Lower = 0Upper = 5
Guess = 2
above
Search for Goodrich
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
Lower = 2Upper = 5
Guess = 3
above
Search for Goodrich
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
Lower = 3Upper = 5
Guess = 4
above
Search for 823-1242
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
Lower = 0Upper = 10
Guess = 5
above
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Search for 823-1242
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
Lower = 5Upper = 10
Guess = 7
below
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Search for 823-1242
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
Lower = 5Upper = 7
Guess = 6
MATCH
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
Using an Index
0 Goodrich Carrie 824-92941 Gump George 232-03122 Schwartz Gertrude 923-31343 Watt Harold 130-23134 Randolph Jane 823-12425 Garcia Jose 238-12346 Anderson Lars 920-32317 Spimoni Maria 921-32948 Bilinski Phrank 123-31239 Jones Phred 723-923210 Clark Terry 764-2354
0 61 82 103 54 05 16 97 48 29 710 3
Last Name
0 81 32 13 54 95 106 47 08 69 710 2
Phone number
• What about first name or city?– another index
Data Organization
• What are we organizing for?• Scale
– 10 - 1,000 - 1,000,000 - 1,000,000,000
• Lists– Unsorted (N/2)– Sorted Log2(N)
• count the digits and multiply by 2.5
• To access in many ways– Use many indices into the same data