moving point type otb research institute for housing, urban and mobility studies 2008-11-06 dagstuhl...

2008-11-06 Dagstuhl

Moving Point Type

1

OTB Research Institute for Housing, Urban and Mobility Studies

A ‘movingpoint’ type for a DBMS

Wilko Quak - TUDelft

2008-11-06 2

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Overview

• Clarification• Design considerations for movingpoint type• Implementation in MonetDB

2008-11-06 3

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Moore’s Law makes some problem go away

50% p/year:

- cpu speed

- mem size

- mem bandwidth

- disk bandwidth1% p/year:

- mem latency

10% p/year:

- disk latency

#points in laser scan

#available GPS logs

#cadastral parcels

2008-11-06 4

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Examples ofmoving pointsdata

2008-11-06 5

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

What do I want with the data?

• Can I find all occurences of missing data because someone used subway?

• What is the optimal distance between busstops to get people to the trainstation as fast as possible?

• Is that man smiling?• Flocks and other patterns?• More……?

2008-11-06 6

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Requirements

• Should work with all kinds of data• Should be extensible (to moving region, or

dynamic integer??? (orthogonal?))• Queries should be understandable• Should work seamlessly together with

point/line/polygon + datetime• I want to store my original measurements in a

reproducably and compact way

2008-11-06 7

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Two possible mappings of movinpoint type to DBMS:

create table person( name string, location point dynamic continuous, salary integer dynamic discrete);

create table person( name string location movingpoint);

This is ortogonal but requires a

highly extensible DBMS to

implement

This is not that bad and is easier to do in existing

DBMS

2008-11-06 8

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Things that should be efficient

• Range queries in time• Range queries on location• Nearest neigbour search (Fréchet distance)• Joins

2008-11-06 9

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Debatable Issues

• What to do with accuracy. (Current implementations of point line an polygon don’t have it. Do we miss it?)

• More interpolation types than: linear or constant.

• Do we want a multi-scale type?

2008-11-06 Dagstuhl

Moving Point Type

10

OTB Research Institute for Housing, Urban and Mobility Studies

MonetDB

A novel spatial column-store DBMS

Martin Kersten - CWINiels Nes - CWIWilko Quak - TUDelftMaarten Vermeij - TUDelft

2008-11-06 11

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

MonetDB design considerations

• Multi-model database kernel support• Extensible data types, operators, accelerators• Database hot-set is memory resident• Simple data structures are better• Index management should be automatic• Do not replicate the operating system• Optimize when you know the situation• Cooperative transaction management

2008-11-06 12

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

MonetDB - Physical data organization• Binary Association Tables

ID Day Discount10 4/4/98 0.19511 9/4/98 0.06512 1/2/98 0.17513 7/2/98 0

OID ID100 10101 11102 12103 13104 14

OID Day100 4/4/98101 9/4/98102 1/2/98103 7/2/98104 1/2/99

OID Discount100 0.195101 0.065102 0.175103 0104 0.065

2008-11-06 13

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Monetkernels

MAPI protocol

JDBC

C-mapi lib

Perl

End-user application

ODBC PHP Python

SQL XQuery

MonetDB product family

2008-11-06 14

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

MonetDBheaplayout

2008-11-06 15

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Redundancy

ID X Y Z T

1 1215324 4341572 3.14159

2006-06-02:16.47

2 1215327 4341579 3.2 2006-06-02:16.57

3 1215400 4341590 10.0 2006-06-02:17.07

4 1215321 4341706 11.0 2006-06-02:16.17

This can be compressed away in heap structure

2008-11-06 16

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Implementation options for heap structure

1. Implement as BLOB with x1,y1,z1,t1,x2,y2,z2,t2,…- Similar to polyline implementations.- Index with rtree index on x,y,z,t- Is reasonably efficient for small objects.

2. Build a kinetic structure in the heap- Will be efficient for querying- Compression schema might be tricky

2008-11-06 17

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

Conclusions / Discussion

• DBMS support for multipoints will make querying collections of moving points easier and more efficient.

• MonetDB looks like a good option for implementation.

2008-11-06 18

OTB Research Institute for Housing, Urban and Mobility Studies

Moving Point Type

MonetDB extension mechanism for types