monetdb/xquery technology preview 1
DESCRIPTION
Stefan Manegold Centrum voor Wiskunde en Informatica Amsterdam http://monetdb.cwi.nl/ - http://pathfinder-xquery.org/. MonetDB/XQuery Technology Preview 1. Stefan Manegold. MonetDB/XQuery. HollandOpen, Amsterdam 31.5.2005. European Pathfinder Team. CWI, Amsterdam (Netherlands) - PowerPoint PPT PresentationTRANSCRIPT
MonetDB/XQuery
Technology Preview 1
Stefan ManegoldCentrum voor Wiskunde en Informatica
Amsterdam
http://monetdb.cwi.nl/ - http://pathfinder-xquery.org/
European Pathfinder Team
• CWI, Amsterdam (Netherlands)– Peter Boncz, Stefan Manegold, Sjoerd Mullender
• University of Twente (Netherlands)– Maurice van Keulen, Jan Flokstra
• University of Konstanz (Germany)– Torsten Grust, Jens Teubner, Jan Rittinger
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
Results: Performance (1)
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
XMark benchmark, 110 MB: MonetDB/XQuery vs. X-Hive & Galax
Results: Performance (2)
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
XMark benchmark, 1.1 GB: MonetDB/XQuery vs. X-Hive
Story
• XQuery Example• Relational XQuery
– System Architecture– XML Encoding
• Science & Reseach• Scalability• Outlook
– Conclusions– Roadmaps– Release & References
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
• For each author, return number of books and receipts
for books published in the past 2 years, ordered by name
let $cat := fn:doc(“www.bn.com/catalog.xml”), (:Documents:) $sales := fn:doc(“www.publishersweekly.com/sales.xml”)
for $author in distinct-values($cat//author) (:Grouping:) let $books := $cat//book[@year >= 2003 and author = $author],(:Sel.:) $receipts := $sales/book[@isbn = $books/@isbn]/receipts (:Join:) order by $author (:Ordering:) return <sales> (:XML Construction:)
{ $author } <count> { fn:count($books) } </count> (:Aggregation:)
<total> { fn:sum($receipts) } </total> </sales>
XQuery Example
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
• For each author, return number of books and receipts
for books published in the past 2 years, ordered by name
let $cat := fn:doc(“www.bn.com/catalog.xml”), Documents $sales := fn:doc(“www.publishersweekly.com/sales.xml”)
for $author in distinct-values($cat//author) Grouping let $books := $cat//book[@year >= 2003 and author = $author],Sel. $receipts := $sales/book[@isbn = $books/@isbn]/receipts Join order by $author Ordering return <sales> XML Construction
{ $author } <count> { fn:count($books) } </count> Aggregation
<total> { fn:sum($receipts) } </total> </sales>
XQuery Example
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
XQuery Systems: 2 Approaches
• Existing “native” XML/XQuery systems are built from scratch– Galax, Saxon, …– X-Hive, Tamino, …– (Still have to) re-invent optimization technology
• Our approach:– Build XQuery system on top of an RDBMS– Leverage mature relational technology
to achieve efficient XQuery processing
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
xx
XML in an RDBMS: XPath Accelerator
Node-based relational encoding of XQuery's data model
f/following: SELECT * FROM pre_post WHERE pre > f.pre AND post > f.post f/descendant: SELECT * FROM pre_post WHERE pre > f.pre AND post < f.post f/preceeding: SELECT * FROM pre_post WHERE pre < f.pre AND post < f.post f/ancester: SELECT * FROM pre_post WHERE pre < f.pre AND post > f.post
Similar queries for all 13 XPath axes
<a> <b> <c/> </b> <d/> <e> <f> <g/> <h/> </f> <i> <j/> </i> </e> </a>
0<a> 1<b> 2<c/> </b> 3<d/> 4<e> 5<f> 6<g/> 7<h/> </f> 8<i> 9<j/> </i> </e> </a>
0<a> 1<b> 2<c/>0 </b>1 3<d/>2 4<e> 5<f> 6<g/>3 7<h/>4 </f>5 8<i> 9<j/>6 </i>7 </e>8 </a>9
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
Science & Research
• More research lead to more optimization– Join Recognition– Embedded XPath processing– Order Awareness
• Various scientific publications
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
Results: Scalability (3)
Unsurpassed scalability • Standard Opteron PC, 8GB RAM, 64-bit Linux• Can process 11GB documents!
Mostly linear scaling with document size
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
Conclusions
• Relational approach Works Is fast Is scalable
• Crucial Optimizations– Join recognition– Embedded XPath processing– Order awareness
• Research turned into open-source release
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
Roadmap
• 30.05.05: MonetDB/XQuery 4.8/0.8 “Mercurius”– Developers Release / Technology Preview 1
• 30.09.05: MonetDB/XQuery 4.10/0.10 “Venus”– Student Release / Technology Preview 2– XUpdate, More Optimization
• 30.12.05: MonetDB/XQuery 4.12/1.12 “Mars”– Final Release– Application Programming Interfaces– End-User Front-Ends
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
Open Source Release & References
• MonetDB + Pathfinder on SourceForge– Mozilla-like License
• MonetDB homepage– http://monetdb.cwi.nl/
• Pathfinder homepage– http://pathfinder-xquery.org/
• Developers website– http://sf.net/projects/monetdb/
You are welcome to join the MonetDB/XQuery team!
Stefan Manegold HollandOpen, Amsterdam 31.5.2005MonetDB/XQuery
XMark 11 GBQ Galax X-Hive MDB/XQ Galax X-Hive MDB/XQ X-Hive MDB/XQ MDB/XQ
1 0.06 0.37 0.05 0.72 1.29 0.41 9.9 1.2 132 0.03 0.45 0.07 0.31 1.75 0.30 33.0 2.4 253 0.14 0.65 0.28 1.76 5.66 1.51 25.1 12.5 1264 0.22 0.10 0.08 2.91 1.00 0.45 18.1 3.8 365 0.05 0.13 0.05 0.63 0.90 0.16 20.7 1.2 116 1.30 1.07 0.02 13.29 10.17 0.05 178.1 0.3 37 2.68 1.57 0.03 30.01 24.84 0.07 278.4 0.4 48 0.16 0.85 0.14 2.12 3.51 0.75 49.1 10.4 2089 113.23 32.25 0.20 DNF 12280.66 0.87 DNF 12.9 289
10 1.74 5.28 0.80 18.61 442.37 5.31 DNF 55.0 188211 2.62 98.91 0.18 DNF 19927.29 3.48 DNF 872.5 DNF12 1.44 23.39 0.14 DNF 5100.19 1.66 DNF 150.7 DNF13 0.03 0.10 0.07 0.66 1.03 0.22 12.9 1.3 1314 1.92 0.72 0.17 99.53 11.16 1.40 110.2 13.7 95915 0.02 0.03 0.09 0.20 0.49 0.28 10.6 1.7 1616 0.03 0.03 0.11 0.46 0.52 0.26 10.9 1.8 1817 0.06 0.09 0.07 0.82 0.85 0.30 11.8 2.8 2618 0.07 0.08 0.04 0.73 0.64 0.13 14.8 0.9 919 1.17 0.67 0.11 14.73 12.15 0.55 254.5 5.3 8820 0.28 0.11 0.24 2.98 1.40 0.62 24.6 4.9 50
11 MB 110 MB 1.1 GB
Stefan Manegold HollandOpen, Amsterdam 31-5-2005MonetDB/XQuery
Results: Performance (4)
XMark performance in seconds: MonetDB/XQuery vs. Galax & X-Hive