dirx – query optimization dirx – query optimization frank huber vde/itg-fg 5.2.4...
TRANSCRIPT
![Page 1: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/1.jpg)
DirX – Query Optimization
Frank HuberVDE/ITG-FG 5.2.4 “Netzdatenbanken”
![Page 2: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/2.jpg)
2
Outline
DirX / LDAP / X.500Query processing in generalQuery processing and optimization in DirX– Rewriting– Optimization– Execution
ResultsRelation to network databases
![Page 3: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/3.jpg)
3
Outline
DirX / LDAP / X.500Query processing in generalQuery processing and optimization in DirX– Rewriting– Optimization– Execution
ResultsRelation to network databases
![Page 4: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/4.jpg)
4
DirX / LDAP / X.500
DirX – LDAPv3, DSMLv2 and X.500 directory serverBase for Identity Management Systems– Identities like persons, organizations,
applications, …Meta Directories
![Page 5: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/5.jpg)
5
LDAP / X.500
Hierarchical, semi-structured data model, organized as treeLDAP – query language:
– Relative to a given base– Boolean expressions– Operators =,>=,<=,~ and wildcards
![Page 6: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/6.jpg)
6
container=mymoviedb
objectclass : containerobjectclass : top
title: Idioterne
objectclass : movieobjectclass : top
year : 2000
...
...cn: Bodil Jørgensen
objectclass : actorobjectclass : person
objectclass : top
first_name : Bodillast_name : Jørgensen
year_born : 1961
cn: Troels Lyby
objectclass : actorobjectclass : person
objectclass : top
first_name : Troelslast_name: Lybyyear_born: 1966
cn: Lars von Trier
objectclass : directorobjectclass : person
objectclass : top
year_of_birth: 1956
Hierarchical, semi-structured data model, organized as treeLDAP – query language:
– Relative to a given base– Boolean expressions– Operators =,>=,<=,~ and wildcards
Example query:/container=mymoviedb,SUB,&((lastName=Lyby)
(year_born=1966))
LDAP / X.500
![Page 7: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/7.jpg)
7
Outline
DirX / LDAP / X.500Query processing in generalQuery processing and optimization in DirX– Rewriting– Optimization– Execution
ResultsRelation to network databases
![Page 8: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/8.jpg)
8
Query processing
![Page 9: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/9.jpg)
9
Outline
DirX / LDAP / X.500Query processing in generalQuery processing and optimization in DirX– Rewriting– Optimization– Execution
ResultsRelation to network databases
![Page 10: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/10.jpg)
10
Query processing in DirX
Q=(/dc=de, SUB, (a||b)&&(c), a)
Q=(/dc=de, SUB, (a&&c)||(b&&c), a)
SBS(dc=de)
∩
∪
AVIdx(a) AVIdx(c) AVIdx(b)
∩ ∩
Materialisation
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
![Page 11: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/11.jpg)
11
Rewriter
Q=(/dc=de, SUB, (a||b)&&(c), a)
Q=(/dc=de, SUB, (a&&c)||(b&&c), a)
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
![Page 12: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/12.jpg)
12
Rewriter
Filter transformation into disjunctive normal form (DNF)Checking satisfiability against schema definitionFinding tautologies, contradictions, redundancies, rangesGrouping disjunctions with same attribute type with IN-operators
![Page 13: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/13.jpg)
13
Optimizer
Q=(/dc=de, SUB, (a&&c)||(b&&c), a)
SBS(dc=de)
∩
∪
AVIdx(a) AVIdx(c) AVIdx(b)
∩ ∩
Materialisation
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
![Page 14: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/14.jpg)
14
Optimizer
Creates „optimal“ query execution plan (QEP)Cost model defines optimalityOutput: procedural QEP– Directed, acyclic graph
edges flow of datanodes operators
![Page 15: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/15.jpg)
15
Search component
SBS(dc=de)
∩
∪
AVIdx(a) AVIdx(c) AVIdx(b)
∩ ∩
Materialisation
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
Explores the search spaceUses methods of factorization and pruning
![Page 16: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/16.jpg)
16
SBS(dc=de)
∩
AVIdx(c)
Materialisation
Search component
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
∩
∪
AVIdx(a)
AVIdx(c)
AVIdx(b)
Materialisation
Explores the search spaceUses methods of factorization and pruning
![Page 17: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/17.jpg)
17
Search component
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
∩
∪
AVIdx(a)
AVIdx(c)
AVIdx(b)
Materialisation
Explores the search spaceUses methods of factorization and pruning
![Page 18: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/18.jpg)
18
Search component
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
SBS(dc=de)
∩
AVIdx(c)
Materialisation
Explores the search spaceUses methods of factorization and pruning
![Page 19: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/19.jpg)
19
Costing Component
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
SBS(dc=de) ∪
AVIdx(b)
∩ ∩
∩
AVIdx(a) AVIdx(c)
Materialisation
C=1250
C=1000
C=200
C=220C=30
C=250
C=10 C=20
![Page 20: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/20.jpg)
20
Costing component
– Cost model for each operator in the physical layer
– Statistical model for bit strings
![Page 21: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/21.jpg)
21
Statistics component
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
![Page 22: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/22.jpg)
22
Statistics component
– Annotates terms with selectivity and other statistical parameter
– Gives statistical information about the whole directory and sub trees
– Uses a specialized histogram type
![Page 23: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/23.jpg)
23
Query generation component
SBS(dc=de)
∩
∪
AVIdx(a) AVIdx(c) AVIdx(b)
∩ ∩
Materialisation
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
![Page 24: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/24.jpg)
24
Query execution engine
SBS(dc=de)
∩
∪
AVIdx(a) AVIdx(c) AVIdx(b)
∩ ∩
Materialisation
Rewriter
rewriting (declarative)
optimization (procedural)
Query (declarative)
Query Execution Plan (procedural)
QEP generation component
Optimizer
Executor
![Page 25: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/25.jpg)
25
Query execution engine
Implements different operators:– SBSIndex()– AVIndex(), AVIndexRange(), AVIndexIn()– Union(), Intersection(), Complement()
Evaluates the generated QEP
Open/Next/Close-Paradigm
Uses „Sideways Information Passing“
![Page 26: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/26.jpg)
26
Outline
DirX / LDAP / X.500Query processing in generalQuery processing and optimization in DirX– Rewriting– Optimization– Execution
ResultsRelation to network databases
![Page 27: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/27.jpg)
27
Results
Zeitersparnis (gesamter Prozess)
0,00
5,00
10,00
15,00
20,00
25,00
30,00
35,00
40,00
45,00
50,00
All Simple Range Not SubStr
Anfragetype
Proz
ent
![Page 28: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/28.jpg)
28
Outline
DirX / LDAP / X.500Query processing in generalQuery processing and optimization in DirX– Rewriting– Optimization– Execution
ResultsRelation to network databases
![Page 29: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/29.jpg)
29
Identity and Access Management @ Mobile Access - Vodafone
Source: www.siemens.com
![Page 30: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/30.jpg)
30
Source: www.siemens.com
Identity and Access Management @ Mobile Access - Vodafone
![Page 31: DirX – Query Optimization DirX – Query Optimization Frank Huber VDE/ITG-FG 5.2.4 “Netzdatenbanken” 2 Outline zDirX / LDAP / X.500 zQuery processing in general zQuery processing](https://reader035.vdocuments.us/reader035/viewer/2022081611/5f073d5d7e708231d41c010e/html5/thumbnails/31.jpg)
31
Questions?