content-based routing. asynchronous middleware© gero mühl2 the need for a distributed...
TRANSCRIPT
Asynchronous Middleware © Gero Mühl 2
The Need for a Distributed Implementation> Disadvantages of a centralized notification
service> Single point of failure> Bottleneck wrt. bandwidth and computing power> Does not exploit locality although most interaction will
be local⇒ Neither reliable nor scalable to wide-area scenarios
> Service must somehow be distributed and replicated
⇒ Multiple cooperating service instances necessary!
Asynchronous Middleware © Gero Mühl 3
Broker Network> Service instances called brokers> Each broker manages a set of
exclusive local clients to which it forwards matching notifications it processes
> Notifications are stepwise forwarded through broker network
> Cyclic forwarding and delivery of duplicates must be avoided!> Easy in acyclic topologies but
these provide no fault tolerance> Complex in topologies with cycles
B1
Localclients
B2
B3
B4
Asynchronous Middleware © Gero Mühl 4
Flooding
> Each notification is delivered to all brokers> A broker forwards an incoming notification to all
neighbors if it was received from a local client and to all other neighbors if it was received from a neighbor
> Advantages> No routing tables necessary> Subscriptions become active immediately> Implementations can easily exploit IP-Multicast
> Disadvantage> Many notifications may be processed and forwarded
unnecessarily wasting computing power and network bandwidth
Asynchronous Middleware © Gero Mühl 5
Efficiency of Flooding
> Efficient if > most notifications need to be delivered to a majority
of brokers or> subscriptions are highly transient
> Not efficient if > filters are highly selective or> locality among subscriptions exists
Asynchronous Middleware © Gero Mühl 6
Content-Based Routing
> Each broker manages a filter-based routing tables
> A routing table TB consists
of a set of routing entries > A routing entry is a pair
(F, D) consisting of a filter F and a destination D which can be a local client or a neighbor broker⇒ Local and remote routing entries
B1
B2 B3
B4
nn
nsub(F)
sub(G)
(F,B2)(G,B3)
(G,B4)
Routingtables
n
n
n n ∈ N(F)n ∈ N(G)
Asynchronous Middleware © Gero Mühl 7
Content-Based Routing (contd.)
Broker B1
Forwarding Engine
Routing Table
(F, B3) (G, B4)(H, B3)
n
n ∈ N(F)n ∉ N(G)n ∉ N(H)
B3 B4
B2
1.n2.
Asynchronous Middleware © Gero Mühl 8
Notification Forwarding in Acyclic Topologies> Each broker forwards an incoming notification
to all neighbor brokers and to local clients for which there is a matching routing entry in the routing table.
> If the notification was received from a neighbor broker, it is not forwarded to that neighbor.
Asynchronous Middleware © Gero Mühl 9
Routing Algorithms
> Routing tables must be updated if subscriptions are issued or revoked> Update processes triggered> Many update processes might be in progress currently⇒ Routing algorithm
⇒ Subscriptions become active gradually⇒ Race conditions and latency⇒ Fully active after corresponding update process terminated
> Forwarding of unsubscriptions only done for efficiency> Delivery of non-matching notifications can be prevented
locally
Asynchronous Middleware © Gero Mühl 10
Routing Algorithms (contd.)
> A correct routing algorithm ensures that 1. Each update process terminates within finite time2. After an update process terminated, all published
notifications matching this subscription are delivered to the subscribing client
3. No duplicates and no non-matching notifications are delivered
Asynchronous Middleware © Gero Mühl 11
Routing Algorithms (contd.)
> Trade-Off> Flooding notifications vs.
filtering at intermediate brokers and updating of routing tables
> Optimal approach heavily depends on current setting!
⇒ In dynamic environments adaptive solutions necessary
Asynchronous Middleware © Gero Mühl 12
Simple Routing
> Only assumption: Filters can be uniquely identified> Each subscription stored in every routing table
> Size of routing tables grows linearly in the number of active subscriptions and the number of brokers
> Each routing table affected by a new/cancelled subscription> #brokers-1update messages necessary if update
information is not batched
⇒ Not scalable!
B4
B1
B2
B3
(F,B3)
(F,B1)
1. sub(F)
2. sub(G)
(F,B3)
G
(F,B1)(G,B2)
(G,B3)
(F,B3)(G,B3)
GG
Asynchronous Middleware © Gero Mühl 13
Simple Routing (contd.)
> New subscription flooded into the broker network> If a broker receives a subscription F from a neighbor B,
>it adds an entry (F, B) to its routing table and >forwards the subscription F to all its other neighbors
> Unsubscription flooded into the broker network> If a broker receives an unsubscription F from a neighbor B,
>it removes the entry (F, B) from its routing table and>forwards the unsubscription F to all its other neighbors
Asynchronous Middleware © Gero Mühl 14
Advanced Content-Based Routing
> Goals> Smaller routing tables> Reduced filter forwarding overhead
> Possible Solutions> Routing algorithms that exploit similarities and merge
subscriptions> Integration of advertisements (announcements of producers)> Trade accuracy vs. efficiency
> perfect routing> Notifications are forwarded only if a matching subscription
exists in respective subnet
> imperfect routing> Notifications may be forwarded without any matching
subscription
Asynchronous Middleware © Gero Mühl 15
Identity-Based Routing
> Uses identity tests for routing decisions
F≡G ⇔ N(F) = N(G)> Avoids
> Forwarding of identical filters> Routing entries with identical filters
for the same neighbor broker
F≡G
geometricalvisualization
n1
n2
B4
B1
B2
B3
(F,B3)
(F,B1)sub(F)
(F,B3)
(F,B3)
F
F
(F,B1)(F,B2)
sub(F)
Asynchronous Middleware © Gero Mühl 16
Covering-Based Routing
> Uses covering tests for routing decisionsF covers G ⇔ N(F) ⊇ N(G)
> Avoids> Forwarding of covered filters> Routing entries with covered filters
for the same neighbor broker
> Forwarding of subscriptions similar to identity-based routing
> Forwarding of unsubscriptions more complex due to uncovered subscriptions
F
G
Asynchronous Middleware © Gero Mühl 17
Covering-based Routing (contd.)
B4
B1
B2
B3
(F,B3)
(F,B1)sub(F)
(G,B3)
(F,B3)
G
G
(F,B1)(G,B2)
sub(G)
> Processing of a new subscription F from a neighbor B> Routing entries regarding B whose filter is covered
by F are dropped> F is forwarded only to those other neighbors to which
no covering subscription was forwarded before which is still active
F G
Asynchronous Middleware © Gero Mühl 18
Covering-Based Routing (contd.)
> Processing of an unsubscription F from a neighbor B> F is forwarded only to those other neighbors to which
no covering subscription was forwarded before which is still active
> Along with F, uncovered subscriptions must be forwarded
B4
B1
B2
B3
(F,B3)
(F,B1)(G,B2)(H,B2)
sub(F)
sub(G)sub(H)
F G
HF
(F,B3)
FG
HF(G,B3)(H,B3)
(G,B3)(H,B3)
Asynchronous Middleware © Gero Mühl 19
Merging-Based Routing
> Filters can be merged> perfectly
N(F) = N(G) ∪ N(H)> imperfectly N(F) =
N(G) ∪ N(H)
> Merging generates new covers
F G
H
GHF
B1
B2
B31. sub(G) B4
(G,B3)
(G,B1)H
2. sub(H)
(G,B1)(H,B2)
(G,B3)
(H,B3)
H F
(F,B3)
Asynchronous Middleware © Gero Mühl 20
Merging-Based Routing (contd.)
> Merging-based routing is perfect iff> only perfect mergers are generated and> mergers are forwarded and cancelled in a way such
that no unnecessary notifications are received later on
> … otherwise it is imperfect
Asynchronous Middleware © Gero Mühl 21
Perfect Merging-Based Routing
> Possible implementation on top of covering-based routing> Brokers merge suitable routing entries (F1, D),…, (Fn,
D) with the same destination D to a single entry (F,
D) such that F is a perfect merger of {F1, …, Fn}
> Merged entries are removed from routing table> Merger is added to routing table and forwarded like a
normal subscription received from D
Asynchronous Middleware © Gero Mühl 22
Hierarchical Routing Algorithms
> One broker is labeled as root broker R
> Any notification and filter is forwarded “upwards” to R
> Notifications are onlyforwarded “downwards” if a matching filter exists
> Leads to smaller routing tables but increases the load imposed on brokers on higher levels such as the root broker
R
B1 B2
B3 B4 B5 B6
1. sub(F)
(F,B3)
(F,B1)
n
n
n
n
2. pub(n)n ∈ N(F)
Asynchronous Middleware © Gero Mühl 23
2. pub(n)n ∈ N(F)
Hybrid Routing Algorithms
> Combine hierarchical with peer-to-peer routing
B0
B1 B2
B3 B4 B5 B6
B7
Peer-to-PeerRouting
HierarchicalRouting
1. sub(F)
(F,B1)(F,B1)
(F,B0)
n
nn
n n
(F,B3)
n
Asynchronous Middleware © Gero Mühl 24
Advertisements
> Producers specify the notifications they will publish
> Advertisements are filters that are forwarded like subscriptions by the help of a second routing table
> Subscriptions are only forwarded to those destinations for which there is an overlapping advertisement
> Test for overlapping: N(F) ∩ N(G) ≠ ∅
G
F
Asynchronous Middleware © Gero Mühl 25
Advertisements (contd.)
G
F F
(G,B4)(H,B5)
Advertisements Routing Table
B4
B1
B3
B5
FH
... ...
1. 2.
Subscription
Advertisements
Asynchronous Middleware © Gero Mühl 26
Advertisements (contd.)
> New advertisement> If B receives a new advertisement from H, B forwards
those overlapping subscriptions to H which can newly be served
> Cancelled advertisement> If B receives a cancelled advertisement from H, B
drops those overlapping subscriptions which can no longer be serviced
Asynchronous Middleware © Gero Mühl 27
Advertisements (contd.)
> Race condition> Published notifications may be dropped if they match only
a newly servable subscription that has not yet arrived!
> Potential solutions> Keep notifications that match only the new advertisement
until all newly servable subscriptions have arrived corrupts FIFO producer ordering
> Block a producer that publishes a notification that matches only the new advertisement until all newly servable subscriptions have arrived introduces some synchronicity
Asynchronous Middleware © Gero Mühl 28
Combining Advertisements with Hierarchical Routing> Advertisements are
distributed using peer-to-peer routing
> Subscriptions are distributed using hierarchical routingbut they are only forwarded “upwards” if an overlapping advertisement exists
> A flag indicates whether for an advertisements an overlapping subscription exists in an upper sub-tree
R
B1 B2
B3 B4 B5 B6
1. adv(F) 2. sub(G)
N(F) ∩ N(G) ≠ ∅
G
(B,B4)
false
Asynchronous Middleware © Gero Mühl 29
Topology Changes
> Easy approach preserving an acyclic topology> A new broker B is connected to exactly one broker B’
>B’ sends all interesting subscriptions to B> Simple Routing: all subscriptions> Identity-based Routing: identical subscriptions are
suppressed> Covering-based Routing: covered subscriptions are
suppressed
>If advertisements are used, they are exchanged instead of subscriptions
> Only leaf brokers can be removed
Asynchronous Middleware © Gero Mühl 30
Supporting Routing Optimizations
> Complexity of routing optimizations depends on the underlying data/filter model> In the most general case not computable> E.g., covering test among relational expressions is
NP-complete
> Scalability vs. Expressiveness
⇒ Expressiveness must be limited to a sensible level
Asynchronous Middleware © Gero Mühl 31
Identity among Filters
> Two filters F1 and F2 are identical, written F1 ≡
F2, iff they contain the same number of attribute
filters and if for each attribute filter in F1 there is
an attribute filter in F2 such that these attribute
filters are identical
> Example of identical filters
F1 = {x ≥ 2} ∧ {y > 5}
F2 = {x ≥ 2} ∧ {y > 5}
≡ ≡≡
Asynchronous Middleware © Gero Mühl 32
Overlapping among Filters
> Two filters F1 and F2 are disjoint iff for an attribute
filter in F1 there exists an attribute filter in F2 that
constrains the same attribute such that these attribute filters are disjoint
> Two filters are overlapping iff they are not disjoint
> Example for disjoint filters
F1 = {x ≥ 2} ∧ {y > 5}
F2 = {x < 1} ∧ {y = 7}
⊓ ⊓⊓
Asynchronous Middleware © Gero Mühl 33
Covering among Filters
> A filter F1 covers a filter F2, written F1 ⊒ F2, iff
for each attribute filter in F1 there is a covered
attribute filter in F2
> Example of covering filters
F1 = {x ≥ 2} ∧ {y > 5}
F2 = {x = 4} ∧ {y = 7} ∧ {z ∈ [3,5]}
⊒ ⊒⊒
Asynchronous Middleware © Gero Mühl 34
Filter Lattice
> Filters are arranged in a lattice according to covering relation
> Lattice is updated according to new and cancelled subscriptions
> Lattice is used by covering-based routing algorithm
F1:={x>5}
F2:={x>8}
F3:={x=9}
F4:={x=6} F5:={x>6 ∧ y>7}
F6:={x>7 ∧ y=9}
F7:={x=9 ∧ y=9}
Asynchronous Middleware © Gero Mühl 35
Merging of Filters
> Two filters F1 and F2 can be merged perfectly iff
they are identical in all but a single attribute filter
> Merging Example
F1 = {x ≥ 2} ∧ {y < 4}
F2 = {x ≥ 2} ∧ {y > 6}
F3 = {x ≥ 2} ∧ {y ∉ [4,6]} (Merger)
≡
Asynchronous Middleware © Gero Mühl 36
Example: Geographic Information Systems (GIS)> F = {(Type = TrafficInformation) ∧
(Location around(Frankfurt,50km))}> G = {(Type = TrafficJam) ∧
(Length ≥ 5km?) ∧ (Location around(Darmstadt,20km))}
⇒ F covers G
> H = {(Type = TrafficJam) ∧ (Location around(Frankfurt,40km))}
> I = {(Type = TrafficJam) ∧ (Location around(Wiesbaden,40km))}
⇒ H and I can be merged imperfectly
X
X
Frankfurt
Darmstadt
X XX
Frankfurt
Wiesbaden
Asynchronous Middleware © Gero Mühl 37
Bibliography
1. A. Carzaniga. Architectures for an Event Notification Service Scalable to Wide-area Networks. PhD thesis, Politecnico di Milano, Milano, Italy, Dec. 1998.
2. G. Mühl. Large-Scale Content-Based Publish/Subscribe Systems. PhD thesis, Darmstadt University of Technology, 2002. Chapter 3
3. A. Carzaniga, D. S. Rosenblum, and A. L. Wolf. Design and Evaluation of a Wide-Area Event Notification Service. ACM Transactions on Computer Systems, 19(3):332--383, 2001.