psirp architectural components part 1
DESCRIPTION
PSIRP Architectural Components Part 1. Walter Wong NomadicLab & HIIT 08.02.2010. Outline. Identifiers Algorithmic IDs Node Internal Architecture Helper Functions Rendezvous System. Background – IP-based Identifier. Network Layer Identifier IP address Topological identifier - PowerPoint PPT PresentationTRANSCRIPT
PSIRP Architectural PSIRP Architectural Components Part 1Components Part 1
Walter WongNomadicLab & HIIT
08.02.2010
OutlineOutlineIdentifiersAlgorithmic IDsNode Internal ArchitectureHelper FunctionsRendezvous System
Background – IP-based Background – IP-based IdentifierIdentifierNetwork Layer Identifier
◦IP address◦Topological identifier◦Refers to a specific location in the
networkTransport Layer Identifier
◦Also IP address (IPsrc/dst, portsrc/dst, protocol)
◦Identifies end-hosts in the Internet
IP Semantic Overload IP Semantic Overload ProblemProblem
Application
Transport
Network
Link
Physical
IP
Socket
200.201.11/24
174.180.23/24
128.17.11.22
IP-based Identification – IP-based Identification – ProblemsProblemsEntangle host location with
identification◦Host-centric approach
Content can’t be addressed alone◦Sub-part of a URL◦Ex.
www.unicamp.br/main/courses/index.html◦ Identified as part of connection parameters◦Ex. TCP sequence number
Limited naming system◦Hinders new features, ex., mobility,
multicast◦Users can’t express their interest in data
regardless of location
PSIRP Identifiers – PSIRP Identifiers – MotivationMotivationExpress user interests
◦WWW users are interested in documents
regardless of their location
◦Currently users express an interest (what) and it is translated to a place (where)
Publish/subscribe◦Users express their interest on data◦Data is asynchronously delivered to
users
PSIRP IdentifiersPSIRP IdentifiersApplication Level Identifier (AId)Rendezvous Identifier (RId)Scope Identifier (SId)Forwarding Identifier (FId)Algorithmic Identifier (AlgId)
Application Level Identifier Application Level Identifier (AId)(AId)Application level identificationBased application requirements
◦Easiness to route, easiness to resolve, randomness, etc
Can be any namespace◦Flat labels◦Structured names (FQDN)◦Global Unique Identifier (GUID)◦Electronic Numbering (ENUM)
Rendezvous Identifier Rendezvous Identifier (RId)(RId)PSIRP network level identifierUniquely identifies a piece of
information256-bit identifier (SHA-256 hash
over the data)Goal: identify the interest
between publishers and subscribers in the rendezvous system
Rendezvous Identifier Rendezvous Identifier (Rid)(Rid)
Publish RIdnut
Subscribe RIdnut
Rendezvous Point
Interest!Doesn’t subscribe RIdnut
Deliver data
Bingo
Scrat
Nut
Scope Identifier (Sid)Scope Identifier (Sid)Also PSIRP network identifier with
256 bits lengthGoal: aggregate a set of Rids into
one group (scope)Represents a collection of whatever
information that Rids representsScope = implicit context of a set of
Rids◦Photo album has a Sid and each photo
has a RidAlso
◦Access control◦Metadata control operations
Scope Identifier (Sid)Scope Identifier (Sid)
RidC
RidB
RidA
Sidfamily_album Sidpublic_album
Forwarding IDForwarding ID256-bit long network identifierGoal: identify path segments in
the forwarding pathDifference:
◦Sid/Rid used in the rendezvous system (slow path)
◦Fid used in the forwarding fabric (fast path)
Can be aggregated in Bloom filters
Forwarding IDForwarding ID
Rendezvous 2
FId21 FId22
FIdA
Rendezvous 1
FId11
FId12
FId13
FId11FId21
FId12
Bloom-filter
Bob Clark
David
FId31
FId22
Forwarding IDForwarding IDSource controlled packet soft state
◦Topology manager creates the Bloom filter
◦+ stateless◦- requires constant updates from the
rendezvous to identify new subscribers◦- recycle Fids after un-subscription
In-network state◦Fid switching state in the forwarding
nodes◦+ Identifies just initial Fid◦- state in the switches
Algorithmic Identifiers Algorithmic Identifiers (AIgIds)(AIgIds)Special class of identifiersInformation collection concept
◦group semantically similar information items
Goal: automatic generation of IDs for different pieces of information
Allow elaborate tests with the identifier
Algorithmic IdentifiersAlgorithmic IdentifiersAllows relationship tests on each
ID:◦Ordering: precedence (does an
identifier precede another one?)◦Composition: does an ID belong to
another ID (e.g. a chunk ID belongs to a file ID)
◦Completeness: do we have all the identifiers composing another information object?
Algorithmic IDsAlgorithmic IDsApplications
◦Automatic deriving identifiers for information fragments
◦Calculating the next identifier for a sequence
◦Deducing the identifier
Subscription ManagementSubscription Management
Subscribecomics
Comics part 1
Comics Part 2
Comics Part 3
Comics Part 6
Comics Part 4
Comics Part 5
Billy
Forwarding State Forwarding State AggregationAggregation
FId11
FId12
FId13
FId21
FId31
FId22
FId32
FIdA
FIdB
FIdC
FId11
FIdA
FId13
FId31 FId32
FIdC
Bloom-filter
RVS
RVS
RVS
Bob Clark
David
Re-encoding the same Re-encoding the same publicationpublication
Publisher
ID1 ID2
IDA IDB IDC
AlgIDhigh_resolution
AlgIDlow_resolution
Publication
Return Path Return Path (legacy client-server model)(legacy client-server model)
Bob Clark
Publish(algIdA)
Subscribe(hash(algIdA))
Subscribe(algIdA)
Publish(hash(algIdA))
RVS
Hey Clark, let’s have a beer after work!Okee dokee, Bob!
Sequence NumberingSequence Numbering
RVS
Publish(algId) Subscribe(algId)
ID2 = hash(ID1)ID3 = hash(ID2)
ID1 = hash(algId)
Alice Beth
Algorithmic IDs – Other Algorithmic IDs – Other scenariosscenariosError control
◦generate error messages identified with IDs derived from the error ID
Reliability◦different algIds can be generated to
receive error messagesAnnouncements
◦prior to a data publication, publishers announce in the algId channel, informing possible subscribers that data will be published in the related algId
Algorithmic ID – General Algorithmic ID – General designdesignDirect Acyclic GraphIdentifiers are related to each other by
functionsSorting algorithms
◦Ordering the identifiersTrees are generated by consecutive
application of one-way functionsSequences can be expressed as the
recursive application of a singular function
Reverse function to walk in both directions
Algorithmic IDs – ExampleAlgorithmic IDs – Example
B = f(A,1)
C = f(A,2)
D = f(C,1)
C = f-1(F,3)A = f-1(C,2)
Node Internal Architecture Node Internal Architecture Blackboard Approach
◦Communication model for sharing memory objects through the network
Uses Memory Object Service◦Simple unreliable page service◦Simple memory object service
Unreliable Page ServiceUnreliable Page ServiceSimple unreliable
page service◦ Pages can be
mapped to packet level transmissions using PageIDs
◦ Allows for identifying and caching individual packets within the network
◦ There is one PageID for each Fid
Memory Pages
Memory Object ServiceMemory Object ServiceWorks on the page
serviceSupports larger
data files (over one memory page)
Supports update on memory objects
Memory PagesMemory Object
Memory Object Creation & Memory Object Creation & PublicationPublication
1. Publisher requests a meta-publication and a memory object2. Memory service returns a memory object with the meta attached
3. Publisher fills the data in the memory object
Blackboard
4. Publisher creates a Rid for the memory object. It can be republished as many times as he needs with equal or different tags
Publisher
Memory Object Memory Object SubscriptionSubscriptionSubscriber subscribers to the MO-RidSubscriber receives an event stream
◦File descriptor which can be read with select
Whenever the publisher publishes some data, the subscriber will be notified and will receive a new memory object representing the corresponding version of the object
Node Internals – Node Internals – NetworkingNetworkingNodes
◦Local input queues, with one Fid for each queue
◦Local output queues, with one Fid for each queue
◦Sender-helper sends the packet identified with the interfaces Fid
◦Network-receiver receives packets from the network
Node Internal ArchitectureNode Internal Architecture
Node Internal - Networking
Publish call
Node Internal ArchitectureNode Internal Architecture
Node Internal - Networking
Subscribe call
MemoryObject
Memory Object Memory Object SegmentationSegmentationPacketiser helper
◦handles memory objects larger than a packet size
◦Partitions objects into a set of packetsAt each local output places, whenever
there is a new object, the packetiser chunks the data and places each of them in the correct output queue scope
There is no memory copy, just a new mapping of the object in the scope directory
Memory Object Memory Object SegmentationSegmentation
Local Blackboard
Helper FunctionsHelper FunctionsNot core network serviceNot traditional applications
◦Actually, it is a set of functions providing additional features for the system
◦“Helps the system!”PSIRP has three categories:
◦Network Management Functions◦Remote Service Functions◦Host Centric Functions
Network Management Network Management FunctionsFunctionsNetwork Management &
Information gatheringInformation gathering for better
planning and management◦Increase performance
In PSIRP, network management functions can subscribe to the link state, receiving information as it becomes available
Network Management Network Management Functions – ExampleFunctions – ExampleCurrent Internet
◦SNMP/ICMP◦GET/SET/TRAP directives◦Managed systems generate
asynchronous messages to notify new events
Network Management Network Management Functions – Unified Link Functions – Unified Link Layer API (ULLA)Layer API (ULLA)Closely resembles publish/subscribeIntroduces an interface to collect
information from the link layerLink aware applications express
their interests by specifying link conditions
Notifications can be asynchronous or periodic
ULLA can be the base for the helper function which subscribes to the link conditions and wait for updates
Remote Service FunctionsRemote Service FunctionsSegmentation function
◦Helps in the segmentation decision◦Extracts data from link layer, such as
MTU, and provides parameters for the fragmentation
Merger function◦Merges partitioned data to get the
original publication
Remote Service FunctionsRemote Service FunctionsForwarding Error
Correction/Content protection◦Error correction in the transmission◦Recover received data without
retransmissions◦Send redundant data together◦Ex. Reed Solomon Block Code (N, K)◦N-K symbol redundancy◦After receiving any K symbols out of
N, the receiver is able to decode the original data
FEC ParametersFEC Parameters◦Currently, the FEC parameters are
statically set, degrading the performance
◦Inaccuracy in the estimation of the underlying channel state, resulting in inefficient bandwidth usage
◦FEC helper can subscribe to the channel conditions in order to generate better parameters
Remote Service FunctionsRemote Service FunctionsContent re-encoding
◦Scenario: heterogeneous subscribers requesting the same video but with different resolution
◦Content provider can’t attend the demand
◦Content provider asks for remote helper functions located closer to the subscribers to perform the translation
Application-specific Application-specific CachesCachesContent caching
◦Offers the same interface as the original content provider (transparent)
◦Applications can select from which cache to retrieve data
◦content providers may instruct caches to serve content on their behalf
◦subscribers can receive data from the closest caches, saving network resources
Host Service FunctionsHost Service FunctionsTransport and other middle network
servicesPSIRP offers shared blackboard for
distributed applications using the network
Analysis and debugging purposes◦Ping, traceroute, tcpdump, wireshark
PSIRP context◦Transport component for rate control and
reliability algorithm for point-to-point delivery
Rendezvous SystemRendezvous SystemMeeting point of interests
between publishers and subscribers
Requirements◦Scalability (Internet scale)◦Efficiency in operation◦Deployability (gradual deployment
and stakeholders incentives)
Rendezvous NetworkRendezvous NetworkComposed of Rendezvous Nodes
(RNs)Organized as a BGP-like inter-domain
hierarchyCollection of RNs from multiple
cooperative ASesRendezvous points (RPs)
◦logical meeting points of interests between publishers and subscribers
◦one rendezvous point for each Sid/Rid pair
Rendezvous Network Rendezvous Network InterconnectionInterconnectionSome approaches
◦Central entity managing signaling reachability: lack of incentives, trust, competition
◦Multiple entities: providers compete for global reachability coverage
◦Fully distributed mechanism: rendezvous networks self-organize to interconnect without any third party infrastructure
Rendezvous Network Rendezvous Network ConnectionConnectionCanon DHT
◦Hierarchical DHT◦Maintains local communication
within its domain◦Caching in the sub-domain exit
between levels in the DHT
Canon Hierarchical DHTCanon Hierarchical DHT
Domain A Domain B
Top-level Domain
Rendezvous System Rendezvous System ExampleExample
Rendezvous System Rendezvous System BootstrappingBootstrapping
Publish OperationPublish Operation
Subscribe OperationSubscribe Operation
SummarizingSummarizingIdentifiers
◦ Many types, special semanticsAlgorithmic ID
◦ New use casesNode Internal Architecture
◦ Blackboard, Input/output queues, memory objects segmentation
Helper functions◦ Adding new functionalities in the
architectureRendezvous System
◦ Matching interests between publishers and subscribers
Questions?Questions?Comments?Thanks!