cat l1: applications to distributed...

CAT L1: Applications to Distributed Computing

Samson Abramsky

Department of Computer Science, University of Oxford

Samson Abramsky (Department of Computer Science, University of Oxford)CAT L1: Applications to Distributed Computing 1 / 20

Overview

In this part of the course (5 lectures), we shall discuss two applications ofalgebraic topology ideas.

Applications to distributed computing. The topology of asynchronouscomputability.

Applications to foundations of quantum mechanics and quantum information.The cohomology of non-locality and contextuality.Not exactly the advertised topic, but also uses sheaf cohomology.

We begin with distributed computing.

Overview

Applications to foundations of quantum mechanics and quantum information.The cohomology of non-locality and contextuality.

Not exactly the advertised topic, but also uses sheaf cohomology.

Overview

Distributed Computing

A large area of study!

All computing nowadays is highly spatially distributed, happens over networks,involves lots of different autonomous components.

The area of Distributed Algorithms or Principles of Distributed Computing is thetheoretical study of what can and can’t be computed in a distributed fashionunder various assumptions, and the complexity of what can be computed.

Distributed computing is very subtle! There is a high degree of non-determinisminvolved. And the assumptions about the computational model have a stronginfluence on which tasks can be accomplished.

The Story Line

Theoretical work in this area looks very complicated and low-level. Differentmachine models, simulations, reductions, tricky counting arguments.

A seminal result in the field:

Fischer, Lynch and Paterson, “Impossibility of Distributed Consensus With OneFaulty Process”, 1985.

A striking generalization and combinatorial characterization:

Biran, Moran and Zaks, “A Combinatorial Characterization of the Distributed1-Solvable Tasks”, 1990.

Purely combinatorial (graph-theoretic, low-dimensional topological),‘computational model free’ characterization.

How to go beyond this?

The decisive breakthrough came around 1993 (full papers published in 1999 and2000), and won the 2004 Godel prize.

The Story Line

It’s Topology!The 2004 Godel Prize for outstanding journal articles in theoretical

computer science is shared between the papers: “The Topological Structure of

Asynchronous Computation” by Maurice Herlihy and Nir Shavit, Journal of

the ACM, Vol. 46 (1999), 858-923, and “Wait-Free k-Set Agreement Is

Impossible: The Topology of Public Knowledge” by Michael Saks and Fotios

Zaharoglou, SIAM J. on Computing, Vol. 29 (2000), 1449-1483.The two papers recognized by the 2004 Godel Prize offer one of the most

important breakthroughs in the theory of distributed computing.The problem attacked is the complete understanding of asynchronous

wait-free deterministic computation in the basic shared memory model. Thesepapers demonstrate that one can avoid the inherent difficulty of analyzing adynamic model, transforming it into a static one by associating computationaltasks with simplicial complexes and translating the question of existence of await-free protocol into (distinct but related) topological questions about thecomplexes. This reformulation allows the introduction of powerful topologicalinvariants, such as homologies, to show the impossibility of numerous tasks,including set-agreement and renaming.

The discovery of the topological nature of distributed computing provides

a new perspective on the area and represents one of the most striking

examples, possibly in all of applied mathematics, of the use of topological

structures to quantify natural computational phenomena.Samson Abramsky (Department of Computer Science, University of Oxford)CAT L1: Applications to Distributed Computing 5 / 20

The Scenario

We have N processes, each running autonomously on their own processor. Sincewe want to regard this as a single system, rather than a collection of separatesystems, we are interested in how they can interact and coordinate to achieve acommon goal.

There are various models which serve as reasonable abstractions of ‘real’ systems,and have been extensively studied.

A major distinction is between synchronous and asynchronous models:

In a synchronous model, there are time bounds, known to each process, onhow long the next actions by other processes should take.

Asynchronous models have no such bounds. This means that what eachprocess can ‘know’ about the state of the network, based on its own localinformation, is much more limited.

Within each type of model, there are two main types: based on message passingand shared memory.

We shall use asynchronous message-passing as our reference model, but theresults will also apply to asynchronous shared-memory.

The ScenarioWe have N processes, each running autonomously on their own processor. Sincewe want to regard this as a single system, rather than a collection of separatesystems, we are interested in how they can interact and coordinate to achieve acommon goal.

The asynchronous message-passing model

To fix ideas . . .

Each process has a unique id.

Events have the form (p,m), where p is a process id and m is the messagecontents.

Each process runs a deterministic program, accessing local state, an inputregister, and an output register.

There is a global message queue (unordered). The primitive operations onthis queue are send(e) and receive(e), where e = (p,m) is an event.

A basic step of the system is determined by some event (p,m) in the queue.Process p receives the message (which is removed from the queue), performssome internal computation, and sends some messages which are placed onthe queue.

A process can halt by writing into its output register (which is ‘write-once’).

System executions are sequences of steps. A complete execution is one inwhich all processes halt. An execution with k failures is one in which N − kprocesses halt, and the remaining processes have no further actions.

To fix ideas . . .

The 1-Resilient Consensus Problem

Each process starts with a bit in its input register. Any initial assignment ispossible.

Initially, there is an undefined value in the output register of each process.

The specification of 1-resilient correctness is:

No accessible configuration has more than one decision value.

There are accessible configurations with both possible decision values.

If at most one process fails, every complete execution will result in someprocess having a decision value.

The FLP Theorem:

TheoremNo asynchronous protocol can solve the 1-resilient consensus problem.

This is an impossibility result: we are proving something about all programs in thismodel.

Note that, if we assumed a centralized control, the problem is trivially computable.

The 1-Resilient Consensus ProblemEach process starts with a bit in its input register. Any initial assignment ispossible.

The FLP Theorem:

Note that, if we assumed a centralized control, the problem is trivially computable.Samson Abramsky (Department of Computer Science, University of Oxford)CAT L1: Applications to Distributed Computing 8 / 20

Parallel Moves/Diamond Lemma

We write Ce- D when the event e can be applied to configuration C resulting

in configuration D; and Cσ- D when a sequence of steps σ can be applied.

Lemma (Parallel Moves)

Suppose that σ and τ are sequences of events involving disjoint sets of processes.

Then whenever Cσ- D and C

τ- E , Dτ- F and E

σ- F .

F�..

........

......

................................

Parallel Moves/Diamond LemmaWe write C

e- D when the event e can be applied to configuration C resulting

σ- F .

F�..

........

......

................................

σ- F .

F�..

........

......

................................

σ- F .

F�..

........

......

................................

Valency

The key notion in the FLP proof.

We say that a configuration is i -valent, i = 0, 1, if every final configurationreachable from it has decision value i ; it is bivalent if there are reachableconfigurations with both possible values.

LemmaIn any protocol for consensus, there must exist a bivalent initial configuration.

Proof Initial states are described by input vectors IN . Two vectors are adjacentif they differ in exactly one component. The adjacency graph is connected.Suppose for a contradiction that all input vectors are univalent. Then we can findadjacent input vectors v , w where v is 0-valent and w is 1-valent. Now consider acomputation in which the process at which they differ fails, and all the otherprocesses run to completion. �

Valency

Forever Undecided

Proposition

Every reachable bivalent configuration has a bivalent successor.

Proof Suppose C is a reachable bivalent configuration, and all its successors

are univalent. Hence it must have a 0-valent successor Ce- D0, and a 1-valent

successor Ce′- D1, where e = (p,m), e′ = (p′,m′).

Case 1: p 6= p′. In this case, we can apply the Parallel Moves lemma to get acontradiction.

Case 2: p = p′. In this case, consider Cσ- E , where σ does not involve p. If E

has a decision value, we can use Parallel Moves with respect to e to conclude thatthis value must be 0, and Parallel Moves with respect to e′ to conclude that thisvalue must be 1. So we can never reach a decision value from C when p fails,contradicting 1-resilience. �

Thus we can find a complete execution which never reaches a decision value,establishing the FLP theorem.

Forever Undecided

Proposition

Forever Undecided

Proposition

Forever Undecided

Proposition

Forever Undecided

Proposition

Forever Undecided

Proposition

Combinatorial Characterization of Distributed 1-SolvableTasks

A significant step was taken in two papers, by Moran and Wolfenhaus, and Biran,Moran and Zaks:

A general description of 1-solvability of distributed decision tasks in anasynchronous model.

A purely combinatorial (implicitly, topological) characterization of1-solvability.

We have a set of input values X and a set of decision valued D. In an N-processnetwork, input vectors are in XN and decision vectors in DN . The taskspecification is a relation

T : XN −→ P(DN).

Examples:

Consensus

Leader election

Renaming

T : XN −→ P(DN).

Examples:

Consensus

Leader election

Renaming

T : XN −→ P(DN).

Examples:

Consensus

Leader election

Renaming

T : XN −→ P(DN).

Examples:

Consensus

Leader election

Renaming

T : XN −→ P(DN).

Examples:

Consensus

Leader election

Renaming

T : XN −→ P(DN).

Examples:

Consensus

Leader election

Renaming

1-Solvability

A protocol species a program Pi to run at the i ’th node of the network for eachi = 1, . . . ,N. As before, the processes communicate by asynchronous messagepassing. (the results also apply if we assume asynchronous shared memory).

A process is faulty in an execution if from some point on, all messages sent bythat process are never received.

A protocol solves a task T if the following conditions hold, for every input vectorx:

If no process is faulty, every process halts with a decision value such thatd ∈ T (x).

If one process is faulty, all the other processes eventually decide.

Note that in the asynchronous model, if process i fails, none of the otherprocesses can ever know this for certain!

1-Solvability

Extendability

A partial output vector is one with some components undefined. A partial vectord is extendable to d ′ if d ′ can be obtained by replacing the undefined componentsin d with defined values.

LemmaIf a protocol 1-solves a task, then in any execution starting in x where process ifails, all the other processes eventually produce decision values, yielding a partialvector d which is extendable to a vector in T (x).

Proof By 1-solvability, all the other proesses will eventually produce decisionvalues. The execution can be extended to a complete one, which must solve thetask. �

Extendability

Connectedness

Write G (AN) for the adjacency graph of the set of vectors AN . GT is thesub-graph of G (DN) determined by the image of T .

A key insight is that, in transforming an input vector into a decision vector,connectedness is preserved. This gives an important necessary condition for1-solvability.

If d is a decision vector in DN , then any N − 1 components uniquely determine itsconnected component in GT .

Proof Given d in GT , let d1 and d2 be any two partial vectors obtaining byomitting one component in d. Let ei be any vector in GT to which di isextendable, i = 1, 2. Both e1 and e2 are adjacent to d, hence in the sameconnected component. �

Connectedness

Results

Theorem

if a task T has a connected input graph I ⊆ XN and a disconnected decisiongraph GT , then T is not 1-solvable.

Proof The proof is by reduction to unsolvability of consensus, i.e. the FLPtheorem. Let C1, . . . ,Ck be the connected components of GT . Given a protocol Pfor T , we construct a new protocol Pc as follows. When a process is about to haltwith a decision value in P, it broadcasts the value to all processes and waits.Since P 1-solves T , eventually N − 1 processes will broadcast their values. Thuseach non-faulty process will eventually receive N − 1 values (including its own).By the previous Lemmas, this partial vector is extendable to a decision vector forT , and the connected component Cj to which this decision vector belongs isuniquely determined. Upon receiving N − 1 decision values which determine Cj ,each process decides on the value given by the parity of j in Pc . Since allprocesses will agree on the connected component, protocol Pc 1-solves distributedconsensus, a contradiction by the FLP theorem. �

Results

Theorem

Results

Theorem

The BMZ Characerization

A remarkable point is that this necessary condition for 1-solvability is given purelyin terms of topological conditions on geometric objects associated with the task,and independently of any computational details.

The full BMZ result gives a necessary and sufficient condition for 1-solvability, interms of connectedness, and another condition related to extendability.

The sufficiency is proved by an ingenious construction of a ‘universal protocol’,which we shall not discuss.

The question of how to generalize this result, e.g. from one to many failures,remained open for a number of years.

The decisive break-through was obtained by explicitly introducing concepts fromalgebraic topology.

Modelling problems and protocols with simplicial complexes

We shall deal with abstract simplicial complexes.

The vertices are of the form (Pi , vi ), where Pi is a process id, and vi is a value(input or output).

A n-simplex is a set of n + 1 vertices. It is properly coloured if each vertex islabelled with a distinct id. The set of process ids associated with an n-simplex Sn

is ids(Sn), and the set of values is vals(Sn).

A decision task for n + 1 processes is given by an input complex I, an outputcomplex O, and a map ∆ which takes each n-simplex in I to a set of n-simplexesof O.

Interpretation:

I consists of possible inputs to the processes participating in the protocol.

O consists of the possible final decision states of the protocol.

∆ specifies which final states are acceptable for each input.

This assignment is extended to lower-dimensional simplices, representing the caseswhere some processes do not run. ∆(Sm) is the set of legal final states when onlythe indicated m + 1 processes run, and n −m processes fail.

Modelling problems and protocols with simplicial complexesWe shall deal with abstract simplicial complexes.

Interpretation:

Examples

Binary consensus.

Renaming.

Examples

Binary consensus.

Renaming.

Examples

Binary consensus.

Renaming.

Examples

Binary consensus.

Renaming.

Protocols

A protocol has an associated protocol complex P consisting of all the reachablefinal states in the protocol. Lower dimensional simplexes correspond to the caseswhere some processes fail.

Example: one-round synchronous protocol.

Let (I,O,∆) be a task specification. If P is a protocol with protocol complex P,we write P(Sn) ⊆ P for the subcomplex of final states reachable under P fromthe initial state Sn.

A protocol solves a decision task (I,O,∆) if there is a process id-preservingsimplicial map δ : P −→ O such that, for every input simplex Sn:

δ(P(Sn)) ⊆ ∆(Sn).

We shall use topological properties of P and O to show that no such map canexist.

Protocols

δ(P(Sn)) ⊆ ∆(Sn).

Protocols

δ(P(Sn)) ⊆ ∆(Sn).

Protocols

δ(P(Sn)) ⊆ ∆(Sn).

Protocols

δ(P(Sn)) ⊆ ∆(Sn).

Protocols

δ(P(Sn)) ⊆ ∆(Sn).

cat l1: applications to distributed...

Documents