Garbage Collecting the World: One Car at a Time

Richard L. Hudson, Ron Monison,~ J. Eliot B. Moss, &David S. Munrot

Department of Computer Science, University of Massachusetts,Amherst, MA 01003, U.S.A. ’

Email: { hudson, moss} @cs.umass.edu

Schoo1 of Mathematical and Computational Sciences,University of St Andrews, North Haugh, St Andrews, Fife,

KY16 9SS, ScotlandEmail: (ron, dave} @dcs.st-and.ac.uk

Presented by: Martin Krogel

Outline

Goals Assumptions DMOS Collector

Rules Example Addressing Objects Pointer Tracking Object Substitution Protocol Car and Train Management Unwanted Relative Problem Cleaning up Trains Safety and Completeness

Related Works Conclusion Future Work Questions?

Goals

Motivation was to provide garbage collection for distributed systems with the following properties.

Safety Completeness Non-disruptiveness Incrementality Scalability Non-blocking

Assumptions

Nodes have local storage and only use message passing.

Ordered delivery of messages. Nodes appear to operate correctly. No bounds on relative computation rates. Events at a node are totally ordered.

DMOS Collector

(Distributed Mature Object Space) Based on MOS and PMOS garbage

collection. Uses trains and cars analogy. Also uses relative ages of trains.

Rules

1) Data locally reachable from roots is copied to a younger train.

2) Data locally reachable from younger trains is copies to those trains. (If from multiple trains, any will do.)

3) Data locally reachable from older trains is copied to another car in the current train.

4) Data locally reachable from other cars of the same train is copied to another care of the same train.

5) All remaining data is unreachable and is reclaimed.

Rules (handling cycles)

0) If no object in a train is reachable from outside the train, collect the entire train.

1) Data locally reachable from roots is copied to a younger train.

2) Data locally reachable from younger trains is copies to those trains. (If from multiple trains, any will do.)

3) Data locally reachable from older trains is copied to another car in the current train.

4) Data locally reachable from other cars of the same train is copied to another care of the same train.

5) All remaining data is unreachable and is reclaimed.

Rules (non-local trains)

0) If no object in a train is reachable from outside the train, collect the entire train.

1) Data locally reachable from roots is copied to a younger train.

2) Data locally reachable from younger trains is copies to those trains. (If from multiple trains, any will do.) If the destination train is not represented on this node, then the node should join the train and create a new car in that train.

3) Data locally reachable from older trains is copied to another car in the current train.

4) Data locally reachable from other cars of the same train is copied to another care of the same train.

5) All remaining data is unreachable and is reclaimed.

Example (Step 1)

Example (Step 2)

Example (Step 3)

Example (Step 4)

Addressing Objects

DMOS objects encode logical address and home node.

Car numbers are not reused, or reused very slowly.

Pointer Tracking Events

Event Description<s, n, o, A, B> This (send) event indicates that node A has sent to node B a pointer to o. The

number n is chosen such that no other event <s, m, o, A, B> has m=n, i.e., n, o, A, and B together uniquely identify the s event for all time and space. This event is said to happen at A (the sender), hence it is A's responsibility to inform H. Intuition: This indicates that a new pointer to o has been created in the virtual channel A -> B. The number n is used to match s events with their corresponding r events.

<r, n, o, A, B> This (receive) event indicates that node B has received a pointer to o sent by node A. The number n is chosen to match the corresponding s event. This event happens at B. Intuition: An r event indicates that the pointer has been deleted from the virtual channel A -> B and has been created in a message receive buffer at B.

<d, n, o, A, B> This (delete) event indicates that node B has deleted from its message buffers the received pointer to o uniquely identified by n, A, and B. The number n corresponds to the s and r messages previously described. This event happens at B.

<+, m, o, A> This event indicates that node A has created a new pointer, uniquely identified by m to object o. This event happens at A.

<-, m, o, A> This event indicates that node A has deleted the specific pointer, uniquely identified by m to object o. This event happens at A.

Node A Sends to Node Ba Pointer to o

Ordering of Events

any(o, Y, E) Indicates whether Y contains any pointers to o

based on information in the given set of events E.

<r, n, o, X, Y>, but no <d, n, o, X, Y> or <+, n, o, Y>, but no <-, n, o, Y>.

absence(o, E) True if and only if there are no pointers to o at

nodes other than o's home node H. any(o, X, E) is false for all nodes X other than H.

Pointer Tracking Optimization

Removing the unique numbers from events. Referring fewer events to H. Further reducing the detail required at H. Piggy-backing and compressing messages. Combining events

Event Effect at H

<s, o, B> +<r, o, B> - +<+, o, B> +<-, o, B> -

inTransit(o, ->B) pointersTo(o, B)

Object Substitution Protocol

When o is substituted by o' H creates KnownNodes(o) list H sends move message [m, o, o'] to known nodes X receives move message X inserts o o' in relocation table

X receives a pointer to o Generate received o message <r, o, X> Generate add o' message <+, o', X> Generate remove o message <-, o, X>

H receives message about o from X Check KnownNodes(o) If X is not present, add it and send move message.

If H' H H sends o' pointer to H' H forwards manipulation messages to H'

Object Substitution Protocol

Cleaning up the Tables Check for absence(o, E) H sends and end of move message [e, o, o'] to X H removes X from KnownNodes(o) H removes o o' from relocation table X removes o o' from relocation table

Multiple Substitutions Opaque Addressing

Car and Train Management

Solving completeness Remembered Sets and Sticky Remembered

Sets Basic train management

Identifying successor(X, n:A) Logical token passing ring Joining a ring

X sends join message [join, X, n:A] X becomes successor of A in ring

Leaving a ring X sends leave message [leave, X, successor(X, n:A),

n:A]

Car and Train Management

Basic train management Identifying successor(X, n:A) Logical token passing ring Joining a ring

X sends join message [join, X, n:A] X becomes successor of A in ring

Leaving a ring X sends leave message [leave, X, successor(X, n:A),

n:A]

Car and Train Management

How a node X should respond to a leave message [leave, Y, Z, n:A]. Case 1: Y = successor(X, n:A), i.e., X is Y's

predecessor: X sets successor(X, n:A) to be Z (Y's successor) and sends the message [left, n:A] to Y.

Case 2: Z = X and X is not in the process of leaving the ring: X sends the message [leave, Y, Z, n:A] to successor(X, n:A).

Case 3: Z = X and X is in the process of leaving the ring: X sends a [leave, Y, successor(X, n:A), n:A] message to successor(X, n:A).

Two Nodes Leave a Train Ring at the Same Time

Car and Train Management

Train Reclamation (using Token Passing) Initial state Starting the token Receiving the token

Rule 1: External pointers in sticky remembered sets for train

Rule 2: No external pointers, but “changed bit” is true Rule 3: No external pointers and “changed bit” is false

Special case: If Y receives token with value Y, there are no external pointers to the train, and it can be reclaimed

Illustration of Correctness of Train Reclamation Algorithm

Assuming no new objects are added to the train

Unwanted Relative Problem

Unwanted Relative Problem

Rejected solutions Disallow moving unwanted relatives into n:A Delay moving relative Only reclaim oldest trains

Accepted solution: Epochs (old/new) When a token is started, all cars are considered old

epoch. Cars added while “changed bit” is false are considered

new epoch Cars added while “changed bit” is false are considered

old epoch If “changed bit” switches from false to true, all cars on

train are considered old epoch “Changed bit” is only set when pointer is detected to old

epoch car in the train

Cleaning up Trains

X receives the token for n:A with value X. (Old epoch cars are unreachable). X deletes old epoch cars X sends [end-of-epoch, X] message.

Y receives end-of-epoch message and Y X. Y deletes old epoch cars Y changes new epoch cars to old epoch cars Y forwards message

X receives end-of-epoch message. X restarts token passing algorithm

If train is oldest, there will be no new epoch cars, so the train will be fully collected.

Safety and Completeness

Safety Object substitution Car / Train reclamation

Completeness Oldest train will eventually be collected

Related Works

Migration Bishop's non-distributed garbage collection algorithm

Reference Counting Bevan's, and Watson & Watson's weighted reference counting. Reference listing. Optimized weighted reference counting with background global

tracing. Tracing

Hughes' global timestamped live object trace Liskov and Ladin suggest centralized time server

Lang, Queninnec, and Piquer's grouped mark/sweep algorithm. Ferreira and Shapiro's multiple site segment replication Maheshwari and Liskov's partitioned garbage collection

Garbage Tracing Vestal's test decrement of suspected cyclic garbage Lins & Jones' combining weighted reference counting with mark and

sweep from deletion points (not roots). Maeda et al. And Fuchs' tracing potentially cyclic garbage

Conclusion

DMOS is a new distributed garbage collection algorithm that satisfy all of the following properties:

Safety Completeness Non-disruptiveness Incrementality Scalability Non-blocking

Future Work

Implementation and practical evaluation. Algorithmic performance analysis. Extension to tolerate node and

communications failures.

Questions?