sombrero: implementation of a single address space paradigm for distributed computing exhibiting...
Post on 21-Dec-2015
219 views
TRANSCRIPT
Sombrero: Implementation of a
Single Address Space Paradigm for Distributed
Computing Exhibiting Reduced Complexity
Alan SkousenArizona State University
Operating Systems Research
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
2
OUTLINE• Basics• Why SASOS?• Computer Evolution• Protection Models• Sombrero Current Architecture• Implementation Effort• Middle Level Architecture and Tools• Research Contributions• Summary
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
3
Many Address Space Operating Systems - MASOS
• Current Operating System Technology is based on Multiple VA spaces commonly known as processes.
• UNIX, Windows NT, Windows 9x, Linux, MACH etc.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
4
Single Address Space Operating Systems - SASOS
• Use only a single address space
• Examples:MS-DOS, many embedded OSs, Mac etc.
• Single VA space OS’s: AS400, Opal, Mungi, Monads, Sombrero etc.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
5
Namespace
• A domain of all possible names each of which can be paired with at most one object.
• Namespaces include: File names, IP numbers, Capabilities, DSM space, the addresses in a virtual address space.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
6
Very Large Single Address Space Operating System – VLSASOS
• Very Large Namespace
• 64 bit address space, 18 Quintillion bytes
• 4GB/s can be allocated for 136 years
• Can be used instead of file systems and other name spaces. Reduces the need for namespace translation.
• Est 30% of code used for trans to/from store
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
7
Why SASOS?• Single namespace allows complexity
reduction due to elimination of code performing namespace translations. Atkinson 30% translation code; Feigen 80% program effort, 65% bug prediction.
• Reduced requirements mean cost of writing programs is reduced.
• Natural persistence, reduced memory copying, and reduced context switching.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
8
Coding example//MASOS file server:
#include <header.h>void main(void){ struct message m1, m2; /* incoming and outgoing messages */ int r; /* result code */
while (1) { receive (FILE_SERVER,&m1); /* server runs forever */ switch(m1.opcode) { case CREATE: r = do_create(&m1, &m2); break; case READ: r = do_read(&m1, &m2); break; case WRITE: r = do_write(&m1, &m2); break; case DELETE: r = do_delete(&m1, &m2); break; default: r = E_BAD_OPCODE; } m2.result = r; /* return result to client */ send(m1.source, &m2); /* send reply */ }}
//MASOS client that uses file server to copy a file:
#include <header.h>int copy(char *src, char *dst) /* procedure to copy file using the server */{ struct message m1; /* message buffer */ long position; /* current file position */ long client = 110; /* client's address */
initialize(); /* prepare for execution */ position = 0; do { /* Get block of data from source file */ m1.opcode = READ; /* operation is a read */ m1.offset = position; /* current position in the file */ m1.count = BUF_SIZE; /* how many bytes to read */ strcpy(&m1.name, src); /* copy name of file to be read to message */ send(FILE_SERVER, &m1); /* send message to the file server */ receive(client, &m1); /* block waiting for the reply */
/* Write the data just received to the destination file */ m1.opcode = WRITE; /* operation is a write */ m1.offset = position; /* current position in the file */ m1.count = m1.result; /* how many bytes to write */ strcpy(&m1.name, dst); /* copy name of file to be written to buf */ send(FILE_SERVER, &m1); /* send the message to the file server */ receive(client, &m1); /* block waiting for the reply */ position += m1.result; /* m1.result is the number of bytes written */ } while (m1.result > 0); /* iterate until done */ return(m1.result >= 0 ? OK : m1.result); /* return OK or error code */}
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
9
Coding cont.
//SASOS Memory object copy
int copy(char *src, char *dst){ FILE_OBJECT *from, *to; from = address(src); /*address() is a new function that obtains an object's*/ to = address(dst); /*address from the NameServer.*/ *to = *from;}
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
10
Server access Active Passive
Client Thread
Server has own
Thread
Send Message
Receive Message
Client Thread
Server uses client
Thread
Calling Thread
Returning Thread
SASOS and MASOS SASOS only
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
11
Computer Evolution• VA space reuse is compelling in 16 and 32 bit
computers.
• Inherent isolation solves protection problem for free.
• It also creates a very large access problem for sharing and communication.
• The process paradigm is now the accepted way.
• Much OS research energy is therefore dedicated to making inter-process access less difficult.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
12
Evolution cont.
• 64 bit processors encourage a new approach since VA space reuse is no longer a compelling issue.
• The protection that came for free in the process paradigm remains a compelling issue.
• A different approach to the access problem is to make protection the issue and get access for free.
• Sombrero represents that paradigm switch.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
13
MASOS Vs SASOS
More Protection
Easier Memory Sharing
•Relatively Secure•Trust is implicit•Limited to Active Services since threads can’t migrate.
•Pointer Translation•Pipes/RPC/Sockets•Files for communication•Distributed Shared Memory
•RAM-Centricity
• Easily Corruptible•Very Trusting•Passive Services•Transparent memory sharing•Simple communication semantics, i.e. no IPC
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
14
Main Issue in VLSASOS Design is the Protection Mechanism
• Two main issues in protection mechanism: Memory protection; Protection Domain Switching.
• Two Protection Models are used: Standard Access Matrix; CPU Access Matrix
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
15
Standard Access Matrix (SAM)
• SAM is an Explicit Protection model that requires user code to invoke.
• SASOS PD Switching is normally based on SAM model. Most use Capabilities, All are Explicit PD Switching models.
U S E RD o m a in s M O 1 M O 2 M O 3 M O 4 P t r P l t U 1 U 2 U 3U S E R 1 r e a d R / W E n t e rU S E R 2 r e a d R W X w r i t eU S E R 3 w r i t e w r i t e
S y s t e m R e s o u r c e s
A c c e s s M a t r i x : U x - U S E R x , M O - M e m o r y O b j e c t , P t r - P r i n t e r , P l t - P l o t t e r .
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
16
CPU Access Matrix (CAM)• CAM is an Implicit Protection model.
• Protection and PD switching are Implicit
• Makes better use of the SAS properties and reduces program complexity even more.
• Used for memory access protection (TLBs)
V A 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 V A 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 . . . . . . . . . . . V A F F F F F F F F F F F F F F F E V A F F F F F F F F F F F F F F F F
R W X R W X R W X R W X R W XG P D - 1 A A AG P D - 2 S - 3 A A A AG P D - 3 A A A A S - 1G P D - 4
A l l 2 ^ 6 4 V i r t u a l A d d r e s s e s / A c c e s s R ig h t s f o r e a c h a d d r e s sP r o t e c t io nD o m a in s
C P U A c c e s s M a t r i x f o r a 6 4 b i t v i r t u a l a d d r e s s s y s t e m - A f o r A C C E P T ; R E J E C T i s i n d i c a t e d b y a b l a n k e n t r y a n dS f o r S W I T C H w h i c h r e t u r n s a G P D n a m e .
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
17
Standard vs. CPU Access Matrix
The standard Access Matrix is good at representing protection policy.
The CPU Access Matrix is good at representing protected access in terms that the CPU can directly use.
By combining the two matrices we get the best of both. This allows implicit (transparent) protection and domain switching.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
18
SOMBRERO HARDWARE• Implements CPU Access Matrix.• Region Protection Lookaside Buffer - RPLB.• Projects hard walls of protection into VA space.• Introduces Implicit PD Switching.• Implements classical OO encapsulation in
hardware: Services don’t need to depend on the compiler for protection. Objects accessible only through entry points.
• Allows dependable passive services.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
19
S W IT C HA D D R E S S
N E W G P DN A M E
G P Dor
T h readA C C E S S R IG H T S
R |W |XS -F LA GR P LB S T O R E D
S W IT C H
C U R R E N T V IR T U A L A D D R E S S G P D B R A T T E M P T E DA C C E S S T YP E
T B R
C O M P A R E T OT O F O R M A H IT
or
T he C urren t C P UV irtual add res s isc om pared to thes tored s w itc h
D ata loadedfrom anR C B .
R P L B in m od e 2 , a .k .a . S R P L B
P r o d u c e sw h e r e X i s 0 o r 1 .
M A T C H M A S K 0 0 0 0 1 1 0 0 1 0 0 0 0 0 0 0
D O N ’T C A R E M A S K 0 0 0 0 0 0 0 1 0 1 1 1 1 1 1 1
R A N G E F U N C T I O N o n
a n d
R A N G E M A S K 0 0 0 0 1 1 0 X 1 X X X X X X X
0 H C H o r D H 8 0 H - F F H
R a n g e f u n c t io n e x a m p le
M A T C H M A S K D O N ’T C A R EM A S K
G P Dor
T hreadA C C E S S R IG H T S
R |W |XS -FLA G
R A N G E M A S KG P D
orT hread
A C C E S S R IG H T SR |W |X
C U R R E N T V IR T U A L A D D R E S S G P D B R A T T E M P T E DA C C E S S T YP E
T B R
R P LB S T O R E D T A G
R P LB LO G IC A L T A G
C O M P A R E T OT O FO R M A H IT
or
T he R ange M askis generated fromthe s tored tag
T he C urrent C P UV irtual address iscom pared to thelog ical tag
D ata loadedin R P LBfrom R C B
RPLB m ode 1 opera tion a .k .a . IRPLB or D RPLB.
To a first approximation the RPLBFunctions in a manner similarto a subnet mask in a network router
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
20
Carrier Protection Domain (CPD)
• Sombrero distinguishes between two PD types:
• General Protection Domain – Memory, Executable code, and PD switches
• Carrier PD – Memory and PD switches. Used by threads to ‘carry’ state. Real thread local storage.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
21
Protection and Resource Access List - PRAL
• Both the standard Access Matrix and CPU Access Matrix data are stored in the PRAL.
• Traversed by CPU during execution or data access on RPLB miss.
• PRAL data is managed through protected system service calls.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
22
GCB 1
PAL
RACL
AD forG-1 M-1
AD forG-1 Sw-2
AD forG-1 M-2
. . . . .
GCB 2
PAL
RACL
AD forG-2 M-1
AD forG-2 M-3
AD forG-2 Sw-1
. . . . .
TCB 1
PAL
AD forT-1 M-2
AD forT-1 M-3
AD forT-1 Sw-2
. . . . .
MOCB 1
RACL
MOCB 2
RACL
MOCB 3
RACL
LEGEND:AD for G-i M-j Access Descriptor containing:1. Access rights of GPDi to MOj
2. Entry in MOj's ACL for GPDi.AD for G-i Sw-j Access Descriptor containing:1. Access rights of GPDi to GPDj and GPDj entry address2. Entry in GPDj's ACL for GPDi.
The PRAL provides CPU Access Matrix data tothe RPLB from the lists of accessible memory objects.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
23
Sombrero Program Instantiation• A Sombrero program instantiation has one or
more entry points (methods). The program can be as trivial and efficient as a subroutine call or as expensive as need be to support any trust relationship. Program methods are like conventional subroutines and are called with an argument list and can return a typed value depending on the entry point. This is the classical model for a class instance in OOP.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
24
Sombrero Current Architecture
• Kernel Services distributed among executive protection domains
• No central kernel and no hardware protected kernel mode
• A few Protection Domain Lock Registers name the protection domain that can access sensitive protected instructions and registers
PALCode executing in physical memory
Alpha CPU
Scheduler NIC Driver
Interrupts
Access Memory
Pager
Locked Interrupt Management
Locked NIC IO Ports
Page Misses
RPLB misses
Application
VA Space
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
25
VLSAS-D-OS
The Sombrero model is extended to the Network using a copyset algorithm known as Token Tracking.
Sombrero allows the network to be viewed as a single large NUMA multiprocessor.
Pointers remain valid across network.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
26
N1 N2 N3 N4 N5
W
A
N1 N2 N3 N4 N5
W
B
Last Known Writer Pointer
writee
Last Known Writer Pointer
The Modified Page Cache Graph: N5 currently holds the writeenabling token W which originated at N1. N6 is attempting toread the Memory object.
W
N1 N2 N3 N4 N5 N6 N1 N2 N3 N4 N5
W
N1 N2 N3 N4 N5
W
A B
CSowner
CS1st
CS2nd
CS3rd
CS4th
A Copy Set graph for aControl Block with anarity of two with theowner node as its root.
CS5th
Last Known Writer Graph Pruning of Last Known Writer Graph
Modified Page Cache Graph
Pruning of Modified Page Cache Graph
CopySet Graph
Distributed Object Copy Set Management
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
27
Network Consistency
• The Sombrero address space remains accessible and consistent across the network by distributing system level data to neighboring nodes.
• Implements selected consistency semantics for each memory object.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
28
Backward compatibility
• General purpose computing allows processor emulation.
• Fully emulated processor can install any OS for that processor.
• VMware uses this approach.
• Had been successful at running Intel programs on NT Alpha. FX32.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
29
Side Effects of Processor Emulation
• Any program running on Sombrero is distributed by default.
• Any OS installed on emulated hardware is therefore automatically distributed.
• End up with any OS plus virtualization and distribution for free.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
30
Implementation Effort• Sombrero constructed on a network of
cooperating computers
W2K Host
NT Development Alpha
NT and Sombrero
Target Alpha
Linux Development Alpha
Boot Loader and PALCode
Source Upload
Module Compile
Sombrero NIC
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
31
Development Tools
• Hardware – Networked W2K, Alpha Linux, Alpha NT 4.0, AlphaBios on Target Alpha
• Languages – C, C++, Alpha assembler• Compilers – VC++, MSC, GCC, GAS, ASAXP• My Custom Tools on Linux for the Sombrero
Compiler: sosbuild, buildsxe, catdebug• My Custom Tools on W2K host: SOSHostdll, SOSHost, SOSDebug, SOSRBuild
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
32
Sombrero Boot Sequence
• Start Sombrero boot loader on Target Alpha• Target contacts Host and requests modules• Host sends modules to Target• Boot loader on Target instantiates system
modules• Boot loader on Target starts Debugger on
W2K Host and transfers control to Sombrero system modules.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
33
Middle Level Architecture
• Middle Level Architecture was developed during implementation to solve issues that became apparent during implementation.
• New hardware design, Compiler support, system strategies, libraries, useful behavior.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
34
Compiler Support
• The GCC based Sombrero Compiler was designed to support IDC via Entry and Return points.
• Every Sombrero program has a class type to represent it.
• Entry points to other instantiations are accessed via proxy program class instantiations
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
35
Runtime Depiction of a proxy method invocationUser PCO
.
. SOSLinker.FindClass(name);
Continue after return. .
SOSLinker PCO . .
FindClass(void*, char name) { return cpointer; }
.
SOSLinker IMO .
Entry Point: FindClass .
Other Entry Points Return Point
User IMO .
Entry Points . .
Return Point
A
B
C D
SOSLinker is a proxy. FindClass is one of its methods: A) Call entry point; B) GP register is updated follow ed by Jump to actual code; C) Function return is to caller return point. Restore user OID to GP register; D) Final return using Auxiliary return.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
36
Some Middle Architecture
• Tail Switch – Allows RPLB to push a one-shot return permission on thread tail stack.
• Semaphores and Locks – Special advantage can be taken of a SASOS to make Semaphores and Locks globally visible without system calls or a lock manager.
• Interrupts can be designed to act directly as a signal to a blocking thread.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
37
More Middle Architecture
• Sombrero Communication Protocol – built over UDP/IP stack
• Library support for heaps and trees
• Intermediate Cache between emulated RPLB and PRAL
• Scheduler, Run time Linker and other system modules.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
38
My Contributions
• RPLB Protection Model• Carrier Protection Domains• Implicit Protection Domain Switching• Kernelless Architecture• Binding Hardware resources to PDs• Policy Programmable System Hierarchies• Entry/Return Point Mechanism• Tail Switch
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
39
• C++ Compiler Support for Return/Entry• Semaphores and Locks• Passive System Services• Signal Interrupts• Proposed Algorithms to Distribute Sombrero• Surrogate Control Blocks - Routing• Reduced Complexity • Did all the actual implementation work
My Contributions Cont.
08/22/2002 Alan Skousen Dissertation Defense - Arizona State University
40
Summary
• The ultimate goal of Sombrero is to provide:– a distributed client/server environment that is
inherently less complex and therefore inherently cheaper to manage and program.
– gets improved performance from the hardware.