1.3. minos kernel - lec.inf.ethz.ch · 4 k 20 19 1 m 0 virtual translationtablebase index...

1.3. MINOS KERNEL

89

System Startup

RPI (2) VC / firmware

• Initialize hardware

• Copy boot image to RAM

• Jump to OS boot image (Initializer)

90

OS Initializer (we!)

• Set stack registers for all processor modes

• Setup free heap list and module list

• Initialize MMU & page table

• Setup interrupt handlers & runtime vectors

• Start timer & enable interrupts

• Initialize other runtime data structures

• Initialize UARTs

• Initialize RAM disk

• Enter scheduling loop on OS

Note that the hardware starts completely unininitialized: no address translation, no caches, one thread, no interrupts enabled, gpios not configured etc., so there is a need for some bootstrapping in the beginning

Modular Kernel StructureThe Big Picture

91

Kernel Runtime

I/OFile System

Modules

Minos

"imports"

Command Interpreter andScheduler

Dynamic Linker:Module Loader

floating point emulation, memory allocation …

Kernel Logging etc.

Memory ManagementDevice Drivers

Modular Kernel StructureMinos Modules in More Detail

92

Kernel Runtime

I/OFile System

Tools

Kernel

PlatformUartMin FPE

Heaps

OFS

RamVolumes

Devices

SerialLog Uart

abstract block device& concrete FS methods abstract

characterdevice

Strings

Tools

Log

API

Module Loader Modules

OSMinos

User Interface

Command Interpreter andScheduler

Dynamic Linker:Module Loader

floatingpointemulation

memoryallocationunit

Builtins

64 bit ops

Kernel ModuleMODULE Kernel;

IMPORT SYSTEM, Platform;

TYPE …

(* types of runtime data structure *)

VAR …

(* global runtime data structures *)

PROCEDURE P* (…); (* exported *)

BEGIN …

(* low level routine *)

END …;

PROCEDURE … (…); (* internal *)

(* low level routine *)

BEGIN …

END …;

BEGIN …

(* runtime initialization *)

END Kernel.

93

PlatformFPE64BuiltinsHeapsUartMinKernelUtilsStringsDeviceUartLogSerialLogOFSIFSRamVolumesToolsModulesMinos

Memory Layout -- Objectives

As simple as possible

Support null-pointer checks via MMU

Classical Heap / Stack layout

1 MB pages

94

0x30000000

devices0x3F000000

0xFFFFFFFF

0x40000000

VC

0x0

kernel.img0x8000

physical address space

032k

768M

1G

4G

Memory Layout: big picture

95

0x3000'0000

devices0x3F00'0000

0xFFFF'FFFF (4G-1)

0x4000'0000

VC

0x0

physical

0x10'0000

0x10'8000

kernel.img0x8000


96

0x3000'0000

devices0x3F00'0000

0xFFFF'FFFF (4G-1)

0x4000'0000

VC

0x0

physical

0x10'0000

kernel.img0x10'8000

initial stack

kernel.imgfirst 1 MB


97

devices

VC

physical

0x30000000 (768 M)

devices0x3F000000

0xFFFFFFFF

0x40000000

kernel.img

VC

virtual

1MB0xFFF00000

kernel.img

1:1

1:1 (strongly ordered)

initial stack

kernel.imgfirst 1 MB

0x3000'0000

0x3F00'0000

0xFFFF'FFFF (4G-1)

0x4000'0000

0x0

0x10'0000

0x10'8000


98

0x30000000

devices0x3F000000

0xFFFFFFFF (4G-1)

0x40000000

VC

0x0

physical

0x30000000 (768 M)

devices0x3F000000

0xFFFFFFFF

0x40000000

kernel.img

VC

virtual

1MB0xFFF00000

0x100000

kernel.img0x108000

first 1 MB

1:1

1:1 (strongly ordered)

initial stack

unmappedkernel.imgfirst 1 MB

Virtual Memory Layout: Heap, Stack, RAMDisk

99

0x3000'0000

kernel.img

virtual

first 1 MB

0x0020'0000

stack (16M)

heap (493 MB)

unmapped

RAM Disk (256M)

0x2000'0000

0x000'00000

0x0010'0000

0x0010'8000

0x1F00'0000

0x1EF0'0000

Exception Table on ARM

100

Type Mode Address* return link(type)**

Reset Supervisor 0x0 undef

Undefined Instruction Undefined 0x4 next instr

SWI Supervisor 0x8 next instr

Prefetch Abort Abort 0xC aborted instr +4

Data Abort Abort 0x10 aborted instr +8

Interrupt (IRQ) IRQ 0x18 next instr +4

Fast Interrupt (FIQ) FIRQ 0x1C next instr +4

* alternatively High Vector Address = 0xFFFF0000 + adr (configurable)

** different numbers in Thumb instruction mode

Virtual Memory Layout: IRQ Table / MMU

101

UNDSP

IRQ Stack

0xFFFEFFFF

16KB

16KB

0xFFFF0000

MMU Table

LDR PC, [PC+0x18]

encoded as hex number 0x359ff018RESET

UNDEF

SWI

4B

4B

4B

4B

IRQ

FIQ

4B

4B

4B

4B

Prefetch Abort

Data Abort

Not assigned

RESET Adr

UNDEF Adr

SWI Adr

Prefetch Adr

4B

4B

4B

4B

0xFFFF0020

Data Abort Adr

Not assigned 4B

4B

IRQ Adr 4B

Fast IRQ Adr 4B

0xFFFEC000

0xFFFE8000

0xFFFE4000

0xFFFE3FFF

0xFFFE7FFF

0xFFFEBFFF

Pipeline!

ABORT Stack

0xFFF00000

911KB

16KB

virtual

1MB0xFFF0'0000

0x000'00000

64KB-64unused

Initialization: Kernel (body)

VAR lnk: PROCEDURE;

...

BEGIN (* do not enter any call here --> link register consistency ! *)

SYSTEM.PUT32(ADDRESSOF(lnk), SYSTEM.LNK());

SYSTEM.LDPSR( 0, Platform.SVCMode + Platform.FIQDisabled + Platform.IRQDisabled );

SYSTEM.SETSP(Platform.SVCSP);

SYSTEM.SETFP(Platform.SVCSP);

SYSTEM.LDPSR( 0, Platform.IRQMode + Platform.FIQDisabled + Platform.IRQDisabled );

SYSTEM.SETSP(Platform.IRQSP);

102

SYSTEM.LDPSR(u, src) [instruction MSR]

u = 0 PSR of current processor mode

u = 1 PSR of saved processor mode

src value to be loaded in PSR, an expression

Initialization: Kernel (body)

VAR lnk: PROCEDURE;

...

BEGIN (* do not enter any call here --> link register consistency ! *)

SYSTEM.PUT32(ADDRESSOF(lnk), SYSTEM.LNK());


SYSTEM.SETSP(Platform.SVCSP);

SYSTEM.SETFP(Platform.SVCSP);

SYSTEM.LDPSR( 0, Platform.IRQMode + Platform.FIQDisabled + Platform.IRQDisabled );

SYSTEM.SETSP(Platform.IRQSP);

103

SYSTEM.LDPSR(u, src) [instruction MSR]

u = 0 PSR of current processor mode

u = 1 PSR of saved processor mode

src value to be loaded in PSR, an expression

store link register globally – we are switching the stack!

disable IRQs, stay in SVC mode

new stack top for this mode

InitializationKernel (body)

SYSTEM.LDPSR( 0, Platform.FIQMode + Platform.FIQDisabled + Platform.IRQDisabled);

SYSTEM.SETSP(Platform.FIQSP);

...


InitMMU;

SetupInterruptVectors;

InitHandlers;

EnableIRQs;

OSTimer;

lnk

END Kernel.

104

continue execution (next body)

InitializationHeap

MODULE Heaps;

PROCEDURE New*( VAR p: ADDRESS; T: ADDRESS);(*allocate record, add tag field of 1 word with offset -4*)VAR size: SIZE; BEGIN

p := heap + 4; SYSTEM.PUT( heap, T ); (*adr of type descriptor (tag) to tagfield of new record*)SYSTEM.GET( T, size ); (*obtain record size from type descriptor*)heap := p + size;ASSERT(heap < heapEnd);

END New; ...

BEGINheapStart := Platform.HeapBase; heap := Platform.HeapBase; heapEnd := Platform.HeapEnd;

END Heaps.

105

heapBase

heapEnd

Stack

heap

Address Translation (1 MB pages)

106

31

Index Offset

20 19 01 M4 K

virtual

MMU lookup

31

PhysAdr[20..31] PhysAdr[19..0] = Offset

20 19 01 M4 K

physical

Translation Table

107

31

Index Offset

2019 01 M4 K

virtual

IndexTranslationTableBasetranslation table register

Translation table (4k entries)

Page Table EntriesPossible entries:

Invalid

Section

(Properties define memory type and sharing attributes)

Page table (2nd level: 4k or 64K pages), Supersection (16 MB pages)

(cf. ARM v7-A Section B.3)

31

Index Offset

20 19 01 M4 K

virtual

IndexTranslationTableBasetranslation table register

Translation table (4k entries)

Ignored 0 0

31 2 1 0

Section base addressTEX

[2:0]AP

[1:0]

SBZ

DomainIMP

C B 1 0

31 20 19 12 11 10 9 8 5 4 3 2 1 014

AP[2]

1516

0

18

S

(cf. ARM Reference Manual v7-A Section B.3)

Initialization: Platform.IdentityMapMemoryVAR pageTable-: POINTER {UNSAFE} TO ARRAY 4096 OF ADDRESS;

CONST NormalMemory = 10D02H; StronglyOrderedMemory = 0C02H;

PROCEDURE IdentityMapMemory-;VAR index: SIZE;BEGIN

pageTable := MMUPhysicalTableBase;FOR index := 0 TO MemorySize DIV MB - 1 DO

pageTable[index] := index * MB + NormalMemory END;FOR index := MemorySize DIV MB TO LEN (pageTable) - 1 DO

pageTable[index] := index * MB + StronglyOrderedMemory END;

END IdentityMapMemory;

BEGINIdentityMapMemory; EnableMemoryManagementUnit;

END Platform.

109

Accessing the System Control Processor

MCR (Move to Coprocessor from Arm Register)MRC (Move to Arm Register from Coprocessor)

110

ARM ProcessorSystemControlProcessor

MMU / MPU

coprocessor

number

opcodes

CPU

register

Coprocessor

registers

coprocessor

number 15

System Control Processor Registers

e.g.

"MCR P15, 0, R0, C2, C0, 0 ; set page table base address"

111

ARM Architecture

Reference Manual ARM v7, B4.1.154

ARM Architecture


ARM Architecture


ARM Architecture


ARM Architecture

Reference Manual ARM

v7A

B4.1.154

ARM Architecture


ARM Architecture


ARM Architecture


ARM Architecture


Cortex A7 MPCore

Technical Reference

Manual

4.2.3.

Cache

ARM processors can support several levels of cache

112

(from ARM Architecture Reference Manual ARM v7-A, Section A3.9.2

Cache Coherency

Caches largely invisible to the application programmer in normal operation.

Cache coherency breakdown:

113

memorycacheagent 1 agent 2reads or writes reads or writes

Cache CoherencyCaches largely invisible to the application programmer in normal operation.

Cache coherency breakdown:

Examples:

agent 1 = processor, agent 2 = DMA controller

agent 1 = processor (instruction cache), agent 2 = processor (data cache)

agent 1 = processor x, agent 2 = processor y

114

memorycacheagent 1 agent 2reads or writes reads or writes

relevant even for

single-core systems

Ensuring Cache Coherency

Not enabling caches in the system

RPI starts with MMU switched off and with caches disabled

Use hardware coherency mechanisms configurable for memory regions

E.g. strongly ordered memory for memory regions containing device registers

Use memory maintenance operations to manage cache coherency issues in software

E.g. when sharing information between processors

E.g. when changing instruction memory from data path

115

Memory types and attributes

Memory typeattribute

Shareableattribute

Otherattribute

Objective

Strongly-ordered Shareable Memory accesses to Strongly-orderedmemory occur in program order.

Device Shareable Memory mapped peripherals that areshared by several processors.

Non-Shareable Memory mapped peripherals that areused only by a single processor.

Normal Shareable Non-cacheableWrite-Through cacheableWrite-Back cacheable

Normal memory that is shared betweenseveral processors.

Non-Shareable Non-cacheableWrite-Through cacheableWrite-Back cacheable

Normal memory that is used only by asingle processor.

116

... we might revisit this topic again when discussing memory ordering in the context of a multi-core kernel.

Cache properties (ARM v7)

Data memory cache: Reads and Writes from one observer to the same physical location, also from different virtual addresses, always happen in program order

No memory barriers required

Instruction caches are never written to or read from by memory load / store operations

The guarantees from above do not necessarily apply to instruction cache, depending on the cache implementation

Changing memory attributes in the page table can require a cache maintenance operation

117

Cache Coherency Issues

Memory Location Update by a Processor not visible to other observers because

1. new updates are still in the writing's processor cache

2. cache of observer contains stale copy of the memory

Two explicit mechanisms to address this

Clean: updates made by a writer made visible to other observers that can access memory at the point to which the operation is performed.

Invalidate: A cache invalidate operation ensures that updates made visible by writers that access memory are made visible to an observer that controls the cache.

[precise definitions in ARM Architecture Manual v7, B 2.2.6]

118

Cache Maintenance

Instruction/Memory Cache can be selectively enabled / disabled

Cache manipulation: CRn = 7, TLB mainpulation: CRn = 8

119

clean data cache

invalidate data / instruction cache, by set / way, by VA, ...

Barriers

ISB -- Instruction Synchronization Barrier flushes the pipeline:all instructions following the ISB are fetched from cache or memory.

Important when code is written to data memory. Example: module loading.

Important when instruction memory changes, e.g. page table / TLB modifications

DMB -- Data Memory Barrier: synchronizes memory accesses and provides memory ordering.

DSB -- Data Synchronisation Barrier: data memory barrier that additionally synchronizes the execution stream with memory accesses.

to be revisited in the multicore context

120Examples in "Barrier Lithmus Tests and Cookbook", ARM, 26.11.2009

When DMB?

DMB ensures that all explicit memory accesses before DMB completebefore explicit memory accesses after DMB

E.g. required:

before writing to MMU Control register

in spinlocks: to prevent that processor reads memory before lock acquired or writes memory after lock released

121

memaccess

memaccess

memaccess

memaccess

DMB

can reorder can reorder

When DSB?

DSB completes only when all memory access instructions before DSB havecompleted

E.g. required:

after page table change

before waking up processes waiting on mutex (WFE / SEV)

122

memaccess

memaccess

memaccess

memaccess

DSBcan reorder

can reordermust complete

When ISB?

ISB completes only when all instructions before ISB have been completedand the pipeline is flushed and branch brediction logic is updated

E.g. required:

TLB refresh, MMU reload

Self modifying code

123

ISB

must complete

Initialization: Kernel.InitMMU

(* Init the memory management unit *)PROCEDURE InitMMU;CONST

Platform.DisableMemoryManagementUnit;Platform.pageTable[0] := 0; (* unmap page *)Platform.pageTable[4095] := 0*MB + StronglyOrderedMemory; Platform.EnableMemoryManagementUnit;

END InitMMU;

124

Enable Memory Management Unit

1. Set page table base address (register c2 c0)2. Enable full access to domain 0 (register c3 c0)3. set memory protection, data and unified cache, branch predicition,

instruction cache and high vector bits in system control register4. Flush And Invalidate DCache lengthy code 5. InvalidateTLB6. InvalidateICache

125

Example (invalidate TLB)

MCR p15, 0, R0, c8, c7, 0 ; invalidate I+D TLBDSBISB

1.3. minos kernel - lec.inf.ethz.ch · 4 k 20 19 1 m 0 virtual translationtablebase index...

Documents