os project 2011 documentation

8/6/2019 OS Project 2011 Documentation

1/29

Faculty of Computer and Information Sciences

Ain Shams University

Third Year Second Semester

2010 - 2011

Dr. Howaida Abdel Al Fattah


2/29

2

Contents

INTRODUCTION .......................................................................................................................... 4Whats new?! ........................................................................................................................................................4Project Specifications .............................................................................................................................................. 5New Concepts........................................................................................................................................................6

The Working Sets ................................................................................................................................................ 6The Page File ...................................................................................................................................................... 6Page Buffering .................................................................................................................................................... 7

OVERALL VIEW........................................................................................................................... 8DETAILS ..................................................................................................................................... 9env_create( )

.........................................................................................................................................................

9Fault Handler .........................................................................................................................................................9env_free( ) .......................................................................................................................................................... 13Dynamic allocation/free ........................................................................................................................................ 13BONUSES .................................................................................................................................. 15First: Freeing RAM when it's Full ............................................................................................................................. 15Second: Buffering Page Tables ................................................................................................................................ 15CHALLENGE!!............................................................................................................................ 15HOW TO TEST YOUR PROJECT .................................................................................................. 16FIRST: Testing Each Part......................................................................................................................................... 16

Testing Page & Table Placement:.......................................................................................................................... 16Testing Page Replacement: ................................................................................................................................. 16Testing Table Replacement:

................................................................................................................................

16Testing Page & Table Replacement: ...................................................................................................................... 16Testing Dynamic Allocation & Free: ...................................................................................................................... 17

SECOND: Testing Whole Project .............................................................................................................................. 17Scenario 1: Env_Free.......................................................................................................................................... 17Scenario 2: Small Programs ................................................................................................................................. 17Scenario 3: QuickSort Program ............................................................................................................................ 17Scenario 4: QuickSort_freeHeap Program .............................................................................................................. 18

APPENDIX I: PAGE FILE HELPER FUNCTIONS............................................................................

19Add a new environment page to the page file ........................................................................................................... 19


3/29

3

Read an environment page from the page file to the main memory.............................................................................. 19Update certain environment page in the page file by contents from the main memory ...................................................20Read an environment table from the page file to the main memory ............................................................................. 20Write a certain environment table to the page file..................................................................................................... 21Remove an existing environment page/table from the page file

..................................................................................

21APPENDIX II: WORKING SET STRUCTURE & HELPER FUNCTIONS............................................. 22Working Set Structure ........................................................................................................................................... 22Working Set Functions .......................................................................................................................................... 22

inline uint32 env_page_ws_get_size(struct Env *e) ................................................................................................. 22inline uint32 env_table_ws_get_size(struct Env *e) ................................................................................................. 22inline void env_page_ws_invalidate(struct Env* e, uint32 virtual_address) .................................................................. 23inline void env_table_ws_invalidate(struct Env* e, uint32 virtual_address) .................................................................. 23inline void env_page_ws_set_entry(struct Env* e, uint32 entry_index, uint32 virtual_address)

.......................................

23inline void env_table_ws_set_entry(struct Env* e, uint32 entry_index, uint32 virtual_address) ....................................... 23inline void env_page_ws_clear_entry(struct Env* e, uint32 entry_index)..................................................................... 23inline void env_table_ws_clear_entry(struct Env* e, uint32 entry_index) .................................................................... 23inline uint32 env_page_ws_get_virtual_address(struct Env* e, uint32 entry_index) ......................................................23inline uint32 env_table_ws_get_virtual_address(struct Env* e, uint32 entry_index) ......................................................23inline uint32 env_page_ws_is_entry_empty(struct Env* e, uint32 entry_index) ............................................................ 24inline uint32 env_table_ws_is_entry_empty(struct Env* e, uint32 entry_index) ...........................................................24inline voidenv_page_ws_print(struct Env* e)........................................................................................................ 24inline voidenv_table_ws_print(struct Env* e) ....................................................................................................... 24

APPENDIX III: FUNCTIONS FOR MANIPULATING PERMISSIONS IN PAGE TABLES AND

DIRECTORY.............................................................................................................................. 25inline void pt_set_page_permissions(struct Env* ptr_env, uint32 virtual_address, uint32 permissions_to_set, uint32

permissions_to_clear) ........................................................................................................................................ 25inline uint32 pt_get_page_permissions(struct Env* ptr_env, uint32 virtual_address ) ....................................................25inline uint32 pd_is_table_used(uint32* ptr_pgdir, uint32 virtual_address) .................................................................. 26inline void pd_set_table_unused(uint32* ptr_pgdir, uint32 virtual_address) ................................................................ 26

APPENDIX IV: PAGE BUFFERING ............................................................................................... 27Structures ........................................................................................................................................................... 27Functions ............................................................................................................................................................ 27

inline void bufferList_add_page(struct Linked_List* bufferList,struct Frame_Info *ptr_frame_info) .................................. 27inline void bufferlist_remove_page(struct Linked_List* bufferList, struct Frame_Info *ptr_frame_info) ............................ 27

APPENDIX V: BASIC AND HELPER MEMORY MANAGEMENT FUNCTIONS .................................. 28Basic Functions .................................................................................................................................................... 28Helpers Functions................................................................................................................................................. 28


4/29

4

Introduction

Whats new?!

Previously:all segments of the program binary plus the stack page should be loaded in the main memory. If theres

no enough memory, the program will not be loaded. See the following figure:

But, wait a minute

This means that you may not be able to run one of your programs if theres no enough main memory for it!! This is

not the case in real OS!! In windows for example, you can run any program you want regardless the size of main

memory do you know WHY?!

YES it uses part ofsecondary memory as a virtual memory. So, the loading of the program will be distributed

between the main memory and secondary memory.

NOW in the project: when loading a program, only part of it will be loaded in the main memory while the whole

program will be loaded on the secondary memory (H.D.D.). See the following figure:

This means that a Page Fault can occur during the program run. (i.e. an exception thrown by the processor to

indicate that a page is not present in main memory). The kernel should handle it by loading the faulted page back to

the main memory from the secondary memory (H.D.D.).

BINRequired Tables

All

Pages

Set of pages

env_create( )

BIN

env_create( )

All

Ifno enough memory, then

dont load it

Group Members: 3-6

Group Registration: you shouldregister your group heredue to FRI 20/5/2011

Due to Date: Week #13 starting from SUN29/5/2011 to THU 2/6/2011. (Subject to change according to lab exam

schedule)
https://spreadsheets1.google.com/spreadsheet/viewform?hl=en&hl=en&formkey=dEUtWC1lRXlsQUY3NGJJVHRDa29mREE6MA#gid=0https://spreadsheets1.google.com/spreadsheet/viewform?hl=en&hl=en&formkey=dEUtWC1lRXlsQUY3NGJJVHRDa29mREE6MA#gid=0https://spreadsheets1.google.com/spreadsheet/viewform?hl=en&hl=en&formkey=dEUtWC1lRXlsQUY3NGJJVHRDa29mREE6MA#gid=0https://spreadsheets1.google.com/spreadsheet/viewform?hl=en&hl=en&formkey=dEUtWC1lRXlsQUY3NGJJVHRDa29mREE6MA#gid=0


5/29

5

Project Specifications

In the light of the studied memory management system, you are required add new features for your FOS memory

manager, these new features are:

For loading only part of a program in main memory, we use the working set concept; the working set is the set of all

pages (or tables) loaded in main memory of the program at run time at any instant of time.

Each program environment is modified to hold its working sets information. The working sets are a FIXED size array

of virtual addresses corresponding to pages that are loaded in main memory.

The virtual space of any loaded user application is as described in lab 6, see figure 1.

User Stack

..

VirtualMemory

256 MB

4 GB

KERNEL_BASE

Free Memory

USER_LIMIT

USER_TOP

8 MBFOS stackKERNEL_STACK_TOP

FOS Kernel

FOS stack

< 1 MB

User Heap

Memory 2 GB

1.0 GB

User Code +Data

0

USER_HEAP_START

USER_HEAP_MAX

USTACKTOP

Figure 1: Virtual space layout of single user loaded program

There are two important concepts you need to understand in order to implement the project; these concepts are the

Working Set and Page File.

1- Load multiple user programsthat are partially loaded in main memory and fully loadedin secondary memory (H.D.D.)

2- Handle page faults during execution3- Exit any of the loaded programs: (i.e. remove it from the main and secondary memory)4- Allow user program to dynamically allocate and free memory space at run-time (i.e.

mallocandfreeMem functions)


6/29

6

New Concepts

The Working Sets

We have previously defined the working set as the set of all pages (or tables) loaded in main memory of the program

at run time which is implemented as a FIXED size array of virtual addresses corresponding to pages that are loaded in

main memory.

SeeAppendix IIfor description about the working sets data structure and helper functions.

The maximum allowed number of pages (or tables) for any running user program in memory is defined inside FOS1,

let's say Npages (or M tables), when the program needs memory (either when the program is being loaded or by

dynamic allocation during the program run) FOS must allow maximum Npages (or M tables) for the program in main

memory.

So, when page fault occur during the program execution, the page (or table) should be loaded from the secondary

memory (the PAGE FILE as described later) to the WORKING SET of the program. If the working set is complete, then

one of the program environment pages (or tables) from the working set should be replaced with the new page (or

table). This is called LOCAL replacement since each program should replace one of its own loaded pages (or tables).

This means that our working sets are FIXED size with LOCAL replacement facility.

FOS must maintain the working sets during the run time of the program, when new pages (or tables) from PAGE FILE

are loaded to main memory, the working set must be updated.

Therefore the working sets of any loaded program must always contain the correct information about the pages (or

tables) loaded in main memory.

The Page File

The page file is an area in the secondary memory (H.D.D.) that is used for saving and loading programs pages (or

tables) during runtime. Thus, for each running program, there is a storage space in the page file forALL pages

needed by the program, this means that user code, data, stack and heap sections areALL written in page file.

(Remember that not all these pages are in main memory, only the working set).

You might wonder why we need to keep all pages of the program in secondary memory during run!!

The reason for this is to have a copy of each page in the page file. So, we dont need to write back each swapped out

page to the page file. Only MODIFIED pages are written back to the page file.

But wait a minute

Did we have a file manager in FOS?!!

NO!

Dont panic, we wrote some helper functions for you that allow us to deal with the page file. These functions provide

the following facilities:

1 defined in inc/environment_definitions.h with name "PAGE_WS_MAX_SIZE" and "TABLE_WS_MAX_SIZE"

The initialization of the working set by the set of pages during the program loading is already implemented

for you in "env_create()", andyou will be responsible for maintaining the working set during the run time of

the program.


7/29

7

1- Add a new environment empty page to the page file.2- Read an environment page (or a table) from the page file to the main memory.3- Update certain environment page (or WRITE a table) in the page file from the main memory.4- Remove an existing environment page (or a table) from the page file.

SeeAppendix Ifor description about these helper functions.

Page Buffering

An interesting strategy that can improve paging performance is page buffering. A replaced page is not lost but rather

is assigned to one of two lists:

1. the free frame list if the page has not been modified, or2. the modified page list if it has.

Note that the page is not physically moved about in main memory; instead, the PRESENT bit for this page is set to 0

and its "Frame_Info" is placed in either the free or modified list.

When a page is to be read in, the frame at the head of the free frame list is used, destroying the page that was there.

When an unmodified page is to be replaced, it remains in memory and its "Frame_info" is added to the tail of thefree frame list. Similarly, when a modified page is to be written out and replaced, its "Frame_info" is added to the

tail of the modified page list.

The pages on the modified list can periodically be written out in batches when their count reaches

"MAX_MODIFIED_LIST_COUNT2" and moved to the unmodified list. A page on the unmodified list is either

reclaimed if it is referenced, or lost when its frame is assigned to another page.

There are two important aspects of these maneuvers

1. The page to be replaced remains in memory. Thus if the process references that page, it is returned to theworking set of that process at little cost (by setting its PRESENT bit to 1).

2. Modified pages are written out in clusters rather than one at a time. This significantly reduces the number ofI/O operations and therefore the amount of disk access time.

2 defined in inc/environment_definitions.h

The loading ofALL program binary segments plus the stack page to the page file is already implemented for

you in "env_create()", you will need to maintain the page file in the rest of the project.


8/29

8

Overall ViewThe following figure shows an overall view of all project components together with the interaction between them.

The components marked with (*) should be written by you. The unmarked components are already implemented.

Figure 2: Interaction among components in project

Exiting

(Unloading)

Handling Page Faults (Fault Handler) *Page Placement

NotBuffered

Dynamic Alloc/Free

Running

Fault when accessing page (or

table) not in main memory

* Remove WorkingSets of Pages and Page

Tables + Directory

Remove all pages

and tables (if any)

from page file

freeHeap( ) *malloc( ) *

Nothing

Created in

Memory

ALL Pages

Added to

Page File

Working Sets

Pages + Tables in

Given Range

ALL Pages

+ Tables

env_run()env_free()

Page Re-placement

Stack

Add Page

IfNew stack page then add it to page file

env_create( )

BINWorking Sets

All

Program

Loaded

Pages Working Set in Memory

Tables Working Set in Memory

All pages in Page File

No tables initially on Page File

Handling Table Faults (Fault Handler) *Table Placement Table Re-placement

NEW

Exist

NEW

Exist

malloc( ) / freeHeap( )

VictimBuffered f.f.l.

mod.l.

f.f.l.

mod.l.

Not

Buffered

Buffered f.f.l.mod.l.

Write Pages

IfModified Buffer reaches Max then write it into page file

mod.l.

f.f.l.Concatenate lists

Victim

f.f.l.

mod.l.

Buffered Frames

* Remove BufferedFrames from both Lists

(make them not buffered)


9/29

9

Details

env_create( )

in kern/user_environment.c (no student code needed, just understand)

After env_create() is executed to load a program binary, the binary will partially loaded in main memory (part of

segments + stack page) and FULLY loaded in page file (all segments + stack page)

Figure 3: View of working set after loading a program binary with env_create(). [(1): * this asterisk means that

Used Bit is set to 1 in page table for this virtual address, (2) WS_Last_index: the WS index to start iterating

from (used in clock algorithm)]

Fault Handler

In function fault_handler() , " kern/trap.c"

Fault: is an exception thrown by the processor (MMU) to indicate that:o A pagecant be accessed due to either its not present in the main memory ORo A page table is not present in the main memory. (see the following figure)

BIN

Segment 1, page 1

Segment 1, page 2

Segment 1, page 3

Segment 2, page 1

Segment 2, page 2

201000 *(1)

202000 *

203000 *

801000 *

Program Binary

Segments and pages

Working Set after

loading

(WS_MAX_SIZE = 5)

Loaded in virtual address 201000



NOT Loaded in

main memory

USTAKTOPPAGESIZE *

First Stack Page


WS_Last_index (2)


10/29

1

Figure 4: Fault types (table fault and page fault)

You should handle any fault in FOS kernel by:a. Allocate a new page (or a new page table) for this faulted page in the main memoryb. Loading the faulted page (or its page table) back to main memory from page file (secondary

memory) if the page (or the table) exists in page file. Otherwise, add it to the page file. You can handle the page (or page table) fault in function fault_handler() in trap.c, as follows

LinearAddress

Page Directory

Dir Entry

Page Table

Table Entry

CR3

32 bits

P=1

LinearAddress

Page Directory

Dir Entry

CR332 bits

4 KB

P=0

TABLEFault

10 bits10 bits 12 bits

DIR PAGE OFFSET

10 bits10 bits 12 bits

DIR PAGE OFFSET

P=0

PAGEFault

CASE1: Table not exist CASE2: Page not exist

If the directory entry of the faulted address is NOT PRESENT then

Handle Table Fault

Else

Handle Page Fault


11/29

Table fault handler should be written as follows: if the size of the table working set < TABLE_WS_MAX_SIZEthen do

Placement:

1.save the directory entry of the faulted address to use it in step 32.allocate a new frame for the table and link it with the directory entry of the faulted address(create_page_table() function will do this for you)3.if saved directory entry = NULL then

1. This is a new table (i.e. not exist in page file)2. So, do nothing here.

4.else1. The table of the faulted address exists in the page file2. So, read it from page file to memory3. Remove it from page file as we will never read it again from page file.

5.update the table working set else, do

Replacement:

1.implement clock algorithm to find the virtual address of the victim table from table WS toreplace (refer toAPPENDIX II)

2.write the victim table in the page file3.remove the victim table by

1. unlink it from the directory entry by setting PRESENT BIT to 0 ( IMPORTANT NOTE:Dont set the whole directory entry to 0 (NULL), this means that the victim table

doesnt exist in both memory and page file, which is not the case here)

2. freeing its frame3. and remove it from the table working set

4.Apply Placement steps here


12/29

2

Page fault handler should work as follows:

Refer to helper functions to deal with flags of Page Table (or page Directory) entries (Appendix III) Refer to the linked lists, basic and helper memory manager functions (Appendix V)

o if the size of the page working set < PAGE_WS_MAX_SIZEthen doPlacement:

1. check if required page is buffered (seeAPPENDIX III), if true, do the following:o

Just reclaim the page by setting present bit to 1, and buffered bit to 0, and set itsframe_info->isBufferedto 0. (seeAPPENDIX III, and IV)

o remove its Frame_Info from: "modified_frame_list" if the page was modified(modified bit = 1), or from "free_frame_list" if the page was not modified (modified

bit = 0) (seeAPPENDIX III and IV)

elseo allocate and map a frame for the faulted pageo read the faulted page from page file to memory

If the page does not exist on page file, then CHECK if it is a stack page. If it is a

new stack page, add a new empty page with this faulted address to page file

(refer toAppendix I and II)

3. update the page working set else, do

Replacement:

4. implement clock algorithm to find the victim virtual address from page working set5. for the victim page:

Prepare its Frame_Info by flagging it as buffered, setting the environment and thevictim virtual address inside this Frame_Info Set the buffered bit to 1 in the victim page table Set the present bit to 0 in the victim page table

6. If the victim page was not modified, then: Add the victim frame to "free_frame_list" at the tail (seeAPPENDIX IV)

Else Add the victim frame to "modified_frame_list" at the tail (seeAPPENDIX IV)

Check the count of modified pages in "modified_frame_list", if it reachesMAX_MODIFIED_LIST_COUNT, reset the counter and do the following:

1. for each frame in the "modified_frame_list"a. update its page in page file (seeAPPENDIX I),b. set the modified bit of its page in its page table to 0

2. Concatenate the "modified_frame_list" to "free_frame_list" (by doing this,the modified list will be cleared) (see helper functions inAPPENDIX V)

2.Apply Placement steps described before


13/29

3

env_free( )

in kern/user_environment.c

For each BUFFERED page in the User Spaceo If the buffered page is MODIFIED then

Remove its "Frame_Info" from the "modified_frame_list" Flag its "Frame_Info" as NOT Buffered (setting isBuffered to 0) Add it to the "free_frame_list"

o Else Just flag its "Frame_Info" as NOT Buffered (setting isBuffered to 0)

Then, you should free :o The pages in the page working seto The page tables the table working seto The directory table

Frees all pages and tables from page file, this code is alreadywritten for youDynamic allocation/free

Functions malloc() and freeHeap() in "lib/malloc.c"

You will need to implement a dynamic allocation function for your user programs to make runtimeallocations, also a free Heap function to free ALL the heap

Your should implement function malloc() (allocates user memory) and freeHeap()(frees all userheap) functions in lib/malloc.c

The user should call the kernel to allocate/free memory for it. But wait a minute!! remember that the user code is not the kernel (i.e. when a user code is

executing, the processor runs user mode (less privileged mode), and to execute kernel code the

processor need to go from user mode to kernel mode)

The switch from user mode to kernel mode is done by a software interruptcalled System Call. All you need to do is to call a function prefixed sys_ from your user code to call the kernel code

that does the job, (e.g. from user function malloc(), call sys_allocateMem() which then will call

kernel function allocateMem() to allocate memory for the user) as shown in figure:


14/29

4

Figure 5: Sequence diagram of dynamic allocation using malloc()

Your kernel code will be in allocateMem() and freeMem() functions in memory_manager.c In allocateMem(), all pages you allocate will not be added to the working set (not exist in main

memory), but they should added to the page file, so that when this page is accessed, a page fault

will load it to memory.

In freeMem(), you will need to:o For each BUFFERED page in the Given Range

If the buffered page is MODIFIED then Remove its "Frame_Info" from the "modified_frame_list" Flag its "Frame_Info" as NOT Buffered (setting isBuffered to 0) Add it to the "free_frame_list"

Else Just flag its "Frame_Info" as NOT Buffered (setting isBuffered to 0)

Set its entry in the page table to NULLo

Remove ONLY working set pages and working set page tables that are located in the givenuser virtual address range. (REMEMBER to update the working sets after removing)

o Remove all pages and all tables in the given range from the page file.

Switch to

Kernel mode

Call

Switch back to

User mode

Call *USERMalloc

KERNELallocateMem *

S/W INTsys_allocateMem()

NOTE: You should do the (*) operations only


15/29

5

BONUSES

First: Freeing RAM when it's Full

When allocating new frame, if there's no free frame, then you should:

1- Free some/all frames from the modified page list and add them to the free frame list2- If the modified list is empty, then you should free at least 1 frame from the user working set by applying the

clock algorithm

Second: Buffering Page Tables

Sometimes during the execution, the program needs a large number of page tables to be loaded in memory at the

same time to use them all. This can lead to large number of TABLE FAULTS.

Since the page tables are always modified (due to continuous changes in status bits), any fault on it cause it to be

written back in the page file (H.D.D) which consume a lot of time and cause the CPU to be idle.

In order to solve this problem, we can apply the BUFFERING concept on the page tables just as we apply it for pages.Your task is to define the required variables in your code and maintain them during the whole code in order to

manage the PAGE TABLE BUFFERING.

CHALLENGE!!In fault_handler() function you are required to write TABLE replacement algorithm using NORMAL CLOCK, normal

clock works with 1 used bitof the table, however, it does not achieve good level of fairness when choosing a victim!.

and it will generate a lot of table faults due to

1- It doesnt care about the pages that each page table refer to2- It deal with all tables as a same regardless how many times each table is used

These two issues may lead to remove table that is either frequently used or contains pages that are currently

used.

Can you write a better algorithm?

Challenge Description

Develop a table replacement algorithm that minimizes the number of table faults occur when testing FOS project.

Every effort is welcomed, search the books, ask faculty staff, search the

internet or even invent your own algorithm.


16/29

6

Testing

How to test your project

To test your implementation, you need to run each of the following test programs. Each program should

test certain part from the project. In addition, after completing all parts, you can test the whole project

using quickSort program. User programs found in "user/" folder:

FIRST: Testing Each Part

If a test succeeds, it will print and success message on the screen, otherwise the test will panic at the error

line and display it on the screen

Testing Page & Table Placement:

test_ptf_stack.c: tests page faults on stack + page & table placement. You need to initialize the Working

set sizes as follows:

PAGE_WS_MAX_SIZE = 20

TABLE_WS_MAX_SIZE = 20

Testing Page Replacement:

test_pf_clock.c : to test your implementation for the clock algorithm for page replacement. You need to

initialize the Working set sizes as follows:



test_pf_stack.c: to test your implementation for handling the new stack pages and the modified pages.

You need to initialize the Working set sizes as follows:



Testing Table Replacement:

test_table_replacement.c, test_table_replacement_2.c:tests page faults on stack + table replacement

(clock algorithm and dealing with page file for writing and reading tables)



Testing Page & Table Replacement:

test_ptf_stack.c : tests page faults on stack + page/table replacement.




17/29

7

Testing Dynamic Allocation & Free:

test_malloc.c : to test your implementation for the malloc() and allocateMem() functions. You need to




test_freeHeap.c : to test your implementation for the freeHeap() and freeMem() functions. You need to




SECOND: Testing Whole Project

You should run each of the following scenarios successfully

Scenario 1: Env_Free1. Make sure that PAGE_WS_MAX_SIZE = 10, and TABLE_WS_MAX_SIZE = 202. run command meminfo

denote free frames as FF1

3. Run user program fos_add4. kill fos_add5. run command meminfo

make sure that free frames = FF1

Scenario 2: Small Programs

All other programs in the "user/" folder should be run successfully with the following Working set sizes



Scenario 3: QuickSort Program

1. Make sure that PAGE_WS_MAX_SIZE = 4, TABLE_WS_MAX_SIZE = 32. Run user program quicksort_heap and test it according to the following steps:

Number of Elements = 3,000

Initialization method : Ascending

Do you want to repeat (y/n) : y

3. The program should sort the array successfully


18/29

8

Scenario 4: QuickSort_freeHeap Program

1. Make sure that PAGE_WS_MAX_SIZE = 500, and TABLE_WS_MAX_SIZE = 202. run user program fos_helloWorld3. run user program fos_add4. run user program quicksort_heap_freeHeap and test it according to the following steps:


Initialization method : Semi randomDo you want to repeat (y/n) : y


Initialization method : Semi random



Initialization method : Ascending



Initialization method : Semi randomDo you want to repeat (y/n) : n

At each step, the program should sort the array successfully and DONT panic

Enjoy your OS

GOOD LUCK


19/29

9

APPENDICES

APPENDIX I: Page File Helper FunctionsThere are some functions that help you work with the page file. They are declared and defined in

kern/file_manager.h and kern/file_manager.c respectively. Following is brief description about those functions:

Add a new environment page to the page file

Description:

Add a new environment page with the given virtual address to the page file and initialize it by zeros.

Function declaration:

int pf_add_empty_env_page( struct Env* ptr_env, uint32 virtual_address);

Parameters:

ptr_env: pointer to the environment that you want to add the page for it.

virtual_address: the virtual address of the page to be added.

Return value:

= 0: the page is added successfully to the page file.

= E_NO_PAGE_FILE_SPACE: the page file is full, cant add any more pages to it.

Example:

In dynamic allocation: let for example we want to dynamically allocate 1 page at the beginning of the

heap (i.e. at address USER_HEAP_START), so we need to add this page to the page file as follows:

int ret = pf_add_empty_env_page(ptr_env, USER_HEAP_START);if (ret == E_NO_PAGE_FILE_SPACE)

panic(ERROR: No enough virtual space on the page file);

Read an environment page from the page file to the main memory

Description:

Read an existing environment page at the given virtual address from the page file.


int pf_read_env_page(struct Env* ptr_env, void *virtual_address);

Parameters:

ptr_env: pointer to the environment that you want to read its page from the page file.

virtual_address: the virtual address of the page to be read.

Return value:

= 0: the page is read successfully to the given virtual address of the given environment.

= E_PAGE_NOT_EXIST_IN_PF: the page doesnt exist on the page file (i.e. no one added it before to

the page file).

Example:

In placement steps: let for example there is a page fault occur at certain virtual address, then, we

want to read it from the page file and place it in the main memory at the faulted virtual address as

follows:


20/29

21

int ret = pf_read_env_page(ptr_env, fault_va);

if (ret == E_PAGE_NOT_EXIST_IN_PF){

...}

Update certain environment page in the page file by contents from the main memory

Description:

Updates an existing page in the page file by the given frame in memory


int pf_update_env_page(struct Env* ptr_env, void *virtual_address, structFrame_Info* modified_page_frame_info));

Parameters:

ptr_env: pointer to the environment that you want to update its page on the page file.

virtual_address: the virtual address of the page to be updated.

modified_page_frame_info: the Frame_Info* related to this page.

Return value:

= 0: the page is updated successfully on the page file.

= E_PAGE_NOT_EXIST_IN_PF: the page to be updated doesnt exist on the page file (i.e. no one add

it before to the page file).

Example:

structFrame_Info *ptr_frame_info = get_frame_info();

int ret = pf_update_env_page(environment, virtual_address, ptr_frame_info);

Read an environment table from the page file to the main memory

Description:

Read an existing environment table at the given virtual address from the page file memory.


intpf_read_env_table(struct Env* ptr_env, uint32 virtual_address, uint32*

tableKVirtualAddress);

Parameters:

ptr_env: pointer to the environment that you want to read its page from the page file.

virtual_address: the first user virtual address of the table (in userspace)

tableKVirtualAddress: the kernel virtual address of the table needed to be read from page file

Return value:

= 0: the table is read successfully to the given table virtual address.

= E_TABLE_NOT_EXIST_IN_PF: the table doesnt exist on the page file

Example:

Assume for example that there is a table fault at certain virtual address, then, we want to read table

from the page file and place it in the working set in main memory:

uint32* ptr_table;create_page_table(faulted_Env->env_pgdir, fault_va, &ptr_table);

pf_read_env_table(ptr_env, fault_va, ptr_table);


21/29

2

Write a certain environment table to the page file

Description:

Write an environment page table to the page file.


intpf_write_env_table(struct Env* ptr_env, void *virtual_address, uint32*tableKVirtualAddress);

Parameters:

ptr_env: pointer to the environment that you want to write its page to the page file.

virtual_address: the first user virtual address of the table (in userspace)

tableKVirtualAddress: the kernel virtual address of the table needed to be written to page file

Return value:

= 0: the table is written successfully to the page file

= E_NO_PAGE_FILE_SPACE: the page file is full, cant add any more pages (tables) to it.

Example:

//write a table to page fileuint32* ptr_table;get_page_table(faulted_env->env_pgdir, (void*) virtual_address,&ptr_table(;int success = pf_write_env_table(curenv, virtual_address, ptr_table);

Remove an existing environment page/table from the page file

Description:

Remove an existing environment page/table at the given virtual address from the page file.


void pf_remove_env_page(struct Env* ptr_env, uint32 *virtual_address);

void pf_remove_env_table(struct Env* ptr_env, uint32 virtual_address);

Parameters:

ptr_env: pointer to the environment that you want to remove its page (or table) on the page file.

virtual_address (page): the virtual address of the page to be removed.

virtual_address (table): the first user virtual address of the table (in userspace)

Example:

Lets assume for example we want to free 1 page at the beginning of the heap (i.e. at address

USER_HEAP_START), so we need to remove this page from the page file as follows:

pf_remove_env_page(ptr_env, USER_HEAP_START);

Or that we want to free 1 table at the beginning of the user heap (i.e. at address

USER_HEAP_START), so we need to remove this table from the page file as follows:

pf_remove_env_table(ptr_env, USER_HEAP_START);


22/29

22

APPENDIX II: Working Set Structure & Helper Functions

Working Set Structure

As stated before, each environment has two working sets with FIXED size that hold the virtual addresses

corresponding to the pages/tables that are loaded in main memory.

The working set is defined as a fixed size array of virtual addresses inside the environment structure structEnvlocated in inc/environment_definitions.h.

Figure 6: Definitions of 2 working sets & their indices inside struct Env

Working Set Functions

These functions are declared and defined in kern/memory_manager.h and kern/ memory_manager.c

respectively. Following are brief description about those functions:

inline uint32 env_page_ws_get_size(structEnv *e)inline uint32 env_table_ws_get_size(structEnv *e)

Description:

Counts the pages/tables loaded in main memory of a given environment

Parameters:

e: pointer to the environment that you want to count its working set size

Return value:

Number of pages/tables loaded in main memory for environment e,(i.e. e working set size)

#define PAGE_WS_MAX_SIZE 5#define TABLE_WS_MAX_SIZE 5

struct WorkingSetElement {uint32 virtual_address; // the virtual address of the pageuint8 empty; // if empty = 0, the entry is valid, if empty = 1 , entry is

empty or free};

struct Env {...//page working set management, array of PAGE_WS_MAX_SIZE entriesstruct WorkingSetElement pageworkingSet[PAGE_WS_MAX_SIZE];

//table working set management, array of TABLE_WS_MAX_SIZE entriesstruct WorkingSetElement tableworkingSet[TABLE_WS_MAX_SIZE];

// used for modified clock algorithm, the next item (page) pointeruint32 page_WS_last_index;

// used for clock algorithm, the next item (table) pointeruint32 table_WS_last_index;};


23/29

23

inlinevoid env_page_ws_invalidate(structEnv* e, uint32 virtual_address)

inlinevoid env_table_ws_invalidate(structEnv* e, uint32 virtual_address)

Description:

Removes page/table entry at the given virtual address from the working set ofe

Parameters:

e: pointer to an environmentvirtual_address: the virtual address to remove from working set

inlinevoid env_page_ws_set_entry(structEnv* e, uint32 entry_index, uint32 virtual_address)

inlinevoid env_table_ws_set_entry(structEnv* e, uint32 entry_index, uint32 virtual_address)

Description:

Sets the entry number entry_index in eworking set to given virtual address after ROUNDING it

DOWN to the start of page (or page table)

Parameters:

e: pointer to an environment

entry_index: the working set entry index to set the given virtual address

virtual_address: the virtual address to set

inlinevoid env_page_ws_clear_entry(structEnv* e, uint32 entry_index)

inlinevoid env_table_ws_clear_entry(structEnv* e, uint32 entry_index)

Description:

Clears (make empty) the entry at entry_index in eworking set.

Parameters:


entry_index: working set entry index

inline uint32 env_page_ws_get_virtual_address(structEnv* e, uint32 entry_index)

inline uint32 env_table_ws_get_virtual_address(structEnv* e, uint32 entry_index)

Description:

Returns the virtual address of the page at entry entry_index in environment eworking set

Parameters:



Return value:

The virtual address ofthe page at entry entry_index in environment eworking set


24/29

24

inline uint32 env_page_ws_is_entry_empty(structEnv* e, uint32 entry_index)

inline uint32 env_table_ws_is_entry_empty(structEnv* e, uint32 entry_index)

Description:

Returns a value indicating whether the entry at entry_index in environment e working set is

empty

Parameters:



Return value:

0: if the working set entry at entry _index is NOT empty

1: if the working set entry at entry _index is empty

inlinevoidenv_page_ws_print(structEnv* e)

inline voidenv_table_ws_print(structEnv* e)

Description:

Print the page (or table) working set together with the used and modified bits. It also show where

the last_WS_indexof the working set is point to.

Parameters:



25/29

25

APPENDIX III: Functions for manipulating permissions in page tables and

directory

inline void pt_set_page_permissions(struct Env* ptr_env, uint32 virtual_address, uint32

permissions_to_set, uint32 permissions_to_clear)

Description:

Sets the permissions given by permissions_to_setto 1 in the page table entry of the given

page (virtual address), and clears the permissions given by permissions_to_clear. The

environment used is the one given by ptr_pgdir

Parameters:

ptr_env: pointer to environment that you should work on

virtual_address: any virtual address of the page

permissions_to_set : page permissions to be set to 1

permissions_to_clear : page permissions to be set to 0

Return value: none

Example1: to set page PERM_BUFFERED bit to 1 and set PERM_PRESENT to 0

Example2: to set PERM_MODIFIED to 0

inline uint32 pt_get_page_permissions(struct Env* ptr_env, uint32 virtual_address )

Description:

Returns all permissions bits for the given page (virtual address) in the given environment page

directory (ptr_pgdir)

Parameters:

ptr_env: pointer to environment that you should work on

virtual_address: any virtual address of the page

Return value:

Unsigned integer containing all permissions bits for the given page

Example: To check if a page is modified:

uint32 page_permissions = pt_get_page_permissions(environment,virtual_address);if(page_permissions & PERM_MODIFIED){

. . .}

pt_set_page_permissions(environment, virtual_address, 0,PERM_MODIFIED);

pt_set_page_permissions(environment, virtual_address,PERM_BUFFERED, PERM_PRESENT);


26/29

26

inline uint32 pd_is_table_used(uint32* ptr_pgdir, uint32 virtual_address)

Description:

Returns a value indicating whether the table at virtual_address in was used by the processor

Parameters:

ptr_pgdir: pointer to environment page directory

virtual_address: any virtual address inside the table

Return value:

0: if the table at virtual_address is not used (accessed) by the processor

1: if the table at virtual_address is used (accessed) by the processor

Example:

inlinevoid pd_set_table_unused(uint32* ptr_pgdir, uint32 virtual_address)

Description:

Clears the Used Bit of the table at virtual_address

Parameters:

ptr_pgdir: pointer to environment page directory

virtual_address: any virtual address inside the table

if(pd_is_table_used(faulted_env->pgdir, virtual_address)){

}


27/29

27

APPENDIX IV: Page Buffering

Structures

In order to keep track of the buffered frames, the following changes were added to FOS:

1- We use one of the available bits in the page table as a BUFFERED bit (Bit number 9) to indicate whether

the frame of this page is buffered or not.

2- Add the following information to the "Frame_Info" structure:

a- isBuffered: to indicate whether this frame is buffered or not

b-va: virtual address of the page that was mapped to this buffered frame

c- environment: the environment that own this virtual address

3- Create a new list called "modified_frame_list"

A new Frame_Info* linked list is added to FOS to keep track of buffered modified page frames,(see

memory_manager.c):

Functions

inline void bufferList_add_page(struct Linked_List* bufferList,struct Frame_Info

*ptr_frame_info)

inline void bufferlist_remove_page(struct Linked_List* bufferList, struct Frame_Info

*ptr_frame_info)

Description:

Adds/Removes the given frame (ptr_frame_info) to/from the given list (bufferList)

Parameters:

bufferList: pointer to a linked list of Frame_Info* elements

ptr_frame_info: a pointer to Frame_Info object that will be added/removedExample:

struct Linked List modified frame list

0111231 123456789

PAGE FRAME ADDRESS 12...31 PA 0 0U/S

R/W

DAVAIL 0 0

P: Present bit (PERM_PRESENT)R/W: Read/Write bit (PERM_WRITABLE)U/S: User/Supervisor bit (PERM_USER)A: Accessed bit (PERM_USED)D: Dirty bit (PERM_MODIFIED)B: Buffered bit (PERM_BUFFERED)AVAIL: Available for system programmers useNOTE: 0 indicates Intel reserved. Dont define.

B

struct Frame_Info* ptr_victim_frame;. . .

bufferList_add_page(&modified_page_list, ptr_victim_frame);


28/29

28

Appendix V: Basic and Helper Memory Management Functions

Basic Functions

The basic memory manager functions that you may need to use are defined in kern/memory_manager.c file:

Function Name Description

allocate_frame Used to allocate a free frame from the free frame list

free_frame Used to free a frame by adding it to free frame list

map_frameUsed to map a single page with a given virtual address into a given allocated frame,

simply by setting the directory and page table entries

get_page_tableUsed by map_frame to get a pointer to the page table if exist, or creating it if

not exist

unmap_frameUsed to un-map a frame at the given virtual address, simply by clearing the page

table entry

get_frame_infoUsed by unmap_frame to get both the page table and the frame of the given

virtual address

Helpers Functions

There are some helper functions that we may need to use them in the rest of the course:

Function Description Defined in

LIST_FOREACH

(Frame_Info*,

Linked_List *)

Used to traverse a linked list.

Example: to traverse the modified list

struct Frame_Info* ptr_frame_info = NULL;

LIST_FOREACH(ptr_frame_info, &modified_frame_list)

{

.

}

inc/queue.h

LIST_CONCAT

(Linked_List* list1,Linked_List* list2)

Used to concatenate list2 to list1, after concatenation

list2 will be empty, and list 1 = list1+list2 inc/queue.h

to_frame_number

(Frame_Info *)

Return the frame number of the corresponding

Frame_Info elementKern/memory_manager.h

to_physical_address

(Frame_Info *)

Return the start physical address of the frame

corresponding to Frame_Info elementKern/memory_manager.h

to_frame_info

(uint32 phys_addr)

Return a pointer to the Frame_Info corresponding to

the given physical addressKern/memory_manager.h

K_PHYSICAL_ADDRESS(uint32 kernel

virtual address)

Used for returning the physical address of a givenkernel virtual address after KERNEL_BASE. This is done

simply by adding ( KERNEL_BASE) to the given address.

Kern/helpers.h


29/29

Function Description Defined in

K_VIRTUAL_ADDRESS

(uint32 physical

address)

Used for returning the kernel virtual address of a given

physical address. This is done simply by adding

KERNEL_BASE to the given address.

Kern/helpers.h

PDX (uint32 virtual

address)

Gets the page directory index in the given virtual

address (10 bits from 22 31).Inc/mmu.h

PTX (uint32 virtual

address)

Gets the page table index in the given virtual address

(10 bits from 12 21).Inc/mmu.h

ROUNDUP

(uint32 value, uint32

align)

Rounds a given value to the nearest upper value that

is divisible by align. Inc/types.h

ROUNDDOWN

(uint32 value, uint32

align)

Rounds a given value to the nearest lower value that

is divisible by align. Inc/types.h

tlb_invalidate(uint32*

page_directory,

uint32 virtual

address)

Refresh the cache memory (TLB) to remove the givenvirtual address from it.

Kern/helpers.c

rcr3()Read the physical address of the current page directory

which is loaded in CR3Inc/x86.h

lcr3(uint32 physical

address of directory)

Load the given physical address of the page directory

into CR3Inc/x86.h

Have a nice project

GOOD LUCK

os project 2011 documentation

Documents