ece344.a0.lie

4Assignment 0, Systems Software, ECE344, Fall 2011

Assignment 0: An Introduction to OS161Testing date: Week of Sept 19, 2011 in lab.ObjectivesAfter this assignment, you should:

Understand how System/161 emulates the MIPS hardware environment onwhich OS161 runs.Be familiar with Subversion (SVN) and GDB (the GNU Debugger).Setup your account to work on OS161Understand the source code structure of OS161, the software system wewill be using this term.Know how to build the OS161 kernel from source.Learn how to modify the OS161 kernel by adding debugging statements andsystem calls.Learn how to submit assignments.

DeliverablesYou will be required to do the following for full marks:

Be able to answer all questions about the source code below (in blue).Submit code that implements the printstring, printint and helloworld systemcalls.Demonstrate that these system calls work with a comprehensive set of testsfor the system calls.

IntroductionIn this assignment, we will introduce:System/161

The machine simulator for which you are building an operating system thisterm.

OS161The operating system you will be designing, extending, and running thisterm.

Subversion (SVN)

Assignment 0: Introduction to OS161 http://www.eecg.toronto.edu/~lie/Courses/ECE34...

1 of 17 14-07-23 04:33 PM

SVN is a source code revision control system. It manages the source les ofa software package so that multiple programmers may work simultaneously.Each programmer has a private copy of the source tree and makesmodications independently. SVN attempts to intelligently merge multiplepeople's modications, highlighting potential conicts when it fails.

GDB (Gnu Debugger)GDB allows you to examine what is happening inside a program while it isrunning. It lets you execute programs in a controlled manner and view andset the values of variables. In the case of OS161, it allows you to debug theoperating system you are building instead of the machine simulator onwhich that operating system is running.

The rst part of this document briey discusses the code on which you'll beworking and the tools you'll be using. You can nd more detailed information onSVN and GDB. The following sections provide instructions on what you must dofor this assignment.What are OS161 and System/161?The code is divided into two main parts:

OS161: the operating system that you will augment in subsequenthomework assignments.System/161: the machine simulator that emulates the physical hardwareon which your operating system will run. This course is about writingoperating systems, not designing or simulating hardware. Therefore, youmay not change the machine simulator.

The OS161 distribution contains a barebones operating system source tree,including some utility programs and libraries. After you build the operatingsystem you boot, run, and test it on the simulator.We use a simulator in OS161 because debugging and testing an operatingsystem on real hardware is extremely diicult. The System/161 machinesimulator has been found to be an excellent platform for rapid development ofoperating system code, while still retaining a high degree of realism. Apart fromoating point support and certain issues relating to RAM cache management, itprovides an accurate emulation of a MIPS processor.By the end of the semester you will have built an OS that can:

Run multiple programs simultaneouslySupport several simple UNIX system calls


2 of 17 14-07-23 04:33 PM

Provide virtual memory and swapOS161 assignments are cumulative. You will need to build each assignment ontop of your previous submission. So you will have to make sure that each of yourassignments work correctly!About SVNMost programming you have probably done has been in the form of 'one-o'assignments: you get an assignment, you complete it yourself, you turn it in, youget a grade, and then you never look at it again.The commercial software world uses a very dierent paradigm: developmentcontinues on the same code base producing releases at regular intervals. Thiskind of development normally requires multiple people working simultaneouslywithin the same code base, and necessitates a system for tracking and mergingchanges. Your will be working in teams of 2 on OS161 and will be developing acode base that will change over the couse of several assignments. Therefore, it isimperative that you start becoming comfortable with SVN, an open sourceversion control system.SVN is a powerful tool, but for OS161 you only need to know a subset of itsfunctionality. The SVN handout contains all the information you need to knowand should serve as a reference throughout the term. If you'd like to learn more,there is comprehensive documentation available here.About GDBIn some ways debugging a kernel is no dierent from debugging an ordinaryprogram. On real hardware, however, a kernel crash will crash the wholemachine, necessitating a time-consuming reboot. The use of a machine simulatorsuch as System/161 provides several debugging benets. First, a kernel crashwill only crash the simulator, which only takes a few keystrokes to restart.Second, the simulator can sometimes provide useful information about what thekernel did to cause the crash, information that may or may not be easilyavailable when running directly on top of real hardware.To debug OS161 you must use a version of GDB congured to understand OS161and MIPS. This is called cs161-gdb. This version of GDB has been patched to beable to communicate with your kernel through System/161.An important dierence between debugging a regular program and debuggingan OS161 kernel is that you need to make sure that you are debugging theoperating system, not the machine simulator. Type:


3 of 17 14-07-23 04:33 PM

% cs161-gdb sys161

and you are debugging the simulator. Not good. The handout Debugging withGDB provides detailed instructions on how to debug your operating system and abrief introduction to GDB. Setting up your accountLogin to the ECF machines. The OS161 tools are accessible fromp1-p170.ecf.toronto.edu. You will need to setup your path to access the OS161tools. Add the following at the end of your ~/.mycshrc le. set path=( /local/packages/cs161/bin $path)

If you use bash, add the following at the end of your ~/.bashrc le. export PATH=/local/packages/cs161/bin:$PATH

Then log out and log back in. Run echo $PATH and you should see the new path.Getting the distributionFirst, download the OS161 source into your home directory.Optional: In addition to OS161, you can also download the distributions forSystem/161, the machine simulator, and the OS161 toolchain. If you aredeveloping on ECF machines, you do not need these additional les, as they arealready installed. If you wish to develop on your home machine at home, you willneed to download, build, and install this package as well. Note that we do notprovide support for installing this package. Also, you must ensure that yourassignment works on the ECF machines.

Make a directory in which you will do all your work. For the purposes of theremainder of this assignment, we'll assume that it will be called ~/ece344. % mkdir ~/ece344 % cd ~/ece344 % mv ../os161-1.11.tar.gz .

1.

Unpack the OS161 distribution by typing % tar xzf os161-1.11.tar.gz

This will create a directory named os161-1.11

2.

Rename your OS161 source tree to just os161, and remove the tarball. % mv os161-1.11 os161 % rm os161-1.11.tar.gz

3.


4 of 17 14-07-23 04:33 PM

Setting up your SVN repositoryEach of you has been assigned a group and a group number. The instructoror lab TAs will provide you this number. Below, we use to denoteyour group number. Important: make sure you have submitted yourgroup members here before starting this step.

1.

Your SVN repository is located at svn+ssh://remote.ecf.utoronto.ca/n/svn/ece344f_. In the document below, this repository path is referred to as$ECE344_SVN. You can set this variable by adding the following to your~/.mycshrc. setenv ECE344_SVN svn+ssh://remote.ecf.utoronto.ca/n/svn/ece344f_

If you use bash, add the following at the end of your ~/.bashrc le. export ECE344_SVN=svn+ssh://remote.ecf.utoronto.ca/n/svn/ece344f_

2.

Run % svn ls $ECE344_SVN

and you should seetags/trunk/

3.

Change directories into the OS161 distribution that you unpacked in theprevious section and import your source tree. % cd ~/ece344/ % svn import os161 $ECE344_SVN/trunk/ -m "Initial import of os161" % svn copy $ECE344_SVN/trunk $ECE344_SVN/tags/asst0-start -m "Tagging initial import."

You can alter the arguments as you like; here's a quick explanation. -m"Initial import of os161" is the log message that SVN records. (If you don'tspecify it on the command line, it will start up a text editor). /trunk is whereSVN will put the les within your repository. $ECE344_SVN/trunk is the SVN URLyou will specify when you check out your system. Run svn ls on this URL andyou will see that the os161 directories have been imported into your SVNrepository. The second svn copy command creates a simple tag of the initialimport of your system that you can later use with svn diff and othercommands. More on that later (or see this tagging explanation).

4.

Now, remove the source tree that you just imported. % rm -rf os161

Don't worry - now that you have imported the tree in your repository, thereis a copy saved away. In the next step, you'll get a copy of the source treethat is yours to work on. You can safely remove the original tree.

5.


5 of 17 14-07-23 04:33 PM

Now, checkout a source tree in which you will work. % svn co $ECE344_SVN/trunk os161

The svn co command creates a working copy of your tree that you cansafely modify in ~/ece344/os161.

6.

Code readingOne of the challenges of OS161 is that you are going to be working with a largebody of code that was written by someone else. When doing so, it is importantthat you grasp the overall organization of the entire code base, understandwhere dierent pieces of functionality are implemented, and learn how toaugment it in a natural and correct fashion. As you and your partner developcode, although you needn't understand every detail of your partner'simplementation, you still need to understand its overall structure, how it ts intothe greater whole, and how it works.In order to become familiar with a code base, there is no substitute for actuallysitting down and reading the code. Admittedly, most code makes poor bedtimereading (except perhaps as a soporic), but it is essential that you read the code.It is all right if you don't understand most of the assembly code in the codebase;it is not important for this class that you know assembly.You should use the code reading questions included below to help guide youthrough reviewing the existing code. While you needn't review every line of codein the system in order to answer all the questions, we strongly recommend thatyou look over at least every le in the kernel.The key part of this exercise is understanding the base system. Your goal is tounderstand how it all ts together so that you can make intelligent designdecisions when you approach future assignments. This may seem tedious, but ifyou understand how the system ts together now, you will have much lessdiiculty completing future assignments. Also, it may not be apparent yet, butyou have much more time to do so now than you will at any other point in theterm.The le system, I/O, and network sections may seem confusing since we have notdiscussed how these components work. However, it is still useful to review thecode now and get a high-level idea of what is happening in each subsystem. Ifyou do not understand the low-level details now, that is OK.These questions are not meant to be tricky -- most of the answers can be foundin comments in the OS161 source, though you may have to look elsewhere (suchas Tannenbaum) for some background information. Make sure that you can


6 of 17 14-07-23 04:33 PM

answer these questions during evaluation.Using CscopeOne very useful tool for navigating source code is cscope. To use cscope, go tothe top level source directory and type "cscope -k -R". Cscope will thenrecursively scan the source directory and build a database of functions andvariables. The cscope interface is fairly self explanatory. You can search thecode for functions and variables, and move between the search results andoperations using the tab key. For more information, consult the cscope I3.Top Level DirectoryThe top level directory of many software packages is called src or source. In UNIX,if the operating system source is installed, it is typically found in /usr/src. The topof the OS/161 source tree is also called src. In this directory, you will nd thefollowing les:

Makefile: top-level makele; builds the OS/161 distribution, including allthe provided utilities, but does not build the operating system kernel.configure: this is an autoconf-like script. It sets up things like `How torun the compiler.' You needn't understand this le, although we'll askyou to specify certain pathnames and options when you build your owntree.defs.mk: this le is generated when you run ./configure. You needn't doanything to this le.defs.mk.sample: this is a sample defs.mk le. Ideally, you won't be needingit either, but if congure fails, use the comments in this le to xdefs.mk.

and the following directories:bin: this is where the source code lives for all the utilities that aretypically found in /bin, e.g., cat, cp, ls, etc. The things in bin areconsidered "fundamental" utilities that the system needs to run.include: these are the include les that you would typically nd in/usr/include (in our case, a subset of them). These are user levelinclude les; not kernel include les.kern: here is where the kernel source code lives.lib: library code lives here. We have only two libraries: libc, the C


7 of 17 14-07-23 04:33 PM

standard library, and hostcompat, which is for recompiling OS/161programs for the host UNIX system. There is also a crt0 directory,which contains the startup code for user programs.man: the OS/161 manual ("man pages") appear here. The man pagesdocument (or specify) every program, every function in the C library,and every system call. You will use the system call man pages forreference in the course of assignment 2. The man pages are HTML andcan be read with any browser.mk: this directory contains pieces of makele that are used for buildingthe system. You don't need to worry about these, although in the longrun we do recommend that anyone working on large software systemslearn to use make eectively.sbin: this is the source code for the utilities typically found in /sbin on atypical UNIX installation. In our case, there are some utilities that letyou halt the machine, power it o and reboot it, among other things.testbin: these are pieces of test code.

You needn't understand every line in every excutable in bin and sbin, but it isworth the time to peruse a couple to see how they work. Eventually, you willwant to modify these and/or write your own utilities and these are good models.Similarly, you need not read and understand everything in lib and include, butyou should know enough about what's there to be able to get around the sourcetree easily. The rest of this code walk-through is going to concern itself with thekern subtree.The Kern SubdirectoryOnce again, there is a Makele. This Makele installs header les but does notbuild anything. In addition, we have more subdirectories for each component ofthe kernel as well as some utility directories. We will ask you questions(highlighted in blue) during the lab so be sure you can nd the answersto them!kern/arch

This is where architecture-specic code goes. By architecture-specic,we mean the code that diers depending on the hardware platform onwhich you're running. For our purposes, you need only concernyourself with the mips subdirectory.

kern/arch/mips/conf


8 of 17 14-07-23 04:33 PM

conf.arch: This tells the kernel cong script where to nd the machine-specic, low-level functions it needs (see kern/arch/mips/mips).Makefile.mips: Kernel Makele; this is copied when you "cong a kernel".

kern/arch/mips/include

These les are include les for the machine-specic constants andfunctions.Which register number is used for the stack pointer (sp) inOS/161?What bus/busses does OS/161 support?What is the dierence between splhigh and spl0?Why do we use typedefs like u_int32_t instead of simply saying"int"?

kern/arch/mips/mips

These are the source les containing the machine-dependent code thatthe kernel needs to run. Most of this code is quite low-level.What does splx return?What is the highest interrupt level?

kern/asst1

This is the directory that contains the framework code that you willneed to complete assignment 1. You can safely ignore it for now.

kern/compile

This is where you build kernels. In the compile directory, you will ndone subdirectory for each kernel you want to build. In a realinstallation, these will often correspond to things like a debug build, aproling build, etc. In our world, each build directory will correspondto a programming assignment, e.g., ASST1, ASST2, etc. Thesedirectories are created when you congure a kernel (described in thenext section). This directory and build organization is typical of UNIXinstallations and is not universal across all operating systems.

kern/conf

cong is the script that takes a cong le, like ASST0, and creates the


9 of 17 14-07-23 04:33 PM

corresponding build directory. See how it is used in the section onBuilding a Kernel

kern/dev

This is where all the low level device management code is stored.Unless you pick a particularly low level nal project, you can safelyignore most of this directory.

kern/include

These are the include les that the kernel needs. The kernsubdirectory contains include les that are visible not only to theoperating system itself, but also to user-level programs. (Think aboutwhy it's named "kern" and where the les end up when installed.)How frequently are hardclock interrupts generated?What functions comprise the standard interface to a VFSdevice?How many characters are allowed in a volume name?How many direct blocks does an SFS le have?What is the standard interface to a le system (i.e., whatfunctions must you implement to implement a new le system)?What function puts a thread to sleep?How large are OS/161 pids?What operations can you do on a vnode?What is the maximum path length in OS/161?What is the system call number for a reboot?Where is STDIN_FILENO dened?

kern/lib

These are library routines used throughout the kernel, e.g., managingsleep queues, run queues, kernel malloc, etc.

kern/main

This is where the kernel is initialized and where the kernel mainfunction is implemented.


10 of 17 14-07-23 04:33 PM

kern/thread

Threads are the fundamental abstraction on which the kernel is built.Is it OK to initialize the thread system before the scheduler?Why (not)?What is a zombie?How large is the initial run queue?

kern/userprog

This is where you will add code to create and manage user levelprocesses. As it stands now, OS/161 runs only kernel threads; there isno support for user level code. In Assignment 2, you'll implement thissupport.

kern/vm

This directory is also fairly vacant. In Assignment 3, you'll implementvirtual memory and most of your code will go in here.

kern/fs

The le system implementation has two subdirectories. We'll talk abouteach in turn.

kern/fs/vfs

This is the le-system independent layer (vfs stands for "Virtual FileSystem"). It establishes a framework into which you can add new lesystems easily. You will want to go look at vfs.h and vnode.h beforelooking at this directory.What does a device name in OS/161 look like?What does a raw device name in OS/161 look like?What lock protects the vnode reference count?What device types are currently supported?

kern/fs/sfs

This is the simple le system that OS/161 contains by default. You canignore this directory for now.

Building a Kernel


11 of 17 14-07-23 04:33 PM

Now it is time to build a kernel. As described in kern/conf, you will need tocongure a kernel and then build it.

Congure your tree for the machine on which you are working.1. % cd ~/ece344/os161 % ./configure --ostree=$HOME/ece344/root

% make

This will build and create the various les needed to run your kernel andput them in $HOME/ece344/rootCongure a kernel named ASST0.2.

% cd ~/ece344/os161/kern/conf % ./config ASST0

This will create the ASST0 build directory in kern/compile. The next step willactually build a kernel in this directory. Note that you should specify thecomplete pathname ./config when you congure OS161. If you omit the ./,you may end up running the conguration command for the system onwhich you are building OS161, and that is almost guaranteed to producerather strange results!

3.

Build and install the ASST0 kernel. % cd ../compile/ASST0 % make depend % make % make install

4.

Note that if you make any changes to your kernel, you only need to repeatstep 4 to install the new kernel. However, if you modify or add testprograms to the ~/ece344/os161/testbin directory, you will need to run make inthe top level directory to make and install the new les from testbin.

Running your kernelChange to your root directory. Copy the default simulator conguration leinto the root directory. % cd ~/ece344/root % cp /local/packages/cs161/bin/sys161.conf.sample sys161.conf

You should take a look at this le, as it describes how to congure thesimulator you will be running your code in.

1.

Run the machine simulator on your operating system. % sys161 kernel

2.


12 of 17 14-07-23 04:33 PM

At the prompt, type p /sbin/poweroff. This tells the kernel to runthe poweroff program that shuts the system down.

3.

Adding Debugging StatementsProgram instrumentation is a common debugging technique where printf()statements (or in the case of kernel code, kprintf()) are added at various pointsin the code to output information about the program's running state. Instead ofadding printf() statements directly, it is often convenient to use macros thatallow instrumentation to be turned on and o easily.Examine the DEBUG macro in kern/include/lib.h. Based on your earlier reading ofthe operating system, add ve useful debugging messages to your operatingsystem. Be sure to add the debugging messages to various parts of the kernel;they should not all be in the same le.From reading the code and comments, try to gure out how to enable anddisable these DEBUG macros. In general, debugging operating systems code isfar more diicult than debugging regular code. To save your sanity, please referto the 10 Commandments of OS161 programming.Adding System CallsA large component of Assignment 0 is adding the following system calls toOS/161:

int _helloworld(void)This system call is handled by having the kernel print "Hello World\n" usingthe internel kprintf() function. It takes no arguments but returns the returnvalue from kprintf to the user level. You will need to understand how to adda new system call number, and build the user and kernel sides of theinterface. Study the existing reboot() system call to get started.

1.

int _printint(int value)This system call passes an integer to the operating system kernel, where itis printed out using the kprintf function. Only print the integer. Notably, donot print a newline after the number. The return value should be the returnvalue from kprintf.

2.

int _printstring(char *string, int numchars) This system call passes a pointer to a string to be printed, and a number ofcharacters to print. The kernel must rst copy the string from the useraddress into a kernel buer, and then print it using kprintf. Your code must

3.


13 of 17 14-07-23 04:33 PM

be careful to check for misuse by the user (for example, the memorypointed to by "string" may not contain a proper null-terminated string oflength numchars). If an error is detected, errno should be set and anappropriate result returned. Otherwise, the return value should be thereturn value from kprintf.

Notice that the system calls include a leading underscore ("_"). This is part of thesystem call name and is how user level programs will use the system call. (seethe User Level Test below).You will need to modify the code in kern/arch/mips/mips/syscall.c to detect your newsystem calls and dispatch appropriate system call handlers. Although thesesystem calls are simple enough to implement full within syscall.c, it is goodprogramming practice to put the handlers in a separate function in a le in thekern/userprog subdirectory. You should name this le simple_syscalls.c.Please read the document Understanding System Calls for additional informationabout how user-level programs are started from the OS/161 kernel, and howsystem calls operate on both the system and user side of the interface. Thisdocument also describes how to test your new system calls.Note that after adding a new le to the kernel source, itis necessary torecongure the ASST0 kernel before building and installing it. Examine thedocumentation at the top of the cong script (~/ece344/os161/kernel/conf/cong). Then edit the ASST0 cong le and rebuild the conguration: % cd ~/ece344/os161/kern/conf % vi ASST0 % ./config ASST0

Testing Your New System CallsQuick Sanity CheckAs a quick test you can modify kern/main/main.c and add a call (in a suitable place)to helloworld(). Build your kernel again. Make sure that your new kernel runsand displays the new message. Note that if you add new les to your kernelsource tree, you have to modify and enter them in the build conguration usingthe "le". command. Consult kern/conf/conf.kern for more documentation on howto do this.User Level TestsThe above test does not check if your system calls are callable from a user level(as opposed to kernel level) program. To test this, add new test programs to


14 of 17 14-07-23 04:33 PM

src/testbin. Use the existing programs as a template. For example, the followingcommands will make a copy of an existing program in testbin: % cd ~/ece344/os161/testbin % mkdir syscalltest % cp argtest/argtest.c syscalltest/syscalltest.c % sed 's/argtest/syscalltest/' argtest/Makefile > syscalltest/Makefile % touch syscalltest/depend.mk

Now modify syscalltest/syscalltest.c to call your new system calls. You could usethe following program to test the "_helloworld()" system call:#include

int main(){

_helloworld();return 0;

}

Modify testbin/Makefile to add an entry for the new syscalltest directory.Build and install the syscalltest program: % cd ~/ece344/os161/testbin/ % make % make install

Now run the syscalltest program by starting the os161 kernel, and entering p/testbin/syscalltest from the kernel prompt.One nal note: All user programs end by calling the "_exit" system call, even ifyour main() function does not end with the C library exit() function. The _exitsystem call isn't implemented yet, and you are not required to implement it forthis assignment. Without it, however, you will trigger the "default" case inmips_syscall when your test programs nish. The default case simply prints thatthe system call is not implemented.For the time being, you can if you wish, implement the _exit system call (similarto how you implemented the _helloworld system call). Basically, you need to adda case into syscall.c to handle the _exit system call. The implementation of thissystem call can simply call thread_exit. A common mistake is to call thread_exitfrom a user program. This is incorrect, since thread_exit is supposed to only beaccessible from the kernel, not from user programs.Again, implementing the _exit system call is an optional task for thisassignment.Sample Test Cases


15 of 17 14-07-23 04:33 PM

Your syscalltest program should call the new syscalls with various inputs, andverify that the return values and output are correct. Here are some test cases toget you started:

Call: _helloworld()Output: Hello World\n(Note this is a Hello World string followed by a newline character, not HelloWorld followed by \ and n characters.)Return value: 12 (the number of characters printed)Call: _printint(88)Output: 88Return value: 2Call: _printint(-1)Output: -1Return value: 2Call: _printstring("Hello",5)Output: HelloReturn value: 5Call: _printstring("Hello",3)Output: nothingReturn value: an appropriate error valueCall: _printstring("Hello",-10)Output: nothingReturn value: an appropriate error value

Preparing your assignment for submissionFinally, you need to prepare your code for submitting your assignment.

Once you are condent that you have completely done your assignment, runsvn commit from the os161 directory so that all modied les are checked inyour repository. Before running commit, you may have to add new les tothe repository using svn add. Please refer to the SVN document for moredetails. Use svn status in the os161 directory to make sure that all yourmodied source les are properly committed.

1.

Tag your repository for the end of asst0: % svn copy $ECE344_SVN/trunk $ECE344_SVN/tags/asst0-end

Remember the tags directory we created earlier? This is versioned like any

2.


16 of 17 14-07-23 04:33 PM

other part of your SVN repository. The step above simply copies from /trunk/into tags/asst0-end. It's also repeatable, so if you realized later you havechanges yet to commit you can repeat the copy using a new version of/trunk/.

What to submit for prep:Finally, create a patch that describes the changes you have made in thisassignment: % svn diff $ECE344_SVN/tags/asst0-start $ECE344_SVN/tags/asst0-end > asst0.patch

1.

Submit the patch le. % submitece344f 0 asst0.patch

2.

2011David Lie - all rights reserved. Last modied: 2011-09-19


17 of 17 14-07-23 04:33 PM

ece344.a0.lie

Documents