Spring 2004ECE569 Lecture 03-1.1

ECE 569 Database System Engineering

Spring 2004

Yanyong Zhang www.ece.rutgers.edu/~yyzhang

Course URL www.ece.rutgers.edu/~yyzhang/spring03

Spring 2004ECE569 Lecture 03-1.2

Warm-up Discussion

Address space, virtual memory

Memory magement

How does file system interact with virtual memory?

What happens when the user makes a “read” call?

Spring 2004ECE569 Lecture 03-1.3

Questions to answer in this class?

If a query tries to access tuple T, how can the system locate where T is?

Spring 2004ECE569 Lecture 03-1.4

Media and File Management

Abstraction Array of fixed length blocks

Size varies dynamically, space permitting

Notation Blocks – File system objects

Pages – Units of virtual address space

Slots – Units of storage on disk

All of these items are identical in size and there is a direct mapping from one to the other.

We will generally use them interchangeably.

Spring 2004ECE569 Lecture 03-1.5

Data structure for Block#define EIGHTK 8192typedef unsigned int FILENO;

typedef unsigned int BLOCKID;

typedef struct {int flip; FILENO

fileno;

BLOCKID blockno;

} BLOCKHEAD;

typedef struct {BLOCKHEAD header;

char contents[EIGHTK-sizeof(header)-2];

int flop;

} BLOCK, *BLOCKP;

Spring 2004ECE569 Lecture 03-1.6

File System API

STATUS create(filename, allocparmp)-- create and allocate a new file STATUS delete(filename)-- delete a file and deallocate space for it STATUS open(filename,ACCESSMODE,FILEID)-- Open file in desired mode and return file handle STATUS close(FILEID)-- Close an open file STATUS extend(FILEID,allocparamp)-- extend existing file by specified amount STATUS read(FILEID,BLOCKID,BLOCKP)-- read contents of specified block into a buffer  STATUS readc(FILEID,BLOCKID,blockcount,BLOCKP) -- read a certain number of block into consecutive buffers in memory. STATUS write(FILEID,BLOCKID,BLOCKP)-- write a buffer to the specified block on disk. STATUS writec(FILEID,BLOCKID,blockcount,BLOCKP)-- write a number of blocks from consecutive pages to disk. 

Spring 2004ECE569 Lecture 03-1.7

Mapping Blocks onto Slots

Disk allocation schemes determine the mapping between blocks onto slots

Issues in disk space allocation: Initial allocation: When a file is created, how many

contiguous slots should be allocated to it?

Incremental expansion: If an existing file grows beyond the number of slots currently allocated, how many additional contiguous blocks should be assigned to that file?

Reorganization: When and how should the free space on the disk be reorganized?

Spring 2004ECE569 Lecture 03-1.8

Free Space Management

+ Bit Map

- One bit represents each block

- Easy to find contiguous free spaces for allocation

+ Free List

- Link free blocks together into a free list

- Can use all techniques for memory free-list management, first-fit, best-fit, etc.

+ fault-tolerance

- extra slots

Spring 2004ECE569 Lecture 03-1.9

Static and Contiguous Allocation

At file creation time, the total number of blocks a file needs is allocated at one time in contiguous slots.

Address translation is easy sb = the slot holding block 0;

sb+k = the slot holding block k

Advantage: Supports both block-direct access and sequential access

Disadvantages: Files cannot grow, cannot shrink

Discussion: database systems by third-party vendor prefer such file organizations

Spring 2004ECE569 Lecture 03-1.10

Extent-based Allocation

Provides many of the advantages of contiguous allocation without many of the problems

The Idea Allocate an initial chunk that is probably big enough

(primary allocation)

If file runs out of space, allocate another chunk (secondary allocations)

Successive allocations increase in size

Characteristics Good clustering allows efficient sequential I/O

More complex address translation than contiguous allocation

Spring 2004ECE569 Lecture 03-1.11

Extent-based Allocation

Disk-A Disk-B Disk-A Disk-A disk-id

extent index

accum-length

14 187 3 214

100 350 600 850

primary 1.secondary 2.secondary 3.secondary

file directory

A B extent directory

Why do we need extent directory?• transparency• easy lock name

Spring 2004ECE569 Lecture 03-1.12

Single-Slot Allocation

extent-based allocation with extent size of 1 slot.

Used by original UNIX file system, and implemented by the idea of i-node.

10 + n + n2 + n3 blocks can be addressed, where n is the number pointers that can be held by one slot (e.g., 8192 / 4 = 2048).

Poor random access performance for large files.

Spring 2004ECE569 Lecture 03-1.13

Mapping Relations to Disks

file system & operating system database system

real disks

File system segments

relations extents

Spring 2004ECE569 Lecture 03-1.14

Buffer management

Database buffer is the mediator between the basic file system and the tuple-oriented file system.

A tuple is addressed as <fileno, pageno, page-offset>

Main purpose To make the pages addressable in main memory

To coordinate the writing of pages to disks with the log manager and recovery manager

To minimize the number of actual disk accesses.

Spring 2004ECE569 Lecture 03-1.15

Logic of Buffer Manager

+ Search in buffer: Check if the requested page is in the buffer. If found, return the address F of this frame to the caller.

+ Find free frame: If the page is not in the buffer, find a frame that holds no valid page.

+ Determine replacement victim: If no such frame exists, determine a page that can be removed from the buffer (in order to reuse its frame).

+ Write modified page: If replacement page has been changed, write it.

+ Establish frame address: Denote the start address of the frame as F.

+ Determine block address: Translate the requested PAGEID P into a FILEID and a block number. Read the block into the frame selected.

+ Return: Return the frame address F to the caller.

Spring 2004ECE569 Lecture 03-1.16

Buffer Management process of access

module

buffer storage area

buffer is accessible from the caller's process (shared memory)

buffer manager interface

readdirect

directory

bufferfix (P, ...) Give me page P

find frame in buffer

determine FILEID and block number

return frame address F

1

2 3

4

5

Spring 2004ECE569 Lecture 03-1.17

Buffer Management vs. Read operation

Share vs private (lost update anomaly)

In buffer management, the clients tell whether they are using the page or not.

Spring 2004ECE569 Lecture 03-1.18

Lost Update Anomoly process A process B process A process B

bufferfix (P,...) bufferfix (P,...)

give copy to caller

give copy to caller

change P to P' change P to P"

rewrite page rewrite page

?

a) Access module in process A requests access to page P; gets private copy.

b) Access module in process B requests access to page P; gets private copy.

c) Both processes try to rewrite an updated version of the page, but these versions are different. Only the version written last will be on disk; this is the "lost update" anomaly.

This implies that the buffer should be shared by multipleprocesses.

Spring 2004ECE569 Lecture 03-1.19

The Need for Synchronization

transaction X

bufferfix(P,...)

page P not in buffer

read block containing page P

return base address in buffer

transaction Y transaction Y

bufferfix(Q,...)

page Q not in buffer; replace P

read block containing page Q

bufferfix(Q,...)

return base address in buffer

a) Transaction X requests access to page P; gets base address in buffer.

b) Transaction Y requests access to page Q; buffer mgr. decides to replace page P

c) Transaction Y gets the base address of Q in the buffer - is same as P's.

Spring 2004ECE569 Lecture 03-1.20

The Fix / Use / Unfix Protocol

+ FIX: The client requests access to a page using the bufferfix interface.

+ USE: The client uses the page and the pointer to the frame containing the page will remain valid.

+ UNFIX: The client explicitly waives further usage of the frame pointer; that is, it tells the buffer manager that it no longer wants to use that page.

Spring 2004ECE569 Lecture 03-1.21

The Fix / Use / Unfix Protocol

page P

page Q

page R

fix page P

use

use unfix page P

fix page R

use

use

use

unfix page R

fix page Q use

use

use unfix page Q

Spring 2004ECE569 Lecture 03-1.22

Buffer Control Blocks

typedef struct {

PAGEID pageid; /* id of page in file*/

PAGEPTR pageaddr; /* base addr. in buffer*/

Int index; /* record within page */

Semaphore *pagesem; /* pointer to the sem. */

Boolean modified; /* caller modif. page*/

Boolean invalid; /* destroyed page */

} BUFFER_ACC_CB, *BUFFER_ACC_CBP;

Spring 2004ECE569 Lecture 03-1.23

Buffer Structure

bufferpool

frames

hash table

buffer control block

buffer control block

buffer control block

buffer control block

buffer control block

buffer control block

p a g e s

frame_index first_bcb

next_in_hclass

mru_page lru_page

buffer access control block

to and from client

index of frame holding the page (address pointer in case of buffer access control block) chain of buffer control blocks in same hash class LRU - chain