introduction of ya ffs 2

7/30/2019 Introduction of Ya Ffs 2

1/25

Company Internal

Rev 1

Introduction of YAFFS2


2/25

Company Internal

Rev 2

General description

YAFFS stands for Yet Another Flash File System.

Sep.2002 announced on linuxdevices.com.

YAFFS1 , support 512-byte page NAND devices.

YAFFS2 , follow-up to YAFFS1, support larger and different

devices.

Work with NAND flash.

also been used as a NOR file system and even as a RAM filesystem.


3/25

Company Internal

Rev 3

Dir & file structure (1/2 - on RAM)

Object

Root Object

Object (/aa)

Object (/bb)

Object (/lost+found)

Object (file)

Object (file)

Object (file)

From YAFFS home:

An Object structure are defined as

yaffs_ObjectStruct in yaffs_guts.h

Every FS entries are

represented as Object

Each partition has Root

Object

Objects are linked

mutually.


4/25

Company Internal

Rev 4

Chunk

Tag Data

Chunk

Dir & file structure (2/2 - on flash)

NAND

Partition

Block

Page

...Block Block

Page

Page

Partition

... ...

Object

Object Header Object Data

Tag Data

Header contains object information (ID, parent

object ID, its kind, etc...). Data structure is defined

as yaffs_ObjectHeaderin yaffs_guts.h

Object is consists of 2 kind of

elements, header& data (But

data is only for regular files).

Each element is stored in a data structure

called chunk. Its size is same as a page sizeof flash memory & it is stored directly into flash.

(Real data are divided into multiple chunks)

Chunks are stored in

each flash page.

Each chunk consists of Tags & Data. Tags has

information represents its usage & status, such

as Object ID. Real data is stored after Tag.


5/25

Company Internal

Rev 5

Checkpoint Checkpoint is a flag that represents FS structure in RAM was successfully

backed up into flash memory, and theres no file system update after that. This flag is also stored in flash memoryjust after all backup process has been done.

When check-pointed? >> while unmounting & just after sync.

Why the backup is needed? YAFFS2 do not have file system header, so scanning entire of partition is needed to restore FS

structure.

It takes for a while...

Restoring RAM status from NAND is much fasterthan that.

When the backup is used

While mounting a partition

CheckPointed?

Y N

Object structure is restored from backup data.Object structure is restored by scanning entire

of partition (refer to the next slide)


6/25

Company Internal

Rev 6

Scanning (How to restore FS object structure)

Chunk

NAND

Partition

Block

Page

...Block Block

Page

Page

Partition

... ...

TagData

Chunk

Tag Data

Chunk

Tag DataDataID

Object

ID

IDID

Object Object

IDID

Parent

ID

Parent

ID

Parent

ID

Get chunk info from each page.

Each Chunk contains Tag that has

object ID.

Create objects from ID stored in Tags

of each Chunk.

Each Tag also contains link information

(ID of parent object)

Relationship between objects are

restored from parent object IDs.

RAM


7/25

Company Internal

Rev 7

FS metadata protection (Or how to write / delete file)

File object

Object Header Chunk

Object Data Chunk

Object Data Chunk

Object Data Chunk

Object Data Chunk.

.

.Object Data Chunk

NAND

Empty Page

Empty Page

Empty Page

Empty Page

Empty Page.

.

.Empty Page

Object Header

Object Data

Object Data

Object Data

Object Data

Object Data

Writing sequenceDeleting sequenceIf crash / power fault would happen while writing...

Object Header was flashed.

Only these data are flashed.

Then..

File size (in object header) will be

shrank to this size at the next

scan, on mounting.

If crash / power fault would happen while deleting...

Empty Page

Empty Page

Object Header was not erased.

These data still remains.

Then..

File size (in object header) will be

shrank to this size at the next

scan & the file remains.

Object Header (File metadata) is

written at first & real data follows.

Real data are erased from the tail &

Object Header is erased at last.


8/25

Company Internal

Rev 8

Further topics...

The followings are keywords for further investigation:

Garbage collection

Cache mechanism


9/25

Company Internal

Rev 9

Reference

YAFFS Home.

http://www.yaffs.net/
http://www.yaffs.net/http://www.yaffs.net/


10/25

Company Internal

Rev 10

End of the slides


11/25

Company Internal

Rev 11

Backup


12/25

Company Internal

Rev 12

YAFFS RAM structures

Device/partition

This is named yaffs_Device. This is requi red to ho ld informat ion relat ing to a yaf fspartition.

NAND Block Information

This is named yaffs_BlockInfoandhold s state that ref lects the current state of the NAND blo cks.

NAND Chunk Information

This is a bit-field attached to the yaffs_Device that holds the cu rrent in-use state of each chu nkin the system. Object

This is named yaffs_Object . object is uniquely identified by an objectId.

Regular data files

Directories

Hardlinks

Symbolic links

Special objects (pipes, devices etc).

File structure

For each file object, YAFFS holds a tree which provides the mechanism to f ind the data chunks in a f i le. The tree consists of

nodes called yaffs_TNodes (tree nodes). Directory structure

The directory structure a l lows ob jec ts to be found by name.

Object number hash table

The object number hash table provides a mechanism to f in d an object f rom i ts ob ject Id. The objectId is hashed to select a hashbucket. Each hash bucket has a doubly linked list of objects that belong in this hash bucket.

Cache

YAFFS provides a read/write cache that significantly improv es per formance for shor t op erat ions.


13/25

Company Internal

Rev 13

Directory & TNode structure

Directory Structure File Structure


14/25

Company Internal

Rev 14

YAFFS Object header(1/2 Object structure on RAM)struct yaffs_ObjectStruct {

__u8 deleted:1; /* This should only apply tounlinked files. */__u8 softDeleted:1; /* it has also been soft deleted */__u8 unlinked:1; /* An unlinked file. The file should be in the unlinkeddirectory.*/__u8 fake:1; /* A fake object has no presence on NAND. */__u8 renameAllowed:1; /* Some objects are not allowed to be renamed. */__u8 unlinkAllowed:1;__u8 dirty:1; /* the object needs to be written to flash */__u8 valid:1; /* When the file system is being loaded up, this

* object might be created before

the data* is available (ie. file data

records appear before the header).*/

__u8 lazyLoaded:1; /* This object has been lazy loaded and is missing somedetail */

__u8 deferedFree:1; /* For Linux kernel. Object is removed from NAND, but is* still in the inode cache. Free

of object is defered.* until the inode is released.*/

__u8 beingCreated:1; /* This object is still being created so skip some checks.*/

__u8 serial; /* serial number of chunk in NAND. Cached here */__u16 sum; /* sum of the name to speed searching */

struct yaffs_DeviceStruct *myDev; /* The device I'm on */

struct ylist_head hashLink; /* list of objects in this hash bucket */

struct ylist_head hardLinks; /* all the equivalent hard linked objects */

/* directory structure stuff *//* also used for linking up the free list */struct yaffs_ObjectStruct *parent;struct ylist_head siblings;

... (Continue to right)

... (Continue from left)

/* Where's my object header in NAND? */int hdrChunk;

int nDataChunks; /* Number of data chunks attached to the file. */

__u32 objectId; /* the object id value */

__u32 yst_mode;

#ifdef CONFIG_YAFFS_SHORT_NAMES_IN_RAMYCHAR shortName[YAFFS_SHORT_NAME_LENGTH + 1];#endif

#ifndef __KERNEL____u32 inUse;

#endif

#ifdef CONFIG_YAFFS_WINCE__u32 win_ctime[2];__u32 win_mtime[2];__u32 win_atime[2];

#else__u32 yst_uid;__u32 yst_gid;__u32 yst_atime;__u32 yst_mtime;__u32 yst_ctime;

#endif

__u32 yst_rdev;

#ifdef __KERNEL__struct inode *myInode;

#endif

yaffs_ObjectType variantType;

yaffs_ObjectVariant variant;

};

typedef struct yaffs_ObjectStruct yaffs_Object;

struct yaffs_ObjectList_struct {yaffs_Object *objects;struct yaffs_ObjectList_struct *next;

};


15/25

Company Internal

Rev 15

YAFFS Object header(1/2 Object structure stored in flash memory)typedef struct {

yaffs_ObjectType type;

/* Apply to everything */

int parentObjectId;

__u16 sum__NoLongerUsed; /* checksum of name. No longer used

*/

YCHAR name[YAFFS_MAX_NAME_LENGTH + 1];

/* The following apply to directories, files, symlinks - not hardlinks */

__u32 yst_mode; /* protection */

#ifdef CONFIG_YAFFS_WINCE

__u32 notForWinCE[5];

#else

__u32 yst_uid;

__u32 yst_gid;

__u32 yst_atime;

__u32 yst_mtime;

__u32 yst_ctime;

#endif

/* File size applies to files only */

int fileSize;

/* Equivalent object id applies to hard links only. */

int equivalentObjectId;

... (Continue to right)

... (Continue from left)

/* Alias is for symlinks only. */

YCHAR alias[YAFFS_MAX_ALIAS_LENGTH + 1];

__u32 yst_rdev; /* device stuff for

block and char devices (major/min) */

#ifdef CONFIG_YAFFS_WINCE

__u32 win_ctime[2];

__u32 win_atime[2];

__u32 win_mtime[2];

#else

__u32 roomToGrow[6];

#endif

__u32 inbandShadowsObject;

__u32 inbandIsShrink;

__u32 reservedSpace[2];int shadowsObject; /* This object header shadows the

specified object if > 0 */

/* isShrink applies to object headers written when we shrink the

file (ie resize) */

__u32 isShrink;

} yaffs_ObjectHeader;


16/25

Company Internal

Rev 16

Rachael NAND MapS

1

B

O

O

T

S

Y

S

T

E

M

U

S

E

R

D

A

T

A

C

A

C

H

E

0 a7b

1d30

2fb

1fd8

M

D

M

L

O

G

A

P

P

L

O

G

1fb0

NAND

PartitionD

A

T

A

B

L

O

C

K

64

P

A

G

E

K

E

R

N

E

L

D

A

T

A

B

L

O

C

K

block

2048 page

D

A

T

A

H

E

A

D

E

R

T

A

G

s


17/25

Company Internal

Rev 17

YAFFS2 tagsField Comment

chunkUsed Status of the chunk: used or unused.

objectId If 0 then this is not part of an object(unused).

chunkId If 0 then this is a header, else a data chunk.

byteCount Only valid for data chunks.eccResult ECC check result.

blockBad Is it a bad block?

sequenceNumer The sequence number of this block.

extraHeaderInfoAvailable Extra info available if this is not zero.

extraParentObjectId The parent object.

extraIsShrinkHeader Is it a shrink header?

extraShadows Does this shadow another object?

extraObjectType What object type?

extraFileLength Length if it is a file.

extraEquivalentObjectId Equivalent object ID if it is a hard link.


18/25

Company Internal

Rev 18

Mount/Scan

Scan backward, read the tags for each chunk in each block.

IsCheckPointed?

Quick mounting.

Restore checkpoint.

Y

Pre-scan all the blocks to determine their states

and Sort blocks by sequence Number

N

If it is the first objectID:ChunkID pair in the file,

slotted it into the file.

Else if the object does not yet exist (invalid),use the

chunkIdand nbytes to set the file extents.

Else If it is a duplicate data chunk , discard.

Else if the chunk lies beyond the objects file scan

size , discard.

Is data

chunk?Y

If duplicated header, suck out resize info thendiscard.

Else if an object header has not yet been found for

this object, figure out object info and hook up to

parent directory.

N(header chunk)


19/25

Company Internal

Rev 19

File Operation

File resize Newsize > Oldsize

Update object header.

Newsize < Oldsize

Delete backwards data chunks out of file length bound.

Update last data chunk if needed. Prune file tree structure to new size, free TNode with NULL chunk.

Update object header.

File deletion Resize file length to 0.

Free TNode.

Remove from directory.

Delete header chunk.

Free INode, un-hash object from hash list.

Write data to file Write new chunks with buffer data to NAND Plug NAND chunk into file structure.

Reconstruct file tree.

Manage map issue.


20/25

Company Internal

Rev 20

Bad block handling

Block is marked as bad when:

A read or write operation fails or if three ecc errors are detected.

A block erase operation fails.

Bad block is retired from use.


21/25

Company Internal

Rev 21

Power failure handling

Reconstruct file system at next mounting.

Backward scanning.

Sequence number(provides a way of organizing the log inchronological order).

Shrink header marker(flag help to calculate real file size).

If there are duplicate data chunks, the latest chunk data is used.

Lost and found

During scan , if no parent has been found. Put this object into

LostNFound directory.


22/25

Company Internal

Rev 22

Block and chunk management


23/25

Company Internal

Rev 23

Block and chunk allocation

Yaffs_AllocateChunk()

allcationBlock

allcationPage

Allocating chunks sequentially from Current block.

As the chunks are allocated, block and chunk management

information including the block's pagesInUse count and the chunk

use bitmap.

When the block is full, another empty block is selected for become

the allocation block. The blocks are selected by searching upwards

from the previous allocation block.


24/25

Company Internal

Rev 24

Flags setting for Rachael

CONFIG_YAFFS_FS=y YAFFS2 file system support

CONFIG_YAFFS_YAFFS1=y For 512-byte page devices.

CONFIG_YAFFS_9BYTE_TAGS is not set Use older-style on-NAND data format with pageStatus byte

CONFIG_YAFFS_DOES_ECC is not set If this config is set, let Yaffs do its own ECC. [We use NAND driver ECC.]

CONFIG_YAFFS_YAFFS2=y For >= 2K bytes per page devices

CONFIG_YAFFS_AUTO_YAFFS2=y Auto select yaffs2 format.

CONFIG_YAFFS_DISABLE_LAZY_LOAD is not set "Lazy loading" defers loading file details until they are required. This saves mount time, but makes the first look-up a bit longer.

CONFIG_YAFFS_DISABLE_WIDE_TNODES is not set Wide Tnodes are only used for NAND arrays >=32MB for 512-byte page devices and >=128MB for 2k page devices. They use slightly

more RAM but are faster since they eliminate chunk group searching.CONFIG_YAFFS_ALWAYS_CHECK_CHUNK_ERASED is not set

Normally YAFFS only checks chunks before writing until an erased chunk is found. This helps to detect any partially written chunksthat might have happened due to power loss.

CONFIG_YAFFS_SHORT_NAMES_IN_RAM = y If this config is set, then short names are stored with the yaffs_Object. This costs an extra 16 bytes of RAM per object, but makes look-

ups faster.


25/25

Company Internal

Rev 25

Ref. Mount

If checkpoint is set, quick mounting.Else Backward scanning.

Pre-scan and sort blocks.

scan backwards, read the tags for each chunk in each block

If it is a data chunk (chunkId > 0) then: If the objectId:chunkId has already been found then this is not the current chunk. Discard. Else if the object is deleted. Discard.

Else if the object does not yet exist(invalid) then this chunk was written since the last object header write forthis object so this chunk must be the last written chunk in the file and must have happened just before anunclean shutdown. We can use the chunkId and nbytes to set the file extents.

Else if the object does exist and the chunk lies beyond the object's file scan size then discard.

Else it's just slotted in to the file.

If it is an object header (chunkId == 0) then: If the object is in the deleted directory then mark the object as deleted and set the object's shrink size to zero.

Else if an object header has not yet been found for this object, then this is the most current object headerand holds the most current name etc. If no data chunks have yet been found for this chunk then the size inthe object header is also the file extents. The file size is used to set the file scan size since at the time it waswritten the file did not extend past that point and we can safely ignore further data chunks for that file thatextend past that point.

Else just use the file size as the scan size if it is smaller than the current file scan size.

Else discard this object header.

introduction of ya ffs 2

Documents