introduction to ubi
TRANSCRIPT
![Page 1: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/1.jpg)
![Page 2: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/2.jpg)
Overview Sub-systems Kernel API & EBA subsystem Wear-leveling subsystem Scanning subsystem
2
![Page 3: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/3.jpg)
UBI – Unsorted Block Images
A volume management system Provides static and dynamic volumes
Wear-leveling across whole flash device
Transparent bad blocks management
Read disturbance handling
Merged in the mainline Linux kernel since v2.6.22
3
![Page 4: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/4.jpg)
4
MTD Device
MTD Partition MTD Partition (UBI Device)
Dynamic UBI VolumeStatic UBI Volume
(0,0) (0,1) ... (0,P) (1,0) (1,1) (1,2) (1,3) (1,4) (1,5) ... (1,Q)
0 1 2 3 4 5 6 7 8 9 ... N
Root Filesystem (UBIFS)Kernel ImageBootloader
![Page 5: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/5.jpg)
5
MTD Layer
UBI Wear-leveling Subsystem
UBI Kernel API
UBI I/O Subsystem
UBI Scanning Subsystem
UBI Erase Block Association SubsystemUBI Initialization
![Page 6: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/6.jpg)
6
fs_write()
fs_read()
Filesystem
ubi_leb_write()
ubi_leb_map()
ubi_leb_read()
ubi_leb_unmap()
ubi_leb_erase()
UBI KAPI
ubi_eba_write_leb()
ubi_eba_map_leb()
ubi_eba_unmap_leb()
ubi_eba_read_leb()
ubi_eba_copy_leb()
UBI EBA
ubi_wl_get_peb()
ubi_wl_put_peb()
ubi_wl_scrub_peb()
ubi_wl_flush()
UBI WL Read from an unmapped LEB
Read from a mapped LEB
Write to a mapped LEB
Write to an unmapped LEB
Map a LEB
Unmap a LEB
Erase a LEB
ubi_io_read()
ubi_io_write()
ubi_io_sync_erase()
UBI IO
ubi_io_read_vid_hdr()
ubi_io_write_vid_hdr()
ubi_io_read_data()
ubi_io_write_data()
ubi_io_read_ec_hdr()
ubi_io_write_ec_hdr()
ubi_io_mark_bad()
![Page 7: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/7.jpg)
![Page 8: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/8.jpg)
Responsible for Management of PEBs Wear-leveling Scrubbing (read disturbance)
Works in terms of PEBs and erase counters
Knows nothing about LEBs, volumes, etc
Internal data structures Four RB-trees and one queue
External interfaces ubi_wl_get_peb() ubi_wl_put_peb() ubi_wl_scrub_peb() ubi_wl_flush()
8
![Page 9: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/9.jpg)
9
erroneous
scrub
free
used 8,33,9 6,6
pq 3,4
drivers/mtd/ubi/ubi.hstruct ubi_device {
...struct rb_root used;struct rb_root erroneous;struct rb_root free;struct rb_root scrub;struct list_head pq[UBI_PROT_QUEUE_LEN];...
}
All good PEBs are managed with four RB-trees, and one queue
Note: These RB-trees use (ec, pnum) pairs as keys
2,5 3,21,1 1,7 7,8Free PEBs
In-used PEBs
Good PEBs
![Page 10: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/10.jpg)
10
erroneous
scrub
free
used 8,33,9 6,6
pq 3,4
drivers/mtd/ubi/wl.cint ubi_wl_get_peb(struct ubi_device *ubi, int dtype)int ubi_wl_put_peb(struct ubi_device *ubi, int pnum, int torture)int ubi_wl_scrub_peb(struct ubi_device *ubi, int pnum)int ubi_wl_flush(struct ubi_device *ubi)
2,5 3,21,1 1,7 7,8
ubi_thread
ubi_wl_get_peb()
ubi_wl_put_peb()
ubi_wl_scrub_peb()
ubi_wl_flush()
![Page 11: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/11.jpg)
11
erroneous
scrub
used 8,33,9 6,6
pq 3,4
ubi_thread
drivers/mtd/ubi/wl.cint ubi_wl_get_peb(struct ubi_device *ubi, int dtype)
1. Pick a PEB from the free RB-tree
according to the hint @dtype
• longterm
• shortterm
• unknown
2. Move the picked PEB to the pq queue
• why pq? why not used?
free 2,5 3,21,1 1,7 7,8
Keep newly allocated PEBs from being moved due to wear-leveling.
shortterm longtermunknown
![Page 12: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/12.jpg)
12
erroneous
scrub
used
1,1
3,9 6,6
pq 3,4
ubi_thread
drivers/mtd/ubi/wl.cint ubi_wl_scrub_peb(struct ubi_device *ubi, int pnum)
1. Move the PEB @pnum from pq/used to
scrub
2. Schedule a wear-leveling request
free 2,5 3,21,7 7,8
Besides wear-leveling, I also take care of scrubbing.
8,3
![Page 13: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/13.jpg)
13
erroneous
scrub
used 3,9
pq 3,4
ubi_thread
drivers/mtd/ubi/wl.cint ubi_wl_put_peb(struct ubi_device *ubi, int pnum, int torture)
1. Remove the PEB @pnum from one of
the in-used RB-trees or pq.
2. Schedule the PEB @pnum for erasure.
3. When the erasure is done without any
error, the PEB will be put back to the free
RB-tree.
free 2,5 3,2
6,6
1,7 7,8
Again, the erasure will be delagated to me.
8,3
1,1
6,6
![Page 14: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/14.jpg)
14
ubi_thread
drivers/mtd/ubi/wl.cint ubi_wl_flush(struct ubi_device *ubi)
ubi_work ubi_work ubi_work ubi_work
erase_worker() wear_leveling_worker()1. Flush all pending works
drivers/mtd/ubi/wl.cstruct ubi_work {
struct list_head list;int (*func)(struct ubi_device *ubi, struct ubi_work *wrk, int cancel);
/* The below fields are only relevant to erasure works */struct ubi_wl_entry *e;int torture;
};
![Page 15: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/15.jpg)
if (!free || (!scrub && !used))return
if (scrub) {e1 = pick the least worn out PEB from the @scrube2 = find_wl_entry(free, WL_FREE_MAX_DIFF)
}else {
e1 = pick the least worn out PEB from the @usede2 = find_wl_entry(free, WL_FREE_MAX_DIFF)
if ((e2->ec – e1->ec)<UBI_WL_THRESHOLD)return;
}
ubi_eba_copy_leb(ubi, e1->pnum, e2->pnum, vid_hdr)
15
drivers/mtd/ubi/wl.cstatic int wear_leveling_worker(struct ubi_device *ubi,
struct ubi_work *wrk, int cancel)
static struct ubi_wl_entry *find_wl_entry(struct rb_root *root, int max)
erroneous
scrub
used 3,9
pq 3,4
free 2,5 3,21,7 6,6
8,3
1,1
7,8
![Page 16: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/16.jpg)
err = sync_erase(ubi, e, wl_wrk->torture);if (!err) {
wl_tree_add(e, &ubi->free);serve_prot_queue(ubi);return ensure_wear_leveling(ubi);
}
if (err == -EINTR || err == -ENOMEM || err == -EAGAIN || err == -EBUSY)return schedule_erase(ubi, e, 0)
else if (err != -EIO)goto out_ro;
/* It is %-EIO, the PEB went bad */if (!ubi->bad_allowed)
goto out_ro;
if (ubi->beb_rsvd_pebs == 0)goto out_ro;
err = ubi_io_mark_bad(ubi, pnum);return err;
out_ro:ubi_ro_mode(ubi) /* switch to read-only mode */return err;
16
drivers/mtd/ubi/wl.cstatic int erase_worker(struct ubi_device *ubi, struct ubi_work *wrk, int cancel)
erroneous
scrub
used 3,9
pq 3,4
free 2,5 3,21,7 6,6
8,3
1,1
7,8
![Page 17: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/17.jpg)
![Page 18: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/18.jpg)
Responsible for Scanning the flash media Checking UBI headers Providing complete information about the UBI flash image
UBI on-flash data structures Erase Counter Header Volume Identifier Header Volume Table
Temporary data structures during scanning process Scan Info Scan Volume Scan Erase Block Four lists: free, erase, corr, alien
Unclean reboot
18
![Page 19: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/19.jpg)
19
Every good PEB has a 64-byte Erase Counter Header
Every good mapped PEB has a 64-byte Volume Identifier Header
A “layout volume” contains two copies of the Volume Table
…
5 7 N860 1 3
0,1 0,P 1,0 1,1 2,0 2,2 2,Q
2 4
0,0 2,1 ...
...
...
PEBs
LEBs
![Page 20: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/20.jpg)
drivers/mtd/ubi/ubi-media.hstruct ubi_ec_hdr {
__be32 magic; /* EC header magic number (%UBI_EC_HDR_MAGIC) */__u8 version; /* version of UBI implementation */__u8 padding1[3]; /* reserved for future, zeroes */__be64 ec; /* the erase counter */__be32 vid_hdr_offset; /* where the VID header starts */__be32 data_offset; /* where the user data start */__be32 image_seq; /* image sequence number */__u8 padding2[32]; /* reserved for future, zeroes */__be32 hdr_crc; /* erase counter header CRC checksum */
} __attribute__ ((packed));
205 7 N860 1 3
0,1 0,P 1,0 1,1 2,0 2,2 2,Q
Every good PEB has a 64-byte Erase Counter Header
…
2 4
0,0 2,1 ...
...
...
PEBs
LEBs
![Page 21: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/21.jpg)
drivers/mtd/ubi/ubi-media.hstruct ubi_vid_hdr {
__be32 magic; /* VID magic number (%UBI_VID_HDR_MAGIC)*/ __u8 version; /* version of UBI implementation */__u8 vol_type; /* volume type (%UBI_VID_DYNAMIC or %UBI_VID_STATIC) */__u8 copy_flag; /* for wear-leveling reasons */__u8 compat; /* compatibility of this volume */__be32 vol_id; /* ID of this volume */__be32 lnum; /* LEB number */__u8 padding1[4]; /* reserved for future, zeroes */__be32 data_size; /* bytes of data this LEB contains */__be32 used_ebs; /* total number of used LEBs in this volume */__be32 data_pad; /* padded bytes at the end of this PEB */__be32 data_crc; /* CRC of the data stored in this LEB */__u8 padding2[4]; /* reserved for future, zeroes */__be64 sqnum; /* sequence number */__u8 padding3[12]; /* reserved for future, zeroes */__be32 hdr_crc; /* VID header CRC checksum */
} __attribute__ ((packed));
215 7 N860 1 3
0,1 0,P 1,0 1,1 2,0 2,2 2,Q
Every good mapped PEB has a 64-byte Volume Identifier Header
…
2 4
0,0 2,1 ...
...
...
PEBs
LEBs
![Page 22: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/22.jpg)
drivers/mtd/ubi/ubi-media.hstruct ubi_vtbl_record {
__be32 reserved_pebs; /* physical eraseblocks reserved for this volume */__be32 alignment; /* volume alignment */__be32 data_pad; /* padded bytes for the requested alignment */__u8 vol_type; /* %UBI_VID_DYNAMIC or %UBI_VID_STATIC */__u8 upd_marker; /* if volume update was started but not finished */__be16 name_len; /* volume name length */__u8 name[UBI_VOL_NAME_MAX+1]; /* volume name */__u8 flags; /* volume flags (%UBI_VTBL_AUTORESIZE_FLG) */__u8 padding[23]; /* reserved for future, zeroes */__be32 crc; /* CRC32 checksum of the record */
} __attribute__ ((packed));
225 7 N860 1 3
0,1 0,P 1,0 1,1 2,0 2,2 2,Q
A “layout volume” contains two copies of the Volume Table
…
2 4
0,0 2,1 ...
...
...
PEBs
LEBs
![Page 23: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/23.jpg)
23
SEB
0
SEB
1
……SEB
0
...SEB
205
SEB
100
…SEB
101
SEB
2
… …
SEB
522
...
...
...
PEBs
Volumes
Scan Info
Scan Volume
Scan Erase Block
“layout volume”(internal)
…corr
free
erase
alien
…
…
…
![Page 24: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/24.jpg)
24
EC hdr is written to a PEB right after the PEB is erased
drivers/mtd/ubi/wl.cstatic int sync_erase(struct ubi_device *ubi, struct ubi_wl_entry *e, int torture){
unsigned long long ec = e->ec;
[... Deleted ...]
err = ubi_io_sync_erase(ubi, e->pnum, torture);if (err < 0)
goto out_free;ec += err;if (ec > UBI_MAX_ERASECOUNTER) {
/** Erase counter overflow. Upgrade UBI and use 64-bit* erase counters internally.*/ubi_err("erase counter overflow at PEB %d, EC %llu", e->pnum, ec);err = -EINVAL;goto out_free;
}
dbg_wl("erased PEB %d, new EC %llu", e->pnum, ec);ec_hdr->ec = cpu_to_be64(ec);err = ubi_io_write_ec_hdr(ubi, e->pnum, ec_hdr);[... Deleted ...]
}
![Page 25: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/25.jpg)
25
Map a LEB L to PEB P Write VID header (with lnum L) to P
Unmap a LEB L to PEB P Schedule P for erasure
Remap a LEB L from PEB P0 to PEB P1 Schedule P0 for erasure
Write VID header (with lnum L) to P1
Copy a PEB P0 which is mapped to L to PEB P1 Write VID header (with lnum L) to P1
Copy contents of P0 to P1
Schedule P0 for erasure
![Page 26: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/26.jpg)
26
Whenever the volume table needs update
(The following speaks in the context of “layout volume”)
Unmap LEB 0
Write updated table to LEB 0
Unmap LEB 1
Write updated table to LEB 1
drivers/mtd/ubi/vtbl.cint ubi_change_vtbl_record(struct ubi_device *ubi, int idx,
struct ubi_vtbl_record *vtbl_rec){
[... Deleted ...]layout_vol = ubi->volumes[vol_id2idx(ubi, UBI_LAYOUT_VOLUME_ID)];[... Deleted ...]
memcpy(&ubi->vtbl[idx], vtbl_rec, sizeof(struct ubi_vtbl_record));for (i = 0; i < UBI_LAYOUT_VOLUME_EBS; i++) {
err = ubi_eba_unmap_leb(ubi, layout_vol, i);if (err)
return err;err = ubi_eba_write_leb(ubi, layout_vol, i, ubi->vtbl, 0,
ubi->vtbl_size, UBI_LONGTERM);if (err)
return err;return 0;
}}
![Page 27: Introduction to UBI](https://reader034.vdocuments.us/reader034/viewer/2022042503/557ea861d8b42ab04c8b5251/html5/thumbnails/27.jpg)
Every piece about MTD and UBI can be found on the MTD website
http://www.linux-mtd.infradead.org/