( 12 ) united states patent ( 10 ) patent no . : us 10 ... · us010095626b2 ( 12 ) united states...

22
US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No .: US 10 , 095 , 626 B2 ( 45 ) Date of Patent : Oct . 9 , 2018 ( 54 ) MULTIBIT NAND MEDIA USING PSEUDO - SLC CACHING TECHNIQUE ( 58 ) Field of Classification Search CPC . . . ..... .. ..... GO6F 11 / 108 ; G06F 11 / 1068 ; G06F 11 / 1072 ; G06F 12 / 0873 ; G06F 12 / 0868 ; G06F 12 / 0246 See application file for complete search history . ( 71 ) Applicant : Toshiba Memory Corporation , Minato - ku , Tokyo ( JP ) ( 56 ) References Cited U .S . PATENT DOCUMENTS ( 72 ) Inventors : Narasimhulu Dharani Kotte , Fremont CA ( US ); Senthil Thamgaraj , Fremont , CA ( US ); Robert Reed , Ei Dorado Hills , CA ( US ); Hitoshi Kondo , San Jose , CA ( US ) 7 , 953 , 931 B2 8 , 144 , 505 B2 8 , 225 , 067 B2 8 , 407 , 191 B1 5 / 2011 Yu et al . 3 / 2012 Lee 7 / 2012 Chu et al . 3 / 2013 Nanda ( Continued ) ( 73 ) Assignee : Toshiba Memory Corporation , Tokyo ( JP ) ( * ) Notice : OTHER PUBLICATIONS Subject to any disclaimer , the term of this patent is extended or adjusted under 35 U . S . C . 154 (b ) by 0 days . ( 21 ) Appl . No . : 15 / 456 , 346 San Disk Ultra II SSD review : A new level of SSD afforadability by Dong Ngo , Reviewed : Sep . 22 , 2014 ( https : // www . cnet . com / products / sandisk - ultra -ii- ssd / review ). ( Continued ) ( 22 ) Filed : Mar . 10 , 2017 ( 65 ) Prior Publication Data US 2018 / 0260331 A1 Sep . 13 , 2018 Primary Examiner Shelly A Chase ( 74 ) Attorney , Agent , or Firm White & Case LLP ( 57 ) ( 51 ) Int . Cl . GO6F 12 / 00 ( 2006 . 01 ) GOOF 12 / 0873 ( 2016 . 01 ) G06F 11 / 10 ( 2006 . 01 ) GIIC 29 / 52 ( 2006 . 01 ) G06F 12 / 0868 ( 2016 . 01 ) ( 52 ) U . S . CI . CPC .. .. .. GO6F 12 / 0873 ( 2013 . 01 ); G06F 11 / 1068 ( 2013 . 01 ); G06F 12 / 0868 ( 2013 . 01 ); G11C 29 / 52 ( 2013 . 01 ); G06F 2212 / 1016 ( 2013 . 01 ) ; G06F 2212 / 214 ( 2013 . 01 ); G06F 2212 / 222 ( 2013 . 01 ); G06F 2212 / 283 ( 2013 . 01 ); G06F 2212 / 305 ( 2013 . 01 ) ; G06F 2212 / 313 ( 2013 . 01 ); G06F 2212 / 7206 ( 2013 . 01 ) ABSTRACT A solid state drive ( SSD ) with pseudo - single - level cell ( PSLC ) caching and a method of caching data in an SSD is disclosed . In one embodiment , the SSD includes a plurality of multibit NAND media devices arranged in one or more channels communicatively coupled to a memory controller . A first portion of the plurality of multibit NAND media devices is configured to operate as a PSLC cache and a second portion of the plurality of multibit NAND media devices is configured to operate as a multibit NAND media storage area . In one embodiment , the PSLC cache comprises a first area for a PSLC write cache and a second area for a PSLC read cache . 20 Claims , 9 Drawing Sheets 500 500 Start - 502 514 Read back data written to multibit NAND LB | Read unsorted LUT corresponding to LCA / LBA Information of PSLC LB Data to be written to multibit NAND LB and write unsorted LUT into volatile memory 516 ECC Threshold Exceeded ? 504 Yes Na Sort LCA / LBA Information corresponding to the pSLCLB Data based on LBA to make pSLC LB Data sequential 506 Store sorted LCA / LBA Information corresponding to PSLC LB Data in sorted LUT in volatile memroy 518 Move multibit NAND LBs exceeding ECC Threshold to pSLC Read Cache 508 5201 Lock LUT and update with corresponding LBAS Write pSLC LB Data corresponding to sorted LUT into volatile memory 510 Write sorted PSLC LB Data into multibit NAND LB End No 512 All PSLC LB Data to be written to multibit NAND LB written ? Yes

Upload: others

Post on 28-Sep-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US010095626B2

( 12 ) United States Patent Kotte et al .

( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent : Oct . 9 , 2018

( 54 ) MULTIBIT NAND MEDIA USING PSEUDO - SLC CACHING TECHNIQUE

( 58 ) Field of Classification Search CPC . . . . . . . . . . . . . . . GO6F 11 / 108 ; G06F 11 / 1068 ; G06F

11 / 1072 ; G06F 12 / 0873 ; G06F 12 / 0868 ; G06F 12 / 0246

See application file for complete search history . ( 71 ) Applicant : Toshiba Memory Corporation ,

Minato - ku , Tokyo ( JP ) ( 56 ) References Cited

U . S . PATENT DOCUMENTS ( 72 ) Inventors : Narasimhulu Dharani Kotte , Fremont

CA ( US ) ; Senthil Thamgaraj , Fremont , CA ( US ) ; Robert Reed , Ei Dorado Hills , CA ( US ) ; Hitoshi Kondo , San Jose , CA ( US )

7 , 953 , 931 B2 8 , 144 , 505 B2 8 , 225 , 067 B2 8 , 407 , 191 B1

5 / 2011 Yu et al . 3 / 2012 Lee 7 / 2012 Chu et al . 3 / 2013 Nanda

( Continued ) ( 73 ) Assignee : Toshiba Memory Corporation , Tokyo

( JP ) ( * ) Notice : OTHER PUBLICATIONS Subject to any disclaimer , the term of this

patent is extended or adjusted under 35 U . S . C . 154 ( b ) by 0 days .

( 21 ) Appl . No . : 15 / 456 , 346

San Disk Ultra II SSD review : A new level of SSD afforadability by Dong Ngo , Reviewed : Sep . 22 , 2014 ( https : / / www . cnet . com / products / sandisk - ultra - ii - ssd / review ) .

( Continued ) ( 22 ) Filed : Mar . 10 , 2017

( 65 ) Prior Publication Data US 2018 / 0260331 A1 Sep . 13 , 2018

Primary Examiner — Shelly A Chase ( 74 ) Attorney , Agent , or Firm — White & Case LLP ( 57 )

( 51 ) Int . Cl . GO6F 12 / 00 ( 2006 . 01 ) GOOF 12 / 0873 ( 2016 . 01 ) G06F 11 / 10 ( 2006 . 01 ) GIIC 29 / 52 ( 2006 . 01 ) G06F 12 / 0868 ( 2016 . 01 )

( 52 ) U . S . CI . CPC . . . . . . GO6F 12 / 0873 ( 2013 . 01 ) ; G06F 11 / 1068

( 2013 . 01 ) ; G06F 12 / 0868 ( 2013 . 01 ) ; G11C 29 / 52 ( 2013 . 01 ) ; G06F 2212 / 1016 ( 2013 . 01 ) ; G06F 2212 / 214 ( 2013 . 01 ) ; G06F 2212 / 222 ( 2013 . 01 ) ; G06F 2212 / 283 ( 2013 . 01 ) ; G06F

2212 / 305 ( 2013 . 01 ) ; G06F 2212 / 313 ( 2013 . 01 ) ; G06F 2212 / 7206 ( 2013 . 01 )

ABSTRACT A solid state drive ( SSD ) with pseudo - single - level cell ( PSLC ) caching and a method of caching data in an SSD is disclosed . In one embodiment , the SSD includes a plurality of multibit NAND media devices arranged in one or more channels communicatively coupled to a memory controller . A first portion of the plurality of multibit NAND media devices is configured to operate as a PSLC cache and a second portion of the plurality of multibit NAND media devices is configured to operate as a multibit NAND media storage area . In one embodiment , the PSLC cache comprises a first area for a PSLC write cache and a second area for a PSLC read cache .

20 Claims , 9 Drawing Sheets

500 500 Start - 502 514

Read back data written to multibit NAND LB | Read unsorted LUT corresponding to LCA / LBA Information of PSLC LB Data to be written to multibit NAND LB and write

unsorted LUT into volatile memory 516 ECC Threshold Exceeded ? 504 Yes Na Sort LCA / LBA Information corresponding

to the pSLCLB Data based on LBA to make pSLC LB Data sequential

506 Store sorted LCA / LBA Information

corresponding to PSLC LB Data in sorted LUT in volatile memroy

518 Move multibit NAND LBs exceeding ECC Threshold to pSLC

Read Cache 508 5201

Lock LUT and update with corresponding LBAS

Write pSLC LB Data corresponding to sorted LUT into volatile memory

510 Write sorted PSLC LB Data into multibit

NAND LB End

No 512 All PSLC LB Data to be written to multibit NAND LB written ?

Yes

Page 2: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2 Page 2

( 56 ) References Cited U . S . PATENT DOCUMENTS

8 , 725 , 932 B2 * 5 / 2014 Yano . . . . . . . . . . . . . . . . G06F 11 / 1441 707 / 682

8 , 825 , 941 B2 9 / 2014 Moshayedi et al . 8 , 886 , 990 B2 11 / 2014 Meir et al . 8 , 954 , 654 B2 2 / 2015 Yu et al . 9 , 037 , 776 B25 / 2015 Yu 9 , 123 , 422 B2 9 / 2015 Yu et al . 9 , 195 , 604 B2 11 / 2015 Nemazie et al . 9 , 298 , 603 B2 3 / 2016 Schuette 9 , 405 , 621 B2 8 / 2016 Yu et al . 9 , 496 , 043 B111 / 2016 Camp et al .

2013 / 0138870 A1 5 / 2013 Yoon et al . 2014 / 0003142 A11 / 2014 Lee et al . 2014 / 0133226 AL 5 / 2014 Marotta et al . 2014 / 0219020 A1 * 8 / 2014 Kwak Glic 16 / 10

365 / 185 . 03 2015 / 0120988 A14 / 2015 Hung et al . 2015 / 0149693 Al 5 / 2015 Ng et al . 2015 / 0205664 AL 7 / 2015 Janik et al . 2015 / 0220433 Al * 8 / 2015 Schmidberger . . . . . GO6F 12 / 0246

711 / 103 2015 / 0262714 A1 9 / 2015 Tuers et al . 2016 / 0041762 A1 2 / 2016 Kanno et al . 2016 / 0225461 A1 8 / 2016 Tuers et al . 2016 / 0284393 A19 / 2016 Ramalingam

OTHER PUBLICATIONS Joe Chang , “ SSD and Internal SQL Server Tuning Strategies ” , 1 page , published Sep . 9 , 2013 .

* cited by examiner

Page 3: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

100

atent

150a

150b

150c

150d

162a - -

?????????????????

- -

152a

152b

1520

152d

162b -

110

124

Oct . 9 , 2018

-

Chhhh

120

154a

154b

1540

154d

162c -

Sheet 1 of 9

-

156a

156b

156c

156d

162d

87

130

140

FIG . 1

US 10 , 095 , 626 B2

Page 4: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

200 2010

Start Start

atent

202 -

• Raw _ Disk _ Capacity _ In _ Bytes • User _ Disk _ Capacity _ In _ Bytes

• OP _ In _ Bytes

2044

• Total _ pSLC _ Disk _ Cache _ Size = ( Raw _ Disk _ Capacity _ in _ Bytes - User _ Disk _ Capacity _ In _ Bytes - OP _ In _ Bytes ) / NAND _ Bits • PSLC Write _ cache _ Size = ( N - DWPD X Y - Years x 365 Days x User _ Disk _ Capacity _ in _ Bytes ) / Num _ PE _ Cyc _ per _ NAND _ Block PSLC _ Read _ Cache _ Size = Total _ pSLC _ Disk _ Cache _ Size - -

PSLC _ Write _ Cache _ Size

Oct . 9 , 2018

• New _ Disk _ Capacity _ Needed _ Apart _ From _ Cache =

User _ Disk _ Capacity _ in _ Bytes - Total _ pSLC _ Disk _ Cache _ Size • New _ Disk _ Capacity _ Can _ Be _ Converted _ into _ pSLC _ Cache / =

NAND _ Bits

Total PSLC _ Disk _ Cache _ Size + =

New _ Disk _ Capacity _ Can _ Be _ Converted _ into _ pSLC _ Cache

206

Sheet 2 of 9

No

New _ Disk _ Capacity _ Can _ Be _ Converted _ into _ pSLC _ Cache < Write _ Flush _ Threshold ?

208

Yes End FIG . 2 .

US 10 , 095 , 626 B2

Page 5: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

300 Initialization

Iteration 1

Iteration 2

Iteration 3

U . S . Patent atent

302

302

302

960GB ( TLC )

302

875GB ( TLC )

960GB ( TLC )

847GB ( TLC )

- wyp 304

304

304

304

64GB ( TLC )

64GB

64GB ( TLC )

64GB ( TLC )

Oct . 9 , 2018

( TLC ) 85GB ( PSLC )

306

306

3061

113GB ( PSLC )

122GB ( PSLC )

306

256GB ( TLC )

Iteration 4

Iteration 5

Final Configuration

Sheet 3 of 9

302

838GB ( TLC )

302 |

835GB ( TLC )

302 ]

835GB ( TLC )

64GB

3041 64GB

( TLC )

64GB ( TLC )

( TLC )

306

306

306

125GB ( pSLC )

126GB ( pSLC )

126GB ( pSLC )

FIG . 3

US 10 , 095 , 626 B2

Page 6: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

1 . Initialization 400

2 . Host Write to PSLC Write Cache

5 . Erase PSLC Write Cache LB

atent

LB - 01 LB - 1 LB - 2 LB - 3 4402

3 . PSLC Write Cache 4 . Move LBs Exceeding Threshold Triggered ECC Correction Threshold

IZBOR VIABOT LB - 1

LB - 1

LB - 2

LB - 2

LB - 3

LB - 3

402

NLB - O LB - 1 LB - 2 LB - 3

LB - 1 LB - 2 LB - 3

. . . . .

> 402 1

4021

* 402

Oct . 9 , 2018

LB - X - 1

LB - X - 1 LB - X LB - X + 1 LB - X + 2

LB - X - 1 LB - X LB - X + 1 LB - X + 2

LB - X LB - X + 1 LB - X + 2

LB - X - 1 LB - X AB - X + 1 LB - X + 2

LB - X - 1 LB - X - X + 1

LB - X + 2

1

404

> 404 |

> 404

> 404

404

Sheet 4 of 9

LB - Y - 1 LB - Y

LB - Y - 1 LB - Y LB - Y + 1 LB - Y + 2

LB - Y - 1 LB - Y VABAVI VIBRAND

LB - Y - 1 LB - Y VAB - 12

IVAB VABWE

LB - Y - 1 LB - Y LB - Y + 1 LB - Y + 2 LB - Y + 3

777 406

| 406

A > 406

VABXB2406 14

406

. LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

FIG . 4

US 10 , 095 , 626 B2

Page 7: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

atent

-

w

500 C Start

- 502 514

Read unsorted LUT corresponding to

Read back data written to multibit NAND LB

LCA / LBA Information of PSLC LB Data to be written to multibit NAND LB and write unsorted LUT into volatile memory

516

- 504

ECC Threshold Exceeded ?

Sort LCA / LBA Information corresponding

Yes

to the pSLC LB Data based on LBA to make pSLC LB Data sequential

| 518

Move multibit NAND

Store sorted LCA / LBA Information

LBs exceeding ECC

corresponding to PSLC LB Data in sorted

Threshold to pSLC

LUT in volatile memroy

Read Cache

508 520

Write PSLC LB Data corresponding to

Lock LUT and update with corresponding

sorted LUT into volatile memory

LBAS

- 510

Write sorted PSLC LB Data into multibit NAND LB

End

Oct . 9 , 2018

No

Sheet 5 of 9

No

512

All PSLC LB Data to be written to multibit NAND LB written ?

FIG . 5

Yes

US 10 , 095 , 626 B2

Page 8: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

U . S . Patent

610

620

630

622

612

B Unsorted LUT

PSLC LB - O

632

624

Sorted LUT

Oct . 9 , 2018

614

PSLC LB - 1

Multibit NAND LB - 0

626

LCA / LBA Information corresponding to PSLC LBS

616

PSLC LB - 2

Sheet 6 of 9

628

PSLC LB Storage

618

Multibit NAND 6341

LB - N

pSLC LB - N

FIG . 6

US 10 , 095 , 626 B2

Page 9: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

1 . Read Cache Full

2 . PSLC Write Cache Threshold Triggered

3 . Move LBs Exceeding ECC Correction Threshold

7007

4 . Erase pSLC Write Cache LB

VIBO

atent

*

* * * * * * *

* * * * *

HOND

NY

* * * * * * * * * * * * *

B2

* * *

camera

BB

EL

y

ZA 702

HA 702

B

$ 702

KAMA 702

* *

Oct . 9 , 2018

LB - X - 1 LB - X VIRB - X417

LB - X - 1 LB - X VABXIV BAXTAR

LB - X - 1 LB - X ALBAXI BXE

LB - X - 1 LB - X VABAXATI

VAB XAYU

I

$ 704

|

704

> 704

704

- -

Sheet 7 of 9

VABNWT

VABAND LBY

LB - Y + 1 LB - Y + 2

VABIA ABA I VABAYAN 1 BYTY WABXAZZA

VABAND ALBAYI LB - Y + 1 LB - Y + 2 LB - Y + 3

IRINA

706

WA _ 706 1706

\ 706

* 706

# *

*

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

FIG . 7

US 10 , 095 , 626 B2

Page 10: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

800 LBS

Host Read Counter 10 100 10000 20000

1 . Evict Victim from PSLC 2 . Move frequently accessed

Read Cache

Multibit NAND LB

VIIKBZO

VIINBLOX BABA LB - 2

atent

B - 22

CAR

YA XB - 3

802

( 802

Z

A

LITLA > 802

Oct . 9 , 2018

0

LB - X - 1 LB - X

LB - X - 1 LB - X ABMW VB MAY

LB - X - 1 LB - X

I VABAVA

11 30000

. L

804

. . .

> 804

. .

> 804

ARNAVAL

VABINI

Sheet 8 of 9

LB

VAB - XVI LB - Y WA?BX41 VXBX4212

KOULU

Y *

500 10000

L

.

806

1806

. . . . . . . . . . . . . .

> 806

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

LB - Z - 1 LB - Z

FIG . 8

US 10 , 095 , 626 B2

Page 11: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

U . S . Patent atent Oct . 9 , 2018 Oct . 9 , 2018 Sheet 9 ore Sheet 9 of 9 US 10 , 095 , 626 B2

| $ 1040

1042 1044 1000

FIG . 9

$ 940 1030 1020

942 944 946

1010

0061 930 920

FIG . 10

910

Page 12: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2

MULTIBIT NAND MEDIA USING media and beyond , a further reduction in performance will PSEUDO - SLC CACHING TECHNIQUE be likely given the probable proportional increase in read

errors necessitating more frequent use of robust ECC FIELD OF THE INVENTION mechanisms and proportional decrease in IOPs due to

5 increasing storage density . This invention is generally related to caching techniques There is , therefore , an unmet demand for SSDs addressing

for a solid state drive ( SSD ) . the performance and host - read error issues with multibit NAND media .

BACKGROUND OF THE INVENTION 10 BRIEF DESCRIPTION OF THE INVENTION

SSDs typically include flash - based memory media , such as NAND media , a flash memory controller , and a memory In one embodiment , an SSD includes a memory control buffer , such as DRAM and / or SRAM . The NAND media can ler , a plurality of multibit NAND devices arranged in one or be a single - level cell ( SLC ) media storing one bit of memory more channels communicatively coupled to the memory data per cell , or the NAND media can be multibit NAND 15 controller , one or more volatile memory devices communi media , such as multi - level cell ( MLC ) media and triple - level catively coupled to the memory controller , and a host cell ( TLC ) media storing two bits of memory data per cell interface communicatively coupled to the memory control and three bits of memory data per cell , respectively . The ler . In one embodiment , a first portion of the plurality of need for higher capacity SSDs capable of storing hundreds multibit NAND media devices is configured to operate as a to thousands of terabytes ( TBS ) of data or more drives the 20 pseudo - SLC ( SLC ) cache , and a second portion of the increasing development and adoption of multibit NAND multibit NAND media devices is configured to operate as a media . To achieve this level of storage capacity , NAND multibit NAND media storage area . In one embodiment , the media with increasing storage density is required . Develop - plurality of multibit NAND media devices are MLC NAND ment of the next - generation of multibit NAND media media devices . In another embodiment , the plurality of quad - level cell ( QLC ) media , capable of storing four bits of 25 multibit NAND media devices are TLC NAND media memory data per cell is ongoing , and it is foreseeable that devices . In yet another embodiment , the plurality of multibit flash - based memory media capable of storing an even NAND media devices are QLC NAND media devices . greater number of bits of memory data per cell will be In one embodiment , the memory controller is configured developed in the future . to write data received via the host interface to one or more

The use of increasingly dense multibit NAND media 30 blocks of the PSLC cache . In one embodiment , the memory presents a number of issues . For example , it is well known controller is configured to transfer data from the one or more that TLC NAND media is more prone to read errors than blocks of the pSLC cache to one or more blocks of the MLC NAND media , which in turn is more prone to read multibit NAND media storage area . The memory controller errors than SLC NAND media . So when a host sends data is further configured to read the data transferred to the to be written to the SSD , the data will be first buffered in the 35 multibit NAND media storage area and determine if an ECC DRAM and / or SRAM to undergo error correction parity error correction threshold has been exceeded for the data encoding before it can be written to the TLC NAND media . read , and if so , write the data exceeding the ECC error Error correction parity encoding of the data slows the correction threshold back to the PSLC cache . In one embodi performance of the SSD because it is very time consuming . ment , the memory controller is further configured to mark

Because TLC NAND media is more prone to read errors , 40 blocks of the multibit NAND media storage area exceeding it also has higher error correction requirements . For the ECC error correction threshold as bad blocks . The example , a low - density parity check ( LDPC ) error correc memory controller is configured to write data to the bad tion scheme may be able to support up to 120 bit - flips per blocks only when all other blocks of the multibit NAND one kilobyte ( KB ) of data read from the TLC NAND media , media storage area that have not been marked as bad blocks whereas a Bose - Chaudhuri - Hocquenghem ( BCH ) error cor - 45 have been occupied with data . rection scheme may be able to support up to 120 bit - flips per In one embodiment , the memory controller is further 2 KB of data read from the TLC NAND media . In the case configured to sequentially write the data contained in the one of BCH error correcting code ( ECC ) with TLC NAND or more blocks of the PSLC cache to one or more blocks of media , the SSD ' s ECC engine would likely not be able to the multibit NAND media storage area . In one embodiment , properly correct data read from the TLC NAND media on a 50 the memory controller is configured to perform an in fairly regular basis ( i . e . the BCH ECC capability is memory sorting within the one or more volatile memory exceeded ) , and a more robust ECC mechanism is required , devices of at least one set of logical block addresses ( LBAS ) such as Quadruple Swing - By Code ( QSBC ) developed by and a set of logical cluster addresses ( LCAs ) corresponding Toshiba which has ten times the error correction capability to the data contained in the one or more blocks of the PSLC of typical BCH ECC . The tradeoff , however , is that more 55 cache to be written to one or more blocks of the multibit robust ECC mechanisms such as QSBC also take longer to NAND media storage area . perform ( measured in milliseconds ( ms ) as compared to less In one embodiment , the PSLC cache includes a first area robust mechanisms such as BCH which are measured in for a PSLC write cache and a second area for a PSLC read microseconds ( us ) ) . cache . In one embodiment , the memory controller is con

Finally , due to the storage density and programming 60 figured to write data received via the host interface to one or requirements of TLC NAND media , writing data to and more blocks of the PSLC write cache . The memory control reading data from TLC NAND media also takes longer to ler is further configured to transfer data from the PSLC write perform as compared to SLC and MLC NAND media . Thus , cache to the multibit NAND media storage area , and for the typical SSDs using TLC NAND media will have substan - data transferred to the multibit NAND media storage area tially reduced input / output operations per second ( IOPs ) as 65 exceeding the ECC error correction threshold , the memory compared to SSDs using SLC NAND media . Further , for controller is configured to write such data to the pSLC read next - generation multibit NAND media , such as QLC NAND cache when the pSLC read cache is not full , and to write

Page 13: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2

such data to the PSLC write cache when the pSLC read storage area . In one embodiment , the method further cache is full . In one embodiment , the memory controller is includes determining if an ECC error correction threshold further configured to write data contained in the one or more has been exceeded for the data read , and for the data read blocks of the PSLC write cache to the multibit NAND media exceeding the ECC error correction threshold , writing such storage after a write threshold of the PSLC write cache has 5 data to the PSLC cache . In one embodiment , the method been exceeded . further includes marking blocks of the multibit NAND

In one embodiment , the storage capacity of the pSLC media storage area exceeding the ECC error correction cache is a sum of an initial storage capacity and a series of threshold as bad blocks and writing data to the bad blocks iterative storage capacities . The initial storage capacity is when all other blocks of the multibit NAND media storage equal to a result of dividing a storage capacity that remains 10 area that have not been marked as bad blocks have been when a specified user data storage capacity and an overpro - occupied with data . visioning ( OP ) storage capacity are subtracted from a total In one embodiment , the method further includes sequen storage capacity of the plurality of multibit NAND media tially writing data contained in the one or more blocks of the devices by a storage density of the multibit NAND media PSLC cache to one or more blocks of the multibit NAND devices . A first iterative storage of the series of iterative 15 media storage area . In one embodiment , the step of sequen storage capacities is equal to the initial storage capacity tially writing data to the one or more blocks of the multibit divided by the storage density of the multibit NAND media NAND media storage area includes sorting , in a volatile devices , and each subsequent iterative storage capacity of memory communicatively coupled to the plurality of mul the series of iterative storage capacities is equal to an tibit NAND media devices , at least one of a set of LBAs and immediately preceding storage capacity divided by the stor - 20 a set of LCAs corresponding to the data contained in the one age density of the multibit NAND media devices . A last or more blocks of the PSLC cache before writing the data to iterative storage capacity of the series of iterative storage one or more blocks of the multibit NAND media storage capacities is less than a write flush threshold value . area .

In one embodiment , a storage capacity of the PSLC write In one embodiment , the method further includes config cache is equal to a result of multiplying a specified number 25 uring the PSLC cache into a PSLC write cache and a PSLC of drive writes per day ( N - DWPD ) of the multibit NAND read cache and writing data received from the host device to media devices , a specified number of years of operation of one or more blocks of the PSLC write cache . The method the multibit NAND media devices , 365 days , and a specified further includes transferring the data contained in the one or user data storage capacity , and dividing the result by a more blocks of the PSLC cache to the multibit NAND media specified number of program / erase ( PE ) cycles of a block of 30 storage area , and for the data transferred to the multibit the multibit NAND media devices . A storage capacity of the NAND media storage area exceeding the ECC error correc PSLC read cache is equal to a difference of a storage tion threshold , writing such data to the pSLC read cache capacity of the PSLC cache and the storage capacity of the when the PSLC read cache is not full , and writing such data pSLC write cache . to the PSLC write cache when the pSLC read cache is full .

In one embodiment , the memory controller is further 35 In one embodiment , the method further includes writing data configured to collect host read statistics for each one of the contained in the one or more blocks of the PSLC write cache one or more blocks of the PSLC read cache and multibit to the multibit NAND media storage area after a write NAND media storage area . In one embodiment , the memory threshold of the PSLC write cache has been exceeded . controller is further configured to perform an eviction pro - In one embodiment , the step of configuring the portion of cess when the PSLC read cache is full . The eviction process 40 the plurality of multibit NAND media devices to operate as includes the memory controller identifying one or more a PSLC cache includes calculating a storage capacity of the blocks of the multibit NAND media storage area that have PSLC cache that is a sum of an initial storage capacity and a greater number of host reads than one or more blocks of a series of iterative storage capacities . In one embodiment , the PSLC read cache and swapping data contained in the one the storage capacity of the PSLC cache is a sum of an initial or more blocks of the multibit NAND media storage area 45 storage capacity and a series of iterative storage capacities . having the greater number of host reads with data contained The initial storage capacity is equal to a result of dividing a in the one or more blocks of the PSLC read cache having a storage capacity that remains when a specified user data fewer number of host reads . storage capacity and an OP storage capacity are subtracted

In one embodiment , a method of caching data in an SSD from a total storage capacity of the plurality of multibit having a plurality of multibit NAND media devices includes 50 NAND media devices by a storage density of the multibit configuring a first portion of the plurality of multibit NAND NAND media devices . A first iterative storage of the series media devices to operate as a PSLC cache and configuring of iterative storage capacities is equal to the initial storage a second portion of the plurality of multibit NAND media capacity divided by the storage density of the multibit devices to operate as a multibit NAND storage area . In one NAND media devices , and each subsequent iterative storage embodiment , the plurality of multibit NAND media devices 55 capacity of the series of iterative storage capacities is equal are MLC NAND media devices . In another embodiment , the to an immediately preceding storage capacity divided by the plurality of multibit NAND media devices are TLC NAND storage density of the multibit NAND media devices . A last media devices . In yet another embodiment , the plurality of iterative storage capacity of the series of iterative storage multibit NAND media devices are OLC NAND media capacities is less than a write flush threshold value . devices . 60 In one embodiment , the step of configuring the pSLC

In one embodiment , the method further includes receiving cache into the pSLC write cache and the PSLC read cache data from a host device connected to the SSD and writing the includes calculating a storage capacity of the PSLC write data to one or more blocks of the pSLC cache . In one cache and a storage capacity of the PSLC read cache . In one embodiment , the method further includes transferring data embodiment , a storage capacity of the PSLC write cache is from the one or more blocks of the PSLC cache to one or 65 equal to a result of multiplying a specified N - DWPD of the more blocks of the multibit NAND media storage area and multibit NAND media devices , a specified number of years reading the data transferred to the multibit NAND media of operation of the multibit NAND media devices , 365 days ,

Page 14: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2 6

and a specified user data storage capacity , and dividing the 156a - d , and 162a - d . The memory controller 120 manages result by a specified number of PE cycles of a block of the the writing , reading , and erasing of data stored on the multibit NAND media devices . A storage capacity of the multibit NAND media of the SSD 100 , and facilitates pSLC read cache is equal to a difference of a storage communication with the host device . In one embodiment , capacity of the pSLC cache and the storage capacity of the 5 the firmware of the memory controller 120 implements the pSLC write cache . flash translation layer ( FTL ) to map LBAs of data from the

In one embodiment , the method further includes collect - host to physical pages and blocks of the multibit NAND ing host read statistics for each one of the one or more blocks media , and implements garbage collection , error correction , of the pSLC read cache and the multibit NAND media and wear - leveling . The memory controller 120 may use storage area , and performing an eviction process when the 10 DRAM 130 and SRAM 140 as buffers for storing data and PSLC read cache is full . In one embodiment , the eviction look up tables ( LUTS ) , for performing error correction parity process includes identifying one or more blocks of the encoding , and the like . The error correction parity encoding multibit NAND media storage area that have a greater may be , for example , Reed Soloman ( RS ) error correction number of host reads than one or more blocks of the PSLC parity encoding or LDPC error correction parity encoding , read cache , and swapping data contained in the one or more 15 etc . blocks of the multibit NAND media storage area having the The multibit NAND media devices 150a - d , 152a - d , 154a greater number of host reads with data contained in the one d , 156a - d , and 162a - d are arranged in four channels 122 , or more blocks of the PSLC read cache having a fewer 124 , 126 , and 128 in communication with the memory number of host reads . controller 120 . While twenty multibit NAND media devices

20 ( 150a - d , 152a - d , 154a - d , 156a - d , and 162a - d ) arranged in BRIEF DESCRIPTION OF THE FIGURES four channels ( 122 , 124 , 126 , and 128 ) are shown in the SSD

100 in FIG . 1 , the specific number of multibit NAND media FIG . 1 is a block diagram of an SSD with PSLC caching , devices and channels are not limited , and may the SSD 100

according to one embodiment of the invention . may include one or more multibit NAND media devices FIG . 2 is a flowchart of steps for calculating a PSLC cache 25 arranged in one or more channels within the scope of the

size , according to one embodiment of the invention . present invention . The number of multibit NAND media FIG . 3 shows the steps of calculating the pSLC cache size devices and channels of such devices in communication with

for a given amount of additional TLC NAND media , accord the memory controller 120 within the SSD 100 will depend ing to the embodiment of FIG . 2 . on the configuration of the SSD 100 , including the overall

FIG . 4 shows the steps of processing a host write to an 30 amount of user data storage specified , the storage density SSD with PSLC caching where a PSLC read cache is not and type of multibit NAND media used , and so forth . In one full , according to one embodiment of the invention . embodiment , the multibit NAND media devices 150a - d ,

FIG . 5 is a flowchart of steps for sequentially writing data 152a - d , 154a - d , 156a - d , and 162a - d are MLC NAND media from a PSLC cache to a multibit NAND media , according to devices , or TLC NAND media devices , or QLC NAND one embodiment of the invention . 35 media devices , or a combination thereof .

FIG . 6 shows the steps of sequentially writing data from As shown in FIG . 1 , the multibit NAND media devices the PSLC cache to the multibit NAND media , according to 162a - d serve as additional storage capacity 160 ( with one one embodiment of the invention . additional multibit NAND media device 162a - d per channel

FIG . 7 shows the steps of processing a host write to an 122 , 124 , 126 , and 128 ) that is configured as PSLC NAND SSD with PSLC caching where a pSLC read cache is full , 40 media to address performance and error issues typically according to one embodiment of the invention . associated with the use of multibit NAND media devices

FIG . 8 shows the steps of a PSLC read cache eviction ( such as the multibit NAND media devices 150a - d , 152a - d , process , according to one embodiment of the invention . 154a - d , and 156a - d ) . For example , for a typical SSD with 1

FIG . 9 is a block diagram of a configuration of an SSD TB of storage capacity without pSLC caching , 960 GB with PSLC caching , according to one embodiment of the 45 would be allocated for user data and the remainder , 64 GB , invention . would be allocated as over - provisioning ( OP ) to act as an

FIG . 10 is a block diagram of a configuration of an SSD additional buffer to assist the memory controller 120 in the with PSLC caching , according to another embodiment of the management of the NAND media . The amount of OP in a invention . given SSD may vary depending on the SSD ' s application ,

50 but typically is 7 % for consumer applications ( i . e . 64 GB for DETAILED DESCRIPTION OF THE 1 TB SSD ) and 28 % for enterprise applications ( i . e . 224 GB

INVENTION for 1 TB SSD ) . To meet the same user storage capacity as would be

FIG . 1 is a block diagram of an SSD 100 with PSLC available in a typical 1 TB SSD ( i . e . 960 GB ) without PSLC caching , according to one embodiment of the invention . As 55 caching , the SSD 100 includes 256 GB of additional storage shown in FIG . 1 , SSD 100 has a host interface 110 that capacity 160 in the form of multibit NAND media devices allows the SSD 100 to be connected to a host device ( not 162a - d , for a total of 1280 GB of multibit NAND media shown ) . The host device may be any suitable device , such as devices 150a - d , 152a - d , 154a - d , 156a - d , and 162a - d . The a computer or a storage appliance . The host interface 110 multibit NAND media devices 162a - d and a portion of the may be any suitable interface that facilitates communication 60 multibit NAND media devices 150a - d , 152a - d , 154a - d , and between the SSD 100 and the host device , such as a 156a - d will form the basis for the PSLC cache for the SSD Peripheral Component Interconnect ExpressTM ( PCIe or PCI 100 . The calculation and provisioning of the multibit NAND Express ) interface , a Serial ATATM ( SATA ) interface , Serial media 150a - d , 152a - d , 154a - d , 156a - d , and 162a - d to form Attached SCSITM ( SAS ) , etc . a PSLC cache is shown and described in connection with

The SSD 100 includes a memory controller 120 in com - 65 FIG . 2 , below . munication with a DRAM 130 , an SRAM 140 , and an array FIG . 2 is a flowchart of steps 200 for calculating a PSLC of multibit NAND media devices 150a - d , 152a - d , 154a - d , cache size , according to one embodiment of the invention .

Page 15: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2

As shown in FIG . 2 , the calculation of the PSLC cache size New _ Disk _ Capacity _ Can _ Be _ Converted _ Into _ pSLC _ is an iterative process whereby an initial storage capacity of Cache is less than Write _ Flush _ Threshold , and the PSLC the PSLC cache is determined based on a storage capacity of cache size calculation concludes . the additional multibit NAND media devices that are pro - FIG . 3 shows the steps of calculating the PSLC cache size vided for the SSD , and the storage density of the multibit 5 for a given amount of additional TLC NAND media , accord NAND media devices used in the SSD . Subsequent storage ing to the embodiment of FIG . 2 . As shown in FIG . 3 , capacities may then be added to supplement the PSLC cache initially an SSD 300 has 960 GB of user storage capacity based on the remaining total capacity of the SSD ( i . e . the 302 ( User _ Disk _ Capacity _ In _ Bytes ) and 64 GB of OP 304 storage capacity provided by the multibit NAND media ( 7 % ) ( OP _ In _ Bytes ) in the form of TLC NAND media . The devices and the storage capacity of the PSLC cache ) . The 10 SSD 300 also has an additional 256 GB of TLC NAND subsequent storage capacities added to supplement the PSLC media 306 added as compared to a typical SSD without cache is determined based on the initial storage capacity of PSLC caching , for a total raw capacity of 1280 GB of TLC the pSLC cache , the storage density of the multibit NAND NAND media for the SSD 300 ( Raw _ Disk _ Capacity _ devices , and a write flush threshold of the multibit NAND In _ Bytes ) . devices . The write flush threshold is a minimum number of 15 In the first iteration of the calculation as described in step PSLC blocks that need be written to one block of the multibit 204 of FIG . 2 , the additional 256 GB of TLC NAND media NAND device , and as such , is also based on the storage 306 is first converted to PSLC cache ( i . e . an initial storage density of the multibit NAND devices as will be explained capacity of the PSLC cache ) . Because TLC NAND media in greater detail below . has a storage density of three bits per cell , the additional 256

Referring to FIG . 2 , at step 202 , the initial configuration 20 GB of TLC NAND media 306 equates to 85 GB of PSLC of an SSD with PSLC caching , such as the SSD 100 shown cache ( Total _ pSLC _ Disk _ Cache _ Size ( 85 GB ) = ( Raw _ Dis and described in FIG . 1 , is determined , including the total k _ Capacity _ In _ Bytes ( 1280 GB ) — User Disk amount of multibit NAND media contained within an SSD Capacity _ In _ Bytes ( 960 GB ) - OP _ In _ Bytes ( 64 GB ) ) / with PSLC caching ( Raw _ Disk _ Capacity _ In _ Bytes ) , the NAND _ Bits ( 3 ) ) . In other embodiments where MLC or desired amount of user storage capacity ( User _ Disk _ Ca - 25 QLC NAND media may be used , for example , NAND _ Bits pacity _ In _ Bytes ) , and the amount of OP ( OP _ In _ Bytes ) will be two and four , respectively , in those embodiments . specified . As previously discussed , OP is typically 7 % for Next , at step 206 of FIG . 2 , a portion of TLC NAND consumer applications and 28 % for enterprise applications . media 302 , originally allocated to user data , is re - allocated At step 204 , an initial pSLC cache size ( Total _ pSLC _ Dis to be converted to PSLC cache 306 ( i . e . a first subsequent k _ Cache _ Size ) is determined by subtracting the User _ Dis - 30 storage capacity added to supplement the initial storage k _ Capacity _ In _ Bytes and OP _ In _ Bytes from the Raw _ D - capacity of the PSLC cache ) , leaving 875 GB of user storage isk _ Capacity _ In _ Bytes , and dividing the result by the capacity 302 remaining ( New _ Disk _ Capacity _ Needed _ A density of the multibit NAND media ( NAND _ Bits ) used part _ From _ Cache ( 875 GB ) = User _ Disk _ Capacity _ In _ ( i . e . 3 for TLC , 4 for QLC , etc . ) . In other words , the Bytes ( 960 GB ) — Total _ pSLC _ Disk _ Cache _ Size ( 85 GB ) ) . additional multibit NAND media 162a - d of the SSD 100 of 35 This adds an additional 28 GB to the pSLC cache 306 FIG . 1 is first allocated to be converted to PSLC cache . ( New _ Disk _ Capacity _ Can _ Be _ Converted _ Into _ pSLC _

In one embodiment , the pSLC cache is further configured Cache ( 85 GB ) / NAND _ Bits ( 3 ) ) , bringing it up to 113 GB to be divided into a pSLC write cache and a PSLC read ( Total _ pSLC _ Disk _ Cache _ Size ( 113 GB ) = ( New _ Disk _ Ca cache . The PSLC write cache size ( pSLC _ Write _ pacity _ Can _ Be _ Converted _ Into _ pSLC _ Cache ( 85 GB ) / Cache _ Size ) is determined by the endurance requirements of 40 NAND _ Bits ( 3 ) + previous Total _ pSLC _ Disk _ Cache _ Size the SSD . As shown at step 204 , the PSLC write cache size ( 85 GB ) ) . At the second iteration , the New _ Disk _ Capaci is calculated by multiplying the specified N - DWPD , how ty _ Can _ Be _ Converted Into pSLC Cache is 85 GB , which many times the entire capacity of the SSD can be overwrit - is larger than the Write _ Flush _ Threshold for TLC NAND ten per day of its usable life without failure during a media ( four ) , which , as shown in FIG . 2 , means that step 206 warranty period , the number of years ( Y - Years ) of the 45 will be repeated as shown in the third iteration of FIG . 3 . In warranty period , 365 days , and the User _ Disk _ Capacity _ other embodiments where MLC or QLC NAND media may In _ Bytes , and dividing the product by the specified number be used , for example , the Write _ Flush _ Threshold will be of program / erase ( PE ) cycles that a NAND block can endure three and five , respectively , in those embodiments . without failure ( Num _ PE _ Cyc _ Per _ NAND _ Block ) . The In the third iteration , an additional 28 GB of user storage PSLC read cache will be the remainder of PSLC 50 capacity 302 is allocated to be converted to PSLC cache 306 Write _ Cache _ Size subtracted from the Total _ pSLC _ ( i . e . another subsequent storage capacity added to further Disk _ Cache _ Size . supplement the initial storage capacity of the PSLC cache ) , At step 206 , further multibit NAND media is allocated for leaving 847 GB of user storage capacity 302 remaining . The

the PSLC cache ( New _ Disk _ Capacity _ Needed _ additional 28 GB of TLC NAND media is then converted Apart _ From _ Cache ) by subtracting the Total _ pSLC _ 55 into PSLC cache 306 , adding an additional 9 GB to PSLC Disk _ Cache _ Size from the User _ Disc _ Capacity _ In _ Bytes , cache 306 , bringing the total pSLC cache 306 to 122 GB . which is then divided by the density of the multibit NAND Again , the New _ Disk _ Capacity _ Can _ Be _ Converted _ In media , NAND _ Bits , resulting in New _ Disk _ Capacity to _ pSLC _ Cache after the third iteration is 28 GB , which is Can Be Converted Into _ pSLC _ Cache , which is then still larger than the Write Flush Threshold . In the fourth added to the Total _ pSLC _ Disk _ Cache _ Size . In this step , a 60 iteration , an additional 9 GB of TLC NAND media is portion of the multibit NAND media 150a - d , 152a - d , 154a - converted into PSLC cache 306 , adding an additional 3 GB d , and 156a - d of the SSD 100 of FIG . 1 is allocated to be to PSLC cache 306 , bringing the total PSLC cache s306 to converted to pSLC cache as well . 125 GB and leaving 838 GB of user storage capacity 302 At step 208 , if the New _ Disk _ Capacity _ Can _ Be _ Con - remaining .

verted _ Into _ pSLC _ Cache is greater than the capacity 65 Once again , the New _ Disk _ Capacity _ Can _ Be _ Convert required for a multibit NAND media write flush e d _ Into _ pSLC _ Cache at the fourth iteration is 9 GB , which ( Write _ Flush _ Threshold ) , then step 206 is repeated until is still larger than the Write _ Flush _ Threshold . In the fifth

Page 16: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2 10

iteration , an additional 3 GB of user storage capacity 302 is read cache 404 ( LB - X + 1 to LB - Y ) ; and ( 3 ) a PSLC write allocated to be converted to pSLC cache 306 , leaving 835 cache 406 ( LB - Y + 1 to LB - Z ) . As previously discussed , the GB of TLC NAND media 302 remaining . The additional 3 PSLC read cache 404 and the pSLC write cache 406 are GB of TLC NAND media is then converted into PSLC cache provisioned multibit NAND media configured to operate as 306 , adding an additional 1 GB to PSLC cache 306 , bringing 5 PSLC NAND media according to the PSLC cache calcula the total pSLC cache 306 to 126 GB . At iteration 5 , the tion shown and described in FIG . 2 . New _ Disk _ Capacity _ Can _ Be _ Converted Into _ pSLC During operation , when the PSLC read cache 404 is not Cache is 3 GB , which is less than the Write _ Flush _ Thresh - full , a memory controller ( not shown ) of the SSD 400 writes old and , as such , the PSLC cache size calculation concludes . data received from a host device coupled to the SSD 400 in In the final configuration , the SSD 300 with PSLC caching 10 the physical blocks of the PSLC write cache 406 ( denoted by has a total of 835 GB of user storage capacity 302 , 64 GB shaded LB ' s LB - Y + 1 to LB - Y + 3 ) . In one embodiment , the of OP 304 , and 126 GB of PSLC cache 306 , for a total of 961 PSLC write cache 406 is configured as a ping pong buffer , GB of storage . whereby the memory controller continues to write host data

In one embodiment , the 126 GB of PSLC cache 306 is to the pSLC write cache 406 until a write threshold is allocated between a pSLC write cache and a pSLC read 15 reached , at which time the memory controller writes a cache . As previously described at step 204 in FIG . 2 , the size portion of the data written to the pSLC write cache 406 to of the PSLC write cache is determined by the endurance the multibit NAND media storage area 402 , and erases that requirements of the SSD 300 . For example , assuming that data from the PSLC write cache 406 in order to maintain the TLC NAND media has a DWPD of three , a requirement sufficient storage space in the PSLC write cache 406 for of three years of operation , and 50 , 000 PE cycles , then the 20 subsequent writes from the host device . PSLC write cache size is about 64 GB ( ( N - DWPD ( 3 ) xY - By configuring the pSLC write cache 406 as a ping pong Years ( 3 ) x365 daysxUser _ Disk _ Capacity _ In _ Bytes ( 960 buffer , the speed of host writes to the PSLC write cache 406 GB ) ) / Num _ PE _ Cyc _ Per _ Nand _ Block ( 50 , 000 ) ) . The pSLC can be maintained . This is because the time to write data to read cache size will be 62 GB ( Total _ pSLC _ multibit NAND media configured as PSLC NAND media , Disk _ Cache _ Size ( 126 GB ) - PSLC _ Write _ Cache _ Size ( 6425 such as the pSLC write cache 406 , is typically much shorter GB ) ) . than the time it would take to erase the data . For example ,

The following tables summarize the PSLC cache sizing of for PSLC write cache 406 configured from TLC NAND various TLC NAND media based SSDs with typical storage media , it only takes 100 us to program the lower page of the capacities according to various embodiments of the present TLC NAND media ( which is the only page that is pro invention . These tables assume the TLC NAND media has 30 gramed in order to operate TLC NAND media as PSLC a DWPD of three , requires three years of operation , and have NAND media ) , whereas it takes about 10 - 11 us to erase just an endurance of 50 , 000 PE cycles . 4 KB of data . Depending on the specific capacity of the TLC

NAND media , an erase operation may be in the ms range . TABLE 1 By not operating the PSLC write cache 406 as a ping pong

35 buffer , the performance of the SSD 400 will be dramatically TLC NAND Media Based SSDs with 7 % OP reduced as the pSLC 406 write cache will be quickly

overwhelmed by writes from the host device . User Raw TLC pSLC pSLC Storage In one embodiment , the write threshold of the PSLC write

Data Read Write Capacity OP cache 406 is determined based on the bandwidth of the host ( TLC ) Capacity Cache Cache ( TLC + PSLC ) ( TLC ) 40 interface between the SSD 400 and the host device and the 640 GB 420 GB 30 GB 31 GB 480 GB 32 GB bandwidth of the multibit NAND media configured as PSLC 1280 GB 835 GB 62 GB 64 GB 960 GB 64 GB NAND media . For example , assuming the host interface is 2560 GB 1670 GB 124 GB 126 GB 1920 GB 128 GB a PCIe interface with four PCIe lanes each operating at 8 5120 GB 3332 GB 257 GB 252 GB 3840 GB 256 GB Gbps , the host interface has a total bandwidth of 32 Gbps ,

or 4 GBps ( 32 Gbps / 8 ) . Assuming a performance variation tolerance of 20 % , the actual speed of the host interface may

TABLE 2 be about 3 . 2 GBps . The bandwidth of the PSLC NAND media depends on the type of multibit NAND media used to

TLC NAND Media Based SSDs with 28 % OP configure the pSLC NAND media , as well as the specific 50 configuration of the multibit NAND media ( i . e . number of User

Raw TLC PSLC PSLC Storage die , number of channels , etc . ) . For example , in a one - die Capacity Data Read Write Capacity eight - channel configuration using TLC NAND media , PSLC ( TLC ) Capacity Cache Cache ( TLC + PSLC ) ( TLC ) NAND media configured from the TLC NAND media will 640 GB 340 GB 35 GB 26 GB 400 GB 112 GB have write speeds of about 1 . 4 GBps . Thus , the PSLC write 1280 GB 675 GB 74 GB 52 GB 800 GB 224 GB 55 cache 406 has a fill ratio of 2 . 3x ( 3 . 2 GBps / 1 . 4 GBps ) . 2560 GB 1348 GB 147 GB 105 GB 1600 GB 448 GB Based on the rate at which the pSLC write cache 406 can 5120 GB 2693 GB 298 GB 210 GB 3200 GB 896 GB be filled by the host device , the write threshold should be

greater than the size of the PSLC write cache 406 divided by FIG . 4 shows the steps of processing a host write to an the fill ratio . Assuming the SSD 400 is the TLC NAND

SSD 400 with PSLC caching when a pSLC read cache 404 60 media based SSD with 7 % OP having a raw TLC capacity is not full , according to one embodiment of the invention . As of 1280 GB of Table 1 , above , the write threshold is 27 GB shown in FIG . 4 , the SSD 400 with PSLC caching is initially ( 64 GB PSLC write cache / 2 . 3x fill ratio ) . This means that at configured into three different storage areas : ( 1 ) a multibit any given point in time , at least 27 GB of the pSLC write NAND media storage area 402 ( logical blocks ( LB ) LB - 0 to cache 406 needs to be available to receive writes from the LB - X ) comprising the multibit NAND media data capacity 65 host device . To ensure the pSLC write cache 406 has and the OP following the PSLC cache calculation and sufficient space to handle writes from the host device , after provisioning shown and described in FIG . 2 ; ( 2 ) a PSLC the write threshold is exceeded , the data written to the blocks

OP

Page 17: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2 ) 12

of the PSLC write cache 406 are subjected to error correction virtually eliminated , which as previously explained is more parity encoding and moved to the multibit NAND media time - consuming . This is because pSLC NAND media is storage area 402 ( denoted by shaded LB - 0 ) , and subse much more robust than multibit NAND media , requiring quently erased from the PSLC write cache 406 . virtually no error correction . Thus , in most instances , data

In one embodiment , the memory controller sequentially 5 that would have otherwise been subject to time - consuming writes the data from the PSLC write cache 406 to the QSBC error correction and decoding if read from the multibit NAND media storage area 402 , as shown in FIGS . multibit NAND media , are simply read from the PSLC read 5 and 6 , as described in greater detail below . Sequentially cache 404 instead without the need for error correction . writing the data to the multibit NAND media storage area Moreover , the data remaining in the multibit NAND media 402 from the PSLC write cache 406 significantly reduces the 10 402 are also confirmed to be correctable using BCH ECC . need to perform garbage collection ( at least one order of This means that the read performance of the SSD 400 will magnitude ) , further reducing the maintenance overhead of be greatly improved because the use of QSBC will be greatly the SSD 400 and improving performance . reduced .

The memory controller subsequently reads back the data The error correction parity encoding and movement of written to the physical blocks of the multibit NAND media 15 data from the pSLC write cache 406 to the multibit NAND storage area 402 ( i . e . the data previously contained in the media 402 , the read of the data written to the multibit NAND physical blocks corresponding to LBs LB - Y + 1 to LB of the media 402 to check if the ECC threshold has been exceeded , PSLC write cache 406 that were written to the physical and the subsequent movement of any data in blocks exceed blocks of the multibit NAND media storage area 402 ing the ECC threshold to the PSLC read cache 404 may be corresponding to LB - 0 ) to determine if the ECC error 20 performed as background operations of the SSD 400 so that correction threshold is exceeded . If so , the data in those the read / write performance of the SSD 400 is not affected blocks of the multibit NAND media storage area 402 when carrying out the process shown and described in FIG . exceeding the ECC error correction threshold are written to 4 . In other words , the steps of processing a host write to an blocks of the PSLC read cache 404 ( denoted by the shaded SSD 400 with PSLC caching where a pSLC read cache 404 region of LB - X + 1 ) . The memory controller marks blocks of 25 is transparent to a host device . From the host device ' s the multibit NAND media storage 402 exceeding the ECC perspective , there will be no appreciable performance dif error correction threshold as " bad " blocks , similar to bad ference between the SSD 400 with PSLC caching as com block management , and the memory controller avoids writ - pared to a conventional SSD device based on SLC NAND ing to those " bad " blocks of the multibit NAND media media . storage area 402 until all other blocks that have not been 30 FIG . 5 is a flowchart of method steps 500 for sequentially marked as “ bad ” blocks have been filled . writing data from a pSLC cache to a multibit NAND media ,

Once the data is written to the multibit NAND media according to one embodiment of the invention . As previ storage area 402 and the PSLC read cache 404 ( if appli ously discussed with respect to FIG . 4 , in one embodiment , cable ) , the memory controller erases the LBs previously by sequentially writing data from the PSLC write cache 406 storing the data in the PSLC write cache 406 . This process 35 to the multibit NAND media 402 after the write threshold is continues until the pSLC read cache 406 is full . The pro - reached , the need for garbage collection for the SSD 400 is cessing of a host write to an SSD with PSLC caching where greatly reduced . a PSLC read cache is full is illustrated in FIG . 7 and To carry out the sequential writing process , at step 502 , a explained in greater detail below . memory controller reads one or more look - up tables ( LUTS ) By processing host writes in the manner shown in FIG . 4 , 40 containing the LCAs and the LBAS ( as necessary ) corre

the SSD 400 with PSLC caching realizes a number of sponding to the data in the pSLC write cache to be moved benefits over a conventional SSD device without PSLC to the multibit NAND media and writes the one or more caching . By initially writing the host data to the PSLC write LUTs to a volatile memory . In one embodiment , the volatile cache 406 rather than the multibit NAND media storage area memory is DRAM . In another embodiment , the volatile 402 ( as would have been done in conventional SSDs ) , the 45 memory is SRAM . At step 504 , the memory controller sorts SSD 400 will have speeds similar to as if it were an SLC the unsorted LUT ( s ) in the volatile memory such that the NAND media based SSD device rather than a multibit LCAs / LBAs corresponding to the data are sequential . At NAND media based SSD device . This is because the data step 506 , the memory controller stores the sorted LUT ( S ) does not need to be error correction parity encoded at the separately from the unsorted LUT ( s ) in the volatile memory . time it is written to the pSLC NAND media ( rather , the data 50 The memory controller maintains the unsorted LUT ( S ) in the is error correction parity encoded when it is written to the volatile memory until the data has successfully been sequen multibit NAND media ) and PSLC NAND media has greatly tially written to the multibit NAND media in the event the improved performance speeds as compared to multibit SSD encounters an unexpected error or failure event during NAND media . For example , the average write time for the sequential writing process . PSLC NAND media configured from TLC NAND media is 55 At step 508 , the memory controller writes data from the about 100 us , compared with about 3 ms for TLC NAND PSLC write cache to the volatile memory based on the sorted media . Read times are also improved , with the average read LUT ( s ) so that the data is stored sequentially in the volatile time of pSLC NAND media at about 30 us , compared with memory . At step 510 , the memory controller writes the about 80 us for TLC NAND media . Moreover , with the sequential data in the volatile memory to the multibit NAND PSLC write cache 406 configured as a ping pong buffer , the 60 media . At step 512 , the memory controller checks to deter PSLC NAND media speeds of the SSD 400 can be sus - mine if all of the data to be written from the pSLC write tained . cache to the multibit NAND media has been written . If not ,

Additionally , by writing blocks that exceed the multibit steps 502 to 510 are repeated until the memory controller has NAND media ECC error correction capabilities ( such as completed writing all of the data to be written form the exceeding the BCH error correction threshold ) to the pSLC65 PSLC write cache to the multibit NAND media . read cache 404 , the need for using a more robust error Once the memory controller has written all of the data to correction mechanism such as QSBC for host reads is be written from the PSLC write cache to the multibit NAND

Page 18: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

13 US 10 , 095 , 626 B2

14 media , at step 514 , the memory controller reads back the into a block of the multibit NAND media 630 , such as the data written to the multibit NAND media to check to see any multibit NAND LB - 0 632 . As previously discussed in con of the written blocks of data exceed the ECC error correction nection with FIGS . 4 and 5 , once the memory controller threshold . If so , at step 518 , the memory controller marks the writes data from the PSLC write cache 610 to a block of the blocks of the multibit NAND media exceeding the ECC 5 multibit NAND media 630 , the memory controller reads the error correction threshold as “ bad ” blocks , and writes the data back to determine if the block exceeds the ECC error data contained within those “ bad ” blocks to the PSLC read correction threshold . The memory controller buffers any cache . Once the memory controller has written all of the data blocks exceeding the ECC error correction threshold in the contained within the “ bad ” blocks of the multibit NAND DRAM 620 and subsequently writes them to the pSLC read media to the PSLC read cache , or if there were no blocks of 10 cache ( not shown ) . After the memory controller sequentially the multibit NAND media exceeded the ECC error correc - writes the data to the multibit NAND media 630 , and moves tion threshold , then at step 520 , the memory controller any blocks exceeding the ECC error correction threshold to updates the original LUT ( S ) containing the LCAs / LBAS the PSLC read cache , the memory controller updates the corresponding to the data originally contained within the LCA / LBA information of the data now residing in the PSLC cache with the new LCA / LBA information of the data 15 multibit NAND media and read PSLC cache ( if applicable ) now written to the multibit NAND media and read PSLC in the corresponding LUT . read cache , if any , and locks the LUT ( s ) . Again , as previously mentioned , by sequentially writing

FIG . 6 shows the steps of sequentially writing data from the data from the pSLC write cache 610 to the multibit the PSLC write cache 610 to the multibit NAND media 630 , NAND media 630 the need to perform garbage collection is according to one embodiment of the invention . As an 20 greatly reduced , thereby reducing the overhead maintenance example , FIG . 6 will be described in connection with the and processing requirements of the SSD with PSLC caching embodiment shown in FIG . 5 . As shown in FIG . 6 , the PSLC and improving overall performance . Moreover , the sequen write cache 610 includes a number of LBs PSLC LB - 0 612 tial writing of the data from the PSLC write cache 610 to the to pSLC LB - N 618 , and the multibit NAND media 630 multibit NAND media 630 may take place in the back includes a number of LBs multibit NAND LB - 0 632 to 25 ground of the operation of the SSD with PSLC caching , multibit NAND LB - N 634 . While shown separately , as thereby minimizing any impact on performance of the previously discussed in connection with FIGS . 1 , 3 , and 4 device due to the sequential writing process . above , the pSLC write cache 610 and the multibit NAND FIG . 7 shows the steps of processing a host write to an media 630 are part of the same SSD with PSLC caching SSD 700 with PSLC caching where a PSLC read cache 704 ( along with OP and PSLC read cache , in one embodiment ) . 30 is full , according to one embodiment of the invention . As As previously discussed in connection with FIG . 5 above , shown in FIG . 7 , similar to FIG . 4 , the SSD 700 with PSLC

the sequential write process begins with the memory con - caching is initially configured into three different storage troller reading the LCA / LBA information corresponding to areas : ( 1 ) a multibit NAND media storage area 702 ( logical the data in the PSLC write cache 610 to be written to the blocks ( LB ) LB - 0 to LB - X ) comprising the multibit NAND multibit NAND media 630 . The memory controller writes 35 media data capacity and the OP following the PSLC cache the LCA / LBA information into a storage area 626 of DRAM calculation and provisioning shown and described in FIG . 2 ; 620 . The amount of LCA / LBA information the memory ( 2 ) a PSLC read cache 704 ( LB - X + 1 to LB - Y ) ; and ( 3 ) a controller reads and then writes into the storage area 626 of PSLC write cache 706 ( LB - Y + 1 to LB - Z ) . As previously DRAM 620 depends on the storage density of the multibit discussed , the pSLC read cache 704 and the PSLC write NAND media 630 . For example , if the multibit NAND 40 cache 706 are provisioned multibit NAND media configured media 630 is TLC NAND media , then the memory control - to operate as PSLC NAND media according to the PSLC ler reads three LBs worth of LCA / LBA information ( for cache calculation shown and described in FIG . 2 . example , LCA / LBA information corresponding to pSLC During operation , when the PSLC read cache 704 is full LB - O 612 to PSLC LB - 2 616 ) and writes them into the ( denoted by the shaded LBs LB - X + 1 to LB - Y ) , a memory storage area 626 of DRAM 620 as all three pages of TLC 45 controller ( not shown ) of the SSD 700 first stores write data NAND media must be written at the same time . In other received from a host device coupled to the SSD 700 in embodiments where the multibit NAND media 630 is MLC physical blocks of the pSLC write cache 706 ( denoted by NAND media or QLC NAND media , then the memory shaded LB ' s LB - Y + 1 to LB - Y + 3 ) similar to when the SSD controller reads two or four LBs worth of LCA / LBA infor - 700 with PSLC caching receives a write data from the host mation and writes them into the DRAM 620 , respectively . 50 device when the PSLC read cache 704 is not full as

Once the memory controller writes the LCA / LBA infor - described in connection with FIG . 4 , above . Again , the mation into the storage area 626 of the DRAM 620 , the PSLC write cache 706 may be configured as a ping pong memory controller reads the LCA / LBA information of the buffer . Once the write threshold of the PSLC write cache 706 data to be written from the pSLC write cache 610 to the is exceeded , the memory controller error correction parity multibit NAND media 630 and writes that information into 55 encodes a portion of the data written to the pSLC write cache an unsorted LUT stored in a storage area 622 of the DRAM 706 and writes that data to the multibit NAND media storage 620 . The memory controller sequentially sorts the LCA area 702 . LBA information in the unsorted LUT within the DRAM As with the operation of the SSD 400 with PSLC caching 620 and writes the sequentially sorted LCA / LBA informa - of FIG . 4 , in one embodiment , the memory controller tion into a sorted LUT stored in a storage area 624 of the 60 sequentially writes the data to the multibit NAND media DRAM 620 . Subsequently , memory controller writes the storage area 702 . In one embodiment , the memory controller data in the PSLC LBs of the PSLC write cache 610 into a sequentially writes the data in the manner described in PSLC LB storage area 628 of the DRAM 620 based on the connection with FIGS . 5 and 6 , above . After the memory sequential LCA / LBA information in the sorted LUT . controller writes the data to LBs of multibit NAND media

After the memory controller sequentially writes the data 65 storage area 702 , the memory controller reads back the data into the pSLC LB storage area 628 of the DRAM 620 , the to determine if the ECC error correction threshold has been memory controller then writes the sequentially sorted data exceeded .

Page 19: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

15 US 10 , 095 , 626 B2

16 If any blocks of the multibit NAND media storage area As shown in FIG . 8 , when the pSLC read cache 804 is

702 exceed the ECC error correction threshold , the memory full , a patrol operation of the memory controller scans the controller marks those blocks as “ bad ” and writes the data host read statistics in the Host _ Read _ Counter to determine contained in those “ bad ” blocks of the multibit NAND the less frequently read data in the PSLC read cache 804 ( i . e . media storage area 702 back to the PSLC write cache 706 , 5 the “ victim ” ) , such as LB - Y of the pSLC read cache 804 rather than to the PSLC read cache 704 as was the case as which has only been read by the host device twice during shown and described in connection with FIG . 4 where the this boot lifetime . The patrol operation similarly identifies PSLC read cache 404 of the SSD 400 was not full . The the LB s of the multibit NAND media storage area 802 that memory controller avoids writing to the " bad " blocks in are read more frequently , such as LB - 2 and LB - 3 which have subsequent writes to the multibit NAND media storage area 10 10000 and 20000 host device reads , respectively . The 702 until all other blocks that have not been marked as “ bad ” memory controller then swaps the data in the least read LBs

of the PSLC read cache 804 with the data in the LBs of the blocks have been filled . Once the memory controller writes multibit NAND media storage area 802 that are read more the data to the multibit NAND media storage area 702 and frequently . back to the pSLC write cache 706 ( if applicable ) , the 15 ! applicable ) , the 15 The memory controller accomplishes this by first writing memory controller erases the LBs previously storing the the data in the least read LBs of the PSLC read cache 804 data in the PSLC write cache 706 . into free LBs of the PSLC multibit NAND media storage

Similar to writing the data in the blocks of the multibit area 802 . After the data has been written to the pSLC NAND media storage area 402 exceeding the ECC error multibit NAND media storage area 802 , the memory con correction threshold to the pSLC read cache 404 as 20 troller erases the PSLC read cache 804 LBs , writes the more described in FIG . 4 , by writing the data in the blocks of the frequently accessed LBs of the multibit NAND media multibit NAND media storage area 702 exceeding the ECC storage area 802 into the LBs of the PSLC read cache 804 error correction threshold back to the pSLC write cache 706 that have been evicted , and erases the data from the LBs of the SSD 700 will continue to have speeds similar to as if it the multibit NAND media storage area 802 . In this manner , were an SLC NAND media based SSD device rather than a 25 when full , the pSLC read cache 804 stores only the more multibit NAND media based SSD device despite the pSLC frequently accessed LBs out of the LBs comprising the read cache 704 being full . Thus , the performance of the SSD multibit NAND media storage area 802 and the PSLC read 700 with PSLC caching can be maintained by using the cache 804 by evicting those less frequently accessed LBs to PSLC write cache 706 when the PSLC read cache 704 is full . the multibit NAND media storage area 802 . Moreover , because blocks that exceed the multibit NAND 30 The eviction process for the pSLC read cache 804 serves media ECC error correction threshold are written to the a number of purposes . It frees up limited PSLC storage space PSLC write cache 704 , there remains virtually no need to use in the SSD 800 that , as previously discussed , has superior a more time - consuming error correction mechanism such as performance characteristics as compared to the multibit QSBC for host device reads . NAND media used to configure the pSLC read cache 804

FIG . 8 shows the steps of a pSLC read cache 804 eviction 35 and the pSLC write cache 806 . Thus , by storing more process , according to one embodiment of the invention . As frequently accessed data in the PSLC read cache 804 , shown in FIG . 8 , an SSD 800 with PSLC caching is again performance of the SSD 800 will also be improved because configured into three different storage areas : ( 1 ) a multibit there will be a reduced need to perform time - consuming NAND media storage area 802 ( logical blocks ( LB ) LB - 0 to error correction on the data most frequently accessed by the LB - X ) comprising the multibit NAND media data capacity 40 host device because , as previously discussed , PSLC NAND and the OP following the pSLC cache calculation and media is much more robust than multibit NAND media and provisioning shown and described in FIG . 2 ; ( 2 ) a PSLC requires virtually no error correction . read cache 804 ( LB - X + 1 to LB - Y ) ; and ( 3 ) a PSLC write Multibit NAND media configured as PSLC NAND media cache 806 ( LB - Y + 1 to LB - Z ) . The SSD 800 also includes a also will have improved P / E characteristics as compared to Host _ Read _ Counter used to maintain statistics on the num - 45 the multibit NAND media , meaning that the PSLC NAND ber of times each LB of the SSD 800 is read or accessed by media will have better endurance and longevity than the a host device . The Host _ Read _ Counter may be stored in a multibit NAND media from which it is configured . For volatile memory of the SSD 800 , and may be maintained on example , TLC NAND media is typically specified to have a per - boot basis of the SSD 800 to record the number of 10 , 000 P / E cycles , whereas PSLC NAND media configured reads of each particular LB for the lifetime of the boot . At 50 as TLC NAND media can have 60 , 000 P / E cycles . Thus , by the outset , the Host Read Counter for each LB is set to ( on storing the more frequently accessed data in the PSLC read every boot . cache 804 , the overall longevity of the SSD 800 can be

During operation of the SSD 800 , as a host device ( not improved . shown ) connected to the SSD 800 writes data to the SSD 800 FIG . 9 is a block diagram of a configuration of an SSD ( denoted by the shaded LBs of the multibit NAND media 55 900 with PSLC caching , according to one embodiment of the storage area 802 , the PSLC read cache 804 , and the PSLC invention . As shown in FIG . 9 , the SSD 900 includes a write cache 806 ) a memory controller ( not shown ) of the memory controller 910 that is communicatively coupled to SSD 800 updates the host read statistics of each LB each volatile memory buffers 920 and 930 . In one embodiment , time the host device reads from a given LB of the SSD 800 . volatile memory buffer 920 is an SRAM device and volatile For example , as shown in FIG . 8 , LB - 0 of the multibit 60 memory buffer 930 is a DRAM device . The volatile memory NAND media storage area 802 has been read by the host buffers 920 and 930 are communicatively coupled to an device 10 times during the current boot lifetime of the SSD array of multibit NAND media 940 configured as a multibit 800 , LB - 0 has been read 100 times , LB - 02 has been read NAND media storage area 942 , a PSLC read cache 944 , and 10000 times , and so forth . When the pSLC read cache 804 a PSLC write cache 946 . is full , the memory controller performs an eviction process 65 During operation of the SSD 900 , the memory controller to evict less frequently accessed ( i . e . , read ) data in the PSLC 910 buffers host writes in volatile memory buffer 920 ( i . e . read cache 804 . SRAM ) , and without performing error correction parity

Page 20: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

17 US 10 , 095 , 626 B2

18 encoding , writes the host write data to the PSLC write cache cache 1044 is exceeded , the memory controller 1010 buffers 946 . As previously discussed , the pSLC write cache 946 blocks of write data from the pSLC write cache 1044 in may be configured as a ping pong buffer . Once a write volatile memory buffer 1030 and writes the write data to the threshold of the pSLC write cache 946 is exceeded , the multibit NAND media storage area 1042 as shown and memory controller 910 buffers blocks of write data from the 5 described in FIGS . 4 and 7 . In one embodiment , the memory PSLC write cache 946 in volatile memory buffer 930 ( i . e . controller 1010 performs an in - memory sorting in the vola DRAM ) , and writes the write data to the multibit NAND tile memory buffer 1030 of the LCAs / LBAs corresponding media storage area 942 as shown and described in FIGS . 4 to the data blocks to be written to the multibit NAND media and 7 . In one embodiment , the memory controller 910 storage area 1042 in order to sequentially write the data to performs an in - memory sorting in the volatile memory 10 the multibit NAND media storage area 1042 as shown and buffer 930 of the LCAs / LBAs corresponding to the data described in FIGS . 5 and 6 . blocks to be written to the multibit NAND media storage area 942 in order to sequentially write the data to the multibit After reading back the blocks of data written to the

multibit NAND media storage area 1042 , the memory NAND media storage area 942 as shown and described in FIGS . 5 and 6 . controller 1010 marks any blocks exceeding the ECC error

After reading back the blocks of data written to the correction threshold as “ bad ” blocks in the multibit NAND multibit NAND media storage area 942 . the memory con - media storage area 1042 , buffers the data contained in those troller 910 marks any blocks exceeding the ECC error " bad " blocks back into the volatile memory buffer 1030 , and correction threshold as " bad " blocks , buffers the data con writes the data into the read cache area of the volatile tained in those “ bad ” blocks back into the volatile memory 20 memory buffer 1030 if it is not full and into the PSLC write buffer 930 , and writes the data to the PSLC read cache 944 cache 1044 if the read cache area of the volatile memory if the PSLC read cache 944 is not full and to the PSLC write buffer 1030 is full . cache 946 if the pSLC read cache 944 is full , as previously As with the pSLC read cache , when the read cache area shown and described in FIGS . 4 and 7 . When the pSLC read of the volatile memory buffer 1030 is full , the memory cache 944 is full , the memory controller 910 performs an 25 controller 1010 may perform an eviction process to swap eviction process to swap less frequently accessed blocks of less frequently accessed blocks of data from the read cache data from the pSLC read cache 944 with more frequently area of the volatile memory buffer 1030 with more fre accessed blocks of data from the multibit NAND media quently accessed blocks of data from the multibit NAND storage area 942 as previously shown and described in FIG media storage area 1042 , similar to the eviction process 8 . The volatile memory buffer 930 may be used to buffer the 30 shown and described in FIG . 8 . The volatile memory buffer data of the blocks being swapped during the eviction pro - 1030 may be used to buffer the data of the blocks being cess . The volatile memory buffer 930 may also store host swapped during the eviction process . The volatile memory read statistics for the blocks of the PSLC read cache 944 and buffer 1030 may also store host read statistics for the blocks multibit NAND media storage area 942 for the eviction of the read cache area of the volatile memory buffer 1030 process . 35 and multibit NAND media storage area 1042 for the eviction

FIG . 10 is a block diagram of a configuration of an SSD process . 1000 with PSLC caching , according to another embodiment Other objects , advantages and embodiments of the vari of the invention . Similar to the SSD 900 of FIG . 9 , the SSD ous aspects of the present invention will be apparent to those 1000 includes a memory controller 1010 that is communi - who are skilled in the field of the invention and are within catively coupled to volatile memory buffers 1020 and 1030 . 40 the scope of the description and the accompanying Figures . In one embodiment volatile memory buffer 1020 is an For example , but without limitation , structural or functional SRAM device and volatile memory buffer 1030 is a DRAM elements might be rearranged , or method steps reordered , device . The volatile memory buffers 1020 and 1030 are consistent with the present invention . Similarly , principles communicatively coupled to an array of multibit NAND according to the present invention could be applied to other media 1040 configured as a multibit NAND media storage 45 examples , which , even if not specifically described here in area 1042 , a pSLC write cache 1044 . detail , would nevertheless be within the scope of the present

Instead of further configuring multibit NAND media 1040 invention . to include a pSLC read cache , such as the PSLC read cache 944 as shown in FIG . 9 , the SSD 1000 uses a portion of the What is claimed is : storage area of the volatile memory buffer 1030 as a read 50 1 . A solid state drive ( SSD ) comprising : cache . The read cache area of the volatile memory buffer a memory controller ; 1030 may be configured to operate in a similar manner as a a plurality of multibit NAND media devices arranged in PSLC read cache , as explained in greater detail below . The one or more channels communicatively coupled to the amount of storage space of the volatile memory buffer 1030 memory controller ; to be allocated as the read cache may vary depending on a 55 one or more volatile memory devices communicatively number of considerations , including the overall storage coupled to the memory controller ; and capacity of the SSD 1000 and the desired performance and a host interface communicatively coupled to the memory longevity of the SSD 1000 . For example , where the SSD controller , wherein 1000 has a storage capacity of 1 TB , 256 MB of the volatile a first portion of the plurality of multibit NAND media memory buffer 1030 may be allocated as a read cache . 60 devices is configured to operate as a pseudo - single

During operation of the SSD 1000 , the memory controller level cell ( PSLC ) cache , 1010 buffers host writes in volatile memory buffer 1020 ( i . e . a second portion of the plurality of multibit NAND media SRAM ) , and without performing error correction parity devices is configured to operate as a multibit NAND encoding , writes the host write data to the PSLC write cache media storage area , and 1044 . Like the PSLC write cache 946 of the SSD 900 shown 65 the memory controller is configured to : in FIG . 9 , the PSLC write cache 1044 may be configured as write data received via the host interface to one or more a ping pong buffer . Once a write threshold of the PSLC write blocks of the PSLC cache ,

Page 21: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

10

US 10 , 095 , 626 B2 20

transfer data from the one or more blocks of the PSLC logical block addresses ( LBAs ) and a set of logical cluster cache to one or more blocks of the multibit NAND addresses ( LCAs ) corresponding to the data contained in the media storage area , one or more blocks of the pSLC cache to be written to one

read the data transferred to the multibit NAND media or more blocks of the multibit NAND media storage area . storage area and determine if an error correction 5 8 . The SSD of claim 1 , wherein the memory controller is code ( ECC ) error correction threshold has been further configured to exceeded for the data read , and mark blocks of the multibit NAND media storage area

when the data read has been determined to exceed the exceeding the ECC error correction threshold as bad ECC error correction threshold , write such data to blocks , and the pSLC cache . write data to the bad blocks only when all other blocks of 2 . The SSD of claim 1 , wherein the PSLC cache com the multibit NAND media storage area that have not prises a first area for a PSLC write cache and a second area

for a PSLC read cache , the memory controller is configured been marked as bad blocks have been occupied with data . to :

write data received via the host interface to one or more 15 9 . The SSD of claim 2 , wherein the memory controller is blocks of the PSLC write cache , further configured to :

transfer data from the PSLC write cache to the multibit collect host read statistics for each one of the one or more NAND media storage area , and blocks of the PSLC read cache and multibit NAND

for the data transferred to the multibit NAND media media storage area , and storage area that exceeds the ECC error correction 20 perform an eviction process when the PSLC read cache is threshold , write such data to the pSLC read cache when full , the eviction process comprising : the pSLC read cache is not full , and write such data to the memory controller identifies one or more blocks of the PSLC write cache when the pSLC read cache is full . the multibit NAND media storage area that have a

3 . The SSD of claim 1 , wherein a storage capacity of the greater number of host reads than one or more blocks PSLC cache is a sum of an initial storage capacity and a 25 of the PSLC read cache , and series of iterative storage capacities , wherein the memory controller swaps data contained in the one the initial storage capacity is equal to a result of dividing or more blocks of the multibit NAND media storage a storage capacity that remains when a specified user area having the greater number of host reads with data storage capacity and an overprovisioning ( OP ) data contained in the one or more blocks of the PSLC storage capacity are subtracted from a total storage 30 read cache having a fewer number of host reads . capacity of the plurality of multibit NAND media 10 . The SSD of claim 1 , wherein the plurality of multibit devices by a storage density of the multibit NAND NAND media devices are multi - level cell ( MLC ) NAND media devices ,

a first iterative storage capacity of the series of iterative media devices , triple - level cell ( TLC ) NAND media storage capacities is equal to the initial storage capacity 35 " T 25 devices , or quad - level cell ( QLC ) NAND media devices . divided by the storage density of the multibit NAND 11 . A method of caching data in an solid state drive ( SSD ) media devices , and having a plurality of multibit NAND media devices , the

each subsequent iterative storage capacity of the series of method comprising : iterative storage capacities is equal to an immediately configuring a first portion of the plurality of multibit preceding iterative storage capacity divided by the 40 NAND media devices to operate as a pseudo - single storage density of the multibit NAND media devices , level cell ( SLC ) cache ; whereby a last iterative storage capacity of the series is configuring a second portion of the plurality of multibit less than a write flush threshold value . NAND media devices to operate as a multibit NAND

4 . The SSD of claim 2 , wherein storage area ; a storage capacity of the PSLC write cache is equal to a 45 receiving data from a host device connected to the SSD ;

result of multiplying a specified number of drive writes writing the data to one or more blocks of the PSLC cache ; per day ( N - DWPD ) of the multibit NAND media transferring data from the one or more blocks of the PSLC devices , a specified number of years of operation of the cache to one or more blocks of the multibit NAND multibit NAND media devices , 365 days , and a speci media storage area ; fied user data storage capacity , and dividing the result 50 reading the data transferred to the multibit NAND media by a specified number of program / erase ( PE ) cycles of storage area ; a block of the multibit NAND media devices , and determining if an error correction code ( ECC ) error

a storage capacity of the PSLC read cache is equal to a correction threshold has been exceeded for the data difference of a storage capacity of the PSLC cache and read ; and the storage capacity of the pSLC write cache . for the data read exceeding the ECC error correction

5 . The SSD of claim 2 , wherein the memory controller is threshold , writing such data to the pSLC cache . configured to write data contained in the one or more blocks 12 . The method of claim 11 , further comprising : of the PSLC write cache to the multibit NAND media configuring the PSLC cache into a PSLC write cache and storage area after a write threshold of the PSLC write cache a PSLC read cache ; has been exceeded . writing data received from the host device to one or more

6 . The SSD of claim 1 , wherein the memory controller is blocks of the pSLC write cache ; further configured to sequentially write the data contained in transferring the data contained in the one or more blocks the one or more blocks of the PSLC cache to one or more of the PSLC cache to the multibit NAND media storage blocks of the multibit NAND media storage area . area ; and

7 . The SSD of claim 6 , wherein the memory controller is 65 for the data transferred to the multibit NAND media configured to perform an in - memory sorting within the one storage area that exceeds the ECC error correction or more volatile memory devices of at least one of a set of threshold , writing such data to the PSLC read cache

Page 22: ( 12 ) United States Patent ( 10 ) Patent No . : US 10 ... · US010095626B2 ( 12 ) United States Patent Kotte et al . ( 10 ) Patent No . : US 10 , 095 , 626 B2 ( 45 ) Date of Patent

US 10 , 095 , 626 B2 21 22

when the PSLC read cache is not full , and writing such 15 . The method of claim 12 , further comprising : data to the pSLC write cache when the pSLC read writing data contained in the one or more blocks of the cache is full . PSLC write cache to the multibit NAND media storage

area after a write threshold of the PSLC write cache has 13 . The method of claim 11 , wherein the step of config been exceeded . uring the portion of the plurality of multibit NAND media 16 . The method of claim 11 , further comprising : devices to operate as a pseudo - single - level cell ( PSLC ) sequentially writing the data contained in the one or more cache includes calculating a storage capacity of the pSLC blocks of the PSLC cache to one or more blocks of the cache that is a sum of an initial storage capacity and a series multibit NAND media storage area . of iterative storage capacities , wherein 17 . The method of claim 16 , wherein the step of sequen 10 tially writing the data contained in the one or more blocks of the initial storage capacity is equal to a result of dividing the PSLC cache to the one or more blocks of the multibit a storage capacity that remains when a specified user NAND media storage area includes sorting , in a volatile

data storage capacity and an overprovisioning ( OP ) memory communicatively coupled to the plurality of mul storage capacity are subtracted from a total storage tibit NAND media devices , at least one of a set of logical capacity of the plurality of multibit NAND media 15 block addresses ( LBAs ) and a set of logical cluster addresses devices by a storage density of the multibit NAND ( LCAs ) corresponding to the data contained in the one or media devices , more blocks of the PSLC cache before writing the data to

one or more blocks of the multibit NAND media storage a first iterative storage capacity of the series of iterative area . storage capacities is equal to the initial storage capacity 20 18 . The method of claim 11 . further comprising : divided by the storage density of the multibit NAND marking blocks of the multibit NAND media storage area media devices , and exceeding the ECC error correction threshold as bad

each subsequent iterative storage capacity of the series of blocks , and iterative storage capacities is equal to an immediately writing data to the bad blocks when all other blocks of the preceding iterative storage capacity divided by the 25 multibit NAND media storage area that have not been

marked as bad blocks have been occupied with data . storage density of the multibit NAND media devices , 19 . The method of claim 12 , further comprising : whereby a last iterative storage capacity of the series is collecting host read statistics for each one of the one or less than a write flush threshold value . more blocks of the pSLC read cache and the multibit 14 . The method of claim 12 , wherein the step of config - o NAND media storage area ; and

uring the PSLC cache into the pSLC write cache and the performing an eviction process when the PSLC read cache pSLC read cache includes calculating a storage capacity of is full , the eviction process comprising : the PSLC write cache and a storage capacity of the PSLC identifying one or more blocks of the multibit NAND read cache , wherein media storage area that have a greater number of host

reads than one or more blocks of the pSLC read the storage capacity of the PSLC write cache is equal to 35 cache , and a result of multiplying a specified number of drive swapping data contained in the one or more blocks of writes per day ( N - DWPD ) of the multibit NAND media the multibit NAND media storage area having the devices , a specified number of years of operation of the greater number of host reads with data contained in multibit NAND media devices , 365 days , and a speci the one or more blocks of the PSLC read cache fied user data storage capacity , and dividing the result 40 having a fewer number of host reads . by a specified number of program / erase ( PE ) cycles of 20 . The method of claim 11 , wherein the plurality of a block of the multibit NAND media devices , and NAND media devices are multi - level cell ( MLC ) NAND

the storage capacity of the PSLC read cache is equal to a media devices , triple - level cell ( TLC ) NAND media difference of a storage capacity of the PSLC cache and devices , or quad - level cell ( QLC ) NAND media devices . the storage capacity of the PSLC write cache . * * * * *