Flash Memory - TU Dortmund€¦ · NOR NAND SLC NAND MLC x2 Price 34.55 $/GB 6.79 $/GB ... Yuan-Hao...

fakultät für informatik informatik 12

technische universität dortmund

Flash Memory

Jian-Jia Chen (Slides are based on Yuan-Hao Chang)

TU Dortmund Informatik 12

Germany 2015 年 01 月 27日 These slides use Microsoft clip arts. Microsoft copyright restrictions apply.

© S

prin

ger,

2010

- 2 - technische universität dortmund

fakultät für informatik

© Y.Chang and J. Chen, informatik 12, 2015

Why Flash Memory

Diversified Application Domains §  Portable Storage Devices § Consumer Electronics §  Industrial Applications § Critical System Components




Layout of Flash Memory

… … (2KB + 64 Byte) (2KB + 64 Byte)

… … (2KB + 64 Byte) (2KB + 64 Byte)

… … (2KB + 64 Byte) (2KB + 64 Byte)

Page: basic write-operation unit.

0 63 … …

0

1023

… …

… …

Block: basic erase-operation unit.

128MB Flash Memory




Characteristics of Flash Memory

Write-Once §  No writing on the same page unless its residing block is

erased! §  Pages are classified into valid, invalid, and free pages.

Bulk-Erasing §  Pages are erased in a block unit to recycle used but invalid

pages.

Wear-Leveling §  Each block has a limited lifetime in erasing cycles.

•  E.g., 10,000 ~ 100,000 erase cycles for each block




Terminology

Valid data: the latest version of data stored in flash Invalid data: not the latest version of data stored in flash Live page: a page that stores valid data Dead page: a pages that stores invalid data Free page: a page that is erased and is ready to store data Free block: a block that is erased and is not allocated to store any data Hot data: frequently updated data §  Valid hot data might become invalid in the near future.

Cold data: non-frequently updated data §  Valid cold data might stay in the same place for a long time.




Dead pages

A B C D A B

Management Issues – Flash-Memory Characteristics

Example 1: Out-place Update




A live page A dead page A free page

This block is to be recycled. (3 live pages and 5 dead pages)

L D D L D D L D

L L D L L L F D

L F L L L L D F

F L L F L L F D


Example 2: Garbage Collection




L L D L L L D

L F L L L L D

L L F L L F D

L

L

D D D D


Live data are copied to somewhere else.

L

D D D D






A live page A dead page

The block is then erased. Overheads: • live data copying • block erasing.

L L D L L L D

L F L L L L D

L L F L L F D

L

L

F F F F F F F F

L







Example 3: Wear-Leveling

L D D L D D L D

L L D L L L F D

L F L L L L D F

F L L F L L F D

100

10

20

15

Erase cycle counts

Wear-leveling might interfere with the decisions of the block-recycling policy.


A

B

C

D




Single-Level Cell (SLC) vs. Multiple-Level Cell (MLC)

A limited bound on erase cycles §  SLC : 100,000 §  MLCx2: 10,000

Bit error probability §  SLC: 10-9

§  MLC: 10-6

1 0VT

LOGIC STATENU

MB

ER

OF

CE

LLS

a) BILEVEL (1 BIT/CELL)

11 00VT

LOGIC STATENU

MB

ER

OF

CE

LLS

b) MULTILEVEL (2 BIT/CELL)

10 01

64MB

SLC Flash

128MB

MLC Flash




MLC vs. SLC (Cont.)

Electronic Engineering Times, July 2005




Price and Read/Write Performance

NOR NAND SLC NAND MLCx2

Price 34.55 $/GB 6.79 $/GB 2.48 $/GB

Read 23.84 MB/sec 15.33 MB/sec 13.5 MB/sec

Write 0.07 MB/sec 4.57 MB/sec 2.34 MB/sec

Erase 0.22 MB/sec 85.33 MB/sec 170.66 MB/sec

*NOR: Silicon Storage Technology (SST). NAND SLC: Samsung Electronics. K9F1G08Q0M. NAND MLCx2: ST STMicroelectronics[1,2]

1. Jian-Hong Lin, Yuan-Hao Chang, Jen-Wei Hsieh, Tei-Wei Kuo, and Cheng-Chih Yang, "A NOR Emulation Strategy over NAND Flash Memory," the 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), Daegu, Korea , August 21-24, 2007. 2. Yuan-Hao Chang and Tei-Wei Kuo, “A Log-based Management Scheme for Low-cost Flash-memory Storage Systems of Embedded Systems”




Flash Memory Management

Management issues §  Write constraints imposed by flash memory §  Scalability issue §  Garbage collection §  Performance considerations

Reliability issues §  Cell error rate problem imposed by MLC flash memory §  Error correction coding vs. wear leveling §  Read/write disturbance problem §  Data retention problem




Typical System Architecture

Virtual File System (VFS)

Shared Library

AllocatorAddress Translation / Block Assignment

Memory Technology Device Layer (MTD)

CleanerGarbage Collection

File Systems (e.g., DOS FAT)

AllocatorAddress Translation / Block Assignment

CleanerGarbage Collection

File-system Management Unit

Native Flash File System Layer(e.g., JFFS2, YAFFS)

Flash Translation Layer (FTL)

Flash Memory

fwrite(file, data)

flash write(block, page)

Control single

...

Application 1 Application 2 ... Application n

Wear Leveler

Wear Leveler




FTL adopts a page-level address translation mechanism. §  The main problem of FTL is on large memory space requirements for

storing the address translation information.

Userdata

.

.

.

Logical BlockAddress

(array index)

Physical BlockAddress

(block,page)

Physical Block Address(block,page)

Access LBA = 3

Address TranslationTable (in main-memory) Flash memory

0,00,10,20,30,40,50,60,71,01,11,21,3

(0,3)(0,1)

(0,6)(0,4)(4,7)(1,0)(2,1)(1,2)(1,3)

0123456789

1011

.

.

.

Sparedata

Spare data

LBA=3;ECC=. . .;

Status=. . .;

Flash Translation Layer (FTL)




A logical address under NFTL is divided into a virtual block address and a block offset. §  e.g., LBA=1019 => virtual block address (VBA) = 1019 / 8 = 127 and block offset = 1019 % 8

= 3

.

.

.

(9,23)

Write data to LBA=1019

.

.

.

NFTL Address Translation Table

(in main-memory)

Free Free Free Used Free Free Free Free

Used Used Used Free Free Free Free Free

A Primary Block

Address = 9

A Replacement Block

Address = 23

VBA=127

Block Offset=3

If the page has been used Write to the

first free page

NAND Flash Translation Layer (NFTL)




FTL or NFTL

FTL NFTL Memory Space Requirements Large Small

Address Translation Time Short Long

Garbage Collection Overhead Less More

Space Utilization High Low

• The memory-space requirements for one 1GB NAND (2KB/Page, 4B/Table Entry, 128 Pages/Block) – FTL: 2MB (= 4*(1024*1024*1024)/2K) – NFTL: 32KB (= 2*4*(1024*1024*1024)/(2K * 128))




Size of Translation Tables

1GB 32GB 1TB 32TB FTL 2MB 64MB 2GB 64GB NFTL 32KB 1MB 32MB 1GB

No matter which kind of granularity of address translation is adopted, the fast growing flash memory capacity would eventually make the translation table too large to be fitted in RAM.




Adaptive Two-Level Mapping Mechanism

The coarse-grained hash table maintains the primary block and its replacement block §  This will be mainly used to identify the primary and

replacement blocks The fine-grained hash table has limited entries to store the excessive live pages in the replacement block

•  When the fine-grained table is full, the replacement policy has to be considered to move some pages to the coarse-grained table

Improve the space utilization. §  The delayed recycling of any primary block lets free

pages of a primary block be likely used in the future.

A Fine-GrainHash Table

A Coarse-GrainHash Table

.

.

.

.

.

.

A

A Primary Block( )

B

C

D

E

A

A Replacement Block( )

B

A

B

A

A

B

B

.

.

.

.

.

.

(A, )

(B, )

AFTL

PPBAcg .δ RPBAcg .δ

)

,,.(

RPBAPPBAVBA

cg

cgcg

δ

δδ

5. +RPBAcgδ

7. +RPBAcgδ

Chin-Hsien Wu and Tei-Wei Kuo, 2006, “An Adaptive Two-Level Management for the Flash Translation Layer in Embedded Systems,” IEEE/ACM 2006 International Conference on Computer-Aided Design (ICCAD), November 5-9, 2006.




New Write Constraints of MLC Flash

Write constraints §  Pages can only be written sequentially in a block. §  Partial page write/programming is prohibited.

Impact on NFTL §  Data can’t be written to any free page of primary blocks.

•  The space utilization in primary blocks is even lower. •  Most writes are forced to be placed in the replacement

block. §  Pages of invalid data can’t be marked as dead.

•  Each read operation should scan pages of the replacement block.




Intuitive and Practical Solution

Level-paging translation tables §  Pages of translation tables

are stored in flash, and are cached in RAM.

Problems: §  Hit ratio of cached pages §  Extra page reads and writes

for translation information §  Crash recovery for

translation table and lost data

Level 2 paging page Off-setLevel 1 paging pageLevel 0 paging page

2 bits9 bits9 bits9 bits

4 bytes

4 bytes

4 bytes

256GB Flash memory

......

......

Page(2KB)

LBA (4 bytes)

PageBlockChip

7 bits17 bits

PBA (4 bytes)

3 bits

empty

1 Page = 2KB1 Block = 27 Pages1 Chip = 217 Blocks

... ...

...

Page table

Outer-page table

2nd outer-page table




Performance Considerations

MLC flash has growing market share: §  Reason: low cost and high density §  Drawbacks: low speed, low endurance, and low reliability

Solutions: §  Hardware multi-channel programming §  Software multi-bank or multi-channel programming




Wear Leveling Er

ase C

ycle

s

0 20 40 60 80 100 0

1000

2000

3000

4000

5000

No Wear Leveling

0 20 40 60 80 100

Dynamic Wear Leveling

Physical Block Addresses (PBA)

0 20 40 60 80 100

Perfect Static Wear Leveling

Intuitive Static Wear Leveling




Static Wear Leveling

Random policy §  It randomly select a block to reclaim after a fixed number

of block erases or write requests. §  It doesn’t track the locality of data acceses, such that it

might move hot data that might turn dead in the near future.




Static Wear Leveling (Cont.)

Random policy with block-erasing table §  Each-bit flag of the table is to indicate whether the corresponding

blocks have been erased. §  Whenever the block erases are not even enough, select blocks whose

corresponding bit flag are not set. Pros and Cons: §  Pros: it can identify the locality of data accesses. §  Cons: the block-erasing table needs extra RAM space even it is

comparatively small.

Yuan-Hao Chang, Jen-Wei Hsieh, Tei-Wei Kuo: Endurance Enhancement of Flash-Memory Storage, Systems: An Efficient Static Wear Leveling Design. DAC 2007: 212-217




The Block Erasing Table (BET)

A bit-array: Each bit is for 2k consecutive blocks. §  Small k – in favor of hot-cold data separation §  Large k – in favor of small RAM space

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Flash

BET 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1 1 1 1

k=0 k=2

ecnt=0 fcnt =0

ecnt=0 fcnt =0

ecnt=1 fcnt =1 ecnt=2 fcnt =2 ecnt=3 fcnt =2

ecnt=1 fcnt =1 ecnt=2 fcnt =2 ecnt=3 fcnt =2 ecnt=4 fcnt =2

: a block that has been erased in the current resetting interval : an index to a block that the Cleaner wants to erase

fcnt: the number of 1’s in the BET ecnt: the total number of block erases done since the BET is reset




ecnt=1998 fcnt =2

An unevenness level (ecnt / fcnt) >= T à Triggering of the SW Leveler

Resetting of BET when all flags are set.

A SimpleStatic-Wear Leveler

0 0 0 0

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

: A block that has been erased in the current resetting interval

1 1 k=2

: An index to a block that the Cleaner wants to erase

fcnt: The number of 1’s in the BET

ecnt: the total number of block erases since the BET is erased T: A threshold, T=1000 in this example

: An index in the selection of a block set

ecnt=1999 fcnt =2 ecnt=2000 fcnt =2

2000 / 2 = 1000 >= 1000 (Ecnt / fcnt >= T)

1 : An index that SW Leveler triggers the Cleaner to do garbage collection

The Cleaner is triggered to 1. Copy valid data of selected block set to free area, 2. Erase block in the selected block set, and 3. Inform the Allocator to update the address mapping between LBA and PBA

After a period of time, the total erase count reaches 2998.

ecnt=2004 fcnt =3 ecnt=2998 fcnt =3 ecnt=2999 fcnt =3 ecnt=3000 fcnt =3

3000 / 3 = 1000 >= 1000 (Ecnt / fcnt >= T)

1

ecnt=3004 fcnt =4

After a period of time, the total erase count reaches 3999.

ecnt=3999 fcnt =4 ecnt=4000 fcnt =4

4000 / 4 = 1000>=1000 (ecnt/ fcnt>=1000) , but all flags in BET are 1à reset BET

Reset to a randomly selected block set (flag)

ecnt=0 fcnt =0




Main-Memory Requirements

512MB 1GB 2GB 4GB 8GB

k=0 256B 512B 1024B 2048B 4096B k=1 128B 256B 512B 1024B 2048B k=2 64B 128B 256B 512B 1024B k=3 32B 64B 128B 256B 512B

MLCx2 (1 page = 2 KB, 1 block=128 pages)




File-System Considerations - Observations

File system NTFS FAT Number of write requests 24,513 179,670 Time taken (min:sec) 4:33 54:21

File system NTFS FAT Number of read requests 14,568 23,528 Time taken (min:sec) 2:53 3:19

Number of files 19,535 Number of directories 1,200 Average file size 11 KB Archive size 215,666 KB

Archive: Linux-2.6.17 source

Write the files to a removable storage device

Read the files from a removable storage device FAT file systems

introduce excessive write requests! why ?

FAT is the default file system for removable

storage devices




File-System Considerations - Observations (Cont.)

FAT #1 FAT #2

Content of File Directory Entry

Directory Entry

FAT #1 FAT #2

Content of File

Directory Entry

Directory Entry

File #1

File #2

• Layout of FAT filesystem LBA 0

https://www.pjrc.com/tech/8051/ide/fat32.html




File-System Considerations

USB Flash Drives

Card Readers

Flash Memory Cards

Hos

t

Applications

Hos

t

Storage Media

Device Controller

Dev

ice

Storage Media

Dev

ice

(a) (b)

File System Drivers Operating System API

Device Drivers

Bus Drivers Filter Drivers

Device Controller

Disk Drivers




Overview

Cache

Transport Unit

Debug Unit

Dispatch Unit

Debug Viewer

USB Mass Storage Device Driver

USB Bus Driver

Filesystem

Identifier

I/O Request

Partition Table & Boot Sectors

File System Layout

I/O Request

Trace

Read and Write

IoPackets

I/O Response

I/O Response

Other IoPackets

Filter Driver

FIFO Queue

Notifications

When ? Cohesive-caching

algorithm




The “Allow/Prevent Medium Removal” Operation Code in the USB Bulk-Only Transport

… hrtimer.c intermodule.c itimer.c kallsyms.c …

<CBW> Write10 LBA=1446 LEN=1

<CSW> Success

<DATA>

…

…

…

…

… FAT File System USB Bus

1st FAT 2nd FAT

File Content Directory Entry

Directory Entry

W 477 1 W 1466 1 W 210576 4 W 210664 9 W 210576 4 W 210576 4 W 210576 4 W 210576 4

…

…

Disk

…

…

<CBW> Allow Medium Removal

<CBW> Prevent Medium Removal

<CSW> Success

<CSW> Success USB Mass Storage Class, Bulk-Only Transport Protocol CBW: Command Block Wrapper CSW: Command Status Wrapper

USB Mass Storage Device Driver

Disk Driver An IoPacket

Send a notification




Worsening Reliability - Narrow Threshold Voltage Window

MLC/TLC/QLC technology must squeeze the available window of threshold voltage for each logical state §  Higher Bit Error Rate §  Lower Endurance

Source: http://liobaashlyritchie.blogspot.tw/

SLC MLC




Reliability Issues

The low-cost MLC flash §  Has lower erase cycles §  Has higher and higher bit error rate.

Ways to improve reliability §  Error correction coding (ECC) à passively

•  Non-erasure code such as BCH and RS §  Wear leveling à positively

•  Distribute block erases as even as possible

Note: Erasure code is used in communication Non-erasure code is used in storage systems




Error Correction Coding

Error correction code of a page is stored in its spare area. §  The time on error correction might be affected by

•  the location of error bits or •  the number of error bits.

§  The space of the ECC hardware is increased as the number of supported error bits increases.

Trend of MLC flash §  The page size is getting larger §  The bit error rate is getting higher

Fast erasing bits §  The fast worn-out flash cells

Vt




Data Retention Problem

The guaranteed data retention: 10 years As the cell size is getting smaller, the number of electrons in

the floating gate of a flash cell is getting smaller. §  For example:

•  A programmed cell can store 10,000 electrons. •  A lost of only 10% in this number can lead to a wrong

read. à A loss of less than 2 electrons per week can be tolerated.

Flash Memory - TU Dortmund€¦ · NOR NAND SLC NAND MLC x2 Price 34.55 $/GB 6.79 $/GB ... Yuan-Hao...

Documents

Transcript of Flash Memory - TU Dortmund€¦ · NOR NAND SLC NAND MLC x2 Price 34.55 $/GB 6.79 $/GB ... Yuan-Hao...