oracle cache fusion cache fusion concepts, data block shipping, and recovery with cache fusion

Oracle Cache FusionOracle Cache FusionCache Fusion Concepts, Data Block Cache Fusion Concepts, Data Block Shipping, and Recovery with Cache Shipping, and Recovery with Cache FusionFusion

ObjectivesObjectives

At the end of this module the At the end of this module the student will understand the student will understand the following tasks and concepts.following tasks and concepts.

Cache fusion conceptsCache fusion concepts Block transfers using cache fusionBlock transfers using cache fusion Cache Fusion and RecoveryCache Fusion and Recovery

OverviewOverview

SynchronizationSynchronization Past ImagesPast Images Cache Fusion ScenariosCache Fusion Scenarios Recovery Methodology and stepsRecovery Methodology and steps Recovery Process ScenariosRecovery Process Scenarios

Synchronization of Concurrent Synchronization of Concurrent TasksTasks

RAC synchronization through Cache RAC synchronization through Cache Fusion:Fusion: Maintains cluster-wide concurrency of Maintains cluster-wide concurrency of

resources resources Ensures integrity of shared dataEnsures integrity of shared data

Data blocks and enqueues are Data blocks and enqueues are synchronized when nodes within a synchronized when nodes within a cluster:cluster: Acquire block or enqueue ownership Acquire block or enqueue ownership Release block or enqueue ownershipRelease block or enqueue ownership

Minimize SynchronizationMinimize Synchronization

Key is to optimally divide tasks Key is to optimally divide tasks among nodes, so that very little among nodes, so that very little synchronization is necessarysynchronization is necessary

Minimize inter-node communication, Minimize inter-node communication, because block access in local cache because block access in local cache is cheapest:is cheapest: Block access in local cache ~ 0.01 msecBlock access in local cache ~ 0.01 msec Block access in remote cache ~ 2.5 Block access in remote cache ~ 2.5

msecmsec Block access on disk ~ 14 msec+Block access on disk ~ 14 msec+

Past ImagesPast Images Past Images introduced in Oracle 9i RAC Past Images introduced in Oracle 9i RAC

to maintain data integrityto maintain data integrity Dirty data block not written to disk Dirty data block not written to disk

immediatelyimmediately Another instance may request the same Another instance may request the same

block for read or writeblock for read or write Image of the block is created at owning Image of the block is created at owning

instance, and is shipped to requesting instance, and is shipped to requesting blockblock

Backup image kept at owning block is a Backup image kept at owning block is a Past Image, and is kept in memory for Past Image, and is kept in memory for local consistencylocal consistency

““Juggling” Data with Juggling” Data with Multiple Past ImagesMultiple Past Images

Multiple Past Image versions Multiple Past Image versions of a data block may be kept of a data block may be kept by different instancesby different instances

Upon a checkpoint, only the Upon a checkpoint, only the current image is written to current image is written to disk; Past Images are disk; Past Images are discardeddiscarded

In the event of a failure, In the event of a failure, current version of block can current version of block can be reconstructed from PIsbe reconstructed from PIs

Since PIs are kept in memory, Since PIs are kept in memory, they aid in avoiding frequent they aid in avoiding frequent disk writesdisk writes

This avoids “disk pinging” This avoids “disk pinging” experienced with 8i OPS due experienced with 8i OPS due to frequent writes to diskto frequent writes to disk

Data is “juggled” in memory, Data is “juggled” in memory, without touching down on the without touching down on the diskdisk

Cache Fusion Scenario 1: Cache Fusion Scenario 1: Read/Read Cache Fusion – GCS Read/Read Cache Fusion – GCS

ProcessingProcessing

Global Cache Service (GCS)

Block Block

SGABufferCache

SGABufferCache (3) Ship

(1) Request

(4) Inform GCS1: (Shared, Local)2: (Shared, Local)

(2) Forward

Locks:(None) (Shared, Local)

Locks:(Shared, Local)

Instance 1 Instance 2

Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

Cache Fusion Scenario 2: Cache Fusion Scenario 2: Write/Write Cache Fusion – GCS Write/Write Cache Fusion – GCS

ProcessingProcessing


Block Block

SGABufferCache

SGABufferCache (3) Ship

(1) Request

(4) Inform GCS1: (Exclusive, Global)2: (Null, Global, Past Image)

(2) Forward

Locks:(None) (Exclusive, Global)

Locks:(Exclusive, Local) (Null, Global, Past Image)



Cache Fusion Scenario 3: Write Cache Fusion Scenario 3: Write Blocks to Disk – GCS ProcessingBlocks to Disk – GCS Processing


Block Block

SGABufferCache

SGABufferCache

(2) Forward

(4) Notify GCS of write and inform GCS1: (Exclusive Local)2: (None)

(5) Flush PI

Locks:(Exclusive, Global) (Exclusive, Local)

Locks:(Null, Global, Past Image) (None)


(1) Write Request

(3) WriteBased on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

Online Instance Recovery StepsOnline Instance Recovery Steps

Instance Failure detected by Cluster Manager Instance Failure detected by Cluster Manager and GCSand GCS

Reconfiguration of GES resources (enqueues); Reconfiguration of GES resources (enqueues); global resource directory is frozenglobal resource directory is frozen

Reconfiguration of GCS resources; involves Reconfiguration of GCS resources; involves redistribution among surviving instancesredistribution among surviving instances

One of the surviving instances becomes the One of the surviving instances becomes the “recovering instance”“recovering instance”

Online Instance Recovery Steps Online Instance Recovery Steps (cont.)(cont.) SMON process of recovering instance starts first SMON process of recovering instance starts first

pass of redo log read of the failed instance’s pass of redo log read of the failed instance’s redo log threadredo log thread

SMON finds BWR (block written records) in the SMON finds BWR (block written records) in the redo and removes them as their PI is already redo and removes them as their PI is already written to diskwritten to disk

SMON prepares recovery set of the blocks SMON prepares recovery set of the blocks modified by the failed instance but not written to modified by the failed instance but not written to diskdisk

Entries in the recovery list are sorted by first Entries in the recovery list are sorted by first dirty SCNdirty SCN

SMON informs each block’s master node to take SMON informs each block’s master node to take ownership of the block for recoveryownership of the block for recovery

Online Instance Recovery Steps Online Instance Recovery Steps (cont.)(cont.)

Second pass of log read begins. Second pass of log read begins. Redo is applied to the data files.Redo is applied to the data files. Global Resource Directory is unfrozenGlobal Resource Directory is unfrozen

RAC Recovery – Solving RAC Recovery – Solving the Mysterythe Mystery

Recovering from a failed RAC Recovering from a failed RAC instance is like solving a Detective instance is like solving a Detective mysterymystery Part of the evidence is missing – by Part of the evidence is missing – by

definition, the failed instance is definition, the failed instance is inaccessible at recovery timeinaccessible at recovery time

There are clues – in this case the There are clues – in this case the lock state and mode of the blocks lock state and mode of the blocks on the surviving nodes.on the surviving nodes.

Tip 1: Whichever node has an Tip 1: Whichever node has an exclusive lock on a given block has exclusive lock on a given block has the most recent version of that the most recent version of that blockblock

Tip 2: If the mode of surviving Tip 2: If the mode of surviving blocks is local, any change made blocks is local, any change made was on one node ony. If the mode was on one node ony. If the mode of surviving blocks is global, of surviving blocks is global, changes have been made on changes have been made on multiple nodes.multiple nodes.

Tip 4: If the block mode is global Tip 4: If the block mode is global and a node has a past image, and a node has a past image, there is a newer version of the there is a newer version of the block on another node.block on another node.

Lock Remastering – (Scenario-Lock Remastering – (Scenario-1)1)

Exclusive, Local None


Remove fromrecovery list

RecoveringInstance A

OpenInstance B

FailedInstance C

Lock Held

LockAquired

XCurrent copyof buffer



Exclusive, GlobalNull, Global,Past Image




OpenInstance B

FailedInstance C

Lock Held

LockAcquired

X

Write to disk



None Exclusive, Local

None Exclusive, Local



OpenInstance B

FailedInstance C

Lock Held

LockAcquired

X

No need to write to disk



Null, Global,Past Image

Exclusive, Global


Exclusive, Global



OpenInstance B

FailedInstance C

Lock Held

LockAcquired

X

Write to disk



None Exclusive, Global


None



OpenInstance B

FailedInstance C

Lock Held

LockAcquired

X

write to disk



None None(Exclusive, Local

Implied)


Keep in recovery list


OpenInstance B

FailedInstance C

Lock Held

LockAcquired

X



None




OpenInstance B

FailedInstance C

Lock Held

LockAquired

X

Send Cr block to A

(Exclusive, GlobalImplied)Null, Global,

Past Image




Null, Global

Exclusive, Global,Past Image

Null, Global



OpenInstance B

FailedInstance C

Lock Held

LockAquired

X

Send Cr block to A


(Exclusive, GlobalImplied)

ReviewReview Name two times that data blocks and enqueues Name two times that data blocks and enqueues

are synchronized.are synchronized. True or false: since communication across the True or false: since communication across the

private interconect is fast, you want to maximize private interconect is fast, you want to maximize the amount of inter-node communication.the amount of inter-node communication.

True or false: multiple instances may hold Past True or false: multiple instances may hold Past Images of the same block at the same time.Images of the same block at the same time.

All cache fusion requests must pass through the All cache fusion requests must pass through the ___.___.

True or false: an instance with an exclusive lock True or false: an instance with an exclusive lock with a global resource role contains the most with a global resource role contains the most current version of a block.current version of a block.

SummarySummary Cache Fusion synchronizes resources cluster-Cache Fusion synchronizes resources cluster-

wide such as:wide such as: Data blocksData blocks EnqueuesEnqueues

Past Images are used to maintain data block Past Images are used to maintain data block integrity integrity No immediate disk write requiredNo immediate disk write required Avoids “disk pinging”Avoids “disk pinging”

The GCS plays a key role in performing the The GCS plays a key role in performing the necessary block transfers necessary block transfers Including lock mode conversionsIncluding lock mode conversions

Upon recovery, GCS remasters cache resources Upon recovery, GCS remasters cache resources of a failed node or nodes on one or more of a failed node or nodes on one or more recovery nodesrecovery nodes

oracle cache fusion cache fusion concepts, data block shipping, and recovery with cache fusion

Documents

local cache

remote cache

cache fusionobjectivesat

data block shipping

global exclusive

data integritydirty

diskcache fusion scenario

current version of block