Performability Analysis of Disk Arrays

S.M. Rezaul Islam IBM Corporation

Boca Raton, FL 33431

Abstract

In recent years, arrays of small inexpensive disks are becoming a cost effective alternative that offers high performance and high capacity. Arrays of small disks contain many more components than a large single disks; therefore, they are prone to random failure. In this paper we are presenting a technique to evaluate the effectiveness of the array in the presence of random failure and repair.

I. Introduction

Redundant arrays of inexpensive disks (RAID) [l] provide high capacity and tolerance to single disk failure by introducing redundancy. In this scheme, protection against data loss is maintained by employing "(N+I)-parity". This form of protection maintains the parity of N data disks in a single disk, which is distributed evenly over all disks. In the event of a failure, the data in the failed disk is reconstructed from the parity and other data i.n rest of the functioning disks.

When a disk fails, it is replaced with a good disk and data is rebuilt. During this data rebuild, the array is accessible by the system, and in spite of a disk failure, there is no data loss to the system. Since the array is servicing user requests and at the same time rebuilding data in the replaced disk, the system experiences a degradation of overall performance. In a mission- critical system with hard deadlines, the array must be able to perform to expectations even when it is in a degraded mode; i.e., the array is not fully operational due to some nonfunctional disks. Therefore, evaluation of disk arrays in a mission-critical system requires unified performance-reliability measures, i.e., performability. Performability, formally introduced by Meyer [41, is defined as the probability that a system reaches a certain accomplishment level over a utilization interval called mission time. In this paper

we applied the technique of performability [5,6] analysis to evaluate effectiveness of disk arrays in a mission-critical system.

11. Performance and Reliability Model

RAID achieves very high levels of fault tolerance by introducing redundancy into the system. In RAID 5 this redundancy is in the form of parity data that is distributed evenly throughout the disks of an array. This parity can be viewed as a part of a code word in an algebraic error-correcting code block. Therefore, in each write operation, one must adjust the corresponding parity data (in a separate disk) in addition to writing to the requested disk. On the other hand, read requests are serviced by directly accessing the requested disk. When a disk fails and it is replaced with a new disk, the data in the new disk is reconstructed from (doing an exclusive-or) remaining disks. Therefore, during rebuild, all disks in the array are 100% utilized even when there are no user request for disk access. If the workload is evenly distributed between the user request and the rebuild process, throughput of the array becomes approximately half of the normal throughput. At the successful completion of the rebuild, the throughput returns to the normal level. However, if another disk fails before rebuild is complete, data in the disk array is lost.

We assume that the failure events are independent and both the lifetime and repair time (time to replace the disk and rebuild data) of a disk follow exponential distribution. Figure 2 shows the three-state Markov model representing the failure and repair action in the disk array. M-0 represents the state of a n o d disk array. When a disk fails, it drops to state M-1, which represents the degraded state of the array. In state M-1 the array undergoes a rebuild operation. (In fact., the array f is t maintains a degraded mode and then moves to the rebuild mode. However, if a spare is available, the array spends no time in the degraded mode as it immediately begins the rebuild operation.) The failure rate of a disk L-d is given by the inverse of the

CH 3381-1/93/$01.00 81993 IEEE 158

MTTF-disk (L-d=l/MTI'F-disk). Similarly the repair rate L r is given by the inverse of M'ITR-disk. In this paper we assume MTlT-disk is 2oo.OOO hours and MITR-disk is 24 hours. At the end of successful rebuild, the array returns to state M-0. However, if another disk fails before rebuild is complete, the state of the array drops to nonoperational state M-2.

The I/O subsystem is composed of support hardware, such as power supplies, cabling, cooling fans and controller, which are shared by multiple disks in the subsystem as shown in Figure 1. These shared hardware with the may of disks forms a string. The entire string fails when any of the support hardware fails. Therefore, the state of the array moves from good operational state M-0 to a nonoperational state M-2 due to a string failure. This is represented by the transition L-s=l/M"F-string. We assume that the nondisk portion of the MTTF-string is 150,000 hours [2]. The mean repair time from a nonoperation state is also assumed to be 24 hours.

We normalized performance of the array when it is in good operational state (i.e.. M-0) to 1. Similarly performance when nonoperational (M-2) is set to 0. We further assume that the normalized performance when in state M-1 is 0.5. This is little bit pessimistic since the sojourn time in M-1 is composed of a system running at degraded mode and a rebuild operation. 0.5 normalized throughput is more appropriate for the rebuild and possibly not appropriate for the degraded mode. To keep our analysis simple, we assumed one normalized performance both for degrade and rebuild mode. The transition rate matrix Q and performance rate (reward vector) F are shown in Figure 3. These are required to compute performability.

111. Performability Analysis

We used the algorithm presented in [5.61 for the performability analysis. We have a three-state repairable system as shown in Figure 2. Our structure- state process is given by [X-t, t > 0}, where X-t describes the state of the system (X-t is either M-0, M-1 or M-2) at time t. We also associate a non- negative reward function f(i) with state i. In OUT case, the reward function is the normalized performance in each state. The function f(i) defines the reward structure of the system, and the stochastic process (f(X-t). t > 0} is the corresponding reward model. If the process X-t is Markovian (as in our case), it is called a Markov reward model. The performability of the system over the mission time t is the probability density function of Y(t)= C f(i)u-i. Where u-i is the sojourn time in state i. The distribution function of Y(t) is referred to as the performability distribution function. This is

S i (x,t) = Rob [ Y(t) < x I X-O=i I.

Here S-i (x,t) is the performability distribution given that the initial state of the system at t=O is i. In our case, the initial state X-0 is M-0. The complement of the distribution S-i(x,t) gives the probability that a system can complete its mission (in our case, numbers of normalized tasks) with in a specified mission time t. That is

Prob [ Y(t) > x I X - k i I = 1 - S-i (x,t).

Once the Q matrix and vector F are known as shown in Figure 3, it is straightforward to compute the Prob[Y(t)>x]. The details of the algorithm are presented in [W.

IV. Results

Figure 4 shows Rob[ Y(t) > x ] for various mission time. The horizontal axis shows the total work done by the system. We are using a normalized unit of week- task. A week-task is the total work done by a non- faulty system in a week. Clearly, the cumulative probability is very close to 1 when value of the mission time is less than the value of the week-task and it drops sharply close to 0.5 at when the value of the mission time approaches value of the week-task. This shows that disk arrays can complete their mission only 50% of the time due to random failure. This is an interesting result and requires further investigation.

V. Conclusion

In this paper, we analyzed performability of disk arrays using Markov reward models. These arrays are becoming increasingly popular as an cost-effective alternative to single expensive disks. We examined the behavior of the disk arrays when they are used in mission-critical systems.

References

Patterson, D.A., Gibson, G., and Katz R.H. "A case for Redundant Array of Inexpensive Disks(RAlD)," Proc. of ACM SIGMOD Conference, pp 109-1 16, Chicago, IL, June 1-3, 1988.

Gibson, G. and Patterson, D. "Designing disk Arrays for high data reliability." Journal of parallel and Distributed Computing 17, 4-27 (1993).

Geist, R., and Trivedi, K., "Reliability estimation of fault-tolerant systems: Tools and techniques," IEEE Comput., Vol 23, no. 7, July 1990.

159

Mayer, J.F., "On evaluating the performability of degradable computing systems," IEEE Trans. Comput. vol C-29, no. 8 Aug 1980.

Islam, S.M.R., and Ammar, H.H., " Performability of the hypercube," IEEE Trans. Rel. vol. 38, no. 5, Dec. 1989.

Islam, S.M.R., and Ammar, H.H.,"Perfombility analysis of distributed real-time systems," IEEE Trans. Comput. vo1.C-40, no. 11, Nov 1991.

Figure 2: State Diagram, ~ = 7 , number of disks,

M-0 M-1 M-2

-NL-d-L s-L-r 0- -L-r

-L-s- (N+1) L-d (N+l)L-d

0.5 0 I F = I 1.0 Controller

disks Figure 3: Q Matrix and reward vector F.

Figure 1: Array Configuration.

e-e 1 W e e k $-4 2 W e e k A-A 3 W e e k E-E 4 W e e k v-v 5 W e e k

Number of Week-Tasks x

Figure 4: Prob[Y(t)>x] vs. number of Week-tasks.

160