ieee transactions on cloud computing 1 ctdgm: a data

IEEE TRANSACTIONS ON CLOUD COMPUTING 1

CTDGM: A Data Grouping Model Based onCache Transaction for Unstructured Data

Storage SystemsDongjie Zhu, Haiwen Du, Yundong Sun and Zhaoshuo Tian, Member, IEEE,

Abstract—Cache prefetching technology has become the mainstream data access optimization strategy in the data centers. However,the rapidly increasing of unstructured data generates massive pairwise access relationships, which can result in a heavy computationalburden for the existing prefetching model and lead to severe degradation in the performance of data access. We proposecache-transaction-based data grouping model (CTDGM) to solve the problems described above by optimizing the featurerepresentation method and grouping efficiency. First, we provide the definition of the cache transaction and propose the method forextracting the cache transaction feature (CTF). Second, we design a data chunking algorithm based on CTF and spatiotemporallocality to optimize the relationship calculation efficiency. Third, we propose CTDGM by constructing a relation graph that groups datainto independent groups according to the strength of the data access relation. Based on the results of the experiment, compared withthe state-of-the-art methods, our algorithm achieves an average increase in the cache hit rate of 12% on the MSR dataset with smallcache size (0.001% of all the data), which in turn reduces the number of data I/O accesses by 50% when the cache size is less than0.008% of all the data.

Index Terms—Data grouping model, correlation analysis, feature extraction method, cache prefetching, distributed storage systems.

F

1 INTRODUCTION

THE fast development of cloud computing and mobilecomputing has enabled massive amounts of unstruc-

tured data, such as images, documents, and videos, bymobile devices using the cloud platforms. Distributed stor-age systems, such as the Hadoop Distributed File System(HDFS) [1] and Ceph [2], have advantages in terms of scala-bility, concurrency, and availability. Therefore, these systemshave become the mainstream storage solution for cloudplatforms and are used by large Internet-based companiessuch as Facebook, Netflix, Yahoo, and Amazon [3]. Until2017, Spotify and Yahoo stored approximately 50% of smallfiles with a size of less than 1 MB, and the number of I/Ooperations on small files accounted for approximately 50%of all I/O operations [4]. In addition, the rapid developmentof data science [5] has increased the pressure on small-sizedunstructured data in distributed storage systems. However,the storage mechanism of the distributed storage system ismainly designed for storing large files, and accessing a largenumber of small-sized unstructured data causes the storagesystem generate a large number of I/O operations on disk(e.g., metadata access, data access), which substantially af-fects the access concurrency and response time of the datacenter [6], [7].

Researchers have used data merging and cache prefetch-ing strategies to improve the data access efficiency of stor-

• D. Zhu and Y. Sun are with the School of Computer Science andTechnology, Harbin Institute of Technology, Weihai, Shandong, China,264209.

• H. Du and Z. Tian are with the School of Astronautics, Harbin Instituteof Technology, Harbin, China, 150001.E-mail: [email protected]

Manuscript received April 27, 2020(Corresponding author: Haiwen Du.)

age systems and to address these problems. Among theseapproaches, the data merging strategy merges small-sizedunstructured data that are strongly correlated into a bigfile and store it in a consecutive location, which reducesthe number of I/O operations and improves the indexingefficiency of the distributed storage system via hierarchicalindexing [8]. The cache prefetching technology predicts andcaches the data that may be accessed at the next momentby estimating the correlation to previously accessed data.The cache prefetching technology has been reported to effec-tively reduce the data access latency, thereby improving theconcurrency of the storage system [9], [10]. These strategieshave produced good results and are widely used in storageefficiency optimization [11], [12], [13], data deduplication[14], [15] and other fields. Data grouping is a strategy thatmerges related data into mutually exclusive groups andaccesses them by group, which effectively reduces the diskI/O of the storage system. The accuracy and computationalcomplexity of the correlation mining algorithm, which is thecore component of the data grouping model, are the keyfactors to address when aiming to improve the data accessefficiency of the storage system [16], which has become apopular issue to study.

However, data in storage systems are saved as blocks ofdifferent sizes that are scattered throughout disks instead ofin one continuous location. This type of storage is criticalfor a multi-core server, but on such servers, the order ofthe blocks in a file is not maintained. Therefore, block layercorrelations are common semantic patterns in storage sys-tems [17]. Additionally, a block layer I/O trace can providedetailed information about request queue operations andis collected by utilities such as blktrace; thus, block-levelcorrelation mining is widely studied [18], [19]. Our study

arX

iv:2

009.

1441

4v1

[cs

.DC

] 3

0 Se

p 20

20


focuses on the data correlation in the block layer. In thepresent study, the term data refers to the continuously saveddata in the block layer.

Data possess fewer valuable features for correlation min-ing. Most existing data correlation analysis methods usethe block address and the access interval between dataas features and estimate data correlations by defining thedistance between data [20], [21], [22], [23]. These correlationanalysis methods determine the data features from relateddata, and such features are called relationship defined fea-tures (RDFs). Moreover, the distributed storage system runsin multi-appli-cation and multi-user environments, and thecorrelations between data change dynamically with time[24]. Therefore, an RDF is an unstable feature that requiresa large amount of primary storage space to save pairwiserelations between data, and it must be recalculated whenaccess patterns change, resulting in substantial computa-tional costs and memory consumption. Additionally, sincethe cache space is limited, the size of the data may affectthe distribution of other data in the cache, which in turnaffects the data correlations. However, existing methods foranalyzing data relationships ignore the impact of the size ofthe data on data correlations.

We proposed a data correlation feature called the cachetransaction feature (CTF). Unlike the RDF, a cache transac-tion reflects the cache distribution of a real storage system.A cache transaction consists of a set of data that appearstogether in the cache; therefore, the data size can be con-sidered in the data correlation analysis, and the data accessfeature is represented as a vector of the cache transactionsin which the data appear. Thus, the feature is determinedby not only the pairwise access relation but also the dataitself, which substantially reduces the memory consumptionrequired for relationship storage.

Furthermore, we propose a data grouping model, CT-DGM, based on a cache transaction. CTDGM aims to obtainaccurate and closely related data groups by merging highlycorrelated data. In terms of computational efficiency, CT-DGM divides the data grouping process into data chunkingand group merging processes. The data chunking algorithmdivides the data into data chunks according to access times,spatial locality and CTF. The group merging process isperformed on highly related data chunks according to thecache transaction, which potentially improves the miningrange and the inner relation strength of the group to pro-vide a more accurate relation mining result. Moreover, thecomplexity and number of calculations will be substantiallyreduced by the two-layer relation mining method. The maincontributions of our study are listed below.

1) A more accurate representation of the relationshipsbetween data: For RDF, the size of the data is often ignored.The CTF incorporates the size of the data into the dataaccess feature. Moreover, it presents the access feature ofthe data as a vector, which makes the value easy to calculate.Therefore, it is suitable for data relation mining in the datacenter and expresses more access features of the data.

2) Calculate the data correlation without a spatiallocality limitation: CTDGM improves the range of rela-tion mining using the two-layer mining algorithm, whichreduces the negative impact of block address mining rangeon the computing efficiency and thus potentially increases

the mining accuracy.3) Ensure that data with strong correlations are accu-

rately mined: The independence of the data group requiresthat data belong to only one group. CTDGM prioritizes themerging of data highly correlated to ensure only the mostclosely related data are merged and thus ensuring a highgroup inner relation strength.

The CTDGM model works on block I/O level with lowcomputation overhead. It can also be used to group andprefetch related data on proxy workloads. Through cross-validation of widely used public datasets, our data groupingmodel significantly improved the cache hit rate, reducedthe number of the disk I/Os, and reduced the data accesslatency.

In Section 2, we introduced the background of andmotivation for the study. We describe the definition andmethod used to calculate the cache transaction in Section3. In Section 4, we provide a detailed description of thedata grouping algorithm. Section 5 presents the evaluationmethodology and results. Related studies are discussed inSection 6. Finally, we provide some conclusions in Section 7.

2 BACKGROUND AND MOTIVATION2.1 Data access feature extractionAs the basis of data access correlation mining, the methodfor extracting access features is a popular topic in the fieldof storage research. Most existing data correlation miningalgorithms are based on an RDF or Ordinal Feature (OF).We provide a detailed explanation of the RDF and the OFbelow.

Ordinal Feature (OF): An OF indicates that the dataare treated as separate units and are uniquely marked asindependent features. Therefore, an OF is a suitable featurefor the calculation of temporal locality. Researchers [25],[26] use the OF-based frequent itemset mining algorithmto analyze the correlations of Web resources and prefetchfiles with high relevance. However, OF does not representthe spatial locality, which reflects the potential correlationbetween data [27]. When an OF is applied to NN-baseddata prefetching strategies, it is represented as a one-hotencoded vector that requires extensive storage resources[28]. Therefore, in recent years, researchers have attemptedto mine data correlations based on RDF.

Relation Defined Feature (RDF): An RDF indicates thatthe access feature of the data is defined by the relateddata. Compared with an OF, an RDF is represented inpairwise relations between data, and it introduces logicalblock addresses as a factor that influences the data corre-lations, which potentially improves the relationship miningaccuracy. Most RDF-based correlation mining models definethe distance between data and regard it as the closenessof the data [11]. However, compared to an OF, the RDFis calculated by determining the distance from other data,causing the data to lose their independent features. Ad-ditionally, the pairwise relations among massive amountsof data must be pre-calculated and stored, which requiresextensive computing and storage resources.

Several RDF-based relation mining models perform therelational calculation only on data with close block ad-dresses to ensure computational accuracy. Otherwise, the


AB

C

D

AB

D

AC A

Time

Blo

ck a

ddr

ess

A, B, and C have a similar block address, but not access time

B and D have a similar access time, but their relation is not mined

Fig. 1. An RDF-based relationship mining algorithm thoroughly exploitsthe relationships among A, B, and C. However, the relationship betweenB and D is difficult to mine.

method may generate n2 relations. These methods ignoredata with large block address distances and have strongtemporal locality, as shown in Fig. 1.

Additionally, we identified the problem in which boththe OF and RDF are unable to perceive the size of thedata. In a real storage environment, the storage systemconcurrently receives access requests for both large data andsmall data. As shown in Fig. 2, when large data are accessed,the relevance of the data in the cache around the accesseddata may also be affected because they occupy most of thecache space.

Therefore, the data size is a notable factor that therelationship mining algorithm should consider. The datanear the larger data in the access sequence should be lessrelevant. We propose a feature representation based oncache transactions, which introduces the size of the data tothe correlation mining process to solve this problem; thus,the degree of access relation between data is dynamicallyadjusted according to the size of the accessed data.

Bb(8 MB) Bc(4 MB) Bd(3 MB)Ba(4 MB)Sa(16 KB) Sb(8 KB) Sc(8 KB)

Weaker Correlation

Sv(16 KB) Sm(128 KB) Sk(16 KB)Su(32 KB)Sa(16 KB) Sb(8 KB) Sc(8 KB)

Stronger Correlation

Access Sequence

Fig. 2. In the case of a limited cache size, a higher correlation isobserved between Sa, Sb, and Sc in the lower access sequence, sincethey are more likely to appear in the cache at the same time.

2.2 Data locality statisticsIn a distributed storage system, data access is characterizedby spatio-temporal locality. Most studies use this featureto perform relation calculations and mine the access rela-tionships among data [22]. However, since a large amount

Fig. 3. The blue line shows the distribution of data with a strong cor-relation in terms of the block address distance and the red line is theCDF of the blue line; we consider two data to be related when theyare always accessed sequentially. Approximately 60% of the data withstrong correlations exist in close block addresses, but a large number ofdata with large distant block addresses (i.e., data with distances greaterthan 1.4e10) still exist.

of data is stored in the storage system, the analysis of thepairwise relationships generated by the data may consumea large amount of computing resources. Therefore, moststudies use the spatial locality of data as a range limitationof the access relation mining model. However, by analyzingthe block address distance of the data with the associatedrelationship in the trace, temporal locality is indeed ob-served between the data with large block address distances,as shown in Fig. 3. As a result, since this method ignores alarge number of relationships, the improvement in efficiencyoccurs at the expense of accuracy.

Researchers proposed Mithril [21] for mining the accessrelation of data with large block address distances, basedon the idea that most related data have similar access times.Mithril mines the access relations among the data with thesame access times. However, we calculated the difference inaccess times to the data with access relevance, as shown inFig. 4. Approximately 70% of the data with access relevancehave the same access times, and the proportion changeslittle under different relation strength limits. Therefore, theexclusive use of the access time of data for limiting the rangeof relation mining is inappropriate. Among the factors listedabove, the relation strength W is defined as the averageaccess interval between data, as shown in Equation (1).

Wx,y =

∑Ax

i=1 min (|seqxi − seqyj |)Ax

, j ∈ Ay (1)

For data x, Ax is total number of accesses to data x, seqxiis the ith access sequence number of data x.

For closely related data, an analysis of the correlationof the data with similar access times and similar blockaddresses is efficient. However, as discussed above, thisapproach may ignore many relationships between blocksthat are farther part in block address. Therefore, this methodshould be used only as a preliminary relationship miningmethod. A correlation mining algorithm that can mine dataaccess relationships for data that are farther apart in blockaddress is needed as a supplement.


TABLE 1Comparation of prefetching approaches.

Algorithm Feature used Space overhead Time overhead Wasted Prefetches Method

AMP [29] OF Low Low Moderate Adaptive Asynchronous PrefetchNN [28] OF Low High High Neural Network

MITHRIL [21] RDF High Moderate Moderate Distance between inverted data access indexTombolo [30] OF High Moderate Moderate Directed graph

C-MINER [17] OF Moderate High Moderate Frequent patternWildanis [20] RDF High Moderate Low Relation graph

CTDGM CTF Moderate Moderate Low Cache transactions and Complete subgraph

Fig. 4. We use 4 different relation strength limitsW < 20,W < 50,W <100 and W < 150 to verify the effect of the difference in access timeson data correlation. Only approximately 70% of the data with accessrelevance have the same number of access times, and the proportionchanges little under different relation strength limits.

2.3 Prefetching algorithms

Existing data prefetching algorithms are mainly dividedinto two classes that 1) analyze association rules amongdata and do prefetch in units of data, and 2) divide datawith strong relationships into groups, store them in con-tiguous locations and prefetch them in groups. We com-pare the popular cache prefetching algorithms in Table 1.For the first type of prefetching algorithm, when a datais accessed, the data related to it are prefetched and putin the cache. On this basis, researchers have proposedprefetching algorithms based on a neural network (NN)[28], frequent items [31], probabilistic graphing [32], anddistance [20]. The total disk I/Os of this method are definedas len (trace)∗(1− accuracy)+sum (prefetch). Regardlessof the accuracy, the reduction of the disk I/O depends onthe prefetching strategy. However, for a data access request,whether it is a cache hit or a miss, new disk access requestsmay be generated because all the data associated with thecurrently accessed data may not be included in the cache.Therefore, this method has a significant effect on the I/Oresponse time, but it may increase the I/O burden of thedisk. In the second type of prefetching algorithm, data areread in groups. When data in the group are read, the entiredata group is prefetched and put in the cache. Therefore, thedata in the group should be strongly correlated. The state-of-the-art graph-based data relationship mining algorithm usesthe data as the point and the relationships between the pairs

of data as edges to establish the relationship graph, andthe complete sub-graphs are the results of the data groupdivision [20]. This type of relationship mining algorithmcomprehensively identifies data groups displaying strongcorrelations. However, this model cannot mine the relationsbetween data with large block address distances since it isbased on the RDF, as discussed in Section 2.2. Additionally,this model cannot provide a clear division method for databelonging to more than one group, as shown in Fig. 5.

A

B

DC

Related in block address

and time

EData C belongs to

two groups

Fig. 5. Data C is strongly related to A/B and E/D, but the RDF-basedcomplete subgraph search algorithm cannot provide a clear divisionmethod for C.

Therefore, a data correlation mining algorithm that effi-ciently mines data relations with significant block addressdistances and divides the data into independent groups isneeded.

3 FEATURE EXTRACTION METHOD BASED ON THECACHE TRANSACTIONS

In this chapter, we present the definition of and methodused to calculate the cache transaction, and propose a dataaccess feature based on the cache transaction to solve theproblems we discussed in Section 2. We use the size of thecache as the basis for partitioning the cache transaction todefine the conditions for the access relationship between thedata. The feature extraction model includes two steps: cachetransaction construction and CTF extraction, as shown inFig. 6.

3.1 Definition of a cache transactionThe cache transaction analyzes the historical data accesstrace and simulates a cache queue with a space size ofM . Whenever M -sized data are cached, we build a cachetransaction using the block address of all the data in thecache queue as the data index. Therefore, a cache transaction


Cache transaction featureBlock access trace

2865143

8

1

0

0

0

Block address

Block size

52489162

16

5135413

8

657242

8

4542346

16

5135413

8

52489162

16

9438123

32

5135413

8

Cache transaction constructor

8341168

8

2865143

8

52489162

16

5135413

8

52489162

16

5135413

8

657242

8

4542346

16

5135413

8

9438123

32

2865143

8

52489162

16

1

0

1

0

657242

8

0

1

0

0

4542346

16

0

1

0

0

5135413

8

1

1

1

0

9438123

32

0

0

0

1

8341168

8

0

0

0

0

Transaction 0 Transaction 1

Transaction 2 Transaction 3

1

0

1

0

Feature vectorData

Fig. 6. Schematic depicting the cache transaction feature extraction method.

is understood as a collection of data indexes that mayappear in the cache together.

We have attempted to incorporate multiple cache strate-gies as a queue for cache transactions. Finally, we selectedFIFO as the strategy since it is the ideal approach in termsof both feature representation and computational efficiency.Unlike LRU, LFU, and other cache strategies, FIFO retainsthe timing characteristics. Although cache strategies suchas LRU and LFU provide a higher cache hit rate for dataaccesses, these strategies cause the cache to be occupied withfrequently accessed data. Thus, most of the relationships weobtain are generated between frequently accessed data, andthe association between data with lower access frequenciesis unable to be fully expressed. Moreover, in real storagesystems, the number of data access logs is extremely large,and data access patterns change rapidly. The efficiency re-quirement of the feature extraction method makes the FIFOalgorithm a better choice.

In this section, we provide a detailed description ofthe method used to calculate the cache transaction. For acache transaction Ti = {b1...bn}, b1...bn represents the blockaddress of data in the cache transaction. When a new datax is accessed, we determine whether bx has already beenadded to Ti. If bx is included in Ti, then no operation isperformed; otherwise, bx will be added to Ti and the FIFOcache. When a cache replacement occurs, the cache replacesthe size counter Out = Out + size (dout), where dout is thedata that was replaced from the cache, and size representsa function that computes the size of a given data. WhenOut ≥ M , Ti will join the cache transaction queue T andstart the calculation of Ti+1. The pseudocode of the cachetransaction division algorithm is shown in Algorithm 1.

Algorithm 1. Cache transaction division

Input: Max cache size MData access sequence d

Output: Cache Transactions T1 : T ← {}2 : Counter ← 03 : Out← 04 : Cache← {}5 : For i in d6 : if bi not in Cache7 : Cache← Cache ∪ bi8 : Counter ← Counter + size (i)9 : while Counter > M10: Counter ← Countersize (Cache [0])11: Out← Out+ size (Cache [0])12: Remove (Cache [0])13: if Out ≥M14: Out← 015: T ← T∪ Cache16: Cache← {}17: Return T

For the size of the cache queue M , we generally setM to less than or equal to the actual cache size in theruntime environment. A smaller M ensures that the cachetransaction contains fewer data than a larger M . Thus, datahave less of a possibility to have the same feature. Therefore,M is defined as the strictness of feature extraction betweendata.

3.2 Data access feature representation based on thecache transaction

According to our previous analysis, a good data featuremust have the ability to calculate independent featuresfor each data. We introduce a data feature representationmethod based on the cache transactions that represents theappearance of each data in the cache to solve this problem.


We invert the cache transactions to obtain the CTF of thedata, as shown in Fig. 7. We design a data feature vectoras V , where the dimension of V is the total number ofcache transactions. For a data i and cache transaction Tj ,if bi appears in Tj , then Vi,j = 1; otherwise Vi,j = 0, asshown in Equation (2).

Vi,j =

{1 bi ∈ Tj0 bi /∈ Tj

(2)

Data A

Data B

Data C

Data X

Cache Transactions

CTF of data

Invert

Data A Data X Data S Data Q Data R Data C Data Y

Data E Data R Data A Data X Data V Data N Data Z

Data A Data X Data T Data U Data O Data P Data H

T0

T1

Tn

T0 T1 T4 T5 T7 T10 Tn

T4 T5 T14 T19 T26 T77 T128

T0 T143T69

T0 T1 T4 T5 T7 T26 Tn

Fig. 7. For cache transactions T0...Tn, after inversion, we obtain thecache transaction features of all the data. We use these features as thebasis for the data chunking method.

We obtain valuable information from this feature. Forexample, the Euclidean distance of two data are regardedas the access relevance of the two data. |Vi,∗| representsthe data access frequency. Compared with RDF and OF,the CTF more conveniently measures the similarity betweendata and represents the independent feature with timingcharacteristics of the data. In addition, CTF is more sensitiveto the size of the data in the access sequence than RDFand OF, as addressed in Section 2.1. An explanation forthis finding is that larger data take up more space in acache transaction, reducing the number of data in the cachetransaction; thus, larger data are correlated with fewer data.

4 DATA GROUPING METHOD

Since CTF has advantages in terms of computability andattribute coverage, in this section, we propose a data group-ing model, CTDGM, based on CTF. The model completelyexploits the spatio-temporal locality features among dataand strictly groups data without group-across based on thestrength of the data relation. The model consists of threeparts. Firstly, the data are divided into different regionsaccording to their access frequency and block address.Secondly, the data are clustered into chunks within eachregion using the CTF. Finally, a data relationship graphis constructed from the relationships between the chunksand a complete subgraph searching algorithm with highcorrelation priority is used to obtain the data groupingresult. A schematic depicting the model is shown in Fig.8.

4.1 Data chunking algorithm based on cache transac-tion features

As discussed in Section 2.1, the generation of all pairwiserelations among the data in a storage system is too costly.In Section 2.2, we described how closely related data dis-play similar numbers of access times and block addresses.Therefore, we initially merge these highly correlated datainto chunks.

We propose a pre-blocking model to limit the amount ofdata input into the chunking algorithm and to improve thecomputational efficiency of the algorithm. The pre-blockingalgorithm uses the access times of the data as the ordinateand the block address of the data as the abscissa to establisha plane. The plane is divided into areas Ai = (x0, x1, y0, y1)where x0 is the left horizontal coordinate of area Ai, x1 isthe right horizontal coordinate of the area, y0 is the lowerordinate of the area Ai, and y1 is the upper ordinate ofthe area. The block addresses are divided into the average-cut regions. Regarding access times, although closer accesstimes indicate stronger relevance, the relationship is non-linear. We assume two cases for Data a and Data b. In thefirst case, |VData a,*| = 1 and |VData b,*| = 11; in the second,|VData a,*| = 100 and |VData a,*| = 110. As we can see, inboth cases, |VData a,*| − |VData b,*| = 10. Where |VData a,*| is thetotal times Data a shows in all transactions. However, inthe second case, a higher probability of a strong correlationexists between the two data than in the first case. Therefore,for access times, the partitioning method should be strictfor data with low access times and loose for data with highaccess times. Assuming that the maximum block address ofthe storage system is Q and that Q is divided into q regions,we number and define the regions using Equation (3):

Ai =

(i

qQ,

i+ q

qQ,

pi

qQ,

pi+1

qQ

)(3)

where p indicates the division coefficient of the accesstimes. A larger value of p indicates a higher tolerance forthe data access gap in the chunking process, and the amountof data in each area is larger than that when p is small. Asample of our data pre-blocking model is shown in Fig. 9.

Then we then clustered the data in each area based ontheir CTFs. For two data, Data x and Data y, the CTFs areVData x,∗ and VData y,∗. We define the relationship distanceDxy between the data using Equation (4).

Dxy = |VData x,∗ − VData y,∗| (4)

We define a strong relationship between Data x andData y if and only if they satisfy the relationship in Equation(5).

Dxy <|VData x,∗|+ |VData y,∗|

2σ, (σ ≤ 1) (5)

The term σ indicates the relationship strength threshold.When σ is smaller, there is less tolerance for the differencein the CTF of the data. When σ = 0, only data with the sameCTF are merged into data chunks. Finally, we use the datawith a strong relationship {C} as the chunks of data. Thepseudocode of the data pre-blocking algorithm is shown inAlgorithm 2.


Cache transactions

2865143

8

5248162

16

5135413

8

5248162

16

5135413

8

657242

8

4542346

16

5135413

8

9438123

32

Transaction 0

Transaction 1

Transaction 2

Transaction 3

Block addressA

ccess

tim

es

Cache transaction feature

ChunkingPre-blocking

Data grouping

Result

Fig. 8. Schematic depicting the data grouping model. Different colored geometries represent different data. During the pre-blocking and chunkingphases, the data are merged into blocks according to the block address, access times and CTF. Data, such as the green and sandy brown blocksare merged into one block. Second, we group the blocks according to the cache transactions and the relationship graph to get the final groupingresult, such as the data in the dashed box in the Result process.

Fig. 9. A sample of the data pre-blocking algorithm on MSR mds 1 trace.

Algorithm 2. Data pre-blocking algorithm

Input: Relation strength parameter σData set DATA

Output: Data chunks C1 : C ← {}2 : for d in DATA3 : k = dblockAddress

m +mlogpdaccessT imes

4 : Ak ← Ak ∪ d5 : n = max (k)6 : for i from 1 to n7 : C ← C ∪Hcluster (Ai, σ)8 : Return C

4.2 Data group mining algorithm

The data chunking algorithm described in Section 4.1 ef-ficiently merges data with strong correlations into chunks.However, the data in a chunk belong to the same area thatwas divided by the pre-blocking algorithm. At this stage, weneed to address only the boundary relationships across ar-eas, which significantly reduces the number of relationships.

Therefore, the data grouping model efficiently mines all datarelationships, which is the bottleneck of existing algorithms.The main purpose of the data grouping algorithm is to com-pletely mine the data relationships across area boundariesthat are ignored by the data chunking algorithm, i.e., tomine the relationships between chunks. At the same time,this method effectively avoids the problem of group-acrossdata.

The chunks that appear in the same cache transactioncould be relevant. The condition of whether the chunk ap-pears in a transaction is similar to that of the data. Namely,if Data x in chunk Cy appears in Ti, then Cy appears in Ti.For chunks Cx and Cy , the access relation between them isdefined as R (Cx, Cy), which indicates the number of timesthey appeared in the same cache transaction. Since stronglyrelated data with spatial locality have been merged intodata chunks, the temporal complexity of this computationaloperation is nearly linear. For data with strong relations, thefrequency at which they occur in the same cache transactionshould be similar to the frequency at which they appearindependently. Therefore, we filter the legitimacy of datarelationships. We define a legal set of data relationships asL and define a relationship threshold parameter α. Whenthe chunks satisfy Equation (6), we add a relationship be-tween Cx and Cy into L. Subsequently, we construct therelation graph using L to mine the data groups with strongcorrelations between chunks.

R (Cx, Cy) ≥ max(|VCx| ,∣∣VCy

∣∣)α, (0 ≤ α ≤ 1) (6)

We propose a complete subgraph search algorithm witha strong relation priority. Initially, we construct a datarelation graph. We use the chunk as a point and the rela-tionship between the chunks as an edge. Then, we add therelationships between chunks in L to the graph accordingto strength from high to low. When we add R (Cx, Cy) tothe graph, we calculate the number of relationships for thegroup in which they are located using Equation (7).


R(GCx

, GCy

)= R

(GCx

, GCy

)+ 1 (7)

Here, GCxis the group to which chunk Cx belongs and

R(GCx

, GCy

)represents the strength of the relationship

between GCxand GCy

. For the merging process, GCxand

GCyare merged to a new complete subgraph when they

satisfy the condition in Equation (8).

R(GCx , GCy

)≥ |GCx |

∣∣GCy

∣∣µ, (0 ≤ µ ≤ 1) (8)

|GCx| is the number of chunks contained in subgraph

GCxand µ is the relationship threshold parameter. A larger

µ indicates that the data in a group have a stronger relation-ship with each other. The pseudocode of the data groupingmodel is shown in Algorithm 3.

Algorithm 3. Data grouping algorithm

Input: Cache transaction T , threshold parameter α, µOutput: Data groups G1 : for t in T2 : S ← {}3 : for d in t4 : S = S ∪ Chunk (d)5 : for Ci, Cj in S6 : R (Ci, Cj) = R (Ci, Cj) + 17 : Sort (R)8 : for r in R9: if R (r.x, r.y) ≥ max (|Vr.x| , |Vr.y|)α10: Gx = Group (r.x)11: Gy = Group (r.y)12: if R (Gx, Gy) ≥ |Gx| |Gy|µ13: Merge (Gx, Gy)14: G← G−Gx −Gy ∪Gx,y

15: return G

Chunk (d) returns the chunk to which data d belongs,Sort (R) sorts the relation strength into ascending order, r.xand r.y are the chunks of the relation r, and Group (r.x)returns the group to which chunk r.x belongs.

Unlike the traditional complete subgraph-based datagrouping model, our algorithm performs data-to-data rela-tionship mining according to the relationship strength fromhigh to low. Two data/groups are immediately merged intoa new group, i.e., a new node in a relation graph whenthey satisfy Equation (8). As shown in Fig. 10, this methodensures that data do not simultaneously belong to multiplegroups at the same time.

5 EXPERIMENTS AND ANALYSISIn this chapter, we evaluate the proposed feature extractionmethod based on the cache transaction and data groupprefetching algorithm. We compare the proposed modelwith popular feature extraction methods and prefetchingalgorithms, including prefetching algorithms based on RDFand OF, such as ANN [28], and Mithril [21].

All experiments described in this paper were performedon a Huawei RH2288 V3 server with a Xeon E5-2620 v4CPU, 64 GB of RAM, 4 TB, 7200 RPM SATA disk, and a 4096-byte block. The system runs CentOS Linux release 7.5.1804,

58

59 5959

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

58

75

63

52

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

C1 C2

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

Initial graph Add edge 75

Subgraph C1 C2 found

Add edge 70


Add edge 68

No subgraph found

Add edge 66, 63, 62

No subgraph found

Add edge 60


C0

C3

Add edge 59, 58

Subgraph C1 C2 C3 C4 found

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

C0 C1 C2

C3

C5 C6 C7

C4

57 66 68

6256

70

53

60

75

63

52

Add edge 57, 53

No subgraph found

Add edge 52


58 58

59585959

595959

5858

5858

Fig. 10. Schematic depicting a high-correlation priority data groupingalgorithm. Nodes in different colors represent different groups, and whitenodes are chunks that do not belong to any group. Black lines representassociations between groups to be mined, and red lines representrelationships between groups that have been mined.

64 bit, with Linux kernel Version 3.10.0. In addition, thealgorithms we proposed can be deployed in a variety ofdistributed storage systems. We evaluate the cache hit rateperformance of our model by block I/O traces. Furthermore,to show the effect on the proxy server, we implemented ourmodel on the controller server in a three-node OpenstackSwift distributed storage system to group the related dataacross storage nodes.

The dataset we selected to test our model is MSR [33],which contains 1-week block I/O traces of enterprise serverson 13 servers at Microsoft Research Cambridge. MSR iswidely used in storage research since it is a publicly avail-able block I/O trace and contains multiple data accessbusiness types. Data centers run in multi-application, multi-user scenarios. These scenarios require a feature extractionmethod and data grouping model that can handle varioustypes of data services. Therefore, the MSR dataset is a suit-able and convincing selection for testing the performanceand universal applicability of our model [34].

The grouping algorithm adjusts the strictness of thegrouping by the threshold parameter. For the cache trans-action size threshold M and the clustering stop threshold µ,a higher threshold µ increases the strictness of the criteriafor data grouping, while the number of data in each groupand the total number of groups decreases. We adjusted theparameters σ in the experiments and analyzed the changesin Fig. 11. As the value of parameter σ increases, the totalnumber of groups decreases, while each group containsmore data than does a smaller σ. Thus, a more rigorouschunking process results in chunks with a finer granular-ity, and the correlation between the data in the chunks isstronger. This approach causes the data grouping process to


work better, but it may lead to a significant increase in thecomputational complexity of the data grouping process.

Additionally, as shown in Fig. 12, a decrease in µweakens the conditions for merging data into groups, butit affects only the results of grouping, and the effect oncomputational complexity is slight. µ should be less than 1,because some of the relations across two cache transactionsmay be ignored by the model. The use of an excessivelystrict division method prevents the chunking and the group-ing model from fixing these errors, and thus some of thestrongly related data are unable to be merged into a groupby the model.

For the parameter M , the number of larger groupsincreases as M increases, but the total number of groupsdecreases, as shown in Fig. 13. We analyze this parameterbecause the increase in the cache transaction size parameterM facilitates the generation of access relationships betweendata. Thus, the number of data groups is easier to determine.However, a larger M also potentially increases the compu-tational complexity of the data grouping process, since thenumber of pairwise relationships between data is increas-ing. At the same time, the size of M should be smaller thanthe actual cache size in the running environment. Otherwise,the grouping model may generate access relationships be-tween data that do not simultaneously appear in the cache.Regarding the number of groups, the stable trends implythat the data in groups still exhibit strong relationships witheach other, confirming the accuracy of the relation miningalgorithm.

We validated the effect of grouping on all datasets ofMSR to verify the effectiveness of our data grouping algo-rithm. We experimented with the cache hit rate in the group-ing model. Compared with the existing cache prefetchingmodel, the model we proposed effectively improves thecache hit rate. Notably, our algorithm significantly improvesthe cache hit rate when the cache space is small, as shownin Fig. 14.

We used three different parameters in the experimentto determine the effect of the cache transaction parameterM on the grouping result. When the cache size is lessthan 0.0016% and the parameter M = 8, the effect of thegrouping is not as good as when the parameter M = 2.However, as the cache size increases, the data group withM = 8 displays a gradually better cache hit rate than thegroup with M = 2. Therefore, the mining scope of the cachetransaction should be adjusted to accommodate changes inthe size of the runtime environment cache.

We show the average hit rate of each algorithm in Table2 to more clearly display the advantages of the proposedalgorithm. The CTDGM model outperforms the existingprefetching model in terms of cache hit rate, particularlywhen the cache space is small. Additionally, a smaller cachetransaction size parameter M works better in smaller run-time cache environments, which also confirms our interpre-tation of the data presented in Fig. 13.

For the system I/Os, we implemented the groupedresults in two forms on proxy server in a three-nodeOpenstack swift distributed storage system. One stores thegrouping result and does not merge the data. When thestorage system accesses one data, data in the same groupare sequentially prefetched into the cache. In the other case,

we merge the data that belong to a group and save them in acontinuous storage area. When the storage system accessesone data, the entire group is prefetched into the cache. Wecount the number of I/Os using these two methods.

As shown in Fig. 15, when data groups are stored incontinuous storage areas, the number of I/Os is up to 60%lower than that of the one-step prefetching method possiblybecause the recall rate of the one-step prefetching methodis relatively low since the presence of mistaken data lead toan increase in the number of I/O requests. However, I/Osdecrease significantly when using the merging approach,because the related data are accessed by one I/O request.Therefore, the RDF-based data correlation mining algorithmcannot easily optimize the number of I/Os since it is unableto provide explicit group divisions of the correlated dataand has a limited relation mining range.

As shown in Fig. 16, we performed a stability test onthe time range of the algorithm mining results to showthe stability of the relationships within the data group. TheMSR prxy dataset has a long period and multiple users, andthe users access interest may change. Therefore, we used thisdataset to test the effect of the algorithm. In the experiment,we used the first 3 million accesses as a training set forthe grouping model. Then, we used the model to verifythe changes in the cache hit rate as the number of accesseschanged.

For the prxy 0 dataset, the overall trend of the cachehit rate for the proposed grouping model is similar to theLRU cache. Notably, at approximately 4.5× 106 and 6× 106

accesses, the cache hit rate of the grouping model decreased,and the hit rate of the LRU cache increased, possibly becausethe storage system has an access pattern that has not beenmined by the grouping algorithm. For prefetch models, thissituation is unavoidable because the model always performsdata prefetching based on existing access patterns. However,its probability of occurrence is relatively low. In the prxy 1dataset, the performance of the cache hit rate is more stablethan the that in the prxy 0 dataset. The cache hit rateof the storage system showed an overall decrease up toapproximately 6.5 × 106 accesses. We analyzed this trendbecause the access pattern changed. Therefore, our groupingstrategy has a relatively stable effect on the cache hit rateover time; thus, we can confirm the stability of our proposedalgorithm.

6 RELATED STUDIES

As a method for effectively improving the access efficiencyof storage systems, prefetching is a key focus area forresearchers in the storage field [29], [35], [36], [37]. With theexpansion of storage space and the development of com-puting power, researchers have shifted their research focusto the optimization of the accuracy and efficiency of datacorrelation mining algorithms. However, data have fewerfeature attributes and are present at a greater quantity thanin other subjects. Therefore, the main focus of data accessrelation mining is the representation of data correlationfeatures and computing efficiency.

6.1 Data access relationship characteristicsIn terms of data access feature representation, researchersinitially used the OF-based approach to estimate data cor-


Fig. 11. Effect of variations in σ on the grouping results.

Fig. 12. Effect of variations in µ on the grouping results.

TABLE 2Averange cache hit rate for prefetching models

0.001 0.002 0.004 0.008 0.016 0.032 0.064 0.128

LRU 0.2908 0.3203 0.3520 0.3901 0.4287 0.4558 0.4933 0.5121ANN 0.3318 0.3590 0.3893 0.4174 0.4512 0.4757 0.5046 0.5223

Mithril 0.4786 0.5326 0.5733 0.5961 0.6139 0.6310 0.6471 0.6631Group (0.1,8,0.5) 0.4724 0.5797 0.6326 0.6581 0.6720 0.6801 0.6857 0.6932Group (0.1,2,0.5) 0.5961 0.6264 0.6440 0.6545 0.6622 0.6698 0.6772 0.6869

relations. OF treats data as an independent object using aunique feature, such as the data storage path and one-hotencoding, to identify data. Most researchers have used themetadata and temporal locality to estimate data correlations.However, data centers have changed their storage strategiesto handle the increasing amount of data. Block storage andobject storage have become the primary storage strategies,making block-based data correlation mining algorithms amainstream approach. Since blocks have fewer independentattribute features than files, researchers use an RDF toestimate data correlations. RDF combines the block addressand the temporal locality of the data to define the datacorrelations. Thus, all properties of an RDF are numericallyoperated. Most studies estimate the correlations by defining

the distance between data. However, the distance definitionmethods adopted by the RDF are mostly static and unableto easily adapt to the dynamic changes in the data accessscenario. Researchers have proposed the use of the overallaccess density of the storage system as a parameter fordynamically defining the distance as a method to solve thisproblem [20]. Since the prefetch algorithm is cache-oriented,as long as two data can appear in the cache at the same time,they have a potential correlation. Additionally, the accessdensity focuses on the level of concurrence and ignores thedata size and cache size.

We propose the cache transaction based on the contentof the cache for data correlation mining, because it is notaffected by changes in the data access scenario and can


Fig. 13. Effect of variations in M on the grouping results.

0.002 0.008 0.032 0.128

0.5

1

hm1

0.002 0.008 0.032 0.128

0.8

0.9

1

mds0

0.002 0.008 0.032 0.128

0.1

0.15

0.2

mds1

0.002 0.008 0.032 0.128

0.4

0.6

0.8

prn0

0.002 0.008 0.032 0.128

0.5

1

prn1

0.002 0.008 0.032 0.1280

0.5

1

proj0

0.002 0.008 0.032 0.1280

0.01

0.02

proj1

0.002 0.008 0.032 0.1280

5# 10-4 proj

2

0.002 0.008 0.032 0.1280

0.5

1

proj3

0.002 0.008 0.032 0.1280

0.2

0.4

proj4

0.002 0.008 0.032 0.1280

0.5

1

prxy0

0.002 0.008 0.032 0.128

0.5

1

prxy1

0.002 0.008 0.032 0.1280.6

0.8

1

rsrch0

0.002 0.008 0.032 0.128

0.5

1

rsrch1

0.002 0.008 0.032 0.128

0.6

0.7

0.8

rsrch2

0.002 0.008 0.032 0.1280

0.5

src10

0.002 0.008 0.032 0.1280

0.01

0.02

src11

0.002 0.008 0.032 0.128

0.5

1

src12

0.002 0.008 0.032 0.1280.75

0.8

0.85

0.9

src20

0.002 0.008 0.032 0.1280

0.5

src21

0.002 0.008 0.032 0.1280

0.1

0.2

src22

0.002 0.008 0.032 0.128

0.6

0.8

stg0

0.002 0.008 0.032 0.128

0.5

1

stg1

0.002 0.008 0.032 0.128

0.6

0.8

1

ts0

0.002 0.008 0.032 0.128

0.5

1usr

0

0.002 0.008 0.032 0.1280

0.005

0.01usr

1

0.002 0.008 0.032 0.1280

0.05

0.1usr

2

0.002 0.008 0.032 0.1280.6

0.8

1wdev

0

0.002 0.008 0.032 0.128

0.8

0.85wdev

1

0.002 0.008 0.032 0.128

0.5

1wdev

2

0.002 0.008 0.032 0.128

0.76

0.78

wdev3

0.002 0.008 0.032 0.128

0.6

0.8

web0

0.002 0.008 0.032 0.128

0.6

0.8

1web

1

0.002 0.008 0.032 0.1280

0.5

1web

2

0.002 0.008 0.032 0.1280

0.1

0.2web

3

Cach

e h

it r

ate (

%)

Cache size (% of the total data size)

ANN-LRU

LRU

Mithril-LRU

GROUP-LRU(σ=0.1,t=8,μ=0.5)

GROUP-LRU(σ=0.1,t=2,μ=0.5)

Fig. 14. Cache hit rate of the MSR dataset. The cache sizes we use here are the percentage of the total data size in traces.

calculate the data correlation stably without adding param-eters. Moreover, the cache transaction introduces the datasize into the access feature, which expands the propertycoverage of the feature. We also designed the CTF based oncache transactions to ensure that the data have independentcomputable features. Since this feature is represented asa vector, it is also suitable for models that require vectorcalculation, such as a neural network. To the best of ourknowledge, the CTF is the first method to establish a vector-based data-independent feature. The proposed method pro-vides new ideas for research into data correlation miningand other fields.

6.2 Data correlation mining algorithm

For the data correlation mining model, the existing studiesare divided into two classes. The first includes online algo-rithms that determine which data to prefetch when an accessrequest arrives, such as Mithril [21] and Tombolo [30].

Mithril prefetches data with the same access times andsmall access intervals as the current data. Therefore, thisalgorithm must establish an index of access frequency forall data stored in memory to ensure that a prefetch decisionis made immediately when data are accessed. This solu-tion quickly adapts to changes in data access patterns anddisplays great flexibility. However, in distributed storagesystems that save massive amounts of data, Mithril requiresa large amount of memory to index the data. Moreover,the scheme does not consider the effect of spatial localityon data access relevance. The calculation of the correlationsamong all data can place a heavy computational burden onthe server.

Tombolo uses a graph-based algorithm by establishing aone-way edge between data that are successively accessed,but it uses a substantial block address distance to forma directed graph. When data are accessed, the nodes thatare reachable from the current node are the candidateset, thereby determining the prefetched data. However, the


0.001 0.002 0.004 0.008 0.016 0.032 0.064 0.128


0

1

2

3

4

5

6

7

8

I/O

tim

es

# 105 proj

0

Group-LRU-I/O

Mithril-LRU-I/O

LRU-I/O

(a)

0.001 0.002 0.004 0.008 0.016 0.032 0.064 0.128


0

2

4

6

8

10

12

14

I/O

tim

es

# 104 rsrch

0

Group-LRU-I/O

Mithril-LRU-I/O

LRU-I/O

(b)

Fig. 15. I/Os in the MSR proj 0 and rsrch 0 datasets

maintenance of this directed graph is costly, since the accesspattern in the data center is highly random and the accessfrequency is high.

The other approach is an offline association miningalgorithm that calculates the correlations among data inadvance. When the data are accessed, the model prefetchesthe associated data and puts them in the cache accordingto a predetermined prefetching strategy, such as the strate-gies described by Wildani [20] and Zhu [12]. The methodreported by Zhu calculates the access distance between datawhose access times differ by less than 10%, and the authorsdefine the distance according to the average access intervalbetween data. This method explicitly classifies the data butdoes not consider the spatial locality and the effect of thedata size on the content of the cache. The method describedby Wildani pre-calculates the access distance based on theblock address, access time offset, and the overall data accessdensity of the storage system. Subsequently, the algorithmfilters out the data access relationships with large blockaddress distances and uses the remaining relationship as theedge, and the data are defined as points so that a completesubgraph search can be performed. Although the subgraphsearching process is dynamic, the graph is costly to maintainsince it must be precalculated and saved in the memory.

0 1 2 3 4 5 6 7 8 9 10

Access sequence # 106

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Cac

he h

it r

ate

Changes in the cache hit rate according to access to prxy0

(a)

0 1 2 3 4 5 6 7 8 9 10

Access sequence # 106

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Cac

he h

it r

ate

Changes in the cache hit rate according to access to prxy1

Group-LRU(cache size=0.004%)



LRU(cache size=0.002%)

(b)

Fig. 16. Changes in the cache hit rate over time as the number ofaccesses to the MSR prxy dataset increases. The cache hit rate of theLRU is used as the baseline to show the change in the access pattern.

Additionally, the algorithm filters out most of the data thatare located far from the block address. However, basedon our statistical results presented in Section 2.2, existingrelationships between the data have a large block addressdistance. Therefore, the block address distance is used tolimit the mining range, which may improve the algorithmcalculation speed, but decrease the accuracy of correlationmining.

6.3 Discussion

We propose the data correlation mining algorithm CTDGMin this paper. Unlike the existing algorithms, our algorithmestimates the access relationships of different granularitiesin two steps, which expands the scope of relation mining.Additionally, the algorithm is designed to clearly define datathat may belong to multiple groups, which solves the across-group problem described in Section 2.3. Therefore, CTDGMstill provides a high cache hit ratio when the cache space issmall (0.001% of the total data size). Thus, our model canbe applied to environments other than distributed storagesystems, and it even serves as a solution for disk internalcaching.


We used the offline association mining algorithm be-cause prefetched data are time-sensitive, particularly in theblock layer. Although the online prefetching models maynot introduce overhead in the I/O critical path since it canrun in the background, the computing time for the groupsmay increase when the I/O concurrency is high. Therefore,the prefetched data may be out of date. However, high-concurrency scenarios are the optimal scenarios for usingprefetching algorithms.

However, the access number covered by the CTF islimited. According to our calculations, for a 4 MB cachetransaction, the CTF can cover approximately 4 TB of dataaccess. The proposed method is superior to the existingfeature extraction method in this regard, and this featureis acceptable when the access characteristics of the dataare relatively stable. However, as the number of data andaccess concurrency in the data center continue to increase,and for complex business scenarios with multiple users andmultiple applications, the data access features must cover alarger access space. We postulate that subsequent studieswill be able to expand these features by optimizing thefeature coding method. Additionally, CTDGM exhibits highprocessing efficiency and low memory consumption. In thetest environment, the memory consumption of 3 millionvisits is less than 1 GB, and the calculation time is less than40 seconds. Thus, our mining algorithm will continue to runon the storage node.

7 CONCLUSION

We propose a CTF extraction method for defining the ac-cess features of the data and a high-correlation-first datacorrelation mining method. Compared to existing methods,our model considers the size of the data in the correlationanalysis, and the relationships between the data are clearlydefined based on the cache. Additionally, using the datachunking algorithm, our method completely exploits thetemporal and spatial locality features between data. It ad-dresses the inability of the traditional method to completelyexploit data relationships due to efficiency problems. Usingthe MSR dataset, the proposed model displays, on average,an average 12% increase in the cache hit rate at a smallcache size (0.001% of the total data size) compared with theexisting RDF and OF-based data correlation mining models.The merged group prefetching model reduces the numberof I/Os by 50% when the cache size is less than 0.008% ofthe total data size, thereby increasing the concurrency of thestorage system. We also propose a data feature representa-tion method based on cache transactions, which provides anew idea for the data correlation mining algorithm.

ACKNOWLEDGMENTS

This work is supported by the Fundamental Research Fundsfor the Central Universities (grant no. HIT.NSRIF.201714),Weihai Science and Technology Development Program(2016DXGJMS15), and the Key Research and DevelopmentProgram in Shandong Province (2017GGX90103).

REFERENCES

[1] K. Shvachko, H. Kuang, S. Radia, R. Chansler et al., “The hadoopdistributed file system.” in MSST, vol. 10, 2010, pp. 1–10.

[2] S. A. Weil, S. A. Brandt, E. L. Miller, D. D. Long, and C. Maltzahn,“Ceph: A scalable, high-performance distributed file system,” inProceedings of the 7th symposium on Operating systems design andimplementation. USENIX Association, 2006, pp. 307–320.

[3] S. Bende and R. Shedge, “Dealing with small files problem inhadoop distributed file system,” Procedia Computer Science, vol. 79,pp. 1001–1012, 2016.

[4] S. Niazi, M. Ronstrom, S. Haridi, and J. Dowling, “Size matters:Improving the performance of small files in hadoop,” in Proceed-ings of the 19th International Middleware Conference. ACM, 2018,pp. 26–39.

[5] J. Wang, J. Yin, D. Han, X. Zhou, and C. Jiang, “Odds: Optimizingdata-locality access for scientific data analysis,” IEEE Transactionson Cloud Computing, 2017.

[6] S. Chandrasekar, R. Dakshinamurthy, P. Seshakumar, B. Praba-vathy, and C. Babu, “A novel indexing scheme for efficient han-dling of small files in hadoop distributed file system,” in 2013International Conference on Computer Communication and Informatics.IEEE, 2013, pp. 1–8.

[7] S. Niazi, M. Ismail, S. Haridi, J. Dowling, S. Grohsschmiedt, andM. Ronstrom, “Hopsfs: Scaling hierarchical file system metadatausing newsql databases,” in 15th {USENIX} Conference on File andStorage Technologies ({FAST} 17), 2017, pp. 89–104.

[8] B. Dong, Q. Zheng, F. Tian, K.-M. Chao, R. Ma, and R. Anane,“An optimized approach for storing and accessing small fileson cloud storage,” Journal of Network and Computer Applications,vol. 35, no. 6, pp. 1847–1862, 2012.

[9] L. Lin, X. Li, H. Jiang, Y. Zhu, and L. Tian, “Amp: an affinity-basedmetadata prefetching scheme in large-scale distributed storagesystems,” in 2008 Eighth IEEE International Symposium on ClusterComputing and the Grid (CCGRID). IEEE, 2008, pp. 459–466.

[10] T. M. Kroeger, D. D. Long, J. C. Mogul et al., “Exploring thebounds of web latency reduction from caching and prefetching.”in USENIX Symposium on Internet Technologies and Systems, 1997,pp. 13–22.

[11] B. Dong, J. Qiu, Q. Zheng, X. Zhong, J. Li, and Y. Li, “A novelapproach to improving the efficiency of storing and accessingsmall files on hadoop: a case study by powerpoint files,” in 2010IEEE International Conference on Services Computing. IEEE, 2010,pp. 65–72.

[12] D. Zhu, H. Du, X. Qiao, C. Liu, L. Kong, A. Li, and Z. Fu,“An access prefetching strategy for accessing small files based onswift,” Procedia computer science, vol. 131, pp. 816–824, 2018.

[13] L. Xiong, Y. Zhong, X. Liu, and L. Yang, “A small file mergingstrategy for spatiotemporal data in smart health,” IEEE Access,vol. 7, pp. 14 799–14 806, 2019.

[14] M. Fu, D. Feng, Y. Hua, X. He, Z. Chen, W. Xia, F. Huang,and Q. Liu, “Accelerating restore and garbage collection indeduplication-based backup systems via exploiting historicalinformation,” in 2014 {USENIX} Annual Technical Conference({USENIX}{ATC} 14), 2014, pp. 181–192.

[15] A. T. Clements, I. Ahmad, M. Vilayannur, J. Li et al., “Decentralizeddeduplication in san cluster file systems.” in USENIX annualtechnical conference, 2009, pp. 101–114.

[16] A. Tamersoy, K. Roundy, and D. H. Chau, “Guilt by association:large scale malware detection by mining file-relation graphs,” inProceedings of the 20th ACM SIGKDD international conference onKnowledge discovery and data mining. ACM, 2014, pp. 1524–1533.

[17] Z. Li, Z. Chen, S. M. Srinivasan, and Y. Zhou, “C-miner: Miningblock correlations in storage systems.” in FAST, vol. 4, 2004, pp.173–186.

[18] X. Ge, X. Xie, D. H. Du, P. Ganesan, and D. Hahn, “Chewanalyzer:Workload-aware data management across differentiated storagepools,” in 2018 IEEE 26th International Symposium on Modeling,Analysis, and Simulation of Computer and Telecommunication Systems(MASCOTS). IEEE, 2018, pp. 94–101.

[19] J. Liao, D. Yin, and X. Peng, “Block i/o scheduling on storageservers of distributed file systems,” Journal of Grid Computing,vol. 16, no. 2, pp. 299–316, 2018.

[20] A. Wildani and E. L. Miller, “Can we group storage? statisticaltechniques to identify predictive groupings in storage systemaccesses,” ACM Transactions on Storage (TOS), vol. 12, no. 2, p. 7,2016.


[21] J. Yang, R. Karimi, T. Sæmundsson, A. Wildani, and Y. Vigfusson,“Mithril: mining sporadic associations for cache prefetching,” inProceedings of the 2017 Symposium on Cloud Computing. ACM,2017, pp. 66–79.

[22] J. Liao, F. Trahay, B. Gerofi, and Y. Ishikawa, “Prefetching onstorage servers through mining access patterns on blocks,” IEEETransactions on Parallel and Distributed Systems, vol. 27, no. 9, pp.2698–2710, 2015.

[23] J. Liao, F. Trahay, G. Xiao, L. Li, and Y. Ishikawa, “Performinginitiative data prefetching in distributed file systems for cloudcomputing,” IEEE Transactions on Cloud Computing, vol. 5, no. 3,pp. 550–562, 2015.

[24] R. A. Hankins and J. M. Patel, “Data morphing: an adaptive, cache-conscious storage technique,” in Proceedings of the 29th internationalconference on Very large data bases-Volume 29. VLDB Endowment,2003, pp. 417–428.

[25] A. Marascu and F. Masseglia, “Mining sequential patterns fromtemporal streaming data,” in Proceedings of the first ECML/PKDDWorkshop on Mining Spatio-Temporal Data (MSTD05), held in con-junction with the 9th European Conference on Principles and Practice ofKnowledge Discovery in Databases (PKDD05, 2005.

[26] R. Mishra and A. Choubey, “Discovery of frequent patterns fromweb log data by using fp-growth algorithm for web usage min-ing,” International Journal of Advanced Research in Computer Scienceand Software Engineering, vol. 2, no. 9, 2012.

[27] V. Jalaparti, C. Douglas, M. Ghosh, A. Agrawal, A. Floratou,S. Kandula, I. Menache, J. S. Naor, and S. Rao, “Netco: Cacheand i/o management for analytics over disaggregated stores,” inProceedings of the ACM Symposium on Cloud Computing. ACM,2018, pp. 186–198.

[28] P. K. Patra, M. Sahu, S. Mohapatra, and R. K. Samantray, “Fileaccess prediction using neural networks,” IEEE transactions onneural networks, vol. 21, no. 6, pp. 869–882, 2010.

[29] B. S. Gill and L. A. D. Bathen, “Amp: Adaptive multi-streamprefetching in a shared cache.” in FAST, vol. 7, no. 5, 2007, pp.185–198.

[30] S. Yang, K. Srinivasan, K. Udayashankar, S. Krishnan, J. Feng,Y. Zhang, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau,“Tombolo: Performance enhancements for cloud storage gate-ways,” in 2016 32nd Symposium on Mass Storage Systems andTechnologies (MSST). IEEE, 2016, pp. 1–14.

[31] D. Banga and S. Cheepurisetti, “Proxy driven fp growth basedprefetching,” International Journal of Advances in Engineering &Technology, vol. 7, no. 3, p. 856, 2014.

[32] J. Griffioen and R. Appleton, “Reducing file system latency usinga predictive approach,” in Proceedings of the USENIX Summer 1994Technical Conference on USENIX Summer 1994 Technical Conference-Volume 1. USENIX Association, 1994, pp. 13–13.

[33] R. Koller and R. Rangaswami, “I/o deduplication: Utilizing con-tent similarity to improve i/o performance,” ACM Transactions onStorage (TOS), vol. 6, no. 3, p. 13, 2010.

[34] F. Wu, B. Zhang, Z. Cao, H. Wen, B. Li, J. Diehl, G. Wang, and D. H.Du, “Data management design for interlaced magnetic recording,”in 10th {USENIX}Workshop on Hot Topics in Storage and File Systems(HotStorage 18), 2018.

[35] H. Lei and D. Duchamp, “An analytical approach to file prefetch-ing,” in USENIX Annual Technical Conference, 1997, pp. 275–288.

[36] M. Li, E. Varki, S. Bhatia, and A. Merchant, “Tap: Table-basedprefetching for storage caches.” in FAST, vol. 8, 2008, pp. 1–16.

[37] X. Ding, S. Jiang, F. Chen, K. Davis, and X. Zhang, “Diskseen:Exploiting disk layout and access history to enhance i/o prefetch.”in USENIX Annual Technical Conference, vol. 7, 2007, pp. 261–274.

Dongjie Zhu received the PhD degree in com-puter architecture from the Harbin Institute ofTechnology, in 2015. He is a assistant professorin School of Computer Science and Technologyat Harbin Institute of Technology, Wehai. Hisresearch interests include parallel storage sys-tems, social computing.

Haiwen Du is currently working toward the PhDdegree in School of Astronautics at Harbin In-stitute of Technology. His research interests in-clude storage system architecture and massivedata management.

Yundong Sun is currently working toward theMaster degree in School of Computer Scienceand Technology at Harbin Institute of Technol-ogy. His research interests include social com-puting and network embedding.

Zhaoshuo Tian is a professor in School of As-tronautics at Harbin Institute of Technology. Hisresearch interests include laser technology andmarine laser detection technology. He is a mem-ber of IEEE.

ieee transactions on cloud computing 1 ctdgm: a data

Documents