ieee transactions on services computing, vol. 11, no. 3, …mchen/min_paper/2018/2018-ieee... ·...

13
Opportunistic Task Scheduling over Co-Located Clouds in Mobile Environment Min Chen , Senior Member, IEEE, Yixue Hao, Student Member, IEEE, Chin-Feng Lai, Senior Member, IEEE, Di Wu , Member, IEEE, Yong Li , Senior Member, IEEE, and Kai Hwang, Fellow, IEEE Abstract—With the growing popularity of mobile devices, a new type of peer-to-peer communication mode for mobile cloud computing has been introduced. By applying a variety of short-range wireless communication technologies to establish connections with nearby mobile devices, we can construct a mobile cloudlet in which each mobile device can either works as a computing service provider or a service requester. Although the paradigm of mobile cloudlet is cost-efficient in handling computation-intensive tasks, the understanding of its corresponding service mode from a theoretic perspective is still in its infancy. In this paper, we first propose a new mobile cloudlet- assisted service mode named Opportunistic task Scheduling over Co-located Clouds (OSCC), which achieves flexible cost-delay tradeoffs between conventional remote cloud service mode and mobile cloudlets service mode. Then, we perform detailed analytic studies for OSCC mode, and solve the energy minimization problem by compromising among remote cloud mode, mobile cloudlets mode and OSCC mode. We also conduct extensive simulations to verify the effectiveness of the proposed OSCC mode, and analyze its applicability. Moreover, experimental results show that when the ratio of data size after task execution over original data size associated with the task is smaller than 1 (i.e., r< 1) and the average meeting rate of two mobile devices ! is larger than 0:00014, our proposed OSCC mode outperforms existing service modes. Index Terms—Task schedule, mobile cloud computing, mobile cloudlets, allocation optimization Ç 1 INTRODUCTION N OWADAYS, due to the explosive increase of mobile devi- ces and data traffic, various innovative technologies have been developed to transfer data more efficiently by the use of large quantities of mobile devices connected with each other. However, as mobile devices have limitations in terms of computing power, memory, storage, communications and battery capacity, the computation-intensive tasks are hard to be handled locally. Fortunately, the paradigm of mobile cloud computing (MCC) enables mobile devices to obtain extra resources for computing, storage and service supply, and may overcome above limitations [1]. Typically, compu- tation-intensive tasks can be uploaded to the remote cloud [2] through cellular network or WiFi. Though WiFi is energy efficient with high data rate, its connections are intermittent in mobile environments. In contrast to WiFi, cellular network provides stable and ubiquitous connections with high cost. In recent years, as a direct short-range communication mode between devices in the same district, device-to-device communication (D2D) has been well studied in terms of its techniques, application cases, and business models [3], [4], [5]. With the enormous increase of mobile devices with high memory and computing power, a new type peer-to-peer communication mode for MCC, called ad hoc cloudlet or mobile cloudlets, has been introduced [6]. In a mobile cloud- let, a mobile device can be either a service node or a comput- ing service requester (referred to as task node). When the connection of D2D is available in the mobile cloudlets, task node can offload the computing task to the cloudlet. The use of mobile cloudlets leads to low communication costs and short transmission delay, however, the intermittent D2D connections may quickly become invalid due to network dynamics. In mobile environment, remote cloud and mobile cloud- lets both have advantages and disadvantages for task off- loading. The keypoint of our work in this paper is to find the compromised service mode by the use of remote cloud and mobile cloudlets to minimize the cost and still ensure good-enough quality of experience (QoE) [7]. As shown in Table 1, remote cloud based service mode has shortcoming of high cost, while the efficiency of mobile cloudlets ori- ented service mode is closely related with user’s mobility. To solve the problem, the paper proposes a new task off- loading mode named “Opportunistic task Scheduling over Co-located Clouds” (OSCC) which divides into three cate- gories including OSCC (back&forth), OSCC (one way-WiFi) and OSCC (one way-Cellular Network). The OSCC mode outperforms remote cloud mode and mobile cloudlets mode due to a better tradeoff between cost and mobility support. Thus, the main contributions of the paper include: M. Chen and Y. Hao are with the School of Computer Science and Technol- ogy, Huazhong University of Science and Technology, Wuhan 430074, China. E-mail: [email protected], [email protected]. C. Lai is with the Department of Engineering Science, National Cheng Kung University, Tainan 701, Taiwan. E-mail: [email protected]. D. Wu is with the Department of Computer Science, School of Data and Computer Science, Sun Yat-sen University, Guangzhou 510006, China. E-mail: [email protected]. Y. Li is with the Tsinghua National Laboratory for Information Science and Technology, Department of Electronic Engineering, Tsinghua Univer- sity, Beijing 100084, China. E-mail: [email protected]. K. Hwang is with the Department of Electrical Engineering and Computer Science, University of Southern California, Los Angeles, CA 90089. E-mail: [email protected]. Manuscript received 19 June 2015; revised 13 June 2016; accepted 28 June 2016. Date of publication 7 July 2016; date of current version 8 June 2018. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference the Digital Object Identifier below. Digital Object Identifier no. 10.1109/TSC.2016.2589247 IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018 549 1939-1374 ß 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See ht_tp://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Upload: others

Post on 19-Sep-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

Opportunistic Task Scheduling over Co-LocatedClouds in Mobile Environment

Min Chen , Senior Member, IEEE, Yixue Hao, Student Member, IEEE, Chin-Feng Lai, Senior Member, IEEE,

Di Wu ,Member, IEEE, Yong Li , Senior Member, IEEE, and Kai Hwang, Fellow, IEEE

Abstract—With the growing popularity of mobile devices, a new type of peer-to-peer communication mode for mobile cloud computing

has been introduced. By applying a variety of short-range wireless communication technologies to establish connections with nearby

mobile devices, we can construct a mobile cloudlet in which each mobile device can either works as a computing service provider or a

service requester. Although the paradigm of mobile cloudlet is cost-efficient in handling computation-intensive tasks, the understanding

of its corresponding service mode from a theoretic perspective is still in its infancy. In this paper, we first propose a newmobile cloudlet-

assisted service mode named Opportunistic task Scheduling over Co-located Clouds (OSCC), which achieves flexible cost-delay

tradeoffs between conventional remote cloud service mode and mobile cloudlets service mode. Then, we perform detailed analytic

studies for OSCC mode, and solve the energy minimization problem by compromising among remote cloud mode, mobile cloudlets

mode and OSCC mode. We also conduct extensive simulations to verify the effectiveness of the proposed OSCC mode, and analyze

its applicability. Moreover, experimental results show that when the ratio of data size after task execution over original data size

associated with the task is smaller than 1 (i.e., r < 1) and the average meeting rate of two mobile devices � is larger than 0:00014, our

proposed OSCC mode outperforms existing service modes.

Index Terms—Task schedule, mobile cloud computing, mobile cloudlets, allocation optimization

Ç

1 INTRODUCTION

NOWADAYS, due to the explosive increase of mobile devi-ces and data traffic, various innovative technologies

have been developed to transfer data more efficiently by theuse of large quantities of mobile devices connectedwith eachother. However, as mobile devices have limitations in termsof computing power, memory, storage, communications andbattery capacity, the computation-intensive tasks are hard tobe handled locally. Fortunately, the paradigm of mobilecloud computing (MCC) enables mobile devices to obtainextra resources for computing, storage and service supply,and may overcome above limitations [1]. Typically, compu-tation-intensive tasks can be uploaded to the remote cloud [2]through cellular network or WiFi. Though WiFi is energyefficient with high data rate, its connections are intermittentinmobile environments. In contrast toWiFi, cellular networkprovides stable and ubiquitous connections with high cost.

In recent years, as a direct short-range communicationmode between devices in the same district, device-to-devicecommunication (D2D) has been well studied in terms of itstechniques, application cases, and businessmodels [3], [4], [5].With the enormous increase of mobile devices with highmemory and computing power, a new type peer-to-peercommunication mode for MCC, called ad hoc cloudlet ormobile cloudlets, has been introduced [6]. In a mobile cloud-let, a mobile device can be either a service node or a comput-ing service requester (referred to as task node). When theconnection of D2D is available in the mobile cloudlets, tasknode can offload the computing task to the cloudlet. The useof mobile cloudlets leads to low communication costs andshort transmission delay, however, the intermittent D2Dconnections may quickly become invalid due to networkdynamics.

In mobile environment, remote cloud and mobile cloud-lets both have advantages and disadvantages for task off-loading. The keypoint of our work in this paper is to findthe compromised service mode by the use of remote cloudand mobile cloudlets to minimize the cost and still ensuregood-enough quality of experience (QoE) [7]. As shown inTable 1, remote cloud based service mode has shortcomingof high cost, while the efficiency of mobile cloudlets ori-ented service mode is closely related with user’s mobility.To solve the problem, the paper proposes a new task off-loading mode named “Opportunistic task Scheduling overCo-located Clouds” (OSCC) which divides into three cate-gories including OSCC (back&forth), OSCC (one way-WiFi)and OSCC (one way-Cellular Network). The OSCC modeoutperforms remote cloud mode and mobile cloudletsmode due to a better tradeoff between cost and mobilitysupport. Thus, the main contributions of the paper include:

� M. Chen and Y. Hao are with the School of Computer Science and Technol-ogy, Huazhong University of Science and Technology, Wuhan 430074,China. E-mail: [email protected], [email protected].

� C. Lai is with the Department of Engineering Science, National ChengKung University, Tainan 701, Taiwan. E-mail: [email protected].

� D. Wu is with the Department of Computer Science, School of Data andComputer Science, Sun Yat-sen University, Guangzhou 510006, China.E-mail: [email protected].

� Y. Li is with the Tsinghua National Laboratory for Information Scienceand Technology, Department of Electronic Engineering, Tsinghua Univer-sity, Beijing 100084, China. E-mail: [email protected].

� K. Hwang is with the Department of Electrical Engineering and ComputerScience, University of Southern California, Los Angeles, CA 90089.E-mail: [email protected].

Manuscript received 19 June 2015; revised 13 June 2016; accepted 28 June2016. Date of publication 7 July 2016; date of current version 8 June 2018.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TSC.2016.2589247

IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018 549

1939-1374� 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See ht _tp://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Page 2: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

� We propose a new OSCC mode for task offloading tosupport high user mobility while saving the commu-nication cost as much as possible.

� We analyze the performance of the OSCC modeextensively in terms of task duration and energy costunder different application scenarios. In this paper,the energy cost mainly includes communication costand processing cost. The communication cost is con-sumed for task offloading, computation result feed-back. The communications can be achieved by eitherD2D link or cellular network. The processing costconsists of processing energy cost in either clouds orlocal nodes. The optimal solution of task allocation isgiven to achieve minimum energy cost. Basically, it’smore energy efficient to allocate more workloads tothe service nodes with higher mobility and largercomputing capacities.

� We identify the design spectrum based on the math-ematical modes. The OSCC mode achieves thetradeoff between remote cloud mode and mobilecloudlets mode. Furthermore, we introduce two dif-ferent kinds of task allocation schemes, i.e., dynamicallocation and static allocation. Under both mobilecloudlets mode and OSCC mode, dynamic alloca-tion exhibits lower cost than static allocation.

� We provide some insights based on the performanceevaluation, and the following question is answered:given a computation task, what is the optimal taskpartitioning strategy, i.e., how many sub-tasksshould be divided to optimize the integrated perfor-mance in terms of task duration and energy cost.

The rest of the paper is organized as follows. In Section 2,we describe the related work. We introduce the Oppor-tunistic task Scheduling over Co-located Clouds mode inSection 3, and then detail the mode in Section 4. We analyzeand optimize the mode in Section 5. Numerical results areshown in Section 6, followed by the conclusion and futurework in Section 7.

2 RELATED WORK

In this section, we survey the existing methods for task off-loading, which are classified into two categories: 1) basedon the remote cloud, 2) with the help of mobile cloudlets.

2.1 Remote Cloud

Along with the development of MCC, mobile users canupload their computing tasks [8], [9] to the cloud and thecloud will return the result to them after the completion of

the computing tasks [10], [11]. This is traditional task off-loading mode as shown in Fig. 1. Mobile devices can offloadcomputing related tasks to the cloud in two ways. One isthrough WiFi for cost saving as shown in Fig. 1a while theother is through expensive cellular network (e.g., 3G/4G/5G) as shown in Fig. 1b in case WiFi is unavailable. There-fore, a major question is: under what situation should themobile users offload the computing tasks to the cloud [12]?Previous work introduced various offloading strategies.Clonecloud [13] has proposed cloud-augmented executionby using cloned virtual image as a powerful virtual unit.Kosta et al. [14] has proposed a dynamic resource allocationand the framework of parallel execution named ThinkAir.As for the parallel task [15] allocation on the mobile devices,Li et al. [16] designed a kind of heuristic offloading scheme.Different from the existed research work, Lei et al. [17] firstconsiders the interactions between the offloading decisionfunction of MCC and the radio resource management func-tion of wireless heterogeneous network (HetNet), and theoffloading decision is made considering both the offloadinggain and the cost of using the HetNet when a Service LevelAgreement (SLA) is established with it. Under this frame-work, the mobile users may enjoy the cloud services withgood QoE regardless of spectrum scarcity. However, theseresearches mainly takes what, when and how to offload thetask from the mobile to cloud. In Flores et al. [18], the mainconsideration is how to offload the tasks to the cloud in realsituation. Provided with stable support from the cellularnetwork, smartphone can offload the computing task to the

TABLE 1A Comparison of Service Modes for Task Offloading

Structure Communication Style Cost Scalability MobilitySupport

Freedom ofService Node

ComputationDuration

Remote Cloud Cellular Network High Coarse High N/A MediumWiFi Low Coarse Low N/A Medium

Mobile Cloudlets D2D Low Coarse Low Low LowCo-Located Clouds D2D Low Medium Medium Medium High

D2D and Cellular Network Medium Fine High High HighD2D and WiFi Low Fine High High High

Fig. 1. Illustration of task offloading through remote cloud service mode:(a) remote cloud service mode via WiFi; (b) remote cloud service modevia cellular networks.

550 IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018

Page 3: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

remote cloud at any time and any places. The advantage ofthis mode is high reliability in the service supply while thedisadvantage is the high cost and delay of the cellularnetwork [19].

2.2 Mobile Cloudlets

The concept of cloudlet was presented in Satyanarayananet al. [20] and then discussed in Miettinen et al. [21]. Thesecloudlets are described as “data center in a box”. Nowa-days, discussions on cloudlets focus on the definition of thecloudlet size, lifetime of cloudlets node life and availabletime to solve a basic problem of the cloudlets: under whatcondition is it feasible for the mobile cloudlets to providemobile application service [6]. Moreover, Wang et al. [22]has proposed a kind of opportunistic cloudlet offloadingmechanism based on mobile cloudlets and Truong-Huuet al. [23] has proposed a kind of stochastic workload distri-bution approach based on mobile cloudlets. However, onlywhen task node is connected with service node, can the taskoffloading be allowed. After the D2D connection is built upbetween the task node and service node, the energy cost iseconomic since the content delivery are carried out throughthe local wireless network (i.e., WiFi and bluetooth). Zhouet al. [24], [25] first propose a distributed information-shar-ing strategy with low complexity and high efficiency. Thelimitation of mobile cloudlets lies in the strict requirementon time because the task node and service node shall haveenough time to offload the computing tasks and treat thefeedback. Once the task node and service node disconnectdue to high user mobility and other factors of networkdynamics while task offloading is not completed, the com-puting will fail.

3 OPPORTUNISTIC TASK SCHEDULING OVER

CO-LOCATED CLOUDS MODE

3.1 Motivation

Along with the rapid development of wireless communi-cation and sensor technology, the mobile devices areequipped with more and more sensors, as well as power-ful computing and perception abilities. Under such back-ground, the crowdsourcing application emerges as a newtype of mobile computing: a large number of users utilizemobile devices as basic sensing units to achieve distrib-uted data, collection and utilization of the perceptiontasks and data through mobile Internet to complete evenlarger and more complicated social perception tasks.The participants who complete the complicated percep-tion tasks with crowdsourcing do not need professionalskills. The crowdsourcing has succeeded in the applica-tions of positioning, navigation, urban traffic perception,market forecasting, opinion mining, etc. which are labor-intensive and time-consuming. Based on the vast quantityof common users, it distributes tasks in a free and volun-tary manner to common users and let them completethe tasks that they can never complete independently.The idea of crowdsourcing also has broad applicationsfor task offloading [18], [26].

In this paper, the task offloading is realized by remotecloud and mobile cloudlets. We consider that either tradi-tional remote cloud or mobile cloudlets exhibits a certain

limitation during task offloading, especially under limitedbandwidth. Given the application of image segmentation asa typical scenario (see Section 4.1 for details), the size ofthe picture taken by a mobile device is generally large.However, the user only care some specific region of interest(ROI). For example, the interest of some users towards awhole picture is only the face image appearing in the pic-ture. Compared to the size of the picture, the size of suchROI is much smaller. In order to achieving energy saving,it’s beneficial to finish the task of image segmentationlocally. However, the transmission of the whole picture tothe cloud is a must using remote cloud service mode. Incomparison, the energy cost for offloading the task to thecloud through the cellular network can be eliminated ineither mobile cloudlets service mode or OSCC mode. How-ever, the use of mobile cloudlets service mode incurs thelimit on user’s mobility. Thus, how to design an optimalsolution to minimizing energy cost while guaranteeing highuser’s QoE is a challenging issue.

3.2 OSCC Mode

In mobile cloudlets, user mobility or network dynamicsmake contacting time of two users short, which decreases theprobability of task completion. However, we assume that thecontacting time via D2D link is enough for a task node totransmit content associated with computation to a servicenode. When the task node and the service node disconnect,the computing of the service node will still carry on until thesub-tasks complete. We call the new service mode as OSCC.A basic feature of OSCC is that the contact between the tasknode and the service node can be either short or long insteadof limiting users’ mobility to guarantee the contact time fortask completion in conventional cloudlet based servicemode. We assume each computing task has a deadline,before which the computing result should be returned fromthe service node to the task node. Based on the location of theservice node upon the sub-task completion, there are threesituations: i) move close to the task node again within D2Dcommunication range, ii) cannot to connect with the tasknode directly by D2D communications, but WiFi can stillwork, iii) in no way to connect the task node by neither D2Dlinks nor WiFi, but the cellular network can still work. Con-sidering three situations above, the OSCC service mode wasclassified into the following three categories.

� OSCC (back&forth): Wang et al. [22] have proposed atask offloading method with the help of cloudlet,used the statistical law of the nodemovement and cal-culated the probability of themeeting of the task nodeand service node for twice at least. In that way, beforethe completion of the required computing tasks, oncethe service node meets the task node again and thesub-tasks of the service node have finished, the resultof the sub-tasks can be transmitted to the task nodesuccessfully. We call the task offloading service modeby mobile cloudlets as “back-and-forth service incloudlet”. However, in this mode, user mobility isalways limited to ensure the secondmeeting betweenthe task node and the service node. Even though,the mobility support of OSCC (back&forth) modeis higher than remote cloud mode through WiFi.

CHEN ET AL.: OPPORTUNISTIC TASK SCHEDULING OVER CO-LOCATED CLOUDS IN MOBILE ENVIRONMENT 551

Page 4: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

Therefore, we mark the mobility support level ofOSCC (back&forth) as themobility support should beleveled as “Medium” in Table 1.

� OSCC (one way-WiFi): Considering that the servicenode may move another cell, where WiFi is avail-able, for example, the owner of the service nodecomes back to his/her home, the sub-task result canbe uploaded to the cloud through WiFi. Generally,the data size of sub-task result (Sresult

sub�tk) is smallerthan the original size of the data associated with the

sub-task (Srecvsub�tk). Let r denote the rate of S

resultsub�tk over

Srecvsub�tk. With the decrease of r, OSCC (one way-WiFi)

outperforms remote cloud service mode more.� OSCC (one way-Cellular Network): In thismode, the eco-

nomic way for computing task result feedback is notavailable. That is, the service node moves to a placewithout WiFi, it needs to upload the sub-task result tothe cloud through cellular network. As for the r value,the small the r is, the better the effect of OSCCmode is.

A typical example is presented to explain the above men-tioned three kinds of OSCC service modes, as shown inFig. 2. David has a computation-intensive task which cannotbe carried out only by his mobile phone. Within his D2Dcommunication range, the phones of David’s three friends,Smith, Alex and Bob are all in idle state. So, David dividesthe computing task into three sub-tasks and transmits themto the three phones via D2D links. Smith is good friend ofDavid and moves together with David, so he always keepscontact with David. After the completion of the task, theresult of his sub-task computation will be transmitted toDavid directly through D2D connection. We assume Alexgoes back to home with available WiFi link, so OSCC (oneway-WiFi) service mode is used. Let’s assume that Bob hasmoved to another cell before the completion of the sub-task,so he uploads the sub-task result to the task node throughOSCC (one way-Cellular Network) service mode.

OSCC is quite efficient in some applications, for example,the data size associated with computing task is huge but theresult data is relatively small. We consider the example ofimage segmentation mentioned in Section 3.1. Comparedwith remote cloud service mode, OSCC mode can transmitthe whole picture through D2D which needs less bandwidthand energy. Compared with mobile cloudlets service mode,OSCC mode features a higher expand ability, as it does notrequire the task node keep contact with service nodesthrough D2D communications all the time or within aregion, so it provides high freedom for the task node andservice node. Therefore, OSCC mode which can be taken asthe compromised mode between remote cloud and mobilecloudlets, achieving more flexibility and cost-effectiveness.It is known to us that the paper first proposes the OSCCmode. In order to understand how to use this new task off-loading mode better, we establish a mathematical modeland provide solutions to some optimization problems. Asfor the OSCC mode, here we give the hypothesis as follows:

� The computing tasks can be divided into multiplesub-tasks.

� According to different applications and the propertiesof the task, we classify the division of the task into twocategories, i.e., cloned task and non-cloned task.

� Each service node will not accept the same clonedtask more than once.

� Packet loss is not considered during the transmissionof network data.

4 OSCC MODE

Assume that there are M mobile nodes in the mobile cloudcomputing network. Let N denote the total number of tasknodes and n denote the amount of sub-tasks for a task node.Task node can communicate with service node only whenthey are within the transmission radius R. That is, task node

Fig. 2. Illustration of the task offloading at OSCC mode: (a) Smith send sub-task result to David via D2D connection; (b) Bob send sub-task result tocloud via 3G; (c) Alex send sub-task result to cloud via WiFi.

552 IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018

Page 5: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

cni and service node snj can communicate if jjLiðtÞ�LjðtÞjj < R, Li and Lj are the positions of the two nodes attime t. The node mobility is i.i.d. Typically, the inter-contactduration of any two nodes follows exponential distributionwith parameter � [27], [28]. Thus, � reflects the averagemeeting rate of two nodes. The probability without contact

within Dt time can be calculated as Pft > Dtg ¼ e��Dt. Thetask node have a total amount of computation task Q whichcan be divided into n sub-tasks. Q can be denoted as

Q ¼Xni¼1

xi; (1)

where xi is the workload assigned to node i. Let’s assumethat service node sni have a per unit process speed ni andthis service node can process sub-task xi. Table 2 describesthe notations and default values used in this paper. In thefollowing section, task duration and energy cost of OSCCmode will be analyzed.

4.1 Task Duration

The task duration consists of time consumed by two proce-dures, i.e., 1) the delivery of task contents, and 2) task resultfeedback. For the sake of simplicity, we assume task nodeknows the capabilities of service nodes. Let t� denote thetask duration. Considering the situation where a task iscomputation-intensive and WiFi is unavailable, the delay oflocal processing (denoted by Q=v, where v is processingspeed of the task node) is typically larger than t�.

A task mainly consists of three components, i.e., process-ing code, data and parameter(s). For a non-cloned task,

a task node divides the task into a certain number of sub-tasks. Each sub-task includes a specific combination ofprocessing code, data and parameter(s). Typically, the datacontents and parameters between two sub-tasks are differ-ent while processing codes probably are identical. For acloned task, it can be copied during task dissemination. Ithas intrinsic feature of random parameter-oriented compu-tation. To illustrate the difference between a cloned taskand a non-cloned task, the characteristics of a cloned taskare detailed as follows:

1) When a service node receives a cloned task, it canfurther duplicate the cloned task and disseminate itto other service nodes. However, a service node willnot accept the same cloned task more than once.

2) A cloned task typically includes processing codewithout data and pre-assigned parameters. When aservice node receives the cloned task, it executes theprocessing code with a stochastic parameter gener-ated by the local machine (i.e., the service node).

3) The computation complexity of executing a process-ing code with various stochastic parameters for abunch of times is the major purpose for a task nodeto allocate cloned tasks to numerous service nodes.

4) Though there exists high redundancy among variouscloned tasks in terms of processing code, the compu-tation activities are different in those service nodeshandling the cloned tasks.

We further give examples about non-cloned task and clonedtask in detail.

TABLE 2Variables and Notation of OSCC Mode

Variable Default Value Explanation

M 500 number of nodes in the cellcni N/A a task node with index i and have computation task to be executedsnk N/A a service node with index k, which serves as available resource for computation offloadingN 45 the total number of task nodes in the celln 10 the amount of sub-tasks for a task nodeK 450 number of total sub-tasks in the cellXðtÞ N/A the number of service nodes at time tSi tð Þ N/A the function of number of sub-tasks assigned for a task node cni at time t� 0.0001 average meeting rate of two nodes in the cellrt;tþDtðiÞ 1/0 whether sni assigns sub-task successfully within Dt

uit;tþDtðkÞ 1/0 whether service node snk gets assignment of sub-task for cni

t� N/A the average time to complete the computation of a whole taskt�s N/A t* under computation clone modeQ 200 size of total computation taskxi N/A size of sub-ask the serve node sni haver 0.5 the ratio of Sresult

sub�tk and Srecvsub�tk

Ecelln!c

2 the per unit communication cost from task node to cloud via cellular network

Ecellc!n

2 the per unit communication cost from cloud to task node via cellular network

Ecloudproc

0.1 the per unit energy cost for computation tasks processed in cloud

ED2D 1 the per unit communication cost from task node to service nodeEnode

proc ðkÞ 0.2 the per unit energy cost for service node snk to process a sub-task locally

r 0.001 the probing cost per time unittd 4,000 deadline for computation task completion timeni N/A per unit process speed of service node sni

CCloud N/A the total energy cost for computation task executed in remote cloudCcloudlet N/A the total energy cost for computation task executed in CCS modeCOSCC N/A the total energy cost for computation task executed in OCS modev 0.5 a weight factor which indicates the emphasis

CHEN ET AL.: OPPORTUNISTIC TASK SCHEDULING OVER CO-LOCATED CLOUDS IN MOBILE ENVIRONMENT 553

Page 6: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

Given an example as shown in Fig. 3a about non-clonedtask, David is the task user (corresponding to task node)and has 20 pictures, which have different images andcontain unique ROI in each picture. There are Bob, Alex,Smith and Suri, four users who can reach David throughD2D communications. As each person has a different smartphone and specific computing power, the computing taskshall be divided into four sub-tasks through “dynamicallocation” which means the four assigned sub-tasks are dif-ferent from each other. For example, Bob and Alex are allo-cated six and seven images respectively while Smith andSuri are assigned three pictures and four pictures. When thepicture is segmented, the ROI (i.e., computation result) willbe sent to task node, which is owned by David. In thisexample, all of the benefits are obtained by David whileBob, Alex, Smith and Suri provide “free services”. Practi-cally, the intrinsic selfish feature of mobile users constitutesthe biggest obstacle for task offloading. For example, mostusers intend to assign tasks to other users while avoidingaccepting the sub-tasks allocated to them. This fact mayresult in failure of OSCC scenario, where most users like tocount on others to help them to execute the tasks whilereluctant to share computing capacity to others. In orderto solve the problem, an incentive mechanism can bedesigned. For example, Bob contributes computing capacityof his mobile phone to execute David’s sub-task. A certainamount of incentive is sent to him. Likewise, Alex, Suri andSmith get more or less rewards from David according totheir workload. Later, their incentives can be used to obtainfavors of speeding up their own computing tasks. However,the design of an incentive mechanism to encourage various

users for collaborations on task offloading is not the focus ofthis paper. We will address this issue in future work.

In the scenario shown in Fig. 3b about cloned task, thetask can be cloned for required times. For example, Davidhas a cloned task to be processed for 20 times while onlyBob and Alex are within his D2D communication scope.Thus, David assigns Bob and Alex to process the clonedtask for 9 times and 11 times, respectively. When Bobreceives the assignment, he handles the cloned task for 6times by himself while seeking the help from Smith to pro-cess the cloned task for the left 3 times. Likewise, Alex canreach Suri via D2D link, and allocates 4 times of cloned taskexecutions to Suri. In summary, the 20 times of cloned taskexecutions are allocated to Bob, Alex, Smith and Suri for 6,7, 3 and 4 times, respectively. In this example, when the ser-vice node receives a cloned task, the cloned task can be cop-ied and distributed to other service nodes, which is similarwith the epidemic model in online social network. Regard-ing the energy cost caused by the flooding of cloned task,task clone enables less energy cost since more D2D opportu-nities are available during cloned task distribution than thecase of non-cloned task offloading.

4.2 Energy Cost

The energy cost is mainly consists of communication costand processing cost. The communication cost includes twoparts, the first one is consumed for offloading task result tocloud (denoted by Ecell

n!c), the other part is for cloud to feed-

back computation result to task node (denoted by Ecellc!n).

ED2D denotes the energy cost via D2D link. The processing

cost includes processing energy cost in cloud Ecloudproc and in

node Enodeproc . Considering the heterogeneous capability of ser-

vice nodes in terms computing power, we give two methodsto distribute the nodes of sub-tasks.

� Static Allocation: As the task node usually has noknowledge about the processing capacity of the ser-vice node, we assume that the task node does not dif-ferentiate the computing capability of all the servicenodes, that is, they have the same processing speedvi and the same amount of workload xi ¼ Q=n. Thetask node will distribute the sub-tasks to the servicenodes evenly. However, the shortcoming of suchassumption is that the service node with largestdelay to submit computation result will cause theincrease of the task duration.

� Dynamic Allocation: Practically, in order to achievehigher delay performance, the task node should notignore the heterogenous capabilities of service nodes.This motivates us to propose “dynamic allocation”strategy. With the information of the computing capa-bility of various service nodes, we can distribute thesub-tasks in a more intellectual way. For example, ifthe service node has stronger computing power, it willreceive a sub-taskwith a larger workload.

5 ANALYSIS AND OPTIMIZATION FOR OSCC MODE

Different service modes have both advantages and disad-vantages and we hope to achieve flexible tradeoff amongvarious modes to decrease energy cost and delay while

Fig. 3. Illustration of task offloading in opportunistic co-located cloudsservice: (a) non-cloned task; (b) cloned task.

554 IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018

Page 7: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

meeting the requirements of user’s QoE. In the followingsection, the delay and energy performance will be analyzed,then the optimization framework will be given.

5.1 Analysis for Task Duration in OSCC Mode

5.1.1 Task Duration in OSCC Mode with Non-Cloned

Task

First, let’s analyze the task duration in the case that the taskscannot be cloned. The task duration consists of time con-sumptions from two main parts, i.e., sub-task distribution,and computation execution. Sub-task distribution phase isthe phase when the task node assigns sub-tasks to servicenodes, including transmitting the contents associated withsub-tasks to the service nodes. Typically, computation delayis much smaller than sub-task distribution delay. For thesake of simplicity, only sub-task distribution delay is con-sidered. Let Dt denote a very small time interval, withinwhich there is only one contact at most. As shown in Table 2,if rt;tþDtðiÞ is 1, it means that a task node mi successfullymeets a service node and assigns a sub-task within Dt, viceversa. Thus, rt;tþDtðiÞ can be defined as follows:

rt;tþDtðiÞ ¼ 1 mi assigns sub-task successfully within Dt;0 otherwise:

(2)

Since the inter-contact duration of any two nodes followsexponential distribution, the probability that mi assigns asub-task successfully can be expressed as follows:

Pfrt;tþDtðiÞ ¼ 1g ¼ 1� ðe��DtÞXðtÞ; (3)

where XðtÞ is the number of service node to the time t. Its

expectation can be calculated as: Eðrt;tþDtðiÞÞ¼1� ðe��DtÞXðtÞ,so the number of service nodes which have no sub-taskassignments can be computed as:

Xðtþ DtÞ ¼ XðtÞ �XNi¼1

rt;tþDtðiÞ: (4)

We can obtain the expectation about Equation (4):

EðXðtþ DtÞ ¼ EðXðtÞÞ �NEðrt;tþDtðiÞÞ: (5)

Letting Dt be close to 0, using the theory of limit, we canobtain the derivation of EðXðtÞÞ as follows:

E0ðXðtÞÞ ¼ limDt!0

EðXðtþ DtÞ � EðXðtÞÞDt

¼ �N�EðXðtÞÞ:(6)

By solving the ordinary differential equation (ODE) (6), wecan finally get the function EðXðtÞÞ as:

EðXðtÞÞ ¼ EðXð0ÞÞe�N�t: (7)

By solving the inverse function of Equation (7), we canobtain the average time of task duration (denoted by t�) asfollows:

t� ¼ln M�N

EðXðt�ÞÞN�

: (8)

Correspondingly, EðXðt�ÞÞ ¼ M �Nn. Aforementionedanalysis is for the case that all of the computations areconsidered.

5.1.2 Task Duration in OSCC Mode with Cloned Task

Now we analysis for the OSCC mode with cloned task. Atbeginning of our analysis, for the sake of simplicity, let us justconsider only one task node. Let SðtÞ denote the number ofservice nodes which have sub-tasks at time t. Let dt;tþDtðmkÞdenote whether mk gets sub-task assignment within Dt.We can obtain

EðSðtÞÞ ¼ Sð0ÞMeM�t

M � Sð0Þ � Sð0ÞeM�t; (9)

where Sð0Þ ¼ 1. Then, the task duration t can be calculatedas follows:

t ¼lnðSðtÞðM�Sð0ÞÞ

Sð0ÞðM�SðtÞÞÞM�

: (10)

Furthermore, let’s assume that there are N task node, andeach sub-task can be cloned. Let SiðtÞ denote the number ofservice nodes which have sub-task assignments of task nodemi to the time t, thenwe can calculate Siðtþ DtÞ as follows:

Siðtþ DtÞ ¼ SiðtÞ þXM�NSiðtÞ

k¼1

uit;tþDtðkÞ; (11)

where uit;tþDtðkÞ denoted whether service node mk getsassignment of sub-task formi. As for Equation (11), utilizingthe methods similar to Equations (5) and (6), we can obtain

E0ðSiðtÞÞ ¼ ðM �NEðSiðtÞÞÞ�EðSiðtÞÞ: (12)

Then, by solving ODE (12), we can compute EðSiðtÞÞ as

EðSiðtÞÞ ¼ e�MtM

M �N þ e�MtN: (13)

Finally, by solving the inverse function of Equation (13), weobtain the average time of the task duration for a single task(denoted by t�s) as follows:

t�s ¼ln ðEðSiðt�sÞÞðM�NÞ

M�NEðSiðt�sÞÞ ÞM�

: (14)

Correspondingly, EðSiðt�sÞÞ ¼ n.

5.2 Analysis for Energy Cost in Remote CloudMode, Mobile Cloudlets Mode and OSCC Mode

In this section, we analyze the energy cost performance forvarious service mode. Let’s consider a worst case where WiFiis not available. For simplicity, it is supposed that there is onlyone task node and its total computing quantity isQwhich canbe divided into n sub-tasks. Since static allocation can bedeemed as the extreme case of dynamic allocation, let’s focuson the case of dynamic allocation.Wedivide thewhole energycost chain into three phases, i.e., task associated contents

CHEN ET AL.: OPPORTUNISTIC TASK SCHEDULING OVER CO-LOCATED CLOUDS IN MOBILE ENVIRONMENT 555

Page 8: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

offloading, execution of sub-tasks, and computation resultfeedback. Then, the total cost of remote cloud based servicemode can be calculated as

Ccloud ¼Xni¼1

ðEcelln!cxi þEcloud

proc xi þ rEcellc!nxiÞ;

¼ QðEcelln!c þEcloud

proc þ rEcellc!nÞ:

(15)

In mobile cloudlets service mode, the major energy costcomes from the use of D2D communications and periodicaldetection of the surrounding nodes. Then, the total cost atmobile cloudlets mode can be calculated as

Ccloudlet ¼Xni¼1

ðED2Dxi þ Enodeproc ðiÞxi þ rED2DxiÞ þMrt�;

¼ Qð1þ rÞED2D þXni¼1

Enodeproc ðiÞxi þMrt�:

(16)

Let ~X ¼ fx1; x2; . . . ; xng denote the solution of task allo-cation, thus minimizing the cost can be specified as the fol-lowing optimization problem:

minimize~X

Ccloudlets

subject toXni¼1

xi ¼ Q

xi � 0 i ¼ 1; 2; . . . ; n:

(17)

The optimization problem is a linear programming problemand can be solved by using a conventional solver, i.e., Matlab.

Considering the mobility of a service node, it may moveto some region without WiFi. In this case, the computationresult feedback can be classified into two situations: (1) theservice node moves back to the proximity of task node andD2D connection is available. Under this situation, D2D linkcan be used to deliver the result of sub-tasks, which is thecase of OSCC (back&forth). (2) otherwise, the cellular net-work is the only choice to transmit the result of sub-tasks,which is the case of OSCC (one way-Cellular Network).

We first give the probability Pi, which denotes the chancethat service node sni meets task node twice. Let ti;1 denote thetime interval when task node meets service node sni forthe first time. Let ti;2 denote the time interval between the firstmeeting and the second meeting for task node and servicenode. Since the time interval follows exponential distributionand i.i.d., Let ti denote td � xi=ni. According to total probabil-ity theorem,

P ðti;1 þ ti;2 � tdÞ ¼Z ti

0

P ðti;1 þ ti;2 � tdjti;1 ¼ xÞ�e��xdx;

(18)

where P ðti;1 þ ti;2 � tdjti;1 ¼ xÞ ¼ P ðti;2 � td � xÞ ¼ 1� e��ðtd�xÞ.Therefore, the Pi ¼ P ðti;1 þ ti;2 � tdÞ can be calculated as

P ðti;1 þ ti;2 � tdÞ ¼Z ti

0

ð1� e��ðtd�xÞÞ�e��xdx;

¼ 1� e��ti � �tie��ti :

(19)

Then, the cost can be calculated as

COSCC ¼Xni¼1

ðxiED2D þEnodeproc ðiÞxi þ rxiPiED2D

þ rxið1� PiÞðEcelln!c þEcell

c!nÞÞ þMrt�:

(20)

Thus, the minimum cost can be computed as

minimize~X

COSCC

subject toXni¼1

xi ¼ Q

xi � 0 i ¼ 1; 2; . . . ; n:

(21)

The optimization problem is hard to solve, we divide thisproblem in two stages. First we maximize P , second weminimize the cost in OCS mode.

Algorithm 1. C Choosing Algorithm

beginnotationr denotes the ratio of Sresult

sub�tk and Srecvsub�tk;

� denotes average meeting rate ;C is remote cloud or mobile cloudlets or OSCC;initializationif situation have stable WiFi then

use remote cloud through WiFi;end ifif r > 1 and delay sensitive then

use remote cloud through cellular network;end ifif � is small and cost sensitive then

use mobile cloudlets;end ifif r < 1, � is large and maximum node freedom then

use OSCC;end if

Return C;

Generally, the cost for the service node to offload thecomputing task to the cloud or the cloud feedbacks theresult to the task node through cellular network is morethan the cost of D2D. Therefore, considering the energy costand delay in different situations, a compromising method isdesired according to the special applications.

� Remote Cloud: If the computing task is highly sensi-tive to delay and users can afford high cost to reacha higher QoE, using remote cloud (cellular network)is not a bad choice.

� Mobile Cloudlets: If the task node is very sensitive tothe communication cost and the service node movesin a small range, then the use of mobile cloudlets isrecommended.

� OSCC: If r is very small, and the service nodesrequire maximum node freedom, choosing OSCC isa best solution.

Now, we give the Algorithm 1 about how to chose remotecloud, mobile cloudlets and OSCC.

5.3 Optimization Framework

Now, we will give the joint optimization for time delay andenergy cost. Due to the different impact of time and energy

556 IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018

Page 9: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

cost, we introduce a weight factor, denoted as v, which indi-cates the emphasis on either time or energy cost. Thus mini-mizing the time and energy cost of a single task node can bespecified as the following problem:

minimize~X

t� þ v � COSCC

subject toXni¼1

xi ¼ Q

xi � 0 i ¼ 1; 2; . . . ; n:

(22)

We use genetic algorithms to solve above problem.A genetic algorithm is a heuristic algorithm based on theevolutionary theory of genetics and natural selection andcan solve this problem. The genetic algorithm is mainly touse the heuristic method to search for the optimal xi.

6 PERFORMANCE EVALUATION

In this section, the proposed OSCC mode will be evalu-ated. We set the meeting rate � is 0.00004 to 0.00032 persecond, M is within the range from 300 to 3,000, and r isset to be 0.001 per second by default based on the previouswork [29].

We considers two aspects for the experiment: (1) Whatkind of impact do task allocation strategies pose on taskduration and energy cost? According to the feature of atask, we classify it into non-cloned task and cloned task.The task allocation strategies can be static and dynamic allo-cation. (2) In order to evaluate our methods, we compareour methods with closely related work. In Chun et al. [13],remote cloud service mode is the major concern. In Liet al. [6], mobile cloudlets was introduced in detail.

6.1 Task Duration

6.1.1 The Time Consumed by Allocating All the

Sub-Tasks with Non-Cloned Task

Because of the relationship among XðtÞ, N , M, � and n,we need to evaluate the impact of each parameter on the

model. In Fig. 4a, we fix M to 500; let n be 10, and set �to 0.0001, while varying N with various values, includ-ing 30, 35, 40 and 45. As shown in Fig. 4a, task durationincreases when N becomes larger. Note that, here, thetask duration is the time when all of the users with taskachieve their goal of distributing all of the sub-tasks tothose mobile users who have no task assignments. FromFig. 4a, we also can observe that Xð0Þ is smaller than M.It is because N users already have tasks, thus Xð0Þ isequal to M �N .

In Fig. 4b, we fix N to 45; n to 10; and � to 0.00001, whilevarying M with 500, 750, 1,000, and 1,250, respectively. Asshown in Fig. 4b, when M ¼ 1; 250, the task completiontime is minimum among all of the scenarios compared. It isbecause there are more chances for task users to meet a ser-vice node to offload task to the node in a shorter period. Incomparison, when M ¼ 500, M �N service nodes are notenough for consequent task offloading process, and thuscausing a larger task duration.

In Fig. 4c, we fix M to 500; N to 45; n to 10, while varying� with 0.00004, 0.00008, 0.00016, 0.00032, respectively, inorder to obtain the impact of � on t and XðtÞ. As shown inFig. 4c, bigger � represents larger probability for a mobileuser to meet with a task node, facilitating the set of sub-tasks to be distributed faster. With the decrease of �, thetask duration increases.

In Fig. 4d, we fix M to 500; � to 0.0001; and K to 450,while varying n with 5, 7, 9, 11, so the N is K=n . As shownin Fig. 4d, task duration increases when n becomes larger. Itis because when n increases with a fix number of total sub-tasks N is decreased. This indicates that, under the fixed Mand �, the smaller n and the biggerN promote the task com-pletion time. From Fig. 4d, we also can observe that Xð0Þis not equal with each other. It is because when n changes,the N also changes. Similar with Fig. 4d, Xð0Þ is equal toM �N .

In Fig. 5a, we fix M to 500; let K to 450; while varying �with various values, including 0.00004, 0.00008, 0.00016 and0.00032. As shown in Fig. 5a, as total sub-tasks is fixed, taskcompletion time decreases when N becomes larger. How-ever when N reach 40, this benefit is not distinctive. We alsocan see that the benefit of increasing N is not significantwhen the � is high.

In Fig. 5b, we fix � to 0.0001; let K set to 450; while vary-ing M with various values, including 500, 750, 1,000 and1,250. As shown in Fig. 5b, like Fig. 5a, as total sub-tasks isfixed, task duration decreases when N becomes larger.From Fig. 5b, We also can see that the benefit of increasingN is not significant whenM reaches 1,000.

Fig. 4. Evaluation onXðtÞ. (a) The impact ofN onXðtÞ; (b) The impact ofM onXðtÞ; (c) The impact of � onXðtÞ; (d) The impact of n onXðtÞ.

Fig. 5. Evaluation on XðtÞ and t�. (a) t�-different � with varying N ;(b) t�-different M with varying N.

CHEN ET AL.: OPPORTUNISTIC TASK SCHEDULING OVER CO-LOCATED CLOUDS IN MOBILE ENVIRONMENT 557

Page 10: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

6.1.2 The Time Consumed by Allocating All the

Sub-Tasks with Cloned Task

In Fig. 6a, we fixM to 500; let n be 10, and set � to 0.0001,whilevaryingN with various values, including 30, 35, 40 and 45. Asshown in Fig. 6a, the impact ofN on SiðtÞ is not distinctive.

In Fig. 6b, we fix N to 45; n to 10; and � to 0.0001, whilevarying M with 500, 750, 1,000, and 1,250, respectively. Asshown in Fig. 6b, bigger M represents more chances for amobile user to meet with a task node. Thus, whenM is equalto 1,250, SiðtÞ increases fastest to reach its maximumof 10.

In Fig. 6c, we fix M to 500; N to 45; n to 10, while varying� with 0.00004, 0.00008, 0,00016, 0.00032, respectively. Asshown in Fig. 6c, bigger � represents larger probability for amobile user to meet with a task node. Thus, when � is equalto 0.00032, SiðtÞ increases fastest to reach its maximum of 10.

In Fig. 7a, we fix M to 500; let K be equal to 450; we vary� with different values, including 0.00004, 0.00008, 0.00016and 0.00032. In Fig. 7b, we fix � to 0.0001; let K be equal to450; We vary M with different values, including 500, 750,1,000 and 1,250.

As shown in Fig. 7, compared with Fig. 5, the task dura-tion t�s of Fig. 7 is far smaller than the task duration t� of Fig. 5under the condition of same N and � or same N andM. It isbecause task clone is allowed. When task node meet an ser-vice node, the service node becomes task node. In otherwords, the number of task node becomes larger. However,

if the task clone is not allowed, the number of task nodestay the same.

6.1.3 Task Duration in Mobile Cloudlet Mode and

OSCC Mode

Fig. 8a has compared the task duration of OSCC mode andmobile cloudlets. From the picture, in the situation that theOSCC can be cloned with a fixed �, the task duration isthe shortest and the task duration of OSCC is shorterthan mobile cloudlets. Along with the increase of �, OSCCpresents a better delay performance, because the increase of�, the task node meets the service node more frequently. Asfor mobile cloudlets, the task duration decreases graduallyfrom a small � (for example, from 0:00002 to 0:0001). How-ever, as � continues to grow, mobile cloudlets task durationstarts to increase because of shortened contacting timewhich leads to inadequate contacting time for the offload-ing, implementation and feedback of sub-tasks.

As shown in Fig. 8b, when � is larger than 0.0003, the per-formance of OSCC starts to not be distinctive. It is becausethe contact duration is too short to guarantee a successfulsub-task offloading.

6.2 Energy Cost in Remote Cloud Mode, MobileCloudlets Mode and OSCC Mode

Fig. 9a has compared the energy cost in the mode of remotecloud and OSCC. Four curves means the energy cost in themode of remote cloud and the energy costs in the mode ofOSCC with different r. As Ecell

n!c; Ecellc!n > ED2D, when r < 1,

OSCC is smaller than remote cloud under normal circum-stances. However, when r > 1, as r increases, The memoryconsumption in OSCCmode also increases and its increasingspeed is faster than remote cloud increasing speed. More-over, whenED2D increases, the cost of OSCC becomes large.

In Fig. 9b, the costs of mobile cloudlets and OSCC arecompared with each other. In these three methods, OSCChas appeared smaller energy cost than the other methodsunder the situation that the computing task can be cloned(i.e., cloned task) when the � value is fixed, because OSCCcan complete the sub-tasks more quickly when it can becloned. When 0:00002 � � � 0:00014, the cost of mobilecloudlets is less than OSCC when it cannot be cloned (i.e.,non-cloned task), because OSCC may needs to upload sub-task results to the cloud when the computing task cannot becloned but mobile cloudlets saves energy accordingly.When � increases, the contacting time gets shorter, possiblyleading to the failure of implementing sub-tasks with

Fig. 6. Evaluation on SiðtÞ. (a) The impact of N on SiðtÞ; (b) The impactofM on SiðtÞ; (c) The impact of � on SiðtÞ.

Fig. 7. Evaluation on SiðtÞ and t�s . (a) t�s-different � with varying N;(b) t�s-different M with varying N.

Fig. 8. Evaluation on task duration. (a) Compared the task completiontime of mobile cloudlets and OSCCmode; (b) OSCCmode task duration-different nwith varying �.

558 IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018

Page 11: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

mobile device. Therefore, OSCC is better than mobile cloud-lets in case of non-cloned task when � � 0:00014.

6.3 Optimization Framework

In this section, we consider the impact of static and dynamicallocation on the experimental results. The performance ofnon-cloned task and clone task is also evaluated. Genetic algo-rithm is used to solve the optimization problem in terms ofenergy cost and task duration. In our experiments, Theweightfactorv about task duration and energy cost is set to be 0.5.

In Fig. 10a shows the comparison of cost in term ofmobile cloudlets with static allocation and dynamic alloca-tion. As shown in the Fig. 10a, the dynamic allocation isalmost smaller than static allocation, this is because the tasknode knows each service node processing cost, so task nodesend large task to the service node which have lower proc-essing cost. when � < 0:00005 and � > 0:00017, the benefitof dynamic allocation is not significant.

In Fig. 10b shows the impact of Enodeproc on energy cost in

terms of static allocation and dynamic allocation. In order toverify the effect of dynamic allocation, random value isapplied. The circle represents the energy performance of

static allocation where Enodeproc is fixed to 0:1, which represents

the same processing capability of service nodes. while thevalues of data points at X-axis mean the value span wherepractical value is generated. For example, 0:2 in X-axismeans the practical value of Enode

proc is obtained between 0:01

and 0:19 in a random fashion; 0:01 in X-axis means the prac-tical value varies from 0:09 to 0:11. As shown in Fig. 10b, thelarger is the interval, the better performance of dynamicallocation can be obtained.

In Fig. 10c shows the comparison of cost in tern of OSCCmode with static allocation and dynamic allocation undernon-cloned task and cloned task. We can see that dynamicallocation with cloned task is the smallest energy cost. Withthe increase of �, energy cost of all of the compared schemesdecreased. In the scheme of dynamic with non-cloned task,the energy cost is decreased with fastest speed. It is becausethe value of � have more effect on non-cloned task thancloned task. When � reaches 0:00018, the impact of duplicat-ing task becomes smaller. It is because the meeting timesincrease in unit time slot, and thus speeding up the distribu-tion of sub-tasks.

In Fig. 10d shows the conjunctive minimization of taskduration and energy cost. we set v ¼ 0:5. In the embeddedfigure in Fig. 10d, with the increase of sub-task number nand when n < 35, the cost decreases. It is because theamount of sub-task allocated to service nodes becomes

smallerwhen total taskQ is fixed andmore sub-task commu-nication with D2D. However, since the number of sub-tasksis increase, it need more time to deliver the task content andthe periodically probing, so the task duration increase. Evenmore, when n > 35, the cost increase since the periodicallyprobing excessive cost due to the task duration. So thereexists a trade-off, we try to decrease time and energy cost byobtaining the solution to the optimal function. As shownfrom Fig. 10d, using genetic algorithms, when n is equal to26, the optimized performance is achieved.

7 CONCLUSION AND FUTURE WORK

7.1 Conclusion

With the explosive increase of mobile devices and data traf-fics, 5G network system needs to realize the resource utiliza-tion more efficiently through novel mobile networkarchitecture designs. The task offloading is an efficient solu-tion to cope with the growing mobile traffic and the associ-ated computation demand. In this paper, by the use ofremote cloud and mobile cloudlets, we propose a new taskoffloading mode, the Opportunistic task Scheduling overCo-located Clouds mode. In the design spectrum of OSCC, itcan be deems as a compromisedmode between remote cloudand mobile cloudlets to achieve high flexibility and betterperformance in terms of energy and delay. To the best of ourknowledge, this paper is the first to propose OSCC mode. Inorder to understand how to use this new task offloadingmode better, we establish amathematical model and providesolutions to some optimization problems.

7.2 Future Work—Workflow Scheduling

In this paper, we only consider that task consists of a bag ofsub-tasks, while there are no dependencies among those sub-tasks. In future work, we will investigate the task including aseries of interactive sub-tasks, generally expressed as directedacyclic graph GðV;EÞ. The vertices V expresses a series of

Fig. 9. Evaluation on energy cost. (a) Compared the cost betweenremote cloud and OSCC mode; (b) Compared the cost between mobilecloudlets and OSCC.

Fig. 10. Evaluation on the optimization framework. (a) Compared thecost between static allocation and dynamic allocation in mobile cloud-lets; (b) The impact of Enode

proc on energy cost in OSCC mode; (c) Com-pared the cost between static allocation and dynamic allocation inOSCC mode; (d) Conjunctive minimization of time and energy cost.

CHEN ET AL.: OPPORTUNISTIC TASK SCHEDULING OVER CO-LOCATED CLOUDS IN MOBILE ENVIRONMENT 559

Page 12: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

tasks and edges E expresses the interaction or dependency insub-task pairs. The distribution of tasks on themobiles ismen-tioned at Gao et al. [30] and a kind of energy-aware offloadingstrategy is proposed based on cloudlet. MuSIC [31] presentsan optimal service allocation mechanism for location andtime-sensitive tasks in cloud and cloudlet environments. Fortask scheduling in cloud, Chun et al. [13] has proposed a costadaptive virtual machine management technology whichrequires lower time and energy cost. As for the computation-intensive task, such as resource scheduling for multimediacontent driven, a kind of resource sensitivemoderate schedul-ing algorithm with higher performance for the clustering ofcloud resources and tasks at Vasile [32]. Ge et al. [33] pro-posed 5G wireless hackhaul networks to balance user taskand BSs task in a distributed network architecture. Moreover,it is the first paper that the task scheduling and energy effi-ciency optimization was derived by a user accessing Markovchain model for random cellular networks [34]. However, allabove existingwork do not consider the workflow schedulingin hybrid cloud and mobile cloudlets environments, so wewill address the issue in the future work regarding worflowscheduling and task allocation inmobile cloudlets and cloud.

ACKNOWLEDGMENTS

This work was supported by the National NaturalScience Foundation of China (grant No. 61272397, 61572538,61572220 and 61300224). Dr. Yixue Hao’s work was sup-ported by the Fundamental Research Funds for the CentralUniversities’, HUST: CX-15-055. Yixue Hao is the corre-sponding author of this paper.

REFERENCES

[1] V. Leung, T. Taleb, M. Chen, T. Magedanz, L.-C. Wang, andR. Tafazolli, “Unveiling 5G wireless networks: Emerging researchadvances, prospects, and challenges [Guest Editorial],” IEEENetw., vol. 28, no. 6, pp. 3–5, Nov./Dec. 2014.

[2] Y. Zhang, M. Chen, S. Mao, L. Hu, and V. Leung, “CAP: Crowdactivity prediction based on big data analysis,” IEEE Netw.,vol. 28, no. 4, pp. 52–57, Jul., 2014.

[3] L. Lei, Y. Zhang, X. Shen, C. Lin, and Z. Zhong, “Performanceanalysis of device-to-device communications with dynamic inter-ference using stochastic petri nets,” IEEE Trans. Wireless Commun.,vol. 12, no. 12, pp. 6121–6141, Dec. 2013.

[4] B. Han, P. Hui, V. A. Kumar, M. V. Marathe, J. Shao, andA. Srinivasan, “Mobile data offloading through opportunisticcommunications and social participation,” IEEE Trans. MobileComput., vol. 11, no. 5, pp. 821–834, May 2012.

[5] X. Wang, M. Chen, Z. Han, D. O. Wu, and T. T. Kwon, “TOSS:Traffic offloading by social network service-based opportunisticsharing in mobile social networks,” in Proc. IEEE Conf. Comput.Commun., 2014, pp. 2346–2354.

[6] Y. Li and W. Wang, “Can mobile cloudletss support mobileapplications?” in Proc. IEEE Conf. Comput. Commun., 2014,pp. 1060–1068.

[7] K. Zheng, X. Zhang, Q. Zheng, W. Xiang, and L. Hanzo, “Quality-of-experience assessment and its application to video services inLTE networks,” IEEE Wireless Commun., vol. 22, no. 1, pp. 70–78,Feb. 2015.

[8] D. Candeia, R. Araujo, R. Lopes, and F. Brasileiro, “Investigatingbusiness-driven cloudburst schedulers for E-science Bag-of-Tasksapplications,” in Proc. IEEE 2nd Int. Conf. Cloud Comput. Technol.Sci., 2010, pp. 343–350.

[9] W. Cirne, et al., “Running Bag-of-Tasks applications on computa-tional grids: The MyGrid approach,” in Proc. IEEE Int. Conf. Paral-lel Process., 2003, pp. 407–416.

[10] T. Taleb and A. Ksentini, “Follow me cloud: Interworking feder-ated clouds and distributed mobile networks,” IEEE Netw.,vol. 27, no. 5, pp. 12–19, Sep./Oct. 2013.

[11] H. Flores and S. Srirama, “Mobile code offloading: Should it be alocal decision or global inference?” in Proc. 11th Annu. Int. Conf.Mobile Syst. Appl. Serv., 2013, pp. 539–540.

[12] M. Barbera, S. Kosta, A. Mei, and J. Stefa, “To offload or not to off-load? the bandwidth and energy costs ofmobile cloud computing,”in Proc. IEEE Conf. Comput. Commun., 2013, pp. 1285–1293.

[13] B.-G. Chun, S. Ihm, P. Maniatis, M. Naik, and A. Patti,“CloneCloud: Elastic execution between mobile device andcloud,” in Proc. 6th Conf. Comput. Syst., 2011, pp. 301–314.

[14] S. Kosta, A. Aucinas, P. Hui, R. Mortier, and X. Zhang, “ThinkAir:Dynamic resource allocation and parallel execution in the cloudfor mobile code offloading,” in Proc. IEEE Conf. Comput. Commun.,2012, pp. 945–953.

[15] W. Cirne, et al., “Scheduling in Bag-of-Task grids: The PAUAcase,” in Proc. 16th IEEE Symp. Comput. Archit. High Perform. Com-put. 2004, pp. 124–131.

[16] M. Jia, J. Cao, and L. Yang, “Heuristic offloading of concurrent tasksfor computation-intensive applications inmobile cloud computing,”in Proc. IEEEConf. Comput. Commun.Workshops, 2014, pp. 352–357.

[17] L. Lei, Z. Zhong, K. Zheng, J. Chen, and H. Meng, “Challenges onwireless heterogeneous networks for mobile cloud computing,”IEEE Wireless Commun., vol. 20, no. 3, pp. 34–44, Jun. 2013.

[18] H. Flores, P. Hui, S. Tarkoma, Y. Li, S. Srirama, and R. Buyya,“Mobile code offloading: From concept to practice and beyond,”IEEE Commun. Mag., vol. 53, no. 3, pp. 80–88, Mar. 2015.

[19] M. Chen, Y. Hao, Y. Li, C.-F. Lai, and D. Wu, “On the computationoffloading at ad hoc cloudlet: Architecture and service modes,”IEEE Commun. Mag., vol. 53, no. 6, pp. 18–24, Jun. 2015.

[20] M. Satyanarayanan, P. Bahl, R. Caceres, and N. Davies, “The casefor VM-based cloudlets in mobile computing,” IEEE PervasiveComput., vol. 8, no. 4, pp. 14–23, Oct.–Dec. 2009.

[21] A. P. Miettinen and J. K. Nurminen, “Energy efficiency of mobileclients in cloud computing,” in Proc. 2nd USENIX Conf. Hot TopicsCloud Comput., 2010, pp. 4–4.

[22] C. Wang, Y. Li, and D. Jin, “Mobility-assisted opportunistic com-putation offloading,” IEEE Commun. Lett., vol. 18, no. 10,pp. 1779–1782, Oct. 2014.

[23] T. Truong-Huu, C.-K. Tham, and D. Niyato, “A stochastic work-load distribution approach for an ad hoc mobile cloud,” in Proc.IEEE 6th Int. Conf. Cloud Comput. Technol. Sci., 2014, pp. 174–181.

[24] L. Zhou, “Specific-versus diverse-computing in media cloud,”IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 12, pp. 1888–1899, Dec. 2015.

[25] L. Zhou, Z. Yang, H. Wang, and M. Guizani, “Impact of executiontime on adaptive wireless video scheduling,” IEEE J. Sel. AreasCommun., vol. 32, no. 4, pp. 760–772, Apr. 2014.

[26] Q. Li, P. Yang, Y. Yan, and Y. Tao, “Your friends are more power-ful than you: Efficient task offloading through social contacts,” inProc. IEEE Int. Conf. Commun., 2014, pp. 88–93.

[27] Y. Li, Y. Jiang, D. Jin, L. Su, L. Zeng, and D. Wu, “Energy-efficient optimal opportunistic forwarding for delay-tolerantnetworks,” IEEE Trans. Veh. Technol., vol. 59, no. 9, pp. 4500–4512, Nov. 2010.

[28] M. Chen, Y. Hao, M. Qiu, J. Song, D. Wu, and I. Humar, “Mobility-aware caching and computation offloading in 5G ultradensecellular networks,” Sensors, vol. 16, no. 7, pp. 974–987, 2016.

[29] X. Wang, M. Chen, Z. Han, T. T. Kwon, and Y. Choi, “Content dis-semination by pushing and sharing in mobile cellular networks:An analytical study,” in Proc. IEEE 9th Int. Conf. Mobile Ad-HocSensor Syst., 2012, pp. 353–361.

[30] B. Gao, L. He, L. Liu, K. Li, and S. A. Jarvis, “From mobiles toclouds: Developing energy-aware offloading strategies for work-flows,” in Proc. ACM/IEEE 13th Int. Conf. Grid Comput., 2012,pp. 139–146.

[31] M. R. Rahimi, N. Venkatasubramanian, and A. V. Vasilakos,“Music: Mobility-aware optimal service allocation in mobile cloudcomputing,” in Proc. IEEE 6th Int. Conf. Cloud Comput., 2013,pp. 75–82.

[32] M.-A. Vasile, F. Pop, R.-I. Tutueanu, V. Cristea, and J. Ko»odziej,“Resource-aware hybrid scheduling algorithm in heterogeneousdistributed computing,” Future Generation Comput. Syst., vol. 51,pp. 61–77, 2015.

[33] X. Ge, H. Cheng, M. Guizani, and T. Han, “5G wireless backhaulnetworks: challenges and research advances,” IEEE Netw., vol. 28,no. 6, pp. 6–11, Nov./Dec. 2014.

[34] X. Ge, B. Yang, J. Ye, G. Mao, C.-X. Wang, and T. Han, “Spatialspectrum and energy efficiency of random cellular networks,”IEEE Trans. Commun., vol. 63, no. 3, pp. 1019–1030, Mar. 2015.

560 IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, MAY/JUNE 2018

Page 13: IEEE TRANSACTIONS ON SERVICES COMPUTING, VOL. 11, NO. 3, …mchen/min_paper/2018/2018-IEEE... · 2019. 11. 16. · Along with the development of MCC, mobile users can upload their

Min Chen is a professor in the School of Com-puter Science and Technology, Huazhong Uni-versity of Science and Technology (HUST). He isthe director of Embedded and Pervasive Comput-ing (EPIC) lab. He was an assistant professor inthe School of Computer Science and Engineer-ing, Seoul National University (SNU) from Sep.2009 to Feb. 2012. He worked as a post-doctoralfellow in the Department of Electrical and Com-puter Engineering, University of British Columbia(UBC) for three years. Before joining UBC, he

was a post-doctoral fellow at SNU for one and half years. He has morethan 200 paper publications. His Google Scholars Citations reached7,200þ with an h-index of 42. His top paper was cited 820þ times.He received Best Paper Award from IEEE ICC 2012, QShine 2008,IndustrialIoT 2016, and IEEE IWCMC 2016. He is a senior member ofthe IEEE.

Yixue Hao received the BS degree from HenanUniversity, Kaifeng, China, in 2013. He is cur-rently working toward the PhD degree in Embed-ded and Pervasive Computing (EPIC) lab led byProf. Min Chen. His research interests includesInternet of Things, body sensor networks, andmobile cloud computing. He is a student memberof the IEEE.

Chin-Feng Lai received the PhD degree from theDepartment of Engineering Science, NationalCheng Kung University, Taiwan, in 2008. He isan associate professor in the Department of Engi-neering Science, National Cheng Kung Universitysince 2016. He received Best Paper Award fromIEEE 17th CCSE, 2014 International Conferenceon Cloud Computing, IEEE 10th EUC, IEEE 12thCIT. He has more than 100 paper publications.He is an associate editor-in-chief of the Journal ofInternet Technology. His research focuses on

Internet of Things, body sensor networks, E-healthcare, mobile cloudcomputing, cloud-assisted multimedia network, embedded systems, etc.He is a senior member of the IEEE since 2014.

Di Wu received the BS degree from the Universityof Science and Technology of China, in 2000,the MS degree from the Institute of ComputingTechnology, Chinese Academy of Sciences, in2003, and the PhD degree in computer scienceand engineering from Chinese University of HongKong, in 2007. He is a professor and the assistantdean of the School of Data and Computer Sciencewith Sun Yat-sen University, Guangzhou, China.During 2007-2009, he worked as a postdoctoralresearcher in the Department of Computer Sci-

ence and Engineering, Polytechnic Institute of NYU, advised by Prof.KeithW. Ross. He is the co-recipient of IEEE INFOCOM2009 Best PaperAward. He is a member of the IEEE.

Yong Li received the BS degree in electronicsand information engineering from HuazhongUniversity of Science and Technology, Wuhan,China, in 2007, and the PhD degree in electronicengineering from Tsinghua University, Beijing,China, in 2012. During July to August 2012 and2013, he was a visiting research associate withTelekom Innovation Laboratories and Hong KongUniversity of Science and Technology, respec-tively. During December 2013 to March 2014, hewas a visiting scientist with the University of

Miami. He is currently a faculty member of the Department of ElectronicEngineering, Tsinghua University. His research interests are in the areasof networking and communications. He is a senior member of the IEEE.

Kai Hwang received the PhD from the Universityof California, Berkeley, in 1972. He is a professorof electrical engineering and computer science,University of Southern California (USC). Prior tojoining USC in 1986, he has taught at PurdueUniversity for 11 years. He has served as thefounding editor-in-chief of the Journal of Paralleland Distributed Computing from 1983 to 2011.He has published 8 books and 250 scientificpapers. According to Google Scholars, his workwas cited more than 15,000 times with an h-index

of 52. His most cited book on Computer Architecture and Parallel Proc-essing was cited more than 2,300 times and his PowerTrust (IEEE-Transactions on Parallel and Distributed Systems, April 2007) paperwas cited over 540 times. He received Lifetime Achievement Awardfrom IEEE Cloudcom-2012 for his pioneering contributions in the field ofcomputer architecture, parallel, distributed and cloud computing, andcyber security. He is a fellow of the IEEE.

CHEN ET AL.: OPPORTUNISTIC TASK SCHEDULING OVER CO-LOCATED CLOUDS IN MOBILE ENVIRONMENT 561