a mobile offloading game against smart attacks

SPECIAL SECTION ON THEORETICAL FOUNDATIONS FOR BIG DATA APPLICATIONS:CHALLENGES AND OPPORTUNITIES

Received April 17, 2016, accepted April 21, 2016, date of publication May 9, 2016, date of current version June 3, 2016.

Digital Object Identifier 10.1109/ACCESS.2016.2565198

A Mobile Offloading Game Against Smart AttacksLIANG XIAO1,2, (Senior Member, IEEE), CAIXIA XIE1, TIANHUA CHEN1,HUAIYU DAI3, (Senior Member, IEEE), AND H. VINCENT POOR4, (Fellow, IEEE)1Department of Communication Engineering, Xiamen University, Xiamen 361005, China2Key Laboratory of Underwater Acoustic Communication and Marine Information Technology Ministry of Education, Xiamen University, Xiamen 361005, China3Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC 27695, USA4Department of Electrical Engineering, Princeton University, Princeton, NJ 08544, USA

Corresponding author: L. Xiao ([email protected])

This work was supported in part by the National Natural Science Foundation of China under Grant 61271242 and in part by theU.S. National Science Foundation under Grant CMMI-1435778, Grant CNS-1016260, Grant ECCS-1307949,and Grant EARS-1444009.

ABSTRACT Mobile devices, such as smartphones, can offload applications and data to the cloud viaaccess points or base stations to reduce energy consumption and improve user experience. However, mobileoffloading is vulnerable to smart attackers that use smart and programmable radio devices, such as universalsoftware radio peripherals, to perform multiple types of attacks, such as spoofing and jamming, based on theradio environment and offloading transmissions. In this paper, a mobile offloading game is investigated thatconsists of three players: a mobile device that chooses its offloading rate, a smart attacker that determinesits attack mode, and a security agent that decides whether or not to initiate full protection for the servingaccess point during the offloading. Nash and Stackelberg equilibria of the offloading game are derived andtheir existence conditions are discussed. A Q-learning-based mobile offloading strategy is proposed formobile devices that are unaware of system parameters, such as the channel conditions, in dynamic radioenvironments. Simulation results show that the proposed offloading strategy can improve the utility of themobile device and reduce the attack rate of smart attackers.

INDEX TERMS Mobile offloading, spoofing, jamming, game theory, smart attacks, Q-learning.

I. INTRODUCTIONWith the proliferation of cloud-based mobile services, mobiledevices such as smartphones and tablets can offload theirapplications and data to the cloud to improve user experi-ence in terms of longer battery lifetime, larger data storage,faster processing speed and more powerful security services.However, data offloading to the cloud via access points (APs)or base stations (BSs) is vulnerable to various types of attacks,such as spoofing, eavesdropping and jamming [1]. A smartattacker can use smart and programmable radio devices suchas Universal Software Radio Peripherals (USRPs) or theWireless Open-Access Research Platform (WARP) devel-oped by Rice University [2] to launch multiple types ofattacks. Compared with traditional single mode attacks,mobile offloading is more vulnerable to smart attacks, as asmart attacker flexibly chooses its attack mode and strengthaccording to the ongoing offloading transmissions and radioenvironment. For example, a smart attacker can launchjamming attacks if it is close to the serving AP and canefficiently block the offloading, or can send spoofing signalswith the MAC address of the mobile device.

Although the detection of spoofing and jamming attackshas been well studied [3], [4], mobile devices still suffer fromtime and energy loss due to false alarms and security lossresulting from missed detection of attacks. Security agents atAPs or BSs can apply both physical-layer and higher-layersecurity mechanisms [5] to protect the offloading process.More specifically, the security agent can apply an advancedsecurity mechanism, possibly by processing the offloadingdata again or changing the session keys, at the cost of higherprocessing and transmission overhead. The offloading rate ofthe mobile device is chosen according to the radio environ-ment, e.g., a mobile device has to process the data locallyunder strong jamming or heavy traffic at theAP or BS [6]–[9].

While game theory has been used to study the interac-tions between attackers and mobile devices [3], [4], [10],mobile offloading usually involves three players: a mobiledevice, a smart attacker and a security agent. In this paper,we investigate the secure mobile offloading game, in whicha smart attacker can perform multiple types of attacks,including spoofing and jamming, a mobile device deter-mines its offloading rate, and a security agent at the AP

VOLUME 4, 20162169-3536 2016 IEEE. Translations and content mining are permitted for academic research only.

Personal use is also permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

2281

www.redpel.com +917620593389


L. Xiao et al.: Mobile Offloading Game Against Smart Attacks

protects the offloading with two modes: a fast mode thatapplies physical-layer security and a safe mode that appliesboth physical-layer and higher-layer security mechanisms.We analyze the Nash equilibria (NEs) of the secure mobileoffloading game, at which none of the three players canincrease its utility by unilaterally choosing a different strat-egy. The sequential interactions among the mobile device,the security agent and the attacker can be modeled as aStackelberg game, in which the mobile device chooses itsoffloading rate first, the smart attacker determines its attackmode afterwards based on the observed offloading rate,and finally the security agent chooses its defense mode(i.e., whether or not to invoke a higher-layer security mecha-nism) based on both the offloading rate and the attack mode.We derive the Stackelberg equilibria (SEs) of this game anddiscuss the impact of the radio channel gains on the securityof offloading.

A mobile device has difficulty obtaining all the systemparameters, such as the channel condition and transmit pow-ers of the attacker in dynamic environments needed to derivethe optimal offloading strategy. As an important reinforce-ment learning technique, Q-learning can be applied by themobile device to derive the optimal offloading strategy via tri-als. Thus, we propose a Q-learning based offloading strategyfor mobile devices to decide their offloading rates based onthe observed state, which consists of the previous actions ofthe opponents and the channel gain in dynamic environments.

Our main contributions can be summarized as follows:• We formulate a secure mobile offloading game to inves-tigate the interactions among a mobile device, a smartattacker and a security agent.

• We derive both the NEs and SEs of the secure mobileoffloading game, and provide conditions under whichthe equilibria exist.

• We propose a Q-learning based offloading strategyfor dynamic radio environments to improve resistanceagainst smart attacks.

The rest of this paper is organized as follows. We reviewrelated work in Section II, and present the system modelin Section III. We formulate the secure mobile offloadinggame and derive the NEs of the game in Section IV, andinvestigate the Stackelberg game in Section V. We presentthe Q-learning based offloading strategy and investigate thedynamic secure offloading game in Section VI. Simulationresults are provided in Section VII. Conclusions are drawnin Section VIII.

II. RELATED WORKMobile offloading has recently attracted extensive researchattention. For example, a mobile offloading algorithm basedon the user’s local load and the availability of intermit-tently connected cloudlets was proposed in [6] to reducethe computation and offloading costs of mobile devices.Task dispatch, transmission and execution for mobile cloudswere investigated in [11], and a semi-Markovian decisionprocess based mobile offloading method was developed to

achieve a balance between performance and battery life.A delayed offloading model analyzes the energy-delay trade-off was developed in [12], which leverages the complemen-tary strengths of WiFi and cellular networks. Two offloadingstrategies for mobile cloud offloading systems based on theenergy-response timeweighted productmetric were proposedin [13] to analyze the energy-performance tradeoff. Basedon the time complexity and data sharing information atthe procedure calls, the offloading scheme proposed in [8]constructs a cost graph and partitions the computational tasksto save energy. The distributed mobile offloading algorithmproposed in [14] formulates a revenue maximization prob-lem for software defined networks. The offloading strategydesigned in [15] applies a hybrid queueing model and quanti-fies the security attributes and their impact on the offloadingperformance.

Privacy and security are critical for mobile offload-ing [1], [16]. Game theory has been applied to investigate net-work security, especially for attacks with uncertainties. Forexample, the interaction between a wireless transmitter anda dual-threat attacker that can implement both eavesdroppingand jamming was formulated in [3] as a zero-sum game. Thejoint threat from an advanced persistent threat attacker andinsiders was formulated in [4] as a two-player game, and theNE of the game was provided. A zero-sum game between atransmitter and an adversary that is either a passive eaves-dropper or an active jammer was investigated in [17], andconditions for the existence of an NE of the game were exam-ined. A stochastic game was studied in [18] to improve thesecrecy and reliability of communication against an adversarythat can implement both jamming and eavesdropping.

A nested two-stage offloading game was formulated in [7],in which the mobile device judiciously decides whether ornot to offload and chooses the portion of an application tooffload, while the cloud allocates the resources for the mobiledevice. A computation offloading game investigated in [9]can achieve efficient offloading. A Q-learning based malwaredetection strategy was proposed in [19] for smartphones withlimited radio bandwidth, in which each smartphone offloadsa portion of its applications to a security server in the cloudto improve the detection speed and accuracy. We formu-lated a secure offloading game for the interactions among amobile device, a smart attacker and a security agent in [20].Compared with our previous work in [20], in this paper, weincorporate radio channel variations in the game model andinvestigate the impact of the channel gains on the NEs of thegame. In addition, we formulate a Stackelberg game, in whichthe mobile device, attacker and security agent choose theirstrategies in sequence based on the observations of the formerplayer’s actions. We provide the SE of the offloading gameand propose a Q-learning based mobile offloading strategyfor dynamic games.

III. SYSTEM MODELWe consider the offloading of a mobile device (T ) in thepresence of a smart attacker (E) under the protection of

2282 VOLUME 4, 2016




FIGURE 1. Illustration of an offloading game consisting of a mobiledevice with offloading rate x , a smart attacker with attack mode y , and asecurity agent that protects the serving AP in defense mode z .

a security agent (D) at the serving AP. As illustrated in Fig. 1,the mobile device offloads its applications and data to thecloud via the AP. The offloading is threatened by the smartattacker that uses USRPs to performmultiple types of attacks,e.g., spoofing and jamming. The offloading rate of the mobiledevice, denoted by x ∈ [0, 1], is defined as the proportion ofthe data that is sent to the serving AP. If x = 0, the mobiledevice processes all the data locally, while x = 1 means thatall the data are processed in the cloud. The offloading rate ischosen as a tradeoff among the fast processing in the cloud,the radio transmission cost and the risks of being attackedduring offloading.

The smart attacker chooses its attack mode against themobile device, denoted by y ∈ {0, 1, 2}, which corresponds tono attack, spoofing and jamming, respectively. More specif-ically, the attacker sends spoofing signals with the mobiledevice’s identity when y = 1, transmits jamming signalsto block the offloading when y = 2, and keeps silent wheny = 0. The security agent operates in two modes: A safemode combining a physical-layer security method such asthe channel-based spoofing detector of [21] and advancedhigher-layer security mechanisms such as the authenticationtechnique presented in [5] for attack detection at the AP isused when z = 1; and a fast mode in which only the physical-layer security mechanism is used if z = 0. The extra cost ofthe safe mode over the fast mode, denoted by β, is positiveby definition, and the cost of the fast mode is ignored forsimplicity.

The channel gain between the AP and the mobile device ismodeled as a Markov chain. More specifically, the channelgain during the offloading at time n, denoted by hnT ∈ HT ={H l

T }1≤l≤NT , is quantized into NT levels, whereH lT is the l-th

level of the channel gain with HmT < H k

T , ∀1 ≤ m < k ≤ NT .The transition probability of the channel gain hnT is defined aspm,k = Pr(hn+1T = H k

T |hnT = Hm

T ). For simplicity, we assumethat the channel gain only changes to its neighboring states,as shown in Fig. 2. Similarly, the channel gain between theattacker and the AP, denoted by hnE ∈ HE = {H

jE }1≤j≤NE ,

is modeled as a Markov chain with NE states with transitionprobabilities denoted by qm,k = Pr(hn+1E = H k

E |hnE = Hm

E ).For simplicity of notation, the time index of the channel gainis omitted if no confusion results. Table 1 summarizes thenotation used in the paper.

IV. SECURE MOBILE OFFLOADING GAMEWe consider a static secure mobile offloading game denotedby G, in which three players make decisions simultaneously

FIGURE 2. Channel model of offloading based on an NT -state Markovchain, in which pm,k is the transition probability.

TABLE 1. Summary of symbols and notation.

in a time slot: The mobile device chooses its offload-ing rate, x ∈ [0, 1], the smart attacker determines itsattack mode y ∈ {0, 1, 2}, and the security agent selectsits defense mode during offloading with z ∈ {0, 1}.Let Cz

y denote the attack cost of the mobile device if theattacker adopts attack mode y and the security agent protectsthe AP via mode z. The cost of an attack to the offloading con-sists of the loss of themobile device due to attack y, the cost tolaunch the attack, and the penalty to the attacker if caught bythe security agent. For instance, if the penalty of the attackerafter being caught is greater than the user’s loss, Cz

y can benegative. On the other hand, the attack detection accuracy atthe AP depends on the defense mode of the security agent,and thus the attack cost depends on z. The attack cost in thefast mode, in which the security agent uses the physical-layersecurity mechanism to protect the serving AP is greater thanthat in the safe mode, i.e., C0

y > C1y ,∀y = 1, 2. If the attacker

keeps silent with y = 0, then the offloading incurs zero attackcost, i.e., Cz

0 = 0,∀z = 0, 1. For simplicity, we define theattack cost matrix C = [C0

0 ,C10 ;C

01 ,C

11 ;C

02 ,C

12 ].

We assume that both the mobile device and the securityagent make decisions to maximize the same utility denotedby u, which depends on the attack mode, the channel gains,and the transmit power of the offloading signals denotedby PT . Without loss of generality, the receiver noise poweris set to be one. More specifically, if the attacker keeps

VOLUME 4, 2016 2283




silent (y = 0), the mobile device benefits from the channelcapacity minus the transmit cost of the offloading. Thus theutility of the mobile device is given by

u(x, 0, z) = x (log (1+ PT hT )− σPT )− zβ, (1)

where β is the extra cost of the security agent in the safe modecompared with the fast mode, and σ > 0 is the positive unittransmission cost.

If the smart attacker sends a spoofing signal without inter-rupting the ongoing offloading in the time slot (y = 1), theutility of the mobile device can be defined as

u(x, 1, z) = x (log (1+ PT hT )− σPT )− Cz1 − zβ, (2)

where Cz1 is the cost of the mobile device under spoofing

attacks in defense mode z.If the attacker sends jamming signals (y = 2) with

jamming power PE during the offloading, the mobiledevice suffers from a low signal-to-interference-plus-noiseratio (SINR), and its utility can be defined as

u(x, 2, z) = x(log

(1+

PT hT1+ PEhE

)− σPT

)− Cz

2 − zβ.

(3)

In this game, the utility of the attacker denoted by uE is givenby

uE (x, y, z) = −u(x, y, z). (4)

In summary, we consider a three-player secure offload-ing game, G = 〈{T , E,D}, {x, y, z}, {u, uE , u}〉, in whichx ∈ [0, 1], y ∈ {0, 1, 2}, and z ∈ {0, 1}. A Nash equilibriumof the static offloading gameG denoted by (x∗, y∗, z∗) is givenby definition as

u(x∗, y∗, z∗) ≥ u(x, y∗, z∗), ∀x ∈ [0, 1] (5)

u(x∗, y, z∗) ≥ u(x∗, y∗, z∗), ∀y ∈ {0, 1, 2} (6)

u(x∗, y∗, z∗) ≥ u(x∗, y∗, z), ∀z ∈ {0, 1}. (7)

No player has motivation to unilaterally deviate from a Nashequilibrium of a game, as its utility decreases in that case.Theorem 1: The static offloading game G has an NE

(1, 0, 0), if

C01 ≤ 0 (8a)

hT >2σPT − 1

PT(8b)

C02 ≤ log

(1+ PEhE + PT hT

(1+ PEhE )(1+ PT hT )

). (8c)

Proof: By Eqs. (1)-(3), if (8a) and (8c) hold, we have

u(1, 0, 0) = log(1+ PT hT )− σPT

≤ min{log(1+ PT hT )− σPT − C0

1 ,

log(1+

PT hT1+ PEhE

)− σPT − C0

2

}= min {u(1, 1, 0), u(1, 2, 0)}. (9)

If (8b) holds, as x ≤ 1, we have

u(1, 0, 0) = log(1+ PT hT )− σPT≥ x (log(1+ PT hT )− σPT ) = u(x, 0, 0). (10)

Similarly, as β > 0, we have

u(1, 0, 0) = log(1+ PT hT )− σPT≥ log(1+ PT hT )− σPT − β = u(1, 0, 1). (11)

By Eqs. (9)-(11), we see that (5)-(7) hold for (1, 0, 0), whichis an NE of G. �Remark: Under a high penalty of being caught (i.e., (8a)

and (8c)), the smart attacker keeps silent (y = 0). If thechannel gain between the mobile device and the AP is high(i.e., (8b)), the mobile device chooses a full offloading rate(x = 1). In this case, the security agent applies the fastmode at a low cost (z = 0), since the attack motivation issuppressed.Theorem 2: The static offloading game G has an NE

(1, 1, 0), if (8b) holds andmax

{0, log

((1+ PEhE )(1+ PT hT )1+ PEhE + PT hT

)+ C0

2

}≤ C0

1

(12a)

β ≥ C01 − C

11 . (12b)

Proof: By Eqs. (1)-(3), if (12a) holds, we have

u(1, 1, 0) = log(1+ PT hT )− σPT − C01

≤ min{log(1+ PT hT )− σPT ,

log(1+

PT hT1+ PEhE

)− σPT − C0

2

}= min {u(1, 0, 0), u(1, 2, 0)}. (13)

If (8b) holds, as x ≤ 1, we have

u(1, 1, 0) = log(1+ PT hT )− σPT − C01

≥ x (log(1+ PT hT )− σPT )− C01 = u(x, 1, 0).

(14)

If (12b) holds, we have

u(1, 1, 0) = log(1+ PT hT )− σPT − C01

≥ log(1+ PT hT )− σPT − C11 − β = u(1, 1, 1).

(15)

By Eqs. (13)-(15), we see that (5)-(7) hold for (1, 1, 0), whichis an NE of G. �Remark: If spoofing is more harmful than jamming to the

mobile device (i.e., (12a)), the smart attacker sends spoofingsignals (y = 1). If the channel gain of the mobile device ishigh (i.e., (8b)), the mobile device chooses a full offloadingrate (x = 1). The security agent applies the fast mode (z = 0)because the cost of the safe mode is high (i.e., (12b)).

2284 VOLUME 4, 2016




Theorem 3: The static offloading game G has an NE(1, 2, 0), if

β ≥ C02 − C

12 (16a)

hT >

(2σPT − 1

)(1+ PEhE )

PT(16b)

log(

1+ PEhE + PT hT(1+ PEhE )(1+ PT hT )

)≤ min{C0

2 ,C02 − C

01 }. (16c)

Proof: By Eqs. (1)-(3), if (16c) holds, we have

u(1, 2, 0) = log(1+

PT hT1+ PEhE

)− σPT − C0

2

≤ min{log(1+ PT hT )− σPT ,

log(1+ PT hT )− σPT − C01 }

= min {u(1, 0, 0), u(1, 1, 0)} . (17)

If (16b) holds, we have

u(1, 2, 0) = log(1+

PT hT1+ PEhE

)− σPT − C0

2

≥ x(log

(1+

PT hT1+ PEhE

)− σPT

)− C0

2 = u(x, 2, 0).

(18)

If (16a) holds, we have

u(1, 2, 0) = log(1+

PT hT1+ PEhE

)− σPT − C0

2

≥ log(1+

PT hT1+ PEhE

)− σPT − C1

2 − β = u(1, 2, 1).

(19)

By Eqs. (17)-(19), we see that Eqs. (5)-(7) hold for (1, 2, 0),which is an NE of G. �Remark: If jamming is harmful to the mobile device

(i.e., (16c)), the attacker sends jamming signals (y = 2).The mobile device chooses a full offloading rate (x = 1)only if the channel gain of the mobile device is high enough(i.e., (16b)). If the extra cost of the safe mode is high(i.e., (16a)), the security agent chooses the fast mode (z = 0).Theorem 4: The static offloading game G has an NE

(0, 0, 0), if max

{C01 ,C

02

}≤ 0 (20a)

hT ≤2σPT − 1

PT. (20b)

Proof: Similar to that of Theorem 1. �Remark: If the channel gain of the mobile device is low

(i.e., (20b)), the mobile device processes the data locally(x = 0). If the penalty of being detected is high, bothC01 and C0

2 are negative (i.e., (20a)), and thus the attackerkeeps silent (y = 0). In this case, the security agent appliesthe physical-layer security mechanism (z = 0).

Other NEs of the static game are given in the followingtheorem.

Theorem 5: The static offloading game G has an NE(0, 1, 0), if (20b) holds and

max{0,C02 } ≤ C

01 < C1

1 + β. (21)

The game G has an NE (0, 1, 1), if (20b) holds and{C11 ≥ max{0,C1

2 } (22a)

β < C01 − C

11 . (22b)

The game G has an NE (0, 2, 0), ifhT ≤(2σPT − 1

)(1+ PEhE )

PT(23a)

max{0,C01 } ≤ C

02 < C1

2 + β. (23b)

The game G has an NE (0, 2, 1), if (23a) holds and{C12 ≥ max{0,C1

1 } (24a)

β < C02 − C

12 . (24b)

The game G has an NE (1, 2, 1), if (16b) holds andβ < C0

2 − C12 (25a)

log(


)≤ min{C1

2 ,C12 − C

11 }. (25b)

The game G has an NE (1, 1, 1), if (8b), (22b) hold and

C11 ≥ max

{0, log


)+ C1

2

}.

(26)Proof: Similar to that of Theorem 1. �

Remark:Themobile offloading rate depends on the currentchannel gain of the mobile device. Take the NE (0,1,0) as anexample. In this case, the mobile device chooses to processthe data locally, as the channel gain is low, as shown in (20b).The smart attacker sends spoofing signals, if the spoofingcost is high as indicated by (21). The security agent uses thephysical-layer security mechanism in the fast mode as thedata is processed locally.

Fig. 3 summarizes the NEs of the game, in which

ψ = log((1+ PEhE )(1+ PT hT )1+ PEhE + PT hT

). (27)

As shown in Fig. 3 (a), the mobile device chooses a fulloffloading rate if Eqs. (8b), (16b) and (26) hold. The attackerperforms spoofing attacks if (12) holds, as the spoofingsignal is difficult to detect by the security agent. If thecost of a spoofing attack is large, the security agent appliesthe safe mode to secure the mobile device. In Fig. 3 (b),if Eqs. (8b), (16b) and (25b) hold, the mobile device offloadsall the data to the cloud via the AP. The attacker sends jam-ming signals if (16) and (25) hold, because jamming signalscan significantly decrease the SINR of the offloading signalat the AP. If the jamming strength is high, the security agentprotects the AP with the safe mode. Fig. 3 (c) corresponds tothe case in which Eqs. (20b), (22a) and (23a) hold and themobile device processes the data locally. More specifically,

VOLUME 4, 2016 2285




FIGURE 3. NEs of the static secure offloading game G, in which themobile device applies a full offloading rate (full rate) or processes thedata locally (local), the smart attacker chooses spoofing, jamming or noattack, and the security agent enters either the fast mode or safe mode.(a) Conditions (8b), (16b) and (26). (b) Conditions (8b), (16b) and (25b).(c) Conditions (20b), (23a) and (22a). (d) Conditions (20b), (23a) and (24a).

the smart attacker sends spoofing signals if (22a) holds, andthe security agent applies both the physical-layer and higher-layer security mechanisms in the safe mode with a low cost.As shown in Fig. 3 (d), the security agent uses a safe modewith a low cost (24b), while the smart attacker performs ajamming attack if the jamming cost to offloading is high (23b)and (24a), and spoofs if the spoofing cost C0

1 is large (21).

V. MOBILE OFFLOADING STACKELBERG GAMEWe consider a mobile offloading Stackelberg game denotedby G′, in which themobile device chooses its offloading rate xfirst, then the smart attacker chooses its attack mode y basedon the observed offloading rate, and finally the security agentdecides whether to start its advanced defense mode accordingto both the offloading rate and the attackmode. For simplicity,each follower is assumed to accurately obtain the action ofa leader. A Stackelberg equilibrium of its offloading gamedenoted by (xSE , ySE , zSE ) is given by definition as follows:

zSE (x, y) = arg maxz∈{0,1}

u(x, y, z) (28)

ySE (x) = arg miny∈{0,1,2}

u(x, y, zSE (x, y)

)(29)

xSE = arg maxx∈[0,1]

u(x, ySE (x), zSE (x, y)

). (30)

At a Stackelberg equilibrium of the offloading game, themobile device as the leader chooses its offloading rate firstto maximize its utility given by (30) considering the responseof the smart attacker as the follower. The attack mode of theattacker is selected to minimize the utility in (29) based onthe observed offloading rate. The security agent chooses itsdefense mode based on both x and y, as shown in (28).Theorem 6: The static offloading game G′ has an SE

(1, 0, 0), if (8b) and one of (31)-(34) hold, withC01 ≤ 0 (31a)

β ≥ max{C0y − C

1y

}, ∀y = 1, 2 (31b)

C02 ≤ log

(1+ PEhE + PT hT

(1+ PEhE )(1+ PT hT )

)(31c)

C02 − C

12 ≤ β < min

{C01 − C

11 ,−C

11

}(32a)

C02 ≤ log

(1+ PEhE + PT hT

(1+ PEhE )(1+ PT hT )

)(32b)

C01 ≤ 0 (33a)

C01 − C

11 ≤ β < min

{C02 − C

12 ,

log(


)− C1

2

}(33b)

β < min{C01 − C

11 ,C

02 − C

12 ,−C

11 ,

log(


)− C1

2

}. (34)

Proof: By (1), as β > 0, we have

u(x, 0, 0) = x (log(1+ PT hT )− σPT )

≥ x (log(1+ PT hT )− σPT )− β = u(x, 0, 1).

(35)

Thus, (28) holds for (x, 0, 0) and we have zSE (x, 0) = 0.

2286 VOLUME 4, 2016




Similarly, by (2), if β ≥ C01 − C

11 , we have

u(x, 1, 0) = x (log(1+ PT hT )− σPT )− C01

≥ x (log(1+ PT hT )− σPT )− C11 − β = u(x, 1, 1).

(36)

Thus, (28) holds for (x, 1, 0), and we have zSE (x, 1) = 0.Otherwise, if β < C0

1 − C11 , we have z

SE (x, 1) = 1.By (3), if β ≥ C0

2 − C12 , we have

u(x, 2, 0) = x(log

(1+

PT hT1+ PEhE

)− σPT

)− C0

2

≥ x(log

(1+

PT hT1+ PEhE

)−σPT

)−C1

2 − β = u(x, 2, 1),

(37)

indicating that zSE (x, 2) = 0. Otherwise, if β < C02 −C

12 , we

have zSE (x, 1) = 1.If (31) holds, by (1), (2) and (3), we have

u(x, 0, 0) = x (log(1+ PT hT )− σPT )

≤ min{x (log(1+ PT hT )− σPT )− C0

1 ,

x(log

(1+

PT hT1+ PEhE

)− σPT

)− C0

2

}= min {u(x, 1, 0), u(x, 2, 0)}. (38)

Similarly, if (32) holds, we have

u(x, 0, 0) = x (log(1+ PT hT )− σPT )


1 − β,

x(log

(1+

PT hT1+ PEhE

)−σPT

)−C0

2

}= min {u(x, 1, 1), u(x, 2, 0)}. (39)

If (33) holds, we have

u(x, 0, 0) = x (log(1+ PT hT )− σPT )


1 ,

x(log

(1+

PT hT1+ PEhE

)− σPT

)− C1

2 − β

}= min {u(x, 1, 0), u(x, 2, 1)} , (40)

and if (34) holds, we have

u(x, 0, 0) = x (log(1+ PT hT )− σPT )


1 − β,

x(log

(1+

PT hT1+ PEhE

)− σPT

)− C1

2 − β

}= min {u(x, 1, 1), u(x, 2, 1)} . (41)

In summary, (29) holds for (x, 0, 0), thus ySE (x) = 0. By (1),if (8b) holds, we have

u(0, 0, 0) = 0 < log(1+ PT hT )− σPT = u(1, 0, 0). (42)

Thus (1, 0, 0) is an SE, if (8b) and one of (31)-(34)hold. �Remark: In the Stackelberg game G′, the security agent

decides its defense mode based on the observed attack strat-egy y. Under a small cost of the safe mode (i.e., (34)) or theattack cost (i.e., (31), (32) and (33)), the smart attacker keepssilent (y = 0), and thus the security agent chooses the fastmode (z = 0). The mobile device chooses a full offloadingrate (x = 1) if the channel gain is high (i.e., (8b)).Theorem 7: The static offloading game G′ has an SE

(1, 1, 0), if (8b) holds, and either (43) or (44) holds, wheremax

{0, log

(1+ PEhE + PT hT

(1+ PEhE )(1+ PT hT )

)+ C0

2

}< C0

1

(43a)


1y }, ∀y = 1, 2 (43b)

and

C01 > 0 (44a)

C01 − C

11 ≤ β < min

{C02 − C

12 ,

log(


)− C1

2 + C01

}. (44b)

Proof: Similar to that of Theorem 6. �Remark: Under a high channel gain (i.e., (8b)), the mobile

device offloads at a full rate (x = 1). If the spoofing attackis effective (i.e., (43a) and (44a)), the attacker sends spoof-ing signals (y = 1). If the cost of the safe mode is high(i.e., (43b) and (44b)), the security agent checks the AP statusin the fast mode (z = 0).Theorem 8: The static offloading game G′ has an SE

(1, 2, 1), if (16b) holds, and either (45) or (46) holds, whereC01 − C

11 ≤ β ≤ C

02 − C

12 (45a)

log(


)≤ min{C1

2 + β,C12 + β − C

01 } (45b)

and β < min

{C0y − C

1y

}, ∀y = 1, 2 (46a)

log(


)≤ min{C1

2 + β,C12 − C

11 }. (46b)

Proof: Similar to that of Theorem 6. �Remark: If the jamming attack is effective

(i.e., (45b) and (46b)), the attacker chooses to jam (y = 2).If the channel gain of the mobile device is large (i.e., (16b)),the mobile device chooses a full offloading rate (x = 1).The security agent applies a safe mode (z = 1) if its cost isreasonable (i.e., (45a) and (46a)).

With the other two SEs given by Lemma 1 in the appendix,Fig. 4 summarizes the SEs of the game G′. If Eqs. (8b), (16b)and (46b) hold, the mobile device applies full offloading.The attack motivation is suppressed if the penalty of beingdetected is high, possibly due to an accurate attack detection

VOLUME 4, 2016 2287




FIGURE 4. SEs of the secure offloading game G′ if Eqs. (8b), (16b)and (46b) hold, in which the mobile device applies a full offloading rate(full rate), the smart attacker chooses spoofing, jamming or no attack,and the security agent enters either the fast mode or safe mode.

by the security agent (i.e., Theorem 6). The security agentapplies the safe mode, if the mobile device suffers from heavyattack loss (i.e., Theorem 8 and Lemma 1). Otherwise, thesecurity agent chooses the fast mode (i.e., Theorems 6 and 7).

VI. DYNAMIC MOBILE OFFLOADING GAMETo model simulations in which a mobile device repeatsoffloading of its applications and data in dynamic radio envi-ronments, we formulate a dynamic mobile offloading gameconsisting of a mobile device, smart attacker, and securityagent that are unaware of the system parameters such as theattack cost (C), the offloading gain and the current channelcondition (hnT ). Based on Q-learning, the mobile device deter-mines its offloading rate based on the actions of its opponentsand the channel condition in the previous time slot. To thisend, the mobile device observes the system state denoted bys at time n given by sn = [yn−1, zn−1, hn−1T ]. In this game,the offloading rate of the mobile device is quantified into Llevels, i.e., xn ∈ {i/(L − 1)}0≤i≤L−1 for simplicity.Let Q(s, x) denote the quality function of the mobile

device for offloading rate x and state s. The value func-tion denoted by V (s) represents the maximum value of thequality function at state s. The mobile device updates itsQ-function based on its utility u and the value function asfollows:

Q(sn, xn

)← (1− α)Q

(sn, xn

)+α

(u(sn, xn

)+ δV

(sn+1

))(47)

V(sn)= max

xQ(sn, x

), (48)

where α ∈ (0, 1] is a learning factor indicating the weightof the current estimate of the function Q in the updateof the quality function, and the discount factor δ repre-sents the uncertainty of the mobile device about the futurerewards.

By applying the ε-greedy policy [22], the mobile devicechooses its offloading rate xn to maximize its currentQ-function with a high probability 1 − ε, while the

other L − 1 rates are taken with equal probability, i.e.,

Pr(xn) =

1− ε, xn = argmaxx Q (sn, x)ε

L − 1, o.w.

(49)

The secure offloading process of the mobile device is summa-rized in Algorithm 1. This learning-based mobile offloadingstrategy is proposed to provide insights into the dynamicmobile offloading game among the mobile device, the smartattacker and the security agent, and is compared with a bench-mark offloading strategy via simulations in the next section.

Algorithm 1 Secure Offloading Strategy With Q-Learning

Initialize Q(s, x) = 0, V (s) = 0, y0, z0, h0TFor n = 1, 2, 3, ...Update the state sn = [yn−1, zn−1, hn−1T ]Choose the offloading rate xn via (49)Offload to the cloud at rate xn

Observe yn, zn and hnTObtain utility uUpdate Q(sn, xn) via (47)Update V (sn) via (48)

End for

VII. SIMULATION RESULTSSimulations have been performed to evaluate the performanceof the dynamic secure offloading game with L = 11, C =[0, 0;−0.1,−0.3;−0.5,−0.8], HT = HE = {0.8, 0.9, 1},NT = NE = 3, PT = PE = 1, σ = 0.1, β = 0.2, α = 0.9,δ = 0.7, and ε = 0.95. We initialize z0, h0T and h0E randomlyand uniformly, and set pm,k = qm,k with

pm,k

=

{0.5, (m, k) ∈

{(1, 2), (NT ,NT − 1), (i, i)|1≤i≤NT

}0.25, 2 ≤ m ≤ NT − 1, k = m± 1.

(50)

Fig. 5 presents the performance of the proposed offload-ing strategy over time, in which both the attacker and thesecurity agent make decisions based on Q-learning. As abenchmark, we evaluated a random offloading strategy, inwhich the mobile device chooses its offloading rate randomlyand uniformly with x ∈ [0, 1]. As shown in Fig. 5 (a), thespoofing rate decreases rapidly since the start of the dynamicgame, e.g., the spoofing rate decreases by 77% to 7% after1000 time slots, which is 43% lower than that of the ran-dom offloading strategy. In addition, the proposed offloadingstrategy reduces the jamming rate faster than the randomoffloading strategy, e.g., the jamming rate of the proposedoffloading strategy converges to 6% after 500 time slots,while it takes the random offloading strategy 700 time slots toreach the same value. As shown in Fig. 5 (b), the utility of themobile device in the proposed offloading system convergesrapidly to a high value, e.g., the utility increases by 53%after 2000 time slots, which is 2.9 times higher than that

2288 VOLUME 4, 2016




FIGURE 5. Performance of the dynamic offloading game, in which thesmart attacker chooses its strategies based on Q-learning, withC = [0,0;−0.1,−0.3;−0.5,−0.8], σ = 0.1 and β = 0.2. (a) Attack rate.(b) Utility of the mobile device.

of the benchmark strategy, because the mobile device learnsthe system parameters quickly and adjusts its offloading rateaccordingly.

In Fig. 6, we investigate the impact of the channel con-ditions on the offloading game, showing that the averageoffloading rate increases with the channel gain, and sharplyincreases to 0.82 with hT = 0.2, which is 64% higherthan that of the benchmark strategy, because the mobiledevice is motivated to offload more data to the cloud undergood channel conditions. Fig. 6 (b) indicates that the pro-posed offloading strategy reduces the attack rates quickly.For example, the average spoofing rate of the smart attackerdecreases by 47% to 9.4% and the average jamming ratedecreases by 8% to 7.1% with hT = 1 compared to thebenchmark strategy. Moreover, the performance advantageof the proposed offloading strategy increases with hT , e.g.,the gain of the proposed strategy regarding the spoofing rateis 31% with hT = 0.4, and increases to 45% with hT = 0.8.The security agent chooses its protection mode to maximizeits utility based on the offloading rate and the attack mode inthe last time slot. As shown in Fig. 6 (c), the average utilityof the mobile device increases with the channel gain, e.g.,the average utility increases by 4.64 times if hT increasesfrom 0.2 to 1. In addition, compared with the random

FIGURE 6. Performance of the dynamic offloading game, in which thesmart attacker chooses its strategy based on Q-learning, withC = [0,0;−0.1,−0.3;−0.5,−0.8], σ = 0.1 and β = 0.2. (a) Averageoffloading rate. (b) Average attack rate. (c) Average utility of the mobiledevice.

offloading strategy, the average utility of the proposedscheme improves by 93% with hT = 1, because the mobiledevice can choose an offloading rate to reach a balancebetween the transmission cost, security risks and cloud per-formance gain experimentally.

VIII. CONCLUSIONSIn this paper, we have investigated a mobile offloading gameagainst smart attacks, in which a security agent protects aserving AP with two defense modes. We have derived the

VOLUME 4, 2016 2289




NEs and SEs of the secure offloading game and providedconditions for their existence under various radio channelscenarios. We have further proposed a Q-learning basedmobile offloading strategy for dynamic radio environments,in which mobile devices derive the optimal offloading ratesexperimentally. Simulation results show that both the util-ity of the mobile device and the security performance areimproved compared with the benchmark strategy. If the chan-nel gain is high, both the mobile offloading rate and theattack strength increase, and the security agent applies anadvanced defense mode with both higher-layer and physical-layer security mechanisms to protect the offloading againstsmart attacks. For instance, in this situation, the averageutility of themobile device increases in one example by 290%and the spoofing rate decreases by 50% compared with therandom strategy.

APPENDIXLemma 1: The static offloading game G′ has an SE

(1, 1, 1), if (8b) and one of Eqs. (51)-(52) hold, where

max{log


)− C1

1 + C02 ,

− C11 ,C

02 − C

12

}≤ β < C0

1 − C11 (51){

−C11 < β < min{C0

y − C1y }, ∀y = 1, 2

C11 ≥ log

((1+PEhE )(1+PT hT )1+PEhE+PT hT

)+ C1

2 .(52)

The game G′ has an SE (1, 2, 1), if (16b) and one ofEqs. (53)-(54) hold, where


1y }, ∀y = 1, 2 (53a)

C02 ≥ max

{log

(1+ PEhE + PT hT

(1+ PEhE )(1+ PT hT )

),

log(


)+ C0

1

}(53b)

C02 ≥ log

(1+ PEhE + PT hT

(1+ PEhE )(1+ PT hT )

)(54a)

C02 − C

12 ≤ β < min

{C01 − C

11 ,

log((1+ PEhE )(1+ PT hT )1+ PEhE + PT hT

)+ C0

2 − C11

}. (54b)

Proof: Similar to that of Theorem 6. �

REFERENCES[1] K. Kumar and Y.-H. Lu, ‘‘Cloud computing for mobile users: Can offload-

ing computation save energy?’’ IEEE Trans. Comput., vol. 43, no. 4,pp. 51–56, Apr. 2010.

[2] P. Murphy, A. Sabharwal, and B. Aazhang, ‘‘Design of WARP: A wirelessopen-access research platform,’’ in Proc. 14th Eur. Signal Process. Conf.,Sep. 2006, pp. 1–5.

[3] A. Mukherjee and A. L. Swindlehurst, ‘‘Optimal strategies for counteringdual-threat jamming/eavesdropping-capable adversaries in MIMO chan-nels,’’ in Proc. IEEE Military Commun. Conf., Oct. 2010, pp. 1695–1700.

[4] P. Hu, H. Li, H. Fu, D. Cansever, and P. Mohapatra, ‘‘Dynamic defensestrategy against advanced persistent threat with insiders,’’ in Proc. IEEEInt. Conf. Comput. Commun. (INFOCOM), Apr. 2015, pp. 747–755.

[5] D. He, C. Chen, S. Chan, and J. Bu, ‘‘Secure and efficient handoverauthentication based on bilinear pairing functions,’’ IEEE Trans. WirelessCommun., vol. 11, no. 1, pp. 48–53, Jan. 2012.

[6] Y. Zhang, D. Niyato, and P. Wang, ‘‘Offloading in mobile cloudlet systemswith intermittent connectivity,’’ IEEE Trans. Mobile Comput., vol. 14,no. 12, pp. 2516–2529, Dec. 2015.

[7] Y. Wang, X. Lin, and M. Pedram, ‘‘A nested two stage game-based opti-mization framework in mobile cloud computing system,’’ in Proc. IEEEInt. Symp. Service Oriented Syst. Eng., Mar. 2013, pp. 494–502.

[8] Z. Li, C. Wang, and R. Xu, ‘‘Computation offloading to save energy onhandheld devices: A partition scheme,’’ in Proc. ACM Int. Conf. Compil.,Archit., Synth. Embedded Syst., Oct. 2001, pp. 238–246.

[9] X. Chen, ‘‘Decentralized computation offloading game for mobilecloud computing,’’ IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 4,pp. 974–983, Apr. 2015.

[10] H. Sun, S. Hsu, and C. Chen, ‘‘Mobile jamming attack and its countermea-sure in wireless sensor networks,’’ in Proc. IEEE Int. Conf. Adv. Inf. Netw.Appl. Workshops, May 2007, pp. 457–462.

[11] S. Chen, Y. Wang, and M. Pedram, ‘‘A semi-Markovian decision processbased control method for offloading tasks from mobile devices to thecloud,’’ in Proc. IEEE Global Commun. Conf. (GLOBECOM), Dec. 2013,pp. 2885–2890.

[12] H.Wu, Y. Sun, and K.Wolter, ‘‘Analysis of the energy-response time trade-off for delayed mobile cloud offloading,’’ ACM SIGMETRICS Perform.Eval. Rev., vol. 43, pp. 33–35, Sep. 2015.

[13] H. Wu, W. Knottenbelt, and K. Wolter, ‘‘Analysis of the energy-responsetime tradeoff for mobile cloud offloading using combined metrics,’’ inProc. IEEE 27th Int. Teletraffic Congr. (ITC), Sep. 2015, pp. 134–142.

[14] L. Liu, X. Chen, M. Bennis, G. Xue, and Z. Han, ‘‘A distributed ADMMapproach for mobile data offloading in software defined network,’’ in Proc.IEEEWireless Commun. Netw. Conf. (WCNC), Mar. 2015, pp. 1748–1752.

[15] T. Meng, K. Wolter, and Q. Wang, ‘‘Security and performance tradeoffanalysis of mobile offloading systems under timing attacks,’’ in ComputerPerformance Engineering, vol. 9272. Madrid, Spain: Springer, Aug. 2015,pp. 32–46.

[16] J. S. Park, K. J. Yi, and J. H. Park, ‘‘SSP-MCloud: A study on securityservice protocol for smartphone centric mobile cloud computing,’’ in ITConvergence and Services, vol. 107. Gwangju, South Korea: Springer,Nov. 2011, pp. 165–172.

[17] A. Mukherjee and A. L. Swindlehurst, ‘‘Jamming games in the MIMOwiretap channel with an active eavesdropper,’’ IEEE Trans. SignalProcess., vol. 61, no. 1, pp. 82–91, Jan. 2013.

[18] A. Garnaev, M. Baykal-Gursoy, and H. V. Poor, ‘‘A game theoretic analysisof secret and reliable communication with active and passive adversarialmodes,’’ IEEE Trans. Wireless Commun., vol. 15, no. 3, pp. 2155–2163,Mar. 2016.

[19] Y. Li, J. Liu, Q. Li, and L. Xiao, ‘‘Mobile cloud offloading for mal-ware detections with learning,’’ in Proc. IEEE Int. Conf. Comput.Commun. (INFOCOM), BigSecurity Workshop, Apr. 2015, pp. 197–201.

[20] L. Xiao, C. Xie, T. Chen, H. Dai, and H. V. Poor, ‘‘Mobile offload-ing game against smart attacks,’’ in Proc. IEEE Int. Conf. Comput.Commun. (INFOCOM), BigSecurity Workshop, San Francisco, CA, USA,Apr. 2016, pp. 249–254.

[21] L. Xiao, L. J. Greenstein, N. B. Mandayam, and W. Trappe, ‘‘Channel-based spoofing detection in frequency-selective Rayleigh channels,’’ IEEETrans. Wireless Commun., vol. 8, no. 12, pp. 5948–5956, Dec. 2009.

[22] L. Busoniu, R. Babuska, and B. De Schutter, ‘‘A comprehensive survey ofmultiagent reinforcement learning,’’ IEEE Trans. Syst., Man, Cybern. C,Appl. Rev., vol. 38, no. 2, pp. 156–172, Mar. 2008.

LIANG XIAO (M’09–SM’13) received theB.S. degree in communication engineering fromthe Nanjing University of Posts and Telecommuni-cations, Nanjing, China, in 2000, the M.S. degreein electrical engineering from Tsinghua Univer-sity, Beijing, China, in 2003, and the Ph.D. degreein electrical engineering from Rutgers University,New Brunswick, NJ, USA, in 2009. She is cur-rently a Professor with the Department of Commu-nication Engineering, Xiamen University, Fujian,

China. Her current research interests include smart grids, network security,and wireless communications.

2290 VOLUME 4, 2016




CAIXIA XIE received the B.S. degree incommunication engineering from Xiamen Uni-versity, Xiamen, China, in 2015, where sheis currently pursuing the M.S. degree withthe Department of Communication Engineering.Her research interests include network securityand wireless communications.

TIANHUA CHEN received the B.S. degree incommunication engineering from Xiamen Uni-versity, Xiamen, China, in 2014, where sheis currently pursuing the M.S. degree withthe Department of Communication Engineering.Her research interests include network securityand wireless communications.

HUAIYU DAI (M’03–SM’09) received theB.E. and M.S. degrees from Tsinghua Univer-sity, Beijing, China, in 1996 and 1998, respec-tively, and the Ph.D. degree from PrincetonUniversity, Princeton, NJ, in 2002, all in electricalengineering.

He was with Bell Labs, Lucent Technolo-gies, Holmdel, NJ, in summer 2000, and withAT&T Labs-Research, Middletown, NJ, in sum-mer 2001. He is currently a Professor of Electrical

and Computer Engineering with NC State University, Raleigh. His researchinterests are in the general areas of communication systems and networks,advanced signal processing for digital communications, and communicationtheory and information theory. His current research focuses on networkedinformation processing and crosslayer design in wireless networks, cognitiveradio networks, wireless security, and associated information-theoretic andcomputation-theoretic analysis.

He has served as an Editor of the IEEE TRANSACTIONS ON COMMUNICATIONS,SIGNAL PROCESSING, and Wireless Communications. Currently he is an AreaEditor in charge of wireless communications for the IEEE TRANSACTIONS

ON COMMUNICATIONS. He co-edited two special issues of EURASIP journalson distributed signal processing techniques for wireless sensor networks,and on multiuser information theory and related applications, respectively.He co-chaired the Signal Processing for Communications Symposium ofthe IEEE Globecom 2013, the Communications Theory Symposium of theIEEE ICC 2014, and the Wireless Communications Symposium of the IEEEGlobecom 2014.

H. VINCENT POOR (S’72–M’77–SM’82–F’87)received the Ph.D. degree in EECS from Prince-ton University, in 1977. From 1977 until 1990,he was on the faculty of the University of Illi-nois at Urbana–Champaign. Since 1990, he hasbeen on the faculty of Princeton University,where he is the Michael Henry Strater UniversityProfessor and the Dean of the School of Engineer-ing and Applied Science. He has also held visit-ing appointments at several universities, including

most recently at Stanford University and Imperial College. His researchinterests are in the area of wireless networks and related fields. Among hispublications in these areas is the recent book Mechanisms and Games forDynamic Spectrum Allocation (Cambridge University Press, 2014).

Dr. Poor is a member of the National Academy of Engineering andthe National Academy of Sciences, and is a foreign member of the RoyalSociety. He is also a fellow of the American Academy of Arts and Sciences,the National Academy of Inventors, and other national and internationalacademies. He received the Marconi and Armstrong Awards of the IEEECommunications Society in 2007 and 2009, respectively. Recent recognitionof his work includes the 2014 URSI Booker GoldMedal, the 2015 EURASIPAthanasios Papoulis Award, the 2016 John FritzMedal, and honorary doctor-ates from Aalborg University, Aalto University, HKUST, and the Universityof Edinburgh.

VOLUME 4, 2016 2291

