ips.3: reinforcement learning based mobile offloading for ...in 2015, about 144 million new malwares...

Xiaoyue Wan, Geyi Sheng, Yanda Li, Liang Xiao, Xiaojiang Du

Speaker: Geyi ShengDept. of Communication Engineering, Xiamen University

Xiamen Fujian, ChinaDec. 7, 2017

IPS.3: Reinforcement Learning Based Mobile Offloading for Cloud-based Malware Detection

Outline

2

Background & motivation Challenges & opportunities from big data

Cloud-based malware detection model for mobile devices Learning based malware detection schemes Hotbooting-Q based malware detection DQN-based malware detection

Simulation results Conclusion

IPS.3: Reinforcement Learning Based Mobile Offloading for Cloud-based Malware Detection, X. Wan, G. Sheng, Y. Li, L. Xiao, X. Du

3

Malware refers to viruses, Trojans, spywares and other intrusive code Used to disrupt computer or mobile operations, gather sensitive information, gain

access to private computer systems, or display unwanted advertising [Idika’07]

The average smartphone infection rate increased 96 percent in the first half of 2016, compared to the second half of 2015 [Nokia’16] Up to 10 million Android smartphones around the world have been infected by

Hummingbad malware that generates fake clicks for adverts, which makes 300,000$ per mon for the malware attacker [BBC’16]

Motivation


Big Data in Malware Detection

4

8000000 Android malware samples

2000000 Android malware samples

In 2015, about 144 million new malwares were found, in which 274 new unknown malware were produced and launched every minute [Check-Point’16]

The number of Android malware samples in Nokia malware database increased by 75 percent in the first half of 2016 [Nokia’16]


Number of Android malware samples in Nokia malware

5

Malware Detection Methods Signature-based detection

Rely on human expertise in creating the label of malicious behaviors Applicable for light computation on smartphones Fail to address zero-day malware: an attack not publicly reported or

announced before becoming active Anomaly-based detection

Applicable to address various types of malwares A large volume of samples required in the training phase High computation complexity & high false alarm rate

Hybrid detection Raise detection rates of known malwares Decrease the false positive rate for unknown attacks


6

Challenges of Malware Detection at Smartphones Big app trace data: A large number of log data is generated by the

applications run at smartphone High storage cost Limited computational speed to run the detection algorithm Detection accuracy limited by the size of the virus database at

the smartphone downloaded from the security servers Zero-day malware attacks

117903 lines of log data


Chart1

Size of the whole operation data

Size of the generated log data

列1

The log data evaluated in Norton security application

8.2

3.2

Sheet1

列1

Size of the whole operation data8.2

Size of the generated log data3.2

若要调整图表数据区域的大小，请拖拽区域的右下角。

Advantages of cloud-based detection: Fast computation to run more advanced and complex detection algorithms More accurate detection with a large-size signature database Address zero-day vulnerabilities

7

Cloud-based Malware Detection System Online cloud anomaly detection for both system and network level data

using dedicated monitoring components based on SVM [Watson’16]


8

Mobile Offloading in the Cloud-based Malware Detection


Smartphone divides real-time app traces and labels with serial numbers Offload a portion of the traces to the cloud for malware

Cloud-based detection vs. local detection Transmission delay, CPU occupying, detection accuracy, storage cost

Mobile users in the malware detection compete for the cloud computational resource and the network bandwidth,

and cooperate to improve the malware detection accuracy at the cloud

Mobile Offloading Model

9 IPS.3: Reinforcement Learning Based Mobile Offloading for Cloud-based Malware Detection, X. Wan, G. Sheng, Y. Li, L. Xiao, X. Du

Repeated interactions among mobile devices in the cloud-based malware detection under time-variant network environment

Q-learning based malware detection: Offloading rate is chosen without knowing the network bandwidth model and the app trace generation model

A model-free reinforcement learning algorithm for an agent to derive the optimal strategy via trial-and-errors

State: Network bandwidth and the offloading rates of the other devices at last time

Q-function: Estimated discounted long-term utility for each state-action pair

Q-function update based on iterative Bellman equation: Estimate of optimal future value

Encourage exploration with ε-greedy policy: Avoid tracking in the local optimum at the beginning

10

Q-Learning based Malware Detection


11

Hotbooting Q-learning based Malware Detection


Hotbooting technique that initializes the Q-value based on the training data in similar scenarios Decrease the random explorations at the beginning Accelerate the learning speed in the dynamic game

App traces generation

Divide APP traces

Q-learning offloading rate selection based on ε-greedy policy

Mobile Device i

AP/BSMachine learning based malware

detection

Cloud server

Emulate a similar environment

Hotbooting preparation

Detection results

Initialize Q-value

Chosen APP traces

12

DQN-based Malware Detection


App traces reminder

DQN loss and stochastic gradient descent

Evaluate utility

Choose offloading rate with ε-greedy

Current system state sequence

Update CNN weights

Q-v

alues

Mobile device i

ϕi(j)

ϕi(k)

Replay memory

Other mobile devices, security server, and the serving AP/BS

Conv 14×4×20

Conv 23×3×40

FC 1180

FC 2Nx+1

CNN

Current state

Local malware detections

Malware detection report Offloading rate vector of other devices Offload the chosen App traces part

Chosen malware detection experience

Simulation Results


0 1000 2000 3000 4000 5000 6000

Time slot

7

8

9

10

11

12

13

14

15

16

17

Aver

age

dete

ctio

n de

lay

Q

HotbootingQ

DQN

0 1000 2000 3000 4000 5000 6000

Time slot

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

Aver

age

accu

racy

gai

nQ

Hotbooting-Q

DQN

0 1000 2000 3000 4000 5000 6000

Time slot

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

Aver

age

Util

ity

Q

Hotbooting-Q

DQN

13

We have formulated a cloud-based malware detection model, in which the mobile devices compete for the limited radio transmission resource and cooperate to improve the malware detection accuracy of the security server.

A hotbooting-Q based mobile offloading strategy has been proposed to improve the malware detection performance compared to the Q-learning based scheme, and the performance is further improved by the DQN-based malware detection.

14 IPS.3: Reinforcement Learning Based Mobile Offloading for Cloud-based Malware Detection, X. Wan, G. Sheng, Y. Li, L. Xiao, X. Du

Conclusion

15

Questions?

[email protected]


mailto:[email protected]

IPS.3: Reinforcement Learning Based Mobile Offloading for Cloud-based Malware DetectionOutline幻灯片编号 3Big Data in Malware DetectionMalware Detection MethodsChallenges of Malware Detection at Smartphones幻灯片编号 7Mobile Offloading in the Cloud-based Malware DetectionMobile Offloading Model幻灯片编号 10幻灯片编号 11幻灯片编号 12幻灯片编号 13幻灯片编号 14幻灯片编号 15

ips.3: reinforcement learning based mobile offloading for ...in 2015, about 144 million new malwares...

Documents