Advances In Malware Detection-An Overview

Heenaa,b,c , B. M. Mehtrea,d

aCenter of excellence in cyber security, Institute for Development and Research in Banking Technology (IDRBT),Hyderabad, India

bSchool of Computer Science and Information Sciences (SCIS), University of Hyderabad, Hyderabad, Indiac heenarao014@gmail.comd bmmehtre@idrbt.ac.in

Abstract

Malware has become a widely used means in cyberattacks in recent decades because of various new ob-fuscation techniques used by malwares. In order toprotect the systems, data and information, detection ofmalware is needed as early as possible. There are vari-ous studies on malware detection techniques that havebeen done but there is no method which can detectthe malware completely and make malware detectionproblematic. Static Malware analysis is very effectivefor known malwares but it does not work for zero daymalware which leads to the need of dynamic malwaredetection and the behaviour based malware detectionis comparatively good among all detection techniqueslike signature based, deep learning based, mobile/IOTand cloud based detection but still it is not able to de-tect all zero day malware which shows the malwaredetection is very challenging task and need more tech-niques for malware detection. This paper describes aliterature review of various methods of malware de-tection. A short description of each method is pro-vided and discusses various studies already done in theadvanced malware detection field and their compari-son based on the detection method used, accuracy andother parameters. Apart from this we will discuss var-ious malware detection tools, dataset and their sourceswhich can be used in further study. This paper givesyou the detailed knowledge of advanced malwares, itsdetection methods, how you can protect your devicesand data from malware attacks and it gives the com-parison of different studies on malware detection.

Keywords- Cyber Security, Malware Detection ap-proaches, Malware Classification, Malware features.

1 IntroductionTechnology nowadays has become so advanced, ev-erything is adapting the digital over the manual way ofworking. Technology has its pros and cons, if it makeslife easier then at the same time it invites the cyber at-tacks, loss of data, giving access to your personal lifeto someone who can misuse it. So the security of ourdevices is very important in today’s cyber world. Theinternet usage is increasing day by day. One draw-back of the widespread use of the internet is that manycomputer systems are vulnerable to attacks and get in-fected with malwares. There are different names formalware for example malicious code, malicious pro-gram or malicious executable. Malware is malicioussoftware which is used with the intention of breach-ing a computer system’s security policy with respect toconfidentiality, integrity and availability of data[33]. Itcan change and remove your system, data without yourknowledge to harm the system. According to multiplestudies the complete 100% malware detection problemis NP complete problem [39][42], but in every studythe researchers always try to get maximum accuracyby using different methods.

1.1 Types of MalwareFigure 1 shows the different types of malware .

1.1.1 Virus

Viruses attach their malicious code to clean code andwait for an unsuspecting user or an automated pro-cess to execute them. Like a biological virus, they canspread quickly and widely, causing damage to the core

1

arX

iv:2

104.

0183

5v2

[cs

.CR

] 8

May

202

1

Figure 1: Types Of Malware

functionality of systems, corrupting files and lockingusers out of their computers. They usually hide withinan executable file.[34].

1.1.2 Worm

Worms get their name from the way they infect sys-tems. Starting from one infected machine, they weavetheir way through the network, connecting to consecu-tive machines in order to continue the spread of infec-tion. This type of malware can infect entire networksof devices very quickly[34].

1.1.3 Trojans

This type of malware hides within or disguises itselfas legitimate software. Acting discreetly, it will breachsecurity by creating backdoors that give other malwarevariants easy access[34].

1.1.4 Spyware

It Hides in the background on a computer and this typeof malware will collect information without the userknowing, such as credit card details, passwords andother sensitive information.Spyware is software that is installed on your computereither directly or inadvertently. A Trojan horse pro-gram is similar to spyware except that it is packagedas another program[34].

1.1.5 Ransomware

Also known as scareware, ransomware comes with aheavy price. They are able to lockdown networks andlock out users until a ransom is paid, ransomware hastargeted some of the biggest organizations in the worldtoday with expensive results[43].

1.1.6 Adware

Adware is malware that forces your browser to redirectto web advertisements, which often themselves seek todownload further, even more malicious software[34].

1.1.7 Rootkit

Rootkit is, a program or, more often, a collection ofsoftware tools that gives a threat actor remote access toand control over a computer or other system. It gets itsname because it’s a kit of tools that (generally illicitly)gain root access (administrator-level control, in Unixterms) over the target system, and use that power tohide their presence[43].

1.1.8 Cryptojacking

Cryptojacking is the unauthorized use of someoneelse’s computer to mine cryptocurrency.Cryptojacking is another way attackers can force youto supply them with Bitcoin, only it works without younecessarily knowing. The crypto mining malware in-fects your computer and uses your CPU cycles to mineBitcoin for your attacker’s profit. The mining softwaremay run in the background on your operating systemor even as JavaScript in a browser window[43].

1.1.9 Malvertising

Malvertising is the use of legitimate ads or ad net-works to covertly deliver malware to unsuspectinguser’s computers. For example, a cybercriminal mightpay to place an ad on a legitimate website. When auser clicks on the ad, code in the ad either redirectsthem to a malicious website or installs malware ontheir computer. In some cases, the malware embed-ded in an ad might execute automatically without anyaction from the user, a technique referred to as a “driveby download”[43].

1.1.10 Fileless

Fileless malware attacks do not download maliciousfiles or write any content to the disk in order to com-promise the systems. The attacker exploits merelythe vulnerable application to inject malicious code di-rectly into the main memory. The attacker can alsoleverage the trusted and widely used applications, i.e.,

2

Microsoft office or administration tools native to Win-dows OS like PowerShell and WMI to run scripts andload malicious code directly into volatile memory[11].

1.1.11 Stealth Malware

A stealth virus is a hidden computer virus that at-tacks operating system processes and averts typicalanti-virus or anti-malware scans. Stealth viruses hidein files, partitions and boot sectors and are adept at de-liberately avoiding detection. Stealth malware are ofdifferent types based on what they are hiding. Stealthmalware uses the hooking technique to divert the orig-inal system call to malware. There are mainly 4 typesof stealth malware rootkits, code mutation, anti em-ulation, targeting mechanism[30]. Rootkit can useuser mode hooking, kernel mode hooking or hybridby combining both to inject malicious code[41]. Codemutation is malware changing its code to hide fromantivirus using mutation engines but it can be detectedvia emulation. The other type of stealth malwaresanti-emulation behaves differently while running in anemulated environment. They sense the environmentand change the behavior according to the environment.The targeted mechanism of stealth malware runs andspreads only on the chosen systems. There are differ-ent countermeasures for component based and patternbased stealth malware. To detect component basedstealth malware we can use the technique of detectinghooks using signature and heuristic methods which re-sult in high false positive rate or Cross-View Detectionand Specification Based Methods in which the outputof API calls are compared with low-level calls that aredesigned to do the same thing.The other countermea-sure is using hardware solution where a clean machinecan be use to monitor another machine for the pres-ence of rootkits/stealth malware. Virtualization tech-niques are also able to detect stealth malware but theseare also vulnerable to anti-mutation malware. For de-tecting pattern based stealth malware signature based,behaviour, heuristic and model based techniques canbe used. There are multiple studies on stealth malwaredetection but no one gives the good result and need tofind more good methods and studies[30][41][44][45].

1.2 Malware Spreading TechniquesEach type of malware has its own unique way of caus-ing havoc, and most rely on user action of some kind.

Some strains are delivered over email via a link or exe-cutable file. Others are delivered via instant messagingor social media. Even mobile phones are vulnerable toattack. It is essential that organizations are aware of allvulnerabilities so they can lay down an effective line ofdefense[22].

1.2.1 Repackaging

Repackaging includes the disassembling of the popu-lar benign applications, then appending the maliciouscontent and finally reassembling and distributing themon other less monitored third party markets. This isdone by reverse-engineering tools.

1.2.2 Drive By Download

It Occurs when a user visits a website that containsmalicious content and downloads malware into the de-vice.

1.2.3 Dynamic Payloads

Uses dynamic payload to download an embedded en-crypted source in an application. After installation, theapplication decrypts the encrypted malicious payloadand executes the malicious code.

1.2.4 Stealth Malware Technique

Stealth Malware Technique refers to an exploit ofhardware vulnerabilities to obfuscate the maliciouscode to easily bypass the anti-malware.

1.3 Malware Evasion Techniques

1.3.1 Anti-security technique

These techniques are used to avoid detection by secu-rity devices and programs as anti-malwares, malwares,firewalls, and any other tools that protect the environ-ment.

1.3.1.1 Fragmentation

The malware splits into several components that onlyexecute when it is reassembled.

3

1.3.2 Anti-sandbox technique

Anti-sandbox technique is used to detect automaticanalysis and to avoid reports on the behavior of mal-ware. This can be done by detecting registry keys, filesor processes related to virtual environments environ-ments.

1.3.2.1 Stalling Delays

The malware simply does nothing for an extended pe-riod. Typically, 10 minutes is sufficient for most sand-boxes[52] to timeout and assume the object is benign.

1.3.2.2 Suspended Activities

The malware postpones these malicious actions whileit is operating within a sandbox.A) Injection or modification of code within other ap-plications.B) Establish persistence and download additionalcode.C)Move laterally across the network.D) Connect to its C&C servers.

1.3.2.3 Rootkits

The malware hides malicious code in the lower layersof the operating system where conventional sandboxtechnology can’t see it.

1.3.3 Anti-analyst techniques

In these techniques, a monitoring tool is used to avoidreverse engineering. The tools might be process ex-plorer or Wireshark to perform monitoring and to de-tect malware analysts[43].

1.3.3.1 ROP Evasion

Return-Oriented Programming (ROP) The malwareinjects functionality into another process without al-tering the code of that process. This is achieved bymodifying the contents of the stack, which is the set ofmemory addresses that tell the system which segmentof code to execute next.

1.3.3.2 User Action Required

The malware avoids doing anything malicious until auser performs a specific action (e.g., a mouse click,

pressing a key, opening or closing a file, or exiting theprogram).

Malware creators might use two or three of theabove techniques to make detection more diffi-cult[43][44].The rest of the paper is structured as follows. Sec-tion(2) gives detailed view of the different type of mal-ware detection Techniques, section(3) contain the de-tails of tools for malware detection, section(4) is aboutthe datasets for malware detection, section(5) containthe comparison of different malware detection studies,section(6) has some challenges from previous studies,section(7) contains the conclusion and the last sec-tion(8) contains the References used in this paper.Allthe tables used are at the end of paper.

2 Malware Detection

Malware detection has multiple stages, which worktogether to detect or classify the malware. Allthe previous study focuses on malware detection inwindows, smartphones and embedded systems(IOT)mainly. The study on malware detection is increasingin smartphones nowadays. The method for detectionof malware is changing day by day as new researchescome based on the increasing complexity of malware.The main malware detection process remains mostlysame for all the studies as following[1]:1) Malware analysis2) Feature Extraction/selection3)classification/detectionThere are mainly 2 types of malware analysis staticmethod and dynamic method which are mainly usedto analyse the malicious file based on various parame-ters. Static Malware Analysis (SMA)[34] where onlybasic analysis is done and malwares are detected with-out executing them. The methods used for static analy-sis are Basic Information Analysis, Structure Analysis,and Control Flow Analysis etc. But malwares whichuses different measures of Polymorphism, Metamor-phism, ShellCode etc. Can not be detected by staticanalysis.Some of the tools[12] for static analysis arein table 1. Dynamic Malware Analysis (DMA)[34] isdone at the time of program execution. This techniqueis useful in analyzing the malwares which uses tech-niques such as Polymorphism, Metamorphism, ShellCode etc. But it is not useful in detecting Zero day

4

malwares. Some of the tools for dynamic analysisare in table 1. Hybrid Analysis [34] is proposed toovercome the limitations of static and dynamic analy-sis techniques. It firstly analyses the signature spec-ification of any malware code and then combines itwith the other behavioral parameters for enhance-ment of complete malware analysis. Due to this ap-proach hybrid analysis overcomes the limitations ofboth static and dynamic analysis. For detecting themalicious file for each technique we need some fea-tures which will become the inputs to the detectionprocess. But a file contains a large number of fea-tures and not all features are beneficial for a particu-lar detection/classification algorithm and a large num-ber of features can increase the execution time so weneed to extract the correct features based on require-ment. Sometimes by just changing the feature selec-tion criteria we can increase accuracy. Sefer Kurnazand Mokhalad Eesee Khudhur[18] used 4 data miningclassification algorithms to classify the malware files.They used SVM, Random Forest, KNN and Hoeffd-ing Tree over the dataset[for win32] containing 12593malicious and 2405 benign files and they have usedweka tool and three sets of features to pass to datamining algorithms. According to them random for-est gives the best results[accuracy 98%] to classify themalicious files based on different parameters like TPR,FPR, Accuracy, Recall, Precision and Receiver Oper-ating characteristically graph. They claim that with thesame algorithm and dataset their system give better ac-curacy than previous study by changing the feature se-lection methods.They used three different sets of fea-ture selection methods "Symmetrical Uncertainty At-tributeEval", "Information Gain (IG)", and " Correla-tion Attribute Eval ", as the best criterion utilized toselect best features and the accuracy by the same algo-rithm increased.

2.1 Malware Detection techniquesIn recent years the studies on malware detection hasincreased. The most used detection techniques in pastare the signature based detection and the behaviourbased detection and some studies used them both incombination as some feature taken from signaturesdetection and some are from behaviour of applica-tion which is under heuristic or hybrid detection. Thesubsection(2.1.3) discuss heuristic approach in detail.Nowday studies of malware detection focus mostly on

mobile devices as the smartphones are the most useddevices and most vulnerable. At the back all of thestudies used machine learning approaches[1]. But wecan classify them according to the platform and detec-tion methodology in different form as deep learningbased detection (which is part of machine learning butit mainly focus on neural networks, so we will discussthem in details), cloud based (as the detection are donesomewhere at remote servers), IOT based(malware inembedded systems) and many more as shown in fig-ure 2.

2.1.1 Signature-based detection

Nowadays pattern matching is the most commonmethod in malware detection, and signature based de-tection is the most popular method in this area [35].Signature is a unique feature for each file, some-thing like a fingerprint of an executable. Signaturebased methods use the patterns extracted from vari-ous malwares to identify them and are more efficientand faster than any other methods. These signaturesare often extracted with special sensitivity for beingunique, so those detection methods that use this sig-nature have small error rates. Where this small errorrate is the main reason that most common commer-cial antiviruses use this technique. These methods areunable to detect unknown malware variants and alsorequire a high amount of manpower, time, and moneyto extract unique signatures. These are the main disad-vantages of these methods. Also, inability to confrontagainst the malwares that mutate their codes in eachinfection such as polymorphic and metamorphic one isanother disadvantage. To tackle these challenges, re-search societies propose completely new malware de-tection families. It can not detect unknown and poly-morphic malware variants.Ömer ASLAN [29] has compared the Static MalwareAnalysis Tools and Antivirus Scanners To Detect Mal-ware and shows that it is difficult to detect malware byonly using one static tool or a few tools. Using onlystatic analysis tools or antivirus software may not beenough as well. To correctly mark a suspicious pro-gram, it is recommended to use static tools with an-tivirus scanners. For unknown malware, the perfor-mance of the antivirus software declined sharply. Thedetection rate declined from 79% to 56% and accu-racy declined from 80% to 65%. These results showthat antivirus software cannot detect zero-day mal-

5

Figure 2: Malware detection Techniques

6

ware. Signature-based detection tools such as antivirusscanners are fast and effective when detecting existingmalware, but it is almost impossible to detect unknownmalware. On the other hand, static detection tools aremore accurate when detecting more complex and zero-day malware. However, static analysis tools cannotdetect a lot of new unknown malware too.

2.1.2 Behavior-based detection

Behavior based malware detection techniques observebehavior of a program to conclude whether it is ma-licious or not [35]. Since behavior based techniquesobserve what an executable file does, they are notsusceptible to the shortcomings of signature-basedones. Simply put, a behavior based detector concludeswhether a program is malicious by inspecting what itdoes rather than what it says. In these methods, pro-grams with the same behavior are collected. Thus, asingle behavior signature can identify various samplesof malware. These types of detection mechanisms helpin detecting malware that keep on generating new mu-tants since they will always use the system resourcesand services in the similar manner. A behavior-baseddetector basically consists of the following compo-nents[35]:

1. Data Collector: This component collects dy-namic/static information about the executable.

2. Interpreter: This component converts raw infor-mation collected by data collection modules intointermediate representations.

3. Matcher: It is used to compare this representationwith the behavior signatures.

There’s a multitude of behaviors that point to poten-tial danger. Here are some examples:

• Any attempt to discover a sandbox environment• Disabling anti-virus or other security controls• Modifying the boot record or other initialization

files to alter boot-up• Installing rootkits• Registering for autostart• Shutting down or disabling system services• Downloading and installing unknown software• Deleting, altering, or adding system files• Modifying other executable programs• Connecting with known malicious sites• Encrypting files that are unrelated to the program

• Adding or modifying user accounts• Dynamic code building to enhance evasion capa-

bilities• Executing a dropped file• Spawning Powershells• Performing any actions that are highly abnormal

Disadvantage of Behaviour based detection:

• Non availability of promising false positive ra-tion(FPR)

• High scanning time.• Can not detect zero day malwares properly.

2.1.3 Heuristic Based Detection:

In heuristic method detectors use the features fromboth signature and behaviour technique and use thatcombined to detect the malware which changes de-pending upon various things. It uses features like APIcalls, CFG, opcode, n-gram, list of DLLs and otherhybrid features. At the back this technique can useany machine learning algorithm to train and test themodel and classify or detect malware. Although ithas a high accuracy rate to detect zero-day malwareto a certain degree, it cannot detect complicated mal-ware. To overcome the disadvantages of signature,and behavioral-based malware detection approachesZ. Bazrafshan[35] gives the survey on heuristic detec-tion methods and machine learning algorithms used.He gives detailed study of the features API Calls,CFG, N-gram, Opcode and hybrid features. He useda machine learning algorithm to generate a patternwhich was similar to signature. Based on the signa-ture, new suspicious programs were marked malwareor benign.

2.1.4 Deep learning Based Detection

Deep learning is part/subfield of artificial neural net-works(ANN) and able to learn without human super-vision, drawing from data that is both unstructuredand unlabeled. This is mainly used to reduce featuresin malware detection. Berman[14]gives the detailedview of various neural networks used in malware de-tection like deep belief network, Recurrent NN, Con-volutional NN,Generative adversarial network, Recur-sive NN and various open datasets with 2k to 4M files.Alzaylaee[2] proposed DL-Droid an application using

7

dynamic analysis and stateful input generation in de-tection of malware in android and compared the de-tection performance and code coverage of the state-ful input generation method with the commonly usedstateless approach using the deep learning system. TheDL-Droid gives 97.8% detection rate (with dynamicfeatures only) and 99.6% detection rate (with both dy-namic and static features) respectively which outper-forms traditional machine learning techniques. It runsthe applications on real mobile phones so that moreaccuracy can be achieved. It used the Dynalog dy-namic analysis framework[31]. Zhongru Ren[10] pro-posed End-to-end malware detection for android IoTdevices using deep learning. This methods resamplethe raw bytecodes of the classes.dex files of Androidapplications as input to deep learning models.Datasetcontains 8K benign applications and 8K malicious ap-plications from play store[46] and virusshare[47]andPrepare two models, first model called DexCNN with93.4% detection accuracy and the second model calledDexCRNN can achieve a detection accuracy of 95.8%.Heba Ziad Alawneh[15] proposes a dynamic malwaredetection approach for android applications. They usedata mining over process execution time and extractprocess control block(PCB) information and apply acombination of CNNs, LSTM, and DNNs on it to iden-tify the malicious application.

2.1.5 IOT and Mobile based detection

Internet of Things (IoT) devices refers to Internet-connected smart devices such as home appliances, net-work cameras, and sensors. The IoT and mobile de-vices are being used more than PCs. Since mobile andIoT devices are becoming more popular among usersday by day, they are also becoming more favorite tar-gets for attackers. Because of that the malware de-tection schema landscape is changing from comput-ers to IoT and mobile devices. Andrei Costin[19]gives the analysis of all currently known IoT malwarefamilies and uploaded this work as open-source mate-rial[59][60]. Xu Jiang[5] proposed Android MalwareDetection Using Fine-Grained Features, which usesthe permission used by application as static featureand evaluates 1700 benign applications and 1600 ma-licious applications and achieves a TP rate of 94.5%.Author claims FDP can detect more malware familiesand only requires 15.205 s to analyze one applicationon average. Moutaz[4] proposed a system for classify-

ing mobile applications using real-world datasets andapplied two feature selection methods, Chi - Squareand ANOVA with 10 supervised ML algorithms andachieved 98.1%detection accuracy with a classifica-tion time of 1.22s on an average application. LiminShen[3] proposes an application behavior- detectionmethod based on multi feature and process algebra fordetecting privilege escalation attacks in Android appli-cations. By analysis of the privilege escalation attackmodel, five features are extracted. Attack model andapplication behaviour is built using process algebra.Dataflow path detection is conducted among the ap-plications to determine those apps constituted a privi-lege escalation attack and DroidBench benchmark testis used to test the accuracy and effectiveness of theproposed method.

2.1.6 Model Checking based detection

In this detection approach, malware behaviors aremanually extracted and behavior groups are coded us-ing linear temporal logic (LTL) to display a specificfeature[1]. Program behaviors are created by lookingat the flow relationship of one or more system callsand define behaviors by using properties such as hid-ing, spreading, and injecting. By comparing these be-haviors, it is determined whether the program is mal-ware or benign. Model checking-based detection candetect some new malware to a certain degree, but can-not detect all new generations of malware. This is avery old Technique.Kinder et al. proposed a flexiblemethod to detect malicious code patterns in executa-bles by model checking [40]. They introduced thespecification language CTPL (computation tree predi-cate logic) which extends the well-known logic CTL(computation tree logic), and describes an efficientmodel checking algorithm. According to the authors,test results demonstrated that the proposed method candetect many worm variants with a single specification.Proposed method has some limitations as, It Can onlydetect worm variants, Some part of the process has tobe done manually, Performance of proposed method islow. To get better results, CTPL can be extended todetect other malware. In addition, more accurate dataintegrity constructions and efficient data structures canbe used to improve the method performance.

8

2.1.7 Cloud based detection

Cloud computing is a very growing technology andnow can be used in the malware detection field by us-ing security as a service. Users can upload any fileand get the result as it is malicious or not. Cloud hascapacity to store large dataset so it can enhance the de-tection performance of any pc or mobile by security asa service. Martignoni presented a framework that en-hances the capabilities of existing dynamic behavior-based detectors. The proposed framework enables so-phisticated behavior based analysis of suspicious pro-grams in multiple realistic and heterogeneous environ-ments in the cloud [38]. The suggested schema forcessample programs to execute in a distributed environ-ment including security labs and potential victim ma-chines. The evaluation results demonstrated that theanalysis of multiple execution traces of the same mal-ware sample in multiple end-users’ environments canimprove the results of the analysis with very smalloverhead. On the other hand, the suggested frameworkraises the privacy and security issues, and is prone tovarious forms of detection and evasion attacks. Solv-ing security related issues and implementing a resis-tant framework against evasion attacks will increasethe framework performance. Andrew McDole, Mah-moud[13] analyzes and compares various Convolu-tional Neural Networks (CNNs) for online detection ofmalware in cloud IaaS. They analyzed seven differentconvolutional neural network models and determinedwhich model is better suited for malware detection incloud IaaS. The analysis shows that the LeNet-5 modelis quick but gives less accuracy. This model gives90% accuracy and can be used in situations where in-correctness is not too costly but a quick prediction isneeded. Yanfang[36] presented a cloud-based schemato improve malware detection results by combiningthe file content and file relations and developed a fileverdict system. The system incorporated into the Co-modo’s Anti-malware products, and empirical studieswere conducted on large daily datasets collected byComodo cloud security center. The authors claims, theaccuracy and efficiency of the Valkyrie system outper-form other popular anti-malware software tools suchas Kaspersky AntiVirus and McAfee VirusScan, aswell as other alternative data mining based detectionsystems.

Figure 3: Features and Machine Learning Algorithmsfor Static Analysis.

Figure 4: Features and Machine Learning Algorithmsfor Dynamic Analysis.

2.1.8 Machine Learning Algorithms

Most detection uses different types of machine learn-ing algorithms for classification and detection. Fig-ures 3 ,4 shows the features and machine learningalgorithms for static and dynamic analysis. For themalware detection we can use any features from fig-ures 3 ,4 based on the type of file, can apply any ma-chine learning algorithm and choose the best algorithmbased upon accuracy, logloss or other performancemeasure as discussed in upcoming section 5. For ex-ample if we have an android(.apk) file and we want toclassify whether it is malicious or legitimate, we canextract permission, api call, or system call feature andcan apply any classification algorithm like SVM, Ran-dom forest etc.

The following are the factors which can affect the

9

machine learning algorithm:

1. Dataset2. Type of features3. Feature-selection algorithm used to select the

most prominent features4. Classification algorithm used to categorize apps

as malicious or clean5. Classifier’s parameter values(Hyperparameters)

3 Tools for Malware DetectionTable 1 shows some tools that are available for detec-tion and analysis.

4 Datasets for Malware DetectionSome frequently used standard datasets and datsetrepositories by researchers are listed in table 2.

5 Comparison of Malware Detec-tion Techniques

The researchers used different algorithms for malwaredetection. Most of the studies uses machine learningtechniques to identify the malware. Table 3 shows thecomparison of different studies in malware detectionwith the malware detection methods used and the ac-curacy of each studies. Tables 4 gives a good com-parison of malware detection techniques, we have dis-cussed in section 2.

5.1 Performance MeasuresSome of the matrices for identifying how good theclassification (performance measures) is are follow-ing:

• Accuracy• Precision• Recall• False positive ratio• F1 score• Area under curve(AUC)• Log loss

These matrices can be derived from:

• True positive(TP)• False positive(FP)• True negative(TN)• False negative(FN)

6 Challenges

The Followings are some challenges in malware de-tection.

1. A little information for classification.2. Comparing a few algorithms.3. High scanning/detecting time.4. High FP and FN.5. Small Datasets.6. Overfitting.7. Can not detect zero day malwares properly.8. Not detecting Hidden Malwares.9. Analysis is done on Datasets instead of Real Time

monitoring.

7 Conclusion

We have summarized 8 detection methods for mal-ware, but no method is completely able to detect allnew generation malware. Only the behaviour basedand the model checking based detection can resist theobfuscation. According to the discussion the deeplearning based and the cloud based also can detect themalware very well but they are not able to detect alltypes of malware completely. As a future work newmethod and work are needed. This paper will help tounderstand the techniques available for malware de-tection till date and it can be used as a good referencefor the further studies.

8 References1. Ö. A. Aslan and R. Samet, "A Comprehensive

Review on Malware Detection Approaches," inIEEE Access, vol. 8, pp. 6249-6271, 2020, doi:10.1109/ACCESS.2019.2963724.

2. Alzaylaee, Mohammed K., Suleiman Y. Yerima,and Sakir Sezer. "DL-Droid: Deep learning basedandroid malware detection using real devices."Computers & Security 89 (2020): 101663.

10

3. Limin Shen, Hui Li, Hongyi Wang, YihuanWang, "Multifeature-Based Behavior of Privi-lege Escalation Attack Detection Method for An-droid Applications", Mobile Information Sys-tems, vol. 2020, Article ID 3407437, 16 pages,2020. https://doi.org/10.1155/2020/3407437

4. Alazab, Moutaz. "Automated Malware Detectionin Mobile App Stores Based on Robust FeatureGeneration." Electronics 9.3 (2020): 435.

5. Xu Jiang, Baolei Mao, Jun Guan, XingliHuang, "Android Malware Detection Using Fine-Grained Features", Scientific Programming, vol.2020, Article ID 5190138, 13 pages, 2020.https://doi.org/10.1155/2020/5190138

6. Alazab, Moutaz, et al. "Intelligent mobile mal-ware detection using permission requests and apicalls." Future Generation Computer Systems 107(2020): 509-521.

7. Y. A. Ahmed, B. Koçer and B. A. S. Al-rimy,"Automated Analysis Approach for the Detectionof High Survivable Ransomware," KSII Trans-actions on Internet and Information Systems,vol. 14, no. 5, pp. 2236-2257, 2020. DOI:10.3837/tiis.2020.05.021

8. Ali, Abdullah; Eshete, Birhanu; (2020). Best-Effort Adversarial Approximation of Black-Box Malware Classifiers. arXiv preprintarXiv:2006.15725

9. Alper Egitmen, Irfan Bulut, R. Can Ay-gun, A. Bilge Gunduz, Omer Seyrekbasan, A.Gokhan Yavuz, "Combat Mobile Evasive Mal-ware via Skip-Gram-Based Malware Detection",Security and Communication Networks, vol.2020, Article ID 6726147, 10 pages, 2020.https://doi.org/10.1155/2020/6726147

10. Ren, Zhongru & Wu, Haomin & Ning, Qian &Hussain, Iftikhar & Chen, Bingcai. (2020). End-to-end Malware Detection for Android IoT De-vices Using Deep Learning. Ad Hoc Networks.101. 102098. 10.1016/j.adhoc.2020.102098.

11. Sudhakar, Kumar, S. An emerging threatFileless malware: a survey and researchchallenges. Cybersecur 3, 1 (2020).https://doi.org/10.1186/s42400-019-0043-x

12. Talukder, Sajedul. (2020). Tools and Tech-niques for Malware Detection and Analy-sis.https://arxiv.org/abs/2002.06819

13. McDole, Andrew, et al. "Analyzing CNN BasedBehavioural Malware Detection Techniques on

Cloud IaaS." arXiv preprint arXiv:2002.06383(2020).

14. Berman, D.S.; Buczak, A.L.; Chavis, J.S.; Cor-bett, C.L. A Survey of Deep Learning Methodsfor Cyber Security. Information 2019, 10, 122.

15. Fei Xiao, Zhaowen Lin, Yi Sun, Yan Ma, "Mal-ware Detection Based on Deep Learning of Be-havior Graphs", Mathematical Problems in Engi-neering, vol. 2019, Article ID 8195395, 10 pages,2019. https://doi.org/10.1155/2019/8195395

16. lawneh, Heba & Umphress, David & Skjellum,Anthony. (2019). Android Malware DetectionUsing Neural Networks & Process Control BlockInformation.

17. Rahim Taheri, Meysam Ghahramani, RezaJavidan, Mohammad Shojafar, Zahra Poora-nian, Mauro Conti,Similarity-based Androidmalware detection using Hamming distanceof static binary features, Future Genera-tion Computer Systems, Volume 105, 2020,https://doi.org/10.1016/j.future.2019.11.034

18. Kurnaz, S. and Mokhalad Eesee Khudhur. “Com-parative and Analysis Study for Malicious Ex-ecutable by Using Various Classification Algo-rithms.” (2018).

19. Costin, Andrei. “IoT Malware : ComprehensiveSurvey , Analysis Framework and Case Studies.”(2018).

20. S. Anderson and P. Roth, “EMBER: Anopen dataset for training static PE mal-ware machine learning models,” 2018https://arxiv.org/abs/1804.04637

21. Pandey, Anjana. (2018). A STUDY ONDIGITAL FORENSICS USING VARIOUSALGORITHMS FOR MALWARE DETEC-TION. International Journal of AdvancedResearch in Computer Science. 9. 85-89.10.26483/ijarcs.v9i3.6084.

22. Amro, Belal. (2017). Malware Detection Tech-niques for Mobile Devices. International Journalof Mobile Network Communications & Telemat-ics. 7. 10.5121/ijmnct.2017.7601.

23. Nikola Milosevic a , Ali Dehghantanha b andKim-Kwang Raymond Choo, "Machine learningaided Android malware classification," Comput-ers & Electrical Engineering, vol. 61, pp. 266-274, 2017

24. A. H. Lashkari, A. F. A. Kadir, H. Gonzalez, K.F. Mbah, and A. A. Ghorbani, “Towards a net-

11

work–based framework for Android malware de-tection and characterization,” in Proc. 15th Annu.Conf.Privacy,Secur. Trust (PST), Aug. 2017

25. Bat-Erdene M, Park H, Li H, Lee H, ChoiMS (2017) Entropy analysis to classifyunknown packing algorithms for malwaredetection.Int J Inf Secure 16(3):227–248.https://doi.org/10.1007/s10207-016-0330-4

26. Narayanan A, Chandramohan M, Chen L,Liu Y (2017) A multi-view context-aware ap-proach to Android malware detection and ma-licious code localization. Empir Softw Eng.https://doi.org/10.1007/s10664-017-9539-8

27. Alam S, Qu Z, Riley R, Chen Y, RastogiV (2017) DroidNative: automating and opti-mizing detection of Android native code mal-ware variants. Comput Secur 65:230–246.https://doi.org/10.1016/j.cose.2016.11.011

28. Bhattacharya, Abhishek & Goswami, Radha.(2016). DMDAM: Data Mining Based Detectionof Android Malware.10.1007/978-981-10-2035-3\_20.

29. Aslan, Ömer. (2017). Performance Compari-son of Static Malware Analysis Tools Versus An-tivirus Scanners To Detect Malware.

30. Ethan M. Rudd, Andras Rozsa, Manuel Gün-ther, and Terrance E. Boult.A Survey of StealthMalware: Attacks, Mitigation Measures, andSteps Toward Autonomous Open World Solu-tions.https://arxiv.org/abs/1603.06028v2.

31. Alzaylaee, M.K., Yerima, S.Y., and Sezer, S.,2016. Dynalog: an automated dynamic analy-sis framework for characterizing android applica-tions. In: 2016 International Conference On Cy-ber Security And Protection Of Digital Services(Cyber Security), pp. 1–8. doi: 10.1109/Cyber-SecPODS.2016.7502337 .

32. S. Alam, R. Horspool, I. Traore, and I.Sogukpinar, “A framework for metamorphic mal-ware analysis and real-time detection,” Comput.Secur., vol. 48, pp. 212–233, Feb. 2015

33. S. K. Pandey and B. M. Mehtre, "Performanceof malware detection tools: A comparison,"2014 IEEE International Conference on Ad-vanced Communications, Control and ComputingTechnologies, Ramanathapuram, 2014, pp. 1811-1817, doi: 10.1109/ICACCCT.2014.7019422.

34. Jyoti Landage, Prof. M. P. Wankhade, 2013,Malware and Malware Detection Techniques : A

Survey, INTERNATIONAL JOURNAL OF EN-GINEERING RESEARCH & TECHNOLOGY(IJERT) Volume 02, Issue 12 (December 2013),

35. Z. Bazrafshan, H. Hashemi, S. M. H. Fard and A.Hamzeh, "A survey on heuristic malware detec-tion techniques," The 5th Conference on Informa-tion and Knowledge Technology, Shiraz, 2013,pp. 113-120, doi: 10.1109/IKT.2013.6620049

36. Y. Ye, T. Li, S. Zhu, W. Zhuang, E. Tas, U. Gupta,and M. Abdulhayoglu, “Combining file contentand file relations for cloud based malware detec-tion,” in Proc. 17th ACM SIGKDD Int. Conf.Knowl. Discovery Data Mining (KDD), 2011

37. kdd(M. Tavallaee, “A detailed analysis of theKDD CUP 99 data set,” in Proc.IEEE Symp.Comput. Intell. Secur. Defense Appl., 2009, pp.1–6.)

38. L. Martignoni, R. Paleari, and D. Bruschi, “Aframework for behavior based malware analysisin the cloud,” in Proc. Int. Conf. Inf. Syst. Secur.Berlin, Germany: Springer, 2009

39. Z. Zuo, Q. Zhu, and M. Zhou, “On the time com-plexity of computer viruses,” IEEE Trans. Inf.Theory, vol. 51, no. 8, pp. 2962–2966,Aug. 2005

40. J. Kinder, S. Katzenbeisser, C. Schallhart, andH. Veith, “Detecting malicious code by modelchecking,” in Proc. Int. Conf. DetectionIntrusions Malware, Vulnerability Assessment.Berlin, Germany: Springer, 2005

41. Gergely Erdelyi, "Hide’n’SeekAnatomy of stealth malware”https://www.blackhat.com/presentations/bh-europe-04/bh-eu-04-erdelyi/bh-eu-04-erdelyi.pdf

42. D. Spinellis, “Reliable identification of bounded-length viruses is NP-complete,” IEEE Trans. Inf.Theory, vol. 49, no. 1, pp. 280–284,Jan. 2003

43. https://www.csoonline.com/article/2615925/security-your-quick-guide-to-malware-types.html

44. https://www.lastline.com/understanding-advanced-threat-malware-detection

45. https://www.techopedia.com/definition/4130/stealth-virus

46. https://play.google.com/47. https://virusshare.com/48. https://www.virustotal.com.49. http://www.malgenomeproject.org/50. https://www.comodo.com/home/internet-

security/updates/vdp/database.php51. http://contagiodump.blogspot.com/

12

52. https://app.sndbox.com/53. https://www.sec.cs.tu-bs.de/ danarp/drebin/54. http://nlp.cs.aueb.gr/software_and_datasets/Enron-

Spam/index.html55. https://spamassassin.apache.org/56. http://arxiv.org/abs/1802.10135

57. https://kilthub.cmu.edu/articles/dataset/Insider_Threat_Test_Dataset/12841247/1

58. https://androzoo.uni.lu/59. http://firmware.re/malw60. http://firmware.re/bh18us

13

Table 1: Tools for malware detectionTool Type Tool Name Description

DetectionTools

Analyse PE Wrapper for a variety of tools for reporting on window PE files.CHKrootkit Linux rootkit detector.MASTIFF Static analysis Framework.MultiScanner Modular file scanner.PEV for analysis of suspicious binar

OnlineScannerandSandbox

Andro Total Online analysis of APKs against multiple mobile antivirus apps.APK Analyzer Dynamic analysis of APKs.Cuckoo sandbox Open source sandbox and automated analysis system.Deepviz Multiformat file analyzer with machine learning classifier.

StaticAnalysisTools

PEid Detects most common packers, crypters and compilers for PE files.

Resource Hacker Used to add, modify or replace most resources within Windows binaries includingstrings, images, dialogs, menus, VersionInfo and Manifest resources.

Dependency walker List the imported and exported functions of a PE file. It also displays a recursivetree of all the dependencies of the executable file.

PEView. Provides a quick and easy way to view the structure and content of 32-bit PE andCOFF files.

apktool Decompile the applications.

IDA PRO Disassembler to generates assembly language source code from machine-executable code.

DynamicAnalysisTools

Rogshot Capture a snapshot of the system prior to executing malware and then immediatelyafterwards.

Process Explorer. Give real-time system information about the running process.Process monitor Realtime troubleshooting tool.Immunity debugger Write exploits, analyze malware, and reverse engineer binary files.

ollyDbg Traces registers, recognizes procedures, API calls, switches, tables, constants andstrings, as well as locates routines from object files and libraries.

Table 2: Datasets for malware detectionSr.No. Dataset Description

1. Knowledge discovery and dissemination(KDD) 199 dataset[37] Approximately 4,900,000 single connection vectors, each contains41 features.

2. Genome Project dataset[49] 1,200 malware samples.3. Virusshare[46] 106,555 bytes different datasets for various malware families.

4. VirusTotal dataset[48] Provide 70 antivirus scanner and URL/Domain blacklisting ser-vices. Need to upload files to check if it is malicious or not.

5. Comodo dataset[50] 79666064 file till 2/12/20, Updates in every 2 days.6. Contigeio dataset[51] 189 malware samples.7. DREBIN dataset[54] 5,560 applications from 179 different malware families.

8. Microsoft dataset[56] Dataset is almost half a terabyte, malware files representing a mixof 9 different families.

9. CERT insider threat dataset v6.2[5][57] Contain multiple dataset over 83 gb files.

10 EnronSpam[55] 30207 emails of which 16545 emails are labeled as ham and 13662emails are labeled as spam.

11 SpamAssassin[55] 6047 messages, with 31% spam ratio.

12. LingSpam[14] Open Source(for Email Spam Check) 2,893 spam and non-spammessages.

13. SNDBOX[53] Free open source (200 MB of files).

14. Ember[20] 500MB, consisting of disassembly and byte- code of around 20Kmalicious samples from nine families.(Open source).

15. Androzoo[58]Contains 13,996,153 different APKs, each of which has been anal-ysed by tens of different AntiVirus products to know which appli-cations are detected as Malware.

14

Table 3: Comparison of malware detection StudiesS.No. Name Method Performance Measures

1. Static analysis tools vs antivirus scanner[29] Signature based For static -max 68.2%For antivirus scanner-max 58.9%

2. Digital Forensic- comparing different algo-rithm[21] Machine Learning

FPR- 0.417%FNP - 0.716802%GNB - 70.18%DecisionTree-99.11%R.Forest - 99.4929%AdaBoost- 98.4534%G.Boosting-98.801%

3. DL-Droid Deep learning framework[2] Deep Learning 97.8% (with dynamic features only)99.6% (with dynamic + static features)

4. End-to-end Md for android IOT[10] proposed 2models Deep Learning DexCNN -93.4%

DexCRNN - 95.8%.

5. Analyzing CNN Based Behavioural Malware De-tection Techniques on Cloud IaaS[13] Cloud based detection

LeNet5 - 89.9%ResNet50 - 90.7%ResNet101 - 87.0%ResNet152 - 88.7%DenseNet121 - 92.1%DenseNet169 - 91.9%DenseNet201 - 91.5

6.Comparative and Analysis Study for MaliciousExecutable by Using Various Classification Algo-rithms[18]

Machine Learning

SVM - 96.12%KNN- 97.87%Ho-effding Tree- 94.5%Random forest-98.12%

7. A framework for metamorphic malware analysisand real-time detection[32]

ACFG (Annotated Control Flow Graph) andSWOD (Sliding Window of Difference and CFG)- CFWeight

In range 94% - 99.6%

8. Android Malware Detection Using Fine-GrainedFeatures[5] Machine learning TP - 94.5%, 15.205 s time on an average applica-

tion.

9. Automated Malware Detection in Mobile AppStores Based on Robust Feature Generation[4] IOT and Machine Learning

Detection Accuracy - 98.1%Classification time- 1.22s on an average applica-tion.

10. Intelligent mobile malware detection using per-mission requests and API calls.[6] Mobile based detection F-measure - 94.31%

11. Automated Analysis Approach for the Detectionof High Survivable Ransomware.[7] Machine Learning and Deep learning based ROC curve of 0.987

F Rate - 0.007

12. Black-Box Malware Classifiers [8] Deep Learning based and machine learning

LGBM -0.88Decision Tree - 0.85Random Forest - 0.87KNN - 0.91

13. Combat Mobile Evasive Malware via Skip-Gram-Based Malware Detection[9] Deep Learning based and machine learning

RF - 95.64% on entire dataset, 95% on evasiveonly samples.Svm-81.94%Decision Tree- 92.04%Random Subspace - 94.86%KNN - 94.48%For test set containing only zero day without in-cluding them in training set RF- 37.36%

14. Malware Detection Based on Deep Learning ofBehavior Graphs[16] Deep Learning based

Precision- 0.986Recall - 0.992F1-Score - 0.989

15.Similarity-based Android malware detection us-ing Hamming distance of static binary fea-tures[17]

Deep Learning basedFor all algorithm and datasets accuracy is greaterthan 90% and and in some cases (i.e., consideringAPI features) are more than 99%.

16. DroidNative: Automating and optimizing detec-tion of Android native code malware variants[27] Mobile based detection rate (DR) - 93.57%

false positive rate - 2.7%

17. Machine learning aided malware classification ofAndroid applications [23] Mobile based F-score - 95.1%

18. Network-Based Framework for Android MalwareDetection [24] Mobile based accuracy - 91.41%

false positive - 0.085

19. Entropy analysis to classify unknown packing al-gorithms for malware detection[25] Machine Learning based Accuracy of - 95.35%

precision - 94.13%

20.A Multi-view Context-aware Approach to An-droid Malware Detection and Malicious Code Lo-calization[26]

Mobile based and machine based average recall - 94%

21. DMDAM: Data Mining Based Detection of An-droid Malware[28] Mobile and machine learning based

TPR rate - 96.70% ,Accuracy is up to 77.13%,Highest F1 score is 0.8583.

15

Table 4: Comparision of Malware Detection Techniques

Malware Detection TechniqueDetectUnknownMalware

Resistant toObfucation

Well-KnownApproach

New Ap-proach

Signature Based × ×√

×Behavior Based

√ √ √×

Heuristic Based√

×√

×Model Checking Based

√ √ √×

Deep Learning Based√

× ×√

Cloud Based√

× ×√

Mobile Based Detection√

× ×√

IOT Based Detection√

× ×√

16