an attack-resilient architecture for the internet of...

3940 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020

An Attack-Resilient Architecturefor the Internet of Things

Hussain M. J. Almohri , Member, IEEE, Layne T. Watson, and David Evans

Abstract— With current IoT architectures, once a single devicein a network is compromised, it can be used to disrupt thebehavior of other devices on the same network. Even thoughsystem administrators can secure critical devices in the networkusing best practices and state-of-the-art technology, a singlevulnerable device can undermine the security of the entirenetwork. The goal of this work is to limit the ability of anattacker to exploit a vulnerable device on an IoT network andfabricate deceitful messages to co-opt other devices. The approachis to limit attackers by using device proxies that are used toretransmit and control network communications. We presentan architecture that prevents deceitful messages generated bycompromised devices from affecting the rest of the network. Thedesign assumes a centralized and trustworthy machine that canobserve the behavior of all devices on the network. The centralmachine collects application layer data, as opposed to low-levelnetwork traffic, from each IoT device. The collected data isused to train models that capture the normal behavior of eachindividual IoT device. The normal behavioral data is then usedto monitor the IoT devices and detect anomalous behavior. Thispaper reports on our experiments using both a binary classifierand a density-based clustering algorithm to model benign IoTdevice behavior with a realistic test-bed, designed to capturenormal behavior in an IoT-monitored environment. Results fromthe IoT testbed show that both the classifier and the clusteringalgorithms are promising and encourage the use of application-level data for detecting compromised IoT devices.

Index Terms— Internet of Things, intrusion detection, unsu-pervised learning, network security.

I. INTRODUCTION

THE Internet of Things (IoT) connects physical devicesto the Internet. Applications deployed on the Internet of

Things include devices that are capable of monitoring andcapturing events, making decisions according to specificationsset by vendors or system administrators, and coordinating

Manuscript received December 9, 2019; revised March 29, 2020 andMay 6, 2020; accepted May 7, 2020. Date of publication May 14, 2020;date of current version July 17, 2020. This work was supported by KuwaitUniversity under Project RQ01/18 and by the National Science Founda-tion SaTC Program under Grant 1804603. The associate editor coordinat-ing the review of this manuscript and approving it for publication wasDr. Eduard A. Jorswieck. (Corresponding author: Hussain M. J. Almohri.)

Hussain M. J. Almohri is with the Department of Computer Science, KuwaitUniversity, Safat 13060, Kuwait (e-mail: [email protected]).

Layne T. Watson is with the Department of Computer Science, VirginiaPolytechnic Institute and State University, Blacksburg, VA 24061 USA, alsowith the Department of Mathematics, Virginia Polytechnic Institute and StateUniversity, Blacksburg, VA 24061 USA, and also with the Department ofAerospace and Ocean Engineering, Virginia Polytechnic Institute and StateUniversity, Blacksburg, VA 24061 USA (e-mail: [email protected]).

David Evans is with the Department of Computer Science, University ofVirginia, Charlottesville, VA 22903 USA (e-mail: [email protected]).

Digital Object Identifier 10.1109/TIFS.2020.2994777

with other devices to apply application-specific policies. IoTdevice vendors typically use conventional open-source soft-ware, mainly modified to suit the purpose of IoT applications,to operate the devices. For example, Amazon FreeRTOS [1],Android Things [2], Raspbian [3], Embedded Linux [4], andRiot [5] are among the growing number of modifiable andopen-source IoT operating systems.

Open-source software modules, although widely deployedand well-tested, often have known and zero-day vulnerabilitieswhich attackers can exploit to achieve unrestricted remotecode execution. For example, in 2019, embedded Linux wasreported to have a root account vulnerability [6]. Such vulner-abilities may endure on many devices as system administratorsmiss the opportunity to patch their devices promptly. Attackerscan perform reconnaissance operations on the target network todiscover the type of software used and find remote exploitationvulnerabilities [7], [8]. With access to a vulnerable IoT device,the attacker can launch attacks from malicious nodes, disturb-ing trust in the network. Precisely, attackers can manipulatethe application on the compromised device to send deceitfulmessages to other devices that will cause them to behavein ways that achieve the attacker’s goals, even though thosedevices have not been compromised directly. Thus, a singleremotely compromised device can undermine the security ofthe entire IoT subnetwork.

This work addresses the problem of securing an IoT subnet-work by using application-level data generated on individualIoT subnetworks and providing an architecture for limiting andcontrolling access to the devices. Our solution aims to tacklethe security problem at the application layer and leveragesthe role of the IoT subnetwork in monitoring the securitystatus, detecting vulnerable devices and healing from attacks.IoT devices primarily exchange application data with remoteclients, which can be used to reason about the security ofthe device itself. As suggested in several previous works[9]–[17], the data generated on IoT devices can be used tolearn models for the behavior of malicious and benign devices.The hypothesis is that machine learning techniques can beutilized in an IoT subnetwork to determine the security statusof each device (secure or compromised). Once a compromiseddevice is detected, the IoT subnetwork can defend itself againstintruders by controlling the interface to IoT devices (forexample, by using virtual drivers [18]).

Several previous works provide models for analyzing datacollected from IoT network traffic. Meidan et al. pro-pose to use device white-listing by using classification [9].

1556-6013 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: University of Virginia Libraries. Downloaded on July 19,2020 at 13:00:05 UTC from IEEE Xplore. Restrictions apply.

https://orcid.org/0000-0002-7738-5864

ALMOHRI et al.: ATTACK-RESILIENT ARCHITECTURE FOR THE IoT 3941

DÏoT [10] (and other similar works, such as [11]–[17])improve anomaly detection by using probabilistic classifi-cation of network packets. Choi et al. present a sensoractivity correlation model that requires no manual labelingbut uses a bit-wise representation of normal sensor activityand activity transition probabilities as ground truth [19].Sikder et al. describe several machine learning models foranalyzing sensor-level data collected from mobile devices [20].Their work is closest to the modeling part of our work as ituses actual user behavior data in a classification model againstmalicious behavior.

In addition to data analysis, architectural solutions areneeded to secure networks against malicious devices. Barreraet al. argue that well-established models for limiting network-level activities by profiling IoT application behavior can beeffective [21]. Deadbolt uses virtual drivers to protect IoTdevices, for example, by providing encryption for networktraffic out of IoT devices [18].

We present an attack-resilient IoT architecture, whichextends the state of the art in two directions. First, the existingIoT data analysis frameworks mainly focus on network trafficdata and apply conventional anomaly detection techniques.While these solutions are useful, they miss the informationfrom the application layer. Our hypothesis is that application-level data assists in reasoning about the validity of dataexchanges across the network. Compared to miscellaneousnetwork traffic, application-level data is more vibrant andhas significantly less noise. Application-level data modelingcan be coupled with other models, such as analyzing mali-cious activities (e.g., DÏoT [10]) to provide a comprehensivedata model for preventing large-scale attacks against IoTsubnetworks. Second, previous works do not integrate dataanalysis with restricted network architecture. Restricting thenetwork architecture simplifies the data analysis to enable asimple policy system to isolate suspicious IoT devices. Thearchitecture and data model provided in this article attempt toaddress these two issues.

In our attack-resilient IoT architecture, a trusted controllermachine acts as a proxy to capture all device communication,as well as application data and metadata (such as file attributes)from all IoT devices. Security of the network depends on thatcontroller machine not being compromised, and preventinga compromised device within the network from being ableto compromise other devices. The collected data representsevents, such as motion, that are captured on individual IoTdevices. Each IoT device sends events data in the form ofevent sequences. Each event sequence represents several eventswithin a specific interval. Our model of event sequencesprovides rich metadata that can be used to guide the analysis ofanomalous behavior across the IoT subnetwork. Compromiseddevices are assumed to generate deceitful event sequencesto pollute the data collected from devices and confound thetrusted controller. Detecting compromised devices requirescontrolling the traffic to and from devices and authenticatingdata from devices. In our model, the controller authenticatesevent sequences by requiring application data to be signedusing verified device keys. As opposed to application instru-mentation proposed by Wang et al. [22], our attack resilient

Fig. 1. A simplified IoT architecture with a cloud-assisted access controlpolicy enforcement. An attacker subnetwork can exploit surveillance devicesand cause undesired incidents.

IoT architecture uses a simple message signing protocol forauthenticating application data.

In summary, the contributions of this work are:1) an attack-resilient IoT architecture in which a centralized

controller dynamically controls and manages IoT deviceinteractions, to provide security for the IoT network evenwhen some devices are vulnerable to remote exploitationattacks (Section III),

2) a model of IoT application data and a data provenanceverification method based on a simple message signingprotocol (Section IV-A),

3) two anomaly detection models, based on classificationand clustering (Section IV-B), and

4) a prototype implementation (Section VI) and experi-mental evaluation using an IoT testbed (Section VII).Evaluation results show that on average 76% of theattacker’s messages originating from a compromiseddevice can be detected. The best success rate of ourmodels was 91% on a 30-day dataset collected from anactual IoT environment.

Next, we present the necessary background and related workon securing IoT subnetworks.

II. BACKGROUND AND RELATED WORK

This section presents background on the Internet of Things,followed by a discussion of the related work.

A. Internet of Things

We adopt a practical definition of the Internet of Things(IoT), among the existing many [23], that characterizes targetIoT networks. We define things as devices that are designedto be connected to the Internet with full or semi-automatedinteractions with external entities (devices or humans). Thesethings are interchangeably referred to as devices or nodesthroughout this article. Because of Internet connectivity, IoTdevices enable a range of applications, for example, providingreal-time monitoring for human operators outside the targetnetwork. An IoT subnetwork is formed by a set of collaborativedevices that are interconnected to serve a specific purpose.

IoT subnetworks often use a centralized control engineentity that interacts with all IoT devices (Figure 1). The controlengine is responsible for data collection from the devices,



enforcing access control policies, and enable interaction withclient applications. Some IoT subnetworks allow direct clientapplication access to individual devices. Alternatively, com-munication with an IoT device can be restricted to designateddevices. Many IoT subnetworks are set up with assistance fromcloud computing providers. For example, the AWS IoT ThingsGraph [24] provides cloud assistance for connecting variousIoT subnetworks, enabling improved collaboration.

1) IoT Security Challenges: IoT subnetworks face severalsecurity challenges. First, IoT subnetworks rely on the securityof individual devices and the software components that executeon these devices. These devices are often developed withsevere cost pressures and time-to-market requirements thatresult is failure to follow known best security practices intheir implementations. Attackers who can find vulnerabledevices may be able to compromise the entire IoT subnet-work. Attackers can use the compromised subnetwork toinject false data into other devices, and thus compromisethe overriding application. In the example IoT environmentof Figure 1, an attacker-controlled neighboring subnetworkattempts to gain access to critical IoT infrastructure, thesurveillance cameras, to inject false data and influence accesscontrol policies employed by the central IoT control engine.Second, IoT subnetworks can be large, comprising hundredsof nodes. With large interconnected subnetworks, maintainingsecurity is challenging. A system of device interaction isrequired to control message exchanges across IoT devices.Third, IoT devices may occasionally be set as part of a fogcomputing [25] environment. IoT subnetworks that participatein a fog allow a portion of the computation to be carried onindividual IoT devices. Heavier computations are offloaded onthe cloud. This requires continuous interaction of various IoTsubnetworks, effectively increasing exposure to remote attacks.

B. Related Work

Numerous works have considered various strategies forsecuring IoT subnetworks. The main approaches use machinelearning models, blockchain-based access control, and pro-gram analysis.

1) Machine Learning: Many existing works employmachine learning techniques for IoT device identification.Device identification is useful and can aid in securing IoTsubnetworks. However, there is much to be done in terms ofdetecting security breaches from active attackers that compro-mise selected vulnerable IoT devices.

Classifying network data is used in a model proposed byMeidan et al. [9]. In this model, devices are identified withthe assistance of manually labeled data. Based on a trainedclassifier, devices are allowed access in an IoT subnetworkif the device type is white-listed. Nguyen et al. developedDÏoT [10], which uses Gated Recurrent Units (GRU) foranalyzing network-level data for anomaly detection in IoTdevices. The proposed model maps packets to symbols withprobabilities calculated by the GRU. The probabilities belowa threshold are marked as anomalies. In this work, we followa similar approach by using probabilistic reasoning to detectanomalies. However, our anomaly detection model differs inthat it uses application-level data and meta as opposed to

network packets, it uses both classification and clusteringto learn a model of benign behavior, and detects vulnerabledevices by inspecting the signatures on application data. Notethat IoT devices are required to supply their correspondingdevice proxies with signed checksums of the generated data.

Wang et al. [22] and Choi et al. [19] use formal graph-basedapproaches for detecting faults and security breaches in IoTdevices. Wang et al. [22] use a form of a directed acyclic graph(DAG) to represent the interactions of various entities andevents across an IoT platform. Their main concern is designinga code instrumentation mechanism to implement the DAG andcollect provenance data to answer questions such as “How dida device turn on at a specific time?” Choi et al. [19] also usemachine learning for detecting deviations in data transmissionsacross sensors and actuators. The data is grouped using corre-lation groups, where the groups are checked against real-timeactivities. The scope of event sequences in the attack resilientIoT architecture (Section III-C), however, is more general andincludes security-sensitive events that are carried by devices.Serror et al. [26] use network filtering and device behavioralrules to detect anomalies in IoT devices. As specifying rulescan be practically a limiting factor, we propose to use machinelearning with minimal user intervention to harden an IoTsubnetwork against remote attackers.

2) Blockchain: Some of the previous works proposed theuse of the block-chain technology for developing access con-trol in vast IoT networks [27]–[35]. Blockchain technologiesuse cryptographic methods for providing decentralized veri-fication across a distrusting network of nodes. One way tosecure IoT subnetworks is to use blockchain as an accesscontrol scheme [28], [31]–[33], [36]. An IoT network canmaintain a cryptographically verifiable blockchain to controlaccess to any group of nodes in the network. For example,the blockchain provides access control verification for nodeswith long-term membership in the network (such as manage-ment nodes). In this work, we do not use the blockchaintechnology because of the increasing cost of verifying achain. Instead, we use short chains of signatures to prove theprovenance of IoT messages across a small IoT subnetwork.

3) Program Analysis: Other approaches use program andinformation flow analysis for ensuring IoT security. Soteriaproposes to use static analysis and model checking tomake sure if an IoT app enforces security policies correctly[37]–[41]. IoT-Guard [38], and similar systems such as [42]and [43], use program instrumentation in IoT applicationsto detect flawed applications and enforce appropriate secu-rity policies. Balliu et al. developed a semantics frameworkto reason about program interactions for IoT systems [44].Program analysis can complement a data-oriented model bysecuring vulnerable software components that are used tooperate IoT devices. While this approach is valuable, we focuson using data to analyze IoT device behavior. Thus, we assumethat vulnerable software components are inevitable and can beexploited remotely.

III. MODEL AND ARCHITECTURE

This section starts by presenting the threat model(Section III-A), an overview of the architecture (Section III-B),



and a detailed design description of the introduced components(Section III-C).

A. Threat Model

Several components are involved from both the defense(i.e., the target IoT subnetwork) and the attack fronts. Thedefense front is an IoT subnetwork that includes a trustedcomputing base, the physical IoT devices and their installedsoftware, and the network that connects the IoT devices. TheIoT subnetwork serves an application that requires several IoTdevices to cooperate and supply data. The system administra-tors (who own and manage the defense front) specify policyrules for the IoT subnetwork, which are applied according todata received from IoT devices. The attacking front includessoftware for remotely collecting reconnaissance data about thetarget IoT subnetwork.

The attacker can use the reconnaissance data to locate anavailable exploit and launch a remote attack. If the attackercompromises an IoT device and modifies messages sent fromthe device or redirects messages to an attacker-controlleddevice, the attack has succeeded. If the attack fails, the attackerattempts a new offense until the attack succeeds or theattacker’s patience or resources are exhausted.

The trusted computing base consists of two components:(1) a controller, and (2) a network of device proxies. Thecontroller is a system that is assumed to be secure againstknown attacks, and regularly monitored for patches and secu-rity updates. The controller includes software components andstorage for collecting data from the IoT subnetwork. Thecontroller has full access to each IoT device and must maintainconnections with all IoT devices in the subnetwork. Thecontroller can manage security policies for the IoT application.The controller also manages firewall rules across the net-work or for individual IoT devices. The controller can deployand control device proxies. Device proxies are implementedas software components and are deployed within the networkinfrastructure of the IoT subnetwork.

B. Overview

IoT devices are exposed to vulnerabilities that enable remoteattackers to gain unauthorized privileged access. Attackers canmanipulate data generated on the compromised IoT devicesand generate and send arbitrary messages to other devices.As discussed earlier, attackers are assumed to succeed incompromising IoT devices that are directly accessible throughthe Internet. Our approach is to (1) dynamically narrow theinterface to each IoT device with a specification restrictingaccess only to the trusted computing base, and (2) detectanomalous data exchanges between IoT devices by collectingand analyzing event data that passes through the network.In this work, messages refer to data (containing events cap-tured by an IoT device), which is sent to another device in thenetwork.

We introduce an attack-resilient IoT architecture to achievethe first goal. The architecture includes a controller componentthat installs a software-based device proxy for each physicalIoT device. The device proxy does not reside on the IoT device

itself. All device proxies are deployed on a coordinating devicein the network. Each device proxy is set to interact with asingle physical IoT device. The corresponding IoT device isalso restricted to interact with the designated device proxy.A message m generated by a device Pi is sent to a deviceproxy V i

j . If the message is a response to an external request(for example, from a client application), m is forwarded to theexternal requesting client. m is also forwarded to the controllercomponent to be included as part of a device exchange dataset.For example, consider a camera Pi in a network Nk (writtenPi ∈ Nk ) that is connected to a device proxy V i

j ∈ Nk (forminga relation). Pi is set to only communicate with V i

j if the requestis outside Nk . This restriction is applied in the network firewallrules. All messages to and from Pi are relayed through andcached in V i

j .To achieve the second goal, we use an anomaly detection

system based on classification and clustering of IoT applica-tion data. We collect and analyze events data from IoT devices.An event represents the data produced from a function onan IoT device. The functions of interest are the ones thatserve the application for which the IoT subnetwork operates.For a surveillance application, an event can be a motion, asound, or physical observations such as sensing smoke inthe monitored environment. These events form the data thatis used to analyze network traffic for anomalies. We areconcerned with two problems. First, the system administratorsshould be able to distinguish fabricated deceitful messagesgenerated by compromised IoT devices. Detecting fabricateddata prevents the controller from making misguided decisions.Second, compromised IoT devices should be identified andconsidered for treatment or quarantine. Detecting and treatingweak IoT devices assists in gradual healing and securityimprovement of the overall IoT subnetwork.

Analyzing events data can be done in two ways. Systemadministrators can physically monitor the environment tocollect data, label events as benign (in case of a certainevent recorded by a benign camera) or malicious (in caseof a fabricated event generated by a compromised camera).The collected labels can be used in a supervised learningalgorithm to develop a model of the environment and detectdeviations from the model. This method requires significanthuman intervention.

Our alternative is to use application data events as the basisfor modeling device behavior, saving the need for manuallabeling. During a training phase, application data are collectedin the attack resilient IoT subnetwork. The data is used to traina binary classifier (Section IV-B). The output of the classifiercaptures the occurrence of events during daily time intervals.The normal behavior is then compared with real-time datagenerated from devices. Anomalous events are detected andused to limit devices that are suspected to be compromised.We also introduce a clustering approach to detect anomaliesin events data. Our clustering approach employs a density-based clustering algorithm (DBSCAN [45]), which is designedto isolate data points that differ from the general trend.Our experiments on a real IoT testbed (Section VII) showthat clustering application event frequencies can successfullyisolate simulated anomalies with at least 70% success rate.



Fig. 2. Implementation of the attack resilient IoT architecture using AWSEC2 instances and a modified Linux machine as the firewall router.

Using the results of the data analysis, the attack-resilient architecture can adjust policies on device proxiesto limit or completely isolate compromised devices, enablingit to quickly recover from device compromises and mitigatethe damage they can cause to other devices and the IoTapplication. While policy specification for IoT subnetworksis an important problem, we defer the discussion of precisepolicy specifications based on anomaly detection results forfuture work.

C. Attack-Resilient IoT Architecture

In this section, we define an IoT subnetwork, the controllercomponent of the attack resilient IoT architecture, the deviceproxies, and the attack resilient IoT subnetwork. The architec-ture is used in a data analysis model that is described later inSection IV-B.

1) IoT Subnetworks: An Internet of Things subnetwork is adigraph Gk = (Uk, Ek) where k is the subnetwork’s universalidentifier, distinguishing it from other subnetworks, Uk is itsset of nodes, Ek a set of edges (ordered pairs of distinct nodes).Two subnetworks Gk and Gl of G are neighbors in G if G hasan edge (u, v) ∈ (Uk×Ul)∪(Ul×Uk). A node in a subnetworkrepresents an IoT device. Nodes are assumed to be at leastconnected to one subnetwork, through an Internet protocol.The set Uk = Vk ∪ Wk of nodes in a network may containboth proxy nodes Vk and physical nodes Wk . An IoT subnet-work serves a particular application and provides one or sev-eral interfaces for authenticated remote clients. Remoteclients, including possible attackers, are also modeled assubnetworks.

2) The Controller: The controller (Figure 2) is a softwaremodule that is hosted on a machine in an isolated subnetwork.The controller is only accessible by the system administratorsand accepts no incoming connections. The controller onlyaccepts incoming network traffic when the connection isinitiated by the controller itself. A small subnetwork Gc

includes the controller node c ∈ Vc. The subnetwork Gc isa neighbor of all subnetworks in G that include at least onedevice proxy, having permanent bidirectional connections withall device controllers in all subnetworks. The controller cancreate or destroy device proxies, receives data from deviceproxies, and stores them in an isolated datastore s ∈ Vc nodewithin the subnetwork Gc. The controller can also query thethird node a ∈ Vc that is responsible for generating a model ofevents behavior according to the data in s. Bidirectional edgesin Ec connect all the three nodes c, s, and a.

The controller receives analysis results from a, consults apolicy file, and applies the appropriate policy. The policies arerules that affect device proxies. A policy can be either

1) terminate (or launch) device proxy x ,2) connect (or disconnect) device proxy x to (or from) IoT

device d ,3) enable (or disable) remote traffic from IoT device d

connected to device proxy x , or4) limit remote access to an IoT device d connected to

proxy x to a set of remote clients.The policy file is a table of conditional statements that indicatethe application of policies according to the results of dataanalysis in a. Because the controller depends on the analysisof an anomaly detection method (described in Section IV-B),false positives can occur. Hence, the controller should onlylimit access to a suspicious IoT device. The limitation caninclude disconnecting some of the remote clients. The policyis applied through the device proxy. When the controller hashigh confidence about the suspicious device being compro-mised, a more restrictive policy can be applied. For example,the corresponding device proxy is terminated. In this case,the device can no longer interact with any other device in thesubnetwork.

3) Device Proxies: A device proxy is a software module thatis installed on a virtual machine within an IoT subnetwork Gk .The purpose of using device proxies is to isolate physicalIoT devices by narrowing their interfaces with remote clients.Also, device proxies ensure that events data, as generatedon IoT devices, are properly reported to the controller. Eachdevice proxy is represented as a node in Uk . Each deviceproxy is launched on a separate virtual machine hosted on aphysical machine (such as a router). The controller sets eachdevice proxy u to connect to a device x ∈ Ul via an edge(u, x), making Gl and Gk neighbors. Note that to isolatea device proxy from network scans and local area networkattacks (such as ARP poisoning attacks [46]), device proxiesand their corresponding physical devices are in separate IoTsubnetworks.

4) Attack-Resilient IoT Subnetwork: An attack-resilient IoTsubnetwork A is an IoT subnetwork that (1) has device proxiesconnected to every physical IoT device, that is, |VA| = |WA|,and ∀x ∈ WA, ∃u ∈ VA : (u, x) ∈ EA, and (2) contains anincomplete bipartite subgraph between the two node subsetsVA and WA, with edges E∗A ⊂ EA that only connect VAand WA. The attack resilient IoT subnetwork transmits normaldevice messages, miscellaneous network protocol messages,and specially signed messages that carry event data headedtowards the controller node. The network structure is dynamicand is managed by the controller node. Nodes and edges areadded and removed according to the policy file maintained bythe controller node (described above).

The data analyzed from individual devices (Section IV-B)are used by the attack-resilient IoT subnetwork as the basesof a defense strategy. The defense uses the model developedfrom benign sensor activities to detect anomalies in the sensordata. When an event sequence S (defined precisely below)arrives, the controller compares S with the normal data. If S isdetected to be an anomaly, the controller increments a counter



Fig. 3. Overview of the two approaches used to detect deceitful device-generated sequences. SVM classifier is the Support Vector Machine classifierand DBSCAN is the density-based clustering algorithm.

for the physical device d� from which S originated. Thecounter is used to decide if the physical device d� shouldbe restricted. When the controller decides that d� may havebeen compromised, one of the defense policies (describedearlier) can be applied. This work focuses on providing thearchitecture and the data analysis models and defers thediscussion of policy design to future work.

IV. ANALYZING IOT DEVICE DATA

In this section, we describe the data needed to recognizemisbehavior of IoT devices (Section IV-A). This data com-prises event sequences that include the events captured by IoTdevices in a time interval. After introducing events, we presenta model for analyzing event data using a binary classifierand a clustering approach (Section IV-B) as highlightedin Figure 3. A Support Vector Machine (SVM) classifier and adensity-based clustering algorithm are used to detect deceitfulsequences generated by devices.

A. Event Sequences

We are concerned with fabricated messages that are gener-ated on compromised IoT devices. The fabricated messagesinfluence the decision by the access control system thatapplies security policies to smart IoT devices (for example,locking or unlocking smart door locks). The core problemis that the access control system is unable to distinguish thefabricated messages. Our approach is to enable provenanceverification of messages across IoT subnetworks by trackingmessage origins as they are passed through each individualIoT device or device proxy. For tracking messages, we defineevents and event sequences and present a model for monitoringsequences of events throughout the network.

1) Definition of an Event: An event ai has the form(M, d, σ, α), where i is the sequence index, M is a list ofmetadata attributes for the event, d is the identifier of thedevice that generated the event, σ is the operating systemservice associated with the event, and α is the data suppliedwith the event.

Formally, an event sequence is a tuple S = (A, δ, o) whereA = (a1, a2, . . ., an) is a sequence of events in order of time,δ is the time interval for which the sequence was created, ando is the device identifier representing the original creator of thesequence. For any sequence S ∈ S = {all event sequences inan attack resilient IoT subnetwork A}, a unique combinationof (δ, o) identifies the sequence. That is, no two sequences arecreated at the same time on the same device.

2) Recording Events: The rationale for using a time-basedsequence model is the assumption that several correlatedevents occur in temporal proximity. Events at various intervalscan also be correlated. Accordingly, devices record events insequences that are only valid during δ. All physical devices anddevice proxies record, transmit and receive event sequencesacross the IoT network. Each device either creates a newsequence for each system service or receives a sequence fromanother neighboring device. The device sets a timer for eachnew sequence. When the timer expires, the device creates anew sequence and waits for events (file creations) to occur.In this model, files created for several consequent physicalmovements can be recorded in a single sequence during thelifetime of the sequence.

The relationship among events in separate sequences isestablished when analyzing the sequences (Section IV-B). Notethat the duration of each time interval δ is platform-dependentand can be arbitrary. The value of δ can significantly affectthe analysis of events as evaluated in Section VII.

3) Authenticity of Events: An attacker can generate andtransmit sequences with arbitrary origins from compromiseddevices. To prevent attackers from faking events, each devicemust sign event sequences transmitted to other devices.Assume that each device has a pair of public and secret keys(PK, SK) such that the public key PK and the device identifierare registered with the cloud controller. To prove authenticityof an event sequence S, the device sending the sequence mustsign the message containing the event sequence. The sendingdevice can append S with a message authentication code(MAC). Using a fast one-way hash function H (x), the deviceappends S with y = H (S) and signs y using its secret key:e = Enc(y)SK . The encrypted checksum, the signature e, is themessage authentication code for S. A device proxy receivesthe signed message from the corresponding IoT device andforwards it to the controller. Note that the attack resilient IoTarchitecture may destroy a device proxy and replace it with adifferent proxy. Further, an attacker may send messages froma compromised device to another IoT device and convincethe recipient device to send the message through its proxy.Preventing this attack requires authenticating the proxy thatforwards the message. Thus, when a recipient device receivesS, the device appends a new subsequence of events to S1,giving a new (3-tuple) event sequence S∗. To authenticate theadditional events in S∗, a new MAC is appended to S∗ whichincludes the previous signature e and the checksum of thenewly added subsequence: e∗ = Enc(e�H (S∗))SK∗ (whereSK∗ is the secret key for the recipient of S).

B. Event Anomalies

Our work is distinguished from previous work sincewe use application data to detect and analyze anomalies(Section III-B). In contrast with network traffic data, appli-cation data is specific and is challenging to generalize. Ourlearning model is based on the observation that (1) IoTapplication data is temporal as captured by event sequences,and (2) attackers generate fabricated events at particular times.Thus application data anomalies can be captured by isolat-ing fabricated sequences either based on the probability of



occurrence or with respect to the relationship with benigndata points. Here, we present a general definition of anomalyfollowed by two models to distinguish benign and anomalousbehaviors.

1) Anomaly Model: Let f (Si ) ∈ IR be a characterizationfunction that receives a sequence Si . The sequence Si is ananomaly if

f (Si ) ≤ τ,

where τ is a generic threshold parameter. The function f isgenerically defined to accommodate the two anomaly detectiontechniques that we use for our architecture. The rationale forusing two models is to compare the effectiveness of each onein increasing the performance of the anomaly detection processas described next.

2) Classification Approach: First, we use a binary classifier(similar to [47]) to decide if a sequence should occur duringfixed time intervals. Fixed time intervals are specified for eachsequence of events in the attack resilient IoT architecture. Theclassifier’s input has two features: an event sequence and atime interval. The output of the classifier is 1 if the datasetsupports that the event sequence is expected to occur duringthe given time interval, and 0 otherwise. A rigid anomalydetection scheme would directly use the output of the classifierto decide if an event is anomalous by setting τ = 0. Instead,we use a flexible anomaly detection scheme by using theposterior probabilities of the classifier’s output [48]. We trainthe classifier with a dataset D of benign event sequences. Thetraining results in a model M(D). In real time, the sequencesgenerated from IoT devices are labeled using the learnedmodel. To detect if a sequence is an anomaly, we set the valueof the function f to the posterior probability obtained fromthe classifier:

f (Si ) = P(y = a|Si ),

where y is the output of the classifier based on the modelM(D), and 0 ≤ a ≤ 1. The choice of a depends on theexpectation that the event occurs. Here, a low probabilitythat Si is expected to occur during a specific time intervaldetermines a low value of f . The sequence is labeled asanomaly when f falls below τ .

3) Clustering Approach: We also cluster the observedsequences using a density-based clustering approach [45]. Thereason for using density-based clustering is to isolate randombehavior by attackers in generating event sequences. In adensity-based clustering method, noise is defined as data pointsthat do not belong to any clusters. We use the definition ofnoise in density-based clustering to detect anomalies. Notethat we do not base the anomaly detection decision on dataattributes from within the IoT devices. That is because suchattributes can be fabricated on compromised devices. We alsorequire that IoT devices be memory-less. In the attack-resilientarchitecture, the IoT device communication history is onlycached on device proxies. Thus, attackers cannot learn theprevious communication patterns. As a result, the number ofevents and the time interval of sequences are arbitrary andmay appear as noise.

To form the clustering problem, a quantitative value for anevent sequence is defined. Recall from Section IV-A that foran event sequence Si , |A| is the number of events recordedfrom device d� during the interval δ. Let xi be the pair(d�, |A|/m(δ)), where m(δ) is the length of the time interval δ.Cluster first on d� (sort by physical device), then on the eventfrequencies |A|/m(δ) (hierarchical clustering).

The input X = {x1, x2, . . ., xn} to the clustering algorithmis a set of event sequence pairs for a single IoT subnetworkapplication. Assume X is collected during a time intervalwith a negligible probability of attack. The event sequencesare generated on multiple devices and a diverse set of timeintervals. When the input X is clustered, most sequences areexpected to join a cluster. The remaining noise, which isassumed to contain anomalies, is not clustered. We modify thedensity-based clustering to include additional auxiliary single-member clusters for each unclustered point. Let C denote thecluster set containing the sequence Si pair xi , computed usinga density-based clustering algorithm. Then the value of thecharacterization function f (Si ) is computed as the cardinalityof the cluster set C relative to the total number of datapoints (pairs):

f (Si ) = |C|n

.

A low value of f (Si ) either indicates that xi is noise or xi hasjoined a sparse cluster. As anomalies are assumed to be theminority events, the function f is expected to detect them.

4) Handling Cluster Noise: A challenge in detecting theanomalies using the clustering approach is with the noise in thetraining dataset of the normal behavior. To detect anomalousbehavior as noise, the already existing noise in the normalbehavior can result in a high rate of false positives. To remedythis problem, the noise from the normal behavior is separatedfrom the clustered behavior. In real time, if an event sequenceis close to a cluster, the sequence is marked as normal.Otherwise, if the sequence is noise, the controller attemptsto find a near neighbor in the noisy normal data. If a closeenough normal noise data point is detected, the sequence ismarked as normal. Otherwise, the sequence is an anomaly.

V. SECURITY ANALYSIS

In this section, the security of the proposed architecture anddata analysis model is presented. The mitigated attacks andsecurity limitations are discussed.

A. Mitigated Attacks

Remote attackers are the primary concern of this work.A remote exploitation attack vector exists because of theusability of the target IoT subnetwork. Enabling remote clientsrequires allowing remote traffic to reach the IoT devices inthe target subnetwork. Thus, the legitimate user can connectdirectly to an IoT device from any IP address using a clientapplication. Our model does not restrict the access of remoteclients. Instead, the network traffic to each IoT device isrestricted to a device proxy that is created in a neighboringsubnetwork (as virtual machines).



Although the traffic is restricted, remote attackers can gainaccess to a vulnerable IoT device. The remote access can bethrough a victim application that accepts attacker manipula-tions, for example, through payloads. A more powerful remoteaccess could be through gaining escalated privileges, allowingfor a stable connection with the victim device.

The data analysis model is used to find IoT devices thatare compromised through remote exploitation attacks. Thedata model relies on events that are generated on each IoTdevice. The attack resilient architecture requires that an eventsequence S be signed by all devices that transmit S. Theremote attacker should continue signing events even thoughthey might be fabricated. Otherwise, the controller does notaccept an unsigned S or one with an unknown key. Thissigning protocol allows the resilient architecture to check theorigin of sequences.

1) Stealth Attacker: The remote exploitation attacker canremain in stealth mode, collect data from the compromisedIoT device, and imitate the victim to attack the classifier or theclustering algorithms that are used to predict device behavior.The objective of this attack imitating the victim is to supplythe controller with fake event contents (for example, replacingthe image of an event on a camera with a fake one). This typeof attack can occur in two ways. One is to use the victimdevice itself as the storage for event behavior. The other wayis to record each normal event sequence that is generated bythe victim and transmit it back to a more powerful device thatis controlled by the attacker.

The use of device proxies allows us to mitigate the firsttype by limiting the storage of the actual IoT device to theminimum, essentially disabling the accumulation of data onthe device, and using device proxies as a cache for the data,in the case of generating many events within a short timeinterval. This approach does not eliminate the data collectionon the device completely. However, it will severely limit thecapability of the attacker to build a complete image of thevictim’s behavior.

The second type of attack is eliminated by the data analysisapproach. The advantage of the controller on the attacker isthe existence of an abundance of data from all connected IoTdevices. This data is collected in a controlled environment, forexample, by disabling remote access completely. The data,as demonstrated in the experiments of Section VII, enablesthe anomaly detection to detect if a device is suspicious ormalicious quickly. In this case, the controller instructs thedevice proxy to limit remote access to the device (for exam-ple, disabling outbound traffic). To overcome this mitigation,the attacker may avoid interfering with the activity of thedevice, allowing it to behave normally. The attacker will onlysend a copy of the collected events back to an attacker-controlled machine. This attack can be mitigated by learningthe legitimate client’s behavior or IP address and comparingthe real-time connection’s behavior to the learned behavior.Also, strict policies can mitigate the attack, for example,by limiting connection to a single client application. Furtherinvestigation of a complete solution to this problem is left forfuture work.

B. Limitations

The resilient architecture depends on the trustworthiness ofdevice proxies, and the controller. Attackers with full knowl-edge of the target environment may simulate the behaviorof IoT devices with a predictive model without transmittinganomalous data from a compromised device. The modelpresented in this work does not provide an immediate solutionto such an attack.

One important challenge in using data input from multipledevices is establishing trust amongst the devices. We assumethis can be done before the system is exposed, and that devicescan establish secure key pairs with the controller. This maynot be possible in typical deployments. One approach to estab-lishing trust is to use blockchain technologies, as proposed byFortino et al., to maintain a record that certifies the reputationcapital of each device [49]. This use of blockchain technologycan further aid the process of collecting and analyzing reliabledata from cooperating devices.

We assume that the model of benign behavior is developedduring a period that can be guaranteed to be free of attacks, butotherwise closely resembles normal use. This may be difficultto achieve in practice. Our anomaly detection scheme mayhave a decreased success rate when the benign behavior of IoTdevices differs substantially from that of the training period.This requires retraining the anomaly detection system torepresent the new behavior. Detecting the change in the benignbehavior can be challenging and requires careful considerationsince it opens the design to poisoning attacks where an attackercan inject data that corrupts the trained model.

Our model could also be extended using the concept ofcausal reasoning [50] to further enhance the detection strengthby the controller. In this scenario, an attacker with fullknowledge of a subset of the devices, potentially with physicalaccess to them, can be detected by using data from othercooperating devices. For example, a causal relationship graphcan be constructed to represent various conclusions using thesequences of events generated by our architecture. Once apattern of causal relations is learned, the attacker with a fullmimicking capability on a device A can still be detected,when a cooperating trustworthy device B generates an eventsubsequence that contradicts the one generated at A.

VI. IMPLEMENTATION

This section provides the implementation details of theattack-resilient IoT architecture. The prototype uses RaspberryPi 4 as the hardware module for IoT devices. The requiredsoftware modifications are minimal and require no changesto the operating system kernel or the network software stack.Apache web server, MySQL database, and AWS Elastic CloudComputing (EC2) are used to develop the required softwarecomponents. The software components include a sequencemanager for the IoT device, the controller, which is imple-mented on the Amazon Web Services cloud, and a proxyservice that is installed on each device proxy. The architecturealso includes a signature store and a data store. The signaturestore is a relational database table of device IP, device MAC



Fig. 4. An implementation of the resilient architecture. The implementationuses Elastic Computing Cloud (EC2) machines in AWS. Rectangles showEC2 machines and shaded shapes are trusted components. Each IoT devicecommunicates through a distinct proxy machine. Requests are only routed toand from the designated proxy machine. SDK is a sequence signed by thedevice key DK . {SDK }P K is a sequence, which is signed by the proxy.

address, and the corresponding binary representation of thepublic key for the device. The data store is a relational databasetable with a table entry for each sequence generated.

The sequence manager receives commands from the deviceproxy, monitors a set of services on the IoT device, recordssequences, and signs each sequence and sends it to thedevice proxy. The sequence manager is a daemon processthat automatically launches when the IoT device boots. Thethreat model does not require trusting the sequence manager.When the attacker tampers with a victim device, the attackereither stops the process and prevents it from transmittingdata or leaves the sequence manager to normally transmitfabricated sequences.

The firewall rules are set and managed by the controller.A basic set of firewall rules are applied on the network routerthat connects the physical IoT devices to the Internet. Therouter restricts each IoT physical device to only communicatewith the corresponding proxy by setting the inbound andoutbound rules. The firewall rules enforce restrictions on theMAC address of the IoT device. The attacker can forge theMAC address of a victim IoT device. Forging MAC addressesis not useful since it prevents the victim from communicatingto the device proxy. The only benefit is in causing a denial ofservice, which is not the concern of this work. The router isimplemented as a regular Linux machine. The firewall rules areapplied as iptables rules. These rules are updated directlyby the controller.

Although firewall rules limit interactions with the IoTdevices, remote attackers can still find ways to exploit a targetIoT device. Suppose that a target IoT device u1 is in a subnet-work G1. A neighboring subnetwork G2 has regular machinesthat are accessible through the Internet with an open trafficrule. If a machine u2 in G2 is compromised by the attacker,then the attacker might access the subnetwork G1. u1 is behindthe firewall, but the attacker might find vulnerabilities in therouter and gain access to u1. The attacker might also gainaccess to u1 by sending malicious requests that pass throughthe corresponding device proxy. The assumption here is thatdevice proxies are trusted, but we do not trust the requests thatare processed through device proxies.

The IoT devices send sequences through the router to deviceproxies (Steps 1 and 2 in Figure 4). Each device proxy runs aweb server. Upon receiving a sequence as a request, the webserver modifies the request to include a signature and forwards

the request to the controller (Step 3). The proxy just signs thismessage with its own key, and forwards it to the controller,which verifies both signatures (Step 4). The signatures aregenerated using Python’s cryptography library [51].

The controller has a web server interface that interacts withrouters and device proxies. Device proxies send sequencesasynchronously to the controller. The controller receives eachrequest and sends the request to a temporary database table forverifying the signature. A thread pool checks the signatures onthe sequences and sends the verified signatures to the signaturestore database. A MySQL database is used to handle the storerequests. The sequences are also recorded (Step 5) to train themachine learning models.

Device proxies are created by the controller. Each deviceproxy is a virtual machine with a predefined machine imagethat has a copy of the web-based proxy as described above. Forimplementation simplicity, AWS EC2 is used to create virtualmachines for device proxies. To reduce network round triptime, virtual machines can be created internally on a VMwareESX server.

VII. EVALUATION OF ANOMALY DETECTION

We evaluate the two anomaly detection methods usingbenign data generated in an IoT testbed. The attacker datais simulated based on two attack strategies that have differentviews of the target environment. The classifier is trained on thebenign training set and is evaluated using the benign test setand the attacker test set. With the benign test set, the classifierachieves high precision and recall. With the attacker testset, the classifier has low recall in many cases (below 0.5).Depending on the behavior of the attacker and the length ofthe interval in which sequences are generated, the classifierproduces different precision and recall values. The clusteringmethod is evaluated using the percentage of noise generatedwhen clustering attack data points. The clustering algorithmhas an average success rate of detecting 76% of attacker-generated messages. In the worst case, the clustering methodmarks 70% of attacker points as noise, thereby detectingthem as anomalies. In the best case, the controller marks91% of attacker points as noise. The presented models maysuit different application and data settings and the choice ofparameters affects the final results. In practical settings, systemadministrators can adjust the system parameters to produce thebest results.

A. Data CollectionThe experimental data was collected from a testbed in two

physically separate locations: a home office and a research lab.For the home office, three sensors were installed: an officeroom, the server and storage room, and the corridor. A singlesensor was installed in the research lab. The sensors capturepictures of any moving objects. The hardware for sensorsis Raspberry Pi 4 machines with an internal camera moduleattached to the board. Each machine has four cores and 4 GBof memory and a 32 GB of SD storage card. Each sensor runsan Ubuntu distribution and the motion1 open-source libraryfor detecting motion using the camera.

1https://motion-project.github.io/



Fig. 5. Hourly frequency of motions for each sensor.

Fig. 6. Frequency histogram for each sensor with unified four-minuteintervals (m(δ) = 4). Each interval corresponds to an event sequence. They-axis shows the normalized frequency of sequences. N is the normalizedfrequency.

Sensors are labeled vi for 0 ≤ i ≤ 3 in the order mentionedpreviously. The data was collected 24 hours a day for about30 days, generating approximately 2500 data points. Someof the sensors were installed during the entire data collectionperiod and some with shorter periods of time. Because of theexperimental nature of the testbed and the changes needed inthe location of the sensors, noisy motions due to changing thelocation were detected. Also, some motions were not capturedby the sensors because they were occasionally turned off.

The activity generated on the sensors corresponds to lessthan ten people. The sensor’s view frame was facing theentrance in all locations but with arbitrary calibration. Further,the corridor sensor was not able to detect far away motion (itsrange is limited to about 15 meters). However, motion of closerobjects were instantly recorded on the sensor’s internal storagewith the file metadata capturing the exact time of the motion.The sensors also sometimes mistakenly interpreted changes inthe lighting as a movement (for example, detecting the naturallight appearing from the window).

The collected data serves as the training data for model-ing normal (benign) behavior in the monitored environment.Figure 5 shows the frequency of events within one-hour inter-vals for each sensor. The figure shows a clear pattern of activityaccording to the time of day. Figure 6 shows the frequencyhistogram of the event sequences with length ≤ 5 in four-minute intervals. The data for Figure 6 provides a key insightthat underlies the application data analysis in this work. Theassumption is that the attacker misses the expected size ofevent sequence lengths, and the event sequences provided bythe attacker are clustered as noise.

B. Design of the Experiments

The collected data is assumed to be benign with nomalicious activity. No device was compromised during the

experiments, and all the users having access to the cameraswere trustworthy. Thus, the data is used as a basis for the cat-egory of benign activities. For experimenting with fabricatedmessages, we present two attacker strategies. Each strategyuses a model to generate fabricated messages.

1) Naïve Random Attacker: The random attacker is thesimplest attacker that is assumed to have a minimal viewof the target IoT subnetwork. The attacker knows the IPaddresses of a subset of the sensors and the purpose of theapplication (e.g., surveillance). The attacker does not haveprior knowledge of the normal behavior of the sensors. Theattacker knows the type of the physical environment in whichthe IoT subnetwork is installed (e.g., a home office). Theattacker’s goal is to pollute the data collected from sensorsand influence the decisions made by a high privilege IoTsubnetwork that interacts with the subnetwork of sensors. Theattacker can gain remote access to a target physical deviceby using a vulnerable service on the target. The attacker candisable reporting a typical event sequence, modify a normalevent sequence to include less or more events, or generatefabricated event sequences when there are no actual eventscaptured by the compromised sensor.

The simulation of a random attacker tests the attackertechniques mentioned earlier. The simulation starts assuminga trained classifier and a clustered dataset with a large datasetof the benign behavior. The simulation measures the attacker’ssuccess by generating attacking sequences during 24 hours.The time at which event sequences are generated is chosenrandomly. Also, the number of events reported in a singlesequence is chosen randomly.

2) Mimicry Attacker: The mimicry attacker can leveragereconnaissance data in an attempt to mimic the behavior ofbenign nodes. This attacker uses an accurate estimation of filesize and a better estimation of sequence length. In particular,we use statistics collected from the dataset of each device tosimulate the estimated average file size associated with eachevent sequence. We also limit the random number generatorfor the attacker to the maximum and minimum sequencelength from the actual dataset. The assumption here is thatthe attacker is able to collect more reconnaissance data, forexample, by spending more time in stealth mode on thecompromised device, but is not able to build its own accuratemodel of the subnetwork so sends random data within theobserved parameters.

C. Classification Results

The results are displayed for each IoT device in the testbed(Section VII-A). The three home office devices are referred toas v0 for the office room, v1 for the server and storage room,and v2 for the corridor. The lab device is referred to as v3.

The results of the experiments show the performance ofthe anomaly detection methodology using various parameters.We use precision (positive predictive value) P and recall(sensitivity) R to measure the accuracy of the classifier:

P = Tp

Tp + Fp, R = Tp

Tp + Fn,



Fig. 7. Performance of the classifier using the training dataset compared to naïve random attacker-generated sequences. P stands for precision, and R standsfor recall. The y-axis shows the value of the metric.

TABLE I

RESULTS OF ATTACKER DATA AND TRAINING DATA CLASSIFIERACCURACY INCLUDING FILE SIZE ESTIMATION. EACH vi

REFERS TO AN IOT DEVICE IN THE TESTBED

(DEFINED EARLIER)

where T/F denote true/false and p/n denote positive/negative.We first test the relevant features that could be used to captureattacker behavior accurately. We use a support vector machine(SVM) classifier to classify the input with the output valuesY = 1 indicating an event was reported and Y = 0 indicatingno event was reported. The input includes the hour of an eventsequence, the average time difference between any two eventsin the sequence, and the average size of the event files in thesequence.

We measure the performance of both the attacker and thetraining dataset using the trained classifier. A shuffled 30% ofthe training dataset is used as a test set. The attacker-generateddataset is also used as a test dataset for the classifier. Theattacker is successful when both precision and recall valuesare high. That is, if precision and recall are high, the attackercan convince the controller that the compromised device isbehaving normally. The attacker’s benefit is in manipulatingthe contents of the event towards malicious intents. Detectingmanipulations of the event contents are out of the scope ofthis work.

1) Feature Selection: Tables I and II summarize the resultsof the classifier’s performance using the training data com-pared with data generated by the naïve random attacker.In Table I, the results are reported with the input to SVMincluding the file size averages. Here, both the precision andrecall values are high enough to provide encouragement thatthe proposed defense could be made to work in practice.In Table II, the file size averages were excluded from the clas-sifier for both the training and attacker data. Here, the attackerfails with devices v1 and v3 but has high precision with theother two while keeping a low recall. This indicates thatif file sizes are not used to train the classifier, the attackersuffers from a high false negative rate. Based on these results,we proceed with only training the metadata of events.

TABLE II

RESULTS OF ATTACKER DATA AND TRAINING DATA CLASSIFIERACCURACY EXCLUDING FILE SIZE ESTIMATION

2) Detection Accuracy: Figure 8 shows the performanceof the classifier using the training and the mimicry attacker.Recall that the mimicry attacker only speculates about thenumber of events reported within a sequence. The mimicryattacker has an accurate estimate of the minimum and themaximum number of events reported within a sequence andthe maximum and minimum values for the average distancebetween two events. These estimates are supplied directlyfrom the training dataset. The dataset for the mimicry attackermight be challenging to obtain. However, simulating thisbehavior aims to stress test the classification approach bygiving considerable advantage to the attacker.

The results show the computed precision and recall for eachdevice for both the training dataset and the attacker-generateddataset using various event sequence interval lengths. Thebars show precision and recall for both the naïve and themimicry datasets in each figure. Note that the classifier hasa high true positive and low false positive for the trainingdatasets of all devices. The attacker generated data has the bestclassifier performance in v0 and v2. Even with the achievedperformance, the attacker data classifier performance still hasa relatively low recall value. In this simulation, we also testedthe accuracy of the classifier by modifying the intervals duringwhich the sequences are captured. The x-axis shows the lengthof sequence intervals in minutes. Longer sequences showbetter attacker performance by achieving higher recall values.This is a natural consequence as longer sequences are likelyto capture the output value of Y = 1.

The naïve random attacker (with no realistic estimation ofthe input data) has a lower success rate compared to themimicry attacker (Figure 7). For example, in the dataset of thefirst IoT device, the mimicry attacker has a considerably highprecision and recall. In contrast the naïve random attacker haslow precision and recall. This indicates that the naïve random



Fig. 8. Performance of the classifier using the training dataset compared to mimicry attacker-generated sequences. P stands for precision, and R stands forrecall. The y-axis shows the value of the metric.

TABLE III

RESULT OF DENSITY-BASED CLUSTERING (CAPTURING FALSE

POSITIVES) WITH VARYING INTERVAL LENGTHS. THE

VALUES SHOW THE RATIO OF NUMBER OF NOISEPOINTS TO NUMBER OF CLUSTERED POINTS

IN THE DATASET. THE IDEAL

RATIO IS 0.0

attacker performs poorly in predicting the benign behavior ofthe compromised IoT device.

D. Clustering Results

The clustering-based anomaly detection uses the density-based clustering algorithm DBSCAN [45]. This clusteringalgorithm attempts to find core samples of high density andclusters other data points accordingly. The algorithm also findsnoise data points that do not belong to any cluster. Twoparameters influence the results of the algorithm: the radius �used to cluster points, and the minimum number m of sampleswithin the radius � to form a cluster. In our experiments,Euclidean distance was used to measure the distances.

We first tabulate the ratio of data points found to be noiseby DBSCAN in Table III. In this table, for each intervallength of m(δ), the dataset is clustered and the number ofnoise data points divided by the number of clustered points iscomputed. Ideally, the clustering algorithm should return nonoise. The chosen parameters of DBSCAN for the experimentsare � = 0.5 and m = 3. The results demonstrate the falsepositive rates. These results are subject to change based ondifferent parameter selections and different datasets.

To measure the performance of anomaly detection on thecollected dataset, we report the fraction of attacker sequencesthat are detected as anomalies. In the simulations, the thresholdT = 1/n (size of a single-point cluster over the total numberof data points) is used, which only considers the noise asanomalies. Algorithm 1 is used to simulate the naive attackerbehavior. Let X be a matrix of inputs (implemented as a list)to the clustering algorithm with each row having two columns:the interval of the sequence, and the number of events in thesequence. Let I be the set of all intervals, and fl and fh

be the minimum and maximum attacker estimations of the

number of events in all sequences. Let random(a, b) bea function that generates a random data point object with avalue in [a, b]. The function DBSCAN(X) generates a list oflabels L by executing the DBSCAN clustering algorithm onthe input set X . noise(p) marks the attacker data point pas a noise point. A label with a negative value marks a noisepoint.

Algorithm 1 Compute the Performance of Clustering-BasedAnomaly DetectionRequire: X , I , fl , fh

1: for i ∈ I do2: p← random( fl, fh)3: push(X , (i, p))4: L ← DBSCAN(X)5: l ← pop(L)6: if l < 0 then7: noise(p)8: push(P, p)9: end if

10: pop(X)11: end for12: return P

Algorithm 1 essentially reclusters each attacker-generateddata point with the rest of the training dataset. The ratiosof the number of noise points to the number of data pointsgenerated by the attacker are demonstrated in Table IV. Highervalues indicate better anomaly detection performance. Notethat the minimum achieved noise ratio is 0.7; that is, 70% ofthe attacker generated data points are marked as noise points.The performance of the mimicry attacker is slightly better thanthat of the naive random attacker for all devices in all intervals.

1) Comparison of Accuracy With Existing Works: We com-pare the detection performance of our model with6thsense [20], a context-aware IoT intrusion detectionsystem. The objective of 6thsense is to analyze user behaviorand detect intrusions with the aid of an IoT network. 6thsenseuses sensor-level detection as opposed to application-leveldetection. It finds sensor activity that is malicious as a resultof executing malware. We use the F-score to compare theperformance of our model to that of 6thsense:

F = 2× (P × R)

(P + R).



TABLE IV

RESULT OF USING DENSITY-BASED CLUSTERS TO DETECT ANOMALIES FROM THE NAIVE RANDOM AND MIMICRYRANDOM ATTACKER-GENERATED DATA. THE RESULTS SHOW THE FRACTION OF

ATTACKER SEQUENCES THAT ARE MARKED AS ANOMALIES

TABLE V

TIME TO CLUSTER ATTACK POINTS OVER THE ENTIRE DATA SETS WITH VARIOUS INTERVAL LENGTHS

The value of F ranges from 0 to 1, with F = 1 indicatingthe best value. Our model achieves an F-score of 0.74–0.94(on average, 0.87) for the random attacker, while achieving0.76–0.94 (on average, 0.86) for the mimicry attacker. Thiscompares favorably with 6thsense, which achieves (depend-ing on the model parameters) a F-score of 0.4615–0.9899(on average, 0.8040) for their Markov Chain based model. Fortheir Naive Bayes model, the F-score range is 0.7615–0.8235(on average, 0.7859). The performance of our models canpotentially be improved by using a feature-rich dataset fromthe underlying IoT sensors. Coupled with the application-leveldataset, the accuracy of the models would likely be improved.

We compared our results previous works [9], [11], [21]using the metrics from Barrera et al. [21], true positive rate(TPR) and false positive rate (FPR). TPR is identical tothe recall metric used in our work. A maximum reportedFPR of 0.1% produces a TPR of 94.01%. Unfortunately,the reported data does not enable the calculation of precision,and consequently the F-score. However, comparing the recallvalues of our work and the reported TPR, our work achieves aminimum of 0.72 and a maximum of 0.94 recall, similar to theresults reported by Barrera et al. [21]. Meidan et al. reporteda classification accuracy of 95%–99% [9]. Bai et al. [11]reported an accuracy of at least 74.8% and at most 80.1%when classifying network traffic data. These results are notdirectly comparable to our work. However, the achievedaccuracy values indicate a high possibility of detecting mali-cious traffic, similar to the one achieved by our clusteringmethod.

2) Execution Time: When detecting a deceitful attacker-generated sequence S using clustering, the execution timedepends on the speed of the clustering algorithm to find thebest-fit cluster for S. When using a classification method,the time depends on the speed of classifying S according tothe trained model. In both cases, the time required to detectif a single sequence S is benign or deceitful is negligible(running on a machine with a 6-core Intel i7 CPU and 32GB ofmemory). Here, the execution time for the clustering method is

presented since it is expected to be a more expensive operationthan the classification method. The detection time involvesfinding the cluster to which S belongs and detecting if S isa noise point (Algorithm 1). To show the time required forprocessing a number of sequences, we measured the time toprocess the attacker sequences generated for the experiments.Note that in our experiments, the simulations were conductedby dividing the data by days of the week and then by intervalsof a day, then aggregating over the days of a week. That is,each dataset measures the number of events for one intervalof a day over a week. Depending on the interval length andthe number of attack data points, the required verification timevaries. Table V shows the execution times needed to processthe entire attacker dataset for each interval. Processing a singleattacker sequence takes at most 22 milliseconds.

E. Discussion

The results for anomaly detection show the effectivenessof using application metadata with the sensor data of thetestbed. While the anomaly detection methods are not perfectin detecting fabricated attacker messages, given enough time,anomaly detection methods can detect malicious behavior fromcompromised devices. For example, the clustering approachcould detect 70% of data generated from the mimicry attackeron the device v0 (when m(δ) = 2). This behavior couldbe monitored by the controller to first limit interaction withthe device by enforcing more restrictive policies using thedevice proxy. If the behavior persists, the controller can decideto segregate the compromised device by using the devicesignature on the sequences. Also, the controller can isolatethe data generated by the compromised device since the timeat which malicious behavior was observed.

VIII. CONCLUSION

We presented an anomaly detection method using applica-tion data along with an architecture that uses device proxiesto control access to devices and collect the relevant data.



The architecture uses the results of the presented anom-aly detection method to decide which devices are compro-mised. Our results demonstrate the potential feasibility ofthe approach through an experimental case study using datagenerated from a typical IoT subnetwork environment with nospecific controls on the environments except for the locationsof the sensors. While the results are promising, many moresettings should be tested to adjust the anomaly detectionmethods better and develop a more comprehensive IoT anom-aly detection system. Our architecture could be extended toinclude a full policy engine that can directly use the resultsof the anomaly detection model to adjust policies across thenetwork. One possible application of the attack-resilient IoTarchitecture model is in enabling interactions among variousIoT devices belonging to neighboring subnetworks. In thiscase, the exchanged data in a controlled environment canbe used to construct a model of inter-device interactions.An anomaly detection method can then be used to detect andisolate malicious exchanges.

REFERENCES

[1] A. Mann. (Feb. 2020). Introducing Amazon Common Software forDevices (Preview): Accelerate Your Adoption of Amazon Device SDKs.[Online]. Available: https://aws.amazon.com/freertos/

[2] D. Smith. (May 2018). Say Hello to Android Things 1.0. [Online].Available: https://android-developers.googleblog.com/2018/05/say-hello-to-android-things-10.html

[3] S. Long. (Feb. 2020). A New Raspbian Update. [Online]. Available:https://www.raspberrypi.org/blog/a-new-raspbian-update/

[4] E. Jensen. (Apr. 2019). Ubuntu is the #1 Embedded Linux for IoT.[Online]. Available: https://ubuntu.com/blog/ubuntu-is-1-for-embedded-iot

[5] H. Will, K. Schleiser, and J. Schiller, “A real-time kernel for wirelesssensor networks employed in rescue scenarios,” in Proc. IEEE 34th Conf.Local Comput. Netw., Oct. 2009, pp. 834–841.

[6] National Vulnerability Database. (Dec. 2019). CVE-2019-18828.[Online]. Available: https://nvd.nist.gov/vuln/detail/CVE-2019-18828

[7] D. Kumar et al., “All things considered: An analysis of IoT deviceson home networks,” in Proc. 28th USENIX Conf. Secur. Symp. (SEC),Berkeley, CA, USA: USENIX Association, 2019, pp. 1169–1185.

[8] M. Frustaci, P. Pace, G. Aloi, and G. Fortino, “Evaluating criticalsecurity issues of the IoT world: Present and future challenges,” IEEEInternet Things J., vol. 5, no. 4, pp. 2483–2495, Aug. 2018.

[9] Y. Meidan et al., “Detection of unauthorized IoT devices using machinelearning techniques,” 2017, arXiv:1709.04647. [Online]. Available:https://arxiv.org/abs/1709.04647

[10] T. Duc Nguyen, S. Marchal, M. Miettinen, H. Fereidooni, N. Asokan,and A.-R. Sadeghi, “DÏoT: A federated self-learning anomaly detec-tion system for IoT,” 2018, arXiv:1804.07474. [Online]. Available:http://arxiv.org/abs/1804.07474

[11] L. Bai, L. Yao, S. S. Kanhere, X. Wang, and Z. Yang, “Automaticdevice classification from network traffic streams of Internet of Things,”in Proc. IEEE 43rd Conf. Local Comput. Netw. (LCN), Oct. 2018,pp. 1–9.

[12] A. S. Hsu, “Automatic Internet of Things device category identificationusing traffic rates,” Ph.D. dissertation, Dept. Comput. Sci., Virginia Tech,Blacksburg, VA, USA, 2019.

[13] O. Salman, I. H. Elhajj, A. Chehab, and A. Kayssi, “A machine learningbased framework for IoT device identification and abnormal trafficdetection,” Trans. Emerg. Telecommun. Technol., Sep. 2019.

[14] A. Bytes, S. Adepu, and J. Zhou, “Towards semantic sensitive featureprofiling of IoT devices,” IEEE Internet Things J., vol. 6, no. 5,pp. 8056–8064, Oct. 2019.

[15] O. Salman, L. Chaddad, I. H. Elhajj, A. Chehab, and A. Kayssi, “Pushingintelligence to the network edge,” in Proc. 5th Int. Conf. Softw. DefinedSyst. (SDS), Apr. 2018, pp. 87–92.

[16] F. Xia, H. Song, and C. Xu, “Securing the wireless environment ofIoT,” in Proc. IEEE Int. Conf. Saf. Produce Informatization (IICSPI),Dec. 2018, pp. 315–318.

[17] I. Hafeez, M. Antikainen, A. Yi Ding, and S. Tarkoma, “IoT-KEEPER: Securing IoT communications in edge networks,” 2018,arXiv:1810.08415. [Online]. Available: http://arxiv.org/abs/1810.08415

[18] R. Ko and J. Mickens, “DeadBolt: Securing IoT deploy-ments,” in Proc. Appl. Netw. Res. Workshop, Montreal, QC,Canada, 2018, pp. 50–57. [Online]. Available: https://mickens.seas.harvard.edu/files/mickens/files/deadbolt.pdf

[19] J. Choi et al., “Detecting and identifying faulty IoT devices in smarthome with context extraction,” in Proc. 48th Annu. IEEE/IFIP Int. Conf.Dependable Syst. Netw. (DSN), Jun. 2018, pp. 610–621.

[20] A. K. Sikder, H. Aksu, and A. S. Uluagac, “6thSense: A context-aware sensor-based attack detector for smart devices,” in Proc. 26thUSENIX Secur. Symp. (USENIX Secur.). Vancouver, BC, Canada:USENIX Association, Aug. 2017, pp. 397–414. [Online]. Available:https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/sikder

[21] D. Barrera, I. Molloy, and H. Huang, “IDIoT: Securing the Internetof Things like it’s 1994,” 2017, arXiv:1712.03623. [Online]. Available:http://arxiv.org/abs/1712.03623

[22] Q. Wang, W. U. Hassan, A. Bates, and C. Gunter, “Fear and logging inthe Internet of Things,” in Proc. Netw. Distrib. Syst. Secur. Symp., SanDiego, CA, USA, Feb. 2018, pp. 1–15. [Online]. Available: http://wp.internetsociety.org/ndss/wp-content/uploads/sites/25/2018/02/ndss2018_01A-2_Wang_paper.pdf

[23] R. Minerva, A. Biru, and D. Rotondi, “Towards a definition of theInternet of Things (IoT),” IEEE (IoT Initiative), New York, NY, USA,Tech. Rep., 2015. [Online]. Available: https://iot.ieee.org/definition.html

[24] AWS IoT Things Graph. (2019). Amazon Web Services IoT. [Online].Available: https://aws.amazon.com/iot-things-graph/

[25] F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, “Fog computing and itsrole in the Internet of Things,” in Proc. 1st Ed. MCC Workshop MobileCloud Comput. (MCC), New York, NY, USA: ACM, 2012, pp. 13–16,doi: 10.1145/2342509.2342513.

[26] M. Serror, M. Henze, S. Hack, M. Schuba, and K. Wehrle, “Towards in-network security for smart homes,” in Proc. 13th Int. Conf. Availability,Rel. Secur. (ARES), New York, NY, USA: ACM, 2018, pp. 18:1–18:8,doi: 10.1145/3230833.3232802.

[27] M. A. Ferrag, M. Derdour, M. Mukherjee, A. Derhab, L. Maglaras,and H. Janicke, “Blockchain technologies for the Internet of Things:Research issues and challenges,” IEEE Internet Things J., vol. 6, no. 2,pp. 2188–2204, Apr. 2019.

[28] H. Shafagh, L. Burkhalter, A. Hithnawi, and S. Duquennoy, “Towardsblockchain-based auditable storage and sharing of IoT data,” in Proc.Cloud Comput. Secur. Workshop (CCSW), New York, NY, USA: ACM,2017, pp. 45–50, doi: 10.1145/3140649.3140656.

[29] F. A. Abadi, J. Ellul, and G. Azzopardi, “The blockchain of things,beyond bitcoin: A systematic review,” in Proc. IEEE Int. Conf. InternetThings (iThings) IEEE Green Comput. Commun. (GreenCom) IEEECyber, Phys. Social Comput. (CPSCom) IEEE Smart Data (SmartData),Jul. 2018, pp. 1666–1672.

[30] S. Biswas, K. Sharif, F. Li, B. Nour, and Y. Wang, “A scalable blockchainframework for secure transactions in IoT,” IEEE Internet Things J.,vol. 6, no. 3, pp. 4650–4659, Jun. 2019.

[31] T. Jiang, H. Fang, and H. Wang, “Blockchain-based Internet of vehi-cles: Distributed network architecture and performance analysis,” IEEEInternet Things J., vol. 6, no. 3, pp. 4640–4649, Jun. 2019.

[32] J. Huang, L. Kong, G. Chen, M.-Y. Wu, X. Liu, and P. Zeng, “Towardssecure industrial IoT: Blockchain system with credit-based consensusmechanism,” IEEE Trans. Ind. Informat., vol. 15, no. 6, pp. 3680–3689,Jun. 2019.

[33] L. Bai, M. Hu, M. Liu, and J. Wang, “BPIIoT: A light-weightedblockchain-based platform for industrial IoT,” IEEE Access, vol. 7,pp. 58381–58393, 2019.

[34] S. Ding, J. Cao, C. Li, K. Fan, and H. Li, “A novel attribute-basedaccess control scheme using blockchain for IoT,” IEEE Access, vol. 7,pp. 38431–38441, 2019.

[35] O. Novo, “Scalable access management in IoT using blockchain:A performance evaluation,” IEEE Internet Things J., vol. 6, no. 3,pp. 4694–4701, Jun. 2019.

[36] A. Giaretta, S. Pepe, and N. Dragoni, “UniquID: A quest to recon-cile identity access management and the Internet of Things,” 2019,arXiv:1905.04021. [Online]. Available: http://arxiv.org/abs/1905.04021

[37] Z. B. Celik, P. McDaniel, and G. Tan, “Soteria: Automated IoT safetyand security analysis,” in Proc. USENIX Annu. Tech. Conf. (USENIXATC), 2018, pp. 147–158.


http://dx.doi.org/10.1145/2342509.2342513

http://dx.doi.org/10.1145/3230833.3232802

http://dx.doi.org/10.1145/3140649.3140656


[38] Z. B. Celik, G. Tan, and P. McDaniel, “IoTGuard: Dynamic enforcementof security and safety policy in commodity IoT,” in Proc. Netw. Distrib.Syst. Secur. Symp., 2019, pp. 1–15.

[39] Z. Berkay Celik, E. Fernandes, E. Pauley, G. Tan, and P. McDaniel,“Program analysis of commodity IoT applications for security and pri-vacy: Challenges and opportunities,” 2018, arXiv:1809.06962. [Online].Available: http://arxiv.org/abs/1809.06962

[40] D. T. Nguyen, C. Song, Z. Qian, S. V. Krishnamurthy, E. J. M. Colbert,and P. McDaniel, “IotSan: Fortifying the safety of IoT systems,” inProc. 14th Int. Conf. Emerg. Netw. EXperiments Technol., Dec. 2018,pp. 191–203.

[41] Z. B. Celik et al., “Sensitive information tracking in commodityIoT,” in Proc. 27th USENIX Secur. Symp. (USENIX Secur.), 2018,pp. 1687–1704.

[42] Z. B. Celik, E. Fernandes, E. Pauley, G. Tan, and P. McDaniel,“Program analysis of commodity IoT applications for securityand privacy: Challenges and opportunities,” ACM Comput. Surv.,vol. 52, no. 4, pp. 74:1–74:30, Aug. 2019. [Online]. Available:http://doi.acm.org/10.1145/3333501

[43] H. Chi, Q. Zeng, X. Du, and L. Luo, “PFirewall: Semantics-awarecustomizable data flow control for home automation systems,” 2019,arXiv:1910.07987. [Online]. Available: https://arxiv.org/abs/1910.07987

[44] M. Balliu, M. Merro, and M. Pasqua, “Securing cross-app interactionsin IoT platforms,” in Proc. IEEE 32nd Comput. Secur. Found. Symp.(CSF), Jun. 2019, pp. 1–6.

[45] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-basedalgorithm for discovering clusters in large spatial databases withnoise,” in Proc. 2nd Int. Conf. Knowl. Discovery Data Min-ing (KDD), AAAI Press, 1996, pp. 226–231. [Online]. Available:http://dl.acm.org/citation.cfm?id=3001460.3001507

[46] D. Bruschi, A. Ornaghi, and E. Rosti, “S-ARP: A secure addressresolution protocol,” in Proc. 19th Annu. Comput. Secur. Appl. Conf.,2003, pp. 66–74.

[47] S. Mukkamala, G. Janoski, and A. Sung, “Intrusion detection usingneural networks and support vector machines,” in Proc. Int. Joint Conf.Neural Networks. (IJCNN), vol. 2, 2002, pp. 1702–1707.

[48] J. Platt, “Probabilistic outputs for support vector machines and compar-isons to regularized likelihood methods,” in Advances in Large MarginClassifiers. Cambridge, MA, USA: MIT Press, 1999, pp. 61–74.

[49] G. Fortino, F. Messina, D. Rosaci, and G. M. L. Sarne, “Usingblockchain in a reputation-based model for grouping agents in the Inter-net of Things,” IEEE Trans. Eng. Manag., early access, Jun. 14, 2019,doi: 10.1109/TEM.2019.2918162.

[50] J. Pearl, Causality. Cambridge, U.K.: Cambridge Univ. Press, 2009.[51] A. Gaynor. (Mar. 2020). Pyca/Cryptography. [Online]. Available:

https://github.com/pyca/cryptography

Hussain M. J. Almohri (Member, IEEE) receivedthe B.S. degree in computer science from KuwaitUniversity and the Ph.D. degree in computer sciencefrom Virginia Tech in 2013. He is currently an Assis-tant Professor with the Department of ComputerScience with Kuwait University. He has co-foundeda mobile payment startup and has advised a numberof software startups in the Gulf region. His researchfocuses on systems and network security. He hasserved as a Reviewer for several IEEE and IETjournals and the Kuwait Journal of Science.

Layne T. Watson received the B.A. degree (magnacum laude) in psychology and mathematics from theUniversity of Evansville, Indiana, in 1969, and thePh.D. degree in mathematics from the University ofMichigan, Ann Arbor, in 1974.

He has worked for USNAD Crane, SandiaNational Laboratories, and General Motors ResearchLaboratories. He has served on the faculties of theUniversity of Michigan, Michigan State University,and University of Notre Dame. He is currently aProfessor with the Department of Computer Science,

Department of Mathematics, and Department of Aerospace and Ocean Engi-neering, Virginia Polytechnic Institute and State University. He has publishedwell over 300 refereed journal articles and 200 refereed conference papers.His research interests include fluid dynamics, solid mechanics, numericalanalysis, optimization, parallel computation, mathematical software, imageprocessing, and bioinformatics. He is a fellow of the National Institute ofAerospace and the International Society of Intelligent Biological Medicine.He serves as a Senior Editor for Applied Mathematics and Computationand an Associate Editor for Computational Optimization and Applications,Evolutionary Optimization, Engineering Computations, and the InternationalJournal of High Performance Computing Applications.

David Evans received the S.B., S.M., and Ph.D.degrees in computer science from MIT. He has beena Faculty Member with the University of Virginiasince 1999, where he is currently a Professor withthe Department of Computer Science. He is theauthor of an open computer science textbook, a chil-dren’s book on combinatorics and computability, anda textbook on secure multiparty computation.


http://dx.doi.org/10.1109/TEM.2019.2918162

an attack-resilient architecture for the internet of...

Documents