data representation and clustering in sensor networks ...joics.org/gallery/ics-1968_1.pdf · in...
TRANSCRIPT
Data representation and clustering in Sensor Networks
Using Unsupervised Learning Algorithm
Sathya Prakash
Racharla1,
Assistant Professor
Computer Science and
Engineering
CVR College of
Engineering [email protected]
V D S Krishna2
Assistant Professor
Computer Science and
Engineering
CVR College of
Engineering [email protected]
S. Srinivas3
Assistant Professor
Computer Science and
Engineering
CVR College of Engineering [email protected]
Avinash
Amaranayani4
Assistant Professor
Computer Science and
Engineering
CVR College of
Engineering [email protected]
Abstract Wireless sensor network (WSN) is one of the most promising technologies for some real-time
applications because of its size, cost-effective and easily deployable nature. Data clustering
and data representation are used for statistical analysis with more efficient. Machine
learning algorithms such as linear regression, SVM models, regularization methods,
reinforcement learning, and neural networks significantly reduce energy consumption. It
provides a comparative analysis of the performance of different methods to help the
designers for designing appropriate machine learning based solutions for clustering and
data aggregation applications. In this paper we analyzed different algorithms for data
representation and clustering in sensor networks. Improved performance can be observed in
unsupervised learning comparing to individual component analysis.
Index Terms—Sensor networks, Regularization, Clustering, Machine learning,
unsupervised learning, support vector machine.
I. INTRODUCTION
Wireless sensor networks as the name suggests are a class of networks where the nodes are
sensor nodes. The nodes which sense, which have the capability of sensing the physical
phenomena that occur around them. These sensing can be of different types a particular
sensor node might be able to sense temperature might be able to sense pressure they can
sense if there is any object that is moving around them sensors can also sense colors can
sense vibration occurring around can sense whether there is any sound around the sensors
and so on. Now the sensor nodes have one of the components as a sensor and these sensor
nodes collectively they form a network which is called the wireless sensor network these
wireless sensor networks are very popular currently and they have gained popularity. Since
over a decade now sensor networks are very popular because of diverse types of applications
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org917
they can be used for tracking an object in a particular terrain [1]. These can be used for
medical purposes for healthcare for space applications for agriculture and so on and so forth
there are large number of different applications of wireless sensor networks wireless sensor
networks are key to the formation of internet of things.
In particular, the designers of WSN should address the issues of reliability, clustering,
security, aggregation, localization, event scheduling, fault detection, and energy-aware
routing. Machine learning is a branch of artificial intelligence (AI) which provides the
capacity to automatically learn and refine from experience without being specifically
programmed. Machine learning concentrates on the creation of computer programs which
can access data and use to learn for themselves [10]. It has an important role in several
applications of WSN due to the following reasons:
WSN usually monitors the dynamic environments WSN may gather information
about unreachable locations in exploratory applications
Since WSN is deployed in complicated environments, it is impossible to develop an
accurate mathematical model to describe the system behavior
Due to the excessive amount of data, network designers may be unable to find the
correlations among them
Integration of WSN with new technologies such as IoT, Cyber-physical system has
been introduced; it needs a greater number of smart decision making
The application of advanced machine learning techniques in WSN has been increased
recently. Machine learning is considered as a field of themes and patterns.
Machine learning algorithms are very flexible to apply for many WSN applications. ML
algorithms are often categorized as supervised, unsupervised and reinforcement. Labeled
training dataset is provided with the supervised learning algorithm. To symbolize the
relationship between input and output, a system model is created using the data set. In case
of unsupervised learning, labeled data is not provided with the algorithm. By finding the
similarities between the input samples, it classifies the samples into different groups
(clusters). In reinforcement learning algorithm, the agent learns by means of communicating
with its surroundings. There is a need for a review of Application of ML algorithms
specifically for clustering and data aggregation since not many papers are specifically
discussing the about the Machine Learning algorithms for Clustering and data aggregation in
WSN.
The paper is organized as follows. The second section contains machine learning algorithms
which are generally used in WSN. Section 3 provides different machine learning algorithms
which are used for clustering and data aggregation. Performance analysis and comparison of
different Machine Learning methods are provided in the fourth section. Section 5 provides
an overview of proposed method and the paper is concluded with section 6.
II. MACHINE LEARNING TECHNIQUES IN WIRELESS SENSOR NETWORKS
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org918
Machine learning methods are broadly classified as supervised, unsupervised and
reinforcement learning.
A. Supervised Learning
Supervised learning has the concept of learning from examples. The model is learned by the
relationship between input and output parameters. This approach is used to resolve the
various issues of WSN such as query processing, localization, event detection, object
targeting, medium access control, and security.
B. Unsupervised Learning
In unsupervised learning, No output vectors or labels are provided with the algorithm. The
datasets are classified by finding the similarities between them. These type of algorithms
mainly used for clustering and data aggregation process in WSN. Unsupervised learning
algorithm determines the concealed relationships and it is used for addressing the problems
in WSN, where the relationship between the variables is complex. Two important algorithms
of this type are PCA (Principle Component Analysis) and K-Means clustering.
C. Reinforcement Learning
Reinforcement learning allows the agent to learn from the environment by interacting with
it. Here, sensor nodes learn to capture the best measurement in order to maximize the
advantage. The most famous reinforcement learning algorithm is Q-learning, in which every
node tries to extract measurements which are expected to increase the rewards. Sensor nodes
regularly update the rewards it achieve based on the action taken at a given state . The
diagrammatic representation of Q-learning method is shown in Fig. 1. Total rewards of
future can be computed by the equation:
𝑄(𝑆𝑡 + 1, 𝛼𝑡 + 1) = 𝑄(𝑆𝑡, 𝛼𝑡) + 𝛾(𝑟(𝑆𝑡, 𝛼𝑡) − 𝑄(𝑆𝑡 , 𝛼𝑡)) − −(1)
(,) indicates the incentive for taking the action at at state St and γ is the rate of learning,
which decides how frequently the learning happens. (Between the values 0 and 1).
Fig. 1: Visualization of Q-learning method
III. CLUSTERING AND DATA AGGREGATION
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org919
In WSN, it is difficult to transmit huge data directly to the sink. Data aggregation is the
effective solution for this problem. In data aggregation, data is collected from neighboring
sensors and the aggregator node choose them based on the aggregation model and the fused
information is sent to the base station with appropriate routing mechanism [9]. The routing
structure determines the efficiency of aggregation process. In WSN, nodes are classified into
three types in connection with aggregation: sensor nodes (sense the data), aggregator node
(performs aggregation function) and the querier node (sends the query). Aggregator node
collects data from many sensor nodes; aggregate the gathered information using aggregation
functions (e.g. COUNT, MAX, SUM, MIN) and then send the output to sink node. This
mechanism eliminates the redundancy in the collected data and improves network life time
by reducing the number of packet transmission. Fig. 2 shows the difference between data
aggregation and non-aggregation models. As the effect of aggregation process, number
Fig. 2: Aggregation and Non aggregation model
of packets and number of collisions are reduced and hence the number of retransmissions
also. Less number of retransmission solves wastage of power and time and increases the
network throughput.
Fig. 3 shows data aggregation in WSN, based on the clusters. From first cluster, the cluster
head n11 sends the value 20 to next cluster through intermediate nodes. Cluster head n1 has
a value 21 and n1 aggregate it with previous value and the aggregated value 41 is forwarded
to base station. From cluster heads n14 and n7 have the values 24 and 25 and the aggregator
combines it send as 49 to Base Station (BS). BS aggregates both the values 41 and 49 came
from four different clusters and send the forward the value as 90. Different works are
proposed for the efficient selection of cluster head and aggregating data using machine
learning techniques.
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org920
Fig. 3: Data aggregation example in a clustered architecture.
A. Location Based Distributed Spectral Clustering
The Distributed spectral clustering algorithm is used to create clusters of sensor
nodes based on their position in a sensor network [3]. For machine learning applications in
sensor networks, a collection of data at an aggregator node make it easier for attackers and
hence leads to data congestion. A strong distributed clustering technique is proposed to
avoid this problem. The algorithm is a combination of distributed eigenvector computation
and K-means clustering. For computing eigenvector of the graph Laplacian, a distributed
power iteration scheme is employed [3]. At steady state, every node meets at a value in the
eigenvector of the algebraic connectivity of the graph Laplacian. By using K-means
algorithm, clustering is done on the eigenvector. Information about the sensor’s location is
used only for creating the topology of WSN. The location based algorithm performs for
every graph structure which is connected.
B. Electing Cluster Head (CH) Using Decision Trees
Ahmed et al.[5] proposed a method based on decision tree for solving the problem of
CH selection. The method employs many important features during the iteration of input
vector by decision tree. The features are level of battery, distance to cluster centroids,
indications of vulnerability and degree of mobility. The simulation shows that the technique
increases the performance of CH elections while comparing with Low Energy Adaptive
Clustering Hierarchy (LEACH) algorithm and Analytical Hierarchy Process (AHP). There
are two phases in CH election, Startup phase and Steady phase. In the first step, a set of
nodes are selected arbitrarily as cluster heads. Enquiry message is broadcasted to all sensor
nodes in the network from the BS (Base Station). Node replies to the base station through
CH by sending the control information. After getting control information from every node,
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org921
decision tree algorithm is employed for selecting a new set of suitable cluster heads. Then it
sends the list of the Cluster Heads to every sensor nodes.
CHs broadcast the notification to all the sensor nodes. After the process, each sensor node
attaches itself with single Cluster Head. This attachment is done by estimating the RSS
(Received Signal Strength) from the Cluster Heads. As the final stage of startup phase, every
node send the request for the attachment to a particular Cluster Head and CHs broadcast the
list of cluster members to other nodes. The steady-state stage is partitioned into frames. In
every frame, nodes send their data to the Cluster Head and CHs transfer the gathered
information to the sink situated in the remote location. After every round, the role of being a
CH rotates to balance the load of working as a Cluster Head. This CH selection can be
repeated either on specified time interval or based on a threshold value in the battery.
C. Data Aggregation With Self-Organizing Map (SOM)
Self-Organizing Map (SOM) is an unsupervised learning technique to map from high
dimensional space to low dimensional space. [4] Discussed unique network architecture:
Cluster-based self-Organizing Data Aggregation (CODA). The model trains the nodes such
that those are able to classify using SOM algorithm and reduces the traffic and energy
consumption. Here, winning neuron is represented as j*, and is having a weight vector w(t).
The weight vector is very close to input vector, x(t).
𝑗∗ = 𝑎𝑟𝑔𝑚𝑖𝑛|𝑥𝑗(𝑡) − 𝑤𝑗(𝑡)| 𝑗 = 1,2, … 𝑁 − −(2)
Winning neurons can be updated as:
𝑤𝑗(𝑡 + 1) = 𝑤𝑗(𝑡) + ℎ(𝑡) (𝑥𝑗(𝑡)𝑤𝑗(𝑡)) − − − (3)
w(t) and w(t + 1) shows the neuron values at time t and t + 1. Gaussian neighborhood
function, h(t) is given:
ℎ(𝑡) = 1√2𝜋𝑗⁄ − − − (4)
Using this architecture, CODA reduces traffic and energy consumption and hence increases
the quality of data.
Visual representation of the map based on the initial random weight vector and the map after
1000 iterations is shown in Fig. 4.
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org922
Fig. 4: Map based on initial random weight vector and the map after 1000 iterations
running
D. Collaborative Data Processing Through K-Means Algorithm
Li et al. discussed the basic notions for distributed detection and tracking of single target
with WSN. From monitoring environment, information is collected by collaborative signal
processing framework. The collaborative data processing method can additionally track
several targets if it uses both K-Nearest Neighbor (KNN) and Support Vector Machine
(SVM). Surveillance system captures a huge amount of data from cameras. It requires high
computation and analysis together, and leads to the implementation of practically feasible
techniques. So Tseng et al [6] proposed iMouse which is Integrated Mobile Surveillance and
Wireless Sensor Networks. iMouse adopts high powered mobile sensors for enhancing
conventional surveillance system.
By using K-Means algorithm, this method classifies the monitored field into different
clusters. Each mobile sensor is repeatedly monitors each cluster. Since the implementation is
simple, the idea of data processing by K-Means is attractive. But still, they are vulnerable to
outliers and initial selection of seeds.
E. Role-Free Clustering With Q-Learning
Forster and Murphy discussed clustering method for WSN known as “Role-Free Clustering
with Q-Learning for Wireless Sensor Networks (CLIQUE) . In Role free clustering, each
node checks its capacity to perform as Cluster Head. Role Free Clustering has no CH
election process and in which each node checks its ability to act as a CH. CLIQUE uses Q-
Learning algorithm along with some of the active parameters like energy level. Every sensor
node is an independent learning agent, and actions are routing options using distinct
neighbors as the next hop closer to the Cluster Head. CH is described as the cluster node
with smallest routing cost to all sinks.
F. Gaussian Process Models for Censored Sensor Readings
Gaussian Process (GP) is a mixture of random variables (stochastic variables) with
parameters of mean and covariance. Ertin proposed a method to initialize probabilistic
fashion of different readings primarily based on Gaussian regression. It is a unique modeling
method to create probabilistic fashions for sensor readings. Gaussian process model for
censored sensor reading is an extended version of the Gaussian technique regression and
applies to sensor readings having continuous values. Consider censored variable as a
combination of normal and binary random variables.
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org923
Gaussian process gives natural way to combine information to determine the parameters of
the procedure. GP fro censored sensor readings represents the anisotropic character of the
propagation traits and makes use of the implicit records from the packet reception related
issues.
G. Learning Vector Quantization for Online Data Compression
Some algorithms may not require the complete knowledge about the topology of network.
Adaptive Learning Vector Quantization (ALVQ) is one such method [8]. It extracts
compressed versions of sensor readings. The use of records correlation and historical
patterns, Adaptive Learning Vector Quantization uses the LVQ method for predicting the
code-ebook by the use of beyond training samples. This algorithm reduces the bandwidth
needed for transmission and increases the recovery of the original data from the compressed
data. For code book construction, data is divided into different data pieces (DP) with same
size 𝑊(𝑊 = 𝑛1/2). For every DP, X in the training set, find the best CDP(Code Book
Piece) that can approximate it with less error. This DP is represented as CDPi and CDPi
updation is:
𝐶𝐷𝑃𝑖 = 𝐶𝐷𝑃𝑖 + 𝛼 [𝑋 − 𝑏
𝑎− 𝐶𝐷𝑃𝑖] − − − (5)
a, b are regression parameters; α is training parameter and 0<α<1.
After testing all the DPs in the training data set, code book is adjusted and transmitted to the
BS. Dynamic Bandwidth Assignment Algorithm (DBA) is introduced here. Representation
of DBA model is shown in Fig. 5.
It is mainly for balancing the qualities of sensor data transmission and compression. The
compression quality (Q1,Q2,…Qk) of every sensor is collected by the cluster head and the
assigned bandwidth for last transmission between sensors and cluster head are B1, B2,. Bk.
The computation of average quality of compression can be done by:
𝑄𝐴 = ∑𝑄𝑖
𝑘⁄
𝑖=1,,,,𝑘
− − − (6)
For later data transmission, CH assigns bandwidth to sensor i as Bi α(Qi-QA). α is adjusting
parameter for bandwidth.
I. Data Aggregation Using Principal Component Analysis (PCA)
Data Aggregation based on PCA examines based on projection and not based on eigen
vectors. It is valid to maintain certain mean squared error (NMSE) threshold in signal
recovery and decrease the energy consumption [2]. The method decreases transmissions to
the sink and one or more aggregation nodes within the sensor network. The algorithm is
implemented with less energy consumption and includes signaling and data compression for
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org924
reduced transmission. Uncompressed data is transmitted by leaf nodes and cluster head
(Aggregator node here) does the processing and send the compressed data to sink in order to
reduce energy consumption and balance the overall energy consumption of the network
Fig. 5: Dynamic Bandwidth Assignment model
.
Readings from different sensors may not be correlated or data from same clusters are more
compressible. PCA based aggregation is not limited to the single-hop network. The main
concentration is on how it handles with non-ideal projection basis and synchronizes between
two sensor nodes and it is applied to any network in which more than one aggregator nodes
compress the data and send to sink. Fig. 6 (a) shows the cluster with data aggregation node
and data block to the sink. In (b), X0, Xk, X2k, X3k represent the data blocks to be sent to the
sink.
Fig. 6: (a) Cluster with a data aggregation node (b) Data blocks to the sink
IV. PERFORMANCE ANALYSIS
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org925
The different Machine Learning methods used for data aggregation and clustering in WSN
such as Support Vector Machine, Neural Networks, Decision Tree, K-Means, Q-Learning
etc,.. are discussed above. The performance of different schemes are compared by
considering the important parameters of the WSN such as delay, complexity, energy
consumption, topology awareness and overhead. Table 1 and 2 compare the methods for
clustering and data aggregation using certain parameters mentioned above. The clustering
complexity is low when it uses K-Means, Q-Learning and Decision Tree. Q-Learning,
Decision Tree and Neural Networks provides very low overhead to the network. The
methods using Q-Learning algorithm and Decision Tree employ less delay for the Wireless
Sensor Network.
TABLE I: Comparative analysis of different machine learning based methods used for
clustering and data aggregation
MECHANISMS MACHINE LEARNING
ALGORITHM DELAY OVERHEAD
Location based distributed
clustering K-Means Low Moderate
Collaborative Data
Processing K-Means Moderate Moderate
Electing Cluster Head Using
DT Decision Tree Low Low
Online Data
Compression LVQ High High
Large Scale Network
Clustering
Using NN
Neural network Moderate Low
Role Free Clustering Q-Learning Low Low
Gaussian
Process Model GPR High Moderate
Data
Aggregation
Using SOM
SOM High High
Data
Aggregation
Using PCA
PCA Moderate High
TABLE II: Performance analysis of different machine learning based methods used for
clustering and data aggregation
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org926
MECHANISMS TOPOLOGY
AWARENESS
BALANCING
ENERGY COMPLEXITY
Location based distributed
clustering K-Means Yes Moderate
Collaborative Data Processing K-Means Yes Low
Electing Cluster Head Using DT Decision Tree Yes Low
Online Data
Compression LVQ No High
Large Scale Network
Clustering
Using NN
Neural network Yes Moderate
Role Free Clustering Q-Learning No Low
Gaussian
Process Model GPR No Moderate
Data
Aggregation Using SOM SOM No Moderate
Data
Aggregation
Using PCA
PCA Yes Moderate
V. PROPOSED DATA AGGREGATION METHOD
An improved clustering and data aggregation method is proposed by assuming
predetermined Cluster Heads are having enough energy for transmission and computation.
Here, cluster formation is based on the similarity of data from the sensor nodes. After node
deployment, the data similarity between the neighboring nodes is measured in terms of data
correlation. An appropriate threshold value can be set. If the correlation metric is greater
than the threshold value, these two data belong to cluster with similar data.
Similar data clusters only require data aggregation process. Dissimilar data clusters normally
send data to CH and finally forward to the sink node. In the case of similar data cluster, a
cluster based data aggregation method using ICA (Independent Component Analysis) is
employed. Data aggregation is done by the CH. ICA is computationally efficient for data
reduction. It minimizes mutual information using the concept of differential entropy.
Aggregated data from similar clusters are forwarded to sink node. Hence the computation
and energy consumption can be reduced due to less number of aggregation processes. The
flow diagram of proposed methods is shown in Fig. 7.
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org927
Fig. 7: Flow diagram of proposed data aggregation method
VI. CONCLUSION
There is an increasing interest in Wireless Sensor Network application now days. Wireless
Sensor Network use data aggregation for saving energy by reducing the number of
transmission. WSN needs innovative solutions to overcome the challenges and limitations
faced by the network. Machine learning algorithm gives a group of methods for increasing
the capability of network to adapt with the dynamic environment. The paper presents a
review of different machine learning algorithms which are used for clustering and data
aggregation in Wireless Sensor Networks. Comparisons of the performance of such methods
are given by Table 1 and Table 2. It compares the methods using selected parameters such as
complexity, delay, overhead, topology awareness and balancing energy consumption. An
improved similarity based clustering and data aggregation using Independent component
Analysis is proposed in order to reduce the energy consumption and computation in the
network.
REFERENCES
[1]. F.Akyildiz, W.Su, Y.Sankarasubramaniam, and E.Cayirci, Wireless Sensor
Networks: A Survey, Elsevier Computer Networks, Vol.38, Issue 4, PP. 393-422,
2002.
[2]. Morell, A. Correa and M. Barceló and J. L. Vicario, Data Aggregation and
Principal Component Analysis in WSNs,IEEE Transactions on Wireless
Communications, vol.15, Issue.6,PP.3908-3919, 2016.
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org928
[3]. G. Muniraju, S. Zhang, C. Tepedelenlioglu and M. K. Banavar Location
Based Distributed Spectral Clustering for Wireless Sensor Networks, IEEE Sensor
Signal Processing for Defence Conference (SSPD), 2017.
[4]. SangHak Lee and TaeChoong Chung, Data Aggregation for Wireless Sensor
Networks Using Self-organizing Map,Springer-Verlag Berlin Heidelberg 2005.
[5]. G. Ahmed, N. M. Khan , Z. Khalid and R. Ramer, Cluster head selection
using decision trees for Wireless Sensor Networks,IEEE International Conference on
Intelligent Sensors, Sensor Networks and Information Processing, 2008
[6]. Y. C. Tseng, Y. C. Wang, K. Y. Cheng and Y. Y. Hsieh, iMouse: An Integrated
Mobile Surveillance and Wireless Sensor System, IEEE Computer Society, Vol.40,
Issue, PP. 60-66, 2008.
[7]. H. He , Z. Zhu and E. Makinen, A Neural Network Model to Minimize the
Connected Dominating Set for Self-Configuration of Wireless Sensor Networks,IEEE
Transactions on Neural Networks, Vol.20, Issue. 6, PP. 973-982, 2009.
[8]. S. Lin , V. Kalogeraki , D. Gunopulos and S. Lonardi, Online Information
Compression in Sensor Networks,IEEE International Conference on
Communications,2008.
[9]. M. A. Alsheikh, S. Lin , D. Niyato and H. P. Tan, Machine Learning in
Wireless Sensor Networks: Algorithms, Strategies, and Applications, IEEE
Communications Surveys Tutorials, Vol.16 ,Issue.4 , PP.1996-2018 , 2014.
[10]. M. B. H. Frej and K. Elleithy, Secure data aggregation model (SDAM) in
wireless sensor networks ,IEEE 14th International Conference on Machine Learning
and Applications (ICMLA), 2015.
Journal of Information and Computational Science
Volume 9 Issue 12 - 2019
ISSN: 1548-7741
www.joics.org929