an untagged human hand motion classification by ... · untagged human hand motion classification by...
TRANSCRIPT
An Untagged Human Hand Motion Classification
by Pyroelectricity
Chun-Ching Hsiao Department of Mechanical Design Engineering, National Formosa University, No. 64, Wunhua Rd., Huwei Township,
Yunlin County 632, Taiwan
Email: [email protected]
Abstract—Pyroelectric sensors are the most suitable for
detecting human motions because they are sensitive to
human motions induced variances in the infrared radiation.
A pyroelectric sensor array is designed and used to build an
event-time sequence in an infrared radiation domain. Then,
the sequence is used to contrast with a password sequence
for further recognizing human hand motions. This is
applied on the intelligent life for controlling doors to open or
close as a pyroelectric lock. For estimating the functions of
the pyroelectric lock, six password sequences are used to
control the proposed device via discreet manipulation of six
persons. The experimental results verify the feasibility and
efficacy of the proposed design. The false recognition is
lower than 2%.
Index Terms—pyroelectricity, human hand motion,
pyroelectric infrared sensors, thermal vision
I. INTRODUCTION
To provide a supplement and alternative to traditional
vision-based approached, pyroelectric infrared
technology is capable of detecting human motion by
sensing their thermal radiation variations. Therefore,
infrared sensors are more suitable for tracking an
untagged human limb motion induced variances in the
infrared radiation. A human limb tracking system based
on infrared technology can be low-cost, low-data-
throughput, fast response speed, small volume, low
power consumption, nonintrusive sensory approaches
without interference, and unnecessary complex
computations compared with the expensive and
complicated computer vision-based system [1], [2].
The pyroelectric effect is the change in spontaneous
polarization that appears in ferroelectric materials with a
time dependent temperature variation. When a
pyroelectric material is applied to a temporal temperature
change, then the net dipole moment of the polar material
is modified and hence the value of the spontaneous
polarization. The pyroelectric material is associated with
the top and the bottom electrode and an electrical charge,
a current will appear as long as the temperature changes.
Considering a pyroelectric material with charge
collection electrodes, the pyroelectric current flows from
these electrodes in an electric load can be written as [3]:
Manuscript received November 19, 2015; revised February 16, 2016.
IP = η × p × A × dT / dt
where, IP is the pyroelectric current, η is the absorption
coefficient of radiation, p is the pyroelectric coefficient, A
is the active surface area, dT/dt is the temperature
variation rate. Pyroelectric materials have been widely
used for dynamic temperature measurements, infrared or
terahertz detection but also waste heat energy harvesting.
Moreover, pyroelectric infrared detectors exhibit broad
and uniform spectral response which makes them
desirable for many applications such as fire monitoring,
gas detection and security system. The operation of
pyroelectric detectors is to convert infrared radiation
energy into heat and further into electrical signal. When
the dimensions and the materials of pyroelectric elements
are determined, various moving velocities of thermal
objects resulting various temperature variation rates will
directly affect the pyroelectric current and voltage
responsivities. Moreover, the temperature variation rate
has a huge maneuverability according to the design of the
patterns, trenches, cavities and structures in the
pyroelectric materials. However, the temperature
variation rate is difficult to extract from pyroelectric
layers by experimental measurements. Some finite
element models were built using the commercial software
package COMSOL to explore the temperature variation
rate in pyroelectric cells, with some designs of cavities by
wet etching, trenches by a precision dicing saw, and
grooves by a sandblast etching technique, in order to
improve the energy conversion efficiency of PZT cells by
pyroelectricity [4]-[6]. Moreover, Hsiao et al. [7] used an
aerosol deposition to grow the ZnO films by laser
annealing for rapidly fabrication on pyroelectric sensors.
The pyroelectric effect has been proven to be
advantageous in smart and intelligent system and focused
on the applications of sensors and power generators.
Recently, the pyroelectric power generations have been
widely used and studied from time-dependent
temperature fluctuations. The efficiency of the generators
was improved when Olsen et al. [8] proposed using the
Ericsson cycle in the pyroelectric energy harvesting. It
enables the generators to approach Carnot efficiency.
However, using the pyroelectric effect for energy
harvesting lies in finding promising temperature
variations or a circulating thermal loading in real
circumstances. Hsiao et al. [9] proposed a novel
pyroelectric harvester integrating solar radiation with
International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016
©2016 Int. J. Electron. Electr. Eng. 426doi: 10.18178/ijeee.4.5.426-429
wind power. A thermal source is caught from solar
radiation and wind is a dynamic power. This design used
a mechanism to convert the rotary energy of the disk
generator to drive a shutter for generating temperature
variations in pyroelectric cells using a planetary gear
system. The pyroelectric harvester could play a
complementary role when the disk generator was inactive
in situations of low wind speed.
The pyroelectric infrared sensors with the visibility
modulations have been capable of identifying and
classifying the human motions with high sensing
efficiency. When the pyroelectric materials are heated,
the net polarization decreases and the entropy increases
due to a reorientation of the dipole moment. The decrease
of polarization results a current for applying to an electric
circuit. Hence, monitoring the current or voltage
responsivities of the pyroelectric cells can probe human
limb motions. The pyroelectric sensor array can further
detect the orientations and velocities of human limb
motions. Human limb tracking could be extensively
applied in surveillance, robotics, intelligent home and gait
biometrics. This study proposes a low-cost human limb
motion tracking system based on a pyroelectric infrared
sensor array, and this system is further applied on the
intelligent life for controlling doors to open or close as a
pyroelectric lock. The pyroelectric lock is based on the
pyroelectric infrared technology with a sensor array for
sensing human limb motion, then it further confirms the
orders for controlling an electromagnetic valve as an
entrance guard.
II. MATERIALS AND METHODS
The pyroelectric lock consists of pyroelectric infrared
sensors, a shadow mask, an electromagnetic valve and a
NI LabVIEW system, which is depicted in Fig. 1. Nine
pyroelectric infrared sensors with D205B type are used to
construct a 3×3 square array. Fig. 2 shows the assembly
of the sensor array for tracking the human hand motions.
The output signals of the pyroelectric sensors are further
treated via an operational amplifier (LM741), as shown in
Fig. 3. The amplifier is a non-inverting amplifier and has
a close-loop gain with a ratio of output voltage to input
voltage as 1 + RO/RI. Moreover, the shadow mask is used
to insulate the interference between the infrared sensors
and restrict the sensing region of infrared sensors. The NI
LabVIEW system consists of a case of NI PXIe-1082, a
controller of NI PXIe-8135, a data acquisition card of NI
PXIe-6366 and NI LabVIEW 2014 software, which
mainly acquires the pyroelectric signals from nine sensors
to further filter, amplify, modulate and digitize the
analogy signals as the digital signals. The digital signals
are transferred into event-time sequences. The sequences
are used to compare with set sequences as password
sequences for further confirming the orders and
controlling the electromagnetic valves as the entrance
guards. For digitizing pyroelectric signals, a simple and
effective signal processing method, as depicted in Fig. 4,
is adopted to integrate pyroelectric signals and time
domain signals as follows:
(1) Calculate the incremental data from the raw data,
and further take them as absolute data fluctuation. The
incremental data is the data difference between current
moment and last moment.
(2) Integrate the incremental data with the raw data as
the characteristic signal.
(3) Set a threshold and further digitize the
characteristic signal.
(4) Smooth the digital signal and integrate the digital
signal and time as the event-time sequence.
Figure 1. Schematic diagram for the setup of the pyroelectric lock.
Figure 2. Schematic diagram for the sensor array.
Figure 3. The electrical circuit of the pyroelectric sensor.
International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016
©2016 Int. J. Electron. Electr. Eng. 427
Figure 4. The treatment flow of the pyroelectric signals.
The Event-Time Sequence (ETS) can be expressed as
a matrix with the number of Pyroelectric Sensors (PS) as
column elements and the duration (DT) as row elements.
EST=ETSi,t=[
𝐸𝑆𝑇1,1 ⋯ 𝐸𝑆𝑇1,𝐷𝑇
⋮ ⋱ ⋮𝐸𝑆𝑇𝑃𝑆,1 ⋯ 𝐸𝑆𝑇𝑃𝑆,𝐷𝑇
]
iϵ{1, 2, …., PS}, tϵ{1, 2, …., DT}
The PS indicates the sensing spatial resolution for a
given sensing area, which is a trade-off between the
required spatial resolution and the limited physical
response of the pyroelectric sensors. The issue of human
limb motion recognition for further confirming the orders
can be solved through measuring motion similarities
between a test and a password sequence in the event-time
domain. A training set is selected from the ETS as a
password sequence (P), and then a test sequence (Q) is
measured from the ETS associated with a query motion.
For evaluating the similarity between the test and the
password sequence, a criterion of the Median Hausdorff
Distance (MHD) is used and expressed as [10]:
MHD(Q, P)=𝑚𝑒𝑑𝑖𝑎𝑛
𝑸𝒕 ∈ 𝐜𝐨𝐥(𝑸)(
𝑚𝑖𝑛𝑷𝒕 ∈ 𝐜𝐨𝐥(𝑷)
(‖𝑸𝒕 − 𝑷𝒕‖))
where col(∙) represents the set of the column vectors of
the embraced matrix, Qt is the column vector extracted
from the test sequence, Pt is also the column vector
extracted from the password sequence, ║∙║ denotes the
Euclidean distance from Qt to Pt, and the operator
‘median’ selects the median ranked value from min ║∙║
over Q. Moreover, the median Hausdorff distance is a
rather robust criterion with less sensitive to outlier
observations. A smaller median Hausdorff distance
represents the more similar between the password and the
test sequence. The exactitude for confirming the orders
can set the median Hausdorff distance. An excessively
small median Hausdorff distance will increase the
difficulty to confirm the orders. However, an excessively
large distance will induce an erroneous judgement. The
set value of the median Hausdorff distance will decide the
sensitivity of the pyroelectric lock to recognize human
limb motion.
III. RESULTS AND DISCUSSION
For validating the usability of the pyroelectric lock,
human right hand motions are adopted to operate the
pyroelectric lock. All exercises are performed in a
30cm×30cm×10cm cube. The pyroelectric sensor array is
placed in front of the human right hand. The distance
between the array and the hand is nearly 10cm. Fig. 1
shows the sketch for using the human right hand with a
fist gesture to operate the pyroelectric lock. The fist
gesture can minimize the size of heat radiation for
confirming the path of hand motions detected by the lock
consisted of the pyroelectric sensor array. Moreover, six
right hand motions as the exercises are transferred to the
password sequences for verifying the capability of the
pyroelectric lock. The images of these motions are
depicted as in Fig. 5. These right hand motions are Z
type(RM1), × type(RM2), ㄇ type(RM3), ㄩ type(RM4),
┴ type(RM5) and ⌂ type(RM6). The database of these
motions is constructed from six persons to perform the
exercises. These persons have various physical
characteristics of 160-180cm in height and 60-80kg in
weight and perform these motions with both the motion
styles and velocities respectively, which provides more
realistic data for testifying the versatility of the
pyroelectric lock.
Figure 5. Schematic diagram for the hand motions as password sequences.
The database from the sensor array capturing six
persons, each person performing six hand motions and
each motion repeated six times, which is built as 216
(6×6×6) sequences. Thus, each motion generates 36 (6×6)
sequences, each sequence is associated with a 9×DT
matrix. Here DT represents the duration of each motion
and the array consists of nine pyroelectric sensors.
Furthermore, 36 sequences built by six persons repeated
six times to present a hand motion, these sequences are
merged as a password sequence for charactering the
motion. Fig. 6 shows the median Hausdorff distances
between the password sequence and the test sequence,
where rm1 to rm6 represent the test motions and RM1 to
RM6 represent the password sequence. It is obvious that
the each test motion is nearest to the corresponding
International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016
©2016 Int. J. Electron. Electr. Eng. 428
password motion due to the smallest median Hausdorff
distance. Fig. 7 shows a confusion matrix between the
password motion and the test motion for presenting a few
false recognition lower than 2%. Hence, the spatio-
temporal sequence can achieve the classification of hand
motions with the low-dimension.
Figure 6. Median Hausdorff distance between the password sequence and the test sequence.
Figure 7. Confusion matrix for the classification of hand motions.
IV. CONCLUSIONS
The spatiotemporal sequence is used to characterize
the human hand motion features by the pyroelectric
sensor array. The test sequence is compared with the
password sequence for recognizing the human right hand
motions with six patterns. The proposed device can
capture the low-dimensional feature sequences in real
time without extra computation. The six right hand
motions can be classified from 216 series data of the
pyroelectric sensor array. This is used on the intelligent
life for controlling doors as a pyroelectric lock. The false
recognition is lower than 2%. The pyroelecticity is
successfully applied on tracking the untagged human
hand motion in the infrared radiation domain without the
need of heavy computations.
ACKNOWLEDGMENT
The authors are thankful for the financial support from
the Ministry of science and Technology (Taiwan) through
Grant No. MOST 104-2221-E-150-013, and the
experimental support from the Common Laboratory for
Micro and Nano Science and Technology at National
Formosa University, and Nano-Electro-Mechanical-
Systems (NEMS) Research Center at National Taiwan
University.
REFERENCES
[1] Q. Guan, C. Li, X. Guo, and G. Wang, “Compressive
classification of human motion using pyroelectric infrared sensors,”
Pattern Recognit. Lett., vol. 49, pp. 231-237, 2014.
[2] B. Yang, J. Luo, and Q. Liu, “A novel low-cost and small-size
human tracking system with pyroelectric infrared sensor mesh
network,” Infrared Phys. Techn., vol. 63, pp. 147-156, 2014.
[3] R. W. Whatmore, “Pyroelectric devices and materials,” Rep.
Progr. Phys., vol. 49, pp. 1335-1386, 1986.
[4] C. C. Hsiao, J. C. Ciou, A. S. Siao, and C. Y. Lee, “Temperature
field analysis for PZT pyroelectric cells for thermal energy
harvesting,” Sensors, vol. 11, pp. 10458-10473, 2011.
[5] C. C. Hsiao, A. S. Siao, and J. C. Ciou, “Improvement of
pyroelectric cells for thermal energy harvesting,” Sensors, vol. 12,
pp. 534-548, 2012.
[6] C. C. Hsiao and A. S. Siao, “Improving pyroelectric energy
harvesting using a sandblast etching technique,” Sensors, vol. 13,
pp. 12113-12131, 2013.
[7] C. C. Hsiao and S. Y. Yu, “Rapid deposition process for zinc
oxide film applications in pyroelectric devices,” Smart Mater.
Struct., vol. 21, 2012.
[8] R. B. Olsen, “Ferroelectric conversion of heat to electrical energy
- A demonstration,” J. Energy, vol. 6, pp. 91-95, 1982.
[9] C. C. Hsiao, J. W. Jhang, and A. S. Siao, “Study on pyroelectric
harvesters integrating solar radiation with wind power,” Energies,
vol. 8, pp. 7465-7477, 2015.
[10] D. M. Mount, N. S. Netanyahu, and J. L. Moigne, “Efficient
algorithms for robust feature matching,” Pattern Recogn., vol. 32,
pp. 17-38, 1999.
Chun-Ching Hsiao is an associate professor of
the Department of Mechanical Design Engineering at the National Formosa
University, Taiwan. His researches focus on
pyroelectric sensors, flexible devices, thermoelectric generators, pyroelectric
harvesters, pyroelectric human-machine
interface and residual stress of thin film.
International Journal of Electronics and Electrical Engineering Vol. 4, No. 5, October 2016
©2016 Int. J. Electron. Electr. Eng. 429