2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
1
Abstract—We propose an IoT framework for next-generation
racket sports training. To validate its performance, a wireless
wearable sensing device (WSD) based on MEMS
(microelectromechanical systems) motion sensors was used to
recognize different badminton strokes and classify skill levels
from different badminton players. The system includes a
customized sensor node for data collection, a mobile app and a
cloud-based data processing unit. The WSD developed is
low-cost, easy-to-use and computationally efficient compared to
video-based methods for analyzing badminton strokes. It offers
the advantage of dynamic monitoring of multiple players in
indoor and outdoor environments. In this paper we present the
hardware design, mobile software implementation, and data
processing algorithms of the system. Twelve right-handed male
subjects wore the WSD on their wrists while each performed 30
trials of different strokes in a real badminton court. The results
show that our system is capable of recognizing three different
actions, i.e., smashes, clears and drops, with an accuracy rate of
97%. The skill assessment function can differentiate between
professional, sub-elite, and amateur players from their stroke
performance. This IoT framework aims to change the way of
racket sports training from experience-driven (subjective) to
data-driven (objective), and which can be easily extended to
analyze the motions and skill levels of players in other racket
sports (e.g., tennis, table tennis and squash) for training and/or
practice.
Index Terms—IoT Wearable Devices, Micro Inertial
Measurement Unit, Sports Analysis, Badminton Strokes, Motion
Assessment
I. INTRODUCTION
acket sports, such as badminton, tennis, table tennis and
squash, are amongst the most popular recreational sports
activities. In addition to physical fitness, racket sports
players need aerobic stamina, agility, strength, speed, and
precision simultaneously. Meanwhile, racket sports are highly
1Department of Mechanical and Biomedical Engineering, City University of
Hong Kong (CityU), Hong Kong SAR, China.
2Department of Electronic Engineering, CityU, Hong Kong SAR, China.
3Department of Computer Science, CityU, Hong Kong SAR, China.
4Shenzhen Academy of Robotics (SZAR), Shenzhen, China. #Contact Authors: Dr. Rosa H. M. Chan ([email protected]) is Associate Professor in the Department of Electronic Engineering at City
University of Hong Kong. Dr. Wen J. Li ([email protected]) is Chair
Professor of Biomedical Engineering at CityU; he is also an affiliated professor at the SZAR.
technical requiring sophisticated motor coordination for agile
steps and racket movements. Generally, a badminton player
should be able to strike a shuttlecock with different motions
(with corresponding forces) called ‘strokes’. This requires
techniques which are constructed and delivered through precise
basic skills, powerful strength, and coordination [1].
Badminton can be characterized by three types of racket
movements: short drops, long clears and hard smashes [2].
Timely execution of these strokes contributes to success in
matches [3]. In recent years, numerous researchers have
investigated the movement of racket sports players [4], since
observable changes from athletes are vital in the coaching
process. The majority of conventional studies focus on using
video-based methods to track athletes’ actions and performance.
However, such video-based methods are not utilized in
day-to-day training due to limitations such as large
computation load [5], costly equipment, and environmental
constraints [6]. Also, adoption of multiple physical markers
tends to incur a marker crossover phenomenon and extra cost.
Thus, conventional means cannot provide timely feedback to
players. Instead, to capture the motion in daily practice outside
the lab, researchers have shifted their efforts to recognize
activities using inertial sensors worn on the body. The
advancement in MEMS and Bluetooth Low Energy (BLE)
technologies makes feasible long-term motion data collection
and transmission in a small package with relatively long battery
life.
Recently, a number of research studies in sports activities
recognition have utilized MEMS motion sensors [7]. For
example, a support system for golf players using wearable
sensors was designed to collect the athletes’ kinematics data [8].
An inertial information database was constructed for
professional horseback riders with 16 motion sensors. Data on
several motion features: elbow angle, knee angle, backbone
angle, hip position, and knee-elbow distance were extracted [9].
Another example is a volleyball application investigating the
frequency of jumps in volleyball games using a tri-axial
accelerometer [10]. On the other hand, there are already similar
research work analyzing strokes and smashing motions in
various racket-related sports using video-based and MEMS
technologies [11]. For example, in a study of baseball players,
researchers collected data from a body sensor network (made of
MEMS-based acceleration sensors) to generate motion
transcripts, and used them to measure coordination among limb
segments and joints [12]. However, each sensor node’s surface
area is about 3. 2cm x 6.6 cm, which makes it impractical to be
worn by baseball players in a real game. The possibility of
using MEMS-based gyroscopes for skill assessments of tennis
IoT for Next-Generation Racket Sports
Training
Yufan Wang1, Student Member, IEEE, Meng Chen1, Xinyu Wang3, Rosa H. M. Chan2*, Senior Member,
IEEE and Wen J. Li1,4*, Fellow, IEEE
R
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
2
players was investigated as well [13]. However, they merely
collected motion data from three sets of two-axis gyroscopes
placed in different locations of four different participants, and
no data segmentation or motion differentiation were reported in
that work. In our previous work [14], we used an inertial
measurement unit to assess the skill level of volleyball spikers.
The system was demonstrated to be stable and capable of
assessing spike-motion differences between professional,
sub-elite and amateur volleyball players. However, the
collected data could only be saved on a local Secure Digital (SD)
card and processed offline. For the device presented in this
work, we have further improved the stability of the sensing
device and implemented the real-time transmission of motion
data via Bluetooth technology. To the best of our knowledge,
the system we describe here is one of the first that is capable of
racket sports action recognition and skill assessment using a
single sensor unit, communicating through Bluetooth Low
Energy technology, and computing in the Cloud.
With proliferation of consumers’ passion for a healthy and fit
lifestyle in recent years, the wearable electronic market has
grown exponentially. Many enthusiastic racket sports players
are looking for a more professional and efficient way to track
their motion data and assess their performance with data
collected during competitions. Amateurs likewise may want to
learn the correct motions for a particular action in a sport in
order to improve their skill level.
Here, using the example of badminton, we propose a novel
and “smart” racket action recognition and skill assessment
system using a low-power MEMS inertial measurement unit
(IMU) with BLE and Cloud Technology. This application
demonstrates the possibilities in using IoT framework for
next-generation racket sports training. The system is capable of
classifying different actions and differentiating skill levels
between professional athletes and badminton amateurs. At the
same time, it provides feedback for the quality of a player’s
smashes and clears. First, we developed a wireless wearable
sensing device (smaller than any existing commercial products
to the best of our knowledge) with an overall size of
18mm×17mm×2mm, to collect inertial data. Second, we
designed a mobile app that can visualize experimental results
and upload data from the experiments to the cloud server.
Third, we used machine-learning algorithms to classify
different badminton actions at 97% prediction accuracy.
Moreover, we have shown that this system can discriminate
skill levels between professional badminton athletes and
badminton amateurs in term of different actions, reaching a
high prediction accuracy of 83.3% for smashes and 90% for
clears. Foreseeably, this framework can be extended to
recognize actions and analyze skill levels of players in other
racket sports, such as tennis, table tennis and squash.
II. SYSTEM SETUP
Here we detail the complete technical solution developed to
build a smart badminton action recognition system based on
low-power Bluetooth communication and Cloud technology. It
contains hardware and software parts as well as sensor
placement locations. The general system setup principles are
shown in Fig. 1. The system consists of a sensor node, a
high-speed camera, a mobile device, a cloud server and a
laptop. We used this system to collect motion data from
badminton players and to recognize different badminton
actions. The general work principle for the system is that data
collected by an inertial measurement unit was sent to a mobile
device through Bluetooth Low Energy (BLE). Once the mobile
phone received the data, it sent the motion raw data to the
remote server via Cloud technology. After collecting all data,
users could then view badminton players’ data with a client
Wi-Fi.
A. Hardware System
Fig. 2 shows the wireless IMU design. It comprises a
microprocessor with Bluetooth wireless communication
module, a MEMS motion sensing chip (with 3-axes
accelerometer and 3-axes gyroscope), an on/off switcher and a
coin cell battery (on the back of the circuit board). DA14583
(Dialog Semiconductor, Reading, United Kingdom) fully
integrates radio transceiver and baseband processor for
Bluetooth Low Energy, which saves space for communication
and processing.
Fig. 1. System setup and sensor placement on human subjects.
Fig. 2. (a) Circuit board and dimension of the Wearable Sensing Device (WSD)
which (b) can be embedded in wrist band.
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
3
TABLE 1 COMPARISON BETWEEN DIFFERENT BLUETOOTH CHIPS [15]
Company Product
Power
Consumption
TX (mA)
Power
Consumption
RX (mA)
Chip Size
Dialog Semiconductor
DA14583 4.9 4.9 5 mm 5 mm
Nordic
Semiconductor nRF51822 8.0 9.7 6 mm 6 mm
Texas
Instruments CC2640 6.1 5.9 5 mm 5 mm
Qualcomm CSR1010 16.0 18.0 5 mm 5 mm
We considered two major parameters while selecting chips
for our BLE module: power consumption and chip size. Table
1 summarizes the specification of this Bluetooth chip and
comparison among its congeneric products. This unit enables a
small and power efficient Bluetooth smart system to
communicate with the motion sensor and send motion data to
the peripheral mobile device. It was programmed using Dialog
Semiconductor Software Development Kits (SDK). Apart from
inertial sensors mentioned above, there are several commercial
inertial sensor systems available, such as MTi0series from
Xsens North America, Inc. and the wireless accelerometer from
Noraxon. However, the ranges of these sensors were not
suitable for our application. According to the results from
Wang et al. [16], a 16g sensor range is enough to analyze all the
badminton actions. Therefore, we mainly considered full-scale
range and chip size for IMU chip selection. Table 2
summarizes the specification of the inertial sensor used in this
study and a comparison among its congeneric products. As
shown in Table 2, we utilized BMI160 from BOSCH because
of its suitable sensor range (16 g) and small size (2.5 mm × 3.0
mm × 0.83 mm). BMI160 is a highly integrated, low power
IMU that provides orientation and acceleration readings in
x-y-z dimensions. Using this sensor module, we kept the entire
WSD size within an 18 mm × 17 mm package that weighs only
2.2 g.
TABLE 2 COMPARISON BETWEEN DIFFERENT IMU CHIPS
Company Product
Gyroscope-
Full-scale Range (°/sec)
/ Sensitivity
(LSB/°/sec)
Accelerometer-
Full-scale Range (g) /
Sensitivity
(LSB/g)
Chip Size
Bosch Sensortec
[17]
BMI160 2000/16.4 16/2048 2.5 mm 3 mm
InvenSense
[18] MPU9250 2000/16.4 16/2048 3 mm 3 mm
ST [19]
LSM9DS1 2000/14.3 16/1366 3.5 mm 5 mm
In this study, a high-speed camera was used for two
purposes. It provides validation for the inertial information
received from sensors, and for auto-segmentation in the data
processing. We considered eight major parameters while
selecting the camera for our system: frame rate, image
resolution, exposure time (shutter speed), sensitivity, bit depth,
colour or monochrome, and camera interface.
Since the users’ playing approach varied, we chose the
BASLER acA2000-165um camera, which is capable of
freezing fast moving objects in an indoor sports center
environment as well as providing high definition. Table 3
shows the specifications of the acA2000-165um camera.
TABLE 3 SPECIFICATIONS OF THE ACA2000-165 µM CAMERA [20]
Product acA2000-165 µm
Resolution 2,048 px×1,088 px
Frame rate 165 fps Mono/Color ±4800 µT
Interface USB 3.0
Exposure control Programmable via camera API Pixel depth 10,12 bits
B. Interface Software
To receive and visualize the IMU data collected from the
BLE peripheral node, we wrote a software application on the
mobile phone. This mobile application is based on the
Evothings framework, a development tool to create the mobile
apps for Internet of Things (IoT). It is an open-source software
developed with Java Script programming language. The
software we developed can be divided into three modules: BLE
Connection, Sensor Data Display and Sensor Data Cloud. The
BLE connection module is based on Evothings and Cordova
BLE Plugin that implements BLE support for Android, IOS,
and Windows 8.1. Fig. 3 shows the inertial sensor data from a
badminton player when he is smashing.
Our system adopts a new cloud-based method to save the
data received from BLE into the remote server. Moreover, this
method also supports building a cloud-based badminton actions
database for the use of other researchers. In this module, the
Cordova HTTP plugin is used to recognize the cloud saving
function. Once the collecting process ends, any user can look
up the sensor data by visiting a designated website.
Fig. 3. Android mobile app software for collecting and displaying 3-axes sensor
data from the WSD.
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
4
III. METHODS
Our experiment was conducted at HU FA KUANG Sport
Centre in City University of Hong Kong. We recruited twelve
right-handed male badminton players, including four amateurs,
four sub-elites and four elite badminton players. Their
demographics are shown in Table 4.
The experimental procedures were reviewed and approved by
the Ethics Committee of the City University of Hong Kong, and
all participants provided written informed consent before
participation. Elite players represented their region and had
played in international competitions more than 10 times.
Sub-elite players played in local competitions but had no
experience playing in international competitions. Amateurs are
badminton beginners who have never played in competitions.
Table 4 shows specific physical information about all the
subjects. As badminton is a wrist-based sport, each
right-handed subject wore the designed sensor on their right
wrist when performing badminton basic training. Such
configuration is comfortable and unobtrusive.
After a 20-minute warmup supervised by a professional
coach, each subject performed 20 straight smashes, short drops,
and long clears, respectively. As shown in Fig. 4, the coach
served the shuttlecock to position “1” (for drops), position “2”
(for smashes), and position “3” (for clears). The subject
performed the actions at different positions. Every subject had
to hit the shuttlecock to the destination inside the right half
court; otherwise, we did not count it as a successful action.
Fig. 5 shows the six-axis synchronized raw data from
players at different levels. Fig. 6 displays the raw data captured
by the WSD. The first two rows show the angular velocity and
acceleration from a clear action, while the second and third two
rows show the inertial information from 6-axes for the drop and
smash actions.
Fig. 4. The positions of badminton players to perform drops (1), smashes (2)
and clears (3).
Fig. 5. Example sensor data recorded at wrist during smash. Angular velocities
of (a) an amateur, (c) a sub-elite and (e) an elite and accelerations of the (b) amateur, (d) sub-elite and (f) elite are crucial to discriminate the skill levels.
Fig. 6. Raw sensor data plot from three different strokes.
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
5
Fig. 7. Data processing flow to recognize badminton activities and assess skill levels from a single wrist-worn sensor.
TABLE 4 SUBJECT DESCRIPTIONS
Subject Age Height Body Mass
Elite A 22 184 cm 80 kg
Elite B 25 179 cm 74 kg
Elite C 22 175 cm 68 kg
Elite D 21 182 cm 75 kg
Sub-elite A 29 175 cm 74 kg
Sub-elite B 26 180 cm 76 kg
Sub-elite C 25 174 cm 70 kg
Sub-elite D 22 176 cm 69 kg
Amateur A 25 183 cm 70 kg
Amateur B 26 179 cm 71 kg
Amateur C 28 174 cm 74 kg
Amateur D 23 170 cm 64 kg
IV. DATA PROCESSING
A. Badminton Actions Recognition System
After data collection, a typical machine learning data
processing method was implemented in our badminton actions
recognition and skill level assessment system shown in Fig. 7.
This framework includes preprocessing, segmentation, feature
extraction, dimensionality reduction and classification. Each
stage of this framework can be implemented using a variety of
methods. To demonstrate performance in this section, we
utilized the Support Vector Machine (SVM) classifier.
In data preprocessing, data points associated with the subject
failing to hit the shuttlecock inside the target area were
removed. We first loaded the raw data Ṧ(t)ij from each subject.
Then we applied a 3-point filter moving average to reduce the
effect of noise and obtain a clearer S(t)ij signal. The statistical
and morphology features were extracted and each dataset Ẋi =
(f1 …fm) was merged into a large matrix Ẋ. Segmentation was
processed automatically by finding the peak of the signal. This
window-based method can realize real-time data processing
[21].
In this study, we extracted 15 statistical features and 3
morphological features as inputs for badminton actions
recognition and classification, as detailed in Table 5. These
features included 1) mean and variance from the six axes and
root mean square (RMS) from three acceleration axes; 2) the
maximum acceleration in x-axis, 3-axis acceleration data and
3-axis angular velocity data. We compiled a badminton actions
(smashes, clears and drops) database from the inertial sensor.
a) Principle Component Analysis (PCA)
Principle components were identified to alleviate the
computing load and bandwidth requirements during
communication with the cloud server. We used PCA to
preprocess the data before classification because PCA shows
better performance compared to nonlinear dimensionality
reduction [22].
Eighteen features extracted from the raw badminton actions’
data can be expressed as vectors, where f=[f1,f2,…,f18]. These
new features are linear combinations of the original features
and can be expressed as fn=[fn1,fn2,…fnm], where m represents
the dimension to be reduced:
1 1 2 2= + +m i i mi mf a f a f a f (1)
where aij are eigenvalues of the covariance matrix. As we have
only one node, (1) can be simplified to
1 1 2 2 m= + +m mf a f a f a f . (2)
b) Support Vector Machine
Support vector machine (SVM) is a supervised learning
algorithm used for solving a binary classification problem. As
shown by the Mercer’s condition [23], SVM exhibits some
distinct advantages such as good generalization ability, and
robustness by free choice of model parameters in processing
high dimensional and linear inseparable problems over other
supervised learning algorithms, such as Naïve Bayes and
Linear discriminant analysis. This algorithm is also suitable to
our application because there are some intersections for clears
and smashes action (patterns are similar). That comprises a
linear inseparable problem.
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
6
TABLE 5 STATISTICAL AND MORPHOLOGICAL FEATURES
No. Symbol Description
1 Aax Mean value of acceleration from x axis
2 Aay Mean value of acceleration from y axis
3 Aaz Mean value of acceleration from y axis
4 Dax Variance of acceleration from x axis
5 Day Variance of acceleration from y axis
6 Daz Variance of acceleration from z axis
7 Agx Mean value of angular velocity from x axis
8 Agy Mean value of angular velocity from y axis
9 Agz Mean value of angular velocity from z axis
10 Dgx Variance of angular velocity from x axis
11 Dgy Variance of angular velocity from y axis
12 Dgz Variance of angular velocity from z axis
13 Max The maximum acceleration from the x-axis
14 Ma The magnitude of the 3-axis acceleration
15 Mg The magnitude of the 3-axis angular velocity
16 RMSax Root mean square of acceleration from x axis
17 RMSay Root mean square of acceleration from y axis
18 RMSaz Root mean square of acceleration from z axis
In this study, as we have three labels including smash, clear,
and drop, we choose a one-versus-one strategy [24] in which
three SVMs are constructed using corresponding data from the
other two classes and then a voting scheme is applied. This is a
binary classification problem solved by SVM:
Given a training data set 1
, , 1, 1n
m
i ii
T y R y
X X
where, X is a m-dimensional matrix; yi is a binary label, which
belongs to either 1 or -1; n is the total number of samples; and i
is the current sample number. The main idea of using SVM is
to map the training data set into a higher-dimensional feature
space and then to classify the training data set with hyperplanes.
The problem that finds the maximum margin hyperplane
(MMH) can be converted to an optimization problem that can
be described as follows:
,
2
argmin1
2b
. .s t ( ) 1, 1,...,iy b i n i
x
(3)
where is a normal vector of a hyperplane and b is an offset of
a hyperplane from the origin along the normal vector.
According to the Lagrangian multipliers under the
Karush-Kuhn-Tucker (KKT) conditions, (3) can be
reformulated as follows
2270
1
( , , )
1
21
nT
i ii
L b
y b
ix (4)
where α represents the Lagrangian multipliers vector. The
derivative of (4) with respect to results in
270
1
n
i i
i
y
ix (5)
The derivative of (4) with respect to b results in
270
1
0n
i i
i
y
(6)
Then, after (5) and (6) are substituted into (4), we obtain a
simplified Lagrangian dual problem.
270 270
1 1
1arg max
2
n n
i i j i ji i
y y
i jx x
. .s t 0, 1,...,i i n
270
1
0n
i ii
y
(7)
Since there are some overlap data from clears and smash
actions, which means that our case is not linear separable, we
add a slack variable ξi and an error penalty constant C to find a
tradeoff between a large margin and an error penalty.
Following the aforementioned procedure, we obtained the
simplified Lagrangian dual problem in the case of non-linear
separable problems as:
270 270
1 1
1arg max
2
n n
i i j i j
i i
y y
i jx , x
. .s t 0, 1,...,iC i n
270
1
0n
i i
i
y
(8)
By using Sequential Minimal Optimization (SMO) algorithm
[25], we can obtain the Lagrange multipliers αi. According to
(4), we can calculate the final and find an optimization
hyperplane. The decision function for classification is:
90
1
sgnn
T
i i
i
d y b
X i jx , x (9)
where yi refers to the class label of a support vector; i and b0
refer to two constants; and X refers to the testing set of
badminton actions samples whose labels are yi. To investigate
the influence of parameters on classification performance, we
randomly chose some parameters for testing tabulated in Table
6.
360 datasets were collected from 12 subjects, each of whom
performed 30 trials for three different actions. We used nine
subjects’ datasets (270 datasets) for the training, and the rest of
the datasets (90 datasets) from a different three subjects for
testing classifier performances. During the training process, we
used 10-fold cross validation to avoid the overfitting problem
and find the best parameters of the SVM classifier.
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
7
TABLE 6 RANDOMLY CHOSEN PARAMETERS OF SVM
Penalty Parameter (C) Gamma Kernel
1 0.0001 Linear
100 0.0005 Polynomial
1000 0.001 RBF
5000 0.005 Sigmoid
10000 0.01
50000 0.1
We compared C values ranging from 1 to 50000, Gamma
values ranging from 0.0001 to 0.1, and several different types
of kernels. We achieved the best classifier when C=1, and when
using the linear kernel function. Table 7 shows the average
classification results when using SVM following PCA
(SVM+PCA).
TABLE 7 SVM+PCA CLASSIFICATION RESULTS OF RECOGNIZING DIFFERENT
STROKES
Actions Precision Recall F1-score
Clears 91% 1.00 0.95
Drops 100% 1.00 1.00
Smash 100% 0.90 0.95
AVERAGE 97% 0.97 0.97
As shown in Table 7, the recognition accuracies for three
different actions (clears, drops, and smash) are 91%, 100% and
100%, respectively. This result demonstrated clear distinction
between different actions. On average, the precision of
classifying different actions can reach 97%, which means our
system is highly effective.
B. Skill Assessment System
Similar to the above analysis, we changed the label from
different actions to different skill levels. We labeled three
different levels (Elite, Sub-elite and Amateur) as shown in the
Table 8. We used nine subjects’ datasets (90 datasets) from
each group for the training, and the rest (30 datasets) from
another three different subjects from each group for testing
classifier performances. 10-fold cross validation was used
again to avoid the overfitting problem and find the best
parameters of the SVM classifier. This was repeated 64 times
to ensure that all possible combinations of testing sets with
three subjects of different skill levels were covered. Table 8,
Table 9 and Table 10 show the skill assessment results in terms
of different actions.
TABLE 8 SVM+PCA CLASSIFICATION RESULTS OF SKILL ASSESSMENT IN
SMASH STOKES
Skill Level Precision Recall F1-score
Elite 100% 0.90 0.87
Sub-elite 70% 0.77 0.74
Amateurs 80% 0.78 0.89
AVERAGE 83.3% 0.82 0.83
As shown in the Table 8, the recognition accuracy of elite,
sub-elite, and amateur players through smash strokes are 100%,
70% and 80% respectively. For skill assessment of clear
strokes as shown in Table 9, the average classification precision
is 90.3%, which demonstrates clear distinction in performance
between amateurs to elites.
TABLE 9 SVM+PCA CLASSIFICATION RESULTS OF SKILL ASSESSMENT IN
CLEAR STROKES
Skill Level Precision Recall F1-score
Elite 100% 1.00 1.00
Sub-elite 82% 0.90 0.86
Amateurs 89% 0.80 0.84
AVERAGE 90.3% 0.90 0.90
TABLE 10 SVM+PCA CLASSIFICATION RESULTS OF SKILL ASSESSMENT IN
DROP STROKES
Skill Level Precision Recall F1-score
Elite 100% 0.45 0.62
Sub-elite 0% 0.00 0.00
Amateurs 0% 0.00 0.00
AVERAGE 33% 0.15 0.21
On the contrary, the classification accuracy of skill level from
wrist motion during drop stokes is very low, particularly for
sub-elite players and amateurs, as shown in Table 10.
C. Comparison of Different Classifiers
For stroke recognition, we compared k-Nearest-Neighbor
(kNN) non-parametric classifier and Naïve Bayes (NB)
classifier, as shown in Table 11, to determine whether
SVM+PCA is the best classifier for our data. We tested
different k values (from 1 to 11) to find the best estimator for
our data, achieving the best model results when k=5. The
results from testing two other algorithms demonstrated that the
computational efficient of PCA+SVM is also sufficiently
accurate.
TABLE 11 BADMINTON ACTIONS CLASSIFICATION ACCURACIES OF DIFFERENT
ALGORITHMS
Classification
Algorithm
Parameters Accuracy
SVM+PCA C = 1,
Gamma =0.0001 97%
SVM C = 1,
Gamma =0.0001 94%
Nearest
Neighbor
K = 5 94%
Naïve Bayes N.A. 90%
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
8
TABLE 12 SMASH LEVEL CLASSIFICATION ACCURACIES OF DIFFERENT
ALGORITHMS
Classification
Algorithm
Parameters Accuracy
SVM+PCA C = 1,
Gamma =0.001 83.3%
SVM C = 1,
Gamma =0.001 74%
Nearest
Neighbor
K = 5 78%
Naïve Bayes N.A. 78%
Table 12 and Table 13 compare skill assessment system
performance in a similar way. The results for smash and clear
are listed only, because the average accuracy of assessing skill
levels in drops is very low. Again, PCA+SVM shows
advantages in dealing with a linear inseparable problem over
k-Nearest-Neighbor and Naïve Bayes classifier.
TABLE 13 CLEAR LEVEL CLASSIFICATION ACCURACIES OF DIFFERENT
ALGORITHMS
Classification
Algorithm
Parameters Accuracy
SVM+PCA C = 1,
Gamma =0.001 90%
SVM C = 1,
Gamma =0.001 86%
Nearest
Neighbor
K = 7 82%
Naïve Bayes N.A. 84%
V. DISCUSSION
Quantifying sport activities is of great interest since it allows
trainers and coaches to assess an athlete’s performance. This
paper presents a smart badminton actions recognition system
comprising of Bluetooth Low Energy technology, MEMS
inertial measurement unit, Cloud technology, and machine
learning algorithms. The complete platform for badminton
actions motion data analysis includes three parts: a wearable
sensor, a mobile app, and cloud server. The data collected by
an inertial measurement unit (IMU) is sent to a mobile phone
through Bluetooth Low Energy (BLE). Once the mobile phone
receives the data, it sends the motion data to the remote server
by Cloud technology. After data collection, users can analyze
badminton players’ data on a server in real time or afterwards.
The results shown in Table 7 indicate that our smart system
can classify at least three different badminton strokes clearly
with an average accuracy of 97%. The system can
automatically provide data statistics of badminton players,
which can help coaches and athletes to learn about real
condition changes during a match or a training session.
Classification results of the action recognition system strongly
support the assumption that wrist motion is crucial in
badminton playing. And, we have shown that different strokes
require distinct wrist motion in execution.
As for the skill assessment system, test samples from elite
players are recognized precisely, showing that the elite players
have a distinctive motion compared to the other two groups of
players in all strokes. It is thus feasible to identify level of
performance from wrist motion data of smash and clear. Based
on the results thus far, we could possibly identify elite players
just by observing their clear strokes. However, drop strokes
are relatively flexible. Moreover, amateurs and sub-elites play
similarly while elite players’ drop strokes are consistently
different from those of amateur and sub-elites. Using this
system, we can compile a database of badminton action
movements from players at different levels, which can then be
used by sports scientists and professional coaches for further
study and research.
Motion analysis is an important factor in building
self-awareness of athletes in playing sports. Using MEMS
sensor to capture motion data can help badminton players or
other racket sports players improve their skills, which will play
a significant role for next-generation racket sports training.
With the advances in MEMS sensors and wireless
communication technology, as well as cloud computing, it is
possible to use wearable sensing devices to automatically
recognize different actions that can provide statistics during
matches, which will allow athletes themselves or their coaches
to assess their performance in real time. In this study, we
investigated some major features in characterizing each data
segment. The performance levels represented by the skill data
were thus estimated by SVM, kNN and NB classifiers.
Comparisons of these classifiers show SVM achieves high
accuracy in stroke recognition (97%) and in assessing levels of
players in executing clear stokes (90.3%). Therefore, we
envision that the IoT framework presented in this paper will
play an important role in sports analysis where wrist actions are
important.
ACKNOWLEDGMENT
The authors thank CityU Men’s Badminton Team and South
China Athlete Association (SCAA) for their participation in
this study. The authors would also like to thank the Hong Kong
Innovation Technology Commission (Project no. UIM/326),
the Hong Kong Research Grants Council (Project no.
CityU/11213817), and the Shenzhen Overseas High Level
Talent (Peacock Plan) Program (grant number:
KQTD20140630154026047) for partially funding this research
project.
REFERENCES
[1] A. Lees, “Science and the major racket sports: A
review,” J. Sports Sci., vol. 21, no. 9, pp. 707–732,
2003.
[2] J. Wang and J. Moffit, “Teaching Badminton Based on
Student Skill Levels.,” Strateg., vol. 22, no. 6, pp.
14–18, 2009.
[3] M. Kwan, M. S. Andersen, C. L. Cheng, W. T. Tang,
and J. Rasmussen, “Investigation of high-speed
badminton racket kinematics by motion capture,”
Sport. Eng., vol. 13, no. 2, pp. 57–63, 2011.
[4] G. Zhu, Q. Huang, C. Xu, L. Xing, W. Gao, and H.
Yao, “Human behavior analysis for highlight ranking
in broadcast racket sports video,” IEEE Trans.
Multimed., vol. 9, no. 6, pp. 1167–1181, 2007.
[5] M. Nagasawa, Y. Hatori, M. Kakuta, T. Hayashi, and
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
9
Y. Sekine, “Smash motion analysis for badminton from
image,” in Proceedings of the IEEE Image Electronics
and Visual Computing Workshop, 2012, pp. 1–8.
[6] Z. Chen, R. Li, C. Ma, X. Li, X. Wang, and K. Zeng,
“3D vision based fast badminton localization with
prediction and error elimination for badminton robot,”
in Proceedings of the World Congress on Intelligent
Control and Automation (WCICA), 2016, pp.
3050–3055.
[7] I. Bisio, A. Delfino, F. Lavagetto, and A. Sciarrone,
“Enabling IoT for In-Home Rehabilitation:
Accelerometer Signals Classification Methods for
Activity and Movement Recognition,” IEEE Internet
Things J., vol. 4, no. 1, pp. 135–146, 2017.
[8] T. Mitsui, S. Tang, and S. Obana, “Support system for
improving golf swing by using wearable sensors,” in
2015 8th International Conference on Mobile
Computing and Ubiquitous Networking, ICMU 2015,
2015, pp. 100–101.
[9] J. Lee, “A Three-Dimensional Motion Anlaysis of
Horse Rider in Wireless Sensor Network
Environments,” Int. J. Adv. Comput. Sci. Appl., vol. 5,
no. 11, pp. 50–55, 2014.
[10] J. M. Jarning, K.-M. Mok, B. H. Hansen, and R. Bahr,
“Application of a tri-axial accelerometer to estimate
jump frequency in volleyball,” Sport. Biomech., vol.
3141, no. May, pp. 1–11, 2015.
[11] S. Akpinar, E. Devrilmez, and S. Kirazci,
“Coincidence-Anticipation Timing Requirements are
Different in Racket Sports,” Percept. Mot. Skills, vol.
115, no. 2, pp. 581–593, 2012.
[12] H. Ghasemzadeh and R. Jafari, “Body sensor networks
for baseball swing training: Coordination analysis of
human movements using motion transcripts,” in 2010
8th IEEE International Conference on Pervasive
Computing and Communications Workshops,
PERCOM Workshops 2010, 2010, pp. 792–795.
[13] A. Ahmadi, D. Rowlands, and D. A. James, “Towards a
wearable device for skill assessment and skill
acquisition of a tennis player during the first serve,”
Sport. Eng., vol. 2, no. 3–4, pp. 129–136, 2010.
[14] Y. Wang, Y. Zhao, R. H. M. Chan, and W. J. Li,
“Volleyball Skill Assessment using a Single Wearable
Micro Inertial Measurement Unit,” IEEE
Access,accepted.
[15] D. Semiconductor, “Bluetooth Low Energy 4.3,” 2016.
[Online]. Available:
https://www.dialog-semiconductor.com/sites/default/fi
les/da14583_ds_3v0.pdf.
[16] Z. Wang, C. Zhao, and M. Guo, “Badminton Stroke
Recognition and Skill Assessment Based on Body
Sensor Networks,” IEEE Trans. Human-Machine Syst.,
vol. 46, no. 5, pp. 1–14, 2016.
[17] Bosch Sensortec, “BMI160 Datasheet,” Prod. Specif.,
pp. 1–4, 2015.
[18] S. Jose, “MPU9250 Datasheet,” Prod. Specif., vol. 1,
no. 408, pp. 1–4, 2016.
[19] ST-Microelectronics, “LSM9DS1 Datasheet,” no.
August, pp. 1–74, 2013.
[20] Basler, “Basler ace User Manual,” 2011. [Online].
Available:
https://graftek.biz/system/files/2576/original/Basler_A
ce_USB_3.0_Manual.pdf?1479057814.
[21] G. Okeyo, L. Chen, H. Wang, and R. Sterritt,
“Dynamic sensor data segmentation for real-time
knowledge-driven activity recognition,” Pervasive
Mob. Comput., vol. 10, pp. 155–172, 2014.
[22] Y. Bengio and P. Vincent, “Out-of-Sample Extensions
for LLE , Isomap , MDS , Eigenmaps , and Spectral
Clustering,” NIPS Proc., pp. 1–10, 2003.
[23] W. M. Campbell, D. E. Sturim, and D. A. Reynolds,
“Support vector machines using GMM supervectors for
speaker verification,” IEEE Signal Process. Lett., vol.
13, no. 5, pp. 308–311, 2006.
[24] E. Approaches, A. Rocha, S. K. Goldenstein, and S.
Member, “Multiclass from Binary : Expanding
One-Versus-All,” IEEE Trans. Neural Networks
Learn. Syst., vol. 25, no. 2, pp. 289–302, 2014.
[25] S. S. Keerthi and S. K. Shevade, “SMO algorithm for
least-squares SVM formulations.,” Neural Comput.,
vol. 15, no. 2, pp. 487–507, 2003.
2327-4662 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2018.2837347, IEEE Internet ofThings Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) <
10
Yufan Wang received his BS degree in
communications engineering from Beijing
Jiaotong University (BJTU) in 2014,
where he was also a volleyball player in
the varsity volleyball team. His BJTU
team won the Chinese University
Volleyball Association (CUVA) Super
Cup Championship in Nanjing in 2010.
Since 2014, he has been pursuing his Ph. D. research work at
the City University of Hong Kong. He has represented CityU
to win the Volleyball Champion of the University Federation of
Hong Kong (USFHK) in 15/16; he has also represented Hong
Kong Men’s Volleyball Team to win the Volleyball Champion
of the Four-Regions (Mainland China, Taiwan, Hong Kong,
and Macau) in 15/16.
His current research interests are in the area of wearable
cyber physical devices, inertial measurement unit, artificial
intelligence, and sports motion analysis.
Meng Chen received his B. S. degree in
software engineering from Shenzhen
Univeristy, Shenzhen, China, in 2009. He
worked as a research assistant in City
University of Hong Kong’s Shenzhen
Reasearch Institute from 2013 to 2015, in
City University of Hong Kong from 2015
to 2016, and in Shenzhen Academy of
Robotics from 2016 to 2017. He has been pursuing his Ph. D.
degree at the City Univerisity of Hong Kong since 2017. His
research interests include wireless sensor networks, network
transmission protocols, and artificial intelligence.
Xinyu Wang is currently a Ph. D. student
at the Department of Computer Science,
City University of Hong Kong. He
received his B. E. degree from East China
Jiaotong University in 2011. He served as
a software engineer in Tencent during
2011 to 2013, developing information
security systems. He was a research
assistant at the City University of Hong Kong from 2013 to
2016, and worked on research topics related to the security of
applications and systems. His research interests include cloud
computing, network security, and big data.
Rosa H. M. Chan (M’01-SM’17) is
currently an Associate Professor in the
Department of Electronic Engineering at
City University of Hong Kong. She
received the B. Eng. (1st Hon.) degree in
Automation and Computer-Aided
Engineering from The Chinese University
of Hong Kong in 2003. She was later
awarded the Croucher Scholarship and Sir Edward Youde
Memorial Fellowship for Overseas Studies in 2004. She
received her Ph. D. degree in Biomedical Engineering in 2011
from the University of Southern California (USC), where she
also received her M. S. degrees in Biomedical Engineering,
Electrical Engineering, and Aerospace Engineering. Her
research interests include mathematical modeling of neural
systems, development of neural prostheses, and brain-machine
interface applications.
Wen J. Li (F’11) received his B. S. and
M. S. degrees in aerospace engineering
from the University of Southern
California (USC), in 1987 and 1989,
respectively, and his Ph. D. degree in
aerospace engineering from the University
of California, Los Angeles (UCLA), in
1997.
He is currently a chair professor in the Department of
Mechanical and Biomedical Engineering, City University of
Hong Kong. From September 1997 to October 2011, he was
with the Department of Mechanical and Automation
Engineering, The Chinese University of Hong Kong. His
industrial experience includes the Aerospace Corporation (El
Segundo, CA), NASA Jet Propulsion Laboratory (Pasadena,
CA), and Silicon Microstructures, Inc. (Fremont, CA). His
current research interests include intelligent cyber physical
sensors, super-resolution microscopy and nanoscale sensing
and manipulation. Dr. Li is an IEEE Fellow and served as the
President of the IEEE Nanotechnology Council in 2016 and
2017.