20/21 MSc ProjectsLEUNG Kwong Sak

(The projects described below is complete projects. Students can choose a part (subset) of them to work on. You can also proposed

your own project ideas to work on with my consent.)

Project 1Diagnosis of Sepsis using Machine learning and Deep Neural Networks

Background

Sepsis is severe disease that threatens patients’ lives. It develops when the immune system is over response to an infection, although the immune system prevents people from many infections. It is caused by the chemicals released by the immune system to fight against an infection. However these chemicals may go throughout all the body and cause inflammation. Severe situation of sepsis may cause septic shock. Based on the report from Centers for Disease Control and Prevention (CDC), over 1.5 million people get sepsis each year and 250,000 Americans get killed by the disease a year. It is the main cause of death in Intensive Care Unit (ICU).

The diagnose of sepsis is based on the symptoms and a series of medical tests. If sepsis is suspected, the doctor will order a blood test to check for complications like infection, clotting problems, abnormal liver or kidney function, decreased amount of oxygen, an imbalance in minerals called electrolytes that affect the amount of water in your body as well as the acidity of your blood. Depending on the results of the blood tests and the symptoms of the patients, the doctor may order other tests, including: a urine test, a wound secretion test, a mucus secretion test.

objective

In this project, we are going to use gene expression levels of long non-coding RNAs (lncRNAs) and message RNAs (mRNAs) to diagnose the sepsis for a patient. It may help doctors to diagnose the sepsis quicker. We try to find out bio-markers for sepsis so that hospital can diagnose sepsis by our bio-markers.

Methodology

LncRNAs bio-markers

We try to find out the lncRNA biomarkers that can classify sepsis well. We are now working on 9 datasets including 6 children and 3 adults. We use one of the datasets as training set and the other 8 for independent test. The first step is to filter all the lncRNAs with independent t test. Then only few lncRNAs are left with a specific p value. These lncRNAs are regarded as significant lncRNAs related to sepsis. In this project we do not use the expression value of one gene, but we use a combination of two genes instead. We try to find out whether the combinatorial two genes can classify sepsis more accurately. The significant lncRNAs are combined one by one as bio-markers. By using these combination of genes, we are able to classify sepsis in a relative high AUC. As comparison we use machine learning methods and Least absolute shrinkage and selection operator (LASSO).

mRNA bio-markers

Then we work on mRNAs. We are now working on 9 datasets including 6 children and 3 adults. We use 5 of the datasets with largest size as training set and the other 4 for independent test. The first step is to screen out all the mRNAs that related to the immune system. Then the independent t-test is applied to screen out the mRNAs that are significant related to sepsis. Then only few mRNAs are

1

left with a specific p value. These mRNAs are regarded as significant mRNAs related to sepsis. The significant mRNAs are combined one by one as bio-markers. By using these combination of mRNAs, we are able to classify sepsis in a relative high AUC. As comparison we use machine learning methods and Least absolute shrinkage and selection operator (LASSO).

As we can see the combination of genes (Com) gives a very high performance thus it probably can be applied to diagnosis.

Future work

The combination of genes has already shown some power. In the future we will combine it with the very heat machine learning methods called deep neural networks. We will first try the combination of genes and then use deep neural networks as classifier hoping that we can build a network for the bio-markers that improves the accuracy for diagnosing sepsis.

*We also plan to develop an accurate, fast and cheap clinical test for sepsis based on the promising results of this research project.

2

Project 2 Identification and prediction of associations

among human gut microbes, diseases and drugsIntroduction of on-going researchThe human gut harbours more than 100 trillion microorganisms, which are referred to as human gut microbiota. They provide more than 3 million genes, which are 150-fold-more genes than the human genome, encoding a rich enzyme resource. The human gut microbiota helps to digest certain foods that cannot be digested by the stomach and small intestine, produce metabolites and hormones necessary for human body, train our immune system, protect against colonization of pathogens, and contribute to larger systemic effects, e.g. “gut-brain axis”. The taxonomic composition of human gut microbiota demonstrates high inter-individual diversity. Nevertheless, it is found in some culture-based studies that a core microbiota is shared by all healthy people, and shifting away of the gut microbiota from its healthy state, is associated to many diseases, including obesity, cardiovascular disease, cancers, major depression, Rapid-eye-movement sleep Behaviour Disorder (RBD) and inflammatory bowel disease (IBD). In this project, we will collect the human gut microbiota data and associations among human gut microbes, microbe products, drugs, diseases and pathways both manually and automatically. Then the taxonomic composition profiles of human gut microbiota and longitudinal profiles will be complied and used to identify the co-occurring and co-varying gut microbe groups, respectively, and investigate their associations with diseases. The associations obtained and identified will be integrated to construct heterogeneous networks and train the embeddings, which maps all the attributes into a common space using graph neural network techniques. The embeddings trained are domain specific and capture the underlying features of individual gut microbes, drugs and diseases. Finally, the associations among gut microbes, drugs and diseases and their embeddings will be modelled into the tensor decomposition framework, and novel predictions of associations will be discovered.

Research Plan and MethodologyTask 1: Collection of human gut microbiota and association data

The development of high-throughput metagenomic sequencing techniques promotes the study of human gut microbiome. Large amounts of human gut microbiome data and associations of gut microbes have been discovered. Although some databases have been constructed integrating discovered knowledge from different studies, e.g. Human Microbe-Disease Association Database (HMDAD), they are far from complete. We are going to collect large-scale data of human gut microbiome and associations among human gut microbes, microbe products, diseases, pathways, drugs, etc. Some existing datasets will be manually collected and integrated, while the associations described in the literature text will be automatically extracted using text mining techniques (Fig. 1a).

Task 2: Analysis of associations between diseases and human gut microbiotaIt is reported by various studies that some diseases are associated with human gut microbiota, including reduced or increased abundance of particular taxa, and significant shift of the community. However, the changes of human gut microbiota in diseases might occur in not only individual microbes, but also in several co-varied microbes, since there might be particular relationships among the microbes in a community. Thereby, we are going to identify co-occurring groups of human gut microbiota in both healthy and diseased individuals respectively to investigate the associations of these microbiota groups with diseases using tensor decomposition (Fig. 1b).

Task 3: Longitudinal analysis of human gut microbiota in IBD

3

We will construct two three-dimensional tensors of longitudinal data of human gut microbiota of healthy individuals and IBD patients, respectively. The dimensions of the each tensor represent the samples, gut microbes and sampling time points, respectively. Each tensor will be decomposed into three low-rank factor matrices with sparsity constraint on the gut microbe loading matrix. By inducing sparse priors in the microbe loading matrix, we will get co-varying groups of human gut microbes. The associations between co-varying groups and diseases, the carriage of gut microbe co-varying groups and specific microbes, and dynamics of gut microbe co-varying groups and related pathways will be analysed (Fig. 1c).

Task 4: Characterization of human gut microbes, diseases and drugs using heterogeneous network embedding

All of the associations among human gut microbes, microbe products, diseases, pathways and drugs collected in task 1, as well as the association between the gut microbe and diseases discovered in this study (in tasks 2 and 3), will be integrated into a heterogeneous network. With the list of binary and trinary potential associations, we will use heterogeneous network embedding space to encode the semantics and relationship between drugs, proteins and diseases (Fig. 1d). We will optimize the embeddings with respect to the potential associations, such that the similarity of the embeddings in the embedding space reflects the true similarity of these medical concepts in real life. The embedding captures the topological information of the nodes in the network. Similar embedding vectors indicate the similarity of the nodes in respect to their associations in the network. Thereby, we are going to investigate the predictive potential of the embedding vectors of gut microbes, diseases and drugs using some basic classification models, e.g. SVM, to predict the associations among the gut microbes, diseases and drugs

Task 5: Prediction of associations among human gut microbes, diseases and drugs based on tensor decomposition

We will use the collected and discovered associations and extracted embeddings in the previous steps to predict new associations among human gut microbes, diseases and drugs via tensor decomposition (Fig. 1e). The loading matrices reflect the functional patterns of the drugs, gut microbes and diseases. We will cluster the drugs, gut microbes and diseases based on their corresponding vectors in the loading matrices into functional groups. Finally, the predicted associations will be further validated using external data.

PublicationsR Wang, S Li, L Cheng, MH Wong, KS LeungPredicting associations among drugs, targets and diseases by tensor decomposition for drug repositioning; BMC bioinformatics 20 (26), 628; 2019

R Wang, S Li, MH Wong, KS Leung Drug-protein-disease association prediction and drug repositioning based on tensor decomposition2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

4

Figure 1. Workflow of the tasks. a Task 1, collection of human gut microbiota and association data. b Task 2, analysis of associations between diseases and human gut microbiota. c Task 3, longitudinal analysis of human gut microbiota in Inflammatory Bowel Disease (IBD). d Task 4, characterization of human gut microbes, diseases and drugs using heterogeneous network embedding. e Task 5, prediction of associations among human gut microbes, diseases and drugs based on tensor decomposition. ωmain, ω1, ω2, ω3 and ωreg are weights for different parts in the objective functions. The meaning of the other variables in the equations are shown in the figure.

5

Project 3Evaluation of Low-Cost PM Sensors and Development of an App and Wearable PM

Exposure Monitoring System

Regarding the many aspects that could be beneficial from the paradigm shift in PM exposure monitoring, this study focuses on addressing the significant biases or errors, which are introduced by the coarse and inadequate PM exposure assessments from conventional approaches, in epidemiological research studying specific pollution-health association. A pilot study investigating the association between the daily PM2.5 exposure levels and the sub-clinical atherosclerosis of the children and adults in Hong Kong and Chongqing was conducted in this research. In order to assess the individual’s everyday PM2.5 exposure level, a dedicated system embodying a wearable monitoring device carried by each subject was developed. Various sensors that are capable of detecting the presence of PM2.5 were evaluated to identify the most suitable one for constructing the wearable monitoring device. Ten wearable devices equipped with the selected sensor were assembled and evaluated. Initial results demonstrated that the developed system is suitable for its intended application. One hundred and twenty more monitoring devices have been manufactured and will be delivered to the subjects in early 2020.Firstly, three PM sensors were selected empirically for performance evaluation by comparing the time synchronized data pairs acquired by these sensors and two collocated reference instruments. Eventually, the PMS-A003 sensor was selected with a comprehensive consideration for constructing the wearable device utilized in the personal PM exposure monitoring system developed in this study.Secondly, a device named Wearable Particle Information Node (WePIN) for measuring personal PM exposure levels was developed and evaluated. The WePIN is compact and lightweight, and suitable for wearable scenario. It also equips with a micro SD card for storing data locally and a Bluetooth LE link for communicating with the mobile application. The WePIN together with the mobile application for data visualization and the back-end server for data management compose the devoted personal PM exposure monitoring system. We are currently drafting a manuscript on this research and we plan to submit the manuscript by March.

Figure 1: 3D images of the PM devices

6

Project 4E-health System with Wearable Electrocardiography (ECG) Device and Atrial

Fibrillation (AF) Detection

In this research, a personalized healthcare system, including a wearable ECG device for acquiring users’ electrocardiography (ECG) data, a mobile application and web interface for data visualization, and a ResNet base Atrial Fibrillation (AF) detector, was developed. This is an on-going research and we have refined the design of the wearable ECG device and made five of them for trial use in Prince of Wales Hospital; secondly we refined the mobile application for better visualization and continuous monitoring; thirdly we improved the AF detector to achieve 84.37% mean F1 measure.Nowadays, almost everyone has his/her own smart phone. Moreover, thanks to the advancing Micro-electromechanical Systems (MEMS) technologies, bio-related sensors are becoming more power- and cost-efficient while having reduced physical dimensions. Utilizing these tiny sensors in wearable devices and use the smart phones for data collection is the current trend in IoT based e-health systems.Although many vital parameters, including blood pressure, RESP, glucose level, electroencephalography (EEG), etc., can be monitored by IoT based e-health system, we focus on the ECG information in our system because we are interested in the Heart Rate Variability (HRV) measurements of users. HRV measures the time interval variations of adjacent heartbeats. Physiologically, HRV is caused by the sympathetic and parasympathetic nervous system. It is reported that diabetes, psychiatric disorders, and cardiac disorders are related to the decreased HRV.In this research, a personalized health-care system based on ECG signals was developed. This system provides a convenient and light way for users to keep track of their health condition via wearable devices and smart phones. Also, the system enables the health advisors to monitor and analyze their clients’ ECG information remotely through a web interface. We proposed a deep residual network (ResNet) based AF detector for classification of short single lead ECG recordings provided by the PhysioNet/Cinc Challenge 2017 and achieved 84.37% mean F1 measure.We are planning to further improve the e-Health platform, Mobile App and the DNN ..

Figure 2: ECG system architecture and the 3D images of the ECG device.

7

Project 5

Two ways of tracking hand gestures for Diagnostics - Smart Glove vs RGB Camera by Computer Vision

1. Smart Glove

Smart Glove is built with different hardware components - Inertial Measurement Unit (IMU) for hand poses, flex sensors for finger gestures and the aruco marker for more accurate tracking.

With this glove, we can measure the bending of the finger, hence the finger tapping sequences. In addition, with the accelerometers and markers, we can measure the movement of the hand in 6 degrees of freedom.

We will adapt this glove for computer/human interfaces, and monitoring and measuring hand and finger movements of patients for medical diagnosis purposes.

8

2. OpenPose and Other Deep Learning Approaches using single RGB camera

First, Openpose only requires 1 RGB camera to work. After capturing the images, it will use deep learning to extract the hands and output the hand poses with each point on joints. With this information, we can capture the angles of the fingers and then analyze the tapping sequences.

Second, if 3-D information is required, we will utilize another deep learning framework to recover the 3-D hand pose from a single image.

References:[1]T. Simon, H. Joo, I. Matthews, and Y. Sheikh, “Hand keypoint detection in single images using multiview bootstrapping,” presented at the Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 1145–1153.

[2]C. Zimmermann and T. Brox, “Learning to estimate 3d hand pose from single RGB images,” presented at the Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 4903–4911.

9

Project 6Smartphone-based indoor positioning system

IntroductionThe deployment of global navigation satellite systems (GNSSs) such as GPS, have enabled accurate, ubiquitous positioning in outdoor environments. However, these systems are not suitable for indoor use because of signal blockage by building materials. In the era of Internet of things (IoT), and with the technological advancements of cells phones, wearables, and wireless networks, positioning, navigation and location-based services in indoor environments are in high demand in a variety of context.

An indoor positioning system (IPS) or indoor localization system (ILS) is a systems used to locate objects or people inside an indoor environment by determining either 1) the physical position with respect to some coordination system, such as earth-centered earth-fixed (ECEF), or 2) the symbolic location, such as kitchen, toilet. Real-world applications depending on IPS are many. To name a few, one can consider the wayfinding and navigation for the visually impaired in shopping mall, the locating of products in a warehouse, the exit finding for large car park, and etc.

However, there is no overall solution based on a single technology which is cheap, accurate and hassle-free. Most of the current systems either require dedicated local infrastructure and customized mobile devices, or require time consuming scene analysis from time to time.

In the past two years, we reviewed some of the existing smartphone-based indoor positioning systems, evaluated a widely used method called fingerprinting on our test sites, and proposed a novel hybrid system with the use of deep learning. In the coming years, we are going to use novel deep learning models and techniques for a hybrid IPS.

Research Plan 2020 Q3&Q4 - Develop an Indoor Positioning GIS & multimodal fingerprint IPS 2021 Q1&Q2 - Integrate the previous IPS with Pedestrian Dead Reckoning (PDR) 2021 Q3&Q4 - Investigate Bluetooth Low Energy (BLE) Angle of Arrival (AoA) based

10

Project 7Objects Tracking AI system using LiDAR Sensor

Major Goals of the System and ApplicationsThe major goals of the system are:

To develop a robust AI system to track human or other object’s movements such as old-people falling without any privacy issue. (Lidar cannot review any face nor body shape information.)

To identify a human or an object from LiDAR Raw data. To develop a fast and precise identification method to enable real-time tracking

monitoring.

Possible Application: Security and safety monitoring of old people or patients with mental problems. Be a part of sensor on Smart Lamppost. Improve the privacy issue compared with using cameras. LiDAR operates without any visible light source.

System FeaturesSeveral Outstanding features of our system:

Integrate LiDAR Sensor and a computer to build up a real-time and precise tracking system.

A high-end LiDAR Sensor to ensure large sensing range and precise data collection A generic Neural Network AI system to recognize multiple data format of LiDAR The AI system can deploy in a low power consumption computer system.

System ArchitectureIn the system, we have an interfacing and a computation components. The interfacing component provides the following features:

Transform the sensor raw data into an optimized computation format. Display the result to the user by a local display system and delivering the system

result to the remote web interface/server.The computation component provides the following features:

Analyze the sensor data by an AI Deep Neural Network System. Separated in 2 systems: Training and inference computer systems. The training computer system will use the sensor data to retrain the AI model used

by the inference to increase the prediction/analysis precision.

11

.Project Deliverables1. An AI System for Object recognition and tracking.2. A user Interface and system API for data access.3. A prototype to test in an old people’s home.

12

Project 8Dementia, RBD and Parkinson Disease Automatic Diagnostic

System

Major Goals of the System and ApplicationsThe major goals of the system are:

To develop a robust AI system to diagnose the patient from Microsoft Azure Kinect skeleton tracking data, voice, finger and hand movement data.

To automate the tradition manual diagnostic to maximize the diagnostic precision and reduce doctor’s workload.

Possible Applications: Automatic diagnostic system and workflow without human involvement. Building up a disease database for increasing the precision of the AI model

iteratively.

System FeaturesSeveral Outstanding features of our system:

Different data handlers pre-process the raw data from the sensor including Skeleton tracking data, voice data and hand movement video data.

Database creator to create a disease feature database. The diagnostic system using the new disease features from the database to improve

the prediction precision. The individual AI model generates a diagnostic result from the 3 separate sets of

data respectively including the Skeleton tracking data, the voice data and the hand movement video data.

A final AI model will use the diagnostic reports/results from the 3 independent AI models to generate the final overall diagnostic report/result.

System ArchitectureIn the system, there have 3 AI subsystems to analyze 3 types of data stream

separately. Sensor interface will transform the sensor raw data to an AI model optimized data

format. 3 Separate AI subsystem will analyze the data independently. Central AI system will generate a patient report with detailed results to the doctor

from the results of the AI subsystems. The database creator will record the patient data. These data can increase the

precision of the AI systems iteratively.

13

Project Deliverables1. An AI System for Dementia Disease diagnostic using 3 types of data individually and

integratively.2. A database of the patients.3. A prototype used for diagnosing mental health patients in Proposal 5.

14

Project 9

Study 1. Developing a novel digital phenotyping panel to systematically assess and monitor the prodromal motor and non-motor features of -synucleinopathy⍺The rapid development of the digital and wearable technology makes the objective, prospective non-invasive and continuous assessment of motor and non-motor features feasible and practical. This study aims to develop a novel application app for smart-phone and incorporate other novel ambulatory digital multi-modal assessment devices to assess the motor and non-motor features of early ⍺-synucleinopathy during both daytime (ambulatory technology assisted assessment) and night-time (in vivo home monitoring to document REM sleep behaviors and injuries). Development of digital assessment (Figure 1)a) App based tasks for smart phone: The app developed in React Native can be run on both Android and iOS phones to collect the motor and nonmotor features using its 3D accelerometer, gyroscope, GPS, timer, and video functions as appropriate. The app will have different screens for different functions for various tasks. The collected anonymous data will be encrypted and automatically uploaded to a cloud server for storage and data processing. More advanced monitoring device (a smart glove being developed by our group, Figure 2) will be used to measure subtle hand and finger movements. The tasks include voice recording, facial expression, rest and postural tremor, finger tap, hand alternative movement, eye tracking.b)Gait analysis: A mobile device called Microsoft Azure Kinect™ applying a laser scanning can track and record movement of human skeleton using the IR and RGB cameras. c) Rest-activity rhythm: Actigraphy device will be placed on the wrist to record the activity data during both daytime and night-time for 7 consecutive days. d)Bowel movement: A device will be developed to record the participant's bowel sounds.e) Video-polysomnography assessment and Microsoft Azure Kinect™: The development of ambulatory night polysomnography and video monitoring (match-stick analysis) would allow a quantitative capture of nocturnal vocalization, movement and injuries at home. Data processing and analysis The relationship of the ambulatory data and other neuropsychiatric assessments will be analyzed both individually and integratively using machine learning and novel deep neural networks (DNN) to perform deep phenotyping and prediction of conversion and trajectory course of RBD.

15

Figure 1. Summarizing the plans for this project

Figure 2. Smart Glove: Finger and hand movement monitoring and measurment system

16

17

Project 10Mindful Flourishing-Promoting college students’ mental well-being

through cultivation of mindfulness with online and offline approaches

Prof Winnie Mak, Psychology Department (PI); KS Leung(Co-I)

Brief Summary: In view of the alarming mental health status of college students in Hong Kong, a proactive approach that not only addresses their symptoms of psychological distress but also builds their capacity to understand their mental health needs and to flourish is necessary for long-term benefits. As college students are a unique group of emerging adults who are going through life transitions, using peers in the mental health promotion programs may serve as important role models for college students to emulate. Furthermore, with the high penetration of social media and smartphone technology in their daily lives, they are excellent means to raise their mental health awareness and practices. The present project proposes to use a parallel, two-pronged approach of online and offline mindfulness training to create a mindfulness community as well as to engage college students in practicing mindfulness and promoting their mental well-being. Through the immersion of college students in mindfulness practice with a peer-led mindfulness ambassador program, taster and interest-based mindfulness workshops, as well as an ecological momentary intervention mobile application and Facebook, we aim to engage college students as active mental health promotion agents to their fellow students and sustain their motivation to integrate and practice mindfulness in their daily lives.

 ObjectiveOnline system of the Mindful Flourishing project aims to provide mindfulness intervention in an environment that is natural to the participants, and help users understand the relations between their mindfulness, mood and stressors at different time and place during their daily lives, and therefore, take better care of their mental well-being. Mindful Flourishing aims to promote the idea of mindfulness on college campuses.

System DesignMain functions of mindful flourishing application are:

Record Now: record physical and mental situation of user, record surrounding of user, improve awareness of oneself and surrounding, get to know one’s inner need;

Focus Now: daily mindfulness practice, improve perceptivity in life; and Satisfy Now: listen to one’s inner voice, reply and satisfy inner need

System Architecture

The front-end components consist of the mindful flourishing mobile app and web interface of content management system(CMS) of the app. Database with a CMS server with app APIs serve as the back-end of the system. Data communication

18

between front-end components and database goes through CMS server and app APIs.

Figure 1. System Architecture

DeliverableA mobile APP (for both Android/IOS system) with:

Daily mindfulness practices for improving mental awareness, focus and health;

Articles and videos related to mindfulness for exploration; Records of taken practices for tracking thinking and behavioural patterns; Simple mental questionnaires for better understanding of oneself; Award system for app gamification; and Basic user account management

A web-based content management system for:

Managing the practice, exploration, questionnaire and other contents available in the APP;

Managing some big sized display elements (e.g. intro videos) in the APP; and Output csv table for research data collecting;

A relational database for:

Storing user information (e.g. basic information, online time) Storing user practice and exploration records

19

Project 11Development of An Online Application for Positive Education

ResearchProf CHIU Chi Yue, Dean of Social Science; KS Leung; Donald Cheung, senior RA

Major Goals of the Platform and Its Online ApplicationThe major goals of the platform and its online application are:

To develop an online application to assess student behaviors and the outcomes of positive education intervention; and

To enable real-time and near-real-time iterative tracking of and feedback to students to facilitate the design of effective teaching pedagogy for positive education.

The sub-goals are: To monitor the patterns and dynamics of learning behaviors of students across years; To evaluate the efficiency and effectiveness of educational practices in positive

education; To identify students with special educational needs; and To study the social relationships among students across years.

System FeaturesSeveral desirable features of this online application are:

It is a general all-in-one platform that is capable of real-time, near-real-time and long-term monitoring and tracking of the learning behavior and well-being of students in a school, which is very useful for developing and improving the curricula, practices, teaching and learning materials for positive education.

Multimodal data will be automatically collected to facilitated long-term research on positive education.

The assessment tools are age-appropriate; it can be used in assessing students from Primary 1 to Secondary 6.

It includes a multimedia web management interface for teachers and researchers to set up surveys and download data for easy analysis.

It has a back-end that manages contents and databases with simple visualization tools.

It supports multimodal responses. It allows easy adjustment of the interface to cater for the different levels of motivation

and cognitive maturity of young children in test taking contexts. It supports tracking data longitudinally. It supports support multilevel data linkage, e.g., students nested within a class or a

teacher. It allows automatic compilation of data from different sources for the same

respondent (e.g., assessment of the same student through self-reports, teacher ratings, parent ratings).

System ArchitectureIn our system, the front-end component (web management interface) is designed for easy access through PC and tablet to provide the following functions:

Design and administrate multimedia questionnaires; Display questionnaires for data collection; and Visualize the raw data via simple charts and graphs.

In our system, all back-end components are hosted in a dedicated server. Most of the data collected will be stored as tables in a SQL database which is ideal to perform advance query

20

between tables. Relationships among students will be stored as a network in a graph database since graph databases are well optimized to perform query on a network than a SQL database. We will include some very basic statistical and visualization tools into our system such that teachers, school management teams and researchers can perform data analysis through the web management interface.

Figure 2. System Architecture

Project Deliverables1. A web-based management interface for:

Managing the questionnaire and other contents available in the platform; Managing the privileges (access control) of user accounts; Providing basic query and tools for teachers to study the data collected; and Providing basic data visualization tools to assist researchers in understanding long-

term trends in the future.2. An efficient database system:

A graph database for storing relationships among students, teachers and parents; and

A relational database for storing other information (e.g. questionnaire, metadata of students)

Project 12

21

Personalized 24-hour Environmental Intelligent Sensor System

A personalized 24-hour environmental intelligent sensor system will be developed. This system first measures the subject’s 24-hour light (intensity and wavelength), noise (intensity and frequency), and air pollution (PM2.5 and CO2) exposures utilizing wearable sensors, indoor and outdoor stationary monitors. In addition, the sleep-wake patterns of each subject measured by a commercial wrist-worn activity tracker (e.g., the “Actiwatch”) are imported into the system. With the personalized light, noise, and air pollution exposures, and the sleep-wake patterns, this system will eventually provide each subject with risk assessment for sleep and emotion problems.

Overall System Architecture 

A comprehensive hardware and software system as shown in Figure 1, including wearable and stationary devices, mobile App, and cloud server, for monitoring the light, noise, and air pollution levels in both personal and street levels will be developed. The wearable device “Brooch” measures a subject’s daily personal light and noise exposures and transmits such information to the subject’s smartphone through Bluetooth Low Energy (BLE) in real-time. Another wearable device “PIN” (Pollution Information Node) measures the subject’s daily personal PM2.5 and CO2 exposures and transmits such information to the subject’s smartphone through BLE in real-time. The mobile App pre-processes the data acquired, which are tagged with time and location information before forwarding to the cloud server. The stationary device “Pod” charges the “Brooch” and “PIN” when the subject is sleeping and uses them to measure the vicinity light and noise intensities and the PM2.5 and CO2 concentrations inside the subject’s bedroom. The “Pod” further pre-processes the data acquired and tagged with time and location (only once when moved) information and forwards to the server. The weatherproof stationary device “PolePod”, which can be deployed on an existing lamp pole or other suitable positions in residential areas, measures the light and noise intensities and the PM2.5 and CO2concentrations outdoor. The “PolePod” pre-processes the data acquired locally, which are tagged with time and location information. All pre-processed data tagged with time and/or location information are transmitted to the cloud server through Wi-Fi or cellular network for archiving and further analysis. The subject’s sleep-wake patterns, which are measured by the activity tracker (e.g., the “Actiwatch”) and downloaded to a PC through USB, are uploaded to the cloud server as well using Ethernet. If a tracker like Fitbit is used, the data can be transmitted through the same BLE-mobile phone-server path mentioned.

22

Figure 1: Overall System architecture.

Sensors and Measured Parameters

The sensors and measured parameters are specified in sub-sections 1-4 and Table 1:

1. Light Sensor

Light sensors are photoelectric devices that convert light energy (photons) whether visible or infrared light into an electrical (electrons) signal. For typical light pollution assessment, a light sensor equipped with an optical filter (allowing light with wavelength 380nm~740nm to pass through) is used to measure the intensity of visual light.

2. Red (R), Green (G), Blue (B), and Infrared (IR) Sensors

RGB and IR sensors are special kinds of light sensors. For RGB sensors, they only measure the intensity of light with wavelength around 615nm, 530nm, and 460nm, respectively. For IR sensors, they measure the intensity of light with wavelength 700nm~1mm.

3. Noise Senor

Noise sensors (microphones) are acoustic electric transducers or sensors that detect sound signals and convert them into electrical signals. For typical noise pollution assessment, a noise sensor is used to measure the acoustic intensity.

4. Carbon Dioxide (CO2) Sensor

Carbon dioxide sensors are nondispersive infrared gas sensors (NDIR) that convert the optical signal decay (absorbed by CO2 molecules) into the

23

concentration of the CO2 gas.

5. Particulate Matter (PM) Sensor

Multiple operation principles are available for PM sensors while the light-scattering is the most applicable principle utilized in commercial low-cost and compact PM sensors. This kind of sensor detects the mass concentration of the PM presented by analyzing the light intensity and pattern scattered by the particles.

6. Temperature, Humidity, and Pressure Sensors

These sensors are used to measure the temperature, relative humidity, and atmospheric pressure. One of the usages of this information is to compensate the other sensors’ drifts caused by change of ambient conditions. These parameters may also affect sleep and mood.

Table 1: Sensors and their measuring parameters.

Sensor TypeParameters Measured

Light Sensor Light Intensity (with wavelength from 380nm to 740nm); Unit: lmRGB & IR Sensors Light Intensities of Red (wavelength 615nm), Green (wavelength

530nm),

Blue (wavelength 460nm), and IR Lights (wavelength 700nm to 1mm); Unit: lm

Noise Sensor Acoustic Intensity; Unit: dBCO2 Sensor CO2 Concentration; Unit: ppmPM Sensor Particulate Matter Mass Concentration; Unit: µg/m3Temperature, Humidity,

and Pressure Sensors

Temperature (Unit: ⁰C), Relative Humidity (Unit: %), and

Atmospheric Pressure (Unit: Pa)

 

Devices

Two wearable devices (“Brooch” and “PIN”) for measuring the subject’s light and noise exposures and air pollution exposure (i.e., PM2.5 and CO2), respectively, will be developed and deployed. A stationary device (“Pod”), which acts as the wearable devices’ charging dock and uses them to measure the vicinity light and noise intensities and the air pollution levels while the subject is sleeping, will also be developed. Moreover, a stationary device (“PolePod”), which measures the light and noise intensities and the air pollution levels at street level, will be developed and deployed on existing lamp poles or other suitable locations in residential areas. A wrist-worn activity tracker (e.g., Actiwatch) will also 24

be adopted to measure the subject’s sleep-wake patterns.

1. Brooch

In order to monitor the subject’s personal light and noise exposures, a wearable device called “Brooch” as shown in Figure 2 will be developed. The “Brooch” is equipped with a light sensor, an RBG-IR sensor, a noise sensor, a temperature-humidity sensor, and a pressure sensor. The intensities of visual light (wavelength 380nm~740nm) are measured by the light sensor; and the intensities of red light (615nm), green light (530nm), blue light (460mm), and IR light (700nm~1mm) are measured by the RGB-IR sensor. The intensities of noise are measured by the noise sensor. The temperature-humidity sensor and pressure sensor measure the ambient conditions that used to compensate the light, RGB-IR, and noise sensors’ drifts caused by change of ambient conditions. Each subject will wear a “Brooch” to monitor his or her personal exposures to light (with different wavelengths) and noise. The data acquired will be transmitted to the subject’s smartphone in real-time through BLE. The “Brooch” is battery powered and should last for 24 hours for daily monitoring.

Figure 2: Scenario with subject wearing the Brooch.

25