using infrared array devices in smart home observation and diagnostics

8/20/2019 Using Infrared Array Devices in Smart Home Observation and Diagnostics

http://slidepdf.com/reader/full/using-infrared-array-devices-in-smart-home-observation-and-diagnostics 1/5



IPASJ International Journal of Electronics & Communication (IIJEC) Web Site: http://www.ipasj.org/IIJEC/IIJEC.htm

A Publisher for Research Motivation........ Email: [email protected]

Volume 4, Issue 1, January 2016 ISSN 2321-5984

Volume 4, Issue 1, January 2016 Page 10

stabilized amplifier and integrated fast ADC, a Proportional To Absolute Temperature (PTAT) sensor is integrated to

measure the ambient temperature of the sensor chip. The output data of both IR and PTAT sensors are stored in the

internal RAM (16-bit result of IR measurement for each individual sensor (64 words) and 16-bit result of PTAT sensor)

and are accessible through I2C digital interface that supports clock speed up to 1MHz. The microcontroller connected to

the MLX90620 sensor can read the different RAM data and, based on the calibration data stored in the internalEEPROM memory (256x8) of the sensor, compensate for difference between sensors to build up a thermal image, or

calculate the temperature at each spot of the imaged scene.

Figure 1 Basic structure of the system

Open source hardware platform PIC32-MAXI-WEB produced by Olimex is a development board equipped with a high

performance microcontroller (PIC32MX795F512L) [7]. It is easy for programing and supports the needed

communication protocols and the necessary ports. It has embedded 100Mbit Ethernet module that allows to develop

easily Ethernet connectivity applications. Along with the PIC KIT3 programmer there are some other useful features of

the microcontroller which are in help in the process of program development and debugging, like the 240x320 TFT

LCD with touchscreen, trimmer potentiometer and the buttons and leads.

The microcontroller periodically samples the sensor, gathers the information from the 64 infrared sensors (pixels) ant

the temperature sensor, then performs calculations for each pixel individually using the result from the temperature

sensor and the calibration constants obtained in the process of sensor manufacturing which are stored in its internal

EEPROM memory. The raw data from the sensor first are converted to human readable values, which in turn are

translated to values for red, green and blue for each corresponding temperature with the help of a look-up table of

values. The RGB values are stored in the microcontroller memory. Those values could be sent through a network to a

client with the help of the Microchip TCP/IP Stack. The stack has all the necessary network management

responsibilities and facilitates the process of establishing connection with the IR sensor via Internet. After the stack is

configured a small web application which is stored in the memory of the microcontroller services all external requests.

Upon request from the client side the developed web application provides the RGB data. The microcontroller is connected to the network using RJ45 connector (Ethernet LAN). The microcontroller could be

accessed via web browser through the router using the fixed IP address. It is possible to enter a mode where

measurements are taken only at the user’s request. In this way the power consumption of the IR sensor is greatly

decreased and the microcontroller has resources to do some other useful work.

3. DATA PROCESSING

The data gathering and further processing is described in this section. As it was mentioned earlier inside of the TO-39

package the MLX90620 sensor contains the 16x4 IR array, thermometer, 65 words (16 bit) of RAM, 256 words of

EEPROM and two internal configuration registers. After the microcontroller establishes communication with the

sensor and initializes the configuration registers the sensor starts measuring the temperature of its field of view and the

proportional to ambient temperature of the package. The data is then being read periodically by the microcontroller

using the I2C digital interface protocol with clock speed supported from the sensor up to 1MHz. In order to extract

some human readable data from the raw 16 bit sequence of zeros and ones obtained from each IR pixel, the

microcontroller has to perform some heavyweight calculations, such as: offset compensation, thermal gradient

compensation, pixel to pixel normalization and emissivity compensation. This step could potentially become a threat

for a bottleneck if a microcontroller with lower computation power and limit of the frames per second visualization wasselected, despite the refresh rate of the sensor being of 0,5Hz to 512 Hz. At that point the raw data is an array of 64







floating point values that represent the temperature that each pixel “sees”. That itself is useful information, if the goal

is to target the temperature of a certain point and simply display it as a floating point value or to alert the system and

start executing some other application specific functions.

In the applications where the goal is to construct a thermal picture based on the values from the array it is necessary to

assign RGB values to each pixel. This process is very flexible as there are many ways of doing that. A verystraightforward approach is used in our system, because the colors of the table are chosen as follows: blue for the low

temperatures, red for the hot, with the transition from light-blue, blue-green, green, yellow and orange in between.

However, the range of the temperature that is measured is selected by the user, depending on the specific preferences

and the current application. The number of the entries of the RGB look-up table and each separate value for red, green

and blue is fixed. Determining which entry of the look-up table to pick is done by determining where inside the user

selected range is the value of the current pixel – if it is below the low value we assume RGB values as if it is the lowest

in the table. If it is above the high boundary we assign the highest RGB values from the table. Anything value in

between means assigning RGB values for the corresponding temperature. This data processing is done by using the

following two relations:

S = (H - L) / N, (1)

where: S is the step, temperature value that separates two entries in the table, H is the high boundary, L is the low

boundary and N is the number of entries in the table.

T = (P – L) / S, (2)

where T is rounded to integer and is the entry of the look-up table and P is the temperature of the pixel.

An option exists that allows the high and low boundaries to be selected automatically for each frame based on the

current highest and lowest measured temperature. By constantly collecting information for each pixel we can construct

a real time representation of the temperature of the surrounding area. Sometimes in the manufacturing process or due

to errors in the calibration process errors could visibly appear in some of the pixels. A practical approach of correcting

these types of errors is applied in the developed system that corrects those pixels and brings them closer to the real

value with minimal deviation. In that case a special calibration sequence is started for the damaged pixel. The

calibration procedure consists of taking measurements from object with consistent temperature up close so that each

pixel should see the same value. The value seen from the good pixels is considered the correct one and the value of the

bad pixel is considered with error. At least two measurements need to be taken for two different temperatures and allthe values are saved. Then the values are plotted on a Cartesian plane and the function y = f(x), where f(x) is a

polynomial so that when we plug in for x we get the approximated real value for y. As higher the degree of the

polynomial as closer will get approximation but also will add more complex calculation load and could slow the system

down if it gets too high. Finally, the information from the 64 pixels is stored in the microcontroller memory in the form

of RGB values. An example of thermal picture constructed from the RGB values and the pixels position in the whole

field of view is shown on Figure 2.

Figure 2 Thermal picture and the pixels position in the sensor field of view







4. APPLICATION OF THE IR SENSOR IN SMART HOME DIAGNOSTICS

Having a way of obtaining a thermal image of a certain area presents a wide variety of applications in smart home,

building observation and diagnostics. Very useful and attractive applications in building diagnostics are thermal loss

and water leakage. The high precision measurement could detect the small temperature gradient throughout the wall,floor or ceiling and determine the source of thermal leakage. In case of water leakage IR sensor could detect the source

even before the water breaks to the surface and damages the interior. In order to demonstrate these possible applications

of our solution we have aligned spatially and matched the image from the web camera and the thermal image from the

IR array sensor. The results of this procedure for the two described cases for building diagnostics are presented on the

next figures. On Figure 3 an example for application of MLX90620 sensor in detection of thermal loss is shown, while

Figure 4 shows an example for application of the sensor in detection of water leakage.

Figure 3 Example for application of MLX90620 sensor in detection of thermal loss

Figure 4 Example for application of MLX90620 sensor in detection of water leakage

In general, with the employment of IR array sensors more complete information for the parameters of certain objects

and processes in home environment could be obtained. This will increase the understanding and simplifies the process

of picking the best reaction to the changes of important objects’ parameters. Additionally, the usage of infrared sensors

instead of simple everyday camera will help to avoid privacy invasion issues but still maintain the necessary

observation duties.

5. CONCLUSION

In this paper a low-cost solution for smart home diagnostics and observation with the use of a new IR array sensor isdescribed. Based on an open source hardware microcontroller and an inexpensive IR sensor the developed system is

considered to be with reasonable cost and affordable for the potential users in their everyday environment. The paper

demonstrates the benefits of having an infrared sensor as a part of the developed monitoring module for detection of

thermal losses and water leakage in a system for smart home diagnostics.

In the future work the IR sensor will be separated away from the main microcontroller thus having a battery powered

device that will communicate wirelessly with the access point. This will give additional flexibility for measurement of

objects in difficult to reach places without the need of wires. The further step is to build a network of those devices in

order to have a way of observing and tracking the subjects in hospital and home environment.

Acknowledgments

The work presented in this paper is supported by project ДФНИ Е02/12 “Investigation of methods and tools for

application of cloud technologies in the measurement and control in the power system” and Melexis MicroelectronicIntegrated Systems.







References

[1]

Li, N., Calis, G., and Becerik-Gerber, B., ‘Measuring and monitoring occupancy with an RFID based system for

demand-driven HVAC operations’, Automation in construction 24 (2012), pp. 89-99.

[2]

Melexis Microelectronic Integrated Systems, White paper ‘MLX90620 for smart & green HVAC’, April 2013.[3]

Hnat, T. W., Griffiths, E., Dawson, R., and Whitehouse, K. Doorjamb, ‘Unobtrusive room-level tracking of people

in homes using doorway sensors’, In Proceedings of the 10th

ACM Conference on Embedded Network Sensor

Systems (2012), ACM, pp. 309-322.

[4]

Teixeira, T., Dublon, G., and Savvides, A., ‘A survey of human-sensing: Methods for detecting presence, count,

location, track, and identity’, Technical report, Embedded Networks and Applications Lab (ENALAB), Yale

University, 2010.

[5] Kumar, S., Marks, T., Jones, M., ‘Improving Person Tracking Using an Inexpensive Thermal Infrared Sensor’,

CVPRW 2014 – Intl. Conf. on Computer Vision and Pattern recognition Workshops, Columbus, USA, June 2014,

pp. 217-224, doi: 10.1109/CVPRW.2014.41.

[6] Melexis, Datasheet IR thermometer 16X4 sensor array MLX90620, http://www.melexis.com/Infrared-

Thermometer-Sensors/ Infrared-Thermometer-Sensors/MLX90620-776.aspx, 2012.

[7] Olimex, Web server PIC32-MAXI-WEB with 32 bit PIC microcontroller PIC32MX795F512L datasheet,

https://www.olimex.com/Products/PIC/Development/PIC32-MAXI-WEB/resources/PIC32-MAXI-WEB.pdf, 2013.

AUTHORS

Galidiya Petrova received MSc. degree in Electronics at Technical University of Sofia in 1985 and PhD

degree in Biomedical engineering in 2001. She is presently Associate professor in the Department of

Electronics, Technical University of Sofia, Plovdiv branch. Her research interests are within: data acquisition

systems, applications of distributed systems in medicine, personalized healthcare and ambient assisted leaving

systems, wireless body sensor networks.

Grisha Spasov received MSc. degrees in Computer engineering at Technical University of Sofia in 1983 and

PhD degree in Computer networks in 2003. He is presently Professor in the Department of Computer systems

and technologies, Technical University of Sofia, Plovdiv branch. His research interests are within: Distributed

embedded systems and distributed applications, Internet of Things, Service oriented architectures for

distributed measurements, Computer networks and Wireless sensor networks.

Vasil Tsvetkov received BSc. degrees in Computer systems and technologies Technical University of Sofia,

Plovdiv branch, Bulgaria in 2015. He is presently MSc student in the Department of Computer systems and

technologies, Technical University of Sofia, Plovdiv branch. His research interests are in the area embeddedsystems, firmware, ASIC design, FPGA prototyping and Internet of Things.

using infrared array devices in smart home observation and diagnostics

Documents