diagnosis of industrial robots using case based … · diagnosis of industrial robots using case...
TRANSCRIPT
Diagnosis Of Industrial Robots Using Case
Based Reasoning
Erik Olsson [email protected]
A Master of Science thesis in Computer Science performed at Mälardalen University and
ABB Robotics during the summer and fall of 2003.
Supervisors:
Mats Jackson, ABB Robotics and Mälardalen University, Dep. of Design and Product Development
Peter Funk, Mälardalen University, Dep. of Computer Science and Engineering
Marcus Bengtsson, Mälardalen University Dep. of Design and Product Development
Abstract
ABB Robotics is one of the worlds leading manufacturers of industrial robots. The manufac-
turing process of an industrial robot involves procedures such as assembly, testing and paint-
ing. The testing procedure is critical when it comes to fault detection.
Faults can often be identified as abnormal sounds coming from one of the gearboxes on the
robot during the testing procedure. The decision whether such a sound should be regarded as a
fault or not can be difficult to make by just listening to it. Today that decision is made by the
testing personnel. This means that inexperienced testing personnel runs a risk of making a
wrong diagnosis and consequently let a faulty robot pass as normal.
This thesis explores automatic fault detection in production of robots. This is accomplished by
letting a computer classify the sound that comes from the gearboxes during the testing proce-
dure. A prototype based on wavelet analysis and case based reasoning has been implemented
in attempt to show that this is possible. The prototype has managed to identify two kinds of
faults on axis 4 on an industrial robot.
This report is a ABB internal version of a thesis required for the degree of Master of Science
in Computer Science at Mälardalen University. It was performed by Erik Olsson during the
summer and fall of 2003.
Acknowledgments
This thesis was written at ABB Robotics AB during the summer and fall of 2003. It was writ-
ten by Erik Olsson and forms the required thesis for the degree of Master of Science in Com-
puter Science at Mälardalen University. I want to thank my supervisors: Peter Funk and Mar-
cus Bengtsson at Mälardalen University and Mats Jackson at ABB Robotics. I also want to
thank Mats Åhgren and Christian Dahlberg for their compliance and the testing personnel for
putting up with all my recordings. At last I want to thank Ian Kaplan at Bear Products Interna-
tional [6] for his excellent implementation of the Wavelet Packet Algorithm.
2
1 Introduction............................................................................................................. 6 1.1 Overview .................................................................................................................. 6 1.2 Background............................................................................................................. 6 1.3 Aim ............................................................................................................................ 7 1.4 Related work ........................................................................................................... 7 1.5 Limitations............................................................................................................... 7
2 Theory .......................................................................................................................... 8 2.1 Introduction............................................................................................................. 8 2.2 Case based reasoning ......................................................................................... 8 2.3 Wavelets................................................................................................................. 10
3 The recorded sound....................................................................................... 16 3.1 Introduction........................................................................................................... 16 3.2 Choice of robot model and axis ...................................................................... 16 3.3 The ABB Industrial robot................................................................................... 16 3.4 Choice of axis....................................................................................................... 17 3.5 The testing procedure ........................................................................................ 18 3.6 Choice of recording device............................................................................... 18 3.7 Attaching the recording device ....................................................................... 18 3.8 Recording the sound .......................................................................................... 20 3.9 The recordings ..................................................................................................... 20
3.9.1 Conclusions .................................................................................................. 22 3.10 Analysis of the axis4 sound ............................................................................. 23
3.10.1 Introduction................................................................................................... 23 3.10.2 The period ..................................................................................................... 23 3.10.3 Basic features............................................................................................... 23
4 The prototype ...................................................................................................... 25 4.1 Introduction........................................................................................................... 25 4.2 Problem description ........................................................................................... 25 4.3 Problem analysis ................................................................................................. 25
4.3.1 Methods ......................................................................................................... 26 4.4 Design..................................................................................................................... 26 4.5 Implementation .................................................................................................... 27
4.5.1 Dividing the signal into windows ............................................................ 27 4.5.2 Analysing the windows and extract frequency information ............ 28 4.5.3 Assemble a feature vector from the frequency information............ 28 4.5.4 Matching the feature vector and produce a guess............................. 29
4.6 Testing.................................................................................................................... 30 4.6.1 Introduction................................................................................................... 30 4.6.2 A normal sounding axis............................................................................. 30 4.6.3 Fault1: Loosely tightened gear wheel fault. ......................................... 32 4.6.4 Fault 2: Notch fault...................................................................................... 33 4.6.5 Faults 3: Notch fault ................................................................................... 34 4.6.6 Three candidate faults ............................................................................... 35 4.6.7 Example1 ....................................................................................................... 35 4.6.8 Example2 ....................................................................................................... 37
5 Conclusions .......................................................................................................... 38 5.1 Result...................................................................................................................... 38 5.2 Future work ........................................................................................................... 38
3
5.3 Summary................................................................................................................ 39 References ..................................................................................................................... 40 Appendix A, Graphical plots of sound recordings ..................... 41 Table 1: Example of cases representing car failures......................................................... 8 Figure 1: Matching cases. ...................................................................................................... 9 Figure 2: The CBR model. (Agnar Aamodt, Norwegian University of Science and
Technology).................................................................................................................... 10 Figure 3: Graphs of the soundwaves produced by a tuning fork (left) and the spoken
word “greasy” (right). (Ofer Levi, Stanford University)............................................. 11 Figure 4: The Mexican Hat wavelet. ................................................................................... 11 Figure 5: A cosine wave. ...................................................................................................... 12 Figure 6: The wavelet passes along the cosine wave and different integral values are
produced. ........................................................................................................................ 12 Figure 7: The wavelet is stretched and shrinked.............................................................. 13 Figure 8: The Daubechies wavelet. .................................................................................... 13 Figure 9: Input with 16 samples. ......................................................................................... 14 Figure 10: The input is divided into a low and a high pass version............................... 14 Figure 11: The result is a tree.............................................................................................. 15 Figure 12: The ABB industrial robot ................................................................................... 17 Figure 13: The Palm handheld computer is mounted in a box with magnets on. ....... 19 Figure 14: Position 1. ............................................................................................................ 19 Figure 15: Position 2. ............................................................................................................ 20 Figure 16: Axis 4 sound........................................................................................................ 23 Figure 17: Axis 4 sound at 6600Hz. ................................................................................... 24 Figure 18: Axis 4 sound at 1140Hz. ................................................................................... 24 Figure 19: The design of the prototype. ............................................................................. 27 Figure 20: Flowchart of the signal division ........................................................................ 27 Figure 21: The matching function ....................................................................................... 29 Figure 22: The frequency spectra of a normal sounding axis. ....................................... 31 Figure 23: The frequency spectra of a loose gear wheel fault. ...................................... 32 Figure 24: The feature vector for a loose gear fault......................................................... 32 Figure 25: Feature vector for the notch fault..................................................................... 33 Figure 26: The frequency spectra of a notch fault. .......................................................... 33 Figure 27: A profile of a notch fault..................................................................................... 34 Figure 28: Feature vector of a notch fault. ........................................................................ 34 Figure 29: A cross section of the tree is taken. ................................................................ 35 Figure 30: A_0729_1235_NoPayloadPos1 ....................................................................... 41 Figure 31: A_0721_NoPayloadPos1 .................................................................................. 42 Figure 32: A_0721_1301_NoPayloadPos1 ....................................................................... 42 Figure 33: A_0721_NoPayloadPos1_NotchFault ............................................................ 43 Figure 34: A_0824_PayloadPos1 ....................................................................................... 43 Figure 35: A_0824_PayloadPos2 ....................................................................................... 44 Figure 36: A_0826_PayloadPos1 ....................................................................................... 44 Figure 37: A_0826_PayloadPos2 ....................................................................................... 45 Figure 38: A_0907_0928_PayloadPos1 ............................................................................ 45 Figure 39: A_0907_0928_PayloadPos2 ............................................................................ 46 Figure 40: D_0907_PayloadPos1 ....................................................................................... 46
4
Figure 41: A_0918_NoPayloadPos1 .................................................................................. 47 Figure 42: A_0918_NoPayloadPos2 .................................................................................. 47 Figure 43: C_0825_PayloadPos1_LooseGearFault ........................................................ 48 Figure 44: C_0825_PayloadPos2_LooseGearFault ........................................................ 48 Figure 45: C_0827_PayloadPos1 ....................................................................................... 49 Figure 46: C_0827_PayloadPos2 ....................................................................................... 49 Figure 47: C_0906_NoPayloadPos1 .................................................................................. 50 Figure 48: C_0906_NoPayloadPos2 .................................................................................. 50 Figure 49: C_0918_PayloadPos1 ....................................................................................... 51 Figure 50: C_0918_PayloadPos2 ....................................................................................... 51 Figure 51: B_0716_NoPayloadPos1_NotchFault ............................................................ 52 Figure 52: B_0730_NoPayloadPos1_NotchFault ............................................................ 52 Figure 53: B_0816_1242_NoPayloadPos2 ....................................................................... 53 Figure 54: B_0914_PayloadPos1 ....................................................................................... 53 Figure 55: B_0914_PayloadPos2 ....................................................................................... 54
5
1 Introduction
1.1 Overview
The report consists of five chapters. Chapter 1 covers the background and the purpose of the
thesis. Chapter 2 gives a brief introduction to the underlying theories of the thesis. If the
reader already has the theoretical background, chapter 2 can be skipped. Chapter 3 considers
aspects of the robot sound and the recording devices used. Chapter 4 describes the design,
implementation and testing of a prototype system that is implemented as a part of the thesis.
Chapter 5 covers the conclusions made during this work and presents visions and ideas of a
further development of the prototype system.
1.2 Background
ABB Robotics AB is one of the worlds leading manufacturers of industrial robots. Their first
robot model, the IRB 6, was manufactured 1974 and was the worlds first microcomputer con-
trolled electrical robot. Since then they have produced and sold over 100 000 industrial robots
all over the world. The model range spans from the IRB 140 with a lifting capacity of 5 kg to
the IRB 7600 that can handle payloads up to 500 kg. Industrial robots are used in many kinds
of industries, from consumer goods industries to automotive and foundry industries.
The main part of the robots are assembled and tested at Finnslätten in Västerås, Sweden. The
assembly process of an industrial robot involves four main steps: assembly, testing, painting
and dressing. It is of great importance that the robots shipping from the factory in Finnslätten
are free from faults.
The testing step is critical when it comes to fault detection. If a robot has any faults due to
poor assembly routines or bad parts etc, it most likely will be discovered during the testing
procedure. When a robot is tested, it is placed in a testing area and a special test program runs
the robot to control its function. During the test program, strange noises sometimes can be
heard from the robot. Noises that can be the results of various defects in gears, bearings and
motors on the robot. Because there is no equipment to analyse these noises - the decision
whether such a noise comes from a faulty bearing, gear or electrical motor or if the noise is in
fact normal is very much up to the sense and feel of the testing personnel.
6
This thesis considers the idea of automatic fault detection. It gives a suggestion to hand assist
fault detection with a computer. A computer is more consistent than a human and a computer
has better memory. This report considers the underlying theory and implementation of a pro-
totype witch goal is to make a diagnosis of the condition of the robot with the aid of sound.
The following pages gives a detailed description of how the work was performed and how the
prototype was implemented.
1.3 Aim
The aim of this thesis is to find out whether sound recordings can be used to separate a faulty
industrial robot from a normal robot with a “smart tool” used by an engineer. Methods and
techniques from artificial intelligence and mathematics, such as case based reasoning and
wavelet analysis will be used for the implementation of a prototype with the aim to solve this
problem. If the problem can be solved, a system based on those principles can be used to aid
fault detection in industrial robots.
1.4 Related work
Several works in related areas has been performed. Anréus and Grante [1] has performed a
thesis at Volvo Wheel Loaders AB. The thesis covers spectral analysis of the sound produced
when wheel loader gearboxes are tested. Benko et al [2] covers fault detection in vacuum
cleaner motors. Mäkinen [3] discusses the advantages of vibration analysis on cooling com-
pressors. Vallin [4] has used electric current measurements in attempt to see if any indication
is given when the gears in axis 5 and 6 on are adjusted to tight or to loose. This thesis goes
one step further with the aim to build a prototype of a flexible and learning “smart tool”, used
by e.g. field engineers working in production.
1.5 Limitations
Only one axis on one robot model will be analysed for validation of the proposed method.
The design and implementation of the prototype will be focused on the sound from that axis.
In chapter 5 there is a discussion of how the ideas and solutions implemented in the prototype
can be extended to other axes and robot models.
7
2 Theory
2.1 Introduction
This chapter gives a short tutorial of case based reasoning and wavelet analysis. The aim with
this chapter is to provide the background theory needed to understand the design and imple-
mentation of the prototype system discussed in chapter 3. This chapter starts by giving a short
tutorial of case based reasoning which is a method used in the prototype. It continues to de-
scribe the theory behind wavelet analysis. In this thesis, wavelet analysis is used as a tool to
extract features that comprise cases in the prototype.
2.2 Case based reasoning
Case Based Reasoning (CBR) is an Artificial Intelligence (AI) I technique that is based upon
the way a human is learning things. If you run into a problem, the first thing you do is to try to
remember if you have solved a similar problem before. If you have, there is a big chance that
you can solve the problem by applying the same technique that you did the last time the prob-
lem occurred. If the problem is slightly different, there is still chance that you can solve it by
applying your old knowledge and alter it to fit the new problem. In this way you can learn
new things by reusing old knowledge. This is the fundamental idea of CBR – applying old
knowledge of problem solving to solve new problems.
A case is an abstraction of a procedure. The procedure can be anything that produces a useful
result. One example is the diagnosis of a car failure:
Case Starter engine
OK?
Is there gas? Headlight OK? Action
Case1 Yes Yes ? Call an expert
Case2 Yes No ? Fill up gas
Case3 No ? No Charge battery
Case4 No ? Yes Call an expert
Table 1: Example of cases representing car failures.
8
Here the case is the car repair procedure. It involves symptom, diagnosis and action.
Cases are stored in a case library for later reuse when similar cases occur. The case library is
updated with newer cases as they pop up. Similar cases might be merged together and old
cases that are no longer used might be deleted. When new cases occur, they are matched with
cases in the case library. One way of matching cases is to use the nearest neighbour algorithm.
Similar case
Border case
Figure 1: Matching cases.
The cases witch are nearest the new case has the best probability to present a correct solution
to the new case. If no cases in the case library are close enough, the new case becomes a new
contribution to the library. If different cases in the case library are within the same distance to
the new case they might be combined to present the correct solution. The new combined case
also becomes a new contribution to the case library. The framework for CBR is a cognitive
model for problem solving that is based on past experience, retrieve-reuse-revise-retain.
9
Figure 2: The CBR model. (Agnar Aamodt, Norwegian University of Science and Technol-
ogy).
2.3 Wavelets
The Wavelet theory is a further development of the Fourier theory. The Fourier theory says
that a periodic signal can be expressed as the sum of a, possibly infinite, series of sines and
cosines.
As an example, the tone of an instrument is a combination of its lowest frequency tone, called
the fundamental frequency of the tone, and the higher-frequency ones, called overtones. The
Fourier theory has the limitation that it cannot represent non-periodic signals. Periodic signals
are signals that repeat itself over a period of time like, for example, the tone from an instru-
ment. Non-periodic signals are for example a speech sound. The word “grease” is not a peri-
odic signal, it contains a mix of frequencies with sharp peaks and slopes. But the sound from
a tuning fork is periodic.
10
Figure 3: Graphs of the soundwaves produced by a tuning fork (left) and the spoken word
“greasy” (right). (Ofer Levi, Stanford University)
A periodic signal is said to be localised in frequency whereas a non-periodic signal is said to
be localised in time.
A wavelet has the advantage that it can do both, the wavelet package transform separates a
wave into “time-frequency packets”. A wavelet consists of a basis function, the mother wave-
let. It can be any function witch positive and negative areas cancel out.
One example is the Mexican hat wavelet. It is called so because it looks like looks like an up-
side down sombrero.
Figure 4: The Mexican Hat wavelet.
The function describing the Mexican hat is:
11
As an example, let’s see how the Mexican hat wavelet is used to analyse a cosine wave.
Let be the cosine wave.
Figure 5: A cosine wave.
The wavelet is shifted alongside the signal and a match is calculated as the integral between
the signal and the wavelet at any given time.
As the wavelet slides along the signal, the absolute value of the integral between the signal
and the wavelet will increase when the wavelet is in phase with the signal and decrease when
the wavelet slides out of phase with the signal. The opposite will occur when the wavelet
slides in phase again.
Figure 6: The wavelet passes along the cosine wave and different integral values are pro-
duced.
12
Now lets stretch and shrink the wavelet along the time axis by changing the duration time of
the wavelet function. When sliding the stretched and shrinked versions of the wavelet func-
tion along the signal, different integral results will be produced.
Figure 7: The wavelet is stretched and shrinked.
As can be seen in the picture, line number three gives the best match against the original sig-
nal. The wavelet transform is shown in dashed lines and the wavelet function is shown in dot-
ted lines. In the end the result will be a collection of time-frequency representations of the
signal, all with different resolutions. The stretched wavelet will match signals with lower fre-
quencies and the shrinked wavelet will match higher frequencies.
The wavelet algorithm that is used in this thesis project is called the Daubechies wavelet
packet transform. The Daubechies wavelet is named after its inventor, the mathematician
Ingrid Daubechies.
Figure 8: The Daubechies wavelet.
13
Wavelet analysis can be used to analyse the frequency spectra in various measurement data,
for example stock market curves, temperatures or, as in this case, sampled sound. Sampled
sound consists of a number of measurements that spans over a certain time. The number of
samples taken each second is called the sampling rate, or the sampling frequency (measured
in Hz). The number of bits used to represent each sample is called the resolution (measured in
bits).
The Daubechies wavelet packet transform algorithm separates the original signal into “time-
frequency-packets”, this is done by applying the wavelet function recursively over and over
again until the input signal is of length 2. As an example, lets assume that the input signal to
the algorithm consists of 16 samples.
Figure 9: Input with 16 samples.
The nature of the algorithm forces the size of the input data to be in a power of two. The algo-
rithm applies the Daubechies wavelet function to the input data and thereby separates the
original data into two units.
Figure 10: The input is divided into a low and a high pass version.
hese units can be seen as a high and a low pass filtered version of the original data. The low
he wavelet function is again applied to each filtered unit and the result is copied into the
T
pass filtered data is in the left of the original data array and the high pass filtered data is in the
right of the original data array.
T
original array. This process is then repeated until the high and low pass units are of length 20
and no more splitting is possible. The result can be seen as a tree with the original data as the
root of the tree and the smallest high and low pass units as the leaves of the tree.
14
Figure 11: The result is a tree.
The depth of the tree is determined by the length of the input data. If the input data is of
length 2n the tree depth will be n. A suitable collection of time frequency packets can be se-
lected by taking a cross section of the tree at a depth. Each sibling in the cross section of the
tree spans the entire time of the original data set. This means that going deeper in the tree
gives at better resolution in frequency but a poorer resolution in time. The best compromise
between time and frequency resolution is to take a cross section in the tree where the length of
each sibling is the same as the number of siblings in the cross section. At a given depth n and
with original data size S, the length of a sibling (or leaf) is S/2n and the number of siblings is
2n.
When a low or high pass filtered version of the data is calculated, the data is interpolated on
the mother wavelet function. The difference between the interpolated values and the original
values is the result that is saved in the array. To learn more about the wavelet packet algo-
rithm, see Ian Kaplan’s page [5] or Amara’s links and tutorials [6].
15
3 The recorded sound
3.1 Introduction
This chapter covers the sound recordings that are used by the prototype for analysis. It first
describes the robot model and axis that is chosen for the recordings. All sound recordings
where made during the testing procedure of the robots. This procedure is also described. The
recording device is explained and after that a discussion of the characteristics of the recorded
sound is given.
3.2 Choice of robot model and axis
Three problems must be solved to obtain the sound data:
1. Choose a robot model and axis to record
2. Choose a recording device
3. Find a way of attaching the recording device on the robot
The robot model selected for sound recording is an industrial robot produced by ABB Robot-
ics. Several robots have been recorded. A robot model that tended to be more error prone dur-
ing testing than other robot models was selected. The axes of this model are exposed to great
torque during operation.
3.3 The ABB Industrial robot
The ABB Industrial robot can handle five different sizes of payloads.
It comes in five versions:
A, B, C, D and E
It is used for process applications in many types of industries. Typical areas are spotwelding,
material handling and machine tending.
16
Figure 12: The ABB industrial robot
The ABB industrial robot has six axes witch gives it a wide area of operation. The axes are
numbered from bottom to top as axis 1 to 6.
3.4 Choice of axis
Axis number 4 was selected for recording. It was selected because the first and second fault
occurred on it (see chapter 4). It therefore became natural to focus on it. Axis 4 rotates the
wrist. Its working range is +300° to -300° and its max speed is 150°/s.
17
3.5 The testing procedure
All robots endure the same testing procedure before they are delivered to the customer.
The testing procedure contains of six steps:
1. Manual operation
2. Separate axis test
3. Brake test
4. Automatic operation test
5. Battery test
6. Dip-control
3.6 Choice of recording device
A Palm handheld computer was selected for recording. It is a small handheld computer with
an integrated microphone.
The selected recording software was Audacity Digital Voice Recorder [7].
3.7 Attaching the recording device
The PDA (personal digital assistance) handheld computer is mounted in a small box. A hole
was drilled in the box to let the sound reach the microphone. Magnets are attached to the
backside of the box to provide a way of attaching the box on the robot. This solution was se-
lected due to some technical problems with the wireless blue tooth microphone that would
enable the user to make the recording by attaching a small magnetic microphone.
18
Figure 13: The Palm handheld computer is mounted in a box with magnets on.
Position 1:
This position is close to the motor that operates the axis 4 but not equally close to the gear
wheels in the gearbox.
Figure 14: Position 1.
19
Position 2:
This position is equally close to the motor as to the gear wheels.
Figure 15: Position 2.
Recordings where performed in both positions se next subchapter for a discussion about the
positions.
3.8 Recording the sound
All recordings were made during the testing procedure.
A button on the handheld starts the sound recording. The test cycle is set to separate axis test
and axis number four is selected. The recording format is WAV, the sampling frequency is
8kHz and the resolution is 16 bits. The recording is in mono. The sound file is then trans-
ferred to a stationary computer in witch the prototype is implemented.
3.9 The recordings
17 different robots have been recorded. 11 of them have two recordings made with two differ-
ent positions of the microphone (the Palm handheld). This was done to find out if a change in
position of the microphone affects the features of the recorded sound. 9 robots had the testing
payload attached and 8 robots had not.
The file names are to be interpreted as robot model_date of recording_payload/no payload
position of microphone_type of fault (if any)
20
Two types of faults have been identified in the recordings, the fault types are:
1. Notch fault
2. Loose gear wheel fault
See chapter 5 for more information about the recorded faults.
All recordings can be seen as plots in the appendix.
The following recordings was performed in position 1:
Recordings performed on robots with no payloads:
A_0721_NoPayloadPos1_NotchFault.WAV
B_0716_NoPayloadPos1_NotchFault.WAV
B_0730_NoPayloadPos1_NotchFault.WAV
0729_1235_NoPayloadPos1.WAV
A_0721_1301_NoPayloadPos1.WAV
A_0721_NoPayloadPos1.WAV
A_0918_NoPayloadPos1.WAV
C_0906_NoPayloadPos1.WAV
Recordings performed on robots with payloads:
C_0825_PayloadPos1_LooseGearFault.WAV
A_0824_PayloadPos1.WAV
A_0826_PayloadPos1.WAV
A_0907_0928_PayloadPos1.WAV
A_0907_PayloadPos1.WAV
C_0827_PayloadPos1.WAV
C_0918_PayloadPos1.WAV
B_0914_PayloadPos1.WAV
D_0907_PayloadPos1.WAV
21
The following recordings are performed in position 2.
Recordings performed on robots with no payloads:
A_0918_NoPayloadPos2.WAV
C_0906_NoPayloadPos2.WAV
B_0816_1242_NoPayloadPos2.WAV
Recordings done on robots with payloads:
C_0825_PayloadPos2_LooseGearFault.WAV
A_0824_PayloadPos2.WAV
A_0826_PayloadPos2.WAV
A_0907_0928_PayloadPos2.WAV
A_0907_PayloadPos2.WAV
C_0827_PayloadPos2.WAV
C_0918_PayloadPos2.WAV
B_0914_PayloadPos2.WAV
3.9.1 Conclusions
Recordings made in position 1 are not as good as recordings in position 2, this includes both
recordings with payloads and recordings without payloads. Even though, some faults have
been identified in position 1. This is because position 1 was the original position and some
recordings were made only in that position.
Robots with payloads have more outstanding peaks in the plots compared to robots without
payloads (see appendix). There is a risk that some of those peaks can be misinterpreted as
faults in the matching function.
22
3.10 Analysis of the axis4 sound
3.10.1 Introduction The sound made from axis4 when the arm rotates from start position to end position has cer-
tain characteristics. These characteristics make the foundation of the way the prototype is im-
plemented and how it behaves.
3.10.2 The period The sound made from axis4 when the arm rotates from start position to end position is called
a period. The period is characterised by a bump in the plot below. Two periods are separated
by a short time of silence. In the plot below the silence can be seen as a thin line between two
bumps. The silence comes from when the robot arm changes its direction of rotation.
-40000-30000-20000-10000
0 10000 20000 30000
0 20000 40000 60000 80000 100000 120000 140000
"vekt.txt"
Figure 16: Axis 4 sound.
The period is taken directly from the raw wav data. This makes up the input to the prototype.
When the period looks like this, it is unfiltered, it contains all frequencies from 1Hz (and be-
low), up to 8kHz witch is the max recording frequency.
3.10.3 Basic features
There is much information hidden in the period that cannot directly be obtained when the pe-
riod is in this format.
The wavelet analysis algorithm, filter out various frequency information from the period and
provides a way of finding “hidden” features in it.
23
At 6600Hz the bumps are easy to distinguish from each other.
-20000-15000-10000-5000
0 5000
10000 15000 20000
0 100 200 300 400 500 600
"plot220"
Figure 17: Axis 4 sound at 6600Hz.
At 1140Hz there is easy to distinguish in which way the arm is rotating. The higher amplitude
bumps rotates the opposite way as the lesser amplitude bumps.
-100000-80000-60000-40000-20000
0 20000 40000 60000 80000
100000
0 100 200 300 400 500 600
"plot38"
Figure 18: Axis 4 sound at 1140Hz.
24
4 The prototype
4.1 Introduction
This chapter covers the implementation of a prototype system. It first describes and analyses
the given problem. Methods are given to solve the problem and a design is presented. The im-
plementation of the design is described and some examples of the testing procedure are pre-
sented.
4.2 Problem description
The task is to implement a prototype that uses sound to diagnose faults in industrial robots.
The prototype must be able to take a recorded sound as input and make a diagnosis of the
sound. To make a diagnosis, the prototype must be able to find features in the sound that cap-
tures the faults in the sound.
The time for the project is limited to 20 weeks. To be able to solve the task within that time,
only one axis on a specific robot model is to chosen for recordings.
4.3 Problem analysis
The implementation of the prototype involves:
1. Design
2. Implementation
3. Testing
25
4.3.1 Methods
Two well known methods will be used in the prototype:
1. Wavelet analysis
Wavelet analysis will be used for feature extraction. Wavelet analysis provides a tool
of making a spectral analysis of the sound. This will aid the feature extraction process
as hidden features can be identified in the various sound spectra.
2. Cased based reasoning
Cased based reasoning will be used for diagnosis. Case based reasoning has an advan-
tage over neural nets because it requires no training data in order to make a diagnosis.
Features of a fault can instantly be saved in the case library and later faults can be
matched to these features.
The prototype is implemented on a regular IBM PC. The operating system is Windows NT
and the programming environment is Microsoft Visual Studio 6.0.
The programming language is C++.
4.4 Design
The prototype takes a wav sound file as input and produces a diagnosis of the sound file as
output. This is achieved in a pipe design involving four steps. Each step in the pipe can be
seen as a function with an input and an output.
The prototype consists of four steps
1. Dividing the signal into periods
2. Analysing the periods and extract frequency information
3. Assemble a feature vector from the frequency information
4. Matching the feature vector and produce a guess
26
1. Dividing
the signal
into peri-
ods
-GuessSignal
Figure 19: The design
4.5 Implementation
4.5.1 Dividing the signal in
The first step before the perio
cording. A function: “calibrat
amplitude level of silence and
When the calibration is finishe
riod is a rotation of the robot a
This is performed in three step
3.
2.
1.
Figure 20: Flo
Explanation of figure 20:
1) Sound data are
2) The period is re3) The rest of the w
2. Analys-
ing the pe-
riods and
extract fre
quency i
formation
n-
of the prototype.
to windows
d division starts is
e” calculates a mean
sound in the recordin
d - the next step is
rm.
s:
AddZero(Signal)
ReadSound(Signal)
FindNoice(Signal)
wchart of the signal
read from the file un
ad from the file. indow is filled with
3. Assem-
ble a fea-
ture vector
from the
frequency
information
to calibrate the amp
value of the record
g.
to divide the signal i
division
til a period is found.
zeros.
4. Match-
ing the fea-
ture vector
and pro-
duce a
guess
litude level of the re-
ing and estimates the
nto periods. Every pe-
27
The length of each audio data block (window) that is passed to the next step in the pipeline is
set to 32768 (215), witch is about 4 seconds of continuous sound. One rotation of the robot
arm lasts for about three seconds. The rest of the window is filled with zeros.
4.5.2 Analysing the windows and extract frequency information
The window has a length of about four seconds. It contains one period of robot arm move-
ment. Every period contains a rotation of the robot arm.
Each window is then sent to the wavelet analysis algorithm for /frequency analysis.
The wavelet analysis algorithm makes a time/frequency packet tree of the input data window.
A cross section at level eight of the wavelet packet tree is taken. Level eight is chosen because
it makes a good compromise between time and frequency resolution. At level 8, there are 256
(28) time/frequency packets.
The sampling rate is 8kHz. This means that every time/frequency packet has a frequency in-
terval of about 30Hz (8000/256).
4.5.3 Assemble a feature vector from the frequency information
One value is taken from each packet in the cross section. The value is the maximum absolute
value in each packet. Each value is saved in the feature vector.
4.5.3.1 The feature vector
The feature vector is a way to represent the sound from a period in an easy and comparable
way. Another important feature of the feature vector is to reduce the dimension of the original
data. The feature vector has a dimension of 256 values. Each value is the maximum absolute
value taken from each packet in the wavelet packet tree.
28
Feature vector data type:
[value1,value2, …, value256]
4.5.4 Matching the feature vector and produce a guess
When a feature vector is assembled, it is normalized, matched and stored in a case library.
The case library consists of vectors from older recordings.
The matching function calculates the Euclidian distance between the new feature vector and
the cases stored in the case library. The result is a list sorted by the distance between the new
feature vector and the cases in the case library.
Case library
Case
Sorted list with results
Retrieve
Case Feature vector
Reuse
Match
Feataure vector
Figure 21: The matching function
29
4.6 Testing
4.6.1 Introduction Two types of faults have been recorded. Here, the faults will be explained in detail and the
special features contained in them will be discussed. The special features in the faults can
later be used to identify similar faults. The faults have been identified by experienced testing
personnel before they were recorded.
The faults are:
1. Loosely tightened gear wheel fault. Caused by a slack between gear wheels in axis 4.
Can be heard as four bumps in a short time interval between each rotation in a period.
2. Notch fault. Caused by a notch on the big gear wheel in axis 4.
Can be heard as a bump in the second rotation of the period.
3. Notch fault. Caused by notches on the big gear wheel in axis 4.
Difficult to hear
4. Three candidate faults (not verified).
Sounds like a loosely tightened gear wheel.
4.6.2 A normal sounding axis
The features of the faults are easier to spot if being compared to a normal sounding axis. A
normal sound from axis 4 during a period of the separate axis test is characterised by a con-
tinuous humming sound. Sometimes (but not always) a light bump can be heard at the end of
the continuing sound. This bump is made when the axis changes its direction of rotation. The
amplitude of this bump is a good starting indication whether the sound from a period is nor-
mal or not. The picture below shows a typical normal sounding axis.
30
Figure 22: The frequency spectra of a normal sounding axis.
The plot shows the frequency spectrum of two normal periods. The z-axis shows the period
length, the y-axis shows the amplitude and the x-axis shows the frequency. The bump at the
end of each period is prominent from the rest of the sound in the period. The amplitude of the
bump is low and consistent in the entire frequency spectra and there are no other prominent
features in the plot. This is an ideal sound. The plot shows a robot with no payload attached.
If a payload is attached, some outstanding peaks in the lower frequency spectra can occur but
the amplitude of the peaks is not as high as in the plots displaying a loose gear fault (see the
appendix for a full collection of plots).
The recorded faults all have some basic characteristics:
1. The loosely tightened gear wheel faults have peaks of abnormal amplitude at the be-
ginning and end of each rotation in the lower frequency spectra.
2. The notch faults have peaks caused by the notches at various positions and frequencies
within a rotation of a period.
In the rest of the chapter, the faults will be explained more in detail.
31
4.6.3 Fault1: Loosely tightened gear wheel fault.
This fault is caused due to a slack between the gear wheels in the gearbox. The sound can be
heard as four bumps in a short interval of time at the end of each rotation in the period. Below
is a plot that shows the sound from a recording performed on a robot of type B.
Figure 23: The frequency spectra of a loose gear wheel fault.
The first thing to notice in the plot is the abnormally high peaks in the lower frequency spec-
tra at the end of each period The peaks coincide with the recorded sound in which fours
bumps in a short interval of time can be heard at the end of each rotation.
The feature vector representing this fault captures the abnormalities in the sound:
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0 50 100 150 200 250 300
line 1
Figure 24: The feature vector for a loose gear fault
The frequency of the most prominent peaks occurs in packets #5 to packet #10 that is in the
frequency spectra from 150Hz-300Hz (5*30Hz-10*30Hz).
32
4.6.4 Fault 2: Notch fault
This fault is caused by a notch on the big gear wheel in the gearbox of axis 4. It can be heard
as a bump in the beginning of the rotation of the period. Equal to the case of the loose gear
wheel fault, this fault is easy to discern in the period. Except from the abnormal amplitude of
the bumps at the end of each rotation, there is a very prominent peak in the beginning of the
rotation. It can be seen in the plot at the lower frequency spectra of the period.
The feature vector representing this fault:
0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.1 0.11 0.12 0.13
0 50 100 150 200 250 300
line 1
Figure 25: Feature vector for the notch fault.
The position of the peaks agrees with the spectral plot and with the bump in the recorded
sound.
Figure 26: The frequency spectra of a notch fault.
33
4.6.5 Faults 3: Notch fault
These faults are caused by (probably several) notches on one of the gear wheels of axis 4. The
faults are difficult to hear in the recordings and can only be discovered by trained testing per-
sonnel. In the plot they are quite easy to spot if seen from the right angle.
Fault 3 has at least one abnormal peak in each period in the spectral plot. The plot shows two
periods:
Figure 27: A profile of a notch fault.
The peaks are captured in the feature vector. Mainly at frequency #175:
0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.1 0.11
0 50 100 150 200 250 300
line 1
Figure 28: Feature vector of a notch fault.
34
4.6.6 Three candidate faults
These faults are not verified but the matching function matches them close to a loose gear
fault. See the example in the next chapter. Their sound and plots resembles the verified loose
gear fault.
See appendix for plots on:
D_0918_PayloadPos1.WAV
A_0907_0928_PayloadPos1.WAV
C_0827_PayloadPos2.WAV
4.6.7 Example1
A robot with a loose gear fault, was recorded on the 25th of August. The name of the sound
file is C_0825_PayloadPos2_LooseGearFault.WAV The sound from a loose gear is character-
ised by four bumps at the end of each rotation.
The sound is copied from the Palm handheld computer into the PC with the prototype.
In the prototype the wav sound file is first divided into windows of a length of 4 seconds.
Each window is analysed in the wavelet packet algorithm and each packet in the cross section
of the time/frequency packet tree at level 8 is examined and 256 values are assembled to a
feature vector.
Figure 29: A cross section of the tree is taken.
The matching function then matches the vector with all the other vectors stored in the library.
35
Beneath, the top ten matches are displayed. The left column displays the name of the re-
cording. The right column shows the Euclidian distance between the vectors. The maximum
distance between two normalized vectors is 1.
Matching with C_0825_PayloadPos2_LooseGearFault.WAV:
C_0825_PayloadPos1_LooseGearFault.WAV = 0.143330
D_0918_PayloadPos1.WAV = 0.144296
A_0907_PayloadPos2.WAV = 0.145872
C_0827_PayloadPos2.WAV = 0.148169
A_0824_PayloadPos2.WAV = 0.149204
A_0721_NoPayloadPos1_NotchFault.WAV = 0.150612
C_0816_1242_NoPayloadPos2.WAV = 0.152333
A_PayloadPos1.WAV = 0.155065
A_0918_NoPayloadPos2.WAV = 0.156066
C_0918_PayloadPos2.WAV = 0.156285
A comparison between the feature vector from C_0825_PayloadPos2_LooseGearFault.WAV
is displayed in the plot below:
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
0 50 100 150 200 250 300
line 1
The feature vector from D_0918_PayloadPos1.WAV displayed in the plot below:
36
0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.1 0.11 0.12
0 50 100 150 200 250 300
line 1
The plots show that they both have prominent peaks at frequency #70. The dip around fre-
quency #165 is also represented in both plots.
4.6.8 Example2 A robot with a notch fault, was recorded on 16th of July. The name of the sound file is
C_0825_PayloadPos2_LooseGearFault.WAV The sound from a loose gear is characterised by
four bumps at the end of each rotation.
Beneath, the top ten matches are displayed. The left column displays the name of the re-
cording.
Matching with 175-2.8_0716_NoPayloadPos1_NotchFault.WAV:
175-2.8_0730_NoPayloadPos1_NotchFault.WAV = 0.011827
125-3.2_0721_NoPayloadPos1.WAV = 0.148176
125-3.2_0824_PayloadPos1.WAV = 0.150532
0729_1235_NoPayloadPos1.WAV = 0.151999
175-2.55_0906_NoPayloadPos1.WAV = 0.152911
175-2.55_0918_PayloadPos1.WAV = 0.154718
175-2.8_0914_PayloadPos2.WAV = 0.156401
175-2.8_0816_1242_NoPayloadPos2.WAV = 0.157412
125-3.2_0918_NoPayloadPos1.WAV = 0.158561
175-2.8_0914_PayloadPos1.WAV = 0.159770
37
5 Conclusions 5.1 Result
A prototype has been implemented as proof the concept. The prototype takes a recording as
input and produces a list of matches as out put. The matches are older recordings stored in a
case database. The prototype has been successful in detecting both types of the recorded faults
A total of 17 different robots have been recorded and used in the evaluation of the prototype.
11 of them have two recording done with two different positions of the microphone. In both
positions of the microphone, faults have successfully been identified.
The prototype is not a “research prototype” and not a complete application and it is not ready
to be used as one either. It is limited to one robot model and one axis and it is not tested on
recordings performed with other types of microphones. The rotation speed of the axis is fixed
in the prototype. It means that other rotation speeds of the robot axis cannot be used.
5.2 Future work
A further development of the prototype is suggested to make it more general.
Desirable improvements include:
1. Variable axis speed in recordings.
2. The possibility to diagnose recordings of other axes and other robot models.
3. A modification of the feature vector to improve the ability of representing faults and
matching of faulty sounds.
38
5.3 Summary
This thesis shows that sound analysis proves useful in detection of mechanical faults in indus-
trial robots. Two types of faults have been detected and identified by a prototype system im-
plemented as a part of the thesis. The prototype is implemented in C++ and uses wavelet
analysis combined with case based reasoning as methods to extract features and make a diag-
nosis of the recorded sound. The prototype is more flexible since one recording of a faulty
sound is sufficient to identify similar faults in the future. No further training or interaction
with experts in sound analysis is necessary. This makes the “smart tool” flexible and it will
improve performance as more recordings of faulty robots are made on field. The aim of the
thesis project where met.
39
References
[1] Anreus H., Grante A. Utveckling av mätmetod för ljudprovning av transmissioner i
anslutning till produktionen. Examensarbete MMK 2003:46 MME 735.
[2] Benko U., Petrovcic J., Juricic D., Tavcar J., Rejec J. Quality End-Test Of Vacuum
Cleaner Motors Based On Sound Analysis.
[3] Mäkinen E. J., Vibrationsanalys på kylkompressorer. Examensarbete YTH-Underhåll,
Högskolan i Karlstad 1994,
http://www.kylteknikern-em.se/vibrationsanalys.htm
[4] Vallin K. Mätdatorer. ABB Robotics avd ROP/PA 97.
[5] Ian Kaplan´s wavelet packet algorithm
http://www.bearcave.com/misl/misl_tech/wavelets/index.html
[6] Various wavelet tutorials
http://www.amara.com/current/wavelet.html
[7] Audacity sound recording software
http://www.audiost.com/advr.htm
40
Appendix A, Graphical plots of sound recordings In this section, all sound recordings are presented as graphical plots. The plots are intermedi-
ate results extracted from within the prototype. They are extracted between step 2 and step 3
in the system model (see chapter 4).
Figure 30: A_0729_1235_NoPayloadPos1
41
Figure 31: A_0721_NoPayloadPos1
Figure 32: A_0721_1301_NoPayloadPos1
42
Figure 33: A_0721_NoPayloadPos1_NotchFault
Figure 34: A_0824_PayloadPos1
43
Figure 35: A_0824_PayloadPos2
Figure 36: A_0826_PayloadPos1
44
Figure 37: A_0826_PayloadPos2
Figure 38: A_0907_0928_PayloadPos1
45
Figure 39: A_0907_0928_PayloadPos2
Figure 40: D_0907_PayloadPos1
46
Figure 41: A_0918_NoPayloadPos1
Figure 42: A_0918_NoPayloadPos2
47
Figure 43: C_0825_PayloadPos1_LooseGearFault
Figure 44: C_0825_PayloadPos2_LooseGearFault
48
Figure 45: C_0827_PayloadPos1
Figure 46: C_0827_PayloadPos2
49
Figure 47: C_0906_NoPayloadPos1
Figure 48: C_0906_NoPayloadPos2
50
Figure 49: C_0918_PayloadPos1
Figure 50: C_0918_PayloadPos2
51
Figure 51: B_0716_NoPayloadPos1_NotchFault
Figure 52: B_0730_NoPayloadPos1_NotchFault
52
Figure 53: B_0816_1242_NoPayloadPos2
Figure 54: B_0914_PayloadPos1
53
Figure 55: B_0914_PayloadPos2
54