physic ad 117
TRANSCRIPT
-
7/27/2019 Physic Ad 117
1/16
Physica D 117 (1998) 283298
Comparisons of new nonlinear modeling techniques
with applications to infant respiration
Michael Small , Kevin JuddDepartment of Mathematics, University of Western Australia, Nedlands, WA 6907, Australia
Received 1 August 1997; received in revised form 23 October 1997; accepted 2 December 1997
Communicated by A.M. Albano
Abstract
This paper concerns the application of new nonlinear time-series modeling methods to recordings of infant respiratorypatterns. The techniques used combine the concept of minimum description length modeling with radial basis models. Our
first application of the methods produced results that were not entirely satisfactory, particularly with respect to accurately
modeling long term quantitative and qualitative features of respiration patterns. This paperdescribes a number of modifications
of the original methods and makes a comparison of the improvements the various modifications gave. The modifications made
were increasing the class of basis function, broadening the range of possible embedding strategies, improving the optimization
of the likelihood of the model parameters and calculating a closer approximation to description length. The criteria used in
the comparisons were description length, root-mean-square prediction error, model size, free-run behavior and amplitude size
and variation.
We use surrogate data analysis to confirm the hypothesis that the recorded data are consistent with the nonlinear models we
construct, and not consistent with simpler models. We also investigate the free-run dynamics of the model systems to see if the
models exhibit features consistent with physiological characteristics observed independently in the respiratory recording. This
involved modeling breathing patterns prior to a sigh and onset of a phenomenon called periodic breathing; and comparing
the period of cyclic amplitude modulation of the free-run dynamics of the models and the period of the subsequent periodic
breathing observed in respiration recording. The periods were consistent in six out of seven recordings. Copyright 1998Elsevier Science B.V.
Keywords: Radial basis modeling; Description length; Surrogate analysis; Infant respiratory patterns; Periodic breathing
1. Introduction
This paper describes an attempt to accurately model
the respiratory patterns of human infants using new
nonlinear modeling techniques. We have identified
periodic fluctuation in regular breathing pattern of
sleeping infants using linear modeling techniques
Corresponding author. Tel.: +618 9380 3348; fax: +6189380 1028; e-mail: [email protected].
[3]. An accurate, reliable and replicable method of
building nonlinear models may further aid the iden-
tification of such subtle periodicities and give some
insight into the mechanisms generating them. Just as
a differential equation model of a system can lead to
greater understanding, so too can numerical, nonlinear
models. In this paper it is our aim to produce accu-
rate models of nonlinear systems (infant respiration)
from a scalar time series measurement of that system
(abdominal volume).
0167-2789/98/$19.00 Copyright 1998 Elsevier Science B.V. All rights reserved
PII S 0 1 6 7 - 2 7 8 9 ( 9 7 ) 0 0 3 1 1 - 4
-
7/27/2019 Physic Ad 117
2/16
284 M. Small, K. Judd/ Physica D 117 (1998) 283298
Initially we used a radial basis modeling algorithm
described by Judd and Mees [1] to model recordings
of the abdominal movements of sleeping infants. Al-
though these radial basis models give accurate short-
term predictions, they were not entirely satisfactory in
the sense that simulations of the models failed to ex-
hibit some characteristics of the original signals. Aftersome alteration of the model building algorithm, much
better results were obtained; simulations of the models
exhibit signals that are nearly indistinguishable from
the original signals.
In this paper we first describe the time series we
will model and briefly review the nonlinear modeling
methods of Judd and Mees [1]. We identify some fail-
ings of simulations of models produced by this algo-
rithm; suggest modifications that may overcome these
problems; and finally demonstrate the improved re-
sults we have obtained.
1.1. Data collection
For this study we collected measurements pro-
portional to the cross-sectional area of the abdomen
of infants during natural sleep (using standard non-
invasive inductive plethysmography techniques). Such
measurements are a gauge of lung volume.
Nineteen healthy infants were studied at one, two,
four and six months of age, in the sleep labora-
tory at Princess Margaret Hospital. The study was
approved by the Princess Margaret Hospital ethics
committee.The unfiltered analog signal from an inductance
plethysmograph was passed through a DC ampli-
fier and 12 bit analog to digital converter (sampling
at 50 Hz). The digital data were recorded in ASCII
format and were then transferred to Unix work-
stations at the University of Western Australia for
analysis.
The only practical limitation on the length of time
for which data could be collected is the period when
the infant remains asleep and still. The cross-sectional
area of the lung varies with the position of the infant.
However, in this study we are interested only in the
variation due to the breathing and so we have been
careful to avoid artifact due to changes in position or
band slippage. We have made observations of up to
2 h that are free from significant movement artifacts,
although typically observations are in the range 5
30 min.
1.2. Pseudo-linear radial basis modeling
We have previously used these data to estimate the
correlation dimension of the respiratory patterns of
sleeping infants [2], and to identify cyclic amplitude
modulation (CAM) in respiration during quiet sleep
[3]. Both these studies concluded that linear model-
ing techniques were unable to model the dynamics
of human respiration. 1 Furthermore, by comparing
the correlation dimension estimates for the data and
surrogates we were able to demonstrate that simula-
tions from radial basis models produced dimension
estimates that closely resembled that of the data [2].This implies that nonlinear models are more accurately
modeling the data than are linear models. However,
these nonlinear models appeared to have difficulty
with some data sets, most notably those with sub-
stantial noise contamination and data exhibiting non-
stationarity. In this paper we attempt to improve the
modeling techniques.
2. Modeling respiration
In this section we introduce the data set that we
will attempt to model. We use correlation dimension
estimation and false nearest neighbor techniques to
determine a suitable embedding dimension and exam-
ine three alternative criteria for embedding lag to de-
duce an appropriate value. We then apply the nonlinear
modeling technique described by Judd and Mees [1] to
this data set and examine the weaknesses of the result.
1 By calculating correlation dimension dc(0) for data embed-
ded in ,3 4 and 5 as a test statistic surrogate analysis of 27
recordings of infant respiration from 10 infants concluded thatthe data were inconsistent with each of the linear hypothesesaddressed by Theiler et al. [4]. The details of these calculationsare beyond the scope of this discussion, see [2,5].
-
7/27/2019 Physic Ad 117
3/16
M. Small, K. Judd / Physica D 117 (1998) 283298 285
Fig. 1. Data. The data we use in our calculations. The solid line represents the data set from which we build our radial basis models.The horizontal axis is time elapsed from the start of data collection and the vertical axis is the output from the analog to digitalconvertor (proportional to cross-sectional area measured by inductance plethysmography). Note the sigh (at about 300 s) and the
onset of periodic breathing following this.
Fig. 2. Periodic breathing. An example of a short episode of periodic breathing after a sigh (at 580 s on the second panel). Smaller
sighs are also present at about 275 and 470 s on the first panel. The horizontal axis is time elapsed from the start of data collectionand the vertical axis is the output from the analog to digital convertor (proportional to cross-sectional area measured by inductanceplethysmography).
2.1. Data
For much of the following sections we illustrate
the calculation and comparison using just one record-
ing, selected because it is a typical representation
of a range of important dynamical features. The data
set we use (see Fig. 1) is from a section of approx-
imately 10 min of respiration of a two-month-old
female in quiet (stages 3 and 4) sleep. These data
exhibit a physiological phenomenon of great interest
to respiratory specialists known as periodic breath-
ing [6,7]. Periodic breathing is simply extreme CAM
the minimum amplitude decreases to zero. Fig. 2
shows an example of periodic breathing. In all other
respects these data are typical of many of our record-
ings. The section which we examine first is from a
period of quiet sleep preceding the onset of periodic
breathing.
-
7/27/2019 Physic Ad 117
4/16
286 M. Small, K. Judd/ Physica D 117 (1998) 283298
Fig. 3. False nearest neighbors. False nearest neighbor calculation for the data illustrated in Fig. 1 (1600 points sampled at 12 .5Hz)embedded with a time delay embedding,
=5 (Rtol
=15).
2.2. Embedding
Using a time delay embedding strategy and appeal-
ing to Takens [8] embedding theorem we produce
from our scalar time series
y1, y2, y3, . . . ,
a d-dimensional vector time series. Consider the
embedding
yt zt = (yt, yt2, . . . , ytd) t > d.To perform this transformation one must first identify
the embedding lag and the embedding dimension d.
We describe the selection of suitable values of these
parameters in the following paragraphs.
Any value of is theoretically acceptable, but the
shape of the embedded time series will depend criti-
cally on the choice of and it is wise to select a value
of which separates the data as much as possible.
General studies in nonlinear time series [9] suggest the
mutual information criterion [10], the autocorrelation
function [11] or one of several other criteria to choose
. Our experience and numerical experiments sug-
gest that selecting a lag approximately equal to one
quarter of the quasi-period of the time series pro-
duces comparable results to the autocorrelation func-
tion but is more expedient. Note that the first zero
of the autocorrelation function will be approximately
the same as one quarter of the quasi-period if the
data are almost periodic. Numerical experiments with
these data have shown that either of these methods
produce superior results to the mutual information
criterion.
Suitable bounds on d can be deduced by using afalse nearest neighbor analysis [12]. Numerical ex-
periments indicate that four dimensions are sufficient
to remove false nearest neighbors from the data, see
Fig. 3. Furthermore, it is at approximately this em-
bedding dimension that the correlation dimension es-
timates appear to plateau. Takens sufficient condition
on successful recreation of the attractor by embedding
requires d > 2dc + 1 where dc is the correlation di-mension of the attractor. For our data with 3 < dc 4(see [2]), this would suggest that d > 8 is necessary.
However, embedding in this dimension offers no im-
provement to the modeling process and our false near-
est neighbor calculations indicate that a much smaller
value of d is sufficient.
-
7/27/2019 Physic Ad 117
5/16
M. Small, K. Judd / Physica D 117 (1998) 283298 287
Fig. 4. Initial modeling results. Free-run prediction and noise-driven simulation of a radial basis model. The plot on the left is a
free-run prediction with no noise, on the right is a simulation driven by Gaussian noise at 10% of the root-mean-square prediction error
(
ti=1
2i
/
N). The horizontal axis is yt for t = 1, . . . , 500, the vertical axis is the output from the analog to digital convertor(proportional to cross-sectional area measured by inductance plethysmography). From 30 trials, 27 of them exhibited fixed points.
2.3. Modeling
We attempt to build the best model of the form
yt+1 = f (zt) + t,where t is the model prediction error and f :
d is of the form
f (zt) = 0 +n
i=1i ytli
+m
j=1j+n+1
zt cjrj
, (1)
where rj and j are scalar constants, 1 li < li+1 d are integers and cj are arbitrary points in
d.
The integer parameters n and m are selected to min-imize the description length [10] as described in [1].
Here () represents the class of radial basis functionfrom which the model will be built. We choose to use
Gaussian basis functions because they appear to be
capable of modeling a wide variety of phenomena.
The data set consists of 20 000 points sampled at
50 Hz. This is oversampled for our purposes and we
thin the data set to one in four points and truncate it
to a length of 1600 (see Fig. 1). We set d = 4 andchoose = 5.
Trials with the modeling algorithm as described in
[1] produced some problems with the model simula-
tions (see Fig. 4). None of the simulations look like
the data. When periodic orbits are evident they are still
unlike the data; the waveform is symmetric, whereas
the data have a definite asymmetry. Moreover the free-
run models often exhibit stable fixed points. This is
extremely undesirable as it is evidently not an accu-
rate representation of the dynamics of respiration
breathing does not tend to a fixed point, usually.
The remainder of this paper shall be concerned with
addressing these problems. These problems are the
result of three main deficiencies in the initial mod-
eling algorithm: (i) it over fits the data; (ii) it does
not produce appropriate simulations; and (iii) models
are not consistent or reproducible. We will attempt to
improve upon these problems whilst considering the
many competing criteria for a good model.
3. Improvements
Before we can attempt to improve our modeling
procedure we must be clear on what we mean by im-
provement. There are several criteria that might be
imposed to achieve a good model.
Modeling criteria measure quantities such as the
number of parameters in the model, its prediction error
and description length. It is desirable to have a model
with few parameters, a small description length and a
small root mean square prediction error.
Algorithmic criteria are concerned with optimizing
the modeling algorithm, to ensure that it searches the
broadest possible range of basis functions as efficiently
-
7/27/2019 Physic Ad 117
6/16
288 M. Small, K. Judd/ Physica D 117 (1998) 283298
as possible. Unfortunately a larger search space comes
at the expense of more computation.
Qualitative criteria consider properties of the dy-
namics of models; for example, the behavior observed
in the simulations of the model. In modeling breath-
ing, for example, we expect something like stable
periodic (or quasi-periodic) solutions; divergence orstable fixed points seem unlikely. Furthermore we ex-
pect the shape of the periodic solution to closely match
the shape of the data and to occupy the same region
of phase space.
Modeling results should also be reproducible and
representative. It does not seem unreasonable to ex-
pect consistent, repeatable results from a modeling
algorithm, both qualitatively and quantitatively. Re-
producibility can be examined by repeatedly modeling
a single data set. Furthermore, the model should be
representative in that when making many simulations
of the model, we ought to obtain time series of whichthe original data are representative. Representativity
can be measured with the assistance of surrogate tests
using a statistic such as the correlation dimension es-
timates or cyclic amplitude modulation.
In the following sections we consider improvements
of the basic modeling procedure by: (i) broadening the
class of basis functions; (ii) using a more targeted se-
lection algorithm; (iii) making more accurate estimates
of description length; (iv) local optimization of nonlin-
ear parameters; (v) using reduced linear modeling to
determine embedding strategies; and (vi) simplifying
the embedding strategies using a form of sensitivityanalysis.
3.1. Basis functions
In this section we introduce a broader class of ba-
sis functions. This will produce an algorithm that is
capable of modeling a wider range of phenomena.
First we expand the embedding strategy so that
instead of radial (spherical) basis functions we in-
troduce cylindrical basis functions. Detailed argu-
ments about the advantages of these basis functions aredescribed elsewhere [13]. Generalize the functional
form (1) to
f (zt) = 0 +n
i=1i ytli
+m
j=1j+n+1
Pj(zt cj)rj
, (2)
where li , rj, j, cj, n, m are as described previously
and Pj d: dj (dj < d) are projections ontoarbitrary subsets of coordinate axes.
The functions Pj can be thought of as a local em-
bedding strategy. Each basis function has a different
projection Pj and so each Pj(zt cj) is dependenton a different set of coordinate axes.
We actually generalize the choice of embed-
ding strategy further by selecting the best lags
from the set {0, 1, 2, . . ., d}, not only subsets of{0, , 2 , . . . , d }. It seems that by allowing the se-lection of different embedding strategies in different
parts of phase space the model gives better free-run
behavior. This indicates that, naturally enough, the
optimal embedding strategy is not uniform over phase
space. Selecting from this larger set of embedding
lags is equivalent to embedding with a time lag of 1 in
d. However, the modeling algorithm rarely selects
more than a d-dimensional local embedding. There-
fore, these improved results are not contrary to our
previous estimates of optimal embedding dimension.
They do allow for an embedding in more than 2 dc +1dimensions (satisfying Takens sufficient condition) if
necessary. As noted earlier the choice of embedding
lag is largely arbitrary.Furthermore, to increase the curvature of the basis
functions we replace the choice of
(x) = exp
x2
2
by
(x,) = exp
(1 ) x
,
where 1 < < R is the curvature 2 and (1 )/ is acorrection factor so that
(x,) dx = 1. Hence,
maintaining consistent notation
2 To prevent large values of the second derivative of f it isnecessary to provide an upper bound R on .
-
7/27/2019 Physic Ad 117
7/16
M. Small, K. Judd / Physica D 117 (1998) 283298 289
(x,) =
2(1 )
x/2
,
and the basis functions become functions of the form
2(1 j)
j
Pj(zt cj)rj
j/2
,
where
(x) = exp x2
2.
Broadening the class of basis functions has in-
creased the complexity of the search algorithm.
Hopefully, it will also have broadened the search
space sufficiently to encompass functions which can
more accurately model the data. To overcome this
increased search space we consider a more efficient
search algorithm.
3.2. Directed basis selection
The method of Judd and Mees [1] involves ran-
domly generating a large set of basis functions
{(z cj/rj)}j = {j}j and evaluating them ateach point of the embedded time series z to give the
matrix V = [1|2| |M]. Following an iterativescheme they repeatedly select columns from this ma-
trix (and the corresponding candidate basis function)
to add to the optimal model.
Instead, we select a new set of candidate basis func-
tions {j}j (and a new matrix V) at each expansion ofthe optimal model. We then identify the column k ofV
that best fits the residuals (orthogonal to the previously
selected basis functions) and select the corresponding
basis function k . All the other candidate basis func-
tions {j}j=1,...,M;j=k are ignored and forgotten at thenext iteration. Because a new set of basis functions
is selected at each expansion, all the candidate basis
functions are much more appropriately placed. 3
The improvement in modeling achieved by this will
require a greater computation time. Furthermore, the
selection of basis functions that more closely fit the
3 Basis functions are selected according to either a uniformdistribution or the probability distribution induced by the mag-nitude of the modeling prediction error.
data may, possibly, increase the number of basis func-
tions allowed by the description length criterion. To
alleviate this problem we introduce a harsher more
precise version of description length.
3.3. Description length
The minimum description length criterion, sug-
gested by Rissanen [10], is used by Judd and Mees
[1] to prevent over fitting. However, the original im-
plementation of minimum description length used by
Judd and Mees only provides a description length
penalty for the coefficient j of each of the radial ba-
sis functions (and linear terms). Each basis function
also has a radius rj and coordinates cj which must
also be specified to some precision, and hence should
also be included in the description length calculation.
In [1] j is j truncated to some finite precision j,
then the description length is expressed as
L(z, ) = L(z|) + L(), (3)where
L(z|) = ln P (z|)is the description length of the model prediction errors
(the negative log likelihood of the errors) and
L() m+n+1
j=1ln
j
is the description length of the truncated parameters,
is an inconsequential constant.
We generalize Eq. (3) and include the finite preci-
sions ofrj and cj. Let represent the vector of all the
model parameters (j, cj, and rj) and the truncation
of those parameters to precision . Then
L(z, ) = L(z|) + L(), (4)where
L() (d+2)m+n+1
j=1
ln
j.
Now the problem becomes one of choosing to
minimize (4). By assuming that is not far from the
-
7/27/2019 Physic Ad 117
8/16
290 M. Small, K. Judd/ Physica D 117 (1998) 283298
maximum likelihood solution (see Section 3.4), one
can deduce that
L(z, ) L(z|) + 12 TQ
+ k ln k
j=1ln j, (5)
where k = ((d+ 2)m + n + 1). Minimizing (5) gives(as in [1])
(Q)j = 1/j,
where Q = DL(z|) is the second derivative ofthe negative log likelihood with respect to all the
parameters.
Although algebraically complicated, this expression
can be solved relatively efficiently by numerical meth-
ods. However, by assuming that the precision of the
radii and the position of the basis function must be
approximately the same, 4 one can circumvent a great
deal of the computational difficulty, and simply cal-
culate the precision of j assuming the same values
for the corresponding precisions of the coordinates cj.
Much of the computational complexity of calculat-
ing description length could be avoided by utilizing
the Schwarz criterion [14]. Indeed, from experience
it appears that the Schwarz criterion gives compara-
ble size models. However, Schwarz criterion does not
take into account the relative accuracy of different ba-
sis functions an important feature of minimum de-
scription length.
3.4. Local optimization of nonlinear model
parameters
Once the best (according to sensitivity analysis) ba-
sis function has been selected we improve on its place-
ment by attempting to maximize the likelihood
P (z|, 2)
= 1(2 2)N/2
exp(y V )T(y V )
22,
4 Since a slight change in radii will affect the evaluation of abasis function over phase space in the same way as an equalsmall change in the position of the basis function.
where y V = is the model prediction error, and2 is the standard deviation of the (assumed to be)
Gaussian error. By setting 2 =ti=1 2i /N and tak-ing logarithms one gets that
ln P (z
|)
=N
2 +ln2
N
N/2
+ ln
ti=1
2i
N/2. (6)
To maximize the likelihood we optimize Eq. (6) by
differentiating ln (t
i=1 2i )
N/2 with respect to rj, cj,
and j. This calculation is algebraically messy, but
computationally straightforward provided a good op-
timization package is used.
3.5. Reduced linear modeling for embeddingselection
Allowing different embedding strategies from such
a wide class (due to the expansion of the class of basis
functions in Section 3.1) increases the computational
complexity of the modeling process. However, to cir-
cumvent this we note that for Gaussian basis functions
the first-order Taylor Series expansion gives
Pj(zt cj)rj
= d
i=1 pi (zt cj)2rj
d
i=1
|pi (zt cj)|rj
, (7)
where pi :d is the coordinate projection onto
the ith coordinate. We then build a minimum descrip-
tion length model of the residual of the form (7). From
this we deduce that the basis functions selected are
a good indication of an appropriate embedding strat-
egy. Although this method is approximate it is hoped
that this will provide a useful and efficient innovation
within the modeling algorithm.
-
7/27/2019 Physic Ad 117
9/16
M. Small, K. Judd / Physica D 117 (1998) 283298 291
Fig. 5. Improved modeling results. Free-run prediction and noise-driven simulation of a radial basis model. The plot on the left is afree-run prediction with no noise, on the right is a simulation driven by Gaussian noise at 10% of the root-mean-square prediction
error (
ti=1
2i
/
N). The horizontal axis is yt for t = 0, . . . , 500, the vertical axis is the output from the analog to digitalconvertor (proportional to cross-sectional area measured by inductance plethysmography).
3.6. Simplifying embedding strategies
Our final, very rudimentary alteration is designed
to account for some of the approximation required in
the reduced linear modeling of the embedding strate-gies. Given an embedding strategy suggested by the
method of Section 3.5, we generate additional can-
didate basis functions by using embedding strategies
whose coordinates are subsets of the coordinates of the
embedding strategy suggested by the linear modeling
methods.
4. Results
After implementing the alterations described in the
preceding section, we again apply our methods tothe same data set. This section describes the results
of these calculations and examines some of the im-
provements in the final model. We also examine the
individual effect of each modification and the effec-
tiveness of this modeling procedure in seven different
data sets (from six infants). Because of its physiologi-
cal significance, all the data sets selected for this anal-
ysis exhibit CAM suggestive of periodic breathing. We
compare dimension estimates for the original data sets
and simulations from the models. Finally, we apply
a linear modeling technique discussed elsewhere [3]
to detect CAM within the respiratory traces of sleep-
ing human infants, and present some results. That is,
we compare the CAM present in the data following a
sigh to that present in the models built from the data
preceding the sigh.
4.1. Improved modeling
Fig. 5 shows a section of free-run prediction, and
noisy simulation for a representative model. Using
an interactive three-dimensional viewer (see Fig. 6),
it is possible to determine that these models also have
many more common structural characteristics than
those created in Section 2.3. The size, placement,
shape and local embedding dimensions of the basis
functions of the models have many similarities.
Importantly, all of these models have a similar free-
run behavior. The free-run predictions are as large (in
amplitude) as the data; this was a substantial problem
with the original modeling procedure. Moreover, thefree-run behavior with noise appears more realistic
and the shape of the simulations mimics very closely
that of the data. Fig. 7 shows a short segment of a
simulation, along with the data. Note the similarities
in the shape of the prediction and the data. Finally,
the simulations exhibit a measurable cyclic amplitude
modulation which we use in Section 4.3.2 to infer
the presence of cyclic amplitude modulation in the
original time series.
4.2. Effect of individual alterations
Table 1 lists some characteristics of models built
from the data in Fig. 1 using various methods. The
-
7/27/2019 Physic Ad 117
10/16
292 M. Small, K. Judd/ Physica D 117 (1998) 283298
Fig. 6. Cylindrical basis model. A pictorial representation of the interactive three-dimensional viewer we used. The axes range from
1.715415 to 3.079051, the same range of values as the data. The point (1.7, 1.7, 1.7) is in the front center, foreground.The cylinders, prisms and sphere represent the placement (cj) and size (rj) of different basis functions with different embeddingstrategies. the X, Y, and Z coordinates shown correspond to yt, yt5, and yt15, respectively. The coloring of the basis functionsrepresents the value of the coefficients (j).
different modeling strategies are: (A) the initial
method (described in Section 2.3); (B) extended ba-
sis functions and embedding strategies (Section 3.1);
(C) directed basis selection (Section 3.2); (D)
more accurate approximation to description length
(Section 3.3); (E) local optimization of nonlin-
ear model parameters (Section 3.4); (F) reduced
linear modeling to select embedding strategies
(Section 3.5); and (G) simplifying embedding strate-
gies (Section 3.6). These alterations to the algorithm
were progressively added in various combinations
and characteristics of the observed models measured.
The initial procedure (A) produced very bad free-
run predictions; 27 out of 30 trials produced simula-
tions with fixed points. Extending the class of basis
functions and adding cylindrical basis functions (B)
vastly improved this (only eight out of 30 simula-
tions did not have periodic (or quasi-periodic) orbits).
Most of the periodic orbits in these simulations were
smaller than the data (did not occupy the same part
of phase space) and one divergent simulation was ob-
served (hence the large standard deviation in Table 1).
This approach decreased the prediction error without
affecting either the model size or description length
(clearly, the required precision of the parameters was
greater).
Directed basis selection (C) greatly increased the
size of the model and decreased error whilst improv-
ing free-run behavior not only in amplitude but also
shape. The increase in computational time could al-
most entirely be due to the greater model size. Improv-
ing the description length calculation (D) decreased
the model size whilst, predictably increasing predic-
tion error. This also caused a surprising increase in
-
7/27/2019 Physic Ad 117
11/16
M. Small, K. Judd / Physica D 117 (1998) 283298 293
Fig. 7. Short-term behavior. Comparison of simulation and data. The solid line is the data, the dotdashed is a free-run prediction,
the dashed is a simulation driven by noise (20% of
ti=1 2i /N). The initial conditions for the artificial simulations are identical
and are taken from the data. The vertical axis is the output from the analog to digital convertor (proportional to cross-sectional area
measured by inductance plethysmography).
Table 1
Algorithmic performance: Comparison of the modeling algorithm with various improvements
Modeling method Nonlinear parameters RMS error MDL Free-run amplitude CPU time (s)ti=1
2i
/
N
A 12.52.4 0.1350.016 1086157 0.000.91 283103A + B 12.52.4 0.1130.011 1090155 1.2231.90 292113A + B + C 24.54.3 0.1040.015 1123198 1.581.04 567167A + B + D 10.72.3 0.1220.016 975191 0.3424.91 744504A + B + C + D 14.53.5 0.1230.018 909210 1.501.09 16261203A + B + C + D + E 9.52.9 0.1410.012 735131 1.591.31 20471253A + B + C + D + F 13.73.6 0.1260.009 87081 1.3117.48 43481959A + B + C + D + E + F 11.03.1 0.1170.013 990119 1.1717.94 58422973A + B + C + D + E + F + G 11.43.2 0.1170.011 980110 1.871.00 57532360
The seven different modeling procedures are the initial routine described by Judd and Mees, and six alterations described in
Section 3. Modeling methods are: (A) the initial method; (B) extended basis functions and embedding strategies; (C) directed basisselection; (D) exact description length; (E) local optimization of nonlinear model parameters; (F) reduced linear modeling to select
embedding strategies; and (G) simplifying embedding strategies. Results are from 30 attempts at modeling the data described inSection 2.1 and Fig. 1. The numbers quoted are (mean value)(standard deviation). Calculations were performed on a SiliconGraphics Indy running at 133MHz with 16 Mbytes of RAM. CPU time is measured in seconds using MATLABs cputime
command.
calculation time an indication of the computational
difficulty solving (Q)j = 1/j, where Q is thesecond derivative with respect to all the model para-
meters (or at least and r). Because there is a harsh
penalty these models are far less likely to be over
fitting the data. Combining the improved description
-
7/27/2019 Physic Ad 117
12/16
294 M. Small, K. Judd/ Physica D 117 (1998) 283298
Fig. 8. Periodic breathing. An example of periodic behavior in one of our data sets. The solid region was used to build a nonlinear
radial basis model. Note that periodic breathing begins immediately after the sigh. The vertical axis is the output from the analog todigital convertor (proportional to cross-sectional area measured by inductance plethysmography).
length calculation and directed basis selection pro-
duced models comparable in both size and fitting error
to before either alteration was implemented (A + B).However, free-run behavior had an amplitude closer
to the mean amplitude of the data and exhibited anasymmetric waveform similar to the data.
Addition of the nonlinear optimization (E) and local
linear modeling (F) routines caused the greatest in-
crease to computational time. Individually these meth-
ods did not offer any considerable improvement to
the other model characteristics. However, many of the
statistics indicate a decrease in the variation between
trials. Combined, these modifications gave a slight im-
provement in prediction error and description length
whilst making the model smaller. They produced more
realistic simulations although the amplitude was
smaller than that of the data.Finally, the simple procedure of checking that sim-
pler embedding strategies would not produce better
(or equally good) results (G) caused a substantial im-
provement. This is perhaps due in part to the previous
optimization and local linear methods, particularly the
approximate nature of the local linear modeling. Re-
moving the coordinates helped produce some appre-
ciable improvement in suitability of the embedding
strategies suggested by the approximate local linear
methods. The local linear methods often produce a
high-dimensional local embedding (many significant
coordinates), eliminating some of these will usually
only slightly increase the prediction error. This sim-
ple addition increases the amplitude to a realistic level
(approximately 1.9 whilst the mean breath size for the
data is about 2.3 5) whilst decreasing the proportion
of fixed point and divergent trajectories to the lowest
level (8 and 0 of the 30 models, respectively) with-
out appreciably changing the description length, pre-diction error, or model size whilst decreasing slightly
the calculation time (and variance in calculation time).
Furthermore, these models have far more structural
similarities (in the size and placement of basis func-
tions) than the previous models have, indicating that
these models are far more consistent.
The remainder of this section is devoted to some
applications of these modeling methods and tests of
their representability.
4.3. Modeling results
From over 200 recordings of 19 infants, we identi-
fied seven data sets from six infants for a more careful
analysis. All seven of these data sets include a sigh
followed by a period of breathing exhibiting cyclic
amplitude modulation (CAM) [3]. Our present discus-
sion examines the analysis of these data sets.
In this section we examine the free-run behavior
of data sets created from seven models of seven data
sets from six sleeping infants. We compare the corre-
lation dimension of the data and simulations from mo-
dels. Following this we compare the period of CAM
5 Note, however, that the data are slightly nonstationary whilstthe model is not.
-
7/27/2019 Physic Ad 117
13/16
M. Small, K. Judd / Physica D 117 (1998) 283298 295
Fig. 9. Surrogate calculations. Comparison of dimension estimates for data and surrogates. The three figures on the left are dimension
estimates (for embedding dimension from 3 to 5, shown from top to bottom) for a model of Bs2t8. The left three plots aresimilar results for a model of Ms1t6. All surrogates are simulation driven by Gaussian noise with a standard deviation of half theroot-mean-square one step prediction error. Each picture contains one dimension estimate for the data (solid line), and 30 surrogates
(dotted). The two data sets used in these calculations are shown in Figs. 1 and 8, respectively.
detected in the free-run predictions from the models to
that visually evident after a sigh. Fig. 8 illustrates oneof the data sets used in our analysis. This is the only
set of data to exhibit periodic breathing, the others
merely exhibited strong amplitude modulation after
the sigh for 2560s ( 1530 breaths). Neverthelessthe change that the respiratory system undergoes after
a large sigh is of great interest to respiratory physio-
logists. We examine the system before and after a sigh
to determine evident physiological similarities in the
mechanics of breathing.
For each of our seven data sets, we identify the lo-
cation of the sigh, and extract data sets of 1501 points
spanning 120 s preceding the sigh. From these data
sets the respiratory rate of each recording was estab-
lished and the period of respiration deduced. Each data
set was embedded in 4 with a lag equivalent to the
integer closest to one quarter of the period. We thenapplied our modeling algorithm.
4.3.1. Surrogate analysis
To determine exactly how similar data and model
simulations are, we employ an obvious generalization
of the surrogate data analysis used by Theiler et al. [4].
The principle of surrogate data is the following. One
first assumes that the data come from some specific
class of dynamical system, possibly fitting a paramet-
ric model to the data. One then generates surrogate
data from this hypothetical system (producing simu-
lations from the parametric model) and calculates var-
ious statistics of the surrogate and original data. The
surrogate data will give the expected distribution of
-
7/27/2019 Physic Ad 117
14/16
296 M. Small, K. Judd/ Physica D 117 (1998) 283298
Table 2
Periodic behavior: Comparison of CAM after apnea (apparent to visual inspection), the second set of results, and CAM detected in
the models limit cycle, the first set of results
Subject Sex Age Model CAM in free-run CAM after sigh
(months) size (breaths) (s) (breaths) (s)
A(As4t2) Male 6 8 (7) 56 a 14 a 5 25Bb(Bs2t8) Female 2 7 (6) 6 9 6 9Bb(Bs3t1) 6 (5) 5 10 5 10
G(Gs2t4) Female 2 4 (3) 5 11 5 9H(Hs1t2) Male 1 5 (3) 89 a 11 a 9 13
M(Ms1t6) Female 1 6 (4) None None 5 14.5R(Rs2t4) Male 2 8 (6) 9 18 8 16
Data sets Ms1t6 and Bs2t8 exhibited periodic breathing. For each data set marked cyclic amplitude modulation (CAM) occurred
after a sigh and was measured by inspection. Radial basis models were built on a section of quiet sleep preceding the sigh, noisefree limit cycles exhibited periodicities that were measured in both time and breaths from the simulation.
a Not strictly periodic but rather exhibited a chaotic behavior. Model size is m + n(m), see Eq. (2).
statistic values and one can check that the original
have atypical value. If the original data have atypical
statistics, then we reject the hypothesis that the sys-
tem that generated the original data is of the assumedclass. One always begins with simple assumptions and
progresses to more sophisticated models if the data
are inconsistent with the surrogate data. Theilers [4]
original paper frames surrogate analysis in the con-
text of noise-driven linear systems; a recent paper [17]
presents surrogate analysis in a more general context,
similar to the above.
In our case, however, we are not interested in
determining what type of system generated the data
at least not at present (we have considered this else-
where [2]). A simpler null hypothesis (for example
[15,16]) consistent with the data does not concern ushere. What is of greater interest to us is determin-
ing if the models really do behave like the data. By
calculating models and generating free-run predic-
tions from those models, we are in fact generating
surrogate data. The similarity of the value of various
statistics applied to data and surrogate can be used to
gauge the accuracy of the model. Fig. 9 shows calcu-
lations of correlation dimension estimates (following
the methods of Judd [18,19]) for data and surrogate.
Our calculations indicate a very close agreement be-
tween the correlation dimension of the data and that of
the simulations. In six of the seven data sets the corre-
lation dimension estimate dc(0) for the data is within
two standard deviations of the mean value of dc(0)
estimated from the ensemble of surrogates for all val-
ues of0 for which both converged. In the remaining
data set the value of correlation dimension differed by
more than two standard deviations only at the small-est values of 0 (the finest detail in the data). In all
calculations, dc(0) for the data is within three stan-
dard deviations of the mean value of dc(0) estimated
from the ensemble of surrogates. With respect to cor-
relation dimension, our models are producing results
virtually indistinguishable from the data.
4.3.2. Application
Previously [3], we have used a form of reduced
autoregressive modeling (RARM) to detect CAM in
the regular breathing of infants during quiet sleep. We
apply nonlinear modeling methods here with two aimsin mind: to demonstrate the accuracy of our modeling
methods; and to further demonstrate that CAM evident
during periodic breathing and in response to apnea or
sigh is also present during quiet, regular breathing.
We have built nonlinear models following the meth-
ods outlined in this paper of the regular respiration
of six sleeping infants immediately preceding seven
sighs and the consequential onset of periodic or CAM
respiration. For each of these models we produce sim-
ulations both driven by Gaussian noise, and without
noise. The noiseless simulations approach a stable pe-
riodic (or chaotic, quasi-periodic) orbit which may ex-
hibit slight CAM. Table 2 summarizes the results of
these calculations.
-
7/27/2019 Physic Ad 117
15/16
M. Small, K. Judd / Physica D 117 (1998) 283298 297
In all but one data set, CAM was present in the free-
run prediction of the nonlinear model. The absence of
CAM in one model may either indicate a lack of mea-
surable CAM in the data or a poor model (the data
are illustrated in Fig. 8). All other data sets produced
nonlinear models that exhibited CAM, the period of
which matched that observed after a sigh during visu-ally apparent CAM.
5. Conclusion
We have successfully modified and applied pseudo-
linear modeling techniques suggested by Judd and
Mees [1] to respiratory data from human infants. We
found that the initial modeling procedure had some
difficulties capturing all the anticipated features of res-
piratory motion (it was not periodic). Some alteration
to the algorithm and a considerable increase to compu-tational time provided results which display dynamics
very similar to those observed during respiration of
infants in quiet sleep (not only did the models exhibit
a periodic limit cycle, but its shape was very similar
to the data).
Correlation dimension and the methods of surro-
gate data demonstrated that the models did indeed pro-
duce simulations with qualitative dynamical features
indistinguishable from the data. Short term free-run
predictions appeared to behave similarly to the data;
and, most significantly, we were able to deduce the
presence of CAM in sections of quiet sleep precedingsighs by observing this behavior in free-run predic-
tions of models built from that data. This confirms our
earlier observations from linear models of tidal vol-
ume [3] and the observation of a (greater than) two-
dimensional attractor in reconstructions from data [2].
Based on the results of Section 4, we are able to
deduce that some of the alterations (specifically ex-
tending the class of basis functions, and directed basis
selection) improved short-term prediction. Other alter-
ations reduced the size of the model (accurate approx-
imation to description length) and improved free-run
dynamics (extending the class of basis function, local
optimization and linear modeling methods to predict
embedding strategies). A combination of these meth-
ods is required to produce an accurate model of the
dynamics.
We conclude that the modeling methods presented
here and in [1] are capable of accurate modeling
breathing dynamics (along with a wide variety of other
phenomena, see for example [20]). Furthermore, we
have presented some evidence that the CAM presentduring periods of periodic breathing (when tonic
drive is reduced) is also present, but more difficult to
observe, during eupnea (normal respiration).
Acknowledgements
We wish to thank Madeleine Lowe and Stephen
Stick of Princess Margaret Hospital for Children for
supplying the infant respiratory data, and for physio-
logical guidance.
References
[1] K. Judd, A. Mees, On selecting models for nonlinear timeseries, Physica D 82 (1995) 426444.
[2] M. Small, K. Judd, M. Lowe, S. Stick, Is breathingin infants chaotic? Dimension estimates for respiratory
patterns during quiet sleep, J. Appl. Physiol., to appear.[3] M. Small, K. Judd, S. Stick, Linear modelling techniques
detect periodic respiratory behaviour in infants duringregular breathing in quiet sleep, Am. J. Resp. Crit. CareMed., 153 (1996) A79.
[4] J. Theiler, S. Eubank, A. Longtin, B. Galdrikian, J.D.Farmer, Testing for nonlinearity in time series: The method
of surrogate data, Physica D 58 (1992) 7794.[5] M. Small, K. Judd, Detecting nonlinearity in experimental
data, Int. J. Bifurc. & Chaos 8 (1998), in press.[6] D. H. Kelly, D.C. Shannon, Periodic breathing in infants
with near-miss sudden infant death syndrome, Pediatrics63 (1979) 355360.
[7] W.B. Mendelson, Human Sleep: Research and ClinicalCare, Plenum Medical Book Company, 1987.
[8] F. Takens, Detecting strange attractors in turbulence,Lecture Notes in Mathematics, vol. 898, 1981, pp. 366381.
[9] H.D.I. Abarbanel, R. Brown, J.J. Sidorowich, L.S.Tsimring, The analysis of observed chaotic data in physical
systems, Rev. Mod. Phys. 65 (4) (1993) 13311392.[10] J. Rissanen, Stochastic Complexity in Statistical Inquiry,
World Scientific, Singapore, 1989.
[11] M.B. Priestly, Non-linear and Non-Stationary Time SeriesAnalysis, Academic Press, New York, 1989.
[12] M.B. Kennel, R. Brown, H.D.I. Abarbanel, Determining
embedding dimension for phase-space reconstruction using
-
7/27/2019 Physic Ad 117
16/16
298 M. Small, K. Judd/ Physica D 117 (1998) 283298
a geometric construction, Phys. Rev. A 45 (1992) 34033411.
[13] K. Judd, A. Mees, Embedding as a modeling problem,Physica D, to appear.
[14] G. Schwarz, Estimating the dimension of a model, Ann.Stat. 6 (1978) 461464.
[15] J. Theiler, On the evidence for low-dimensional chaos in an
epileptic electroencephalogram, Phys. Lett. A 196 (1995)
335341.[16] J. Theiler, P. Rapp, Re-examination of the evidence
for low-dimensional, nonlinear structure in the humanelectroencephalogram, Electroencephalogr. Clin. Neuro-
physiol. 98 (1996) 213222.
[17] J. Theiler, D. Prichard, Constrained-realization Monte-Carlo method for hypothesis testing, Physica D 94 (1996)221235.
[18] K. Judd, An improved estimator of dimension and somecomments on providing confidence intervals, Physica D
56 (1992) 216228.[19] K. Judd, Estimating dimension from small samples,
Physica D 71 (1994) 421429.
[20] K. Judd, A. Mees, Modeling chaotic motions of a stringfrom experimental data, Physica D 92 (1996) 221236.