Discovering Dynamic Models

Lecture 21

Dynamic Models: Introduction

Dynamic models can describe how variables change over time or explain variation by appealing to mechanisms.

The informatics systems in this lecture address the discovery of three types of models.

Each system combines domain knowledge provided by an expert with heuristic search enacted by a computer.

Gretl: supports the discovery of descriptive, statistical models of time series.

MECHEM: generates qualitative models that explain the results of chemical reactions.

Prometheus: creates quantitative models that explain the dynamics of complex systems.

ARIMA Models

ARIMA models are commonly used in statistical time series analysis, particularly within the field of econometrics.

ARIMA stands for

Autoregressive: Xt is a linear combination of the prior p values and a random process Zt.Xt = a1Xt-1 + … + apXt-p + Zt

Integrated: a differencing method to remove a trend.e.g., Xt is replaced with Xt – Xt-1 for estimation

Moving Average: Xt is a linear combination of the random processes in the previous q values.Xt = μ + Zt + b1Zt-1 + … + bqZt-qAn ARIMA model describes the time series for one variable

and may be used for forecasting.

VARIMA models extend these to support multiple variables.

The Gretl Environment

Gretl is an informatics tool used by econometricians that supports a wide variety of statistical analyses.

The main window shows basic information about the data

Gretle supports several, customizable plots.

This data in this plot show seasonality and a general upward trend.

Input for Gretl

Gretl can discover the parameters of a model, but not its structure.

Scientists tell Gretl

• the variables to be described;

• the type of model to use; and

• the structure of the model.

Gretl supports ARIMA and other types of time-series models.

The structure of an ARIMA model reflects the number of terms.

Search in Gretl

Gretl uses maximum likelihood estimation to fit ARIMA parameters.

Xt = a1Xt-1 + … + apXt-p + Zt + μ + b1Zt-1 + … + bqZt-q

Where Z0 = 0, and

Zt = Xt – μ – (b1Zt-1 + … + bqZt-q)

This window shows the resulting parameters and fitness scores.

This graph shows the predictions (in blue) from an ARIMA model.

The forecast is in blue and its 95% confidence interval in green.

The observed values are in red.

The MECHEM Environment

MECHEM is an interactive tool that generates plausible reaction pathways for chemical interactions.

Scientists interact with MECHEM through a graphical interface as presented in the figure.

Clockwise from the upper left, the interface shows

• the main menu;

• the current reaction;

• an example mechanism;

• a set of constraints; and

• the output log.

Knowledge in MECHEM

1. Every conjectured intermediate has at most two O atoms

2. O atoms cannot be bonded to O atoms

3. Reject Eley-Rideal mechanisms

4. The site M must be present in all steps

5. No multiple occurrences of any reactant pair on the left-hand side of steps

6. CH4 cannot appear on the left-hand side of a step

Constraints on the structure of candidate mechanisms.

starting materials: CO, H2

catalyst site: Mdual catalyst sites: MMobserved C1 products: CO2, CH3OH, CH4, H2O

Reaction-specific information including chemical products.

MECHEM also includes a general set of parameters, such as, “Consider at most 7 conjectured species.”

The purpose of all this knowledge is to rule out a large number of implausible reaction-pathways.

Search in MECHEM

MECHEM uses heuristic search through a space of symbolic structures to identify reaction pathways.

The search has several features:

• it favors pathways with few species and steps;

• it ensures the unique generation of candidate pathways;

• it requires balanced chemical equations; and

• it limits steps to at most two reactants and two products.

These aspects involve highly general constraints that limit the search space before a scientist adds prior knowledge.

1. H2 + MM 2MH➞

2. CO + MM M➞ 2CO

3. MH + M2CO M➞ 2CHOM

Partial reaction pathway discovered by MECHEM.

The Prometheus Environment

Recall that Prometheus supports quantitative process models that relate variables through numeric processes.

Scientists can build models from generic processes that appear in domain-specific libraries.

Knowledge in Prometheus

Generic processes encode the knowledge used by Prometheus to discover quantitative process models.

generic process exponential_loss variables: S{species}, D{detritus} parameters: [0, 1] equations: d[S,t,1] = 1 S

d[D,t,1] = S

generic process grazing variables: S1{species}, S2{species}, D{detritus} parameters: [0, 1], [0, 1] equations: d[S1,t,1] = S1

d[D,t,1] = (1 ) S1d[S2,t,1] = 1 S1

generic process nutrient_uptake variables: S{species}, N{nutrient} parameters: [0, ], [0, 1], [0, 1] conditions: N > equations: d[S,t,1] = S

d[N,t,1] = 1 S

Generic processes specify

• which variables may interact,

• the equations that govern the dynamics of their interaction,

• ranges on the parameters associated with the processes.

Scientists also give Prometheus a list of variables that should appear in the model.

Search in Prometheus

Unlike Gertl and MECHEM, Prometheus carries out multi-stage search for a model’s structure and its parameters:

1. Find all ways to instantiate known generic processes with specific variables, subject to type constraints;

2. Combine instantiated processes into candidate generic models subject to additional constraints;

3. For each generic model, carry out search through parameter space to find good coefficients;

4. Return the parameterized models with the best overall scores (e.g., sum of squared error).

Like MECHEM, Prometheus returns multiple models for the scientist to inspect and possibly refine.

To this end, the environment supports incremental revision.

Discovering Dynamic Models: Summary

The informatics systems discussed in this lecture covered three types of dynamic models.

Nevertheless there were commonalities as each system

Moreover, researchers have applied these systems to generate new knowledge in scientific domains.

• used domain knowledge such as a model structure, constraints on candidates, or model components;

• used search techniques to discover model parameters, structures, or both;

• worked interactively with scientists to explain their data.