learning algorithms for life scientists

Artificial Intelligence and Learning Algorithms

Presented By Brian M. Frezza 12/1/05

Game Plan

• What’s a Learning Algorithm?• Why should I care?

– Biological parallels

• Real World Examples• Getting our hands dirty with the algorithms

– Bayesian Networks– Hidden Markov Models– Genetic Algorithms– Neural Networks

• Artificial Neural Networks Vs Neuron Biology– “Fraser’s Rules”

• Frontiers in AI

HardMath

What’s a Learning Algorithm?

• “An algorithm which predicts data’s future behavior based on its past performance.”– Programmer can be ignorant of the data’s

trends.• Not rationally designed!

– Training Data– Test Data

Why do I care?

• Use In Informatics– Predict trends in “fuzzy” data

• Subtle patterns in data• Complex patterns in data• Noisy data

– Network inference– Classification inference

• Analogies To Chemical Biology – Evolution– Immunological Response– Neurology

• Fundamental Theories of Intelligence– That’s heavy dude

Street Smarts

• CMU’s Navlab-5 (No Hands Across America)– 1995 Neural Network Driven Car– Pittsburgh to San Diego: 2,797 miles (98.2%)– Single hidden layer backpropagation network!

• Subcellular location through fluorescence– “A Neural network classifier capable of recognizing the patterns of all major subcellular

structures in fluorescence microscope images of HeLa cells” M. V. Boland, and R. F. Murphy, Bioinformatics (2001) 17(12), 1213-1223

• Protein secondary structure prediction• Intron/Exon predictions• Protein/Gene network inference• Speech recognition• Face recognition

The Algorithms

• Bayesian Networks• Hidden Markov Models• Genetic Algorithms

• Neural Networks

Bayesian Networks: Basics

• Requires models of how data behaves– Set of Hypothesis: {H}

• Keeps track of likelihood of each model being accurate as data becomes available– P(H)

• Predicts as a weighted average – P(E) = Sum( P(H)*H(E) )

Bayesian Network Example

• What color hair will Paul Schaffer’s

kids have if he marries Redhead? – Hypothesis

• Ha(rr) rr x rr: 100% Redhead

• Hb(Rr) rr x Rr: 50% Redhead 50% Not

• Hc(RR) rr x RR: 100% Not

• Initially clueless:– So P(Ha) = P(Hb) = P(Hc) = 1/3

Bayesian Network: Trace

Ha: 100% RedheadHb: 50% Redhead 50% Not

Hc: 100% Not

Redhead 0Not 0

HypothesisHistory

Likelihood's

1/31/31/3

P(Hc)P(Hb)P(Ha)

= P(red|Ha)*P(Ha) + P(red|Hb)*P(Hb) + P(red|Hc)*P(Hc)= (1)*(1/3) + (1/2)*(1/3) + (0)(1/3)

=(1/2)

Prediction: Will their next kid be a Redhead?

Bayesian Network:Trace


Hc: 100% Not

Redhead 1Not 0

HypothesisHistory

Likelihood's

01/21/2

P(Hc)P(Hb)P(Ha)


=(3/4)




Hc: 100% Not

Redhead 2Not 0

HypothesisHistory

Likelihood's

01/43/4

P(Hc)P(Hb)P(Ha)


=(7/8)




Hc: 100% Not

Redhead 3Not 0

HypothesisHistory

Likelihood's

01/87/8

P(Hc)P(Hb)P(Ha)


=(15/16)


Bayesian Networks Notes

• Never reject hypothesis unless directly disproved

• Learns based on rational models of behavior– Models can be extracted!

• Programmer needs to form hypothesis beforehand.

The Algorithms


• Neural Networks

Hidden Markov Models(HMM)

• Discrete learning algorithm– Programmer must be able to categorize predictions

• HMMs also assume a model of the world working behind the data

• Models are also extractable• Common Uses

– Speech Recognition– Secondary structure prediction– Intron/Exon predictions– Categorization of data

Hidden Markov Models: Take a Step Back

• 1st order Markov Models:– Q{States}– Pr{Transition}– Sum of all P(T) out of state = 1

Q1

Q4

Q2

Q3

P1

P2

1-P1-P2

P3

1-P3

1

1-P4

P4

1st order Markov Model Setup

• Pick Initial state: Q1

• Pick Transition Probabilities:

• For each time step– Pick a random number 0.0-1.0

Q1

Q4

Q2

Q3

P1

P2

1-P1-P2

P3

1-P3

1

1-P4

P4

P1 P2 P3 P4

0.6 0.2 0.9 0.4

1st order Markov Model Trace

• Current State: Q1 Time Step = 1• Transition probabilities:

• Random Number:– 0.22341

• So Next State:– 0.22341 < P1

• Take P1

– Q2

Q1

Q4

Q2

Q3

P1

P2

1-P1-P2

P3

1-P3

1

1-P4

P4

P1 P2 P3 P4

0.6 0.2 0.9 0.4


• Current State: Q2 Time Step = 2

• Transition probabilities:


• So Next State:– No Choice, P = 1– Q3

Q1

Q4

Q2

Q3

P1

P2

1-P1-P2

P3

1-P3

1

1-P4

P4

P1 P2 P3 P4

0.6 0.2 0.9 0.4


• Current State: Q3 Time Step = 3• Transition probabilities:


• So Next State:– 0.97412 > 0.9

• Take 1-P3

– Q4

Q1

Q4

Q2

Q3

P1

P2

1-P1-P2

P3

1-P3

1

1-P4

P4

P1 P2 P3 P4

0.6 0.2 0.9 0.4


• Current State: Q4 Time Step = 4

• Transition probabilities:

• I’m going to stop here.

• Markov Chain:– Q1, Q2, Q3, Q4

Q1

Q4

Q2

Q3

P1

P2

1-P1-P2

P3

1-P3

1

1-P4

P4

P1 P2 P3 P4

0.6 0.2 0.9 0.4

What else can Markov do?

• Higher Order Models– Kth order

• Metropolis-Hastings– Determining thermodynamic equilibrium

• Continuous Markov Models– Time step varies according to continuous

distribution

• Hidden Markov Models– Discrete model learning

Hidden Markov Models (HMMs)

• A Markov Model drives the world but it is hidden from direct observation and its status must be inferred from a set of observables. – Voice recognition

• Observable: Sound waves• Hidden states: Words

– Intron/Exon prediction• Observable: nucleotide sequence• Hidden State: Exon, Intron, Non-coding

– Secondary structure prediction for protein• Observable: Amino acid sequence• Hidden State: Alpha helix, Beta Sheet, Unstructured

Hidden Markov Models: Example• Secondary Structure Prediction

ObservableStates

HiddenStates

UnstructuredAlphaHelix

BetaSheet

His Asp Arg Phe Ala Cis Ser Gln Glu Lys

Leu Met Asn Ser Tyr Thr Ile Trp Pro ValGly

Hidden Markov Models: Smaller Example

• Exon/Intron Mapping

0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx

Hidden State Transition Probabilities

0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA

Observable State Probabilities

To

Fro

m

Hid

den

Sta

te

Observable

0.010.890.1

ItIgEx

Starting Distribution

Hidden Markov Model

• How to predict outcomes from a HMM

• Brute force:

– Try every possible Markov chain• Which chain has greatest probability of

generating observed data?

– Viterbi algorithm• Dynamic programming approach

Viterbi Algorithm: Trace

0.80.020.18It

0.010.90.09Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx


Example Sequence: ATAATGGCGAGTG

Exon = P(A|Ex) * Start Exon = 3.3*10-2

Introgenic = P(A|Ig) * Start Ig = 2.2*10-1

Intron = P(A|It) * Start It = 0.14 * 0.01 = 1.4*10-3

G

T

A

G

A

G

C

G

G

T

A

A

T

1.4*10-32.2*10-13.3*10-2A

IntronIntrogenicExon


0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx



Exon = Max( P(Ex|Ex)*Pn-1(Ex), P(Ex|Ig)*Pn-1(Ig), P(Ex|It)*Pn-1(It) ) *P(T|Ex) = 4.6*10-2

Introgenic =Max( P(Ig|Ex)*Pn-1(Ex), P(Ig|Ig)*Pn-1(Ig), P(Ig|It)*Pn-1(It) ) * P(T|Ig) = 2.8*10-2

Intron = Max( P(It|Ex)*Pn-1(Ex), P(It|Ig)*Pn-1(Ig), P(It,It)*Pn-1(It) ) * P(T|It) = 1.1*10-3

G

T

A

G

A

G

C

G

G

T

A

A

1.1*10-32.8*10-24.6*10-2T

1.4*10-32.2*10-13.3*10-2A



0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx



Exon = Max( P(Ex|Ex)*Pn-1(Ex), P(Ex|Ig)*Pn-1(Ig), P(Ex|It)*Pn-1(It) ) *P(T|Ex) = 1.1*10-2

Introgenic =Max( P(Ig|Ex)*Pn-1(Ex), P(Ig|Ig)*Pn-1(Ig), P(Ig|It)*Pn-1(It) ) * P(T|Ig) = 3.5*10-3

Intron = Max( P(It|Ex)*Pn-1(Ex), P(It|Ig)*Pn-1(Ig), P(It,It)*Pn-1(It) ) * P(T|It) = 1.3*10-3

G

T

A

G

A

G

C

G

G

T

A

1.3*10-33.5*10-31.1*10-2A

1.1*10-32.8*10-24.6*10-2T

1.4*10-32.2*10-13.3*10-2A



0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx



Exon = Max( P(Ex|Ex)*Pn-1(Ex), P(Ex|Ig)*Pn-1(Ig), P(Ex|It)*Pn-1(It) ) *P(T|Ex)Introgenic =Max( P(Ig|Ex)*Pn-1(Ex), P(Ig|Ig)*Pn-1(Ig), P(Ig|It)*Pn-1(It) ) * P(T|Ig)Intron = Max( P(It|Ex)*Pn-1(Ex), P(It|Ig)*Pn-1(Ig), P(It,It)*Pn-1(It) ) * P(T|It)

G

T

A

G

A

G

C

G

G

T

2.9*10-44.3*10-42.4*10-3A

1.3*10-33.5*10-31.1*10-2A

1.1*10-32.8*10-24.6*10-2T

1.4*10-32.2*10-13.3*10-2A



0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx




G

T

A

G

A

G

C

G

G

7.8*10-56.1*10-57.2*10-4T

2.9*10-44.3*10-42.4*10-3A

1.3*10-33.5*10-31.1*10-2A

1.1*10-32.8*10-24.6*10-2T

1.4*10-32.2*10-13.3*10-2A



0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx




G

T

A

G

A

G

C

G

7.2*10-51.8*10-55.5*10-5G

7.8*10-56.1*10-57.2*10-4T

2.9*10-44.3*10-42.4*10-3A

1.3*10-33.5*10-31.1*10-2A

1.1*10-32.8*10-24.6*10-2T

1.4*10-32.2*10-13.3*10-2A



0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx




G

T

A

G

A

G

C

2.9*10-52.2*10-64.3*10-6G

7.2*10-51.8*10-55.5*10-5G

7.8*10-56.1*10-57.2*10-4T

2.9*10-44.3*10-42.4*10-3A

1.3*10-33.5*10-31.1*10-2A

1.1*10-32.8*10-24.6*10-2T

1.4*10-32.2*10-13.3*10-2A



0.80.020.18It

0.010.50.49Ig

0.20.10.7Ex

ItIgEx


0.20.50.160.14It

0.250.250.250.25Ig

0.140.110.420.33Ex

CGTA


To

Fro

mH

idd

en S

tate

Observable

0.010.890.1

ItIgEx




4.7*10-103.6*10-111.1*10-10G

1.2*10-91.2*10-101.4*10-9T

9.2*10-94.1*10-104.9*-9A

8.2*10-82.7*10-98.4*-9G

2.0*10-79.1*10-91.1*10-7A

1.8*10-63.5*10-89.1*10-8G

4.6*10-62.8*10-77.2*10-7C

2.9*10-52.2*10-64.3*10-6G

7.2*10-51.8*10-55.5*10-5G

7.8*10-56.1*10-57.2*10-4T

2.9*10-44.3*10-42.4*10-3A

1.3*10-33.5*10-31.1*10-2A

1.1*10-32.8*10-24.6*10-2T

1.4*10-32.2*10-13.3*10-2A


Hidden Markov Models

• How to Train an HMM– The forward-backward algorithm

• Ugly probability theory math:

• Starts with an initial guess of parameters• Refines parameters by attempting to reduce the

errors it provokes with fitted to the data.– Normalized probability of the “Forward probability” of

arriving at the state given the observable cross multiplied by the backward probability of generating that observable given the parameter.

CENSORED

The Algorithms


• Neural Networks

Genetic Algorithms

• Individuals are series of bits which represent candidate solutions– Functions– Structures– Images– Code

• Based on Darwin evolution– individuals mate, mutate, and are selected

based on a Fitness Function

Genetic Algorithms

• Encoding Rules– “Gray” bit encoding

• Bit distance proportional to value distance

• Selection Rules– Digital / Analog Threshold

– Linear Amplification Vs Weighted Amplification

• Mating Rules– Mutation parameters– Recombination parameters

Genetic Algorithms

• When are they useful?– Movements in sequence space are funnel shaped

with fitness function• Systems where evolution actually applies!

• Examples– Medicinal chemistry– Protein folding– Amino acid substitutions– Membrane trafficking modeling– Ecological simulations – Linear Programming– Traveling salesman

The Algorithms


• Neural Networks

Neural Networks

• 1943 McCulloch and Pitts Model of how Neurons process information– Field immediately splits

• Studying brain’s– Neurology

• Studying artificial intelligence– Neural Networks

Neural Networks: A Neuron, Node, or Unit

Σ(W)- W0,c

Activation Function Output

Wa,c

Wb,c

W0,c

(Bias)

Wc,n

a z (Bias)

Neural Networks: Activation Functions

Sigmoid Function(logistic function)

Threshold Function

Zero point set by bias

In In

out out+1 +1

Threshold Functions can makeLogic Gates with Neurons!

000

011

01∩

Logical AndW0,c = 1.5

Wb,c = 1

Wa,c = 1

A

B

Σ(W)- W0,ca z (Bias)

Output

If ( Σ(w) – Wo,c > 0 )

Then FIRE

Else

Don’t

(Bias)

And Gate: Trace

W0,c = 1.5

Wb,c = 1

Wa,c = 1

-1.5

Off

Off

Off-1.5 < 0(Bias)

And Gate: Trace

W0,c = 1.5

Wb,c = 1

Wa,c = 1

-0.5

On

Off

Off-0.5 < 0(Bias)

And Gate: Trace

W0,c = 1.5

Wb,c = 1

Wa,c = 1

-0.5

Off

On

Off-0.5 < 0(Bias)

And Gate: Trace

W0,c = 1.5

Wb,c = 1

Wa,c = 1

0.5

On

On

On0.5 > 0(Bias)


W0,c = 0.5

Wb,c = 1

Wa,c = 1

A


If ( Σ(w) – Wo,c > 0 )

Then FIRE

Else

Don’t

(Bias)

010

111

01U

Logical OrB

Or Gate: Trace

W0,c = 0.5

Wb,c = 1

Wa,c = 1

-0.5

Off

Off

Off-0.5 < 0(Bias)

Or Gate: Trace

W0,c = 0.5

Wb,c = 1

Wa,c = 1

0.5

On

Off

On0.5 > 0(Bias)

Or Gate: Trace

W0,c = 0.5

Wb,c = 1

Wa,c = 1

0.5

Off

On

On0.5 > 0(Bias)

Or Gate: Trace

W0,c = 0.5

Wb,c = 1

Wa,c = 1

1.5

On

On

On1.5 > 0(Bias)


W0,c = -0.5

Wa,c = -1


If ( Σ(w) – Wo,c > 0 )

Then FIRE

Else

Don’t

(Bias)Logical Not

10

01!

Not Gate: Trace

W0,c = -0.5

Wa,c = -1

-0.5

Off

On0.5 > 0(Bias)

0 – (-0.5) = 0.5

Not Gate: Trace

W0,c = -0.5

Wa,c = -1

-0.5

On

Off-0.5 < 0(Bias)

-1 – (-0.5) = -0.5

Feed-Forward Vs. Recurrent Networks

• Feed-Forward– No Cyclic connections– Function of its current

inputs– No internal state other

then weights of connections

• “Out of time”

• Recurrent– Cyclic connections– Dynamic behavior

• Stable• Oscillatory• Chaotic

– Response depends on current state

• “In time”

– Short term memory!

Feed-Forward Networks

• “Knowledge” is represented by weight on edges– Modeless!

• “Learning” consists of adjusting weights• Customary Arrangements

– One Boolean output for each value– Arranged in Layers

• Layer 1 = inputs

• Layer 2 to (n-1) = Hidden

• Layer N = outputs– “Perceptron” 2 layer Feed-Forward network

Layers

Input Output

Hidden layer

Perceptron Learning

• Gradient Decent used to reduce error

• Essentially: – New Weight = Old Weight + adjustment– Adjustment = α X error X input X d(activation function)

• α = Learning Rate

CENSORED

Hidden Network Learning

• Back-Propagation

• Essentially: – Start with Gradient Decent from output– Assign “blame” to inputting neurons proportional to

their weights– Adjust weights at previous level using Gradient

decent based on “blame”

CENSORED

They don’t get it either:Issues that aren’t well understood

• α (Learning Rate)

• Depth of network (number of layers)

• Size of hidden layers– Overfitting

– Cross-validation

• Minimum connectivity– Optimal Brain Damage Algorithm

• No extractable model!

How Are Neural Nets Different From My Brain?

1. Neural nets are feed forward– Brains can be recurrent with feedback loops

2. Neural nets do not distinguish between + or – connections

– In brains excitatory and inhibitory neurons have different properties• Inhibitory neurons short-distance

3. Neural nets exist “Out of time”– Our brains clearly do exist “in time”

4. Neural nets learn VERY differently– We have very little idea how our brains are learning

“Fraser’s” Rules

“In theory one can, of course, implement biologically realistic neural networks, but this is a mammoth task. All kinds of details have to be gotten right, or you end up with a

network that completely decays to unconnectedness, or one that ramps up its connections until it basically has a seizure.”

Frontiers in AI

• Applications of current algorithms• New algorithms for determining

parameters from training data– Backward-Forward– Backpropagation

• Better classification of the mysteries of neural networks

• Pathology modeling in neural networks• Evolutionary modeling

learning algorithms for life scientists

Education