Biochemistry and Molecular GeneticsHuman Medical Genetics and Genomics

Computational BioscienceConsortium for Comparative Genomics

University of Colorado School of MedicineDavid.Pollock@UCDenver.edu

www.EvolutionaryGenomics.comwww.ProteinEvolution.org

Multiple Sequence Alignment (MSA); short version

BIOL 7711 Computational Bioscience

University of Colorado School of MedicineConsortium for Comparative Genomics

Sequence Alignment ProfilesMouse TCR Va

Why a Hidden Markov Model?Data elements are often linked by a

string of connectivity, a linear sequence

Secondary structure prediction (Goldman, Thorne, Jones)CpG islands

Models of exons, introns, regulatory regions, genesMutation rates along genome

Why a Hidden Markov Model?

Complications?Insertion and deletion of states (indels)Long-distance interactions

BenefitsFlexible probabilistic framework

E.g., compared to regular expressions

Multiple Sequence Alignment

Generalize pairwise alignment of sequences to include more than twoLooking at more than two sequences

gives us much more information Site-specific information

Sites are different!E.g., which amino acids, coevolutionProcess of change at a site

Evolutionary/phylogenetic relationshipsShorter branches, dissect compound indels

Sample MSA: cFOSFOS_RAT MMFSGFNADYEASSSRCSSASPAGDSLSYYHSPADSFSSMGSPVNTQDFCADLSVSSANF 60FOS_MOUSE MMFSGFNADYEASSSRCSSASPAGDSLSYYHSPADSFSSMGSPVNTQDFCADLSVSSANF 60FOS_CHICK MMYQGFAGEYEAPSSRCSSASPAGDSLTYYPSPADSFSSMGSPVNSQDFCTDLAVSSANF 60FOSB_MOUSE -MFQAFPGDYDS-GSRCSS-SPSAESQ--YLSSVDSFGSPPTAAASQE-CAGLGEMPGSF 54FOSB_HUMAN -MFQAFPGDYDS-GSRCSS-SPSAESQ--YLSSVDSFGSPPTAAASQE-CAGLGEMPGSF 54 *:..* .:*:: .***** **:.:* * *..***.* :.. :*: *:.*. ...*

FOS_RAT IPTVTAISTSPDLQWLVQPTLVSSVAPSQ-------TRAPHPYGLPTPS-TGAYARAGVV 112FOS_MOUSE IPTVTAISTSPDLQWLVQPTLVSSVAPSQ-------TRAPHPYGLPTQS-AGAYARAGMV 112FOS_CHICK VPTVTAISTSPDLQWLVQPTLISSVAPSQ-------NRG-HPYGVPAPAPPAAYSRPAVL 112FOSB_MOUSE VPTVTAITTSQDLQWLVQPTLISSMAQSQGQPLASQPPAVDPYDMPGTS----YSTPGLS 110FOSB_HUMAN VPTVTAITTSQDLQWLVQPTLISSMAQSQGQPLASQPPVVDPYDMPGTS----YSTPGMS 110 :******:** **********:**:* **... ::. .**.:* : *: ..:

FOS_RAT KTMSGGRAQSIG--------------------RRGKVEQLSPEEEEKRRIRRERNKMAAA 152FOS_MOUSE KTVSGGRAQSIG--------------------RRGKVEQLSPEEEEKRRIRRERNKMAAA 152FOS_CHICK KAP-GGRGQSIG--------------------RRGKVEQLSPEEEEKRRIRRERNKMAAA 151FOSB_MOUSE AYSTGGASGSGGPSTSTTTSGPVSARPARARPRRPREETLTPEEEEKRRVRRERNKLAAA 170FOSB_HUMAN GYSSGGASGSGGPSTSGTTSGPGPARPARARPRRPREETLTPEEEEKRRVRRERNKLAAA 170 :** . * *.::: :::.. .: .: : .** : * *:********:******:***

Optimal MSAUse Dynamic Programming?Optimal alignment algorithm exists,

but is _________n is the number of sequencesL is the length of the longest sequence

10 sequences of length 100 take 21010010~1023 operations, around 1 million years at 3GHz

Exponential algorithms strike againSo, approximation approaches?

Progressive MSAStart with pairwise alignments of closely related sequences

Add more distantly related sequences one at a timeComplexity proportional to L2*n

Requires a priori assumptions about the phylogenetic relationships

Can be estimated from all pairwise comparisons

Unfortunate circularity to this approachSATCHMO method tries to estimate both at once

MSA score based on sum of pairwise scores

Can be weighted to reduce influence of similar sequences

Gaps in Progressive MSAsHow to score gaps in MSAs?

Want to align gaps with each other over all sequences. A gap in a pairwise alignment that “matches” a gap in another pairwise alignment should cost less than introducing a totally new gap.

Possible that a new gap could be made to “match” an older one by shifting around the older pairwise alignmentChange gap penalty near conserved domains of

various kinds (e.g. secondary structure, hydrophobic regions)

CLUSTALW2 http://www.ebi.ac.uk/Tools/clustalw2/ is the most widely used Progressive MSA program

Greedy AlgorithmsBest alignment of each new sequence to the existing alignment

Then never revisit the decisionEven if changing an old decision (e.g. moving around the gaps in a previous alignment) could increase the score, this approach doesn't do it.Approach is called “greedy” because it uses the best solution at the current step, then moves on

Have to hope that the best solution to a part of the problem will be good solution for the whole problemThis is a common way to resolve exponential problems

Problems with Progressive MSA

Depends crucially on the quality of the pairwise alignments, particularly among the closest matches (which are aligned first)

Small errors propagate to whole alignmentThere is no suitable resolution to the problem of gap penalties over multiple sequencesWorks reasonably well for closely related sequences.

Even then, manual adjustments are common

Iterative MSA MethodsStart with a reasonable

approximation to the optimal MSAe.g. by using a progressive methodThen “tweak” to improve itCommon CS idea, called “optimization”

Various optimization techniques tried

e.g., GAs and simulated annealing Key is the scoring function for the

whole MSAAlso, what steps (tweaks) to take that

are likely to improve the score

Block Based MethodsStart with short local alignments

(blocks) Then reduce the problem to aligning the

regions between the blocks“Divide and conquer”

Another common CS approach to exponential problems

How to find the blocks? DALIGN (local alignment methods)DCA (divide and conquer alignments)Tmsa (identify patterns and use them to

define blocks)

MSA DatabasesE.g., Pfam, Tigergam, etc.

Look them up and surf

Protein motifsRecall that local alignments can identify similar regions in non-homologous proteinsThese regions (sometimes called domains) often have shared structure and/or functionExample: Zinc-finger DNA binding motifHow to define them?

Consensus sequenceRegular expressionProfile (probability for each amino acid at each position)

ProSite consensus sequences

Recognizing ProSite patternsL14 Ribosome pattern: [GA]-[LIV](3)-x(9,10)-[DNS]-G-x(4)-[FY]-x(2)-[NT]-x(2)-V-[LIV]Some matching sequences:

GIIIACGHLIPQTNGACRTYILNDRVVGVLLWQPKHCSNAADGAWAWFAATAAVLALIVEANIIILSISGRATTFHATSAVI

ProSite patterns can be translated into regular expressions, although the bounded length patterns (e.g. [LIV](3,5) are unwieldy to write down as regexps

ProfilesRather than identifying only the

“consensus” (i.e. most common) amino acid at a particular location, we can assign a probability to each amino acid in each position of the domain

1 2 3A 0.1 0.5 0.25C 0.3 0.1 0.25D 0.2 0.2 0.25E 0.4 0.2 0.25

Applying a profileCalculate score (probability of

match) for a profile at each position in a sequence by multiplying individual probabilities.

Uses a “sliding window”

To calculate a significance value, normalize by the probability of match to random sequence

For sequence EACDC:EAC = .4 * .5 * .25 = .05ACD = .1 * .1 * .25 = .0025CDC = .3 * .2 * .25 = .015

1 2 3A 0.1 0.5 0.25C 0.3 0.1 0.25D 0.2 0.2 0.25E 0.4 0.2 0.25

Using motifsGreat for annotating a sequence

with no strong homologsINTERPRO is an uniform interface to

many different motif methods and databases

ProSitePrints (fingerprints = multiple motifs)ProDom (like Pfam, but for domains)SMART (mobile domains)

Creating profilesGiven a local MSA, creating a profile is straightforward

Calculate frequency of each amino acid at each position

What about zero frequencies?Could be sampling errors, not real zero probabilitiesZero probabilities always make zero scores!

RegularizationPseudocountsDirichlet mixtures (blend in background frequencies)

Better regularizersAdding one pseudocount is too large

and too uniform in the small MSA caseInstead, can add a fraction

proportional to the overall frequency of occurrence of the amino acidMight want to add different

pseudocounts depending on the actual count (add more to smaller counts, especially 0)Can use substitution matrices to

estimate

Feature alphabetsAmino acids can be grouped by their

characteristics:Size, hydrophobicity, ionizability, etc.An amino acid is generally in more than

one groupCan set different regularizers

(pseudocounts) for each different featureMost useful when there are multiple

featuresotherwise many amino acids get same

pseudocount

Dirichlet mixture priorsFanciest (and near optimal)

regularizerAllows several dimensions (like a

feature, but not predefined), each of which has a different weight for each amino acid.Each pseudocount depends on the

prior probability of seeing a particular distribution in a position

Add more “prior” to more unusual observations

Pseudocount falls off with more observations

A Simpler not very Hidden MM

Nucleotides, no Indels, Unambiguous Path

G .1C .3A .2T .4

G .1C .1A .7T .1

G .3C .3A .1T .3

A0.7

T0.4

T0.3

1.0 1.0 1.0

A Toy not-Hidden MMNucleotides, no Indels, Unambiguous

PathAll arcs out are equal

Example sequences: GATC ATC GC GAGAGC AGATTTC

BeginEmit G

Emit A

Emit C

Emit TEnd

Arc Emission

A Simple HMMCpG Islands; States are Really Hidden

Now

G .1C .1A .4T .4

G .3C .3A .2T .2 0.1

0.2

CpG Non-CpG

0.8 0.9

The Forward AlgorithmProbability of a Sequence is the

Sum of All Paths that Can Produce It

G .1C .1A .4T .4

G .3C .3A .2T .2

0.10.2

Non-CpG

0.8

0.9 G

CpG

G .3

G .1

.3*(

.3*.8+

.1*.1)=.075.1*(.3*.2+.1*.9)=.015C

The Forward AlgorithmProbability of a Sequence is the

Sum of All Paths that Can Produce It

G .1C .1A .4T .4

G .3C .3A .2T .2

0.10.2

Non-CpG

0.8

0.9 G

CpG

G .3

G .1

.3*(

.3*.8+

.1*.1)=.075.1*(.3*.2+.1*.9)=.015C

.3*(

.075*.8+

.015*.1)=.0185

.1*(

.075*.2+

.015*.9)=.0029

G

The Forward AlgorithmProbability of a Sequence is the

Sum of All Paths that Can Produce It

G .1C .1A .4T .4

G .3C .3A .2T .2

0.10.2

Non-CpG

0.8

0.9 G

CpG

G .3

G .1

.3*(

.3*.8+

.1*.1)=.075.1*(.3*.2+.1*.9)=.015C

.3*(

.075*.8+

.015*.1)=.0185

.1*(

.075*.2+

.015*.9)=.0029

G

.2*(

.0185*.8+.0029*.1)=.003.4*(.0185*.2+.0029*.9)=.0025A

.2*(

.003*.8+

.0025*.1)=.0005.4*(.003*.2+.0025*.9)=.0011

A

The Forward AlgorithmProbability of a Sequence is the

Sum of All Paths that Can Produce It

G .1C .1A .4T .4

G .3C .3A .2T .2

0.10.2

Non-CpG

0.8

0.9 G

CpG

G .3

G .1

.3*(

.3*.8+

.1*.1)=.075.1*(.3*.2+.1*.9)=.015C

.3*(

.075*.8+

.015*.1)=.0185

.1*(

.075*.2+

.015*.9)=.0029

G

.2*(

.0185*.8+.0029*.1)=.003.4*(.0185*.2+.0029*.9)=.0025A

.2*(

.003*.8+

.0025*.1)=.0005.4*(.003*.2+.0025*.9)=.0011

A

The Viterbi AlgorithmMost Likely Path

G .1C .1A .4T .4

G .3C .3A .2T .2

0.10.2

Non-CpG

0.8

0.9 G

CpG

G .3

G .1

.3*m(

.3*.8,

.1*.1)=.072.1*m(.3*.2,.1*.9)=.009C

.3*m(

.075*.8,

.015*.1)=.0173

.1*m(

.075*.2,

.015*.9)=.0014

G

.2*m(

.0185*.8,.0029*.1)=.0028.4*m(.0185*.2,.0029*.9)=.0014A

.2*m(

.003*.8,

.0025*.1)=.00044.4*m(.003*.2,.0025*.9)=.0050

A

Forwards and BackwardsProbability of a State at a Position

G .1C .1A .4T .4

G .3C .3A .2T .2

0.10.2

Non-CpG

0.8

0.9 G

CpG

C G

.2*(

.0185*.8+.0029*.1)=.003.4*(.0185*.2+.0029*.9)=.0025

A

.2*(

.003*.8+

.0025*.1)=.0005.4*(.003*.2+.0025*.9)=.0011

A

.003*(

.2*.8+

.4*.2)=.0007.0025*(.2*.1+.4*.9)=.0009

Forwards and BackwardsProbability of a State at a Position

G C G A A

.003*(

.2*.8+

.4*.2)=.0007.0025*(.2*.1+.4*.9)=.0009

Homology HMMGene recognition, identify distant

homologs

Common Ancestral SequenceMatch, site-specific emission probabilitiesInsertion (relative to ancestor), global emission probsDelete, emit nothingGlobal transition probabilities

Homology HMM

start

insert insert

match

delete delete

match end

insert

Homology HMMUses

Score sequences for match to HMMCompare alternative modelsAlignmentStructural alignment

Multiple Sequence Alignment HMM

Defines predicted homology of positions (sites)

Recognize region within longer sequenceModel domains or whole proteins

Can modify model for sub-familiesIdeally, use phylogenetic tree

Often not much back and forthIndels a problem

Model ComparisonBased on

For ML, takeUsually to avoid

numeric errorFor heuristics, “score” isFor Bayesian, calculate

Parameters, Types of parameters

Amino acid distributions for positionsGlobal AA distributions for insert statesOrder of match statesTransition probabilitiesTree topology and branch lengthsHidden states (integrate or augment)

Wander parameter space (search)Maximize, or move according to

posterior probability (Bayes)

Expectation Maximization (EM)

Classic algorithm to fit probabilistic model parameters with unobservable statesTwo Stages

MaximizeIf know hidden variables (states), maximize

model parameters with respect to that knowledge

ExpectationIf know model parameters, find expected

values of the hidden variables (states)Works well even with e.g., Bayesian

to find near-equilibrium space

Homology HMM EMStart with heuristic (e.g., ClustalW)Maximize

Match states are residues aligned in most sequencesAmino acid frequencies observed in

columnsExpectation

Realign all the sequences given modelRepeat until convergenceProblems: Local, not global

optimizationUse procedures to check how it worked

Model ComparisonDetermining significance depends

on comparing two modelsUsually null model, H0, and test model,

H1

Models are nested if H0 is a subset of H1

If not nestedAkaike Iinformation Criterion (AIC) [similar

to empirical Bayes] or Bayes Factor (BF) [but be careful]

Generating a null distribution of statistic

Z-factor, bootstrapping, , parametric bootstrapping, posterior predictive

Z Test MethodDatabase of known negative controls

E.g., non-homologous (NH) sequencesAssume NH scores

i.e., you are modeling known NH sequence scores as a normal distribution

Set appropriate significance level for multiple comparisons (more below)

ProblemsIs homology certain?Is it the appropriate null model?

Normal distribution often not a good approximation

Parameter control hard: e.g., length distribution

Bootstrapping and Parametric Models

Random sequence sampled from the same set of emission probability distributions

Same length is easyBootstrapping is re-sampling columnsParametric uses estimated frequencies, may include variance, tree, etc.

More flexible, can have more complex nullPseudocounts of global frequencies if data limit

Insertions relatively hard to modelWhat frequencies for insert states? Global?

Homology HMM ResourcesUCSC (Haussler)

SAM: align, secondary structure predictions, HMM parameters, etc.

WUSTL/Janelia (Eddy)Pfam: database of pre-computed HMM

alignments for various proteinsHMMer: program for building HMMs