statistical analysis of rna-seq data - tom van dooren · aims of the lecture quantitative analysis...
TRANSCRIPT
![Page 1: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/1.jpg)
Statistical analysis of RNA-seq data
Marie-Laure Martin-Magniette
IPS2 Institut des Sciences des Plantes de Paris-Saclay
UMR AgroParisTech/INRA Mathematique et Informatique Appliquees
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 1 / 55
![Page 2: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/2.jpg)
Table of contents
1 Introduction
2 Normalization
3 Differential analysis
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 2 / 55
![Page 3: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/3.jpg)
Aims of the lecture
Quantitative analysis of gene expressionOverview of the different steps of the analysisIt is not exhaustive
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 3 / 55
![Page 4: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/4.jpg)
Design of a transcriptomic project
Biological question↓ ↑
Experimental design
choice of the technology and type of analysis↓
Data acquisition↓
Data analysis
normalization, differential analysis, clustering, ...↓ ↑
Validation
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 4 / 55
![Page 5: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/5.jpg)
HTS data characteristics
Some statistical challenges of HTS dataDiscrete, non-negative, and skewed data with very large dynamicrange (up to 5+ orders of magnitude)Sequencing depth (= “library size”) varies among experimentsTotal number of reads for a gene ∝ expression level × length
Gene 1 Gene 2
Gene 1 Gene 2
Sample 1
Sample 2
To date, most methodological developments are for experimentaldesign, normalization, and differential analysis...
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 5 / 55
![Page 6: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/6.jpg)
Table of contents
1 Introduction
2 NormalizationDifferent types of normalizationComparaison of 7 normalization methodsConclusions on normalization
3 Differential analysis
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 6 / 55
![Page 7: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/7.jpg)
Normalization
DefinitionNormalization is a process designed to identify and correcttechnical biases.Two types of biascontrolable biases: the construction of cDNA librariesuncontrolable biases: sequencing process
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 7 / 55
![Page 8: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/8.jpg)
Two types of normalization
Within-sample normalizationEnabling comparisons of genes from a same sampleNot required for a differential analysisNot really relevant for the data interpretationSources of variability: gene length and sequence composition(GC content)
Between-sample normalizationEnabling comparisons of genes from different samplesSources of variability: library size, presence of majority fragments,sequence composition due to PCR-amplification step in librarypreparation‘(Pickrell et al. 2010, Risso et al. 2011)
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 8 / 55
![Page 9: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/9.jpg)
Between-sample normalization: the scaling factor
DefinitionFor sample j , let Ygj be the raw count for gene g.The normalized count is defined by:
Ygj
sj,
where sj is the scaling factor for the sample j .
Three types of methods:Distribution adjustmentMethod taking length into accountThe Effective Library Size concept
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 9 / 55
![Page 10: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/10.jpg)
Distribution adjustment
Let n be the number of samples in the project
Total read count normalization (Marioni et al. 2008)
sj =Nj
1n∑n
`=1 N`
, where Nj =∑
g
Ygj
Median
sj =mediangYgj
1n∑n
`=1 mediangYg`
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 10 / 55
![Page 11: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/11.jpg)
Method taking length into account
RPKM: Reads Per Kilobase per Million mapped readsMotivation greater lane sequencing depth and transcript length=> greater counts whatever the expression levelAssumption read counts are proportional to expression level,transcript length and sequencing depth (same RNAs in equalproportion)Method divide gene read count by total number of reads (inmillion) and transcript length (in kilobase)
Ygj
Nj ∗ Lg∗ 103 ∗ 106 (1)
RPKM method is an adjustment for library size and transcriptlengthAllows to compare expression levels between genes of the samesampleUnbiased estimation of number of reads but affect the variability.(Oshlack et al. 2009)
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 11 / 55
![Page 12: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/12.jpg)
Method based on the Effective Library Size
Relative Log Expression (RLE)compute a pseudo-reference sample: geometric mean acrosssamples (less sensitive to extreme value than standard mean)
(n∏
`=1
Yg`)1/n
calculate normalization factor
s̃j = mediangYgj
(∏n
`=1 Yg`)1/n
normalize them such that their product equals 1
sj =s̃j
exp[1n∑
` log s̃`]
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 12 / 55
![Page 13: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/13.jpg)
Method based on the Effective Library Size
Trimmed Mean of M-values (TMM)Assumption: the majority of the genes are not differentially expressed
●●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●● ●●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●●
●●
●
●
●
●
●
●
●● ●
●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
● ●●
● ●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
● ● ●●●
● ●●●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●●
● ●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●●
●●●●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●●
●
●●●
●
●
●●
●●
●
●
● ● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●
●
●
●● ●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
● ●
● ●
●
●● ●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
● ●●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●● ●
● ●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●●
● ●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●● ●
●● ●●
●
●
● ●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●●
●●
●
●
●
●
●
●●
●
●●
●
● ●● ●
●
●●● ●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●
●●
● ●
●
●
●● ●
●
●
●
●
●●
●
●
●
●●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●● ●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
● ●●
●
●
●
●●
●
●
●●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●●●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●●
●
●
●●
●●
●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●● ● ●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
● ●
●
●
●
●●●●
●●
●
●
● ●
●
●
●●●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●●
●●
●
●
●
●
●●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●
●
●
●
●●
●
● ●
●
●
●●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
● ●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
● ●
● ●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
● ●
●●
●●
●●●
●
●●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●● ●
●
● ●●
● ●
●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
● ●●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●● ●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
● ●●
●
● ●
●
●
●●●
●●
●
●
●●
●
●
●
● ●●
●
●●
●●
●
●
●
●●
●●●
●
● ●
●
●●
●●
●
●
●
●
●
●
●●
●
●
● ●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●● ●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●●
●●
● ●
●
●
●●
●●
●
●
●
●
●
●●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●●
●●●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●
●
●●●
●
●
●
● ●
●
●
●
●
●●
●
●
●●●
● ●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
● ●
●
● ●● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●
●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●
●●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●●
●●●
● ●
●●● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●● ●
●
●●
●
●
●
●
● ●●●
●
●
● ●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●●
●
●
●
●
● ●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
● ●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
● ● ●
● ●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●● ●
●●
●●●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
● ●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
● ●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
●
●
●●
●
●
●● ● ●
●
● ●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
● ●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
● ●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
●●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●● ●●
●
●●
●
●
●
●● ●●●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
● ●● ●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
● ●●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●● ●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●● ●
●
●●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●● ●●
●
●● ●
●
●●
●●
● ●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●●
● ●
●
●
●
● ●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
● ●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
●
● ●●
●
●
●
●●
●●
●
●●●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●● ●●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
● ●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●● ●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
● ●
●
●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●
●●●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
● ●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
● ●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
● ●
●
●
●
●●
●●
●
●
● ●●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
● ●
●
●
●●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ● ●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
● ●●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●●
● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●● ●
●
●●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●●
●
● ● ●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
●
●
● ●
●
●
● ●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
● ●●
●●
●●
●●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ● ●
●
●
●
● ●●
●
●
● ●
●
●
●● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
● ●
●●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●●●● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
● ●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●
●●
●
●
●●
●
● ●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●●
●
●
●●● ●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●
●● ●
●
●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●●
●
● ●
● ●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●●
●
●
● ●● ●
●
●
●
●
●
● ●
●
●●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●●
●
●
●●
●●
●
●●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
● ●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●● ●
● ●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●● ●
●
●
● ●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●● ●
●
●
●●
●
●
●
●●●
● ●
●●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●
●●
●
● ●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●●
●
●
●
●
●
● ●
●
●
●
●
●●
●● ●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●●
● ●●
●
●
●●●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
● ●●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●
●●●●● ●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●
●
●
● ●
●
● ●●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
● ●
●
●
●
●●
●
●●
●
●
●
●
●
●●● ●
●
●
●
●●
●
● ●
●●
● ●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●●
●
●
●
●
●●
●
●●
● ●
●
●●●●
●●●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●● ●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
● ●
●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
● ●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
● ●●
●
●
●
●
●
●●●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
● ●
●
●
●●
●●
●●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●● ●
●●
●
●
●●
●●
●●
●
●●
●
●
●●
● ●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●●●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●●
●
●
● ●
●
●
●
●
●
●
●
●●
●
● ●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●●
●●
●●
●●
●
●
●● ●
●
●
●●●
● ●
●
●●●
●
●●
●●●
●●
●
●●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
●
●
●
●●
●
●
● ●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
● ●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ● ●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●●●
●
●
●●
● ●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●● ●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●●●
●
●
●
●●●
●●
●
●●
●●
●
●●●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
● ●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●
●●
●
●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●
●● ●
●
●●
●
●
●
●●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
● ●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●●
●●
● ●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●● ●
●
●
●
●
●
●●
●
●
●
●●
●
●● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●
● ●
●
●
●
●
●
●
● ●
●●
●●
● ●
●
●
●
●
●
● ●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
● ●●●
● ●●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
● ●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
● ● ●
●
●
●● ●
●
●
●
●● ●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
● ●
●
●●●●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●● ●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●● ●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●●
●
●
●
●● ●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●● ●●
● ●
●
●
●● ●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●● ●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●● ●●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
● ●
●
● ●
● ●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●
● ●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
● ●●
●●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●●
●
●
●●
●
● ●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●●
●● ●
● ●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●●
●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●●
●
● ●
●
●●
●
●
●
●●
●
●●● ●
●
●
●
● ●●
●
●
●
●●
● ●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
● ●
●
●
●●
●
●
● ●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●● ●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●●● ●
●
●
●
●
●●●
●●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●● ●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●●
●
● ●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
● ●
●
●
●●● ●●
●
●
●
●
●
●
●
● ●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
● ●
● ●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●● ●
●●
●
●
●
●
●●
●●●
●●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●●
●
●
● ●
●●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●
●●●
●●
●
●
●●
●
●●
●●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●●●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●●●
●
● ●
●
●●
●
●●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
● ●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ● ●●
●●
●●
●
●
●
● ●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●● ●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
● ●
●
●
●
●●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●
●●●
●
●●●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
● ●●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●● ●
●●
●
●
●●
●
●
●
●●●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
● ● ●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●●●
●
●
●
●
●
●●
●●
● ●
●
●
●
●
●
●●
●●
●
●●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●● ●●
●
●
●
●
●●
●●
●●●
●
●
●
●●
●●●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●●
●
●
●
●
● ●
●
●
● ●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
● ●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
● ●
●
●
●●
●
● ●
●
●●
●
●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●● ●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●● ●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
● ●●
●
●
●
●●
●
●● ●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●● ●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●● ●
●
●
●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ●●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●●
● ●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●
●●
●
●
●● ● ●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
●
●●
● ●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●● ●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●
●●
●
●
● ●●
●
● ●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●
●
●●
●
●
●●●
●
● ● ●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●
●●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
● ●
●
●
●
●
●
●●
● ●
●
●
●
●
●●●
●●
●●●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●● ●
● ●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●
●
● ●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
● ●
●●
●
● ● ●
●
●●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
● ● ●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●● ● ●●●
●
● ●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●● ●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
● ●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
● ●
●● ●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
● ●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●●
●
●●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●●
●●●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
● ●
●
●
●
●●
●
●
●
●
●
●●
●●●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●● ●
●●●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
● ●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●●
●●●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
● ●
●●● ●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●
●●
●●●
●
●
●
● ●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●● ●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●●●
●
●●
●●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●●
●
●●
●
● ●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●● ●
●●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
● ●
●
●● ● ●
●
●●
●●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●● ●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●● ●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
● ●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
● ●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
● ● ●
●
●
●● ●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
● ●●
● ●
●
●●
●
●● ●
●
●●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
●●
●●
●
●
●
●
●●●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●●
●
●
● ●
●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●●
●●
●●
●
●
●
●
●
●
●
●● ●
●
●
● ●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
● ●
●
●
●
● ●
●
● ●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
● ●
●
●●●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●● ●
●
●
● ●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
● ●●
●
● ●
●
●
●
●
●
●
●
●
●
●
● ●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
● ●●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
● ●
●
●●
●
●
● ●
●
●●
●
●
●●
●
●
●● ●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
● ●
●
●●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●
●
●●
●●
●
●●
●
● ●●
●●
●
●●
●
●
●●
●
●●●
● ● ●
●
●
●●
●
●
●
●●●
●
●
●●●
●
●
●●
●
●● ●
●
●●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●●
● ●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●● ●
●
●
●
●
●
−25 −20 −15 −10
−10
−5
05
10
averaged expression
expr
essi
on d
iffer
ence
Filter on genes with nul countsFilter on the resp. 30% and 5%more extreme values of M r
gj andAr
gj
where
M rgj = log2(
Ygj/Nj
Ygr/Nr)
Argj = [log2(
Ygj
Nj) + log2(
Ygr
Nr)]/2
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 13 / 55
![Page 14: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/14.jpg)
TMM normalization
AlgorithmSelect the reference r as the library whose upper quartile isclosest to the mean upper quartile.
Compute weights w rgj = (
Nj−YgjNj Ygj
+Nr−YgrNr Ygr
)
Compute TMM rj =
∑g∈G? w r
gj Mrgj∑
g∈G? w rgj
Defines̃j = 2TMM r
j
Normalize them such that their product equals 1
sj =s̃j
exp1n∑
` s̃`
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 14 / 55
![Page 15: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/15.jpg)
Which normalization method ?
At lot of different normalization methods...Some are part of models for DE, others are ’stand-alone’They do not rely on similar hypothesesBut all of them claim to remove technical bias associated withRNA-seq data
QuestionsWhich one is the best ?Which criteria are relevant for this choice ?
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 15 / 55
![Page 16: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/16.jpg)
Normalized data distributionWhen large diff. in lib. size, TC and RPKM do not improve over the rawcounts.
Example: Mus musculus datasetM.-L. Martin-Magniette Analysis of RNA-Seq data INRA 16 / 55
![Page 17: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/17.jpg)
Lists of differentially expressed (DE) genes
For each dataset(gene x method) binarymatrice:
1: DE gene0: non DE gene
Jaccard distancebetween methodsdendrogramm, Wardlinkage algorithm
Consensus matriceMean of the distancematrices obtained from eachdataset
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 17 / 55
![Page 18: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/18.jpg)
Type-I Error Rate and Power (Simulated data)Inflated FP rate for all the methods except TMM and DESeq
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 18 / 55
![Page 19: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/19.jpg)
So the Winner is ... ?
In most casesThe methods yield similar results
However ...Differences appear based on data characteristics
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 19 / 55
![Page 20: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/20.jpg)
Conclusions on normalization
RNA-seq data are affected by technical biaises (total number ofmapped reads per lane, gene length, composition bias)Csq1 : non-uniformity of the distribution of reads along thegenomeCsq2 : technical variability within and between-sampleNormalization by gene length isn’t required for the differentialanalysis.Normalisation is necessary and not trivial.
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 20 / 55
![Page 21: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/21.jpg)
Conclusions on normalization
Differences between normalisation methods when genes withlarge number of reads and very different library depths
TMM and DESeq : performant and robust methods in a differentialanalysi context
Risso et al (2014) proposed a new method (RUVSeq). It is a factoranalysis based on a suitably-chosen subset of negative controlgenes
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 21 / 55
![Page 22: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/22.jpg)
Table of contents
1 Introduction
2 Normalization
3 Differential analysisIntroductionTest for RNA-seq data
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 22 / 55
![Page 23: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/23.jpg)
Key steps for a test procedure
The aim is to identify a significant difference between two meansRequired data are measurements in the two conditionsIt is performed with an hypothesis test
H0={There is no difference}versus
H1={There is a difference}
Construct a test statistic (a function of the observations)Decide the most relevant hypothesis with respect to the teststatistic
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 23 / 55
![Page 24: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/24.jpg)
Statistical hypothesis test
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 24 / 55
![Page 25: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/25.jpg)
Statistical hypothesis test
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 25 / 55
![Page 26: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/26.jpg)
Hypothesis test: modeling the variability
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 26 / 55
![Page 27: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/27.jpg)
Decision table
H0 true H0 falseno difference difference
Do not reject H0 Right decision Wrong decisiontype II error
Reject H0 Wrong decision Right decisiontype I error
Acceptable errorsIn a type I error, the null hypothesis is really true but the statisticaltest has led you to believe that it is false. It is a false positive.In a type II error, the null hypothesis is really false but the test hasnot picked up this difference. It is a false negative.
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 27 / 55
![Page 28: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/28.jpg)
Decision rule
Construction of a testFormulate the two hypothesesConstruct the test statisticDefine its distribution under the null hypothesisCalculate the p-valueDecide if the null hypothesis is rejected or not
Definition of a p-valueIt is the probability of seeing a result as extreme or more extreme thanthe observed data, when the null hypothesis is true
Decision ruleThe null hypothesis is rejected if the p-value is lower than a giventhreshold
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 28 / 55
![Page 29: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/29.jpg)
Multiple testing
The result of a test can be viewed as a random variable:0 if the result is a true positive1 if the result is a false positive
By definition, P(to be a false positive)=α
QuestionPerform G=10000 tests at level αWhat is the expected number of false-positive ?
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 29 / 55
![Page 30: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/30.jpg)
Contingency table for multiple hypotesis testing
True Falsenull hypotheses null hypotheses
Declared True Negatives False Negatives Negativesnon-significant
Declared False Positives True Positives Positivessignificant
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 30 / 55
![Page 31: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/31.jpg)
P-value adjustment
Adjust the raw p-values to controlFWER = P(FP > 0) (Bonferroni procedure)FDR = E(FP/P)ifP > 0 or 1 otherwise (Benjamini-Hochbergprocedure)
Calculating adjusted p-valuesStart with (unadjusted) p-values for G hypotheses
1 Order the p-values p(1) < . . . < p(G)
2 Multiply each p(i) by its adjustment factorBonferroni: ai = GBenjamini-Hochberg : ai =
Gi
3 Let p(i) = aip(i)4 Set p(i) = min(p(i) , 1) for all i
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 31 / 55
![Page 32: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/32.jpg)
Test for RNA-seq data
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 32 / 55
![Page 33: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/33.jpg)
Distribution for count data
From D. Robinson and D. McCarthy
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 33 / 55
![Page 34: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/34.jpg)
Notations and framework
Let Yg11, . . . ,Yg1n be independent counts from condition 1,
Yg1` ∼ NB(s`λg1, φg)
Let Yg21, . . . ,Yg2n be independent counts from condition 2,
Yg2r ∼ NB(srλg2, φg)
s` the library size of sample `λgj the proportion of the library for this particular gene g incondition j .We want to test
H0 = {λg1 = λg2} vs H1 = {λg1 6= λg2}
Need to estimate λg1, λg2 and φg
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 34 / 55
![Page 35: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/35.jpg)
Exact Negative Binomial TestRobinson & Smyth (2008) Biostatistics
If librairie sizes are equal (Anderson & Boullion, 1972)s` = s for ` = 1, . . . ,nYg11 . . .+ Yg1n ∼ NB(nsλg1, φg/n)Yg21 . . .+ Yg2n ∼ NB(nsλg2, φg/n)There exists a sufficient statistic for λg1, λg2
φg can be estimated independently from λg1, λg2
The normalisation is performed to get librairies with equal size
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 35 / 55
![Page 36: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/36.jpg)
Limitations of the exact test
It is assumed that mg1` = s` × λg1
Consequently it is only developed when two conditions areavailable
However reality is more complex
Even in simple design, replicates can vary with others factors suchas time of sampling, ...
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 36 / 55
![Page 37: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/37.jpg)
GLM framework for RNAseq data
Let Ygjvr be the counts of reads for gene g in the sampledescribed by the uplet (j , v , r)Generalized Linear Model allows to decompose a function of themean of the observations
We assumeYgjvr ∼ NB(µgjvr , φg)
withlog(µgjvr ) = log(sjvr ) + log(λgjvr )
wheresjvr is the library size for sample described by (j , v , r)log(λgjvr ) = f (g, j , v , r) for example
log2(λgjvr ) = Intercept + αgr + βgj + γgv + δgjv
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 37 / 55
![Page 38: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/38.jpg)
Inference and test
Parameters are those describing the mean and the dispersion φg .
They are estimated by the quasi-likelihood estimators: φg areestimated and parameters describing the mean are thenestimated by maximizing a function of the data and the dispersion
Test based on the likelihood ratio test or the Wald test
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 38 / 55
![Page 39: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/39.jpg)
Dispersion estimation
Few replicates to accurately estimate the dispersion parameterDESeq: φg is a smooth function of λg = λg1 = λg2
edgeR: empirical Bayesian procedure to estimate φg
... and many, many more methods!
Soneson & Delorenzi (2013, BMC Bioinf) compared 11 methods using simulations:No single method is optimal under all circumstances ...
Nookaew et al. (2012, NAR) compared microarray & RNA-seq differential analyses:Importance of mapping to estimate gene expression level
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 39 / 55
![Page 40: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/40.jpg)
Example
Consider a project where a wild-type plant and three mutants arestudiedAvailable data are three biological replicates of the geneexpression of each type of plantThe aim is to find genes affected by each mutationHow can you answer this question ?
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 40 / 55
![Page 41: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/41.jpg)
Three statistical frameworks
A negative binomiale distribution (2008)- Expression = library size ×λcondition
A Generalized Linear Model (2012)- allows us to decompose the expression- each condition is described by several factors
log(λcondition) = Cst + αgenotype + βstress + γgenotype,stress
- Effect of each factor is testedA linear model (2014)
- data are transformed to work with a Gaussian- allows us to decompose the expression
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 41 / 55
![Page 42: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/42.jpg)
In practice
Do we filter genes with low expression (yes or no)
How to model the gene expression (NB, GLM or LM and whichfactors are important to consider ?)
Which method to estimate the variance of the gene expression(several methods)
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 42 / 55
![Page 43: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/43.jpg)
Large evaluation study
We want to answer these questions
How the statistical models fit RNA-seq data ?Are DE genes discriminated from NDE genes ?Are the false-positives controlled ?Are the methods powerful (able to find the truly DE genes)
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 43 / 55
![Page 44: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/44.jpg)
Which kind of data is relevant for an evaluation ?
Real data:More realistic... but no extensively validated data yet available
Simulated data:Truth is well-controlled... but what model should be used to simulate data? How realisticare the simulated data? How much do results depend on the modelused?
Our idea was to create synthetic data
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 44 / 55
![Page 45: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/45.jpg)
Which kind of data is relevant for an evaluation ?
Real data:More realistic... but no extensively validated data yet available
Simulated data:Truth is well-controlled... but what model should be used to simulate data? How realisticare the simulated data? How much do results depend on the modelused?
Our idea was to create synthetic data
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 44 / 55
![Page 46: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/46.jpg)
Creation of synthetic datasets
H0 genes
Validated
Unknown status
H0 full dataset
H1 rich dataset
Leaves vs Leaves Buds vs Leaves
qRT-PCR
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 45 / 55
![Page 47: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/47.jpg)
Creation of synthetic datasets
H0 genes
Validated
Unknown status
H0 full dataset
H1 rich dataset
random sub-selection
random sub-selection
Synthetic dataset
Leaves vs Leaves Buds vs Leaves
qRT-PCR
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 45 / 55
![Page 48: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/48.jpg)
Creation of synthetic datasets
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 45 / 55
![Page 49: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/49.jpg)
Definition of the truth
the set of truly DE genes251 DE genes identified by qRT-PCR among 332 randomly chosengenes
the set of truly NDE genesThe proper identification is not straightforwardDefinition of two setsNDE.union: may include some genes that are not truly NDENDE.inter: may exclude some truly NDE genes.
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 46 / 55
![Page 50: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/50.jpg)
The 3 frameworks described by 9 methods
edgeR and DESeq are NB-based method
Expression = library size × λtissue
glm edgeR and DESeq2 are GLM approaches
log(λcondition) = Cst + αtissue + βbiological replicate
limma-voom is a linear modelData are transformed with the voom method
Expression = Cst + αtissue + βbiological replicate
All methods except DESeq are also applied on filtered dataIn each method, FDR is fixed at 5 %
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 47 / 55
![Page 51: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/51.jpg)
Evaluation criteria
How the statistical models fit RNA-seq data ?→ study of the p-value distribution
Do p-values well discriminate DE and NDE genes ?→ ROC curves
Are the false-positives controlled ?→ proportion of truly NDE declared DE
Are the methods powerful (able to find the truly DE genes)→ proportion of truly DE declared DE
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 48 / 55
![Page 52: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/52.jpg)
Definition of a ROC curve
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 49 / 55
![Page 53: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/53.jpg)
Distribution of the p-values
MethodWhen no difference is expected, histogram of the p-values areexpected to be uniform histogramFor each synthetic dataset, 100 evaluations of the uniformdistribution of 1000 genes randomly chosen in the full H0 datasetare performed
the raw p-values are not properlycalculated (67% of tests arerejected after a strict FP control)
test statistic values are smallerfor linear or generalized linearmodels
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 50 / 55
![Page 54: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/54.jpg)
Discrimination of DE and NDE genes
Methodsort raw p-values into ascending ordercompare them with the truthconstruct a ROC curve and calculate AUCAUC close to 1 indicates a good discrimination
For linear model or glm, the AUCis high and independent of theproportion of full H0 datasets
For NB-based method, the AUCsteadily decrease with theincrease of the proportion of fullH0 dataset when it is larger than0.3-0.4
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 51 / 55
![Page 55: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/55.jpg)
FDR estimation
MethodProportion of truly NDEamong the declared DEExpected value : 5%
For NB-based method, both bounds are close to 0For DESeq2, the FDR is always lower than 5%For glm edgeR, the interval generally contains 5%For limma-voom, the FDR control is more variable but the filteringstep stabilizes its behavior
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 52 / 55
![Page 56: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/56.jpg)
Are truly DE declared DE ?
MethodProportion of truly DE genes among the declared DE genes
LM or GLM based-methodsshow a high TPRFor NB-based methods, the TPRis a function of the full H0dataset proportion.The variance-mean relationshipmodeling and the data filteringseem to have only a limitedimpact.
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 53 / 55
![Page 57: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/57.jpg)
Definition of an indicator of quality
An histogram with a peak at the right side = analysis of bad qualityLet’s play a game : which analysis is correct ?
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 54 / 55
![Page 58: Statistical analysis of RNA-seq data - Tom Van Dooren · Aims of the lecture Quantitative analysis of gene expression Overview of the different steps of the analysis It is not exhaustive](https://reader035.vdocuments.us/reader035/viewer/2022071006/5fc38a6b6d2a4a403b6e2cc8/html5/thumbnails/58.jpg)
Conclusions
modeling ≥ filtering ≥ dispersion
Synthetic data are a relevant frameworkForget edgeR and DESequse glm edgeR, DESeq2 or limma-voominclude biological replicate as a factorfiltering allows methods to control FDR
M.-L. Martin-Magniette Analysis of RNA-Seq data INRA 55 / 55