1 / 30 data mining with biomart
TRANSCRIPT
1 / 30
Data Mining with BioMartData Mining with BioMart
www.ensembl.org/biomart/martviewwww.biomart.org/biomart/martview
2 / 30
What is BioMart?What is BioMart?
• A data export tool
• A quick table generator
• A web interface to mine Ensembl data
3 / 30
BioMart- Data miningBioMart- Data mining
• BioMart is a search engine that can find multiple terms and put them into a table format.
• Such as: mouse gene (IDs), chromosome and base pair position
• No programming required!
4 / 30
General or Specific Data-TablesGeneral or Specific Data-Tables
• All the genes for one species
• Or… only genes on one specific region of a chromosome
• Or… make BioMart select genes
(I.e. all transcripts that match a microarry probe set, GO term, or InterPro domain).
5 / 30
ResultsResults
Tables or sequencesTables or sequences
6 / 30
The First Step: Choose the The First Step: Choose the DatasetDataset
Dataset: Current Ensembl, Human genes
7 / 30
The Second Step: FiltersThe Second Step: Filters
Filters: Define a gene set
8 / 30
Attributes attach informationAttributes attach information
Attributes: Determine output columns
9 / 30
QueryQuery
For the human CFTR gene, export the Entrez Gene ID(s) and matching Affy HG U133-PLUS-2 probeset(s)
10 / 30
Query:Query:
For the human CFTR gene, export the Entrez Gene ID(s) and matching Affy HG U133-PLUS-2 probeset(s)
• In the query:
Filters: what we know
Attributes: what we want to know.
11 / 30
Query:Query:
For the human CFTR gene, export the Entrez Gene ID(s) and matching Affy HG U133-PLUS-2 probeset(s)
• In the query:
Filters: what we know
Attributes: what we want to know.
12 / 30
Query:Query:
For the human CFTR gene, export the Entrez Gene ID(s) and matching Affy HG U133-PLUS-2 probeset(s)
• In the query:
Filters: what we know
Attributes: what we want to know
13 / 30
A Brief ExampleA Brief Example
Use the current Ensembl (archives are also available)
SelectHomo sapiens
genes
14 / 30
Select the Genes with FiltersSelect the Genes with Filters
Expand the GENE panel to enter in the gene ID(s).
Expand the ‘GENE’ panel.
ClickFilters
15 / 30
Filters (and Count)Filters (and Count)
Click “Count” to see if genes passed through your filters.
Change this to HGNC curated name. Enter “CFTR” in the box.
16 / 30
Attributes (Output Options)Attributes (Output Options)
Click on ‘Attributes’
‘Attributes’ allows you to output information.
17 / 30
Attributes (Output Options)Attributes (Output Options)
Select ‘EntrezGene ID’
18 / 30
Attributes (Output Options)Attributes (Output Options)
Select the Affy Platform ‘HG U133-PLUS-2’ in the
‘Microarray’ section
19 / 30
The Results Table - PreviewThe Results Table - Preview
For the full result table: click “Go” or View “ALL” rows.
20 / 30
Full Result TableFull Result Table
Ensembl Gene ID for CFTR
Ensembl Transcript
IDs
EntrezGene ID
Affy HG probeset
21 / 30
Other Export Options (Attributes)Other Export Options (Attributes) Sequences: UTRs, flanking sequences, cDNA
and peptides, etc
Gene IDs from Ensembl and external sources (MGI, Entrez, etc)
Microarray data
Protein Functions/descriptions (Interpro, GO)
Orthologous gene sets
SNP/ Variation Data
22 / 30
BioMart around the BioMart around the world…world…
BioMart started at Ensembl…
To where has it travelled?
23 / 30
Central PortalCentral Portal
www.biomart.org
24 / 30
WormBase WormBase
25 / 30
HapMapHapMap
Population frequencies
Inter- population comparisons
Gene annotation
26 / 30
DictyBaseDictyBase
27 / 30
GRAMENEGRAMENE
www.gramene.org
28 / 30
The Potato CenterThe Potato Center
29 / 30
How to Get ThereHow to Get Therehttp://www.biomart.org/biomart/martview
http://www.ensembl.org/biomart/martview
• Or click on ‘BioMart’ from Ensembl
30 / 30
Worked ExampleWorked Example
• Follow the worked example on pg 26
• Then, do the exercises on pg 34 (answers on pg 37)
This module should do the following:• Show you how to export multiple data types from
Ensembl for gene IDs or chromosomal regions.
31 / 30
Ensembl Core DatabasesEnsembl Core Databases
Relational Database• Normalised• Each data point stored only onceTherefore:• Quick updates• Minimal storage requirementsBut:• Many tables• Many joins for complicated queries• Slow for data mining applications
32 / 30
Normalised SchemaNormalised Schema
gene_id gene.symbol
9970 SMAD1
1712 SMAD2
8240 SMAD3
1967 SMAD4
… …
gene_id transcript
9970 ENST00000302085
1712 ENST00000262160
1712 ENST00000356825
8240 ENST00000327367
1967 ENST00000342988
… …
gene_id stable_id
9970 ENSG00000170365
1712 ENSG00000175387
8240 ENSG00000166949
1967 ENSG00000141646
… …
33 / 30
BioMart DatabaseBioMart Database
Data warehouse• De-normalised• Query-optimisedTherefore:• Fast and flexible• Ideal for data miningBut:• Tables with apparent “redundancy”• Needs rebuilding from scratch for every release
from normalised core databases
34 / 30
De-Normalised SchemaDe-Normalised Schema
gene_id transcript_id gene.symbol
ENSG00000170365 ENST00000302085 SMAD1
ENSG00000175387 ENST00000262160 SMAD2
ENSG00000175387 ENST00000356825 SMAD2
ENSG00000166949 ENST00000327367 SMAD3
ENSG00000141646 ENST00000342988 SMAD4
… … …
35 / 30
SPECIES
FOCUS
REGION
SNP
PROTEIN
HOMOLOGY
GENE
EXPRESSION
REFSEQ
INTERPRO
GO
SWISSPROT
EMBL
AFFYMETRIX
FASTA
FILE
EXCEL
TEXT
GTF
HTML
DATASET FILTER ATTRIBUTES
Information FlowInformation Flow
REGION
SNP
PROTEIN
HOMOLOGY
GENE
EXPRESSION