value-based prioritization. why do we need a process for prioritization? transparency different...

Value-Based Prioritization

Why do we need a process for prioritization?

• Transparency• Different target communities• Different opinions• Allows evaluation of new projects

Allocate our limited resources to those activities that best support our mission

"The iPlant Collaborative will facilitate the transformation of the life sciences research and

education communities through the enablement of novel, collaborative, interdisciplinary

research, and educational activities requiring cyberinfrastructure, and by providing the

necessary tools, services, and related training to apply the enabling CI to questions that were

previously difficult or impossible to efficiently address."

• Facilitate the transformation of life sciences and education

• Enable novel, collaborative, interdisciplinary research and education

• Provide tools, services and training to target questions that were previously difficult or impossible to efficiently address.

iPlant Strategy: The Scientific Enablement Plan

• Enable generating & storing genotypes• Enable high-throughput phenotyping• Enable inclusion of Environment in G2P

models• Enable HP modeling, especially for ecology

and agriculture

Molecular Breeding

Biodiversity Preservation

Evolutionary Relationships

Ecological Modeling

Crop Modeling

Climate Model ProductsComparative

Genomics

Sequencing & Assembly

Image-based Phenotyping

Modeling

Genotypes Phenotypes

Dark green-solid foundationLight green-in progressWhite-new areaYellow-logical grouping

Synthetic Biology

Predictive Biology

Association Studies

Environmental Data

EnvironmentsMolecular

Phenotyping

Trait Data

Annotation

Phylogenetic Support

Enabling Science

PathwaysNetworks

iPlant City

Current Status

Complete infrastructureModerate adoptionSome areas of wider adoption

Our Objective

…cannot be built from this stage

Needs: Increased Adoption

Needs: Build What Users Need (drives adoption, retention and innovation)

We should only do what has the potential to bring value to the user

Our Prioritization Criterion

Usefulness(e.g. BLAST)

Innovation(e.g. GxE)

A Note of Caution

THE PROCESS

Prioritization

• Produce a list of all deliverables across all projects (EOT component implicit)

• List is shared (Google Doc), every person in SciPlant can add what they think is missing (review period: 3 days)

• After end of review period every person in SciPlant will receive a copy of the list of deliverables

Prioritization

• Every person independently scores every deliverable on 2 separate criteria (1 week):– Potential value to the community (1: no value – 9: "killer

app"). In rating deliverables, people should also take into account how certain they are of their value assessment. Low certainty should push scores towards 5.

– Perceived technical difficulty (1: easy – 5: extremely hard). Low certainty should push the scores towards 3.

• Every participant will have incomplete knowledge but the consolidation of all scores will provide a "community wide consensus".

Prioritization• Scores for each individual will be normalized to the mean• For every deliverable the final score will be the sum of the

normalized individual scores• Priorities will be placed on a matrix according to they value and

technical difficulty• We expect to be able to identify 5 clusters: in order of priority:

1. Low hanging fruits: hi val, easy2. New features: hi val, hard3. Bells & whistles: low val, easy4. Routine: mid val and difficulty5. Backburner: low val, hard

• The allocation of a deliverable to a cluster cannot be changed.

EasyExtremely hard

Extremely valuable

No Value

x

x

x

x

x

xx

x

x

xx

x

x

x

x

x

Prioritization

• Independently from the deliverable prioritization, members of SciPo will rank by consensus the 12 projects from the Science Enablement Plan. If consensus cannot be reached, individuals will rank the projects independently and the final ranking will be determined by the average rank.

• The project priority will be used to prioritize the individual deliverables within a cluster.

Community Feedback

• SAB will review the whole priority list and will be able to make recommendations that will then be discussed in SciPo.

• The "high value" items will be displayed on our webpage, with a timeline for the short-term ones ("low hanging fruits").

• Users will have a mechanism to vote on priorities and possibly suggest new deliverables.

• If users provide their own resources, the priorities can be raised.

• The list will also be made into a slide that can be used to elicit informal feedback when presented at conferences etc.

New Projects

• The project rank and the deliverable ranks with corresponding matrix will provide a baseline against which to score new and community submitted projects.

• The prioritization process should be repeated every year to determine the new priorities.