stages of development (sod) national institute on disability, independent living, and rehabilitation...

Copyright National Institute on Disability, Independent Living, and Rehabilitation Research

Stages of Development (SoD)

National Institute on Disability, Independent Living, and Rehabilitation

Research (NIDILRR)

9/16/2015 1


PurposeThese PowerPoint slides provide additional

information on NIDILRR’s proposed Stages of Development (SoD).

These slides should be read in conjunction with the Stages of Development (SoD) Overview.

Questions pertaining to use of these slides can be emailed to SoDComments.

We will respond as quickly as possible!

9/16/2015 2

http://neweditions.net/meetings/nidilrr-stages-development







mailto:[email protected]


Introduction

NIDILRR’s Mission

“Generate new knowledge and promote its effective use in improving the ability of persons with

disabilities to perform activities of their choice in the community,

[And]To expand society's capacity to provide full

opportunities and accommodations for its citizens with disabilities.”

9/16/2015 3

http://www.acl.gov/Programs/NIDILRR/About/About.aspx#mission


IntroductionNIDILRR achieves this mission by funding grants

comprising research, development, training, technical assistance, capacity building, and

knowledge translation activities.

Research and development activities receive the largest portions of these investments.

NIDILRR seeks to optimize benefits to Persons with Disability derived from these investments.

9/16/2015 4


IntroductionPreviously, NIDILRR developed and integrated the

Stages of Research (SoR) into its Long Range Plan (2013-2017).

The SoR have been used to structure and improve: solicitations, expert recruitment for peer-review

panels, peer-review protocols, annual reports, grant monitoring, performance metrics, strategic planning, and dissemination of findings.

NIDILRR anticipates similar uses for the proposed Stages of Development (SoD).

9/16/2015 5

https://www.federalregister.gov/articles/2013/04/04/2013-07879/national-institute-on-disability-and-rehabilitation-research-long-range-plan-for-fiscal-years


ConceptsResearch – comprises activities to create new knowledge using appropriate scientific methods. Research findings

are often documented in peer-reviewed articles and disseminated through journal publications.

Development – comprises activities to create new or improved products using appropriate development

methods. Development products are often adopted by targeted population(s) and used for intended purpose(s).

9/16/2015 6


SoD Overview

The proposed SoD has three stages: the Proof of Concept, Proof of Product, and Proof of Adoption.

Each SoD stage comprises associated: activities, outputs, and outcomes common to many

development projects.

Successful outcomes for SoD stages would support that a product is (respectively): Feasible, Realizable,

and Beneficial.

9/16/2015 7


SoD Overview

SoD Uses might include: proposal preparation; grant management, monitoring, reporting, and

evaluation; portfolio assessment and management; programmatic analysis and planning; and

dissemination.

The proposed SoD are intended to provide: Descriptive Guidances rather than Prescriptive

Requirements.

9/16/2015 8


SoD in Detail

The following slides provide information pertaining to SoD concepts along with examples.

Each slide includes a descriptive title (upper right hand corner). Many slides also include graphics.

When graphics and text occur on the same slide: please read left-hand text boxes from top-to-

bottom; and then read right-hand text boxes from top-to-bottom.

9/16/2015 9


SoD in Detail

The following slides provide exemplars and examples to stimulate thought and encourage

comment.

Many details have been omitted to avoid undue complexity.

We have included many (but not all) important concepts. We have tried to present these concepts

in a reasonably logical order.

We have also omitted many references used to construct the proposed SoD.

9/16/2015 10


SoD Stages

PROOF OF ADOPTION

PROOF OF CONCEPT

PROOF OF PRODUCT

D

First Stage activities transform the Product IDEA

into a Proof of Concept.

Second Stage activities transform

the Proof of Concept into a

Proof of Product

The development Product Idea is

determined by the investigator.

For example, the Product Idea might

derive from a literature review, prior investigator

work, market research, or some

combination.

Development (D) comprises 3

Stages, 3 Outcomes, and

associated Stage Outputs and Activities.

PRODUCT IDEA

Stage 1

Stage 2

Stage 3Third Stage

activities transform a Proof of Product into an Adopted

Product.

9/16/2015 11


SoD Outcomes

PROOF OF ADOPTION

PROOF OF CONCEPT

PROOF OF PRODUCT

A successful First Stage outcome would

support that the Product Idea is

Feasible.

For example, key product concepts,

components, & systems might be demonstrated.

A successful Second Stage outcome would

support that the Product is Realizable.

For example, an integrated & working prototype might be

demonstrated.

A successful Third Stage outcome would

support that a Product is Beneficial.

For example, a Product might be

demonstrated to be substantially adopted

and used for its intended purpose by its target population.

Stage 1

Stage 2

Stage 3

PRODUCT IDEA

9/16/2015 12


Target Market(s)

Products are associated with a Target Market(s) and Intended Use(s).

The Target Market(s) comprises those persons who will use the Product.

The Target Market might for example comprise Individuals with Disability (IWD) using the Product; or caregivers, providers, researchers, developers or others using (or purchasing) the Product in a manner benefitting IWD.

Product Examples

IWD

(Outcomes) Researchers

(OT, PT) Providers

(Engineers, Designers) Developers

Target Market Examples

(Wayfinding) Device

(Screen Reader) Application

(Service Provision) Method

(Wheelchair) Standard

(Usability Assessment) Tool

9/16/2015 13


Product Types

Development Products can usually be

associated with “Types”

Examples:

• Models (e.g. for service provision)

• Tools (e.g. for usability measurement)

• Standards (e.g. for manual wheelchairs)

• Applications (e.g. for task planning)

• Devices (e.g. for wayfinding)

• …

9/16/2015 14


Partner

Organizations

Development Projects are generally accomplished in collaboration with Partner

organizations.

Examples:

• Universities

• Manufacturers

• Distributors

• Professional Organizations

• Standards Organizations

• …

9/16/2015 15


Products and

Partners

Product Adoption (and earlier Stage activities & outcomes) is facilitated by collaboration with one or more Partner Organizations.

In practice, the Partner Organization often provides the path/means by which a Product reaches its Target Market.

Product Examples

Play Store

Manufacturer

Online Distributor

Professional Organization

Standards Organization

Partner Examples

(Wayfinding) Device

(Screen Reader) Application

(Service Provision) Model

(Wheelchair) Standard

9/16/2015 16


Partner Initiated

Requirements

Partner organizations often introduce investigators to Product requirements that are relevant to one or more stage’ activities, outputs,

and outcomes.

For this (and closely-related) reason(s), early identification of and collaboration with appropriate Partner organization(s) often improves

outcomes for all SoD stages.

Partner Organization (Product) Examples for Design, Testing ofAT Company Inc. (Wayfinding Device)

for Distribution ofOnline Software Distributor (Task Application)

for Use in PracticeProfessional Association ([Service Provision] Method)

Stage Requirement Examples

Protocols for Development of

Standard Organization ([Wheelchair] Standard)

9/16/2015 17


Stage Assessments

Each stage normally has investigator-generated

requirements which they propose to achieve.

Corresponding, each stage normally has

investigator-generated

assessments used to evaluate whether

requirements have been met.

PROOF OF ADOPTION

PROOF OF CONCEPT

PROOF OF PRODUCT

Stage 1

Stage 2

Stage 3

PRODUCT IDEA

9/16/2015 18


Stage 1

Assessment(s)

Meeting First Stage requirements

supports that the Proof of Concept is

Feasible

Examples:

• design evaluation (assess device compliance to design requirements; e.g. of device)

• bench testing (assess compliance to design, performance,… specifications; e.g. devices, applications)

• standards testing (assess compliance to safety, design, performance,… standards; e.g. of devices, applications)

• psychometric testing (assess validity, reliability,…; e.g. of tools)

• user testing (assess usability, utility,…; e.g. of methods, devices, applications…)

• …

PROOF OF ADOPTION

PROOF OF CONCEPT

PROOF OF PRODUCT

Stage 1

PRODUCT IDEA

9/16/2015 19


Stage 2

Assessments

Meeting Second Stage requirements supports

that the Product is Realizable.

Examples:

• user trials (usability, utility,… assessed by users)

• provider trials (e.g. efficiency, effectiveness, consistency… assessed by providers)

• partner trials (e.g. product design requirements… assessed by Partner)

• …

PROOF OF ADOPTION

PROOF OF CONCEPT

PROOF OF PRODUCT

Stage 2

PRODUCT IDEA

9/16/2015 20


Stage 3

Assessments

Meeting Third Stage requirements supports

that the Product is Beneficial.

A member of the Target Population only Benefits

if the product is Used (and Retained) for its

Intended Purpose.

Examples:

• Awareness (of product)

• Persuasion (positive disposition towards product)

• Decision (to obtain product)

• Use (to try product)

• Retention (to continue using product)

• Breadth (of target population adopting the product)

• …PROOF OF ADOPTION

Stage 3

PROOF OF CONCEPT

PROOF OF PRODUCT

PRODUCT IDEA

9/16/2015 21


Product Adoption and Adoption

Metrics

Specific adoption metrics depend on the Product and its Target Population and Intended Use.

One or more specific adoption metrics can be identified by the investigator for each generic adoption metric.

Awareness of…

Persuasion towards…

Decision to adopt…

Use of…

Retention of…

E.g. Hits, citations…

E.g. Disposition towards

E.g. Purchase, downloads

E.g. Use at one month

E.g. Use at one year

Generic Adoption Metrics Specific Metric Examples

9/16/2015 22


Product Adoption and Technology

Transfer

Technology Transfer comprises the plans

and strategies used to overcome barriers and to facilitate adoption of a development Product

by its Target Market and for its Intended

Use.

Technology transfer generally involves collaboration with

Partner organizations.

Example: RERC investigators are required to prepare a

Technology Transfer Plan by the end of the grant’s 1st year.

Example: SBIR investigators are required to prepare

Commercialization [Technology Transfer] Plans in the beginning of Phase II

(corresponding to the beginning of SoD Stage 2).

The NIDILRR-funded Center on Knowledge Translation for Technology Transfer

…

Provides guidance (based on best

practices) to other NIDILRR-funded

grantees to support development of

Technology Transfer Plans.

9/16/2015 23

http://kt4tt.buffalo.edu/




Stage Iteration

In-Stage Iteration (of some or all Stage activities) is normal and expected.

Across-Stage Iteration may occur for good reason, but is not typical; and may (or may not) indicate that a development project has

encountered a significant problem.

Consequently, Across-Stage iteration would normally involve NIDILRR notification and might require NIDILRR approval.

For example, First Stage design requirements (e.g. for a device) might be found to be inconsistent with a Partner initiated requirements (e.g.

for performance) found in the Second Stage.

The investigator might wish to modify First Stage requirements and repeat the stage. The investigator would start by discussing their plan

with their NIIDILRR project officer.

9/16/2015 24

Research & DevelopmentResearch and Development are often inter-dependent and inter-twined.

However, Products deriving from Research or Development are very different.

PROOF OF ADOPTION

PROOF OF CONCEPT

PROOF OF PRODUCT

Stage 2

PRODUCT IDEA

Stage 1

Stage 3Product Adoption

Study

Development Products (E.g. Adopted Models, Methods, Tools, Standards, Apps, Devices,

…)

Research Products (E.g. Publications, Proceedings,

Seminars,…)

Product User Trialing Study

Market Research Study

System Bench Testing Study

New

Kn

ow

led

ge

25


Development Example: SBIR Grants

SBIR grants and RERC development projects provide

a nice illustration of SoD concepts.

SBIR grantees are normally small businesses.

SBIR Phase I and Phase II correspond closely to SoD First and Second Stages.

SBIR investigators are required to write a Commercialization

[Technology Transfer] Plan in Phase II.

The Plan is the means by which the SoD Third Stage (Product Adoption) is to be

accomplished. ADOPTION

PRODUCT IDEA

PROOF OF CONCEPT

PROOF OF PRODUCT

Stage 1

Stage 2

Stage 3

Phase I < 1 Year

Phase II 2 Years

Phase II Commercialization

Plan

9/16/2015 26


Development Example: RERC

Grants

RERC grantees are normally universities.

Target Markets for RERC development projects may

include (for example) IWD, caregivers,

providers, researchers, or developers.

Product Types for RERC development projects may

include (for example) models, methods, tools, standards, applications,

or devices.

RERCs investigators are required to develop a

Technology Transfer Plan in the first grant year. PROOF OF

ADOPTION

IDEA

PROOF OF CONCEPT

PROOF OF PRODUCT

Stage 1

Stage 2

Stage 3

RERC Partner Organizations may include (for example)

manufacturers, distributors, professional organizations, or standards organizations.

RERC development projects may encompass one ore more of the SoD Stages

(Proof of Concept, Proof of Product, and Proof of

Adoption).

9/16/2015 27

Thank you! We appreciate your

assistance!

Copyright National Institute on Disability, Independent Living, and Rehabilitation Research 9/16/2015 28

stages of development (sod) national institute on disability, independent living, and rehabilitation...

Documents

development activities

independent living

development products

development projects

stages of research sor

research findings

new knowledge

proof of product