fedramp jab prioritization 10_18_2016.pdf

Advancing Government through Collaboration, Education and Action

Cloud Community of Interest

FedRAMP Accelerated Feedback

Joint Authorization Board (JAB) Prioritization

Government – Cloud Service Provider (CSP) – Third Party

Assessor Organization (3PAO)

Release Date: October 18, 2016

Advancing Government through Collaboration, Education and Action 2

FedRAMP High-Level Prioritization Criteria



• Purpose: Ensure that cloud service offerings selected for Joint

Authorization Board (JAB) Provisional Authority to Operate (P-ATO)

meet U.S. government’s (USG) cloud strategy and FedRAMP strategy.

• Outcome: JAB identifies the highest priority cloud service offerings

the JAB must authorize.

• Assumptions:– FedRAMP Director will grant a CSP the status of “FedRAMP Ready”

for the cloud service offering that demonstrates the ability to

successfully complete the JAB authorization process and have an

acceptable risk posture, as defined by the FedRAMP Readiness

Assessment.

– The number of FedRAMP cloud service offerings selected will be in

accordance with the capacity of the JAB.

3



(continued)

• Current United States Government (USG) Market Use and

Demand

– Current usage of CSP offering:These CSP offerings in-use at multiple Agencies will measure the potential Return on

Investment (RO)I for a given JAB P-ATO. Identified by # of CSP offerings:

• In use at more than one Agency with an Agency ATO

• With a current Agency ATO interested in transitioning to JAB P-ATO

• Used across Agencies not reported and that FedRAMP has no ATO on file for the

CSP offering

– Demand for a CSP offering: number of Agencies that have expressed interest in a CSP

offering that have not yet received an ATO

• Impact to USG – Watermark and Innovation

– 70% of government cloud systems are at the “moderate watermark” per National

Institute of Standards & Technology (NIST) definition

– Highly innovative cloud products that may offer potential cost savings

4



(continued)• Confidence and Commitment of CSP Excellence

– Organizational Maturity

• Understanding of FedRAMP and requirements

• Systems that belong to mature organizations with processes in place

– Capability Maturity Model Integration (CMMI) Level 3+

• Systems with very low risk associated with critical capabilities review

• Corporate culture with IT security in mind, previous audits available

– Project Management

• Timeliness of Deliverables, Deadlines (self-imposed/FedRAMP required) are

met

• Established communication channels with FedRAMP Program Management

Office (PMO)Resources allocated and committed to project

– CSP Dedication to Excellence

• Relationship building/established

• CSPs desire and commitment to JAB path

5



JAB Prioritization – Government

Prema Nair, NIH

Ann Marie Keim, NASA, Sharon Ehrenberg, VA


6


• Federal Market

– Generate an increased interest for suppliers to obtain

FedRAMP authorization.

• Benefits

Streamline testing analytics

Opportunities for innovation

Contribute to RFIs

Accelerate award process

ROI

JAB Prioritization


• Areas of consideration

– Category Deployment Models to increase similar product

interest and opportunity methods desired by agencies in:

Public Cloud

Community Cloud

Hybrid Cloud

Private Cloud

[But Private Cloud not a consideration for JAB path,

just agency authorization path?]

JAB Prioritization


Opportunity Tools

• Federal Business Opportunities (FBO)

– Utilizing the search feature for Cloud allows for a

snapshot of current requirements

– https://www.fbo.gov/

• Federal Procurement Data System (FPDS-NG)

– Utilizing the reports feature for Cloud provides previous

award history for all federal agencies

– https://www.fpds.gov/fpdsng_cms/index.php/en/

9

https://www.fbo.gov/

https://www.fpds.gov/fpdsng_cms/index.php/en/


Training methods

• Current knowledge/training methods for Cloud

guidance and options

National Defense Industry Association (NDIA) – central

scheduling and marketing of events specific to Cloud

General Services Administration (GSA) - gsa.gov/cloud

Department of Defense (DoD)

Department of Homeland Security (DHS)

National Aeronautics & Space Administration (NASA) Solutions

for Enterprise Wide Procurement (SEWP)

FedRAMP.gov

Private Industry

10


• Existing strong commercial capability offering

New to the market, or ground-breaking new capability to the

Federal market - may find this entry harder.

So many companies out there do not want to go through the

hoops –expense/time of FedRAMP.

How can we make it attractive for these companies to bring their

capabilities into the federal market?

This would help with the need to have more of a selection of

similar product.

Otherwise if restricted to FedRAMP authorized only, amounts to

sole source, so there needs to be more competition.

Market penetration in the Federal market


Company with Previous Federal Capabilities

• Prioritization for an established company providing

assurance they will be around for more than a few

years.

• Historical validity and stability of a company, or

maturity model.

• However does this present a barrier to new

businesses and new capabilities or competition?

12


Possible Exit Strategy for Non-Compliant CSPs

• JAB to de-prioritize CSPs that are not maintaining their

security posture

– Offload to agency if they have one as a customer for

Continuous Monitoring.

– Or if no agency, held to an Improvement Plan with a 3 to

6 month ‘graduation’ before revoking their authorization.

• More available time for JAB as a result.

• Possible Criteria:

– Patching, end of life support for hardware or software.

– Data breaches, lack in what they are doing to address.

them.

13


JAB Prioritization Recommendations

• Have an Inter-Agency board Advisory Committee

– Consisting of small, medium, and large agencies with

varying cloud experience – expert to less experienced.

– 7 to 9 members with an Office of Management & Budget

(OMB) MAX page for collaboration.

– To give input into FedRAMP of what types of services

would be useful/desirable, possibly already are in

commercial but not yet in Federal for prioritization.

– Committee to also track other important inputs such as

Cloud Computing Summit, Cloud Security Alliance, and

others.

14


JAB Prioritization Recommendations

• Procurement requirements should be considered.

• Prioritize on CSP that have contractual security

requirement language with acquisitions or third

parties / have been audited.

• Contract procurement language should reflect

FedRAMP requirements to be considered.

15



JAB Prioritization – CSP

Bobbie Browning, Browning Partners

Nate Johnson, Microsoft


16


Prioritization Steps

1) Ability to achieve FedRAMP Ready status

2) Illustrate Cloud-service testing

3) Accommodate innovation and Small or Socio-

economically designated CSP solution

– Necessary services

– Competitive services

– Compelling technologies

– PMO “Shark Tank” Capability Interview to contribute to

final priority

Criteria for JAB Prioritization


• Evaluated by the 3PAO as part of FedRAMP ready

process

• Must be completed and of adequate quality for

assessment completion

• Provides an additional gating function to ensure CSPs

are ready to move forward with assessment and

prioritization after FedRAMP Ready determination

Evidence of completed documentation


Illustrate Cloud-service testing

• Commercial CSP proven in another vertical

• Document compliance in CSP Questionnaire

• Aligns with Federal priorities: OMB, Chief

Information Council (CIO) Council etc

– Financial - highly resilient and required

– Infrastructure and/or security > fundamental to federal

– Human Resources (HR) > employee engagement

– Retail > citizen and beneficiary services

19


Accommodate Innovation and Small

or Socio-economically designated CSP

• Adopt a Portfolio Management strategy

– Necessary services

– Competitive services

– Compelling technologies

• Establish PMO “Shark-Tank” Capability Interview to

contribute to final priority

• Incorporate consideration for small businesses in

the scoring of these Capability Interview CSP

presentations

20


Criteria to participate in Accelerated JAB

Provisional Authorization to Operate (P-ATO)

1) Create qualification scheme template

2) Establish a weighted-score method to rank CSPs

3) Share expectations to participate in JAB-P-ATO

4) Direct CSP to complete Questionnaire

5) Apply weighted-score to completed Questionnaire

6) Rank CSPs to determine readiness for FedRAMP Ready

Audit

7) Schedule 3PAO to perform FedRAMP Ready Audit

8) Identify gaps, determine & communicate next step

9) Obtain CSP commitment to satisfy gaps

21


Criteria to participate in Accelerated JAB P-ATO

(continued)

• FedRAMP Ready – CSP get a 3PAO & validate

– 3PAO assessment of rated maturity – (Rank 1-5)

• Evidence of quality and cloud testing

– Documentation package readiness and quality (Rank 1-

5)

• PMO/JAB initial review of quality

– Shark-Tank Capability Interview - CSP defend (Rank 1-5)

• Capability needed – Based on demand research: RFI’s, Sources

Sought, and any other services responsive to agency demand

• Ground-breaking capability matching Govt Directives

• Introduces competition into the market of existing capabilities

being used

• Commercial CSP proven in another vertical with capabilities that

the govt doesn’t yet know they need 22


Prioritization Score Example

23

43

5

FedRAMP Ready Maturity(3PAO assessed)

Documentation Quality Capability Interview

Total CSP Score = 12 out of possible 15


Potential CSP Questionnaire

• How is your organization structured to pursue a JAB P-ATO

and maintain it operationally, once achieved?– Dedicated security officer?

– HR security training for all new employees and process to recertify annually?

– Sales trained, and with what frequency, to field questions from customers

– Additional comments?

• What P&P demonstrate your organizational maturity?– Do you have a ticket system to document every system change?

– Describe how, and with what frequency, you establish and communicate

priorities across the organization.

– How do you validate all stakeholders are aligned with those priorities?

• How would you describe the resiliency of your infrastructure?

– Disaster Recovery

– Redundancy

– Monitoring Systems

– System Scans

24


• Adopt a Portfolio Management strategy

• Allocate rotating PMO resources for quarterly, day-long

prioritization sessions to establish the CSP queue

• Include Agency reviewers in CSP presentations

• Conduct 15 – 30 minute CSP presentations similar to VC:

– Solution capabilities

– Market diversification

– Government market penetration – not a requirement

– Security compliance experience

• Caucus reviewers immediately following presentation for an

up/down decision to proceed to next step

• Notify CSP of disposition within 2 weeks of presentation

• Leverage “FedRAMP High-Level Prioritization Criteria”

PMO “Shark-Tank” Capability Interview


Considerations to Increase Success of CSPs

• Communicate expectations to CSP

– Rigor & benefits of a JAB Technical Review

• Access to compliance experts

• Positive operational impact: “smart stuff” in detailed requirements

– Initial investment in infrastructure and resources -

dependent upon:

• Existing resilience at beginning of the process

• Approach to satisfying requirements e.g. redundancy

– Recurring investments – one CSP example:

• Operations – 2 Full Time Employees (FTEs)

• Security – 1 FTE

26


Considerations to Increase Success of CSPs

(continued)• Supplement PMO resources

– Recommend CSP hire 3PAO Consultants > ROI

– Establish a CSP Mentor Program – “If only we had known. . .or had an

experienced CSP to ask. . .”

• Absence of clarity

• Subject to many judgment calls

• Lessons learned from a (Software as a Service) SaaS CSP

about the JAB process

1) Get the whole village engaged

2) Get organized, very, very organized

3) Embrace the change

4) Get help

5) Educate your Federal customers

6) The FedRAMP JAB P-ATO is the beginning of something, not the end

27


Recommendations for PMO Resources

• Allocate PMO resources to accommodate

Continuous Monitoring and new certifications

– Maintain existing CSPs – ConMon: 50%

– Support 8 to 12 new certifications annually: 30%

– Rotate PMO staff through Shark-Tank reviews 20%

• Identify criteria for participation in Shark-Tank

– Create a qualification scheme

– CSP complete questionnaire that is quantifiable

– Establish a score that illustrates ranking

• Assess ability to impact the competitive landscape

• Consider dynamic allocation of PMO resources to

respond to external pressures that change priorities 28



JAB Prioritization – 3PAO

Maria Horton, Emesec

Abel Sussman, Coalfire


29


JAB Prioritization 3PAO Recommendation(s)

• Objective:

– Identify 3PAO community Recommendations on

concrete criteria for CSP designation as a JAB

Prioritized Solution

• Detail the methodology for designation

• Methodology of Working Group

– Focus group discussions from a variety of 3PAOs

• Self selected group –

• Must be members of ACT-IAC

– Outside vetting may enhance receptivity by cloud

community

30


Assumptions

• JAB Prioritization 3PAO Working Group

recommendations to be shared and vetted for input

or feedback with:

– Government Working Group

– CSP Working Group

• Presentation to GSA FedRAMP PMO

– Potential input from FedRAMP PMO may alter

recommendations:

• JAB timeline of “prioritized evaluation” (is this hard or soft?)

– First in- First out to evaluation? Or other commitment schedule

• Non-punitive follow up to “prioritized” designees for JAB Board

31


Prioritizing Innovative & Necessary Services

• Different ways to define innovative

– May reflect risks, new goals, etc. More likely to be one-of needs by

CIOs

– Question: How do we acknowledge innovative services that

government does not know it needs (yet) and prioritize without

politicizing?

• Example may be a Cyber Security Prevention Tool for Terrorist events

• Necessary services interpreted differently by Agency

– Quantity needs may be seen as necessary such as sought-after tools

echoed by many agencies

• Recommend: The number of needed Agencies that reflect broad use be

defined and publicized

– Question: How many agencies are required to equate broad

Government use?

32


Factor 1: Timing for CSP Participation

• Recommend development of “Cohorts” - a wave or phased

release of those submitted into the accelerated program

– Timing of the cohorts are not defined but 2-4 releases per year

seems to be reasonable

– Few cohorts in the beginning to pilot program and gain feedback

– CSPs should know the cohort timings and criteria for submission and

for review.

• Understand no reprisals for not being selected or later moving out of the program

• Open Questions:

– How do we determine the number of CSPs selected for the Cohort?

– Do all Cohort slots need to be filled?

• Retain openings if CSPs do not meet certain threshold

• Reserved openings for small business, congressional input, etc

33


Factor 1: CSP Selection Criteria

• Completed FedRAMP Ready

– No measurable criteria other than FedRAMP Ready testing is

currently done

– Maturation of Security Program (May submit a mix) as determined by

FedRAMP Ready designation of Level I-V

• Category of services

– Determined by Government strategic needs/CIO Council

– Examples

• Infrastructure

• Admin / Back Office

• Apps for Gov’t Employees

• Apps for Citizens/Beneficiaries

• Security?

34


Factor 2: Selection Variables

• “Clean” or Complete documentation

– If quality of paperwork is an indicator, this should become

a FedRAMP Ready element

• Not currently part of FedRAMP Ready - just the technical

evaluation

• Contractor size or socio-economic designations

• Unique Congressional interest

• Other variables?

35


Sample

• Cloud Solution

– Identified in one or more Agency Strategic Plan

– Has one or more Capital Expenditures (CPIC or OMB

Exhibit 300’s) dated within the last 2 years

• Makes it current and not a long standing issue

– The CSP has been in business at least 24 months

• Goes to stability

• Is in commercial use -

– The FedRAMP Ready evaluation has been completed

within the last 6 months

36


Evaluating the Selection Criteria

• Selection criteria for Prioritized CSPs need::

– Need to be quantitative in nature

– Clearly defined criteria to prevent bias perception

• Selection criteria and weights assigned can change (per fiscal year or

cohort) to reflect Government needs

– An example of the variable / weighting system follows:

37

CSP Solution 1

Score Weighting Total

(1-5)

Criteria 1 1 20% 0.2



Criteria 4 5 15% 0.75


2.75Total

CSP Solution 2

Score Weighting Total

(1-5)




Criteria 4 1 15% 0.15

Criteria 5 4 25% 1

2.85Total


Marketing Suggestions for JAB Prioritization

• Specific criteria for prioritization must be publicized and achievable by all

businesses to demonstrate a balanced playing field

– Consider piloting the initial criteria variables and ask for industry feedback

• Defined Prioritization elements need to be measurable and possibly

identified in the FedRAMP Ready process.

– Would this allow just large companies to jump ahead since they may have

more funding and resources available?

– The more elements added to FedRAMP Ready, results in a bigger hurdle for

small businesses and new Federal businesses

– Cause and Effect: Likely to result in more CSPs needing to complete all

documents prior to FedRAMP Ready

38


Open Questions

• Customers talk about the cost of compliance, and how do we equate this to ROI? How can we articulate

this?

• Can CSPs leverage 18F and/or Compliance Masonry to streamline documentation maintenance

(ultimately allowing them to reallocate resources to technical maintenance for Continuous Monitoring)?

• If a CSP is selected for JAB prioritization because they have demonstrated maturity in through

FedRAMP Ready, and the CSP isn’t ready to present to the JAB in a year, what happens?

– Is there or will there be an extension or waiver process? Are they “pushed” back to regular process?

• Should we limit categories or types of solutions will be included within the priorities?

– Recommend limiting IaaS to 1 per year -- several already exist

– Recommend how to identify the unique needs that are a result of current timing

– Determine if at least 2 competitors should be JAB ready for those unique innovations

• How do we address the complaints, accusations or inherent barriers to small business and potential

innovators?

– Each step adds overhead costs to entering the Federal Market

– Plan in advance how to address Congressional Inquiries

– Determine if 1-2 slots are left for those businesses using Congressmen to push from their local areas

– How to eliminate politics if this isn't what the JAB should handle

39


End of Presentation – Open Discussion

40


Thank You to Our Contributors

Doug Noakes, Booz Allen Hamilton; Maria Horton,EmeSec; Abel Sussman,

Coalfire; Bruce Hamilton, EY; Daniel Lee, Censeo Consulting; Kyle

Hendrickson,BRMi; Richard Beutel,Cyrrus Analytics; Saif Rahman, Quzara;

Bobbie Browning, Browning Partners; Ken E. Stavinoha, Ph.D, CISCO

Systems; Marilyn Hays, HPE; Brian Cram,IBM; Erica Poskaitis, Oracle;

Stacy Cleveland, HPE; Nate Johnson, Microsoft; Eric Adams, IBM; Prema

G. Nair, NIH; Phillip D. Dixon, Dept of Labor; Michael Christopherson, GSA;

John Frary, CSRA; Michael Cassidy, DOJ; Sharon Ehrenberg, VA; Ann

Marie Keim, NASA; Roopangi Kadakia,VA; Monette Respress,Noblis

41

fedramp jab prioritization 10_18_2016.pdf

Documents