Surviving and Thriving on NASA Audits….

One Organization’s Storyof

Turn Around and Success

Emily Bird BroderickSafety and Quality Assurance

Space Department

emily.broderick@jhuapl.edu

2

Divisions ofThe Johns Hopkins University

School of Arts & SciencesWhiting School of EngineeringSchool of Professional Studies in Business & Education

School of Hygiene & Public HealthSchool of MedicineSchool of Nursing

Applied Physics Laboratory

Nitze School of Advanced International Studies

Peabody Institute

3

The Johns Hopkins University Applied Physics Laboratory

The Applied Physics Laboratory (APL) is a not-for-profit, federally chartered University Affiliated Research Center (UARC) for engineering, research and development.

Our 399-acre campus is home to more than 4,200 women and men working on more than 400 programs that protect our homeland and advance the nation's vision in research and space science.

The core purpose of JHU/APL is to enhance the security of the nation through the application of science & technology.

APL is firmly committed to space science & technology and other challenging areas that are also synergistic with our core purpose. The Lab also carries on important work in areas (e.g., biomedicine) that are synergistic with JHU as a whole.

Our goal is to make critical contributions to critical challenges.

4

The Applied Physics LaboratorySpace Department

The APL Space Department has successfully developed and flown over 150 space science instruments since 1970.

We have assisted NASA in space exploration from the Sun to Pluto, most recently with the STEREO and New Horizons missions.

Currently, the Space Department has over 35 active projects and programs ranging from National Security Space studies of operational conceptions to full NASA space missions such as Radiation Belt Storm Probes (RBSP).

5

The APL Space DepartmentSafety & Quality Assurance Group (SQA)

SQA is responsible for Mission Assurance activities including: Systems Assurance Management Hardware Quality Engineering Software Assurance System Safety Engineering Configuration Management Quality Inspection Supplier Quality Management Internal Audits ESD Compliance and Training Operator and Inspector Workmanship Certification AS9100/ISO Implementation Space Department Staff Training and Certification Records Corrective Action Management and Tracking Metrics Development and Analysis Process Improvement Initiatives

6

Setting the Stage… Assessments were conducted by NASA’s team at APL in 2003 and 2005. Over 75

findings were issued (not counting observations) during the two assessments.

APL received “red” and “yellow” scores with report findings such as:

“Top management commitment to quality and safety is not apparent.”

“Development and maintenance of formal quality records, plans, and procedures are incomplete and inconsistent.”

“No formal corrective/preventative action process is in evidence.”

“The demonstrated level of commitment

to a systems approach to Safety/Mission

Assurance falls short of supporting

NASA’s commitment of Safety First.”

7

Facing the Challenge… In June of 2007, APL still had over 60 open items with NASA from the audits held 3-

5 years prior.

While many topics had been addressed, there had been little follow-through with the sponsor, and little organization of responses to close the items.

The new team hired into APL was tasked with resolving the open issues and ensuring that APL’s Mission Assurance activities were sufficiently implemented and understood to ensure success upon NASA’s return assessment in 2008.

The mission objective from management was clear:

“APL’s Safety and Mission Assurance

activities must be as robust, internalized,

standardized, and exceptional as our

design, engineering, and science.”

8

Changing the Paradigm… Management has to be committed to supporting

Mission Assurance improvement initiatives.

Management includes Program Management, Engineering, Fabrication, Integration and Test, Facilities, Operations, Business Services, Human Resources, and IT.

In most places, Mission Assurance would not exist without everyone else. “They” make sure “we” have jobs. Treat them as your customers.

Your harshest critics are often those who have NEVER tried to play by the rules, and those who have ALWAYS try to play by the rules. Listen to both.

The most critical managers to reach and the most important staff to partner with are the ones in the trenches trying to get the work done. Partner with them whenever possible.

Be willing to do whatever it takes to make Mission Assurance successful. Be known as the team willing to go the extra mile. Roll up your sleeves and help.

9

Preparing for success at your location…

1. Start from where you are.

2. Clean up the past.

3. Set management expectations appropriately.

4. Approach the audit as a project and run it accordingly.

5. Identify your greatest risk areas and place more emphasis on fixing them.

6. Define a team of subject matter experts who will

be critical during the audit.

7. Partner with the folks you need information from

to help them prepare.

8. Meet regularly with your core implementation

team to get status updates and give deadlines.

10

Preparing for success at your location…

9. Take the audit as an opportunity to get facilities involved with extra cleaning,

pick-up, facility fixes, etc.

10. Throw stuff out.

11. Walk your facilities with your Safety Officers and Managers.

12. Hold All-Hands meetings to explain what NCAS is and what they will be doing.

13. Hold special sessions to educate staff on how the audit will be conducted based

on how involved they will be.

14. Make certain no major system upgrades are

planned close to the time of your audit.

15. Hold your “No’s”.

16. Encourage, reward, cheerlead, and smile.

11

Preparing for success with NCAS…

1. Believe the NCAS team when they say they

are coming to help and partner.

2. Engage the NCAS team as early in the process

as possible.

3. Ask NCAS questions about what tools will help

you prepare for the audit.

4. Ask for NCAS assistance in areas where you

need expertise.

5. Communicate openly with NCAS about what

programs or activities will be in-scope for the audit.

6. Agree on the standards that will be used to conduct the audit (AS9100, ISO).

7. Ask for special visits to close out old findings.

12

Operating during the Audit…

1. Set up a “Mission Control” area for

the audit team to congregate in.

2. Have your IT Department preload laptops

with access to the tools needed to

retrieve QMS documents and records.

3. Have copies of all contracts, SOWs,

and related documents in the room.

4. Schedule your staff in blocks of time and ask them to come to you for the audit.

5. Create and distribute a contact list with staff cell phone numbers.

6. Prepare management to be available at the end of each day for the debrief.

7. Have facility owners meet you to walk the auditors through each area.

13

Taking steps after the audit…

1. Before the auditors leave, schedule a

date for them to return and review your progress.

2. Enter findings in your Corrective Action system.

3. Meet with your management team to review

all findings and assign proper personnel and

levels of severity.

4. Track your progress toward completion weekly with notification of status to

senior management.

5. Contact NCAS is you are unclear about an item after they have left rather than

guessing about what they meant.

6. Don’t confuse corrective and preventive actions. Fix the problem found, then

create a separate preventive action if there are other steps to take.

14

Celebrating success… During the 2008 NCAS Assessment, APL received only 8 non-compliances, 21

observations, and 7 positive observations, all but four of which have already been implemented and found effective.

Report findings included comments such as:

During this audit, APL demonstrated that it is a model for organizations required to redefine and transform their safety, quality, and mission assurance systems....

It is clear...that APL has fully embraced the need for a robust quality management system and has put the necessary staff and resources in place to support their commitment to continuous improvement.

...more positive observations were cited during this APL audit than during any supplier audit in recent memory. These systemic improvements, are a sign of the laboratory’s commitment to excellence.