Engineering Engineering
Connections/ReflectionsConnections/Reflections
OpportunitiesOpportunities
Kimberly Gavaletz, Vice PresidentKimberly Gavaletz, Vice PresidentLockheed MartinLockheed Martin
Corporate Internal AuditCorporate Internal AuditNovember 2004November 2004
GSFC – Dec 2004.ppt 2
AgendaAgenda
IntroductionsIntroductions
Connections – Engineering and AuditConnections – Engineering and Audit
Reflections - Lessons LearnedReflections - Lessons Learned
Opportunities Opportunities
GSFC – Dec 2004.ppt 3
The People of Lockheed MartinThe People of Lockheed Martin
130,000 Employees130,000 Employees 55,000 Scientists and Engineers55,000 Scientists and Engineers 30,000 Software and Systems Engineers30,000 Software and Systems Engineers 6 CMMI Level-5 and Level-4 Companies6 CMMI Level-5 and Level-4 Companies Operations in 45 States and 56 CountriesOperations in 45 States and 56 Countries
We Never Forget Who We’re Working For ™We Never Forget Who We’re Working For ™We Never Forget Who We’re Working For ™We Never Forget Who We’re Working For ™
GSFC – Dec 2004.ppt 4
ResponsibilityResponsibility
ProduceProduce
TechnicalTechnical
ResultsResults
ProvideProvide
““Positive”Positive”
Financial Financial ReturnsReturns
GSFC – Dec 2004.ppt 5
ObjectivesObjectives
RisksRisks
ControlsControls
MonitoringMonitoring
AssessmentAssessment
The ChallengeThe Challenge
Mission Success Mission Success
EmployeesEmployees
ChangesChanges
Customer RelationshipsCustomer Relationships
Reputation Reputation
Information SecurityInformation Security
Compliance with LawsCompliance with Laws
Areas of Business Risk Areas of Business Risk (examples)(examples)
GSFC – Dec 2004.ppt 6
Connections - Connections - Engineering SourcesEngineering Sources
LM21 (Lean and Six Sigma Initiatives)LM21 (Lean and Six Sigma Initiatives)
Program Management Council, EV Council, Program Management Council, EV Council,
Engineering Process Improvement Council…Engineering Process Improvement Council…
Program or Company: Product Assurance, Quality, Program or Company: Product Assurance, Quality, Process Integrity OrganizationsProcess Integrity Organizations
Independent External Assessment and Certification Independent External Assessment and Certification Functions (ISO, SEI, EV, VPP, Consultants)Functions (ISO, SEI, EV, VPP, Consultants)
Internal Processes - ICE, IBR, PAR, NAR, SARInternal Processes - ICE, IBR, PAR, NAR, SAR
Audit (Observed Areas of Excellence, Compliance, Audit (Observed Areas of Excellence, Compliance, Programs, I/T & Advisory)Programs, I/T & Advisory)
GSFC – Dec 2004.ppt 7
Internal Audit’s Responsibility Internal Audit’s Responsibility
EvaluateEvaluate Risk ManagementRisk Management Internal ControlsInternal Controls GovernanceGovernance
Provide Early WarningProvide Early Warning
Proactive SupportProactive Support Transfer Best PracticesTransfer Best Practices Improve PerformanceImprove Performance
PROTECTPROTECTIMPROVEIMPROVE
GSFC – Dec 2004.ppt 8
Strategy & Strategy & PlanningPlanning
Shelly PaupShelly Paup
Audit OperationsAudit Operations
Brad OwensBrad Owens
Audit ServicesAudit Services
Reggie CombsReggie Combs
Corporate Internal AuditCorporate Internal Audit
Kimberly GavaletzKimberly Gavaletz
Audit Plan Completion
Governance Execution Information Technology
Advisory Services
BLDP
Tools
Audit Plan
Audit Council
Leverage ResourcesLeverage ResourcesAcross LMAcross LM
Optimize AuditOptimize AuditEngagementsEngagements
Enhance QualityEnhance Quality
Audit & Ethics Audit & Ethics CommitteeCommittee Business Area
Points of Contact• Corporate Kimberly Gavaletz• Space Systems Brad Owens• IS&S Brad Owens• Aeronautics Shelly Paup• Electronic Systems Reggie Combs• I&TS Shelly Paup
President President and CEOand CEO
GSFC – Dec 2004.ppt 9
Corporate Internal AuditCorporate Internal Audit
PalmdalePalmdalePalmdalePalmdale
AlbuquerqueAlbuquerque(DOE)(DOE)
AlbuquerqueAlbuquerque(DOE)(DOE)
DenverDenverDenverDenverSunnyvaleSunnyvaleSunnyvaleSunnyvale
MariettaMariettaMariettaMarietta
Valley ForgeValley ForgeValley ForgeValley Forge
ChelmsfordChelmsfordChelmsfordChelmsford
BethesdaBethesdaBethesdaBethesda
OrlandoOrlandoOrlandoOrlando
Ft. WorthFt. WorthFt. WorthFt. Worth
ScottsdaleScottsdaleScottsdaleScottsdale
Personnel LocationsPersonnel LocationsPersonnel LocationsPersonnel Locations
Operations ConceptOperations Concept
Personnel Reside in the Field… Personnel Reside in the Field…
Projects Staffed Based on SkillsProjects Staffed Based on Skills
Standardized Audit Program Standardized Audit Program
(Tailored as Needed)(Tailored as Needed)
Travel to Location for Fieldwork... Travel to Location for Fieldwork...
Staff Profile Staff Profile
100 %100 % Bachelor DegreeBachelor Degree 26 %26 % Masters DegreeMasters Degree 14 Yrs Avg. Business Experience14 Yrs Avg. Business Experience 6 Yrs Avg. Internal Audit Experience6 Yrs Avg. Internal Audit Experience 48 %48 % Certified -- 25 Different CertificationsCertified -- 25 Different Certifications
GSFC – Dec 2004.ppt 10
PlanningPlanningPlanningPlanning
ResolutionResolution
EngagementEngagement
• Closure ProcessClosure Process
Mission Success – Mission Success – Audit Plan ExecutionAudit Plan Execution
• Risk Assessment Risk Assessment
• CommunicationsCommunications
GSFC – Dec 2004.ppt 11
Audit Plan Coverage - ExampleAudit Plan Coverage - Example
5 Business Areas5 Business Areas
1500+ Programs / 1500+ Programs /
Contracts Over $5M Contracts Over $5M
38 Businesses38 Businesses
Internal Controls & Financial Internal Controls & Financial
International Compliance International Compliance
Programs Execution AuditsPrograms Execution Audits
IT Security / Controls / Disaster IT Security / Controls / Disaster RecoveryRecovery
Mgmt. Requests, Process Mgmt. Requests, Process Assessments & Pre-Implementation Assessments & Pre-Implementation Reviews Reviews
Audit UniverseAudit Universe
Audit CoverageAudit Coverage
GSFC – Dec 2004.ppt 12
AgendaAgenda
IntroductionsIntroductions
Connections – Engineering and AuditConnections – Engineering and Audit
Reflections - Lessons LearnedReflections - Lessons Learned
Opportunities Opportunities
GSFC – Dec 2004.ppt 13
Program Execution AuditsProgram Execution Audits
Assessing Assessing Effectiveness of Effectiveness of Program Controls In: Program Controls In: Program PlanningProgram Planning Risk ManagementRisk Management Program Perf. Mgmt.Program Perf. Mgmt. Systems EngineeringSystems Engineering Software/Hardware Dev.Software/Hardware Dev. Production and Material Production and Material
OperationsOperations Subcontract Mgmt.Subcontract Mgmt. Program Status Program Status
CommunicationsCommunications Customer SatisfactionCustomer Satisfaction
Business Self-Assessments Business Self-Assessments Evaluate:Evaluate:
Key Business Processes are Key Business Processes are Effective & Measured to Effective & Measured to Standards of ExcellenceStandards of Excellence
Early Warning Systems in Early Warning Systems in PlacePlace
Continuous Improvement Continuous Improvement Plans in Place & MonitoredPlans in Place & Monitored
Lessons Learned & Best Lessons Learned & Best Practices Incorporated Into Practices Incorporated Into Key ProcessesKey Processes
GSFC – Dec 2004.ppt 14
Lessons Learned Lessons Learned (Issue Examples )(Issue Examples )
EVMSEVMS
• Baseline Not in Place and/or MaintainedBaseline Not in Place and/or Maintained• Techniques Not UtilizedTechniques Not Utilized• Cost & Schedule Not IntegratedCost & Schedule Not Integrated• Not Fully Implemented (Lack of Mgt Support)Not Fully Implemented (Lack of Mgt Support)• Training, Knowledge of BenefitsTraining, Knowledge of Benefits
Resulting Cost Growth “Surprises” Due to Resulting Cost Growth “Surprises” Due to Inability to Forecast Performance & at Inability to Forecast Performance & at
Completion CostsCompletion Costs
Subcontract ManagementSubcontract Management
• S/C Plan Not in PlaceS/C Plan Not in Place• Failure to Meet Tech Req. Failure to Meet Tech Req. • S/C Qualification ProcessS/C Qualification Process• Parts Obsolescence Not AddressedParts Obsolescence Not Addressed
Resulting Delivery Issues,Resulting Delivery Issues, Stop WorkStop Work
EACs/Financial ReportingEACs/Financial Reporting
• Comprehensive EACs Not Performed PeriodicallyComprehensive EACs Not Performed Periodically• Costs Offset by Future Revenue Not Officially Agreed to By CustomerCosts Offset by Future Revenue Not Officially Agreed to By Customer• Risks Not Covered in Contract Status Reviews Risks Not Covered in Contract Status Reviews
Systems EngineeringSystems Engineering
• Contracts Lack Sufficient Definition of Customer Requirements & Acceptance CriteriaContracts Lack Sufficient Definition of Customer Requirements & Acceptance Criteria• Program Plans Not in PlaceProgram Plans Not in Place• Change Control IssuesChange Control Issues• Drawing Changes Not Completed TimelyDrawing Changes Not Completed Timely
GSFC – Dec 2004.ppt 15
Lessons Learned Lessons Learned (Issue Examples)(Issue Examples) Risk Management & Future Risk Exposure
• Cost, Technical, Subcontract, Schedule Risk Cost, Technical, Subcontract, Schedule Risk Items Not CapturedItems Not Captured
• ““Culture” Doesn’t Exist for Risk Culture” Doesn’t Exist for Risk Identification & MitigationIdentification & Mitigation
• Lack of Mitigation Plans and Activities Lack of Mitigation Plans and Activities
Resulting In Cost ImpactsResulting In Cost Impacts
Program Management Process
• Inadequate Procedures to Define Inadequate Procedures to Define Process & Training IssuesProcess & Training Issues
• No Resource Allocation PlansNo Resource Allocation Plans• Lack of Authority for PMsLack of Authority for PMs• Critical Staffing ShortfallsCritical Staffing Shortfalls• Return to Green Plans Not in PlaceReturn to Green Plans Not in Place
Proposals and Program Planning
• Plans Not Carried Forward to Achieve Proposal ChallengesPlans Not Carried Forward to Achieve Proposal Challenges• Risks and Issues MinimizedRisks and Issues Minimized• Risks Not Carried Forward in Program ExecutionRisks Not Carried Forward in Program Execution• Program Plans Not Developed and/or Not Utilized Program Plans Not Developed and/or Not Utilized
GSFC – Dec 2004.ppt 16
IT Audit CoverageIT Audit Coverage – – Issue Examples Issue ExamplesNetwork and Internet Security
• Absence of approval or knowledge of the Absence of approval or knowledge of the total inventory of Network Connectionstotal inventory of Network Connections
• Unnecessary ports and services openUnnecessary ports and services open
• No IDS System or perceived need by No IDS System or perceived need by management for reviewing logs management for reviewing logs
• Unapproved firewall products in use Unapproved firewall products in use
• Lack of modem sweepsLack of modem sweeps
Disaster Recovery
• Non-existent or outdated Risk Assessment Non-existent or outdated Risk Assessment
• RA done w/o data owner or management RA done w/o data owner or management input/approvalinput/approval
• New systems brought on-line since the last New systems brought on-line since the last RA and not evaluated RA and not evaluated
• Disaster Recovery Plan is Disaster Recovery Plan is outdated/incompleteoutdated/incomplete
• Off-site storage requirements not Off-site storage requirements not consideredconsidered
Electronic Information Protection
Lack of understanding by employee's of Lack of understanding by employee's of what is sensitive and who has accesswhat is sensitive and who has access
Management commitment to safeguarding Management commitment to safeguarding sensitive informationsensitive information
Employee-managed file shares not Employee-managed file shares not configured properlyconfigured properly
Operating System Controls
Terminated employee/contractor accounts that Terminated employee/contractor accounts that still exist and are activestill exist and are active
Banner statements don’t comply with the policyBanner statements don’t comply with the policy
Anti-virus software not installed or out of dateAnti-virus software not installed or out of date
Systems not patched, not configured properly, Systems not patched, not configured properly, & critical system files not protected from & critical system files not protected from external or internal threatsexternal or internal threats
Sys Admins not adequately trained and/or Sys Admins not adequately trained and/or unfamiliar with policies/handbookunfamiliar with policies/handbook
GSFC – Dec 2004.ppt 17
Advisory ServicesAdvisory Services
Special Audits & Advisory ServicesSpecial Audits & Advisory Services Key InitiativesKey Initiatives Process Improvements & EffectivenessProcess Improvements & Effectiveness
Management RequestsManagement Requests
Ethics & Other Special InvestigationsEthics & Other Special Investigations
GSFC – Dec 2004.ppt 18
AgendaAgenda
IntroductionsIntroductions
Connections – Engineering and AuditConnections – Engineering and Audit
Reflections - Lessons LearnedReflections - Lessons Learned
Opportunities Opportunities
Resources
Evolution
GSFC – Dec 2004.ppt 19
Audit Resources Audit Resources
PUSH PUSH PULLPULL
““Waiting List”Waiting List” ““Waiting Line”Waiting Line”
Audit Council Subject Matter Experts (SME)
Technical Partners External Institutes/Forums
SupportingSupporting
INTERNALINTERNALAUDITAUDIT
• RotationalRotational
• Subject Matter ExpertsSubject Matter Experts
GSFC – Dec 2004.ppt 20
• Management RequestsManagement Requests
• Program Execution ReviewsProgram Execution Reviews • Ongoing Risk AssessmentOngoing Risk Assessment (Headlines & Metrics)(Headlines & Metrics)
Audit Program EnhancementsAudit Program Enhancements
Continuous Process ImprovementContinuous Process Improvement
Reactive - > Proactive - > PreventiveReactive - > Proactive - > Preventive
• EducationEducation
• Risk Indicators Risk Indicators
• SharingSharing
• “ “Keep It Closed”Keep It Closed”• Self-AssessmentSelf-Assessment
• Ethics InvestigationsEthics Investigations
• “ “Post Mortem” Post Mortem”
SupportSupport