engineeringconnections/reflections opportunities opportunities kimberly gavaletz, vice president...

Engineering Engineering

Connections/ReflectionsConnections/Reflections

OpportunitiesOpportunities

Kimberly Gavaletz, Vice PresidentKimberly Gavaletz, Vice PresidentLockheed MartinLockheed Martin

Corporate Internal AuditCorporate Internal AuditNovember 2004November 2004

GSFC – Dec 2004.ppt 2

AgendaAgenda

IntroductionsIntroductions

Connections – Engineering and AuditConnections – Engineering and Audit

Reflections - Lessons LearnedReflections - Lessons Learned

Opportunities Opportunities


The People of Lockheed MartinThe People of Lockheed Martin

130,000 Employees130,000 Employees 55,000 Scientists and Engineers55,000 Scientists and Engineers 30,000 Software and Systems Engineers30,000 Software and Systems Engineers 6 CMMI Level-5 and Level-4 Companies6 CMMI Level-5 and Level-4 Companies Operations in 45 States and 56 CountriesOperations in 45 States and 56 Countries

We Never Forget Who We’re Working For ™We Never Forget Who We’re Working For ™We Never Forget Who We’re Working For ™We Never Forget Who We’re Working For ™


ResponsibilityResponsibility

ProduceProduce

TechnicalTechnical

ResultsResults

ProvideProvide

““Positive”Positive”

Financial Financial ReturnsReturns


ObjectivesObjectives

RisksRisks

ControlsControls

MonitoringMonitoring

AssessmentAssessment

The ChallengeThe Challenge

Mission Success Mission Success

EmployeesEmployees

ChangesChanges

Customer RelationshipsCustomer Relationships

Reputation Reputation

Information SecurityInformation Security

Compliance with LawsCompliance with Laws

Areas of Business Risk Areas of Business Risk (examples)(examples)


Connections - Connections - Engineering SourcesEngineering Sources

LM21 (Lean and Six Sigma Initiatives)LM21 (Lean and Six Sigma Initiatives)

Program Management Council, EV Council, Program Management Council, EV Council,

Engineering Process Improvement Council…Engineering Process Improvement Council…

Program or Company: Product Assurance, Quality, Program or Company: Product Assurance, Quality, Process Integrity OrganizationsProcess Integrity Organizations

Independent External Assessment and Certification Independent External Assessment and Certification Functions (ISO, SEI, EV, VPP, Consultants)Functions (ISO, SEI, EV, VPP, Consultants)

Internal Processes - ICE, IBR, PAR, NAR, SARInternal Processes - ICE, IBR, PAR, NAR, SAR

Audit (Observed Areas of Excellence, Compliance, Audit (Observed Areas of Excellence, Compliance, Programs, I/T & Advisory)Programs, I/T & Advisory)


Internal Audit’s Responsibility Internal Audit’s Responsibility

EvaluateEvaluate Risk ManagementRisk Management Internal ControlsInternal Controls GovernanceGovernance

Provide Early WarningProvide Early Warning

Proactive SupportProactive Support Transfer Best PracticesTransfer Best Practices Improve PerformanceImprove Performance

PROTECTPROTECTIMPROVEIMPROVE


Strategy & Strategy & PlanningPlanning

Shelly PaupShelly Paup

Audit OperationsAudit Operations

Brad OwensBrad Owens

Audit ServicesAudit Services

Reggie CombsReggie Combs

Corporate Internal AuditCorporate Internal Audit

Kimberly GavaletzKimberly Gavaletz

Audit Plan Completion

Governance Execution Information Technology

Advisory Services

BLDP

Tools

Audit Plan

Audit Council

Leverage ResourcesLeverage ResourcesAcross LMAcross LM

Optimize AuditOptimize AuditEngagementsEngagements

Enhance QualityEnhance Quality

Audit & Ethics Audit & Ethics CommitteeCommittee Business Area

Points of Contact• Corporate Kimberly Gavaletz• Space Systems Brad Owens• IS&S Brad Owens• Aeronautics Shelly Paup• Electronic Systems Reggie Combs• I&TS Shelly Paup

President President and CEOand CEO


Corporate Internal AuditCorporate Internal Audit

PalmdalePalmdalePalmdalePalmdale

AlbuquerqueAlbuquerque(DOE)(DOE)

AlbuquerqueAlbuquerque(DOE)(DOE)

DenverDenverDenverDenverSunnyvaleSunnyvaleSunnyvaleSunnyvale

MariettaMariettaMariettaMarietta

Valley ForgeValley ForgeValley ForgeValley Forge

ChelmsfordChelmsfordChelmsfordChelmsford

BethesdaBethesdaBethesdaBethesda

OrlandoOrlandoOrlandoOrlando

Ft. WorthFt. WorthFt. WorthFt. Worth

ScottsdaleScottsdaleScottsdaleScottsdale

Personnel LocationsPersonnel LocationsPersonnel LocationsPersonnel Locations

Operations ConceptOperations Concept

Personnel Reside in the Field… Personnel Reside in the Field…

Projects Staffed Based on SkillsProjects Staffed Based on Skills

Standardized Audit Program Standardized Audit Program

(Tailored as Needed)(Tailored as Needed)

Travel to Location for Fieldwork... Travel to Location for Fieldwork...

Staff Profile Staff Profile

100 %100 % Bachelor DegreeBachelor Degree 26 %26 % Masters DegreeMasters Degree 14 Yrs Avg. Business Experience14 Yrs Avg. Business Experience 6 Yrs Avg. Internal Audit Experience6 Yrs Avg. Internal Audit Experience 48 %48 % Certified -- 25 Different CertificationsCertified -- 25 Different Certifications


PlanningPlanningPlanningPlanning

ResolutionResolution

EngagementEngagement

• Closure ProcessClosure Process

Mission Success – Mission Success – Audit Plan ExecutionAudit Plan Execution

• Risk Assessment Risk Assessment

• CommunicationsCommunications


Audit Plan Coverage - ExampleAudit Plan Coverage - Example

5 Business Areas5 Business Areas

1500+ Programs / 1500+ Programs /

Contracts Over $5M Contracts Over $5M

38 Businesses38 Businesses

Internal Controls & Financial Internal Controls & Financial

International Compliance International Compliance

Programs Execution AuditsPrograms Execution Audits

IT Security / Controls / Disaster IT Security / Controls / Disaster RecoveryRecovery

Mgmt. Requests, Process Mgmt. Requests, Process Assessments & Pre-Implementation Assessments & Pre-Implementation Reviews Reviews

Audit UniverseAudit Universe

Audit CoverageAudit Coverage


AgendaAgenda






Program Execution AuditsProgram Execution Audits

Assessing Assessing Effectiveness of Effectiveness of Program Controls In: Program Controls In: Program PlanningProgram Planning Risk ManagementRisk Management Program Perf. Mgmt.Program Perf. Mgmt. Systems EngineeringSystems Engineering Software/Hardware Dev.Software/Hardware Dev. Production and Material Production and Material

OperationsOperations Subcontract Mgmt.Subcontract Mgmt. Program Status Program Status

CommunicationsCommunications Customer SatisfactionCustomer Satisfaction

Business Self-Assessments Business Self-Assessments Evaluate:Evaluate:

Key Business Processes are Key Business Processes are Effective & Measured to Effective & Measured to Standards of ExcellenceStandards of Excellence

Early Warning Systems in Early Warning Systems in PlacePlace

Continuous Improvement Continuous Improvement Plans in Place & MonitoredPlans in Place & Monitored

Lessons Learned & Best Lessons Learned & Best Practices Incorporated Into Practices Incorporated Into Key ProcessesKey Processes


Lessons Learned Lessons Learned (Issue Examples )(Issue Examples )

EVMSEVMS

• Baseline Not in Place and/or MaintainedBaseline Not in Place and/or Maintained• Techniques Not UtilizedTechniques Not Utilized• Cost & Schedule Not IntegratedCost & Schedule Not Integrated• Not Fully Implemented (Lack of Mgt Support)Not Fully Implemented (Lack of Mgt Support)• Training, Knowledge of BenefitsTraining, Knowledge of Benefits

Resulting Cost Growth “Surprises” Due to Resulting Cost Growth “Surprises” Due to Inability to Forecast Performance & at Inability to Forecast Performance & at

Completion CostsCompletion Costs

Subcontract ManagementSubcontract Management

• S/C Plan Not in PlaceS/C Plan Not in Place• Failure to Meet Tech Req. Failure to Meet Tech Req. • S/C Qualification ProcessS/C Qualification Process• Parts Obsolescence Not AddressedParts Obsolescence Not Addressed

Resulting Delivery Issues,Resulting Delivery Issues, Stop WorkStop Work

EACs/Financial ReportingEACs/Financial Reporting

• Comprehensive EACs Not Performed PeriodicallyComprehensive EACs Not Performed Periodically• Costs Offset by Future Revenue Not Officially Agreed to By CustomerCosts Offset by Future Revenue Not Officially Agreed to By Customer• Risks Not Covered in Contract Status Reviews Risks Not Covered in Contract Status Reviews

Systems EngineeringSystems Engineering

• Contracts Lack Sufficient Definition of Customer Requirements & Acceptance CriteriaContracts Lack Sufficient Definition of Customer Requirements & Acceptance Criteria• Program Plans Not in PlaceProgram Plans Not in Place• Change Control IssuesChange Control Issues• Drawing Changes Not Completed TimelyDrawing Changes Not Completed Timely


Lessons Learned Lessons Learned (Issue Examples)(Issue Examples) Risk Management & Future Risk Exposure

• Cost, Technical, Subcontract, Schedule Risk Cost, Technical, Subcontract, Schedule Risk Items Not CapturedItems Not Captured

• ““Culture” Doesn’t Exist for Risk Culture” Doesn’t Exist for Risk Identification & MitigationIdentification & Mitigation

• Lack of Mitigation Plans and Activities Lack of Mitigation Plans and Activities

Resulting In Cost ImpactsResulting In Cost Impacts

Program Management Process

• Inadequate Procedures to Define Inadequate Procedures to Define Process & Training IssuesProcess & Training Issues

• No Resource Allocation PlansNo Resource Allocation Plans• Lack of Authority for PMsLack of Authority for PMs• Critical Staffing ShortfallsCritical Staffing Shortfalls• Return to Green Plans Not in PlaceReturn to Green Plans Not in Place

Proposals and Program Planning

• Plans Not Carried Forward to Achieve Proposal ChallengesPlans Not Carried Forward to Achieve Proposal Challenges• Risks and Issues MinimizedRisks and Issues Minimized• Risks Not Carried Forward in Program ExecutionRisks Not Carried Forward in Program Execution• Program Plans Not Developed and/or Not Utilized Program Plans Not Developed and/or Not Utilized


IT Audit CoverageIT Audit Coverage – – Issue Examples Issue ExamplesNetwork and Internet Security

• Absence of approval or knowledge of the Absence of approval or knowledge of the total inventory of Network Connectionstotal inventory of Network Connections

• Unnecessary ports and services openUnnecessary ports and services open

• No IDS System or perceived need by No IDS System or perceived need by management for reviewing logs management for reviewing logs

• Unapproved firewall products in use Unapproved firewall products in use

• Lack of modem sweepsLack of modem sweeps

Disaster Recovery

• Non-existent or outdated Risk Assessment Non-existent or outdated Risk Assessment

• RA done w/o data owner or management RA done w/o data owner or management input/approvalinput/approval

• New systems brought on-line since the last New systems brought on-line since the last RA and not evaluated RA and not evaluated

• Disaster Recovery Plan is Disaster Recovery Plan is outdated/incompleteoutdated/incomplete

• Off-site storage requirements not Off-site storage requirements not consideredconsidered

Electronic Information Protection

Lack of understanding by employee's of Lack of understanding by employee's of what is sensitive and who has accesswhat is sensitive and who has access

Management commitment to safeguarding Management commitment to safeguarding sensitive informationsensitive information

Employee-managed file shares not Employee-managed file shares not configured properlyconfigured properly

Operating System Controls

Terminated employee/contractor accounts that Terminated employee/contractor accounts that still exist and are activestill exist and are active

Banner statements don’t comply with the policyBanner statements don’t comply with the policy

Anti-virus software not installed or out of dateAnti-virus software not installed or out of date

Systems not patched, not configured properly, Systems not patched, not configured properly, & critical system files not protected from & critical system files not protected from external or internal threatsexternal or internal threats

Sys Admins not adequately trained and/or Sys Admins not adequately trained and/or unfamiliar with policies/handbookunfamiliar with policies/handbook


Advisory ServicesAdvisory Services

Special Audits & Advisory ServicesSpecial Audits & Advisory Services Key InitiativesKey Initiatives Process Improvements & EffectivenessProcess Improvements & Effectiveness

Management RequestsManagement Requests

Ethics & Other Special InvestigationsEthics & Other Special Investigations


AgendaAgenda





Resources

Evolution


Audit Resources Audit Resources

PUSH PUSH PULLPULL

““Waiting List”Waiting List” ““Waiting Line”Waiting Line”

Audit Council Subject Matter Experts (SME)

Technical Partners External Institutes/Forums

SupportingSupporting

INTERNALINTERNALAUDITAUDIT

• RotationalRotational

• Subject Matter ExpertsSubject Matter Experts


• Management RequestsManagement Requests

• Program Execution ReviewsProgram Execution Reviews • Ongoing Risk AssessmentOngoing Risk Assessment (Headlines & Metrics)(Headlines & Metrics)

Audit Program EnhancementsAudit Program Enhancements

Continuous Process ImprovementContinuous Process Improvement

Reactive - > Proactive - > PreventiveReactive - > Proactive - > Preventive

• EducationEducation

• Risk Indicators Risk Indicators

• SharingSharing

• “ “Keep It Closed”Keep It Closed”• Self-AssessmentSelf-Assessment

• Ethics InvestigationsEthics Investigations

• “ “Post Mortem” Post Mortem”

SupportSupport

engineeringconnections/reflections opportunities opportunities kimberly gavaletz, vice president...

Documents