abstract... · web view2019-10-02 · enhancing the usaf corrosion survey process via advanced...

ENHANCING THE USAF CORROSION SURVEY PROCESS VIA ADVANCED ANALYTICS INTEGRATION

Nicole de Vries, M.B.A, M.S.A., PavCon, LLC

Casey Jones, AFLCMC/EZP-AFCPCO

Security Review #78ABW-2019-0077 (19-05313)

Keywords: Big Data Analytics, Aerospace Ground Equipment, Machine Learning

ABSTRACT

The Air Force Life Cycle Management Center, Product Support Engineering Division (AFLCMC/EZP) sup-ports implementing technologies to the Air Force. An office within AFLCMC/EZP, the Air Force Corrosion Prevention and Control Office (AFCPCO) regularly conducts surveys of aircraft and aerospace ground equipment (AGE). In the past, these surveys have been more reactive than proactive. Representatives would interview Subject Matter Experts (SMEs) and go into the field to survey a base’s Corrosion Preven-tion and Control Program (CPCP). By integrating big data analytics, the surveys now take a more proac-tive approach to exploring the key drivers of corrosion. In addition to seeking top drivers from SMEs from Air Force (AF) Program Offices, Depots, and the field, the AFCPCO now uses several machine learning and statistical techniques to uncover trends in data before conducting surveys. As new issues are uncov-ered in the field, additional searches and calculations are performed to evolve and refine the survey proto-col via a feedback loop. This flexible, recursive process leads to more focused and directed surveys and incorporates data collection during surveys to validate findings. This also serves to calculate corrosion costs and benefits through integration of new corrosion technologies and more efficient CPCPs. From data to information and knowledge to application, this paper concerns the evolution, setbacks, and suc-cesses of the AFCPCO survey process.

INTRODUCTION

Data has become a pervasive part of life, creating opportunity to leverage data to make better use of resources. For example, “The exponential growth in the amount of biological data means that revolutionary measures are needed for data management, analysis and accessibility” (Howe). The same statement could potentially apply to the Aerospace and Corrosion Field. The Air Force Corrosion Prevention and Control Office (AFCPCO) saw this opportunity and began a direct effort to invest in analysis by infusing data into the corrosion survey process.

It is a long-held theory that growth in technology is often proceeded by a growth in measurement. “Advancement in measurement is fundamental to technological progress and to scientific growth” (Astain). With well-established systems, many data sources and maintenance (Mx) databases, the Air Force (AF) has progressed measure-ments, leading to a wealth of potential data prime for actionable analysis.

1

Paper No. 2019-0000

2019 Department of De-fense – Allied Nations

Technical Corrosion Con-

Corrosion surveys in the past have produced valuable insights and recommendations but were driven primarily by qualitative investigation at operational bases and component repair shops. The new data-infused approach takes the best from the qualitative investigation and adds a quantitative layer to complement.

This new approach has been successfully tested in the field for one round of surveys and the body of this paper will explore each step of the infusion process, related analytical techniques, and resulting outcomes.

The data provided within this article is for sample purposes only and does not reflect actual AF data.

EXPERIMENTAL PROCEDURE

Brief Survey History

The AFCPCO conducts Major Command (MAJCOM) corrosion surveys every five years. Corrosion surveys are different from inspections, rather they serve to observe methods, share information and best practices, as well as make recommendations. The AFCPCO is in a unique position to observe activity in all locations of all airframes and take the best from each to help raise the entire fleet.

The Business Process Management Life Cycle

As seen in Figure 1, the standard steps in process management include: design, model, execute, monitor, and optimize. This iterative approach allows for agile adjustment and continued growth of the resulting framework.

2



Figure 1Business Process Life Cycle Corrosion Survey Approach

Design

In the design phase, qualitative and quantitative investigation are underway. Qualitative data comes from obser -vations one can make with their senses, things one can: touch, smell, see, hear, and taste. On the other hand, quantitative data are made with instruments and are measurable (numerical).

Qualitative investigation includes interviewing field Subject Matter Experts (SMEs), attending informative meetings like Corrosion Prevention Advisory Boards (CPABs), Corrosion Technical Interchange Meetings (CTIMs), Aircraft Structural Integrity Program (ASIP) reviews or others hosted by the central and various Program Offices (POs). For any given airframe, this detective work establishes known areas of concern. During the qualitative investiga-tion the team also researches potential sources of quantitative data. SMEs are asked, “Where are people enter -ing maintenance actions taken related to a weapon system?” This can yield various sources, including: field main-tenance databases (e.g., IMDS, G081, etc); requests for additional engineering support (107s & 202s); Depot, Contracting, or Original Equipment Manufacturers (OEMs).

The quantitative investigation expands on the qualitative investigation through the inclusion of structured data. This additional data is acquired from the sources discovered in qualitative investigation. Once in hand, an analyst discusses the data with SMEs to better understand what fields are relevant and then apply statistical modeling techniques to confirm known issues or identify additional corrosion drivers.

Cleaning, Classifying, and Strengthening Data with Advanced AnalyticsData entered into Mx systems can be subjective or qualitative in origin. Coding or data entry structures may also limit, or positively or negatively reinforce certain record entry behaviors that can skew data. During analysis this can be kept in mind and mitigated. One way to ensure the corrosion scope focuses on relevant records is through establishing an identification logic that relies on codes in addition to other indicators, such as language, to mark records of interest. This “Corrosion Net” can be cast or reused on various datasets to identify corrosion records. This document references the AFCPCO’s “Net”, which was developed from combining the logical nets of four other sources and initial work done a few years ago by the AFCPCO to identify corrosion language. From the records identified, various phrase parings (n-grams) found through key narrative fields were examined, and new keywords were identified in an iterative approach until the net was developed to its current state today. Sections of data not identified by the net can be periodically reviewed for false positives (items that are corrosion related but not identified by the net) to keep the net updated and as robust as possible. Periodic sections of identified records can also be reviewed to validate existing net logic. Machine Learning Classification is another way to use data science to strengthen data. For corrosion netted records, the analysis seeks to identify the type of corrosion action that took place (corrective, preventative, wash, etc). The AFCPCO developed a method that uses a trained data set to classify other new data into corrosion action categories. These categories are used to look at the rela-tionships between corrosion actions. For example, it analyzes to what extent inspections drive Mx or identifying outlying aircraft with low inspection but high Mx or vice versa.

Establishing Data LevelsThe netted and classified data is broken into geographic classifications so that it can be analyzed from the fleet-wide perspective, by MAJCOM, by Environmental Severity Index (ESI), by base, or per aircraft. Fleet wide break -outs yield overall trends. MAJCOM breakouts show the difference between Depot and field or other unit compar -isons. Analysis of the ESI can reveal how Mx behaviors shift over environmental severities;. base breakouts show the trend of an individual base; and aircraft breakouts show the activity for an individual tail. These break-

3



outs can be used in field to help direct the conversation while on a survey and used “on the fly” while in AF back shops or observing aircraft on the flight line or in Mx hangers.

Establishing KPIsOnce different levels have been established for the data, Key Performance Indicators (KPIs) can be overlain. Five primary KPIs are Number of Occurrences, Labor Hours, Corrosion Action Taken, Cost, and Non-Mission Capabil -ity Impact. All these items can be aggregated over time per item or aircraft to explore visual and statistical trends. The AFCPCO is working to build in-house capabilities for all KPIs and currently has the ability to view of Number of Occurrences, Labor Man Hours, and Corrosion Action Taken. Initially, Visual Methods are used to explore KPI data. This includes line charts, pie charts, heatmaps, etc. over the various geographic areas and action taken cat-egories.

Figure 2 includes a Labor Man Hour trend over time by MAJCOM. For example, the Air Mobility Command (AMC) is showing a downward trend over time in Labor Man Hours, while Air Force Materiel Command (MTC) is showing an increase. This shift can be explored with SMEs to help understand the context of corrosion for the fleet. The spikes in each line can also be explored and in this case correlate back to specific Time Compliance Technical Order (TCTO) changes.

Figure 2Example KPI Over Time by MAJCOM

4



Figure 3Example KPI Over Time by MAJCOM

The line portion of Figure 3 demonstrates the relationship between occurrences and Labor Man Hours. The dot-ted trend lines show an increase in number of corrosion related records and Labor Man Hours at this base. Gaps between the lines are key to note, such as the gap in 2009, where Labor Man Hours spiked considerably more than number of records. The complementing heatmap below can be used to observe the activity of a KPI over time. It appears that the fourth quarter (Q4) reveals increased activity for this base. This theory could easily be further statistically explored with regression or analysis of variance (ANOVA) Models.

Figure 4 includes an example of exploring Corrosion Action Taken across MAJCOM over time. It is interesting to note linear time trends, as well as how much each action represents the whole. In this case, Continental United States (CONUS) field units incurred more paint and treatment action, while Depot and Pacific areas incurred more remove and replace actions.

5



Figure 4

Action Taken Over Time

Model (via Advanced Analytics)

This basic context and overview of data is a solid foundation for statistical methods and modeling. One way to identify item and airframe corrosion drivers is by generating a box plot and exploring quantiles via analytics tools such as R or Tableau. KPIs for each item or aircraft can be plotted, and the distribution can be explored to deter-mine outliers. As with all life, resources are limited and a cut point was established to explore items above 75 th

percentile. Cut points were calculated for each geographic grouping and if an item or aircraft appears in the top 75th percentile for any grouping, the item is flagged as a potential outlier or high driver. Use of this method “nets” high drivers. This cut list is then reviewed with SMEs. A box plot example is included here:

Box Plot by First 2 WUC Box Plot by First 3 WUC Box Plot by First 4 WUCAMC 3,940 lbr hrs – MTC 1,012 lbr hrs , etc AMC 949 lbr hrs - MTC 336 lbrhrs, etc AMC 276 lbr hrs – MTC 129 lbr hrs, etc

6



Figure 5Work Unit Code (WUC) Box Plot by Numerical Groupings

Figure 5 is an example of using box plots and Work Unit Codes (WUCs) to identify corrosion drivers of potential interest. Most WUCs represent a specific part of the aircraft. WUCs are five digits, and each digit represents a greater specificity over system, sub-system, and Line Replaceable Unit (LRU). Maintainers enter a WUC during Mx data entry while they are working on an aircraft. At times, this data can be misleading as WUCs entered dur -ing data entry can be lumped under specific codes or entered at the system or subsystem level. By setting cut points across geographic locations, but also by grouping on this code, the analysis can identify both specific parts and systems of interest. Items of interest are grouped by this code.

Histograms are another helpful method for identifying or validating cut points. By sorting aircraft or items into bins of activity, and looking at the shape, decisions can be made about the most logical place to set the cut threshold depending on situational resources and nature of the population. This agile approach allows a user to address a particular percentage of the outlying population as desired. Along with box plots, understanding the distribution of data reveals or validates analytic items of interest.

After review with the SMEs, items that remain on the list of interest go through a second round of investigation. The relevant WUCs are identified for each area via AF Technical Orders. Corrosion-related records are filtered for selected WUCs and examined. Natural Language Processing methods are applied to narrative fields and used to score phrases and explore the most relevant n-grams found. From this, the language used to describe a particular area can be determined. For example, a formal name for an area of interest may be “ground refuel con-trol panel”’ however Mx data records may reference this component by “GRCP,” “Gnd fuel pnl,” and many other variations. Once a word list has been established, the net is recast for the item of interest, to include both WUCs and “word triggers.” This data is then summarized for the specific issue or area of interest, such as the GRCP. Figure 6 shows an example of the mini dashboard prepared for each interest item (system, sub-system, or LRU) of interest.

7



Figure 6Items of Interest Dashboard Example: Ground Refuel Control Panel

The Item list has now been reviewed by SME, enhanced with more data, and is now developed into a rubric to help guide the corrosion survey while the team visits each base. For the first iteration, a PDF form was devel -oped, and each area of interest was outlined. Standard items for capture, as identified by SMEs, are noted. An example of the data collection sheet can be seen in Figure 7. After collecting data per tail, data was extracted from the PDF to Microsoft Excel and used to validate or add depth to the findings. From this data collection de-finitive conclusions can be established, such as X tails showed signs of standing water in the gear pod, etc.

8



….Figure 7

Example Data Collection FormExecute

The rubric is then executed in the field. Embedded data summaries in the rubric can help with on-the-fly informa -tion observed about each hot spot area. Additional summaries, like the tail summary in Figure 8, supplement the rubric and can help proritize what tails to survey. The box plot in Figure 8 demonstrates if any tails are considered outliers in terms of distribution of the selected KPI. The heat map is useful to see the areas and heat of activity for a specific tail.

Figure 8Box Plot of Tails, and High Item Drivers

Monitor and Optimize

During execution, feedback is taken and then incorporated into the form. The steps below outline the Monitor and Optimize steps of the iterative business process life cycle.

Monitor Optimize

Several methods of data collection were tried, span-ning from paper to tablet. Selecting tiny drop downs on a tablet can be time consuming and cumbersome; paper methods must be keyed in later. Each way has benefits and pain points.

The survey teams are using both routes and develop future rubrics to be both print or tablet friendly.

Items were first listed alphabetically. Users had to It became quickly apparent that the items should be in

9



jump over the form to enter information. order of the survey to ease the collection of data and review of consistent areas.

Several items of interest could not be reviewed at all while on a flight line.

Items were moved to an appendix of the rubric where no data is collected, but the breakout is available for reference.

Other areas of interest were identified. New areas added as needed.

Forward Thinking

An agile and living business process, future investment and developments are underway in the data infusion process, including:

Each visualization in this document can be thought of as a widget or a piece of the whole story. The AFCPCO is currently organizing these widgets into a series of interactive, drillable dashboards.

There are also many on-going data efforts in the Condition Based Maintenance Plus (CBM+) area, partic-ularly predictive corrosion efforts. There is potential for cross-over as projects advance.

The current corrosion net could be used as a training set for other corrosion nets and could switch to a machine learning approach.

Areas of interest undergo a second round of data investigation involving language. It would be ideal to in-tegrate the language before identifying high drivers. Discussion is underway on correcting the item of in -terest by observing the WUC and key trigger words in the narrative fields.

Analytical techniques like market basket analysis or Natural Language Processing could be used to better classify the noun of a record.

A custom AF dictionary for language processing could also be useful. Current stemming and lemmatiza-tion techniques are limited by language differences and custom abbreviations for the industry.

CONCLUSIONSThe AFCPCO Corrosion Survey Process has been infused with data. Simple statistical methods can be applied to enhance and explore data. In the corrosion survey process, the survey team successfully utilizes: visualization tactics, machine learning classifiers, Natural Language Processing, corrosion nets, establishment of KPIs, and statistical methods for exploring distributions (box plots, histograms). The resulting data collection rubric devel-oped from this process makes for better use of time in the field during the survey and helps add shape to data col -lection. Advanced analytics are allowing predictions to be made for the survey and has proven valuable for guid -ing survey activity. It has been integrated through the entire corrosion survey process; before, during and after. The outcome of this enhancement brings data to the hands of its owners, allowing for better trend identification, as well as ability to see impact and return on investment.

REFERENCES

1. Howe, Doug, et al. The future of biocuration. Nature, 2008. 455, 47-50: https://www.nature.com/articles/455047a

10



https://www.nature.com/articles/455047a

https://www.nature.com/articles/455047a

2. Astain, A.V. National Bureau of Standards Annual Report. United States Department of Commerce, 1957. 154, 1: https://archive.org/details/annualreport1957223nati/page/n1

11



https://archive.org/details/annualreport1957223nati/page/n1

abstract... · web view2019-10-02 · enhancing the usaf corrosion survey process via advanced...

Documents