1nasa customer survey 07/31/98 07:29 pm nasa customer survey summary report prepared by science...

1NASA Customer Survey 07/31/98 07:29 PM

NASA CUSTOMER SURVEYSUMMARY REPORT

Prepared by Science Applications International Corporation (SAIC)

Arlington, Virginia

August 1998


Organization of Report

Population

Approach

Introductory Questions

ASTT Program Three Pillars and Ten Goals

Level of Satisfaction with NASA’s ASTT Program• Facilities and Services

• Technology Transfer

• Program Planning

Level of Satisfaction with NASA

Resource Management

Effectiveness in Meeting National Needs

Closing Questions – Executive Level Only


Survey Population Data

Executive Mid-Level Manager

Survey Population 101 72

Customer Groups

Direct Industry 63 65

Indirect Industry 38 7

Benchmark Companies 50 59

Aviation Advisory Committee Members 11 0

Universities 18 3

Aviation Associations 8 3


Industry Groups

DIRECT INDUSTRY • Large Civil Transport Manufacturers• High Performance Aircraft Manufacturers• Aircraft Engine Manufacturers• Rotorcraft Manufacturers• GA/Commercial/Sport/Business/Other Aircraft Manufacturers• Launch Vehicle Manufacturers• Launch Vehicle Engine/Propellant Manufacturers• Aircraft Instrument/Avionics Manufacturers• Specialized Aerospace Services• Department of Defense Entities• Other Federal Agencies

INDIRECT INDUSTRY • Aviation Metal/Metal Component Manufacturers

• Non-Metallic Aviation Materials Manufacturers

• Air Carriers

• Satellite Manufacturers and Other Launch Service Customers

• Universities (Major ASTT Grantees)

• Aviation Associations

• Aerospace Analysts and Advocacy Groups


Survey Approach

Quantitative Questions

Statistical Analysis (Mean, Standard Deviation)

Ratings from 1 to 10• 1 to 4 - low• 5 to 7 - medium• 8 to 10 - high

Qualitative Questions (open-ended)

Followed several Quantitative Questions

YES/NO Questions

Sorting Items into Categories

Comparisons with 92 and 95 Surveys


Survey Approach Summary

Executive Mid-Level Manager

Method of Interview

Personal Visit 86 14

Telephone 15 58

Average Interview Duration (Minutes) 56 41

Level of Understanding (1 to 10 scale) 7.7 7.9

(Interviewer Assessment)

Level of Interest (1 to 10 scale) 8.2 7.2(Interviewer Assessment)

ASTT Mission Area to Which Answers Apply

Aeronautics 63 47

Space 14 15

Aeronautics and Space 24 10


Overall Ratings by Customer Group

Category Mean Rank

Aeronautics Advisory Committee 7.02 1

Universities 6.85 2

Government 6.69 3

Associations/Publications 6.58 4

Mid-Level Managers 6.55 5

Benchmark Companies 6.53 6

Direct Industry 6.50 7

Executives 6.41 8

Indirect Industry 6.36 9

Overall 6.47


1. What is your overall level of interest in the ASTT Program?

• 96 percent of the total respondents said they had a high or medium level of interest in the ASTT Program. This level was up from the 1995 survey when the total was 92 percent.

• 79 percent of the total said they had a high level of interest. This compares to 75 percent in the 1995 survey.

• NASA’s Aviation Advisory Committee (AAC) had the highest level of interest at 100 percent in the high category.

• University interest dropped from 100 percent in 1995 to 90 percent in 1998.

• The group that had the lowest level of interest in ASTT was Indirect Industry.

• This question had the highest mean value of all quantitative questions in the 1998 survey.

8 17

24

17

75

69

100

79

10 90

0% 20% 40% 60% 80% 100%

95 All

98 IndirectIndustry

98 University

98 AAC

98 All

Low Medium High

8.2

9.7

8.9

7.6

8.2

Mean


1b. Select all the NASA mission areas that greatly interest you.

51%

45%

33%

45%

73%

42% 41% 41%

16%

57%

33%

0%

20%

40%

60%

80%

100%

a b c d e f g h i j k

• Critical Technology (Base R&T) was selected as the area of greatest interest by 73 percent of the respondents.

• Other areas of high interest were Advanced Subsonic Transport (57percent) and High-Speed Research (51 percent).

• High Performance Aircraft (45 percent), High Performance Computing (45 percent), Space Transportation Technology (41 percent) and General Aviation (41 percent) all had moderate interest.

• Hypersonics (33 percent ) and Rotorcraft (33 percent) had the lowest level of interest in the specifically named categories.

a. Advanced Subsonic Transportb. High Speed Researchc. High Performance Aircraftd. Hypersonicse. High Performance Computingf. Critical Technology (R&T Base)

g. Facilities and Services

h. Space Transportation Technology

i. General Aviation

j. Rotorcraft

k. Other


2. How well do you understand the mission of the ASTT Program?

• The level of understanding of the ASTT Program for all respondents dropped from 55 percent in 1995 to 44 percent in 1998 in the “well” category. The mean value also dropped from 7.0 in 1995 to 6.4 in 1998.

• AAC Members had the highest level of understanding with a mean value of 8.8.

• Benchmark Customers had the second highest level of understanding with a mean value of 6.8.

• The group with the lowest level of understanding was Association/Publication with a mean value of 4.9.

• This question’s overall mean score of 6.4 ranked 14th of 30 quantitative questions.

20 25

46

33

55

15

91

4423

38

17

9

35 49

0% 20% 40% 60% 80% 100%

95 All

98 Assoc/Pubs

98 Benchmark

98 AAC

98 All

Poorly Moderately Well

8.8

6.4

6.8

4.9

7.0

Mean


3. How well are the current mission areas of NASA’s ASTT Program aligned to your organization’s needs?

• Government rated NASA’s ASTT Program mission areas best aligned to their needs with a mean value of 7.0.

• The Benchmark Companies and Direct Industry rated the mission areas next well aligned with a mean value of 6.1.

• Total scores were close to 1995’s scores, only dropping one percent from 26 percent to 25 percent in well aligned, but the poorly aligned category grew from 23 percent to 27 percent.

• Associations/Publications rated the goals least aligned with none scoring in the “well” category.


23 51

55

48

26

91

2527

45

24

9

46 30

0% 20% 40% 60% 80% 100%

95 All

98 Assoc/Pubs

98 Benchmark

98Government

98 All


5.8

7.0

6.1

4.5

6.1

Mean


3b. What current mission areas of NASA’s ASTT Program are most important to your organization?

• Organizational interest closely paralleled individual interest (Question 1b) but at a level approximately 8 to 10 percent less.

• Critical Technology (Base R&T) showed the most interest with 61 percent of the respondents.

• Another area of significant interest was Advanced Subsonic Transport (49 percent).

• High Speed Research (39 percent), High Performance Computing (38 percent), High Performance Aircraft (34 percent), Space Transportation Technology (34 percent), Facilities and Services (32 percent), General Aviation (30 percent), and Rotorcraft (30 percent) all had moderate interest.

• Hypersonics (23 percent ) had the lowest level of interest in the specifically named categories.

39%34%

23%

38%

61%

32% 34%30%

8%

49%

30%

0%

20%

40%

60%

80%

100%


a. Advanced Subsonic Transport

b. High Speed Research

c. High Performance Aircraft

d. Hypersonics

e. High Performance Computing

f. Critical Technology (R&T Base)


h. Space Transportation

Technology

i. General Aviation

j. Rotorcraft

k. Other


4. How well is NASA’s ASTT program accomplishing its mission?

• 63 percent of the total respondent group rated their general satisfaction in the middle range and most individual groups followed this pattern.

• The mean value for the 98 Survey (6.0) dropped by 0.3 as compared to the 95 Survey (6.3).

• Government Customers gave the highest rating with a mean value of 6.7.

• Mean values for Benchmark Customers was 6.1 which is slightly higher than the total population.

• Lower scores for accomplishing mission are consistent with lower scores for understanding of ASTT mission (Question 2).


15 57

53

63

28

33

1919

26

11

16

56

65

21

18

0% 20% 40% 60% 80% 100%

95 All

98 IndirectIndustry

98 Benchmark

98 Government

98 All


6.0

6.7

6.1

5.6

6.3

Mean


4b. What current mission areas is NASA’s ASTT Program accomplishing the best?

• Respondents typically only selected one or two items, unlike the previous selection questions where they picked several choices.

• Four mission areas were the most common selection; High Speed Research (31 percent), Advanced Subsonic Transport (29 percent), Critical Technology (R&T Base) (22 percent), and Space Transportation Technology (21 percent).

• High Performance Computing (12 percent) received a moderate rating.

• General Aviation (8 percent); High Performance Aircraft, Facilities and Services and Rotorcraft (all at 7 percent) and Hypersonics (5 percent) are accomplished least well according to the respondents.

31%

7%5%

12%

22%

7%

21%

8%

2%

29%

7%

0%

10%

20%

30%

40%

50%


a. Advanced Subsonic Transport

b. High Speed Research

c. High Performance Aircraft

d. Hypersonics

e. High Performance Computing

f. Critical Technology (R&T Base)


h. Space Transportation

Technology

i. General Aviation

j. Rotorcraft

k. Other


5. How familiar were you with the Pillars and Goals before this survey?

• Over two-thirds of the respondents were somewhat to very familiar with the Pillars and Goals before the survey.

• Those most familiar were the AAC Members (91 percent) followed by Benchmark Companies at 49 percent.

• The groups least familiar were Associations/ Publications with 62 percent being unfamiliar and Indirect Industry with 51 percent.

• This question’s overall mean score of 6.1 ranked 21st of 30 quantitative questions.

• There was no comparable question on the 95 or 92 surveys.

62 23

20

25

15

91

4332

51

9

23 28

29

49

0% 20% 40% 60% 80% 100%

98 Assoc/ Pub

98 IndirectIndustry

98 Benchmark

98 AAC

98 All

Unfamiliar Moderate Familiar

6.1

8.7

6.8

4.8

4.4

Mean


5b. How did you learn about the Pillars and Goals?

• Most respondents learned about the Pillars and Goals through interaction with NASA, either through working with NASA or from Committee work. These are indicated in the “Other” column.

• The next most frequent way of learning about the Pillars and Goals was from the NASA Administrator’s Speech.

• The least frequent way of learning about the Pillars and Goals was from the Internet.

26%

12%

5% 3%

54%

0%

10%

20%

30%

40%

50%

60%

a b c d e

a. NASA Administrator’s Speech

b. Pamphlet Mailed to You

c. Letter Introducing the Survey

d. From the Internet

e. Other


6. How well are the Goals aligned to your organization’s needs?

• Mean values of ratings for this question were closely bunched ranging from a high of 7.1 for AAC Members to a low of 6.1 for Associations/ Publications

• 42 percent of respondents ranked the Goals as well-aligned to their organization.


38 38

35

38

23

70

42

14

20

28

20

38

10

48

37

0% 20% 40% 60% 80% 100%

98 Assoc/ Pub

98 IndirectIndustry

98 Benchmark

98 AAC

98 All


6.6

6.9

Mean

7.1

6.4

6.1


6b. Which goals are best aligned to your organization’s needs?

Total Responses

62 Safety55 Reduced Cost of Air Travel52 Next Generation Design Tools34 Environmental Compatibility34 General Aviation Revitalization28 High Speed Research/Travel23 Access to Space23 Global Civil Aviation22 Revolutionary Technology Leaps19 Revolutionize Space Launch Capability14 Reduce Noise12 Experimental Aircraft

Many respondents identified specific Pillars or Goals; some identified subsets of Pillars or Goals.


6c. Which important Goal, from your organization’s perspective, is missing?

• Work with DoD where commonality applies -- [need] Goal to share and integrate cost-effective R&D with DoD• Greatly concerned about future of subsonic transport R&T. Basic is being neglected.• Development of future rotorcraft concepts..., working with FAA to develop infrastructure for civil

transportation.• No materials research for airframe technology for general aviation.• Aeronautics and Earth Science not connected -- NASA ignores internal customers.• High performance military aircraft.• Next generation avionics.• FANS, satellite communications technology, composite wing, wet wing.• Reduce cost to lower Earth orbit applies to only part of mission -- need Goals that cover launch to orbit

mission, transfer orbits and on issues with space probe.• Future engineering education should be in the Goals; important to next generation.• Economic viability of the contemplated technologies.• Human factors research.• Product quality.• Fundamental flow physics.• Infrastructure development.• There isn’t anything missing.

Respondents provided various ideas regarding goals that should be/could be added.


7. How well are the Goals aligned to national needs?

• Total respondents overwhelmingly ranked NASA’s Goals as being very well aligned to national needs scoring 66 percent in the “well” category. Total “well” and “moderately well” rating was 97 percent.

• Benchmark Customers and Universities ranked this question very high with 70 percent in the “well” category.

• The lowest “aligned well” rating was Government at 54 percent, which also had the second highest “poorly aligned” rating of 15 percent.

• Mean values closely followed the percentile ratings.

• This question’s overall mean score of 7.9 ranked 2nd of 30 quantitative questions.

15 31

8

31

54

70

66

17

3

30

29

70

75

0% 20% 40% 60% 80% 100%

98 Government

98 Assoc/Pub

98 Universities

98 Benchmark

98 All


7.9

8.1

Mean

8.1

7.3

7.1


7b. Which goals are best aligned to national needs?

Total Responses

71 Safety

70 Reduced Cost of Air Travel

43 Environmental Compatibility

34 High Speed Research/Travel

33 Global Civil Aviation

28 Revolutionize Space Launch Capability

25 Next Generation Design Tools

23 General Aviation Revitalization

21 Access to Space

15 Revolutionary Technology Leaps

14 Reduce Payload Cost to Orbit

8 All Goals Are Aligned

7c. What important Goal, from a national perspective, is missing?

Many of the suggestions for National Goals were similar to those provided in Question 6c in response to missing Goals from the Customer’s organizational needs.

Most respondents identified specific Pillars or Goals; some respondents identified subsets of Pillars or Goals. These results closely parallel the results of Question 6b regarding the alignment to organizational needs.


8. Overall, how well are NASA’s current programs aligned to the Goals?

• Overall, the respondents thought that NASA’s current programs were moderately well aligned to the Goals. However, within the Customer Groups, the rankings for this question were nearly the reverse of those observed in most other questions.

• The highest rankings were given by the Associations/Publications and the Universities with a mean value of 6.9.

• The lowest rankings within the customer groups were given by the Benchmark Customers and the Direct Industry Customers. They each had a mean value of 6.3.

• This question’s overall mean score of 6.4 ranked near the middle at 16th of 30 quantitative questions.

6.4

6.9

Mean

6.9

6.3

6.313 64

61

58

23

40

28

11

10

15

15

42

50

47

24

0% 20% 40% 60% 80% 100%

98 Benchmark

98 DirectIndustry

98 Universities

98 Assoc/Pub

98 All



8b. Which current ASTT programs are best aligned to the goals?

Total Responses

49 High Speed Research/Travel31 Advanced Subsonic Transport31 Experimental Aircraft (X - 33, X - 34, X - 36, Hyper - X, Future - X)24 Safety14 Next Generation Design Tools13 General Aviation Revitalization12 Advanced General Aviation Transport Experiments12 Global Civil Aviation12 Revolutionize Space Launch Capability11 Reduced Cost of Air Travel10 Access to Space

Most respondents provided answers that related to specific Pillars, Goals, or ASTT Programs.


9. What is your level of satisfaction with NASA’s ASTT Facilities and Services?

• The level of satisfaction with ASTT facilities and services dropped significantly as compared to the 1995 survey. The mean value in 1995 was 6.8, in this survey it dropped to 6.0

• Government Customers showed the greatest satisfaction with facilities and services with a mean value of 7.0; the AAC had the second highest mean value of 6.5.

• Associations and Publications had the lowest mean value of 4.9; Indirect Industry had the next lowest mean value at 5.4.

• This question’s overall mean score of 6.0 ranked moderately low at 22nd of 30 quantitative questions.

12

56

32

18

9

47

33

47

36

36

49

41

11

21

45

55

21 30

0% 20% 40% 60% 80% 100%

95 Survey

98 Assoc/Pub

98 IndirectIndustry

98 AAC

98Government

98 All

Low Medium High

6.0

7.0

6.5

5.4

4.9

6.8

Mean


9b. Which NASA facility and service issues are most important to your organization?

36%

68%

58%

63%

19%

35%

38%

60%

6%

75%

42%

32%

39%

10%

0% 20% 40% 60% 80%

a

b

c

d

e

f

g

h

i

j

k

l

m

n

Facility or Service Issue Percent

1. Quality of the Data 75

2. Level of Technical Expertise 68

3. Timeliness of the Results 63 High

4. Availability of Facility or Service 60

5. State-of-the-Art Facilities 58

6. Protection of the Data 42

7. Ease of Technology Transfer 39

8. Familiarity and/or Trust 38 Moderate

9. Charge for Using Facility or Service 36

10. Ownership by the U.S. Government 35

11. Computational Quality 32

12. Convenience of Facility or Service 19

13. Ownership by U. S. Government 10 Low

14. Location of the Facility or Service 6

a. Charge for using the facility or service

b. Level of technical expertise

c. State-of-the-art facilities

d. Timelines of the results

e. Convenience of the facility or service

f. Ownership of the intellectual property

g. Familiarity and/or trust

h. Availability of the facility or service

i. Location of the facility or service

j. Quality of the data

k. Protection of the data

l. Computational quality

m. Ease of technology transfer

n. Ownership by the U. S. Government


63%

56%

53%

41%

35%

31%

23%

8%

0% 10% 20% 30% 40% 50% 60% 70%

Contracts with Industry

Informal TechnicalDiscussions

TechnologyDemonstrations

Written TechnicalReports

Non-ContractCooperative Programs

Conferences,Symposia and

Government/IndustryPersonnel Exchanges

Other

7.49

10. Which methods of technology transfer work best for your organization?

These results differ somewhat from results of the 1992 survey. In that survey, customers were asked to give the various technology transfer methods a ranking from 1 to 10 with 1 being unfavorable and 10 being favorable. These results are shown in the right-hand columns.

92 Survey

7.01

Mean Rank

4

1

6.81 6

7.07 2

6.85 5

7.03 3

5.98 7


11. How would you rate NASA’s program planning?• Program planning was rated higher in this survey

(mean = 6.0) than in the 95 survey (mean = 5.6).

• Program planning was rated much higher in this survey than in the 92 survey when 53 percent of the respondents indicated that NASA’s program planning was inadequate.

• This question was rated higher by Associations and Publications (mean = 7.3) and Universities (mean = 6.7). It was rated lower by Benchmark Customers (mean = 5.9) and lowest by Government Customers (5.8)

• 58 percent of respondents answered “Yes” to the question, “Does NASA adequately involve your organization in its program planning for ASTT?”

• 62 percent of respondents answered “Yes” to the question, “Does NASA plan research programs to provide results in a form, structure and format useful to you?”

• This question’s overall mean score of 6.0 ranked quite low at 23nd of 30 quantitative questions.

34

27

24

18

46

45

59

36

40

54

21

27

17

45

60

23 23

0% 20% 40% 60% 80% 100%

95 Survey

98Government

98Benchmark

98Universities

98 Assoc/Pub

98 All

Low Medium High

6.0

7.3

6.7

5.9

5.8

5.6

Mean


12. How would you rate NASA’s Outreach and External Communication?

21

17

19

10

50

55

50

29

40

49

29

28

31

62

60

19 32

0% 20% 40% 60% 80% 100%

95 Survey

98 DirectIndustry

98Benchmark

98Universities

98 AAC

98 All

Low Medium High

6.3

7.4

7.3

6.4

6.3

6.2

Mean• Outreach and External Communication was rated

slightly higher in this survey (mean = 6.3) than in the 95 survey (mean = 6.2).

• The AAC (mean = 7.4) and University groups (mean = 7.3) rated this question high. Benchmark Customers (mean = 6.4) and Direct Industry Customers (mean = 6.3) rated the question lower than the overall mean of 6.3.

• 68 percent of respondents answered “Yes” to the question, “Does NASA ask for your input on plans and

issues important to your organization?”

• 67 percent of respondents answered “Yes” to the question, “Does NASA use your input?”

• 65 percent of respondents answered “Yes” to the question, “Does NASA provide feedback on plans/issues

important to your organization?”

• This question’s overall mean score of 6.0 ranked near the middle at 17th of 30 quantitative questions.


13. What is your overall level of satisfaction with NASA?

13b. In what areas are you most satisfied with NASA?• Increasing attentiveness to the needs of U. S. industry; improved communication ... in executing contracts.• Good job getting input from industry; good facilities and people.• Quality work, quality reports, results and availability.• Headquarters' interest in having a positive national impact through technology development.

13c. In what areas are you least satisfied with NASA?• Inability to do meaningful multi-year planning.• Inability to change with changing needs of industry.• Technology transfer - cost to industry; complex paperwork; “nightmare” multi-page licenses.• NASA competition with industry.

14

16

7

12

56

52

53

52

30

32

40

35

0% 20% 40% 60% 80% 100%

95 Survey

98 Executive

98 Mid-Level

98 All

Low Medium High

6.6

7.0

6.3

6.4

Mean

• The Customer’s Overall Level of Satisfaction with NASA was rated slightly higher in this survey (mean = 6.6) than in the 95 survey (mean = 6.4).

• There was a significant difference (p<0.05) in the mean values of the Executives (mean = 6.3) and the Mid-Level Managers (mean = 7.0) on this question.

• This question’s overall mean score of 6.6 ranked near the top one-third of the questions at 11th of 30 quantitative questions.


14. What is your overall level of satisfaction with NASA’s ASTT products?

14b. With what products were you most satisfied? Responses Responses

• High Speed Research/Travel & High Speed Civil Transport 23 Technical Reports/Symposia 13 Computational Fluid Dynamics/Computational Analysis 17 Use of NASA Facilities 12

• Research and Technology Base 15 Design and Analysis Tools 10• Advanced Subsonic Transport/Subsonic Research 14 Reduce Noise 10• Experimental Aircraft (X-33, X-34, X-36, Hyper-X, Future-X) 13 Rotorcraft/Tiltrotor 9

14c. What products were you least satisfied with?• Those that don’t have a cooperative character...

• Full-cost [funding] resulting in budget cuts and diminished expectations.

• Technology transfer between larger corporations and NASA and government (as on HSR program).

17

7

15

55

60

53

57

28

40

40

28

0% 20% 40% 60% 80% 100%

98 DirectIndustry

98 AAC

98 University

98 All

Low Medium High

6.4

7.1

6.9

6.3

Mean

• The Customer’s Overall Level of Satisfaction with NASA’s ASTT Products was rated moderately high. Overall, 85 percent of the respondents rated their satisfaction as “medium” or “high.”

• University Customers (mean = 7.1) and AAC Customers (mean = 6.9) had a higher rating than the overall population. Direct Industry Customers (mean = 6.3) had a lower rating than the overall population.

• This question’s overall mean score of 6.4 ranked in the middle at 15th of 30 quantitative questions.


15. How does your organization’s overall level of satisfaction with NASA compare with its satisfaction 3 to 6 years ago (before the last surveys)?

15b. In what area(s) has your level of satisfaction increased?• Alignment of research to the Pillars; six years ago there was inefficiency and lack of focus.• Shift from micro-management to a partnership relationship.• Involvement and visibility of NASA plans and goals.• Just in being part of their activities; we’re in partnership with them; our relationship has grown by leaps and bounds.

15c. In what area(s) has your level of satisfaction decreased?• Systems integration emphasis, computational resources, and test facility readiness.

• Availability of wind tunnel test facilities; reduced emphasis on R&T base for aeronautics.

• I think the most important is the condition of NASA’s deteriorating facility support.

• Difficulty in dealing with NASA as a customer.

6.5

6.6

6.328

14

15

35

69

51

50

36

13

35

34

19

0% 20% 40% 60% 80% 100%

95 Survey

98 University

98Benchmark

98 All

Low Medium High

6.1

Mean • The Customer’s felt that their satisfaction with NASA had improved over the last 3 to 6 years. They rated this question slightly higher in the current survey (mean = 6.5) than in the 95 survey (mean = 6.3).

• 85 percent of respondents rated this question as “medium” or “high.” Benchmark Customers (mean = 6.6) provided slightly higher ratings, and University Customers (mean = 6.1) provided slightly lower answers than the overall population.

• This question’s overall mean score of 6.5 ranked slightly above the middle at 13th of 30 quantitative questions.


16. NASA provides key laboratory, computing, and test facilities.

16b. What NASA facilities do you view as being essential to the conduct of leading edge R&D?The overwhelming answer to this question was wind tunnels, followed by computational resources. The

comments below characterize these and other comments.

• Modern wind tunnel test facilities and world class computational resources.• Space test facilities at Stennis Center.• Long duration test and space exposure tests.• Airborne test beds, simulation and flight facilities.

7.6

8.2

7.56

7

34

21

15

31

59

74

85

62

5

0% 20% 40% 60% 80% 100%

98Benchmark

98 University

98Government

98 All

Disagree Partially Agree Agree

8.1

Mean • The respondents strongly felt that NASA provides key facilities.

• 93 percent of respondents rated this question as “partially agree” or “agree” and 62 percent rated it “agree.” Government Customers (mean = 8.2) and University Customers (mean = 8.1) provided high ratings, and Benchmark Customers (mean = 7.5) provided slightly lower answers than the overall population.

• This question’s overall mean score of 7.6 ranked near the top at 3rd of 30 quantitative questions.


17. NASA downsizing has affected its ability to support its customers.

17b. In what ways, if any, has NASA downsizing affected your organization?Some respondents felt that NASA downsizing had a negative effect:• Foundational research is not getting done.• Primarily, loss of key NASA expertise and missing expertise is not being replaced.• We are starting to reduce our alliance on technologies developed by NASA.Other respondents felt that NASA downsizing had positive effects:• Very positive effect - less micro-management - more of a partnership.• Haven’t downsized enough.• Helped since NASA is now less competitive with industry.• Its been positive; it’s made them take a hard look at programs and ... it helped in that they are more focused.

5.8

7.0

5.936

35

29

30

45

32

35

40

36

33

18

30

0% 20% 40% 60% 80% 100%

98Benchmark

98 University

98 AAC

98 All


6.4

Mean • The respondents were equally divided in response to this question.

• 65 percent of respondents rated this question as “partially agree” or “agree” and 33 percent rated it “agree.” AAC Customers (mean = 7.0) and University Customers (mean = 6.4) provided high ratings, while Benchmark Customers (mean = 5.9) were more inclined to disagree than the overall population.

• This question’s overall mean score of 5.8 ranked near the bottom at 26th of 30 quantitative questions indicating significant disagreement that downsizing has affected NASA’s ability to support customers.


18. NASA is a key contributor to the education of scientists and engineers.

18b. In what ways can NASA improve the education and training of our next generation of scientists and engineers?

The main themes were more funding, cooperative programs, and more internships. Also NASA involvement in university, secondary and elementary schools was identified as being important.

Strengthen university relationships that include industry.• Put more money into foundational research in universities best equipped to do that type of research.• Re-instituting university programs programs where NASA provides seed money to university engineering programs.• Be more involved in [recommending] curricula of engineering schools. Have more open house events to stimulate

younger students.• Public relations programs aimed at younger folks to kindle their interest early on.• Target younger children, even in elementary grades.

7.0

8.0

6.718

15

38

43

10

32

44

44

81

52

10

14

0% 20% 40% 60% 80% 100%

95 Survey

98Benchmark

98 AAC

98 All


6.9

Mean• The respondents strongly felt that NASA is a key

contributor to the education of scientists and engineers. The respondents of the current survey (mean = 7.0) gave higher ratings to this question than did the respondents of the 95 survey (mean = 6.7).

• 84 percent of respondents rated this question as “partially agree” or “agree” and 52 percent rated it “agree.” University Customers (mean = 8.0) provided high ratings, and Benchmark Customers (mean = 6.9) provided slightly lower ratings than the overall population.

• This question’s overall mean score of 7.0 ranked near the top at 4th of 30 quantitative questions


19. NASA cooperates with other organizations to make cost-effective use of limited R&D resources, skills and facilities.

19b. In what ways can NASA improve cooperation with other organizations?More communication and coordination was the primary theme of the responses.• Talk to and coordinate with DoD -- and don’t just talk -- [perform] some deeds.• Expand levels of cooperation with other government agencies as well as corporate world; develop and participate

in [joint] technical programs.• By helping its personnel to understand the realities and mechanisms of the commercial marketplace.• Improve communication; become more aware of industry programs to create cooperative programs.• [Be] more pro-active in searching for organizations to cooperate with.

6.3

7.0

6.125

22

43

46

64

43

32

33

36

35

21

0% 20% 40% 60% 80% 100%

95 Survey

98Benchmark

98 AAC

98 All


6.3

Mean • The respondents were mixed in their rating of NASA’s cooperation with other organizations. The respondents of the current survey (mean = 6.3) gave slightly higher ratings to this question than did the respondents of the 95 survey (mean = 6.1).

• 78 percent of respondents rated this question as “partially agree” or “agree” and 35 percent rated it “agree.” AAC Customers (mean = 7.0) provided higher ratings than the overall population and Benchmark Customers (mean = 6.3) provided ratings equal to the overall population.

• This question’s overall mean score of 6.3 ranked in the lower half at 19th of 30 quantitative questions


20. The ASTT program effectively balances fundamental (R&T base) research with its focused-program research.

• Fundamental research; if fundamental research is done, the focused-program research will be done better.• [NASA] should be looking at 5 to 10 year technology; universities should be looking at 10 to 15 year technology;

industry should look at 2 to 5 year technology.• More fundamental research; industry better understands market need; leave focused-program to industry.• More focused-program [research] because it is a more effective use of resources.• Focused-program research to become more efficient -- pull research rather than push research.• More focused-program research, because it offers more incentive for industry involvement that would lead to

commercial products.

5.9

7.011 33

42

56

2929

0% 20% 40% 60% 80% 100%

98Government

98 All

Disagree Partially Agree Agree Mean • The respondents were mixed in their rating of NASA’s balance of fundamental versus focused-program research. Government Customers (mean = 7.0) provided much higher ratings than the overall population.

• This question’s overall mean score of 5.9 ranked in the lower half at 25th of 30 quantitative questions.

• Respondents of the current survey called for significantly more fundamental research than did the respondents of the 92 survey.

16

33

35

20

45

34

4

12

0% 20% 40% 60% 80% 100%

92 Survey

98 Survey

No Opinion FundamentalBalanced Focused-Program

20b. Should NASA pursue more fundamental research or focused-program research, and why?

Some thought balance was good; others presented arguments for both fundamental and focused research.


21. The ASTT program effectively balances analytical, computational, ground-based experimental and flight research.

21b. Should NASA change its mix of analytical, computational, ground-based experimental and flight research, and why?

Most respondents felt the balance was good now, but the mix needs to be constantly scrutinized. There was some call for more flight testing.

Yes, not enough flight research for aeronautics; maintain ground-based experimental facilities.• More emphasis on flight research [ will produce] more rapid progress. • In general, they are pretty well balanced, but caution is necessary on flight research to make sure the

costs/benefits are scrutinized.• The mix is fine and doesn’t need to be changed.

6.8

7.6

5.911

13

78

41

27

43

11

47

64

44

11

0% 20% 40% 60% 80% 100%

98 Assoc/Pub

98Benchmark

98 AAC

98 All


6.9

Mean • The respondents generally approved of NASA’s mix of research methods. The overall mean was quite high at 6.8.

• There was some variation within the customer groups. AAC Customers (mean = 7.6) provided higher ratings than the overall population; Benchmark Customers (mean = 6.9) provided slightly greater than the ratings of the overall population; Associations and Publications gave a low rating.

• This question’s overall mean score of 6.8 ranked quite high (in the upper quarter) at 6th of 30 quantitative questions


22. The ASTT engineers and scientists play a key role in the development of innovative or enhanced technical solutions.

22b. In what areas have NASA engineers & scientists played a key role?• Human factors, structures, propulsion and computational fluid dynamics -- Space travel is the biggest• Atmospheric physics; earth physical science -- Development of high performance rocket propulsion• Lifting-body ceramic and metallic thermal protection systems; supercritical wing; coke-bottle design of aircraft• Composite technology; computer fluid analytical (CFA) tools; and noise research -- Atmospheric modeling

22c. In what area have they not played a key role?• They’ve tried to play a role in system integration, but unsuccessfully -- Solution of critical operational problems• General aviation and airline safety -- Development of cost effective launch systems• Have not achieved leadership in information technology, for example, redundant/fault-tolerant information systems• Reducing cost of bringing advanced technology to market and reducing the time to certification of key technologies

7.0

7.3

6.912

11

47

33

37

47

41

58

53

43

8

0% 20% 40% 60% 80% 100%

98Benchmark

98Government

98 University

98 All


7.3

Mean • The respondents generally felt that NASA’s engineers and scientists play a key role in finding technical solutions. The overall mean was quite high at 7.0.

• There was only slight variation within the customer groups as the mean values ranged from 6.9 for Benchmark Customers to 7.3 for University and Government Customers.

• This question’s overall mean score of 7.0 ranked quite high (in the upper quarter) at 5th of 30 quantitative questions


23. How effective is NASA’s ASTT program in performing fundamental research?

6.3

7.8

6.0

10

8

30

54

51

67

42

54

61

40

22

25

58

27

6

20 26

0% 20% 40% 60% 80% 100%

95Effectiveness

95Contributions

98Benchmark

98Government

98 Assoc/Pub

98 All

Ineffective Moderately Effective Effective

6.7

Mean• As compared with the 95 survey(which contained

two question relating to performance of fundamental research) the 98 survey showed much lower mean values. The 98 survey had a mean of 6.3. The 98 survey had a mean score of 7.1 for effectiveness and a 7.5 score for contributions to national needs.

• The customer groups presented varied results in rating the effectiveness NASA’s ASTT Program in performing fundamental research. The mean values ranged from 6.0 for Benchmark Customers to 7.8 for Associations and Publications.

• This question’s overall mean score of 6.3 ranked in the lower half at 18th of 30 quantitative questions.

7.1

7.5


24. How effective is NASA’s ASTT Program in improving national security?

5.1

5.4

5.5

20

45

58

40

46

43

45

43

21

11

17

12

18

37

49

36

42 15

0% 20% 40% 60% 80% 100%

95Contribution

98 Mid-Level

98 Executive

98Benchmark

98 AAC

98 All


4.8

Mean • As compared with the 95 survey the 98 survey scored much lower mean values. The 98 survey had a mean of 5.1. The 98 survey had a mean score of 6.0 for contribution to national needs.

• The customer groups presented very low results in rating the effectiveness NASA’s ASTT Program in improving national security. The mean values ranged from 4.5 for Mid-Level Managers to 5.4 for AAC Members.

• The statistical data showed there was a significant difference in the mean values of the Executive Customers (mean = 5.5) and the Mid-Level Managers (mean = 4.5). For this question, there is a high likelihood that the two customer groups represent different population groups.

• This question’s overall mean score of 5.1 ranked lowest in the entire survey at 30th of 30 quantitative questions.

4.5

6.0


25. How effective is NASA’s ASTT Program in developing/ transferring technologies for economic competitiveness?

5.8

6.5

5.8

16

9

39

40

55

64

58

56

45

40

19

27

32

26

20

11

24 19

0% 20% 40% 60% 80% 100%

95Effectiveness

95Contributions

98Benchmark

98 AAC

98 University

98 All


6.4

Mean• As compared with the 95 survey the 98

survey scored much lower mean values. The 98 survey had a mean of 5.8 for effectiveness. The 95 survey had a mean score of 6.9 for effectiveness and 6.6 for contribution to national needs.

• The customer groups had generally low mean values in rating the effectiveness NASA’s ASTT Program in developing/transferring technologies for economic competitiveness. The mean values ranged from 5.8 for Benchmark Customers to 6.5 for University Customers.

• This question’s overall mean score of 5.8 ranked in the lowest quarter of the survey at 28th of 30 quantitative questions.

6.6

6.9


26. How effective is NASA’s ASTT Program in maintaining superiority of US aircraft and engines?

6.7

7.4

6.7

15

9

46

45

53

27

50

52

38

36

36

64

45

11

18

5

12 36

0% 20% 40% 60% 80% 100%

95Achievement

98Government

98Benchmark

98 AAC

98 University

98 All


7.1


survey scored slightly higher mean value. The 95 survey had a mean of 6.6 for national goals achievement. The 98 survey had a mean score of 6.7 for effectiveness.

• The customer groups had moderately high mean values in rating the effectiveness NASA’s ASTT Program in maintaining the superiority of U. S. aircraft and engines. The mean values ranged from 6.1 for Government Customers to 7.4 for University Customers.

• This question’s overall mean score of 6.7 ranked in the highest third of the survey at 9th of 30 quantitative questions.

6.1

6.6


27. How effective is NASA’s ASTT Program in developing a superior, affordable, global air transportation system?

5.7

6.7

6.4

18

13

57

56

64

63

36

52

24

17

27

25

45

9

27

18

28 20

0% 20% 40% 60% 80% 100%

95Achievement

98Benchmark

98Government

98 University

98 AAC

98 All


6.5


survey scored a lower mean value. The 95 survey had a mean of 6.2 for national goals achievement. The 98 survey had a mean score of 5.7 for effectiveness.

• The customer groups had very low mean values in rating the effectiveness NASA’s ASTT Program in developing a superior, affordable global transportation system. The mean values ranged from 5.6 for Benchmark Customers to 6.7 for AAC Members.

• This question’s overall mean score of 5.7 ranked very low in the survey at 29th of 30 quantitative questions.

5.6

6.2


28. How effective is NASA’s ASTT Program in developing superior, affordable, space launch systems?

6.1

6.2

6.6

14

45

47

43

46

27

52

44

39

29

39

27

30

16

28

18

23 33

0% 20% 40% 60% 80% 100%

95Contribution

98 Executives

98 Mid-Level

98 AAC

98Benchmark

98 All


5.1

Mean • As compared with the 95 survey the 98 survey scored a slightly lower mean value. The 95 survey had a mean of 6.4 for contributions to national needs. The 98 survey had a mean of 6.1 for effectiveness.

• The customer groups had very mixed mean values rating the effectiveness in developing superior, affordable space launch systems. The mean values ranged from 5.1 for AAC Members to 6.6 for Mid-Level Managers. The AAC Members gave this question their lowest rating in the 98 survey.

• The statistical data showed there was a significant difference in the mean values of the Executive Customers (mean = 5.8) and the Mid-Level Managers (mean = 6.6). For this question, there is a high likelihood that the two customer groups represent different population groups.

• This question’s overall mean score of 6.1 ranked low in the survey at 20th of 30 quantitative questions.

5.8

6.4

• There was one question on the 95 survey regarding contributions to national needs in space. The 95 survey question and the 98 survey question are slightly related in that regard and a comparison is presented herein.


29. How effective is NASA’s ASTT Program in ensuring long-term environmental compatibility of aerospace systems?

6.7

7.4

6.6

12 52

56

62

50

53

59

36

31

30

50

47

8

13

9 32

0% 20% 40% 60% 80% 100%

95Contribution

95Achievement

98Benchmark

98Government

98 University

98 All


7.1

Mean • There were two questions on the 95 survey dealing with environmental technology. One related to contribution to national needs and the other related to national goals achievement.

• The 98 survey scored between the two 95 survey questions. The 95 survey had a mean of 6.4 for national goals achievement 6.8 for contribution to national needs. The 98 survey had a mean score of 6.7 for effectiveness.

• The customer groups had generally high mean values in rating the effectiveness NASA’s ASTT Program in ensuring long-term environmental compatibility of aerospace systems. The mean values ranged from 6.6 for Benchmark Customers to 7.4 for University Customers.

• This question’s overall mean score of 6.7 ranked in the upper third of the survey at 8th of 30 quantitative questions.

6.4

6.8


30. How effective is NASA’s ASTT Program in making partnerships with government, industry, and universities?

6.8

6.9

6.7

16 34

46

52

49

58

52

50

33

33

42

42

9

14

21

0

10 38

0% 20% 40% 60% 80% 100%

95Effectiveness

95Achievement

98 IndirectIndustry

98Benchmark

98Government

98 All


6.9

Mean • There were two questions on the 95 survey dealing with partnerships. One related to effectiveness and the other related to national goals achievement.

• The 98 survey scored between the two 95 survey questions. The 95 survey had a mean of 6.3 for national goals achievement and 7.0 for effectiveness. The 98 survey had a mean score of 6.8 for effectiveness.

• The customer groups had generally high mean values in rating the effectiveness NASA’s ASTT Program making partnerships with government, industry and universities. The mean values were tightly bunched and ranged from 6.5 for Indirect Industry Customers Customers to 6.9 for several customer groups.

• This question’s overall mean score of 6.8 ranked in the upper quarter of the survey at 7th of 30 quantitative questions.

6.3

7.0


31. How effective is the overall ASTT contribution to US R&D needs?

31b. What can ASTT do to increase its overall effectiveness in contributing to US R&D needs?The main themes were more funding, more fundamental research, and working more closely with customers.• More stable, long term programs; more emphasis on validation.• NASA needs to excel in the use of critical research and development methods with both industry and other branches

of the government, especially DoD and FAA.• Maintain a high level of communication in trying to understand industry’s needs.• Perform better market research to validate the need for the research.• Continue looking at business-based metrics; watch U. S. competitive needs.

6.6

6.9

7.313

12

29

56

55

57

59

31

36

31

9

13

0% 20% 40% 60% 80% 100%

95Effectiveness

98Benchmark

98 AAC

98 All


6.6

Mean

• The respondents felt that NASA is an effective contributor to U. S. research and development needs. However, the respondents of the current survey (mean = 6.6) gave lower ratings to this question than did the respondents of the 95 survey (mean = 7.3).

• 88 percent of respondents rated this question as “partially agree” or “agree” and 31 percent rated it “agree.” AAC Members (mean = 6.9) provided slightly higher ratings than the overall population. Benchmark Customers (mean = 6.6) provided ratings consistent with the overall population.

• This question’s overall mean score of 6.6 ranked near the middle at 12th of 30 quantitative questions


32. Has NASA’s planning worsened or improved over the last few years?(Asked to Executive Customers only)

32b. What can NASA do to improve its planning?• Provide a forum that goes beyond workshops and actually sets up programs -- smaller group of key industry

participants to boil down ideas into programs; do more feasibility studies with industry before launching a program.• Be more responsive to industry’s needs -- not so much planning as it is attitude. • More planning in cooperation with DoD elements in aeronautics and space.• Better execution of plans.• Match planning to budget; avoid stop - starts in programs due to budget fluctuations.• Maintain commitments once established.• Narrow focus -- be better at fewer things.

6.4

7.3

5.923

18

57

47

45

51

20

34

45

32

9

19

0% 20% 40% 60% 80% 100%

98 IndirectIndustry

98Benchmark

98 AAC

98 All

Worsened Same Better

6.6

Mean

• Most respondents (51 percent) felt that NASA’s planning had stayed the same over the last few years. However, more respondents (32 percent) felt that planning had improved as compared to those who felt that it had worsened (18 percent).

• AAC Members (mean = 7.3) provided significantly higher ratings than the overall population

(mean = 6.4). Benchmark Customers (mean = 6.6) provided ratings slightly higher than the overall population. Indirect Industry Customers

(mean = 5.9) rated this question much lower than the overall mean.

• This question was not included in the count of 30 quantitative rating questions because it was asked only of Executive Customers. However, the overall mean score of 6.4 ranked near the middle of the 30 quantitative questions


33. Have you participated in a previous NASA Aero Survey?

• 21 Executive Level respondents (out of a total of 101) participated in the 1992 survey.

• 35 Executive Level respondents participated in the 1995 survey.

• 13 Executive Level respondents participated in both prior surveys.

33b. If you participated in the 1992 and/or the 1995 survey, did the results of the survey in any way change your organization’s relationship with NASA? If so, how?

The responses were mixed concerning changes in relationships with NASA.

• Yes, between 1992 and 1995 industry became more involved in planning through workshops.• Yes, helped put more relevancy in NASA which led to Pillars and Goals.• Yes, increased efforts in joint industry/NASA planning.• Created an awareness of the advantages of working with NASA.

• No apparent impact -- however, NASA should be praised for attempt.• No, I don’t know if I ever saw results of the survey.


34. Compared to previous NASA Aero Surveys (1992 and 1995), how would you rate this one? (Asked to Executive Customers only)

34b. How can future surveys be improved? • Offer opportunity to get into specifics in different areas of ASTT.• Have Pillars and Goals ahead of time.• Need more questions on how NASA measures their effectiveness against customer need.• ... A few weeks before the survey, provide a list of fundamental areas that will be discussed.• Survey well done; it can’t be improved.• By dividing them into specific areas, such as: global civil aviation, space transportation, etc.• Continue the in-person interviews.• Emphasize more narrative questions and answers. There are techniques to analyze narrative answers.

6.9

7.1

6.614 43

62

71

61

43

33

29

345

5

0% 20% 40% 60% 80% 100%

98Government

98Benchmark

98 IndirectIndustry

98 All

Low Medium High

6.8

Mean

• Most respondents (95 percent) rated the 98 survey as “medium” or “high.”

• Indirect Industry Customers (mean = 7.1) provided slightly higher ratings than the overall population

(mean = 6.9). Benchmark Customers (mean = 6.8) provided ratings slightly lower than the overall population. Government Customers (mean = 6.6) also rated this question lower than the overall mean.

• This question was not included in the count of 30 quantitative rating questions because it was asked only of Executive Customers. However, the overall mean score of 6.9 ranked in the top quarter of the 30 quantitative questions


35. For the next survey, what new question(s) should be asked?

There are a wide range of suggested questions, but respondents were most interested in rating NASA’s performance and follow through. They also wanted to be involved in future NASA planning and strategy development processes.

• Specific questions within the mission areas — a two-level questionnaire.• What changes would you make to improve NASA effectiveness in achieving its mission? What diverts or

interferes with NASA being effective?• Break into space and aeronautical segments.• Are NASA's expenditures properly balanced between programs?• More questions aimed at university programs, training, and engineers.• Rate performance of different NASA levels: Center, HQ Staff, and Administration• A lot of the questions from this survey are the right ones. Maybe some about how we are doing on the

implementation of the recommendations would improve and smooth the survey effort. Kind of the "big stick" approach.

• Pretty complete. Presupposes that interviewee knows the breakdown of the budget.• Ask some fundamental questions on US aeronautics policy and what would affect the customer's predilection to

using technology.• Questions about individual center's performance, especially, performance at headquarters.• What specific areas do you work with or have worked with in the past year with NASA? How frequently do you

interact with NASA?


36. What do you want to say to NASA administrator about ASTT Program?

• Hold the course, balance is good, good support from industry -- don't overreact to public statements.• That the ASTT Program is the most efficient dollars NASA spends. The ASTT industry/NASA roles are a more

appropriate approach for space development (not including exploration) than currently allocated. For all the criticism, it’s much more efficient when industry contributes funds and resources collaboratively.

• Don't sacrifice aeronautics work due to pressure of financing space transportation or other space activities.• Strengthen university programs to develop next generation scientists and engineers.• The subsonic section has inadequate advocacy with the public and is not on par with the space activity.• Thank you for covering the wide range in ASTT from general aviation to high-speed aeronautics.• We are on a good course. Things have certainly been improving.• NASA should consider and address specific needs in industry, especially increased competitiveness. Keep up

research in propulsion area and noise reduction area. Emphasize creature comforts and safety in GA.• It needs a heavier emphasis on space technology.• Goals are good. NASA needs to open up as going through goals. Allows for realistic programs to achieve goals.• I'm very happy to see that the ASTT is stronger focused and more dynamic. I like the way the program is now and

not the way it was before.• More cooperation with military, there's been an attempt but not "tight" enough.• NASA should focus on space and let the aeronautics industries handle aeronautics research.• DoD is interested in being more closely related with NASA than in past. Could use a true personnel exchange.• Loss of key personnel, key capability, need to recruit the best personnel to stay on top of programs.• Quit using taxpayers’ money to fund competition because you are discouraging private investment ...• Don't let large industry drive high-speed civil. Follow the recommendations of National Research Council on HSCT.

Is it necessary for NASA to be so broad? Aeronautics should stick to aeronautics ...• Job well done!! • Keep providing results to hold confidence of Congress to hold the program. But, reach out to industry and identify

specific programs to which those funds should be targeted.

1nasa customer survey 07/31/98 07:29 pm nasa customer survey summary report prepared by science...

Documents