2008 faeis annual longitudinal assessment with a comparison to the 2007 survey results the purpose...
TRANSCRIPT
![Page 1: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/1.jpg)
2008 FAEIS Annual Longitudinal Assessment
With a Comparison to the 2007 Survey Results
The purpose of the FAEIS annual evaluation is to develop longitudinal and trend analyses in order to assess progress and develop program components for the future.
![Page 2: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/2.jpg)
Objectives
Describe selected demographic and other variables that have potential association with FAEIS use.
Determine the level of usefulness of FAEIS data resources and applications.
Determine the level of satisfaction with selected components of FAEIS.
Determine levels of adoption of FAEIS and factors that explain the levels of adoption.
Determine the relationship of selected demographic and usage variables (involvement, use, experience, primary job title) with perceived usefulness and satisfaction with FAEIS.
![Page 3: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/3.jpg)
Methodology
2007- Panel of Experts - 7 FAEIS project staff, USDA representatives and chair of the FAEIS Panel
2007 - FAEIS Panel Review and Approval
2007 - Instrument Development 2008 - Population 1070 Users; 285
Respondents 2008 - 2 Follow Up Reminders
![Page 4: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/4.jpg)
2008 Participants
Staff Faculty Department Head College Associate/Assistant Deans College Dean or VP University level administration “Government” added with recoding
![Page 5: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/5.jpg)
Question 1 in 2008 My Involvement with FAEIS Includes: (select as many as apply)
(all possible combinations displayed; N=285)
![Page 6: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/6.jpg)
Question 1 My Involvement with FAEIS Includes: (select as many as apply)
(all possible combinations displayed; N=285)
Involvement Type 2007Percent of Total
2008Percent of Total
Data Entry 54 53.3
User of FAEIS Products 8 9.1
Data Entry & Analytical 7 7.7
Analytical & User of FAEIS products
3 6.0
Analytical 6 5.6
Data entry & User of FAEIS products
5 5.6
None 3 2.8
Data entry, Analytical, & User of FAEIS products
4 2.5
No answer 2 2.5
Analytical, Product developer, & User of FAEIS products
0.003 2.1
Data entry & Product developer
3 1.1
Data entry, Analytical, Product developer, & User of FAEIS products
2 0.7
Data entry, Analytical & Product developer
2 0.4
Analytical & Product developer 0.003 0.4
Product developer 1 0.4
Product developer & User of FAEIS products
0.003 ---
![Page 7: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/7.jpg)
Question 1Involvement with FAEIS of Survey Respondents(“none” and no answer responses eliminated)
2007 Percent of Total
(N=308)
2008 Percent of Total
(N=285)
Data Entry 81 75
User of FAEIS Products
21 27
Analytical 25 15
Product Developer
9 5
Analysis – in 2008 there were fewer data entry survey respondents, more users of FAEIS products, but fewer respondents indicated that they are analyzing and developing their own products, compared to 2007 survey respondents.
![Page 8: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/8.jpg)
Question 2 How often have you used FAEIS in the past 12 months?
(N=285)
2008
![Page 9: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/9.jpg)
Question 2 How often have you used FAEIS in the past 12 months?
2007 Percent of Total (N=308)
2008 Percent of Total
(N=285)
Not at all 26.3 27.4
1-2 times 47.4 44.9
3-4 times 16.2 16.5
More than 4 times
9.7 11.2
No answer 0.3 --
Analysis – slight increase in the more frequent use categories (3-4 times & >4 times) in 2008
![Page 10: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/10.jpg)
Question 3 My total experience using FAEIS is?
(N=285)
2008
![Page 11: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/11.jpg)
Question 3 My total experience using FAEIS is?
2007 Percent of Total (N=308)
2008 Percent of Total
(N=285)
More than 3 years
45 49.5
1 to 3 years 31 23.2
Less than 1 year
22 24.9
No answer 2 2.5
Analysis –greater number of “experienced” users (> 3 yrs) and greater number of “less experienced” users (<1 yr) in 2008 compared to 2007.
![Page 12: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/12.jpg)
Question 4My professional position is best described as?
(N=285)
2008
![Page 13: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/13.jpg)
Question 4My professional position is best described as?
2007 Percent of Total (N=308)
2008 Percent of Total (N=285)
Staff 49 46.7
Faculty 18 7.7
Department Head 6 19.3
College Associate/Assistant Dean
16 15.4
College Dean or VP 5 7.7
University level Administration
5 2.5
Government -- 0.7
No answer 1 --
Analysis – More administrators (non-staff/faculty) participated in the survey in 2008 (44.9%) vs 2007 (32%).
![Page 14: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/14.jpg)
Question 5My level of adoption (use) of FAEIS is best
described as? (N=285)
2008
![Page 15: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/15.jpg)
Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis)
2007 Percent of Total (N=290)
2008 Percent of Total (N=285)
1=unaware 7.5 10.2
2=aware of it but do not use it 42.2 40.0
3=Use it but not sure of long term use
26.9 27.0
4=Use it and it is integral to my job 15.0 15.8
5=Integral to my job and expect long term use
6.1 6.7
6=So important as to never be without FAEIS
1.0 0.4
Mean Score 2.73 2.70
![Page 16: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/16.jpg)
Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis)
2007Mean
2008Mean
Data Entry Users Only
2.49(N=159)
2.30(N=152)
All Other Users 3.16(N=117)
3.24(N=125)
Total 2.78(N=276)
2.73(N=277)
1=unaware
2=aware of it but do not use it
3=Use it but not sure of long term use
4=Use it and it is integral to my job
5=Integral to my job and expect long term use
6=So important as to never be without FAEIS
![Page 17: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/17.jpg)
Question 6Please rate the usefulness of the data resources in FAEIS (for
student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.). (1 = not useful, 5 = very useful)
2008
![Page 18: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/18.jpg)
Question 6Please rate the usefulness of the data resources in FAEIS
(for student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.).
Score
2007 Percent of Total (N=192)
2008 Percent of Total (N=176)
1, Not useful 7 7
2 12 13
3 28 28
4 29 31
5, Very useful 24 21
Mean score 3.50 3.47
(“no answer” and “not used” responses eliminated)
Analysis – no significant changes between ‘07 & ‘08
![Page 19: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/19.jpg)
Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment, faculty hiring and
fundraising). (1 = not useful, 5 = very useful)
2008
![Page 20: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/20.jpg)
Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment,
faculty hiring and fundraising). (1 = not useful, 5 = very useful)
Score
2007 Percent of Total (N=160)
2008 Percent of Total (N=158)
1, Not useful 9 8
2 21 20
3 36 34
4 19 26
5, Very useful 15 12
Mean score 3.09 3.13
(“no answer” and “not used” responses eliminated)
Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 34%, ’08 = 38%). Lower percentage of “not used” answer in 2008 (’07 = 43%; ’08 = 36%).
![Page 21: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/21.jpg)
Question 8Describe your level of satisfaction with FAEIS components. (i.e., newsletter help desk,
data entry, report builder, instructions) 1=not satisfied; 5=very satisfied
2008
![Page 22: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/22.jpg)
Question 8 Describe your level of satisfaction with FAEIS components. (i.e., newsletter help
desk, data entry, report builder, instructions) 1=not satisfied; 5=very satisfied
Score
2007 Percent of Total (N=222)
2008 Percent of Total (N=208)
1, Not satisfied 6 5
2 9 7
3 33 33
4 32 37
5, Very satisfied 20 18
Mean score 3.50 3.56
(“Unknown,” “no answer” and “not used” responses eliminated)
Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 52%, ’08 = 55%). Similar percentage of “not used” answer (’07 = 23%; ’08 = 23.7%).
![Page 23: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/23.jpg)
2008 Annual FAEIS Evaluation Did the type of involvement (Q1) have a significant relationship with ratings of: Frequency of use (Q2) Experience with FAEIS (Q3) Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)
Respondents listing “data entry” as their only level of involvement were compared to all other levels of involvement. “Not used” and “no answer” responses for Q6, Q7 and Q8 were removed from the analysis.
There were significant differences for “Frequency of Use (Q2),” “Level of adoption” (Q5) and “Usefulness of data” (Q6), and Usefulness of FAEIS to your institution (Q7) and “Satisfaction with FAEIS components” (Q8) between the two groups. There was not a significant difference between the means of the two groups for Q3. Respondents listing “data entry” as their only level of involvement had significantly lower means for Q2, Q5, Q6, Q7 and Q8 than the other group.
![Page 24: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/24.jpg)
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Adoption of FAEIS (Q5)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Adoption of FAEIS” (Q5). As the frequency of use increases, so does the rating of the adoption of the FAEIS.
F=45.0 (P<0.05)
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
![Page 25: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/25.jpg)
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of data (Q6)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of data” (Q6). As the frequency of use increases, so does the rating of the usefulness of the data.
F=58.2 (P<0.05)
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
![Page 26: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/26.jpg)
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of FAEIS (Q7)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of FAEIS” (Q7). As the frequency of use increases, so does the rating of the usefulness of FAEIS.
F=25.2; P<0.05
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
![Page 27: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/27.jpg)
2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Satisfaction with FAEIS components (Q8)
Yes, there is a positive correlation between “Frequency of use” (Q2) and “Satisfaction with FAEIS Components” (Q8). As the frequency of use increases, so does the rating of the satisfaction with FAEIS components.
F=26.1;P<0.05
Key:How often have you used FAEIS in the past 12 months?
1=Not at all2=1-2 times3=3-4 times4=more than 4 times
![Page 28: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/28.jpg)
2008 Annual FAEIS Evaluation
Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Level of adoption (Q5)
For Level of Adoption (Q5), the “Less than One Year of Experience with FAEIS” group was significantly lower than the other two “level of experience” groups. This seems to reason – the less experience with FAEIS, the less the level of adoption of FAEIS.
![Page 29: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/29.jpg)
2008 Annual FAEIS Evaluation Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Usefulness of data (Q6) Usefulness of FAEIS to your institution (Q7) Satisfaction with FAEIS components (Q8)
There was not a significant difference between the three groups for “Usefulness of data” (Q6), “Usefulness of FAEIS to their Institution” (Q7) and “Satisfaction of FAEIS Components” (Q8) .
![Page 30: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/30.jpg)
2008 Annual FAEIS Evaluation Did the professional position (Q4) have a significant relationship with ratings of : Frequency of use (Q2) Years of experience with FAEIS (Q3) Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)
The only significant difference was with the “government” group for Frequency of use (Q2) and Level of adoption (Q5). This group had higher means than the other groups for both questions.
![Page 31: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/31.jpg)
2008 Annual FAEIS Evaluation Did the level of adoption (Q5) have a significant relationship with ratings of: Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)
Groups responding “unaware,” & “aware but do not use” to Q5 were compared to the other response groups for Q5 - “Use it, but not sure of long term use;” “Use it and it is integral to my job;” “Integral to my job and expect long term use;” & “So important as to never be without FAEIS.”
There were significant differences for all three of the questions above between the two groups. The means of the “unaware,” & “aware but do not use” groups were always lower for each of the questions when compared to the other group.
![Page 32: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/32.jpg)
2008 Annual FAEIS Evaluation
Responses by non data entry users only 2007 Mean
2008 Mean
“My level of adoption (use) of FAEIS is best described as:” 3.16 3.24
SCALE:1=unaware2=aware of it but do not use it3=Use it but not sure of long term use4=Use it and it is integral to my job5=Integral to my job and expect long term use6=So important as to never be without FAEIS
![Page 33: 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal](https://reader035.vdocuments.us/reader035/viewer/2022062422/56649f0e5503460f94c2216b/html5/thumbnails/33.jpg)
2008 Annual FAEIS Evaluation
Responses by all users 2007 Mean
2008 Mean
Rate the usefulness of the data resources in FAEIS. Scale: 1=Not useful, 5=Very useful.
3.50 3.47
Rate the usefulness of FAEIS at your institution for internal applications. Scale: 1=Not useful, 5=Very useful.
3.09 3.13
Describe your level of satisfaction with FAEIS components. Scale: 1=Not satisfied, 5=Very satisfied.
3.50 3.56