key findings from the evaluation of fcpson: years 1-3 · measures in the evaluation have captured...

Key Findings from the Evaluation of FCPSOn: Years 1-3

Jennifer R. Morrison, PhD Steven M. Ross, PhD Kelsey L. Risman, MA Alan J. Reid, PhD

June 2020

ii

© Johns Hopkins University, 2020

Contents EXECUTIVE SUMMARY: ............................................................................................. 1

Goal Area 1: Provide equitable access to learning materials/supports at home and at school. .................................................................................................................. 1

Goal Area 2: All students will demonstrate growth in development of Portrait of a Graduate attributes. ............................................................................................... 2

Goal Area 3: All FCPS teachers effectively integrate technology into instructional practice. ................................................................................................................ 2

Goal Area 4: Support all FCPS students in the safe and responsible use of technology. ............................................................................................................................ 3

Goal Area 5: Prepare students with the technology skills essential for success in tomorrow’s workforce. ........................................................................................... 4

Key Findings from the Evaluation of FCPSOn: Years 1-3 .............................................. 1

Goal Area 1: Provide equitable access to learning materials/supports at home and at school. .................................................................................................................. 4

Goal Area 2: All students will demonstrate growth in development of Portrait of a Graduate attributes. ............................................................................................. 12

Goal Area 3: All FCPS teachers effectively integrate technology into instructional practice. .............................................................................................................. 18

Goal Area 4: Support all FCPS students in the safe and responsible use of technology. .......................................................................................................................... 27

Goal Area 5: Prepare students with the technology skills essential for success in tomorrow’s workforce. ......................................................................................... 30

Appendix ................................................................................................................ 33

FCPSON KEY FINDINGS: YEARS 1-3 1


EXECUTIVE SUMMARY: Key Findings from the Evaluation of FCPSOn: Years 1-3

The current document includes key findings from the first three years of the ongoing evaluation of FCPSOn. FCPSOn is an instructional initiative in Fairfax County Public Schools that involves the distribution of personal laptops to students, targeted professional development for teachers, and strategic supports for school leadership teams that support technology integration in teaching and learning. The initiative seeks to support students and teachers as the district moves beyond high-stakes testing to emphasize students’ content area knowledge and the skills and attributes articulated in the Portrait of a Graduate.

Goal Area 1: Provide equitable access to learning materials/supports at home and at school.

Key findings suggest that this goal area is the most highly visible and accomplishable goal of the initiative, and perhaps the most universally supported goal of FCPSOn.

• Every student has access to an FCPS-provided computer. Findings

indicated that while distribution of laptops may not have been uniform across

Phase One schools, such distributions improved over time, particularly between

year 1 and year 2. Teachers and SBTSs in year two noted widespread

improvements in the management of the physical computer, and teachers in

year three focus groups reflected on distribution as being “fine” or otherwise

uneventful.

• Every student has access to learning materials/supports in both virtual

and physical learning environments at school and at home. Parent and

student questionnaire responses indicate that Internet is nearly universal at

home for Phase One students, though student survey responses also indicated

that there are still several hundred students in Phase One schools who would

benefit from more active intervention by FCPS to connect students with Internet

access.

• Every student has access to learning materials/supports in both virtual

and physical learning environments at school and at home. Guaranteed,

equitable access to learning tools was identified as a key strength or benefit of

the initiative by all stakeholder groups, including students, across each year of

implementation. The usefulness or assistive contributions of students’ personal

devices is also consistently one of the most frequently noted benefits of the

initiative, especially among students and parents. Further, students’ responses to

self-efficacy questionnaire items indicate they feel comfortable with and realize

benefits from routine computer usage for school and home learning. Over time,



Google Classroom became referenced as an important component for connecting

students with peers, their teachers, and content. Consistent use of Google

Classroom by teachers is perceived by parents, teachers, and SBTSs, particularly

in year three, as nearly eliminating several legitimate barriers to student effort

and success outside of the classroom.

Goal Area 2: All students will demonstrate growth in development of Portrait of a Graduate attributes.

In general, key findings related to Portrait of a Graduate (POG) attributes indicate that the framework has become more prominent among stakeholders in Phase One schools, but implementation still varies school-to-school. Multiple data sources confirm that students as communicators and collaborators are the most encouraged and demonstrated attributes. Further, perceptions among principals, SBTSs, and teachers regarding the development of students’ POG skills have improved throughout

the initiative.

• Every student has access to learning experiences aligned with Portrait of a Graduate outcomes. Evaluation findings suggest that Phase One schools have, overall, focused more explicitly on promoting these attributes in the third year compared to previous years, but emphasis at the school level still varies to a moderate degree. Teachers’ perceptions supported the belief that POG would contribute to student success and that they possessed confidence in its implementation. Student responses reiterated this belief, despite the variability in the ways and frequency with which technology was used in the classroom. Perceptions from many parents indicated a lack of awareness of the initiative and of its components.

• Every student will meet grade-level expectations of Portrait of a

Graduate outcomes. Based on survey responses, students generally agreed

that they are good communicators, collaborators, ethical and global citizens,

creative thinkers, and goal-directed and resilient individuals. Overwhelmingly,

the POG skills most referenced by students in focus groups, regardless of grade

level, were collaboration and communication. Similar to student respondents,

teachers were more likely to perceive an impact on communication,

collaboration, and creative and critical thinking as compared to the other

skills. Their perceptions of student’s POG skills improved across all three years of

the initiative for each of the attributes.

Goal Area 3: All FCPS teachers effectively integrate technology into instructional practice.

Over time, evaluation findings indicate that teachers have become more skilled in the integration of technology and, in year three, have increased their use of technology



for higher-order learning and in facilitating collaboration amongst students. Importantly, technology is seen as a means to provide instruction to meet the needs of individual students. Teachers conveyed improved perceptions of professional development, particularly in terms of their choice in what they participated in as well as increased opportunities for collaboration with their peers.

• Teachers will create learner-centered environments as defined by the

FCPS Learning Model. Multiple data sources indicate that the initiative allows

teachers to better meet the individual needs of students and that technology

enables students to more easily access content that is suitable for individual

goals, interests, needs, and learning styles. Teacher survey responses illustrated

increases over time in cooperative/collaborative learning and providing student

choice and differentiated learning experiences.

• Teachers will implement meaningful learning experiences as outlined

by the FCPS Learning Model. Multiple data sources from all three years of the

evaluation suggest that increased student engagement in learning is among the

most robust findings and key impacts of the initiative on students. In year one,

teachers frequently described increased engagement in terms of student

excitement over novel learning experiences. Over years two and three of the

evaluation, they strongly associated increased engagement with increased

involvement in and accountability for learning. Interviews and focus groups with

principals, SBTSs, and teachers revealed that teachers had become more skilled

over time at integrating technology to facilitate differentiation, collaborative

learning, project-based learning, and self-directed learning among students.

• Teachers will participate in professional learning that supports goal 3.

Teacher survey results indicate that teachers’ perceptions of specific professional

development topics have improved over time. This improvement in teacher’s

perceptions of professional development represents a major success of the

initiative. Teacher feedback regarding professional development was related to

improved choices for professional learning opportunities in year three as well as

more time spent in collaborative teams and other peer-to-peer learning

opportunities. In terms of professional development needs, teachers consistently

note across years the need for more peer-to-peer learning, time to lesson plan,

content-specific learning opportunities, and diverse learning options that target

teachers at variable stages of tenure and practice.

Goal Area 4: Support all FCPS students in the safe and responsible use of technology.

All stakeholders stressed the importance of students’ safe and responsible use of technology. While teachers and parents expressed concerns in off-task behavior in early years, it appears that these concerns have diminished over time. Relatedly, year three



principal interviews indicated a more focused approach to digital citizenship in school buildings.

• Every student will develop awareness of safe and responsible use of

technology. Survey and focus group data suggest that responsible use of

technology, including maintaining on-task behavior through classroom

management, is a persistent concern for teachers and parents, though its

prominence in feedback from adults diminished from year one to year three.

• Every student will follow digital citizenship with best practices

including maintaining a healthy media balance and well-being. Digital

citizenship is identified by educators each year of the evaluation as an area

where more guidance from the county is needed. Based on interviews with

principals, it appears that more structured and intentional approaches to digital

citizenship in school buildings emerged in year three of the initiative. Findings

related to maintaining a healthy balance of media use are mixed. Students and

teachers indicated each year in focus groups that it seemed as though students

were on their devices more each year than the year prior. However, focus group,

observations, and survey data in year three indicate that teachers emerged in

year three as more judicious users of technology.

• Every student will follow best practices of online privacy and

security. Evaluation findings from all three years do not suggest that

stakeholders have pervasive concerns about students’ online privacy and

security. Evaluation findings indicate that FCPS-issued laptops appear to be

appropriately restrictive and are perhaps too restrictive, as students and teachers

in years two and three of the evaluation indicated that device restrictions inhibit

some school-related content and programs.

Goal Area 5: Prepare students with the technology skills essential for success in tomorrow’s workforce.

Few data sources directly assessed goal area 5 in early years of the evaluation. For the current summary, we have extracted findings mostly from qualitative data sources (e.g., interviews and focus groups) where related themes emerged prominently in responses to other questions.

• Every student is proficient with "real world" tools that are essential for

workplace success. Educators in all three years of the evaluation were more

likely to emphasize positive impact of FCPSOn on students’ process skills such as

communication and problem-solving, more than on technical skills such as coding

and basic troubleshooting. Educators noted in all three years of the evaluation

that while students’ technical skills are described as “improving,” their lack of



basic technical skills (i.e., typing) are seen as a barrier to successful

implementation.



Key Findings from the Evaluation of FCPSOn: Years 1-3

The current document includes key findings from the first three years of the ongoing evaluation of FCPSOn. FCPSOn is an instructional initiative in Fairfax County Public Schools that involves the distribution of personal laptops to students, targeted professional development for teachers, and strategic supports for school leadership teams that support technology integration in teaching and learning (see Figure 1). The initiative seeks to support students and teachers as the district moves beyond high-stakes testing to emphasize students’ content area knowledge and the skills and attributes articulated in the Portrait of a Graduate.

Figure 1. FCPSOn initiative goals.

FCPSOn implementation has occurred in phases. Phase One of the initiative

(2016-17) included the entire Chantilly Pyramid and six additional high schools (eLearning Backpack schools) across the district—Annandale, Lee, Mount Vernon, Falls Church, Justice, and Fairfax Adult High Schools. The Chantilly Pyramid schools implemented FCPSOn in all grades during this first year whereas the eLearning Backpack (eLB) schools implemented the initiative in a phased approach across years. Phase Two of implementation (2019-20) involved all FCPS high schools. This document presents key findings from the evaluation of the first three years of the initiative, which occurred at Phase One schools only. Table 1. below summarizes the methodology and data sources employed in each year of the evaluation.



Table 1. Overview of data collection activities by year.

Year 1

(2016-17)

Year 2

(2017-18)

Year 3

(2018-19)

n n n

Participants

Chantilly Pyramid

eLearning Backpack

(phased implementation)

Questionnaires

Student survey 7,125 7,059

Teacher survey 657 1,004 1,458

Parent survey 1,147 1,940

Focus Groups

Student 3 7 7

Teacher 4 7 7

Parent 2 3 4

SBTS 2

Principal 2

Interviews

SBTS 7 7

Principal 15 13

Observations

Classroom 22 44

In the following sections, we present findings from prior year’s evaluations according to the five goal areas. Table 2. presents data sources for goal areas.



Table 2. Data sources aligned to goal areas. Data source

Goal 1. Provide equitable access to learning materials/supports at home and at school.

All surveys; all interviews and focus groups

Goal 2. All students will demonstrate growth in

development of Portrait of a Graduate attributes. All surveys; observations; all interviews

and focus groups

Goal 3. All FCPS teachers effectively integrate technology

into instructional practice.

All surveys; observations; all interviews

and focus groups

Goal 4. Support all FCPS students in the safe and

responsible use of technology.

Teacher and parent surveys;

observations; all interviews and focus groups

Goal 5. Prepare students with technology skills essential for success in tomorrow’s workforce.

Student and parent surveys; all interviews and focus groups

It is important to note that revisions were made to questionnaires each year (see Appendix) and that only teachers were surveyed in year one. All populations were surveyed in year two. Goal areas 1 and 3 are the most frequently represented goal areas in each year; goal areas 4 and 5 are the least represented. For the most part, each goal area received more attention, in terms of total related questionnaire items, each year as the evaluation was expanded and refined. In terms of specific differences in questionnaire items across the three years of the evaluation, only select items were asked across all three years. Specific items and timing are presented in Appendix A.

In the next section, we present key findings from all three years of the evaluation of FCPSOn in Phase One schools. Key findings emerged from data sources described above and are presented by goal area.



Goal Area 1: Provide equitable access to learning materials/supports at home and at school. Sub-goals for this goal area include that every student has access to an FCPS-provided computer, Wi-Fi at home, and learning materials and supports in virtual and physical learning environments at home and school. This goal area applies to the distribution of devices and providing supports and additional resources that ensure not only that access is assured but that students are successful in their new learning environment. At school, that means teachers who are prepared to leverage virtual classroom spaces and digital tools to ensure flexible access to materials. At home, that means students have access to a computer and Internet and are prepared to use their computer to accomplish school-related tasks.

Key findings suggest that this goal area is the most highly visible and accomplishable goal of the initiative, and perhaps the most universally supported goal of FCPSOn. We present findings related to each of the sub-goals in the following sections. Every student has access to an FCPS-provided computer. Interviews with principals and SBTSs suggest that distribution of laptops was managed at the school level and not uniform across Phase One schools. Further, laptop distribution practices varied among eLearning Backpack schools due to the multi-year grant and laptop provisioning. Within Phase One schools, it appears that some schools hosted informational sessions for parents and students prior to distribution, though focus groups with parents in year one indicated that some were unaware of laptop distribution prior to the event. Some evidence suggests that physical distribution had improved over time, particularly between year 1 and year 2. Teachers and SBTSs in year two noted widespread improvements in the management of the physical computer, and teachers in year three focus groups reflected on distribution as being “fine” or otherwise uneventful.

While our evaluation did not directly assess distribution of laptops to students, FCPS

reported that all students in Phase One schools received laptops each year (see Table 3.). There were some students, though, that opted out of receiving an FCPS-issued laptop. Table 3. Laptops purchased centrally for students in FCPSOn Phase One schools. Laptop

Distribution SY 16-17

Laptop Distribution SY 17-18

Laptop Distribution SY 18-19

Chantilly Pyramid1



Brookfield ES 449 449 449 Greenbriar East 516 516 516

Greenbriar West 640 640 640 Lees Corner ES 435 435 435

Oak Hill ES 587 587 587 Poplar Tree ES 458 458 458

Franklin MS 886 886 886 Rocky Run MS 1252 1252 1252

Chantilly HS 2746 2746 2746 eLearning Backpack

Annandale HS 1129 1305 1925 Falls Church HS 995 1073 1743

Justice HS 1643 1724 2520 Lee HS 952 1023 1593

Mount Vernon HS 2076 2076 2500 1New laptops were purchased centrally and distributed in 2016-17. Schools would have purchased additional laptops due to enrollment increases in 2017-18 and 2018-19. Every student has access to Wi-Fi at home.

Parents (year three only) and students (years two and three) were asked via a

questionnaire item if they currently had Internet at home (see Figure 2). In year two, 6,615 students responded to this item and in year three, 4,950 students responded, as did 1,664 parents. Responses indicate that Internet is nearly universal at home for Phase One students, though student survey responses also indicated that several hundred students in Phase One schools may benefit from more active intervention by FCPS to connect students with Internet access.

Figure 2. Proportion of students with Internet at home.

97.2% 97.1% 99.2%

2.8% 2.9% 0.8%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

Studentsn = 6615

Studentsn = 4950

Parentsn = 1664

Year 2 Year 3

Internet at Home

Yes No



FCPS provided MiFi (mobile hotspot) to some students each year of the initiative (see Table 4). Table 4.

Distribution of MiFi to students in Phase One Schools. MiFi Distribution

SY 16-17 MiFi Distribution

SY 17-18 MiFi Distribution

SY 18-19

Chantilly Pyramid Brookfield ES 75 75 75

Greenbriar East 6 6 6 Greenbriar West 3 3 3 Lees Corner ES 3 3 3

Oak Hill ES 1 1 1 Poplar Tree ES 5 5 5

Franklin MS 20 26 31 Rocky Run MS 6 6 6

Chantilly HS 28 28 28 eLearning Backpack

Annandale HS 40 40 50 Falls Church HS 12 22 22

Justice HS 12 12 23 Lee HS 24 49 79

Mount Vernon HS 16 27 27

Every student has access to learning materials/supports in both virtual and physical learning environments at school and at home.

Guaranteed, equitable access to learning tools was identified as a key strength or benefit of the initiative by all stakeholder groups, including students, across each year of implementation. In fact, in all three years of the evaluation, the importance of equitable access was frequently noted in focus groups and interviews with educators, and in open-ended responses in the teacher and parent surveys. Roughly one in five teachers commented in the year two teacher survey that it was a major strength of the initiative. In year three student and teacher focus groups, learning accessibility was the most frequently noted impact of the initiative on student learning.

The usefulness or assistive contributions of students’ personal devices is also

consistently one of the most frequently noted benefits of the initiative, especially among students and parents. It was the most frequent response in students’ open-ended comments in year two (27% of open-ended responses mentioned the laptop is helpful) and year three (30% of open-ended responses mentioned), and was the most



frequently noted positive impact on student learning by parents in year three focus groups.

Responses from Phase One students related to technology beliefs appear to be

relatively stable between year two and year three (see Figure 3). We did observe slight increases in the proportion of students who strongly agreed to statements between years. Self-efficacy responses indicate students feel comfortable with and realize benefits from routine computer usage for school and home learning.

Figure 3. Yearly comparison of students’ questionnaire responses regarding technology beliefs. Note: + indicates < 5.0%

In year two of the evaluation, interviews, focus groups, and first-hand

observations began to identify the digital classroom (Google Classroom) as a pivotal point of connection between students and teachers and between students and content/materials. Consistent use of Google Classroom by teachers is perceived, by parents, teachers, and SBTSs particularly in year three, as nearly eliminating several legitimate barriers to student effort and success outside of the classroom. Relatedly, Phase One teachers indicated extremely high agreement with questionnaire prompts regarding to the role of technology in instructional practices and planning/administration (see Figure 4). Levels of agreement increased over time, particularly from year one to two.

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

6.3%

6.1%

+

+

12.0%

12.9%

13.6%

14.9%

12.7%

13.6%

15.9%

15.2%

35.2%

41.6%

37.7%

39.2%

45.5%

48.2%

39.8%

45.0%

42.8%

36.1%

40.1%

36.1%

30.7%

26.5%

39.5%

34.1%

Year 3n = 5564

Year 2n = 7283

Year 3

n = 5564

Year 2n = 7283

Year 3

n = 5564

Year 2n = 7283

Year 3

n = 5564

Year 2

n = 7283

Students' Technology Beliefs

N/A or No Opinion Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

Without a computer, it would be difficult to be successful in school.

My computer works well.

My computer is an important part of every school day.

My computer makes turning in homework and completing assignments easy.



Figure 4. Teachers’ questionnaire responses regarding the role of technology in their practices. Note: + indicates < 5.0%

Teacher survey responses also suggest that usage of electronic assessments has

been fairly frequent and consistent in years two and three (see Figure 5). Teachers were, though, more likely to use digital resources like online textbooks and web-based platforms in year three as compared with year two (36.7% vs. 53.5% “frequently”).

Figure 5. Frequency of teaching practices using technology. Note: + indicates < 5.0%

Relatedly, Phase One teachers’ technology efficacy was high in both years while

slightly increasing from year two to year three (see Figure 6).

+

+

+

+

+

+

+

+

6.6%

+

+

8.1%

10.6%

9.9%

16.7%

6.3%

7.1%

24.7%

36.7%

39.3%

29.8%

38.0%

37.8%

26.9%

48.4%

46.8%

46.4%

53.2%

51.6%

39.9%

Year 3

n = 790

Year 2

n = 1057

Year 1n = 744

Year 3n = 790

Year 2n = 1060

Year 1n = 744

Technology and Teacher Practices

Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

Technology is an integral part of my instructional planning and administration this year.

Technology is an integral part of my instructional practices this year.

+

11.3%

+

6.1%

8.9%

17.2%

14.6%

11.8%

36.8%

34.9%

36.5%

33.2%

53.5%

36.7%

44.7%

48.8%

Year 3n = 794

Year 2

n = 1014

Year 3n = 789

Year 2

n = 995

Frequency of Teaching Practices

Never Rarely Moderately Frequently

Deliver electronic formative or summative assessments?

Use digital resources (online textbooks, web-based platforms, software programs, etc.)?



Figure 6. Yearly comparison of teachers’ questionnaire responses related to technology self-efficacy. Note: + indicates < 5.0%

An increase in flexible access to learning (i.e., time and place) was noted by multiple stakeholder groups in focus groups and interviews, particularly in year three of the initiative. As will be discussed in more detail within Goal 3, in year three, Phase One teachers indicated increased levels of agreement compared to year two (> 10 points) to receiving sufficient professional development in multiple areas, including in creating a learner-centered physical and virtual environment. These are the learning environments where students’ flexible access is key.

Increase in student accountability for learning is a key finding across the three

years of the evaluation. Increased accountability, particularly in year three of the evaluation, is described as directly related to more frequent access to school-related materials. There are “no excuses” for students to not be involved in their learning. By year three, students appeared to meet expectations established by their teachers related to increased accountability and involvement in their learning.

Interestingly, we did see changes over time in the frequency students used their

laptops for various activities. As shown in Figure 7, students were less likely to use their laptops for communication, assessment, and media-related activities during year three than year two. An important consideration here is that in year three, the questionnaire specifically asked the frequency the media-related activities were for supporting learning, whereas year two items were general in nature. That is, frequency of use in year two may have included media use for both personal (non-educational) and learning-related use.

+

+

+

+

5.9%

9.5%

+

14.5%

18.4%

+5.4%

+

47.3%

48.9%

45.1%

49.5%

30.5%

21.5%

48.4%

45.0%

Year 3

n = 829

Year 2

n = 1081

Year 3

n = 827

Year 2n = 1082

Teachers' Technology Self-efficacy


With proper training, I am confident in my ability to learn new digital resources and tools.

I can deal with most technical difficulties I encounter.



Figure 7. Frequency of student device use in years two and three.

27.3%

5.6%

33.1%

31.5%

20.4%

37.1%

5.6%

+

6.0%

14.5%

7.8%

17.3%

19.1%

9.1%

14.3%

25.3%

22.7%

13.8%

8.1%

32.3%

37.8%

7.8%

19.9%

19.2%

20.7%

9.2%

12.8%

10.5%

20.1%

11.4%

16.1%

12.9%

20.0%

19.4%

18.7%

30.8%

31.1%

21.7%

19.7%

15.8%

15.8%

18.4%

19.1%

25.1%

13.8%

15.6%

13.6%

15.8%

12.9%

10.9%

10.1%

26.0%

32.4%

29.8%

16.4%

17.4%

14.1%

12.6%

30.5%

27.5%

17.0%

61.6%

29.5%

34.8%

25.7%

75.9%

18.4%

75.7%

13.2%

77.0%

34.6%

35.2%

51.5%

14.6%

51.5%

12.0%

67.8%

38.0%

56.7%

Year 3

n = 5050

Year 2

n = 6700

Year 3n = 5050

Year 3n = 5050

Year 3

n = 5050

Year 2

n = 6700

Year 3n = 5050

Year 2n = 6700

Year 3

n = 5050

Year 2

n = 6700

Year 3n = 3989

Year 3n = 5050

Year 2

n = 6700

Year 3

n = 5050

Year 2

n = 6700

Year 3n = 5050

Year 2n = 6700

Year 3

n = 5050

Year 2

n = 6700

Frequency of student laptop use

Never Once a week 2-3 Times Per Week Almost every day Daily

Work on school work or complete homework at home.

Design PowerPoint presentations, drawings, or web pages.

Take a test or quiz.

Submit homework.

View your grades.

Play educational games / games that support your learning.

Watch TV or YouTube vidoes that support your learning.

Listen to music that supports your learning.

Access Google Classroom / G Suite for Education.

Access Blackboard (FCPS 24-7).

Type notes during class.



Note: + indicates < 5.0%

Teachers also indicated high agreement (over 75%) each year that students had improved in their use of technology as a learning tool (see Figure 8).

Figure 8. Teachers’ perceptions of students’ use of technology for learning. Note: + indicates < 5.0%

Parent perceptions of their child’s experiences in an FCPSOn school were

assessed each year of the evaluation. Outcomes related to goal area 1 include whether the school-issued laptop was integral to their child’s learning and the degree to which students were motivated to use technology (see Figure 9). Questionnaire data indicate that parent perceptions declined slightly between years two and three of the initiative. However, this finding may relate more to the differences in the parent samples that responded to the questionnaire across years than to perceptions of FCPSOn becoming less favorable.

Figure 9. Yearly comparison of parents’ technology perceptions. Note: + indicates < 5.0%

+

+

+

+

+

+

16.0%

14.9%

14.3%

47.1%

51.3%

44.1%

31.7%

29.1%

37.9%

Year 3n = 780

Year 2n = 1040

Year 1n = 742

Student Impact


My students have improved in their use of technology as a learning tool this year.

5.7%

5.4%

0.0%

+

+

+

+

0.0%

6.5%

5.4%

19.8%

16.1%

0.0%

16.8%

13.9%

34.0%

27.1%

0.0%

35.4%

28.1%

36.8%

47.2%

0.0%

36.7%

48.2%

Year 3n = 1330

Year 2

n = 517

Q1

Year 3n = 1331

Year 2

n = 517

Parents' Technology Perceptions

Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree

The laptop provided by FCPS is an integral part of my child's learning experiences.

My child is motivated to use the laptop to complete homework, assignments, and other school-

related tasks.



Goal Area 2: All students will demonstrate growth in development of Portrait of a Graduate attributes.

Sub-goals for this goal area include that every student has access to learning experiences aligned with Portrait of a Graduate outcomes and that every student will meet grade-level expectations for Portrait of a Graduate outcomes as measured by their performance on an end of year Portrait of a Graduate presentations of Learning (in development). These skills include students as communicators, collaborators, ethical and global citizens, creative thinkers, and goal-directed and resilient individuals. The following sections detail the findings related to each of these sub-goals.

In general, key findings related to Portrait of a Graduate (POG) attributes

indicate that the framework has become more prominent among stakeholders in Phase One schools, but implementation still varies school-to-school. Multiple data sources confirm that students as communicators and collaborators are the most encouraged and demonstrated attributes. Further, perceptions among principals, SBTSs, and teachers regarding the development of students’ POG skills have improved throughout

the initiative.

Every student has access to learning experiences aligned with Portrait of a Graduate outcomes.

The POG framework was implemented inconsistently at the school level. Some principal respondents indicated that they have not explicitly implemented POG, whereas others work to make it more visible by integrating the attributes into the school improvement plan or by choosing one attribute to focus on for a full school year. Most principals implied that their school aimed to create an awareness of POG practices, viewing it more as a “framework” rather than something to formally implement and measure. Responses from SBTSs reiterated this finding; POG attributes were tied directly to teaching and learning rubrics in most Chantilly Pyramid schools but were not as formalized in eLearning Backpack schools. According to SBTS respondents, improvement in POG attributes was directly related to increased student choice or project-based learning. Despite the inconsistency in implementation practices, nearly all teachers were knowledgeable of the program (see Figure 10).

+

+

14.2%

21.2%

84.6%

77.1%

Year 3

Year 2

Teachers' Knowledge of Portrait of a Graduate

No Somewhat Yes

I am knowledgeable of the FCPS Portrait of a Graduate.



Figure 10. Teacher’s knowledge of Portrait of a Graduate for years two and three. Note: + indicates < 5.0%

Teachers predominantly expressed positive perceptions of POG. Figure 11 displays how teachers were confident that POG attributes would lead to student success and that teachers felt confident planning student opportunities around POG. Teachers were slightly less optimistic, however, that technology would be a key component in their approach to implementing POG.

Figure 11. Teachers’ perceptions of POG attributes, planning, and approach. Note: + indicates < 5.0%

Most students agreed that they used their computers to create products that show what they have learned. And, there was a slight uptick in agreement from years two and three that students felt their computer helped them become more responsible for school success (see Figure 12).

+

+

+

5.6%

+

+

22.8%

17.7%

16.4%

39.1%

46.9%

44.5%

30.7%

30.7%

35.8%

Year 3n = 780

Year 3

n = 823

Year 3

n = 842

Teachers' Perceptions of Portrait of a Graduate


I am confident that Portrait of a Graduate attributes will contribute to greater success for FCPS

students, in later education and work experiences.

I feel confident planning opportunities for students to practice and develop Portrait of a Graduate

skills.

Technology is a key component in my approach to cultivating Portrait of a Graduate attributes and

outcomes in students.



Figure 12. Students’ technology use in years two and three. Note: + indicates < 5.0%

Students reported some variability in the frequency with which they used their laptops for different types of activities. Figure 13 illustrates how students’ personal laptops were used and how this changed dramatically between years two and three. For example, in year two, students indicated that their laptops were used most frequently to receive feedback from other students and from their teachers. This frequency dropped substantially in year three, where students instead reported that communication and collaboration with others drove the most frequent use of their laptops.

+

+

+

+

+

+

7.0%

7.6%

+

27.2%

28.2%

13.6%

37.0%

37.3%

49.3%

20.6%

19.6%

27.0%

Year 3

n = 5564

Year 2

n = 7283

Year 3

n = 5564

Students' Technology Use

I Don't Know Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

I use my computer to create products that show what I've learned

My computer helps me to be more responsible for my success in school.

7.3%

33.5%

12.8%

33.5%

13.8%

16.1%

23.7%

26.8%

23.7%

20.8%

22.9%

21.0%

16.2%

23.9%

21.9%

16.2%

12.5%

19.4%

28.0%

27.3%

13.7%

20.0%

20.1%

13.7%

11.0%

19.8%

25.7%

51.7%

12.8%

16.4%

57.9%

12.8%

56.6%

26.2%

12.5% 11.0%

Year 3

n = 5050

Year 2

n = 6700

Year 3n = 5050

Year 3

n = 5050

Year 2n = 6700

Year 3

n = 5050

Year 2

n = 6700

Year 3n = 5050

Frequency of student laptop use

Never Once a week 2-3 Times Per Week

Communicate with other students or my teacher.

Receive feedback from other students.

Receive feedback from teachers.

Collaborate with other students from home.

Collaborate with other students during class.

Communicate with other students or my teacher.

Receive feedback from other students.

Receive feedback from teachers.

Collaborate with other students from home.

Collaborate with other students during class.



Figure 13. Students’ reported frequency of personal laptop use for activities that support Portrait of a Graduate skill development in years two and three.

Parents lacked awareness of the FCPSOn initiative and how the POG attributes apply to their child(ren). Figure 14 shows that although most parent respondents were aware that their child(ren) attended an FCPSOn Phase One school, a fewer number were knowledgeable of the FCPSOn initiative. Similarly, roughly one-third of parent respondents were unaware of the POG attributes.

Figure 14. Parents’ awareness and perceptions of POG and FCPSOn.

Every student will meet grade-level expectations of Portrait of a Graduate outcomes.

Students responded to questionnaire items that focused on POG attributes. Based on survey responses, students generally agreed that they are good communicators, collaborators, ethical and global citizens, creative thinkers, and goal-directed and resilient individuals. Overwhelmingly, the POG skills most referenced by students in focus groups, regardless of grade level, were collaboration and communication. Students often noted increased productivity during group work, ease of access in communicating with students and teachers, and better project development. Students also noted an increased sense of availability on behalf of their teachers and themselves and the ability to easily communicate with teachers regarding assignments and class. This is due in part to the use of Google Classroom to communicate with others and to access learning materials outside of class. Of all POG skills, the attribute that students agreed with the least is the tendency to use time wisely while on their own. This finding is supported by other data from interviews and

16.7%

13.7%

0.0%

22.3%

19.2%

0.0%

34.1%

29.6%

39.3%

37.9%

0.0%

15.6%

12.2%

0.0%

31.1%

31.7%

44.0%

48.4%

0.0%

62.1%

68.6%

0.0%

34.8%

38.7%

Year 3

n = 1752

Year 2n = 1164

Year 3

n = 1752

Year 2

n = 1164

Year 3

n = 1967

Year 2

n = 1164

Parents' FCPSOn Perceptions

No Somewhat Yes

I am knowledgeable of the Portrait of a Graduate attributes as they apply to my child(ren).

Prior to completing this survey, I was aware my child is enrolled at a FCPSOn Phase One School.

I am knowledgeable of the FCPSOn initiative.



focus groups, which suggests some students are having difficulty staying on task with their personal devices.

The teacher questionnaire asked classroom teachers the degree to which they felt that technology had impacted their students’ POG skills. Although teachers initially reported a high level of agreement that students embodied these attributes during the first year, these perceptions were strengthened throughout Phase One (see Figure 15).

Figure 15. Yearly comparison of teachers’ perceptions of students’ Portrait of a Graduate skills.

++

+

+

+

+

+

+

+

+

+

+

+

+

++

+

+

+

+

+

+

+

+

+

+

+

5.2%

5.3%

+

+

5.4%

7.1%

6.6%

+

5.3%

5.0%

5.1%

6.6%

6.3%

17.6%

28.5%

24.3%

17.3%

26.0%

26.1%

31.4%

36.7%

38.4%

18.9%

26.7%

23.5%

25.7%

34.0%

31.0%

52.0%

47.3%

51.3%

52.7%

50.0%

48.0%

43.0%

40.2%

39.6%

51.4%

48.4%

49.5%

48.9%

42.5%

47.3%

23.8%

15.3%

17.8%

25.2%

15.4%

21.0%

15.8%

11.1%

12.9%

22.8%

15.4%

20.8%

16.6%

12.7%

14.2%

Year 3n = 790

Year 2n = 1068

Year 1

n = 742

Year 3n = 790

Year 2

n = 1068

Year 1n = 742

Year 3

n = 790

Year 2n = 1068

Year 1n = 742

Year 3n = 790

Year 2n = 1068

Year 1

n = 742

Year 3n = 790

Year 2

n = 1068

Year 1n = 742

Teacher Perceptions of Students' Portrait of a Graduate Skills Improvement

Not Applicable Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

Being goal directed and resilient individuals.

Creative and critical thinking.

Ethical and global citizenship.

Collaboration.

Communication.




Similar to student respondents, teachers in both groups were more likely to

perceive an impact on communication, collaboration, and creative and critical thinking as compared to the other skills. Teachers’ perceptions of student’s POG skills improved across all three years of the initiative for each of the attributes, with the largest gains from year one to year three taking place in students’ communication and collaboration skills (+6.7 pts and +8.9 pts, respectively), and the smallest gains in students being goal directed and resilient individuals and creative and critical thinkers (+4.0 pts and +3.9 pts, respectively).



Goal Area 3: All FCPS teachers effectively integrate technology into instructional practice.

Sub-goals for this goal area include that teachers will create learner-centered environments and implement meaningful learning experiences, as defined/outlined by the FCPS Learning Model, and will participate in professional learning experiences that support learner-centered environments and meaningful learning experiences. This goal area applies to the actual teaching practices of teachers and learning experiences of students, and the district- and school-level supports for teachers and students that cultivate effective technology integration practices by teachers. All three years of the evaluation have included more items intended to capture goal 3 outcomes than any other goal area. Teachers will create learner-centered environments as defined by the FCPS Learning Model.

The FCPS Learning Model posits that learner-centered environments are places where students feel a part of a positive and culturally responsive learning environment; students engage in learning based on their goals, strengths, needs, interests, and learning styles; and students have access to learning in physical and virtual learning environments.

Multiple data sources indicate that the initiative allows teachers to better meet the individual needs of students and that technology enables students to more easily access content that is suitable for individual goals, interests, needs, and learning preferences. Interviews with SBTSs and principals in year one suggested that English-language learners were particularly and immediately positively impacted by the initiative. In year two, educators were more likely mention multiple student populations, including gifted and talented students and students with physical disabilities, that benefit from new ways of learning. In year three, educators emphasized meeting individual student needs in a broader and more inclusive sense—that is, all students regardless of any special education status, were significantly more likely to be provided more personalized learning experiences.

In focus groups, teachers elaborated on the changes in instructional practice that

result in more student-centered learning experiences. In year one, teachers at all three grade levels frequently mentioned increased or easier differentiation and communication with students, including usage of email and formative feedback, and emerging flexibility in time, place, and pace of completing assignments. In year two of the evaluation, teachers again frequently described their classroom as “more flexible” in terms of how students demonstrate learning and the place and pace of completing tasks. In year three, teachers emphasized increased student choice and student-directed learning.



Teacher survey responses (see Figure 16) provide more information related to teachers’ instructional practices each year of the initiative. Increases in cooperative/collaborative learning are notable, as are the percentage of teachers who report providing student choice and differentiated learning experiences moderately or frequently.

Figure 16. Frequency of Teaching Practices. Note: + indicates < 5.0%

Other teacher survey items indicate that teacher attitudes toward technology and learner-center environments were stable in years two and three of the initiative. For example, over three-quarters of teachers in both years agreed that the use of

6.3%

8.1%

+

+

+

+

+

7.0%

7.1%

10.0%

+

30.6%

27.7%

+

+

17.9%

18.6%

7.6%

4.5%

8.1%

33.9%

27.0%

31.5%

7.5%

+ 6.4%

35.2%

36.4%

40.0%

33.5%

42.5%

50.3%

43.4%

42.2%

41.4%

41.4%

46.5%

41.7%

48.3%

43.0%

27.8%

27.8%

56.1%

64.2%

36.2%

29.1%

48.4%

52.7%

48.5%

17.8%

19.4%

16.8%

43.8%

49.7%

Year 3n = 790

Year 2n = 1050

Year 3n = 795

Year 2

n = 1060

Year 1

n = 744

Year 3

n = 784

Year 3

n = 792

Year 2

n = 1049

Year 1

n = 744

Year 3

n = 776

Year 2

n = 1038

Year 1

n = 744

Year 3

n = 797

Year 2

n = 1057



Facilitate differentiated learning experiences?

Promote individualized learning (e.g., students working alone, with or without a computer)?

Provide student choice in topic or tool used to demonstrate learning?

Provide whole class instruction or lecture of 20 minutes or longer?

Facilitate cooperative/collaborative learning?

Involve students in designing their own learning experiences ?



technology is supportive of personalizing the time, place, path, and pace of instruction for students. Teachers will implement meaningful learning experiences as outlined by the FCPS Learning Model.

The FCPS Learning Model posits that meaningful learning experiences are those where students engage in inquiry-based learning opportunities that foster curiosity and require students to develop communication and collaboration skills, and where students' motivation and engagement are increased when they confront and make contributions to solving real-world problems.

Multiple data sources from all three years of the evaluation suggest that increased student engagement in learning is among the most robust findings and key impacts of the initiative on students. In year one focus groups, parents described increased engagement in terms of excitement to use computer programs. Parents in years two and three described their children as responsible, accountable students who are highly engaged with learning because it is so accessible.

Teacher feedback in focus groups mirrors parent perceptions (see Figure 17). In year one, teachers frequently described increased engagement in terms of student excitement over novel learning experiences. Over years two and three of the evaluation, they strongly associated increased engagement with increased involvement in and accountability for learning. Teacher survey responses affirm that teachers were overwhelmingly likely each year to agree that technology contributes positively to student engagement, with notable increased agreement between years two and three.

Figure 17. Student Engagement Items from the Teacher Questionnaire. Note: + indicates < 5.0%

In year one focus groups, students affirmed the impressions of parents and educators by linking their interest in the initiative and increased engagement to

+

+

+

5.5%

6.5%

+

14.4%

27.9%

22.4%

42.8%

43.0%

44.6%

34.4%

20.8%

27.1%

Year 3n = 783

Year 2

n = 1044

Year 1

n = 742

Student Engagement


The student engagement in my classroom has improved this year.

Technology has contributed positively to student engagement in my classroom this year.



excitement over computer programs and digital media. In year two, student engagement was frequently associated with increased creative opportunities and project-based learning that involved using their personal device. Students also conveyed their appreciation for how their device helps them stay organized and complete school-related tasks. In year three of the evaluation, high engagement was most frequently explained in terms of how much easier their personal device makes accessing school content and completing school-related tasks. Students also emphasized increased personalization and independence as positive impacts on their learning in year three. Relatedly, the majority of students indicated agreement on the survey that classes are more interesting when a computer is used for learning (year two: 71.6%, year three: 70.6%). Multiple data sources also suggest that teachers’ tendency to create and/or facilitate meaningful learning experiences emerged prominently in year three of the initiative. While principals and SBTSs described their teachers in terms of growth each year, they were more likely to describe specific teacher practices in years two and three that reflect goal 3 outcomes. For example, in year two, teachers were described as introducing more opportunities for student communication and collaboration, and as emerging classroom “facilitators.” In year three, teachers were described as competent computer users who were comfortable with a variety of instructional approaches that involved technology. Teacher survey items (see Figure 18) reflect teachers’ agreement that the initiative supports instructional practices, and that the majority of teachers feels they are efficacious technology users.

Figure 18. Teachers' Technology Self-Efficacy. Note: + indicates < 5.0%

+

+

++

+

+

+

+

+

+

14.8%

18.6%

10.3%

6.6%

12.7%

46.5%

44.7%

44.5%

51.0%

47.4%

33.3%

32.2%

43.4%

41.2%

37.8%

Year 3n = 822

Year 2

n = 1068

Year 3

n = 827

Year 2

n = 1081

Year 3n = 821



FCPSOn supports the use of a broad array of instructional strategies to support my students’

learning.

Integrating technology into instruction supports learning.

I am confident that I integrate 1:1 technology effectively in my classroom.



In interviews and focus groups, educators and students elaborated on the specific changes to instructional practices observed in each year of the initiative. Key findings include:

• According to principals and SBTSs in year one, teachers were “more willing to

take risks” and classrooms had become decentralized; students noted increased

usage of educational software, and parents noted increased communication

between teachers and students.

• In year two, principals and SBTSs again noted the decentralized classroom and

an increase in small-group instruction and student-directed learning, as well as

increased student choice and more personalized, communicative experiences;

teachers again noted increased feedback and communication with students, and

increased personalized learning, and they described their classrooms as “more

flexible;” students also noted “more flexibility” and choice in how they

demonstrate learning, and described more collaborative and project-based

learning experiences.

• In year three, SBTSs and principals noted increased differentiation, collaborative

learning, project-based learning, self-directed learning among students and, in

high school, increased usage of a “workshop model;” teachers are described as

“more creative” and “smarter users” of technology, who provide more feedback,

and have higher expectations for student accountability; the centrality of Google

Classroom to classroom management emerged prominently in year three

conversations about instructional practice with SBTSs, teachers, and students;

students also again described increased flexibility and student choice in learning.

Teacher survey items (see Figure 19) affirm that teachers have employed more

project-based learning and cross-curricular connections in year three than in year one.

6.4%

6.1%

11.6%

+

+

+

25.1%

21.7%

30.3%

19.4%

14.2%

24.1%

42.7%

44.6%

39.0%

44.6%

46.5%

40.9%

25.8%

27.6%

19.1%

33.3%

37.3%

30.2%

Year 3n = 780

Year 2

n = 1036

Year 1n = 743

Year 3

n = 784

Year 2

n = 1053

Year 1n = 744



Fostering cross-curricular connections

Facilitate project-based or other inquiry-based approaches to learning



Figure 19. Frequency of Teaching Practices related to IBL and Cross-Curricular Connections. Note: + indicates < 5.0%

Stakeholders were prompted each year to describe what, if any, impact the initiative had on student achievement each year. While survey data suggest that the majority of teachers agree that technology contributed positively to student achievement (see Figure 20), interviews and focus groups each year suggest that educators were more likely to talk about positive impacts on how students learn. The most prominent impact noted by educators is an increase in student-directed learning. Educators also described particularly positive impacts on writing skills due to ease of revision and feedback and in science classrooms through virtual labs and simulations.

Figure 20. Teacher Perceptions of Impact on Student Achievement. Note: + indicates < 5.0%

Teachers will participate in professional learning that supports 3.1 and 3.2. We identified two key changes in how teachers were supported by principals and SBTS over the first three years of the initiative.

• Principals began the initiative as “cheerleaders,” taking a highly flexible and

sidelined approach to classroom implementation while supporting the logistics

of device distribution and cultivating positive culture around technology

integration. By year three, principals had established more concrete

expectations for teachers and took a more active role in identifying and

meeting specific teacher needs in their building.

• The SBTS role shifted from primarily supporting teachers’ general knowledge

of computers and digital programs in year one, to more specific and

intentional instructional coaching of teachers one-on-one or in small groups in

year three.

Teacher survey items (see Figure 21) indicate that the majority of teachers perceived feeling informed of expectations associated with being in a Phase One school, and that their school culture supported technology-enhanced instruction.

+

+

5.0%

+

17.0%

18.6%

45.0%

44.7%

31.4%

32.2%

Year 3

n = 783

Year 2

n = 1076

Student Impact


Technology has contributed positively to student achievement in my classroom this year.



Figure 21. Teachers' Technology Self-Efficacy (continued). Note: + indicates < 5.0%

Teacher survey items further described teacher perceptions of professional development in year three. Over two-thirds (69.0%) of Phase One teachers in year three agreed that professional development opportunities are appropriate given their experience teaching in a 1:1 environment.

Additional teacher survey items (see Figure 22) indicate that teachers’

perceptions of specific professional development topics have improved over time. This improvement in teacher’s perceptions of professional development represents a major success of the initiative and is observed alongside teacher feedback related to improved choices for professional learning opportunities in year three as well as more time spent in collaborative teams and other peer-to-peer learning opportunities.

+

+

+

+

+

++

+

7.5%

7.7%

+

+

16.0%

16.7%

12.8%

5.5%

7.6%

8.0%

47.8%

47.7%

49.3%

48.6%

50.5%

46.1%

34.4%

24.3%

27.2%

44.1%

39.2%

42.9%

Year 3n = 809

Year 2

n = 1074

Year 1

n = 749

Year 3

n = 828

Year 2n = 1074

Year 1

n = 749



The culture of my school supports the use of technology-enhanced instruction to support student

learning experiences.

I was adequately informed of the expected role of my school as an FCPSOn Phase One School.



Figure 22. Teachers' Perceptions of Professional Development Received. Note: + indicates < 5.0%

In all three years of the initiative, SBTSs and principals were most likely to

describe teacher preparedness to implement effective technology practices in terms of a range or continuum, based on teacher experience and willingness to participate. While educators were more likely to compliment teacher progress and describe overall positive shifts in the majority of teachers’ practices, they also indicated each year, particularly in high schools, that pockets of teachers either had not implemented at all or were mostly using technology.

All educator groups—teachers, SBTS, and principals—were asked in interviews or focus groups about the type of professional development or areas of support teachers need to reach their potential as efficacious technology users. Table 5. below summarizes the feedback from all educator groups in each year of the evaluation. Educators consistently note needs for more peer-to-peer learning, more time to lesson plan, content specific learning opportunities, and diverse learning options that target teachers at variable stages of tenure and practice.

+

+

+

+

+

+

+

+

+

12.1%

+

9.1%

+

6.8%

+

10.3%

+

14.30%

18.8%

25.0%

18.4%

22.4%

13.4%

18.4%

15.5%

20.3%

16.30%

44.9%

39.2%

48.6%

45.3%

52.9%

49.2%

47.3%

44.9%

41.90%

22.4%

15.2%

22.6%

17.1%

25.7%

19.1%

25.3%

17.7%

23.40%

Year 3

n = 802

Year 2

n = 1074

Year 3

n = 804

Year 2

n = 1074

Year 3

n = 805

Year 2

n = 1074

Year 3

n = 806

Year 2

n = 1074

Year 1

n = 749

Sufficient Professional Development Received


Creating collaborative learning experiences with my students.

Differentiating instruction through technology rich, blended learning activities.

Engaging my students in higher-order learning activities.

Designing personalized learning experiences.

To support blended learning.



Table 5. Stakeholder feedback related to needed professional development and supports. What type of professional development opportunities do you feel would benefit teachers? What areas do teacher need more support?

Year 1 Year 2 Year 3 SBTSs Basic tools and

computer usage; more peer-to-peer learning; more emphasis on instructional design.

Maintain support for new teachers; more PD on POG skills; content specific PD

Supporting new teachers and those with tenure; time to master specific tools; more peer-to-peer learning.

Principals More peer-to-peer learning; more time for lesson planning.

More collaborative teams and lesson planning; PD targeted at resisters.

Classroom management; content-specific teaching; lesson planning.

Teachers PD opportunities that are relevant and appropriate, and content and grade specific.

More peer-to-peer learning.

Time to lesson plan; exposure to new tools; peer-to-peer learning; Google Classroom certification.



Goal Area 4: Support all FCPS students in the safe and responsible use of technology. Sub-goals for this goal area include that every student will develop awareness of safe and responsible use of technology, will develop digital citizenship, including maintaining a healthy media balance and well-being, and follow best practices related to online privacy and security. This goal area specifically applies to maintaining on-task and appropriate behaviors at home and school, the development of positive digital citizenship in students, and the maintenance of students’ privacy and security while online.

Findings related to safe and responsible use of technology, digital citizenship, privacy and security, and maintaining balance of use have been extracted from qualitative and quantitative data sources. Every student will develop awareness of safe and responsible use of technology. Survey and focus group data suggest that responsible use of technology, including maintaining on-task behavior through classroom management, is a persistent concern for teachers and parents, though its prominence in feedback from adults diminished from year one to year three. In year one, off-task behavior was the second most frequently mentioned challenge for teachers. The challenge of adapting analog curriculum to the new modality of learning was the most frequently cited challenge. In year two, students’ misuse of their device, or keeping students on task, was the most frequently mentioned challenge by teacher survey respondents. Off-task behavior did not appear prominently in educators’ concerns in year three, thereby suggesting improvement in goal 4 focuses.

Multiple data sources, including first-hand classroom observations, suggest that while concerns about on-task behavior are valid, pervasive off-task use was not observed in Phase One schools. For example, classroom observations in year three revealed that in the majority of classroom observations (39 of 44 observed), students were noted as on task and engaged in classroom activities. In a minority of classrooms (n = 5), one or more students were observed engaging in off-task activities. Not all off-task observations involved computer use. Of the 32 total observations where device use was part of classroom activities, just four noted inconsistencies with digital citizenship norms (e.g., an elementary student was observed playing a non-educational game in class). Educators also noted that teachers’ efforts at classroom management improved, particularly during the first year of implementation, and that educators were generally impressed by students’ ability to adapt to a 1:1 environment and use their device to complete school-related (i.e., on-task) work.



Every student will follow digital citizenship with best practices including maintaining a healthy media balance and well-being.

Digital citizenship is identified by educators each year of the evaluation as an area where more guidance from the county is needed. In all three years of the evaluation, increased guidance on digital citizenship is among the most frequent recommendations made by school-based adults. It is also an enduring challenge to classroom implementation. Based on interviews with principals, it appears that more structured and intentional approaches to digital citizenship in school buildings emerged in year three of the initiative. Teachers also indicated in survey responses (see Figure 23) that they felt more prepared in year three to implement effective digital citizen practices.

Figure 23. Teacher Perceptions of Professional Development related to Digital Citizenship. Note: + indicates < 5.0%

Again, digital citizenship concerns by educators were consistently couched in a need for more guidance from leaders, in school buildings or at the district level. Digital citizenship practices are highly important to parents. In the parent survey in years two and three, parents were asked about the importance of digital citizenship and responsible use. Parents in both Phase One groups indicated the highest levels of agreement (greater than 90%; see Figure 24) that positive digital citizenship and appropriate online behaviors are important for their children to acquire as part of their overall development.

+

+

7.6%

13.2%

19.3%

20.6%

48.8%

44.6%

22.4%

16.8%

Year 3n = 800

Year 2

n = 1074

Sufficient Professional Development Received


Implementing effective digital citizenship practices.



Figure 24. Parents' Technology Perceptions. Note: + indicates < 5.0%

Findings related to maintaining a healthy balance of media use are mixed. Students and teachers indicated each year in focus groups that it seemed as though students were on their devices more each year than the year prior. Further, in year two classroom observations, students were observed using their personal device in just over half of all observations while in year three, roughly 70% of classrooms observed involved students’ use of a computer or other technology. However, focus group, observations, and survey data in year three indicate that teachers emerged in year three as more judicious users of technology. For example, year three teachers were described by SBTSs and principals as notably “smarter users” of technology, and teachers described themselves, in focus groups and through survey items, as more comfortable and confident users of technology. Student survey items (reported in goal area 3) also indicate that students in year three engaged in less frequent use of their device overall, and particularly less passive use of their personal device (e.g., viewing videos) in year three compared to year two. So, while usage may be more frequent, it is likely that usage has changed such that increased frequency of use has not resulted in a less healthy relationship with technology for students. Every student will follow best practices of online privacy and security.

Evaluation findings from all three years do not suggest that stakeholders have pervasive concerns about students’ online privacy and security. Mention of student privacy and safety did appear in teacher language in year three; mention of student safety appeared in parent focus groups in year one. However, concerns over privacy and security are not a major finding in any year of the evaluation report, meaning that while concerns are noted, they do not appear at any frequency which would suggest that the district is not addressing this important outcome. Evaluation findings indicate that FCPS-issued laptops appear to be appropriately restrictive and are perhaps too restrictive, as students and teachers in years two and three of the evaluation indicated that device restrictions inhibit some school-related content and programs.

+

+

+

+

+

+

26.1%

12.6%

68.4%

83.4%

Year 3

n = 1382

Year 2

n = 1149

Parents' Technology Perceptions

Strongly disagree Somewhat disagree Neither agree nor disagree Somewhat agree Strongly agree

Positive digital citizenship and appropriate online behaviors are important for my child(ren) to

acquire as part of their overall positive growth and development.



Goal Area 5: Prepare students with the technology skills essential for success in tomorrow’s workforce.

Goal area 5 includes one sub-goal, that every student is proficient with “real world” tools that are essential for workplace success. This goal area applies to the actual technology skills—typing, coding, creating digital content, and navigating software and programs—that may be essential to workforce success. Few quantitative measures in the evaluation have captured outcomes related to goal area 5. For the current summary, we have extracted findings mostly from qualitative data sources (e.g., interviews and focus groups) where related themes emerged prominently in responses to other questions. Key findings from the evaluation of years 1-3 include:

• Educators in all three years of the evaluation were more likely to

emphasize positive impact on students’ process skills such as

communication and problem-solving, than on technical skills such as

coding and basic troubleshooting.

• Educators note in all three years of the evaluation that while students’

skills are described as “improving,” their lack of basic technical skills (i.e.,

typing) are seen as a barrier to successful implementation. This finding

was less pronounced in year three as in the previous years.

Findings from year one of the evaluation indicate that goal 5 may have been given less emphasis than the preceding goals at the start of the initiative. Educators indicated that students were “strengthening” their skills, but that students’ technology skills stood to improve and that the lack of basic skills inhibited students’ ability to maximize the benefits of having a personal computer. Year one findings also revealed that teachers likely needed more professional development related to providing students the technology experiences that would result in real-life, applicable skills. The second and third years of the evaluation captured more information related to goal 5 outcomes through focus groups, observations, and interviews. Findings from teacher focus groups and the teacher survey indicated that teachers were more often incorporating technology during inquiry- and project-based learning activities. These modes of learning involve cultivating technology skills that are transferrable from school to work experiences—research, design, and presentation. Again, though, these are process skills rather than technical computer skills. During the year two evaluation, technology was more likely to be described by students and teachers as a tool for managing teaching and learning, and students and parents emphasized in focus groups that students’ personal device helps or makes completing tasks easier. These findings provide further evidence that students in year two were more likely to use their computer in ways that may translate to workplace skills.



Similar to year one, though, some findings from the year two evaluation indicated that goal 5 outcomes continued to receive less focus than other goals by the initiative. Most salient is the suggested need for professional development of teachers to provide the type of learning experiences that cultivate transferrable technical skills. In the teacher survey (presented in Goal area 1), teachers were less likely to agree they integrated online or digital activities, which would require students to navigate software and/or programs. And in focus groups, teachers expressed concerns about their preparation to create learning experiences that directly reflect the goals of FCPSOn. Finally, year two findings from the student survey related to student usage indicated that students engaged in relatively frequent use of their device for passive activities—assessments and media—rather than in ways that demonstrate or cultivate transferrable technology skills.

Year three findings indicate that students’ technology skills have begun to

improve after three years of implementation. Over time, students may have acquired more effective guidance from their teachers: one of the most prominent findings to emerge was that teachers were notably “smarter” users of technology. Teachers were also more likely to agree they were prepared to design learning experiences that promoted higher-order thinking skills, which very likely include critical and computational thinking, and that students had improved in their use of technology as a learning tool. In focus groups, teachers noted that students find resources faster, are better at typing, and are more proficient using software and educational programs.

Student survey responses related to students’ technology self-efficacy overall (e.g., the degree to which students believe in their ability to use technology to accomplish goals) demonstrate stability related to student skills (see Figure 25). Students were slightly more likely to agree in year three that using a computer feels natural to them as compared with year two. This finding may relate to students becoming more comfortable with technology over time.

+

+

+

+

+

+

+

+

5.5%

7.1%

+

+

18.1%

19.5%

5.0%

5.0%

38.6%

41.8%

40.2%

40.6%

31.4%

26.6%

50.6%

50.1%

Year 3

n = 5564

Year 2

n = 7283

Year 3n = 5564

Year 2n = 7283

Students' Technology Beliefs

I Don't Know Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

I know how to use my personal laptop to complete assignments and homework.

Using a computer during learning feels natural to me.



Figure 25. Students' Technology Beliefs.


More evidence related to goal 5 is found in how educators and students talked

about the impact of the initiative on learning and student engagement. Both groups noted increased flexibility in terms of time, place and pace of learning, and an increase in self-directed learning. Teachers also described improvement in students’ research skills. These findings suggest that students have been asked to become more responsible users of technology, and that technology use coincided with the development of functional use of technology to solve problems and complete tasks.



Appendix Table 1. Number of questionnaire items in each year by goal areas. # of items in Year 1 # of items in Year 2 # of items in Year 3

Goal Area Student Teacher Parent Student Teacher Parent Student Teacher Parent

1 4 5 5 1 5 5 2

2 1 3 3 4 4 5 4

3 3 1 9 2 1 9 2

4 0 0 1 2 0 2 2

5 0 1 0 1 1 0 1

Table 2. Summary of questionnaire items by goal area across years. Years 1 – 3

n Years 2 - 3

n Year 3 only

n

Goal area 1 Teacher questionnaire 4 2 Student questionnaire 6 Parent questionnaire 2 1

Goal area 2 Teacher questionnaire 1 3 1 Student questionnaire 4 Parent questionnaire 4

Goal area 3 Teacher questionnaire 9 12 3 Student questionnaire 1 Parent questionnaire 2

Goal area 4 Teacher questionnaire 1

Parent questionnaire 2 Goal area 5

Student questionnaire 1 Parent questionnaire 1

Goal area 1

Year

1

Year

2

Year

3

Teacher survey items

I can deal with most technical difficulties I encounter when using

computers and other digital resources and tools available to me. x x



I have received sufficient professional development on developing

learner-centered physical and virtual environments. x x

In an average month, how frequently do you deliver electronic formative

or summative assessments? x x x

The use of technology is an integral part of my instructional practices this year.

x x x

The use of technology is an integral part of my instructional planning and administration (preparing lessons, grading, data management, etc.)

x x x

My students effectively use technology as a learning tool this year. x x x

Student survey items

My computer makes turning in homework and completing assignments

easier x x

My computer is an important part of every school day x x

My computer is central to my success at school. x x

My computer works well. x x

How frequently do you use your computer for the following school-

related activities? x x

Do you have internet at home? x x

Parent survey items

The laptop provided by FCPS is an integral part of my child’s learning

experiences. x x

My child(ren) is/are motivated to use their laptop to complete homework,

assignments, and other school-related tasks. x x

Do you have internet at home? x

Goal area 2

Year

1

Year

2

Year

3


I am knowledgeable of the FCPS Portrait of a Graduate x x

I feel confident that FCPSOn and Portrait of a Graduate attributes will

contribute to greater success for FCPS students, in later education and work experiences

x x

I am able to provide my students with necessary supports and resources

to develop their Portrait of a Graduate Skills. x x

In an average month, how frequently do you plan opportunities for students to practice POG skills?

x

My students demonstrate effective Portrait of a Graduate skills as a [communicator, collaborator, ethical and global citizen, creative and

critical thinker, goal-directed and resilient individual]

x x x


I use my computer to create products that show what I've learned x x

My computer helps me to be more responsible for my success in school. x x

Communicator, collaborator, global citizen, goal directed, critical thinking inventories.

x x

How often do you use a school-issued laptop for the following SCHOOL-

RELATED activities? x x

Parent survey items

I am knowledgeable about the FCPSOn initiative x x



Prior to completing this survey, I was aware my child(ren) is/are enrolled

in a school that participates in FCPSOn. x x

I am knowledgeable of the Portrait of a Graduate skills promoted by the FCPSOn initiative.

x x

I feel that Portrait of a Graduate describes attributes that will be helpful

to my child’s future success in education and work experiences.

x x

Goal area 3

Year

1

Year

2

Year

3


In an average month, how frequently do you Provide whole class

instruction or lecture for 20 minutes or longer? x x

In an average month, how frequently do you Facilitate cooperative/collaborative learning?

x x x

In an average month, how frequently do you Provide student choice in

topic or tool used to demonstrate learning? x x x

In an average month, how frequently do you Promote individualized learning (e.g., students working alone at desk or personal computer)?

x x

In an average month, how frequently do you Facilitate differentiated

learning experiences? x x x

In an average month, how frequently do you Allow students opportunities to support each other’s learning (e.g. peer conferencing, peer feedback,

etc.)?

x

In an average month, how frequently do you Facilitate project-based or other inquiry-based approaches to learning?

x x x

In an average month, how frequently do you Foster cross-curricular

connections? x x x

The use of technology is supportive of personalizing the time, place, path, and pace of instruction for my students this year.

x x x

Technology has contributed positively to student achievement in my classroom this year

x x

Technology has contributed positively to student engagement in my

classroom this year. x x x

I enjoy using technology in my classroom. x x

Integrating technology into instruction supports implementing meaningful

learning experiences. x x

I am confident that I integrate 1:1 technology effectively in my classroom.

x x

FCPSOn supports a broad array of instructional strategies that I use to

support my students’ learning. x x

I know what digital tools and resources are available to support instruction.

x x

I was informed of my school's role as an FCPSOn Phase One school. x x x

The culture of my school supports the use of technology-enhanced

instruction to support student learning experiences. x x x



I have received sufficient professional development on Personalizing OR providing choice of time, place, pace, or path for student learning.

x x

I have received sufficient professional development on engaging my

students in higher-order (inquiry, problem-solving, analysis/synthesis) learning activities.

x x

I have received sufficient professional development on creating

collaborative learning experiences with my students. x x

I have received sufficient professional development on differentiating instruction through technology-rich, blended learning activities.

x x

I have opportunities to participate in professional learning that allow a choice in what I can focus on.

x

Professional development opportunities provided are appropriate given

my experience teaching in a 1:1 environment. x


Classes are more interesting when I use my computer for learning. x x

Parent survey items

The laptop provided by FCPS has contributed positively to my child(ren)’s achievement in school this year.

x x

The laptop provided by FCPS is an integral part of my child’s learning

experiences. x x

Goal area 4

Year

1

Year

2

Year

3


I have received sufficient professional development in implementing

effective digital citizenship practices. x x

Parent survey items

It is important that in using technology, my child(ren) act responsively in

communicating with others and accessing information. x x

My child(ren) are supported by their teacher and school in the safe and responsible use of FCPS technology

x x

Goal area 5

Year 1

Year 2

Year 3


How often do you use a school-issued laptop to design PPT, drawings, or

web pages? x x

Parent survey items

Technology skills are important for my child(ren) to acquire as a 21st century citizen(s).

x x

key findings from the evaluation of fcpson: years 1-3 · measures in the evaluation have captured...

Documents