Subject: Update on releasing raw data, answers to other outstanding questions

Hi everyone,

We are working through the privacy issues that need to be addressed to release any raw data, so I

wanted to give everyone an update of where we are.

Survey Data

For the survey, there are two sources that we would need permission to release data: CDE and Jeffco, as we collected their responses through a separate contract, since as the contracting clients they own this data. We have reached out to both. So far, we have heard back from CDE

and are working with CDE data security staff and other data security experts to ensure that any release of data is done in a way that guarantees the anonymity that we promised respondents.

Our initial approach would be to remove all identifying variables including:

• District name

• School name

• District characteristics – size, region, geographic location, need

• Respondent ID

• IP address

• Date taken

• Latitude/longitude information

• Specific respondent role (we have recoded specific roles into the general categories of district, school and teacher)

• Grade range served by school

• Grade level taught by teacher

We have also randomized response order and separated out open response items into a different

file. We have started manually going through these and removing any specific district or school names from all the open responses, but we still have concerns that individuals could be identified

by the specific assessment they are referring to, general writing style and content of feedback so we are exploring this issue. Other issues were are exploring include whether respondents’ district or school could be identified based upon the combination of assessments they report using, and if

the implied intent was that survey results would only be used in context of the Task Force’s work, meaning that a public release of data would violate this implied understanding.

I will keep everyone updated as the conversations around the release of the survey data progress.

Cost Collection Data

For the five districts that we spoke with, we explicitly stated to participants (and shared in writing) that all cost collection information would not be specifically attributed to their district

but instead be used to calculate per pupil cost estimates. As such, we see no way that we could share this individual level information without violating this agreement which we are unwilling

to do. Even removing district names, the size of the district could easily be inferred by the resources identified and then that would essentially identify the specific district. As such, we will not be sharing the raw cost collection data.

Other Outstanding Questions Regarding Costs

Different cost estimates- As was mentioned in the presentation, APA received the final cost data the same day we provided the draft report to the Task Force. In finalizing the report, and the cost figures, a relationship between size and cost per pupil became clear. APA felt the best way

estimate possible costs would be to use either the average of the five districts or the weighted average for the districts. The average total cost figure, as shown in the presentation and on page

31 of the updated report posted on the Task Force’s website, is $78,378,666. Statewide costs using the weighted average are $61,129,722. The weighted average weights each district’s cost figure by the district’s enrollment when creating the average figure. When examining costs to the

state, it makes sense to use the weighted average if there are differences in cost by size of district and the costs being examined are on a per pupil level. This was the case for this analysis. APA

believes that continuing to use the range figures would have provided an inappropriate estimate of the possible total costs to the state based on the data collected. The range provided both a lower and higher estimate than we felt should be used since they do not take into account the

impact of different levels of costs for different size districts. APA did not change the per pupil figures in the report, so information on the possible range of costs on a per pupil basis are still

available.

Exclusion of technology costs- we did not hear from the five districts that technology was a consistent direct cost area as some had technology initiatives without assessments, and that any hardware or connectivity upgrades would also be used for instruction. We did narratively

discuss that districts would incur capacity costs if they did not have technology in place and we are going to make sure that it is clear in the final report that these costs due exist, even if they are

not incurred by all districts.

Thank you,

Amanda

Amanda Brown

Senior Associate, APA Consulting

1120 Lincoln St, Suite 1101 | Denver, CO 80203

720.227.0088| arb@apaconsulting.net