department of procurement services - scsk12.org ii rfp... · this rfp? response: yes, ... response:...
TRANSCRIPT
Department of Procurement Services
Date: November 28, 2017
Addendum II
RE: RFP#12152017 Decision Analytics & Information Management
1). The SCBE previously had an Ed-Fi based solution, what happened to it? Response: The Ed-Fi based solution proved to be unsustainable due to a number of issues including significant changes in source applications, reliance on substantial amounts of customization and lack of internal skillsets to maintain the application moving forward.
2). Was it limited in some way or cost prohibitive, which the SCBE is trying to rectify with this RFP? Response: Yes, and Yes. The Ed-Fi solution did not fully meet the expectation of district and school-level stakeholders. The requirements framed in the RFP are intended to address the gaps experienced with the Ed-Fi solution through a sustainable holistic architecture. This RFP is meant to solve underlying issues previously missed and provide a sustainable path forward for the district.
3). Page-18 (#11): Does the Decision Analytics Platform have to be built using Microsoft PowerBI or can it be used with the vendors’ BI tool? Response: The Decision Analytics platform does not have to be built in Microsoft PowerBI. The district recently transitioned to Office 365 and is looking to leverage that investment to the greatest extent possible. Our goal is to deliver a single point of entry for our end users, so they are no longer receiving pieces of data from multiple sources or having to figure out where they should go for different information points. Meeting
our end user needs (outlined on pp 5-7 of the RFP) in a sustainable and efficient manner is our focus with the RFP.
4). Page-18 (#12): Integration with Microsoft PowerBI (already licensed and installed) to provide Ad Hoc Visualization and Reporting Capabilities – Is this the only option for Ad hoc visualization and reporting, or can the vendor provide their own?
Response: The district expects to continue to use PowerBI going forward due to all SCS staff having access to leverage the tool through our Office 365 agreement. Alternative BI proposals should be explained and accounted for in the submitted proposal. The proposal will be assessed based on how well it meets the outlined requirements.
5). Specifically, since this is a non-construction contract, will a bond be required if the submitted bid is in excess of $100,000.00? Additionally, will a bond be required if the submitted bid is less than $100,000.00?
Response: We reserve the right to request bonding.
6). p. 8, Capabilities Overview & Architecture, Architecture Map - What existing Ed-Fi technologies (API, operational data store, XML ingestion) are installed at Shelby County Schools? Please specify the Ed-Fi versions and their level of deployment and use.
Response: The Ed-Fi ODS and dashboards are not currently active. Previous evaluation identified an upgrade to version 2.x would be required in conjunction with the remapping to multiple application changes, which are identified in the application listing in the appendix.
7). p. 8, Capabilities Overview & Architecture, Architecture Map - Are the Ed-Fi
dashboards installed at SCS? Have they been extended for risk indicators?
Response: The Ed-Fi dashboards are not currently linked to active data. A highly customized risk indicator was developed, but experienced limitations to only 9-12 and was not adaptive over time.
8). p. 8, Capabilities Overview & Architecture, Architecture Map - Are there any existing
connectors or integrations with the Ed-FI ODS, via the API or via batch XML?
Response: No APIs were used due to the ODS version in use 1.5. Usefulness will be limited due to the amount of application changes that have occurred. 9). p. 8, Capabilities Overview & Architecture, Architecture Map - What version of
PowerSchool will be integrated?
Response: PowerSchool version 11.x in the Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production environment.
10). p. 8, Capabilities Overview & Architecture, Architecture Map - Do you have an existing
data warehouse? What reporting is driven from the data warehouse? Is there a desire to source data into the data warehouse from the new Ed-Fi infrastructure?
Response: The inactive Ed-Fi ODS was our attempt at a data warehouse and is not use for any reporting. The warehouse solution should align with the proposed architecture.
11). p. 46, diagram provided in Appendix C - do all of the connections to/from PowerSchool
exist today? What is the makeup of each of those? API, batch, csv, etc.
Response: There are currently no APIs in use among any applications in our environment. IT does not have a comprehensive listing of connections between applications. Connections are functioned through a combination of batch, CSV and XML point-to-point data transfers that are stood up for each individual request. The proposed iPaaS should address the management of connections and data flows.
12). p. 21, 2.0 – Joint Ventures - Would SCS consider a “best-of-breed” solution that would
allow for the two main components of the solution, a.) Information Management, and b.) Decision Analytics, to be provided by separate, but complementary, vendors under multiple awards? Or is it important to SCS that all the components are provided in a single technical solution from one vendor or multiple vendors in partnership?
Response: SCS is looking for single technical solution. A best of breed, if proposed, should be submitted through a partnership.
13). p. 18, #15 – Tools for streamlined Federal and State Compliance Reporting - How is state and federal reporting accomplished today? Is there a database that collects data for that purpose? In what form is data sent to the state?
Response: Data is uploaded to the State EIS.
14). p. 16, #7 – Research-based and Validated Frameworks - Does SCS use any existing
research-based frameworks for at-risk indicators? If so, please provide details.
Response: The previous at-risk modeling was developed for SCS by an outside contractor. It focused on identifying students in grades 9-12 that were at-risk of not graduating on time. The model was dependent on the Ed-Fi ODS and dashboards that are no longer active.
15). p. 21, SECTION B – SCOPE OF SERVICES, Paragraph 1.0 The Services, Capabilities
requirements, Sub-Section F. Technical Requirements, Clause 20. Data Security
Policies, Procedures & Practices, bullet No. 4, states the following, “Ownership: Shelby County Schools maintains ownership of all data and associated business rules during the entire duration of platform usage, and maintains the right to have data and business rules returned or purged upon our request.”
Please confirm, is it the District’s intent that ‘ownership’ refers only to data and business
rules either owned by the District prior to the effective date of a final contract to be
awarded or to data and business rules created for the District during the performance of
the Services under the terms of such final contract? Also, to the extent such other data
and business rules are necessary for performance or provision of the Services, will the
district consider securing a license from the Respondent for use of all other data and
business rules owned by either the awarded Respondent or other third-parties?
Response: This statement refers only to data and business rules either owned by the
District prior to the effective date of a final contract to be awarded or to data and
business rules created for the District during the performance of the Services under the
terms of such final contract. We would consider securing a license if outside data is
required.
16). p. 21, SECTION B – SCOPE OF SERVICES, Paragraph 2.0 Joint Ventures, first sentence,
states the following, “Respondents are encouraged to enter into joint ventures for the
purposes of responding to this RFP and providing the Services.”
Please confirm, in addition to establishing Joint Ventures, will the District permit
Respondent to enter into one or more sub-contracting agreements with third-parties in
order to perform or provide the Services?
Response: SCS is looking for single technical solution. We encourage the respondent to
enter into Joint Ventures with Local Business, Minority Owned Businesses, Women
Owned Businesses
17). p. 22, SECTION B – SCOPE OF SERVICES, Paragraph 4.2 Description of the Respondent.,
Sub-paragraph 4.2.4, states the following, “Disclosure of the volume of sales the
Respondent has had in each of its past two fiscal years;”
If the Respondent is a non-public company, may the Respondent provide a sales range
as its response and can this specific response be claimed as an exemption from the
Tennessee Public Records Act?
Response: Yes, you can provide a sales range, however there will be no exemption.
18). p. 24, SECTION B – SCOPE OF SERVICES, Paragraph 5.0 Compensation., states the
following, “The successful respondent will be compensated upon the completion of the
project’s objectives and upon meeting or exceeding the District’s Scope of Services
requirements. To the extent the provisions of this paragraph contradict any provision
regarding compensation set forth in Section C of this RFP, the provisions set forth in this
paragraph shall govern.”
Will the District consider a compensation structure whereby the Respondent is
compensated based on mutually agreed to milestones and/or deliverables where such
milestones and/or deliverables are met or provided to the District prior to the
completion of all of the project’s objectives? Also, to the extent one or more licenses
are necessary to provide the Services, will the District compensate the Respondent for
use of such licenses prior to the completion of all of the project’s objectives?
Response: Yes, SCS will consider.
19). p. 24, SECTION B – SCOPE OF SERVICES, Paragraph 6.0 Bonding Requirements., states
the following, “The successful contractor shall be required under the Contract to comply
with the bonding requirements listed below, if any.”
As the contemplated Services do not include construction related services, please
provide details with respect to any specific bonding requirements associated with the
contemplated Contract.
Response: We reserve the right to request bonding.
20). p. 30, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 9.3 Compensation.,
states the following, “The successful respondent will be compensated upon the
completion of the project’s objectives and have met or exceeding the District’s Scope
of Services required. More specific guidance regarding compensation may be set forth
in Section B of this RFP.”
Will the District consider a compensation structure whereby the Respondent is
compensated based on mutually agreed to milestones and/or deliverables where such
milestones and/or deliverables are met or provided to the District prior to the
completion of all of the project’s objectives? Also, to the extent one or more licenses
are necessary to provide the Services, will the District compensate the Respondent for
use of such licenses prior to the completion of all of the project’s objectives?
Response: Yes, SCS will consider.
21). p. 31, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 9.6 Performance.,
states the following, “SCBE and the successful contractor each shall agree to fully
perform all of their respective obligations under the Agreement. However, failure of
SCBE to perform such obligations shall not automatically relieve the successful
contractor of its obligation to perform under the contract.”
If the provision or performance of certain deliverables by Respondent associated with
the Services are contingent upon the performance of certain obligations by SCBE (e.g.
payment obligations, review and approval by SCBE personnel of specifications or
deliverables, the provision of access to SCBE’s application environment and data, etc.),
please describe how SCBE would contemplate Respondent performing such contingent
obligations in a timely matter in the event SCBE fails to perform its obligations which are
necessary for Respondent to complete its obligations?
Response: Dependencies and required parallel milestones will be accounted for and identified through the project planning phase. The initiative will establish an on-going partnership with SCS and the selected vendor working collaboratively to ensure overall success.
22). p. 35, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 10.22 Insurance.,
states the following, “By submitting a Proposal in response to this RFP, the Respondent
certifies that, if awarded a contract, it will have the insurance coverage required for
performance of the Services, if any, at the time the work commences.”
Please provide the specifications for the insurance coverage required, if any?
Response: Please reference Section 10.0 General Terms and Conditions, 10.22
Insurance.
23). p. 35, SECTION C – INSTRUCTIONS TO RESPONDENTS, Paragraph 10.24 Ownership of
Computer Programs and Data., states the following, “Ownership of all computer
systems, programs, software, data, materials, documentation or similar products
purchased, created or compiled in connection with the performance of the Services or
the performance of obligations under any contract resulting from or related to this RFP,
now or hereafter, shall vest completely and exclusively with SCBE. Upon expiration of
the term of the contract, the successful contractor will relinquish and convey to SCBE
any right it may have in such computer systems, programs, software, data, materials,
documentation or similar products.”
Certain intellectual property owned by Respondent or a third-party may be required to
provide the contemplated Services and fulfill Respondent’s obligations. Therefore, will
SCBE consider a term license to use certain computer systems, programs, software,
data, materials, documentation or similar products procured, created or compiled in
connection with Respondent’s performance of the Services or the performance of its
obligations under the contract contemplated to be awarded?
Response: We would consider securing a license if outside data is required.
24). Does real-time mean instantaneous or do you mean "near" real-time? What is
acceptable for real-time?
Response: “Real-time” is used in the context of actual stakeholder statements.
Currently, dashboards and most reports are only available with a minimum of a day lag
due to overnight load processes. Acceptable has not been clearly defined. A minimum
expectation is that as data entry/corrections are made in source systems there is an
established expectation for when it will be reflected in dashboards and reports within
the same day.
25). What tools and technologies do you currently use to track students? Who is responsible
for inputting data into these tools / technologies? How do the end-users and key
stakeholders currently use these existing technologies?
Response: The bulk of student information is entered and collected through our student
information system, PowerSchool. Individuals at the school-level are responsible for
data entry.
26). What data do you currently collect to support high school graduation and dropout
prevention?
Response: Tableau dashboards are in use to provide views of attendance, discipline,
grades and credit accumulation as individual dashboards. There is also Excel file based
tracking of student cohort lists to document students that have moved to other schools.
27). What data do you currently collect to support college and career readiness?
Response: ACT scores, TN Ready, NWEA MAP (as a 3x year assessment of progress),
Certification progress (for career readiness). All data is intended to align with the State
of TN standards for college and/or career readiness.
28). How do you currently identify at-risk students? At what age / grade?
Response: Our at-risk identification is not currently in use. When previously in use, it
covered 9-12 grade students. The intent is to be able to identify students at-risk of not
graduating on time and whether or not they are college or career-ready through
predictive characteristics, such as attendance, discipline, grades, assessment
performance and other measures.
29). How often do the structures of datasets change? What are some examples of this type
of change?
Response: Dataset structure changes might be from changes in the files the State sends
to SCS or source changes due to new application versions. A typical year might have 2-4
occurrences.
30). How often do your needs change? What are some examples of changing needs?
Response: Changes might be driven by changes in State requirements or as new
programs are stood up within SCS. New or expanded student programs are common
year to year within the district as new areas of focus are identified.
31). How do you define "like" organizations? Is your focus primarily on benchmarking against
k12 / educational organizations? Do you compare against industry organizations as
well?
Response: We expect to focus on primarily on other large, urban school districts.
32). What are all of the factors that will go into calculating percent up time (scheduled
maintenance, patching from vendor software, etc...)?
Response: Percent up time should be based on the percentage of time the solution is
live and accessible to users. We do not expect interruptions in availability due to
maintenance or version upgrades.
33). Can demonstrated experience in building accurate and impactful predictive analytics
models be used in lieu of existing historical models?
Response: Yes.
34). Which browsers and versions do you consider a modern browser?
Response: Current releases of Chrome, Firefox, Edge, Safari.
35). What disaster recovery and backup solutions do you currently use or plan to use?
Response: PowerSchool is operated through the vendor’s cloud environment and
follows their standard protocol. On-premise applications operate through a nightly roll-
over with a production and QA server refresh. As more of the application environment
moves to the cloud, it is intended to leverage the standards available through the cloud
environment of Oracle (future ERP) and Microsoft Azure.
36). What data retention requirements do you currently have?
Response: Retention requirements vary widely in areas of student and personnel
information. Not all data types have a defined and documented retention standard.
37). How is the SCS/board currently meeting this need? Which vendor(s) currently provide
the reporting and data warehousing services?
Response: There is some reporting integrated into existing source applications, but not a
single interaction point for all reporting. Tableau Server is in use for attendance,
discipline and grades dashboards available to district level users and school leaders. The
background information for the Tableau dashboards is manually pulled through a series
of stored procedures from the PowerSchool application. There is additional reporting
delivered through individual spreadsheets from a variety of sources.
38). Which analytical solutions are currently used? For this RFP to perform all required 5
types of analytics, any preference which statistical tools should be used such as SAS, R
etc.?
Response: Tableau and PowerBI are both currently in use. There is a small Research
group that leverages SAS/SPSS for statistical analysis.
39). Can some of the resources work remote on this project?
Response: Yes.
40). Any need to integrate Web APIs/Web services to extract or feed data from/to any 3rd
party system(s)? If yes, please provide details.
Response: There is a desire to include data from the National Clearing House and State
of TN systems. Currently, this data is received through flat file transfers.
41). Any preference on what cloud environment should be used for implementing the
solutions such as AWS, Azure, Google Cloud etc.?
Response: Our Office 365 is currently delivered through an Azure cloud environment.
42). What will be the project management process and engagement from SCS/board? Please
provide details.
Response: The project will be owned and run through the Strategy & Performance
Management division within SCS. We look forward to working with the selected vendor
during the contract negotiation phase to iron out detailed roles & responsibilities that
will ensure the overall success of the effort.
43). How large is the data size? How many years of historical data is available for predictive
analytics? Any preference on which modelling techniques should be used such as
Regression, Neural Networks, Decision Tree for predictive analytics?
Response: Looking at less than a terabyte of data. For historical data, it is recommended
only data after the merger & demerger years (2014-15 and forward) are used to provide
a representative student population.
44). Can SCBE identify the composition of the evaluation committee? How many
representatives will be on the committee and what are their roles within the school
district?
Response:
Title Department
Buyer Procurement
Financial Systems Advisor
Finance
Manager Information Technology
Manager Information Technology
Manager Performance Mgt.
Director Research & Performance Mgt.
Instructional Leadership Director
Office of Schools
Executive Director
Planning & Accountability
Manager Office of Schools
Instructional Leadership Director
Office of Schools
a. Can identify which department(s) will be leading this project?
Response: Performance Management (housed in Strategy & Performance Management)
45). The budget award for substantially similar projects for similar size school districts has
been approximately $850,000 in the first year and $100k - $250k annually for
subsequent years depending on the audience served, the amount of data to be
integrated and other factors. Is that budget comparable to SCBE’s budget for this
project?
Response: Yes.
46). Is this project funded through a grant?
Response: No.
47). Please clarify: how will the new Decision Analytics Information Management System,
including the dashboards, will work with the current Ed-Fi dashboards that SCBE has
already implemented, as described here: http://www.scsk12.org/edfi/? Will those
dashboards and data warehouse continued to be used and supplemented by this
system, or will they be sunsetted with the new system?
Response: Already sunsetted, see above questions on topic.
a. If the Ed-Fi data warehouse will still be used, where in the architecture diagram
on page 8 of the RFP does that data warehouse fit in?
Response: n/a
48). At the recent Ed-Fi Summit, Chris Moffatt, Ed-Fi’s Director of Technology pointed to
some challenges with the Ed-Fi Operational Data Store, including that a separate
Operational Data Store must be implemented for each school year. As a result, any year
on year or other longitudinal query would have to navigate the complexities of that
architecture. Is longitudinal data a priority for SCBE? Is that what Shelby County wants
or is it seeking a high performing, proven, sustainable education data warehouse from
an established ed tech vendor?
Response: Longitudinal query ability is a REQUIREMENT. Any solution unable to deliver a
longitudinal view of an individual student or cohort of students would not meet the
minimum requirement of the RFP.
49). Please confirm: Per the diagram on page 8 of the RFP, SCBE is seeking a Microsoft SQL
Server Data Warehouse and not an Ed-Fi ODS as part of the proposed solution.
a. If SCBE is seeking an Ed-Fi data warehouse, we are having a hard time identifying
a fully implemented Ed-Fi data warehouse, even though Ed-Fi has been around
for over 6 years. If SCBE wishes to have an Ed-Fi data warehouse as part of this
solution, can the district point us to one based on Ed-Fi as an example? We want
to model our proposal off a proven successful data warehouse implementation.
Response: We are interested in successfully following the Ed-Fi data standard. Our
preferred environment is Microsoft SQL due to current district skillsets and environment
uses of related products.
b. What is SCBE planning on doing with its existing Ed-Fi data warehouse?
Response: The existing Ed-Fi ODS is not operational due to multiple changes in source
applications and a version upgrade requirement to the 2.x releases.
50). Appendix C: Can we assume that all of the district’s data sources listed in Appendix C
will provide data in the Ed-Fi data standard? If so, which version of the standard will
they adhere to?
Response: The expectation is that part of the proposed solution will account for
transforming source data to the Ed-Fi data standard. All data sources will not necessarily
be covered by the Ed-Fi data standard. For data not covered by the Ed-Fi data standard,
the source systems standard and definitions should be carried through to the data store.
The current Ed-Fi data standard is version 2.1.
51). Regarding the Ed-Fi requirements, would the district consider a proven education data
warehouse solution used by districts serving over 20 million students throughout the
United States that aligns with the Ed-Fi data elements, but does not use the Ed-Fi
standard?
Response: What is the differentiation between Ed-Fi data elements and Ed-Fi data
standard? The Ed-Fi platform is NOT required to deliver the Ed-Fi data elements and
standard.
52). Page 8: Please clarify: how does SCBE currently use the Impact TN Ed-Fi Dashboard I
identified in the diagram on page 8?
Response: SCS is not currently using the Impact TN Ed-Fi Dashboards. The intended path
forward from the TN DOE is to sunset EIS and move all school districts to an Ed-Fi ODS.
The state reporting part of the any proposal should account for how the State of TN DOE
will require data be submitted to them.
a. Besides the Impact TN dashboards, are there other Ed-Fi standards-based
solutions that the system will feed data into?
Response: None currently planned or in-use.
53). Regarding PowerBI (several places in the RFP): Does SCBE anticipate that the out-of-the-
box dashboards use the PowerBI platform, or will PowerBI only be used for the
advanced analytics and ad-hoc reporting?
Response: Either approach is acceptable. A key interest is in providing as seamless an
interaction as possible for our end users at the school-level, while delivering the
actionable insights required. How a proposal most effectively achieves the requirements
outlined in the RFP is up to the individual vendor.
54). Regarding data access and management on page 15: How many users does SCBE
anticipate having for each type of defined role?
Response: The largest defined role type is that of our teachers (approximately 6,500
district-wide). Additional role types, such as Counselors, Assistant/Vice Principals,
Principals and District personal total less than 2,000.
55). Regarding the timeline described on page 13, with the anticipated implementation date
of July 15, 2018, what are the district’s priorities and preferences on how it would like to
roll out the new system to users?
Response: At-Risk Student identification is the highest priority. Project success is having
training and adoption processes in place prior to the 2018-19 school year. A school-level
adoption plan is part of this priority. Our primary goal is to deliver actionable
information in an easy to digest form to our teachers and school administrators from
the start of the year. SCS is also currently implementing the Oracle Fusion ERP and must
not be interrupted.
56). General solution question: In addition to a production environment, does SCBE also
want a training/test environment?
Response: Yes.
57). General solution question: Is SCBE expecting to migrate longitudinal data into the new
system? If yes, how many years and for which sources?
Response: Yes, back to the 2014-15 school year. We want to make this as complete a
picture as possible, but the capability to bring this information from all sources will have
to be evaluated as part of the initial assessment.
58). Regarding the Training and Support Plan described on page 19: What District resources
would be involved in the project and to what extent (i.e. how much of an FTE for each
role)?
Response: The Performance Management team (5 FTE) will own and manage the
project. There will be partial FTE from other teams to support adoption and
communication across the district.
59). Regarding the Training and Support Plan described on page 19: Does the district have a
Level 1 Help Desk? Is the vendor’s support line expected to be Level 1 or Level 2
support?
Response: Yes, there is a Level 1 Help Desk that covers general access issues. Vendor
support line should cover troubleshooting and usage questions specific to the delivered
solution. A robust FAQ and training plan is part of a successful support plan to minimize
end user tickets.