South Dakota has emerged as a leader in applying over-the-counter data (OTCD) research and standards to its statewide longitudinal data system (SLDS), known as Student Teacher Accountability and Reporting System (STARS). SD-STARS consolidates and integrates data available to the South Dakota Department of Education (SDDE), such as assessments, finances, personnel and certification records, student information system (SIS) data, and more. SD-STARS facilitates education stakeholders’ analysis and use of this data to improve instruction and student performance.
State departments of education offering high-quality longitudinal data systems doubled in 2010 and were expected to be complete in all states by the end of 2011 (Herbert, 2011). While these education data systems provide educators with data, they have traditionally done so without adequate support to use that data correctly. This has applied:
- whether the data systems are provided at the state, county, or district level, and
- whether the data systems are built and maintained locally (in-house) or by an outside agency (vendor)
SDDE is a clear exception to that rule. SDDE officials actively researched best practices for reporting education data, so that data reports are accompanied by clear guidelines on how to understand and use the data. Metro Nashville Public Schools (MNPS) and its guide use was one source of inspiration. As far back as 2013, SDDE utilized OTCD research concerning the use of interpretation guides (also called reference guides) and used guide templates modeled after those shown to be significantly effective. The quantitative study on which these templates were modeled involved 211 educators of varied roles and backgrounds and produced evidence educators’ data analysis accuracy improved by 436% (i.e., more than quadrupled) when the guides were used.
SDDE’s data interpretation guides – or DIGs – offer all of the guide components shown to be successful. Each DIG features an image of the particular data report to which the guide is tailored, and contains the following report-specific sections:
- Quick Reference (Description, Questions Report will Help Answer, Intended Audience, Data Details, Common Misunderstandings/Warning)
- Instructions (Generating Report, Reading Report, Downloading)
- Essential Questions (e.g., how to use the report to help answer key questions)
- Frequently Asked Questions
- Report Details (History, Report Responsiveness, Problems?)
- Security (Roles with Report Access, Security Consideration)
These guides help to ensure those who use SD-STARS reports don’t achieve the mere 11% to 48% accuracy studies suggest is typical of educators’ inferences concerning data they view in ordinary data reporting environments. Rather, these guides can more than quadruple that accuracy so stakeholders can truly use data to improve instruction and student success.
Yet SDDE did not stop with guides. They also provide other supports to help educators use the SD-STARS data successfully. For example, they offer:
- Answers to frequently asked questions about SD-STARS
- An entire SD-STARS implementation guide with clear steps to guide districts through implementation
- A planning document for staff trained to share what they learned about SD-STARS with their districts (i.e., train-the-trainer)
The step-by-step, targeted lessons SDDE provides are in line with the OTCD help system standards. There is much support for resources, such as evidence that a shorter, targeted manual or user-friendly help system causes users to need 40% less training time and to successfully complete 50% more tasks than they would have accomplished with only access to a full-sized manual.
As the Data Quality Campaign (DQC) (2011) asserts, the power of statewide data systems will not be realized until education analysts and researchers – as opposed to just information technology staff – are involved in the full scope of the systems’ design. Applying research on best practices for education data reporting is paramount. SDDE is an example from which other states can learn as they work to make their SLDSs not just warehouses of data, but true resources for improving learning.
