The Data Imperative, Part I: Challenges Along the Way

Editor’s note: This week, EP is posting a three-part series about challenges of data use in education from a recent report from Mathematica Policy Research, how to solve those challenges, and how we can harness the power of data. Read Part II and Part III.

Data is no longer the new kid in school.

While many education organizations still grapple with data collection, analysis, and use, working with data has been around for well over a decade. (The 2002 Elementary and Secondary Education Act/No Child Left Behind required state accountability systems and specifies student testing.)

As a sector, we’ve made progress in working with data since, but challenges persist. If “working with data” sounds simple, we all know it’s not.

To be meaningful, the right data must first be collected, both in quality and quantity. It must be analyzed correctly. Then it must be applied correctly. Resulting analyses and findings must be accessible to all stakeholders who need it do their jobs better. And those stakeholders need to know what to do with the results for it to make a difference.

That’s a lot of steps, and each involves a lot of complex work. No wonder that most organizations – in education and beyond – face similar challenges working with data. [Tweet this.]

But let's dig deeper. In a recent report, “Changing Education Agencies from the Inside Out: Year 2 Report on the Strategic Data Project and Education Pioneers,” Mathematica Policy Research (MPR) looked inside 12 education agencies to assess their data capacity and use, challenges encountered, and lessons learned.

(More specifically, MPR assessed the impact of Education Pioneers and Strategic Data Project fellows who had been placed with those organizations. The report was funded by the Bill & Melinda Gates Foundation to study their investments in Education Pioneers’ and the Strategic Data Project’s programs to place data leaders in education organizations.)

The 12 agencies vary in type – state agencies, school districts, and one charter management organization – and face numerous challenges around gathering, storing, and using data. Similar themes emerge for many of the organizations and the obstacles they encounter, including:

• Data quality. Four agencies reported challenges with data consistency and reliability, including few procedures to ensure data entry by staff is accurate and reliable, poor quality data in general, and lack of reliable data sources.

• Data infrastructure. Six agencies reported challenges with data systems that were outdated, cumbersome, or not integrated. They experienced difficulty analyzing data from different departments that used separate systems or were slowed down by cumbersome systems that resulted in a small number of experts handling all requests.

• Staff capacity. Eight agencies reported a lack of staff capacity to work with data, due to varying levels of skills to make data-driven decisions, including knowledge of or training for interpreting data, a lack of enthusiasm about using data, and discomfort with data use.

Sound familiar? For most of us, I think it might. [Tweet this.]

So how do we solve these challenges? In tomorrow’s post, we dive into two actionable steps that you can take to start solving those challenges. 

Next up:

Part II: Solving education’s data challenges

Part III: Harnessing the power of data

If your organization needs technically skilled data experts who can conduct complex analyses, Education Pioneers can meet those human capital needs.