Better Data, Better Decisions
Districts in Minnesota and New Mexico demonstrate the power of good
By Allan Olson
Data-driven decision-making in education is gaining strong momentum,
accelerated by the passage of the No Child Left Behind legislation and
the need to document yearly progress. The goal of this new movement is
to use accurate, timely information to identify student strengths and weaknesses;
foster individual improvement and progress; ensure the academic growth
of every child; and inform program improvement decisions.
Truly effective data-driven decision-making requires that educators
identify and collect the right information, based not only on NCLB requirements
but on each school district's needs. Equally important is expert use of
data and a willingness to challenge and perhaps even change existing systems.
Accountability is key, too, for data-driven education to fulfill its
potential for student improvement.
Collecting the Right Type of Data
An Education Commission of the States study of six exemplary school
districts has found that the collection of achievement data is of most
value in meeting the NCLB challenge that all students learn consistently
and to high standards within 12 years. Meeting this mandate puts new emphasis
on gathering data to gauge annual progress, best provided by tests that
1) align to state standards, 2) measure growth from year to year, 3) prove
sufficiently challenging for each child, 4) provide results comparable
across school districts, and 5) enable fast, effective reporting to schools
Implementing such tests and using the resulting data to shape and guide
educational efforts will be key to successful assessment programs.
Targeting the Individual Student
Over the past six years, the Bloomfield, New Mexico, school district
has discovered a way to find the data needed to successfully guide instruction
and help all students learn. A 1,500 square-mile district with 3,300 students
of diverse ethnicity, many mobile and economically disadvantaged, Bloomfield
initiated a comprehensive assessment plan in 1996 to better guide instruction
to help all students reach their true educational potential.
Early in the process, Bloomfield educators realized that standardized,
norm-referenced tests needed to be augmented to gather the quality data
needed to help all students learn. According to Superintendent Harry Hayes,
"The state test is an external audit we accept as a necessity, but it's
not very useful for informing or guiding teaching."
What was needed, and eventually initiated: RIT scale-based testing that
identifies where a student is performing on a cross-grade learning continuum
and shows whether that child is growing sufficiently to meet the required
exit standards. These tests are based on a measurement scale called RIT,
or Rasch Unit. This scale enables the sensitive measurement of student
growth in achievement across a continuum of instruction. With such testing,
Bloomfield educators are able to measure student progress over time and
gather data from any school in the far-flung district.
"These tests are accurate at both the individual child and group level,"Hayes said, "and are aligned with our curriculum and standards. We can
see where each student is academically, where he or she is going, and what's
needed for improvement."
Embracing the Mission
With good data in hand, and an active plan to guide the way, districts
are well-positioned to become more mission-driven. In Bloomfield, every
staff member, from administrators and teachers to bus drivers and janitors,
can clearly state the district's mission and make strategic choices daily
based on that goal: helping all students learn successfully to enjoy fulfilling
"Everyone in the district," Hayes said, "contributes to a shared vision
of learning success for students preparing for fulfilling lives as literate
voters in a diverse democracy."
Putting the Right Data to the Right Use
Even with quality information and a clear mission, effective decision-making
can only occur when an educator knows how to use the data. To ensure program
acceptance (a proactive response to the "Who needs more testing?" complaint)
and program value, training is imperative.
Learning to evaluate and use data to inform instruction is key to Bloomfield's
success, with disaggregation especially valuable. Hayes said, "We're able
to answer key questions -- how is a particular ethnic group doing in reading?
How can we make improvements? We can disaggregate further to see if the
learning styles of males and females differ - which, by the way, they do.
Knowing this, we're able to accommodate those differences and get students
more engaged in what they're learning, and more successful."
At Naaba Ani Elementary School, two fourth-grade teachers relied on
data from fall reading tests to assign reading groups and identify specific
skills to be strengthened among individual students. Results from mid-year
testing showed student gains ranging from five to 17 points.
Bloomfield has found that using RIT scale-based tests and gathering
achievement data provide the kind of real-world, practical information
teachers need to ensure individual student improvement.
Redefining a District's Culture
In addition to enabling educational improvements, data-driven decision
making can challenge timeworn preconceptions about existing processes.
The 130-school, 46,000-student Minneapolis school district has found that
use of accurate, timely assessment data often extends beyond student improvement,
and can actually challenge basic conceptions about existing educational
systems and cultures.
After initiating a successful RIT-based testing system and receiving
a government-sanctioned variance, Minneapolis schools stopped using IQ
tests to screen for special-education students. Instead, schools use data
from the academic growth measure administered annually to all students.
That information enables the district to discover ways to keep students
with special needs in regular classrooms, fostering a culture that views
students as part of a learning continuum, not tied to a particular group
It's an important step, according to Dave Heistad, director of research,
evaluation and assessment. "Kids in poverty, kids who are highly mobile,
English language learners and special-ed students ... tend to be doing
poorly all over the country. One of the benefits of the No Child Left Behind
legislation is that it's bound to increase awareness of the huge gaps that
exist between the educational 'haves' and 'have-nots.'"
To help narrow this gap, Minneapolis now uses data from the annual academic
growth measure. Teachers have found that typical special-education students
fall two to three years below peers in reading skills, vocabulary and comprehension.
According to Heistad, "If you rely on grade-level testing with these
students, they'll just be wiped out. They'll give up. They'll never want
to take another test. It's a disservice to those kids to rely on grade-level
tests because in the long run, it doesn't help you find out how much growth
is really taking place and what students really know."
Instead, Minneapolis has turned to RIT-scale testing. Initial testing
provides an indication of how far behind standards the individual student
falls, and more intensive assessment is regularly conducted to determine
whether he or she needs additional help. "If you test a fifth-grader reading
at a second-grade level," Heistad said, "you test him at the appropriate
level, get a good scale score, and then monitor that student's growth compared
to the norm." In this way, rather than being excluded from testing as in
the past, special-education students are now part of the same accountability
system as other children.
This "problem-solving model" has resulted in a major cultural change,
with "special education" viewed differently. According to Heistad, "With
most kids, it's just that they're learning to read more slowly than others.
Rather than those old self-fulfilling prophecies that a child is damaged
or can't learn, our testing process shows us what we can do to maximize
performance and lead to progress in the curriculum."
In the bigger picture, students at every level will be better served
if educators focus learning more on the individual child than on grade-level
activities. Good data helps educators first determine the nature and quality
of growth each child needs, and then allocates the resources to meet those
Data Drives Improvement
In both Bloomfield and Minneapolis, assessment data drive all improvement
processes. Management meetings are held quarterly in Bloomfield, where
teachers and principals review achievement data showing trends in various
goal areas -- by subject, by teacher, by student, by ethnicity and over
time. This data is then used to review the school improvement plan and
strategies for continued progress, which entails fairly intensive ongoing
training for teachers.
In Minneapolis, reliance on data-driven decision making finds the district's
special-ed students demonstrating good growth and meeting the district's
goal to learn faster, get back to mainstream settings and ultimately meet
standards. The program has proved so successful that after being monitored
by the national Office of Civil Rights, Minneapolis' problem-solving model
is being recommended to other states and districts.
Accountability Counts, Too
An effective data-driven model must incorporate a combination of quality
test data, commitment, conversation and collaboration to demonstrate accountability
and ensure constant, intentional improvement in learning.
Test data identifies student strengths and weaknesses and a school's
strategies to accelerate progress, while growth data can keep the public
informed and involved in the school system. Growth data also can be used
to evaluate growth patterns in a given year, why progress has or has not
been made, which curriculum and teaching practices are effective and which
are not, and what solutions can be implemented for change, if needed.
Effectively collecting, evaluating and using quality data to strengthen
and evolve our current educational system as needed, is essential to meeting
the goals of the No Child Left Behind legislation. As Bloomfield and Minneapolis
have learned, that takes a willingness to change the system as needed to
help each individual student.
Accurate data will do a lot to lead the next generation of data-driven
decision making, building accountability programs and mission-driven, future-thinking
cultures focused on the growth of all students. According to the Education
Commission of the States, "effective use of data by educators and their
community will change how they respond to accountability demands. When
timely and useful data are available to educators, and effective interventions
are at their fingertips, the focus of accountability can move toward 'fixing
the problem' rather than 'fixing the blame.'"
Allan Olson is the executive director of the Northwest Evaluation
Association, a nonprofit organization committed to helping K-12 school
districts meet their federal assessment goals. He can be reached at firstname.lastname@example.org.
More information about NWEA is available at www.nwea.org.