Idaho student engagement surveys get mixed reviews and results
Published at | Updated at
The latest batch of student engagement data has been held up as validation by one administrator, criticized by another and ignored by others.
State Department of Education officials released state-, district- and school-level student engagement data last month, showing that overall student engagement dropped by almost 13 percent last year, down to 52.6 percent.
Student engagement data is important in Idaho because it is one of the accountability metrics Superintendent of Public Instruction Sherri Ybarra and the State Board of Education chose to use in the state plan to comply with the Every Student Succeeds Act. Officials chose student engagement over other potential indicators, including chronic absenteeism, after administrators and educators pushed back saying they couldn’t be responsible for student attendance.
Meanwhile, the State Department of Education’s communication staff ignored Idaho Education News’ requests for information about how student engagement data is used and what happens next.
Across the state, students’ level of engagement with their school experience varies widely, from 87.2 percent of students engaged at Grace’s Thatcher Elementary to 20.9 percent of students engaged at Plummer-Worley’s Lakeside Junior High.
Upper Carmen Public Charter School founder Sue Smith said it’s no accident her school has one of the highest levels of student engagement in Idaho.
In developing the school in 2005, Smith prioritized time on task and that each child be engaged in learning. The school uses multi-grade classrooms, which means teachers will have the same students for four years to get to know them.
It also means the students know each other very well.
“One neat thing is each year that active learning culture is transmitted to a new kindergarten student by older students in the classroom. They soon pick up on and become a part of the overall learning environment,” she said.
As measured by student surveys and reported by the State Department of Education, 74.3 percent of Upper Carmen’s students were engaged during the 2018-19 school year. That places Upper Carmen, located near Salmon, in the 90th percentile statewide.
Smith said she focused on student engagement because she saw “tremendous wastes of time” in her previous education jobs. At Upper Carmen, prep time is outside of the school day, the students work actively in groups of varying sizes to stimulate their attention and the schedule is structured to maximize the amount of contact between students and teachers.
“Teachers set up high expectations and are dedicated to help each child meet their individual goals,” she said.
Like many of the schools with high levels of student engagement, Upper Carmen is an elementary school (only serving grades K-3), with small student-to-teacher ratios (only 56 students attend the school).
Overall student engagement in Idaho fell by nearly 13 points last year, down to 52.6 percent, Idaho Education News reported last month.
In the Fremont Joint School District, Parker-Egin Elementary had one of the highest levels of student engagement, at 83.7 percent. Superintendent Byron Stutzman said two things stand out about the school: small class sizes (K-5 enrollment is 88) and a strong dynamic between parents, students and the school, which is located outside St. Anthony in Parker.
Fremont educators also focus on students being actively engaged in learning.
“One thing we talk about in leadership team meetings is how we build on compliancy and make them engaged,” Stutzman said. “There is a difference between engaged students and compliant.”
Stutzman said he hadn’t spent much time with the SDE’s engagement data, or the survey questions used to develop the report, but he believes strong engagement is a byproduct of the overall climate at the school and a natural result of doing a lot of other things well.
How the state measures student engagement
The SDE uses the Glossary of Education Reform’s definition of student engagement, defining it as “the degree of attention, curiosity, interest, optimism and passion that students show when they are learning or being taught.”
In order to measure engagement, students completed a 20-question survey.
“Their responses provide evidence about student engagement across the behavioral, emotional and cognitive domains,” the SDE wrote on its online school accountability report cards.
Here are two of the questions from the elementary surveys, along with responses students can choose:
If the classwork is hard to do, I…
- Work harder.
- Try my very best.
- Get nervous and scared.
- Don’t do the work.
- Realize I need to be a better thinker and not just memorize information.
What do you enjoy most about your school?
- When I use technology.
- There’s not much I enjoy.
- I enjoy lunch and PE.
- I don’t have to work very hard.
- Being with friends
Are surveys a valid measure of student engagement?
Coeur d’Alene Charter Academy Principal Dan Nicklay doesn’t think the surveys are valid and thinks the data can be misleading to the public. He said the questions are ambiguous and the answers show whether kids are engaged with the survey, not with school. He also doesn’t think student engagement is something that can be measured objectively anyway.
“When you give the survey, you have to take into account the kids, for whatever reason, they’re bored or think it’s stupid or they’re being malicious or they resent being surveyed,” Nicklay said. “They are going to answer a certain way and not necessarily give their true thoughts. Talk about the reliability of the instrument itself, it’s ridiculous. I don’t think there is any value in it at all.”
According to the SDE’s student surveys, Coeur d’Alene Academy had 46.2 percent student engagement, below the state average of 52.6 percent. Although the survey says less than half of the school’s students are engaged, the school is one of the highest performing in the state. In math, 86 percent of students were proficient last year, as were 90.3 percent of students in English/language arts and 94 percent for science.
It’s unclear if or how the SDE uses the engagement data beyond posting it on accountability reports. Two SDE spokespeople, Kris Rodine and Scott Phillips, ignored emailed messages Idaho EdNews sent Aug. 28 seeking more information about student engagement data and requesting interviews with relevant SDE experts.
Overall, Nicklay is concerned there is too much of an emphasis on making everything measurable and comparable in education.
“We’re in spot right now where there is an attempt to measure and quantify and put numbers on absolutely everything, and certain things can’t be measured, and I think you’re banging your head against the wall trying to do it.”
Originally posted on IdahoEdNews.org on Sept. 18, 2019.