Categories: News

Reviewing a District’s “Return on Instruction” After a Tech Implementation

Posted on

This article first appeared on The Learning Counsel. By Cebron Walker.

“Digital learning is the way of the future. However, assessing where we stand, and how teachers and students are doing, can be a difficult task now that many have thrown out textbooks in favor of an iPad or Chromebook. Finding that answer is what leadership at Houston ISD tackled and have recently made public. They reviewed their “ROI”—or Return on Instruction—after their PowerUp initiative and the creation of the “HUB,” a district-wide system for all students and teachers to access digital resources and tools.

Director IT, Education Technology of Houston ISD, Beatriz Arnillas shared the details of their digital curriculum and technology strategy and the outcomes. She explained that it began with the high school level in academic year 2013-2014. The stated goal of this work was to enhance their students’ ability to be college and career ready at the end of their high school years.

The research study was controlled to a specific and pre-determined cohort—eleven selected high schools based on their mix of ethnicities and income levels.

Teachers from this cohort received their laptop computers in August of 2013 so they could begin familiarizing themselves with them through the fall semester. Professional development was conducted, focusing on teaching the teachers how to manage the device in a responsible way, how to take care of the device, and also to introduce them to the instructional software and instructional applications they would be using in the classroom to enhance student levels of activity and to focus more on learner-centered instruction.

“After the first five months, after we were sure our teachers were ready to instruct in a new 1-to-1 environment and with digital curriculum, the students received their laptops,” stated Arnillas. “The first step we took with cohort one students (in the eleven chosen high schools) was a lesson on how to be safe online and how to utilize technology effectively for learning.”

While to many it might sound like that first year was focused on the device, Houston leaders considered it vital that the teachers and students were ready for a complete shift in the traditional teacher/student relationship. “The truth of the matter is that teachers really needed to learn how to manage their classroom environment with technology in it,” stated Arnillas. “We made sure that instructors knew how to keep students on task and how to help them be autonomous and manage their time in a wise way, and to ensure they were familiar with all the technology and digital resources that were available to them.”

During that first year, there was also a focus on working with the principals of each high school to adjust their leadership approach and ensure they believed in the program and how it would make a significant transformational change in their schools and each campus’s climate. Beatriz acknowledges that it was a maturation process. “At first the use of the technology was sort of ‘shallow,’ so to speak, but application and enthusiasm grew as all involved saw what could be done with it to enhance, and transform teaching and learning.”

After that first year, learning and instruction could be seen to shift with the group. Year two began with all students and teachers having their device and knowing how to use them.

The itslearning digital teaching and learning platform was also introduced at the beginning of year two (2014-2015). At Houston ISD they have dubbed this platform, the “HUB” as itis their central place for all things digital.

“In essence, it’s a platform that has a digital learning-object repository,” stated Arnillas. “A learning management system (LMS) and a curriculum management system, all put together so that students and teachers can access their content, communicate, collaborate, see assignments and the teachers can deliver quizzes and assessments that are aligned to standards.”

Beatriz shared several of the slides which showed increased proficiency in Math (algebra 1) and English with the initiative when comparing the first year (2013-2014) to the second year of the initiative (2014-2015) when it was in full swing within cohort one.

The chart here, shows the comparisons for all students and then columns which break down results by ethnicities. What is being compared is the year-end course exam results.

“What can be seen here, through these numbers, is a positive trend in just about every course and ethnic,” stated Arnillas. “And I would hesitate to say that Caucasian student results in English don’t show an improvement because our sample of Caucasian students wasn’t high enough. In other words, we did not have enough Caucasian students in this group of schools to really draw a conclusion from that but, if anything, what this says to us is technology is a great equalizer and that was one of our goals.”

In the graph below, the STAAR (State of Texas Assessments of Academic Readiness) score improvements are presented for cohort one for ALL students, Hispanic, African American and Economically Disadvantaged. In fact, almost 90% of students in the PowerUp pilot had an increase in English scores and 55% an increase in Math scores, even as the STARR test became more rigorous over the test periods. (English percentage of improvement is Blue and Math percentages of improvement is shown in Orange).

Beatriz expanded on her explanation of the results and advantages of the initiative. “As with any district, many of the middle-class kids have access to technology at home, but the minority students and the low-income students do not,” she stated. “So, it would make sense that having access to a device that has a lot of very valuable and effective learning materials in it, available 24/7 would empower students that are economically disadvantaged.”


There are a couple key factors of success in the initiative, explained Beatriz, first was to allow all students to “own” their devices and take responsibility for them, just as if they would take their textbooks home in a traditional setting. “The studies and literature says that the difference between the programs where students can and cannot take the device home is huge,” she said. “The point is to empower them—to give them choice and help them become an autonomous lifelong learner.” In order to do that, Houston ISD leadership considered it very important to put the technology in their hands, to be able to use it anytime and from anywhere.

Another factor was the central HUB which facilitates everything the students and teachers do. “It contains digital resources from more than 25 different publishers, all searchable by keywords and learning standards,” stated Arnillas. “A student or a teacher can go to the library online, put in a keyword and cross-reference the keyword with a standard and then they receive a number of results, just like you would do in Google but, in this case, all the results that come up are highly curated materials the district has adopted for digital content that is meta-tagged to learning standards.”

Beyond the curriculum assets themselves, there are assessments and analytics tools in the HUB. “Assessments can be linked to standards so a teacher can prepare an assessment and each question can be linked to a specific student expectation in the TEKS (Texas Essential Knowledge and Skills which are the state standards for what students should know and be able to do). After a student takes an assessment, teachers can generate a report and see how many of their students have mastered, to a certain percentage, each learning object and those who haven’t mastered it yet. Along with this report is a tailored student-by-student list of options/tools in the library that can be used to complete a student’s understanding. It supports mastery-based instruction, not pass or fail.”

We asked Beatriz to summarize, after all the qualitative and quantitative research they did in collaboration with the University of San Diego, the primary success points:

  • Number One is Anytime, Anywhere 24/7 Learning.  Access to the device and the digital curriculum on the HUB wherever they are, anytime.
  • Second is the importance of central office strategic leadership and alignment.  This program was, from day one, governed by the Chief School Support Officer, the Chief Technology Officer and the Chief Academic Officer. No decision was made without one of those departments present. (For the first two years there were members of these three divisions meeting once-a-week for an hour and making all the decisions together.This means that the professional development department, the leadership department, the instructional tech, curriculum department, IT infrastructure and communications department met and made all the strategic decisions together every week.)
  • Third is cross-departmental collaboration. “This is of course related to number two,” stated Arnillas. “Number three can’t happen without the strategic alignment from the top. Furthermore, we would cross check our progress by meeting with principals at least three to four times per year just to double-check with them that everything was working and ask them, ‘These are the decisions we’re making, what do you think?  Is this going to work?’  Because, of course the principals are the ones that really understand their populations and, believe me, each school is a universe and it’s a completely different culture from the other schools.”

“These are preliminary results to be sure,” Arnillas stated. “And today, three years later, all 66,000 high school students in Houston ISD have devices and are experiencing a 21st century teaching and learning environment.”

“As a country, we are at the beginning stages of transforming—using technology in a way that helps prepare our students for college and career. We are also struggling to level education so, no matter what background or conditions a child has as they grow up, they have opportunities for an excellent education, careers and college opportunities. Even though we as educators always say ‘the technology is not the goal,’ everybody agrees that the technology is the tool that truly enables us to scale-up, personalize learning, focus on mastery and close the gaps.”

For further information on Houston ISD’s digital transformation and how they prepared their network to go 1-to-1, read this Journal, featuring an interview with Lenny Schad, Houston’s CTIO.”