As educators rethink how they make technology decisions, these are the critical factors to consider

You never know what you might find in a school closet. Old floppy disks. Mimeograph paper. Textbooks from the 1970s.

What Wiley Brazier V discovered when he walked into a storage closet at his former middle school was far more jaw dropping: 50 brand-new iPads, still in the box, with a year-old delivery date.

“Technology can be a waste. It can be wasted,” says the Baton Rouge principal. When schools buy new devices without a plan for implementing them, or when they purchase a shiny new tool without considering its instructional value, today’s hottest technology can end up gathering dust in the shadows.
It happens more often than technology directors care to admit. An analysis of K-12 district spending found that, on average, 67% of educational software licenses go unused. Given that districts use as many as 548 edtech products each month and spend a total of $13.2 billion a year on technology, that equates to a lot of wasted money – and even more wasted opportunities for students.

What all this points to is that edtech procurement needs fixing.

According to a recent Digital Promise study, school districts are frustrated by what many describe as a largely “hit or miss” approach to vetting their technology purchases. In places where teachers have a lot of autonomy in choosing technology, districts may end up with a hodgepodge of apps that don’t work together. On the flip side, districts that don’t involve teachers in technology purchases at all often struggle with low adoption rates because the software doesn’t meet classroom needs. 

Although more than 60% of teachers believe they should be the primary decision-makers regarding technology in the classroom, only 38% are even consulted. Bringing teachers into the procurement process from the beginning is critical to ensure the policies and processes work at both the classroom and district
levels. Collaborating with teachers helps build buy-in on both sides as educators become more supportive of the district’s constraints and administrators become more considerate of teachers’ needs.

“Edtech procurement has to be done with a lot of thought and intentionality,” says Susan Bearden, chief innovation officer for the Consortium for School Networking (CoSN). “I think a lot of times people can be swayed by the ‘ooh, bright shiny’ phenomenon, or perhaps they’re being moved by edtech vendors who make it sound like their product is going to cure all the problems in education. It’s important for schools and educators to do their research and not to make their technology purchases impulsively.”

But what does that research look like? What boxes should an edtech product check before getting the green light? To help nudge districts toward a more rigorous and comprehensive vetting process, ISTE partnered with Project Unicorn to identify five pillars of edtech selection. Educators can use these critical factors as a rubric for evaluating individual digital tools as well as a guide for redesigning their procurement processes to become more nimble, inclusive and responsive to teachers’ and students’ needs.

“The act of choosing a product or resource to use in a teaching and learning setting is a really difficult one that comes with a lot of contextual factors,” says Mindy Frisbee, senior director of learning partnerships for ISTE. “Context is really important, but these pillars are the critical areas to consider when looking at products. They should be part of the conversation no matter where you get reliable edtech product information.” 

1. Data interoperability

Imagine a middle school teacher investigating why a student is struggling in their class. They pull up the student’s performance records across all classes and discover high competency in science. By drilling down into which specific tasks the student has performed in science class, the teacher can gain valuable insights into the student’s intrinsic motivation and leverage their passion for science to improve achievement in another subject area.

But that can only happen in a technology ecosystem where all digital tools work together as a cohesive whole. In a Digital Promise survey, 67% of respondents identified the lack of data interoperability as a widespread problem, and more than half agreed it’s an urgent one.

Data interoperability is the factor that can elevate edtech from a tool of convenience to a driver of student achievement.

“Interoperability is pretty much one of our non-negotiables,” says Brian Seymour, director of instructional technology for Pickerington Local School District in Ohio. He added that every approved app must be able to sync with the district’s edtech management platform. “If they can’t work with us on that, it’s almost an immediate no. Being able to sync is one of the biggest considerations for us right now so we can pull data out and put it in dashboards.”

In districts where schools and teachers have a lot of autonomy in choosing digital tools, building interoperability into the procurement process can be a struggle.

“If a city has 190 schools, and assessment and curriculum tools are selected by the school for a specific population, that often leads to the question of how do you educate the folks who are making the decisions at the school level in the district ecosystem so they’re privileging interoperability?” says Erin Mote, executive director and
co-founder of InnovateEDU.

One way is to encourage on-the-ground decision-makers to seek out vendors that have signed Project Unicorn’s pledge to prioritize interoperability.

“It’s really about easing teacher time so they’re not hand-coding data out of a PDF and putting it into a grade book. How easy it is for you to get data out of that tool in a way that’s usable and actionable? I think at its best, interoperability means that teachers, district administrators, parents and even students have real-time access to data about learning and learning outcomes.”

2. Student privacy

If educators want to harness the power of student data, they must also be prepared to protect it. That means creating a districtwide data privacy policy that aligns with legal privacy standards – and making sure every tool they use complies with it.

A growing body of state and federal privacy laws, from the Family Educational Rights and Privacy Act (FERPA) to the Children’s Online Privacy Protection Act (COPPA), places the burden on school districts to safeguard student privacy in a technology landscape where some vendors mine user data in exchange for offering free tools and services.

“In my experience, most people really aren’t aware of the amount of information they’re giving away when they download an app or sign up for web services,” Bearden says.

Part of the problem is that many edtech vendors don’t clearly communicate what their privacy policies actually are. Vetting classroom tools for data privacy compliance often requires educators with no legal expertise to wade through hard-to-understand documents in an attempt to parse the provider’s intent. Some vendors, especially startups, don’t have policies in place at all.

In California, where districts are required to vet every vendor on 14 criteria, cooperative agreements like Silicon Valley Education Foundation’s Unified EdTech Contract are helping districts improve legal compliance by keeping tabs on which products meet federal and state requirements. Other resources, such as “The Educator’s Guide to Student Data Privacy” from ConnectSafely, can help teachers navigate the legal morass.

“I read all of our vendors’ privacy policies to make sure they align with what we want. If they haven’t agreed to the Student Privacy Pledge yet, I ask them to sign our internal privacy policy,” Seymour says, adding that he has discontinued services the district has been using for years when the vendors refused to sign. “No privacy policy is an immediate no.”

3. Standards alignment

In a district that spends a million dollars or more per year on educational software, there’s no room for tools that merely digitize traditional practices, such as worksheets or flashcards, without adding any curricular value. That’s why alignment with learning standards is the first thing Seymour considers when vetting an edtech tool.
Frisbee concurs. 

“When you’re looking at an edtech product, you want to know it’s following best practices for digital pedagogy,” Frisbee says. “What does it look like when you’re an effective digital learner? What kinds of skills and knowledge do you need, across a content area or grade level? Any product should be integrating these practices.”

Regardless of a district’s specific mission or learning goals, choosing technology that aligns with the ISTE Standards helps ensure students not only master a content area but develop their digital learning skills in the process.

“If you have a product that has integrated what is outlined in the ISTE Standards for Students, it has integrated learning opportunities for students to demonstrate their proficiency,” Frisbee says. “But has it been built in explicitly, so they’re not only doing it but they know why they’re doing it?”

To fully align with ISTE’s Empowered Learner standard, for example, a tool should not only allow students to set their own learning goals, but it should also help them understand why it’s important to do so – and reflect on whether they’ve met them. Similarly, tools that allow students to practice computational thinking and design skills need to also impart foundational knowledge about the process so learners can apply it to new situations.

“It’s important to have that metacognition as part of the learning process,” she says.

Vetting a tool based on standards alignment can be time-consuming, but ISTE’s Seal of Alignment program offers a powerful shortcut. In addition to discovering edtech products that have already undergone a rigorous standards-based review process, educators can find specialized information on how the tool addresses specific components of the ISTE Standards,” Frisbee says.

4. Research and evidence

In the ever-expanding edtech universe, pinpointing a solution that actually works is like trying to find extraterrestrial life using
an off-the-shelf telescope. Nine in 10 educators admit they rely on general web searches to gather information about edtech, while 59% base their procurement decisions on recommendations from peers.

“There are thousands of programs out there in the world,” Seymour says. “How do you know this one will truly be the one that’s going to make a difference?”

Educators prefer to make decisions supported by sound research and evidence of a product’s efficacy, but such research often doesn’t exist. Even when edtech providers do offer evidence to back up their claims, it may be viewed as untrustworthy; 76% of educators don’t believe vendors are qualified to conduct reliable research about their products. Additionally, much of the evidence that is available comes from local, small-scale pilots that may not take into account contextual differences across districts.

“The question is not just whether the evidence is reliable or something works. What’s more important is to understand the context of it,” says Katrina Stevens, director of learning science at Chan Zuckerberg Initiative. “For example, if somebody uses a tool in an urban environment and you’re in a rural area where you don’t have access to the same things, you need to find evidence that answers questions like: Are they similar enough to my district where it would work? Do I need to tweak it?”

Until edtech vendors are able to back up their claims with more robust research, the onus is on educators to conduct their own inquiry, whether that means running a pilot before implementing a solution districtwide or seeking out similar districts that have used the tool successfully.

“What do I want the solution to do? What evidence exists for it already? Was it gathered in places that look like me? How will I know whether it’s working? Those basic questions are a good way to guide how you’re making decisions,” Stevens says.

Another consideration that can help teachers vet the effectiveness of technology is to examine whether each tool is grounded in learning science principles, which are already backed by research. Stevens recommends using Transcend Education’s primer, “Designing for Learning,” as a guide for evaluating whether a tool incorporates
learning science principles.

“Look at strategies the learning sciences have defined as successful for learning and figure out whether the programs support those types of things – things like retrieval practice,” says Seymour, a participant in ISTE’s Course of Mind learning sciences initiative, which explores how to incorporate the learning sciences into edtech procurement. “Look at how the brain works. Is there some type of that rationale built into the program?”

ISTE’s Edtech Advisor can also help educators with tool selection. This platform, available to ISTE members, allows educators to rate and review edtech tools and apps based on their experience using them in their classrooms in their particular contexts.

5. Implementation and ongoing support

When technology gathers dust in a storage closet, it’s often because the school or district either didn’t have an implementation plan in place or didn’t provide enough training and support for teachers. In a nationwide survey by Common Sense Media, 31% of educators reported that they’re not able to use technology because of a lack of training. More than 60% of teachers said they receive insufficient communication from their district about the edtech available for classroom use.

Before making a purchase, it’s important to consider whether the right policies, staff and training programs are in place to support the tool’s ongoing use. Otherwise, the technology is likely to offer a poor return on investment.

At Oak Park School District 97 in Illinois, senior director of technology Michael Arensdorff tracks ROI through Learn-Platform, which allows him to see how often each app is used. Through ongoing
tracking and analysis, he’s able to pinpoint which tools aren’t working as intended and why.

“Do we renew next year, or do we need to gather more data? Do we need to provide more professional learning to maybe a grade level or a certain building or the whole project? We can identify where we need more professional learning around how the tool should be used in the classroom.”

Thoughtful implementation, ongoing training and adequate support are just a few of the conditions needed to get the most value from edtech purchases. The ISTE Essential Conditions offer a framework for developing the elements necessary for effective implementation and use of technology. When incorporated into the procurement process, they can help districts successfully leverage digital tools to improve student outcomes.

Procurement varies from district to district, but the five pillars represent the key areas every educator needs to consider when making decisions about edtech. As districts increasingly focus on revamping their procurement processes to reflect these best practices, increasing teacher awareness is critical.

“It’s important that districts include teachers when talking about the procurement process,” Bearden says. “They need to make sure they clearly communicate policies around procurement to their school communities, not just once but regularly.”

Nicole Krueger is a freelance writer and journalist with a passion for finding out what makes learners tick.