# data[].attributes.body
One of the foundational questions instructional designers are taught to ask about Ed-Tech is: "do you really have a good idea for an instructional use of this tool, or do you just want to have the latest cool thing you have heard about in your class for the sake of having it?"  //
Because of that, we shoot down a lot of tools before we even get into the harder questions about data, surveillance, equity, etc. Something I have to remind myself of all the time is that we still need to ask the harder questions no matter what the answer to that question is.

Matt (does NOT research potatoes Academia dot edu)
@grandeped 1:33 AM · Nov 15, 2019·Twitter Web App

# data[].attributes.body
The University of Sheffield was using Twitter to improve its internal engagement and visibility when it took the bold step of using this channel to provide communications around a disruptive technical infrastructure upgrade in 2012. The aim was to move away
from a culture of secrecy around maintenance work and keep the community informed at every step. Previously,
communications had typically consisted of one announcement to make people aware of the timing and impact of
maintenance then a follow-up announcement when the work was complete. This allowed others to rearrange their
work around the maintenance period leaving the IT team in peace. Users asking for updates were dismissed in a
similar way to children on a long car journey persistently asking “are we there yet?” The upgrade did not go as planned. There was a risk in exposing these setbacks but the transparency
about the process gained widespread support. Services that should have been restored by 9am were not operational until the afternoon. Instead of complaining, university colleagues sent messages of support for all the work that had gone in and for how the communications had been managed.

"Hundreds of tweets of support came from CiCS colleagues, University partners and followers at other institutions. This had ceased to be an inconvenience imposed by the IT department and had become a team event that we were all part of. The sys admins were not
seen as failing, instead they were modern day heroes who
had worked through the night and who would not stop
until their work was done"

Universities and Colleges Information Systems Association. (2015). Social media toolkit: A practical guide to achieving benefits and managing risks.

# data[].attributes.headline
Government supplies IT to Malaysian classroom teachers

# data[].attributes.subject
Government supplies IT to Malaysian classroom teachers

# data[].attributes.body
"the government just introduced ICT in a drastic and sudden manner without proper consideration for the teachers who are the real implementers, the situation on ground and the students who are at the receiving end. Most probably that is why I don’t use it and some other problems associated with it.”

"The Science teachers all identified lack of consultation
as a prime factor that contributes to less use of ICT in teaching ... “The process is top
down and we have to follow no objection if we object, no point. The government is
too egoistic and business minded.”

"“Normally they will not consult us they use experts and direct us to use”.

"We need to know more about how to use it to teach the subject, how to use it to do some
other things like preparing exam, recording result etc”

"“ICT is interesting and very
useful especially for weak students but for good students, they don’t really like it
because it is too slow for them; they understand quickly and prefer working more
exercises”

Adedokun-Shittu, N. A., & Shittu, A. J. K. (2012). Critical issues in evaluating education technology. In R. Luppicini (Ed.), Cases on Technologies for Educational Leadership and Administration in Higher Education: https://doi.org/10.4018/978-1-4666-1655-4

# data[].attributes.body
http://calleam.com/WTPF/?page_id=3
worth a look, about project management of general projects

also https://www.technologyreview.com/s/401465/10-technology-disasters/

https://er.educause.edu/articles/2016/6/avoiding-failure-with-higher-education-technology-projects

https://edtechmagazine.com/higher/article/2019/10/it-leaders-meet-faculty-and-staff-halfway-classroom-technology-integration?utm_source=Direct

# data[].attributes.body
https://hbr.org/2019/10/why-companies-do-innovation-theater-instead-of-actual-innovation
Stage 1: Organizational Theater. Hire consultants and reorg to a matrix structure. Everyone is busy navigating the new organizational structures and reporting lines, and very little of substance is accomplished.

Stage 2: Innovation Theater. Lots of innovation activities, such as hackathons, design thinking classes, innovation workshops. Folks start to use innovationspeak, but the core business operations are little changed.

Stage 3: Process Theater. Efforts are made to change processes such as procurement, personnel, security, legal, etc., in order to execute on an innovation mandate. As these process changes are divorced from long-term and durable strategic investments, they are as durable as building sandcastles on the beach.

# data[].attributes.attachment_link.readable_text
In his recent Harvard Business Review article " Why Companies Do 'Innovation Theater' Instead of Actual Innovation," Steve Blank's focus was on companies and government agencies. While there are lots of problems with applying the analysis in the article to higher education -- the most significant of which is that we do not produce a product like many industries -- Blank's understanding of how organizations often approach innovation initiatives theatrically, as opposed to strategically, is worth considering in the context of higher education.

The HBR article identifies three stages company and government agencies typically go through when enacting an innovation agenda. These three stages (synthesized below) may sound familiar to many academics.

Stage 1: Organizational Theater. Hire consultants and reorg to a matrix structure. Everyone is busy navigating the new organizational structures and reporting lines, and very little of substance is accomplished.

Stage 2: Innovation Theater. Lots of innovation activities, such as hackathons, design thinking classes, innovation workshops. Folks start to use innovationspeak, but the core business operations are little changed.

Stage 3: Process Theater. Efforts are made to change processes such as procurement, personnel, security, legal, etc., in order to execute on an innovation mandate. As these process changes are divorced from long-term and durable strategic investments, they are as durable as building sandcastles on the beach.

Sound familiar?

In our travels to college and university campuses, and in our discussions with colleagues at peer institutions, we have witnessed the full range of institutional learning innovation efforts. At some schools, learning innovation is identified by leadership and embraced by faculty and staff as core to the identity and culture of the institution. At other schools, learning innovation projects come to life as expressions of a provost's or president's vision, but they fail to take root in the operations of the institution and therefore seldom survive the inevitable leadership turnover.

The university equivalent of organizational theater sometimes involves consultants, but it often starts with a new leader coming to campus. What results is a snow globe reorg, with lots of shaking and movement, but almost always very little change in how teaching and learning occur once the snow settles.

This is often a result of the reorg coming before the articulation of a learning innovation strategy. Ideally, any campus reorganization should be in the service of a strategy. Moving the organizational pieces around by merging groups or changing reporting lines can be satisfactory, as it feels like rapid change in a normally slow-to-change academic environment, but not necessarily successful long term.

The reality is that a reorg can cause damage as easily as it can unleash creative productivity. Learning organizations within a university, say a center for teaching and learning and an academic computing unit, develop their own cultures and maybe most importantly often have vastly different scopes and mandates. Combining groups together in the name of greater collaboration and communication almost always sounds like a smart move -- after all, who wants to defend organizational silos? -- but can lead to scope creep, reduction of services and often a resistance to innovation rather than an embracing of it.

It is not that reorgs to advance learning innovation are bad. It is that the reorg should not only align and support the strategy for learning innovation. It needs to do so while recognizing the importance of supporting ongoing institutional needs -- how we find new ways to transport goods should not come at the expense of no train running on time. At the same time, any reorganization should also recognize the incredible value of the creative, thoughtful, innovative people who carry out this work. Too often institutions of higher education undervalue the very people who make the institutions what they are, assuming they are disposable in the name of efficiency or superficial strategy.

A reorg strategy should be developed and refined with the faculty and nonfaculty educators who will be implementing the projects and initiatives that have been identified. If the community of educators agrees with (or at least understands) the learning innovation strategy, then a campus reorg will have a better chance of being productive, rather than theatrical.

The activities of innovation theater that the HBR article identified for stage two have their own higher ed analogs. Among a certain set of elite institutions, the shiniest set of activities that flew under the banner of learning innovation over the past few years were MOOCs. With the MOOC hype bubble now deflated, new campus initiatives may loudly proclaim the virtues of mobile learning (everyone gets an iPad) or the virtues of learning analytics (everybody gets a dashboard).

Learning innovation initiatives that rest on a foundation of new technologies (iPads, dashboards) or new technology-enabled programs (MOOCs) are excellent at generating noise. What technology-based campus initiatives are less good at is enabling sustainable and resilient campus change. This is especially true in the domain of teaching and learning, an area where institutional structures and norms are deeply entrenched.

Institutional investments in technology in the name of promoting innovation will have little impact if the foundational structures in which teaching and learning occur remain unaltered. No technology will compensate for an institutional faculty strategy that prioritizes the hiring of adjunct and other contingent faculty over the investment in tenure-track positions. Technology will be ineffective in improving student learning if the educators doing the teaching are not well supported and securely employed.

The higher ed learning-innovation equivalent of HBR 's stage three of process theater is where the analogy to higher education is harder to align. Higher ed is not in the business of creating products. We've argued elsewhere that this is the biggest misconception of the disruption movement of the past six or seven years.

Still, there are processes that can create the illusion of innovation that are worth unpacking. For instance, the process of moving existing residential degree programs online is not innovative in and of itself.

Online education is not an innovation.

What would be innovative would be to invest in fundamental changes in how the program is delivered, or whom it is delivered for. These fundamental changes may be to pair faculty with a learning design team and to build the courses around a combination of professor's subject matter expertise and an instructional designer's knowledge of learning science. Equally innovative would be to develop an economic model for the online degree program that allows for the price of tuition to be radically lower than the on-ground program equivalent.

How colleges and universities will avoid learning innovation theater is to commit to rigorously aligning new educational programs to a broader strategy. This strategy should include a commitment to a relational, as opposed to a transactional, model of education. The well-being of the educator, as well as the success of the student, must also be understood as a feature of a well-designed learning-innovation strategy.

The good news, as we discovered in writing a book, Learning Innovation and the Future of Higher Education, is that an ever-increasing number of colleges and universities are moving beyond the stages of learning innovation theater. Having learned from the disappointments of technologically driven initiatives and educational outsourcing, most schools are moving toward a more deliberate and systematic approach to learning innovation.

This shift in strategic focus toward institution-led learning innovation across the higher ed ecosystem is still new, and largely untested for resilience or impact. Determining if the pivot away from learning innovation theater, and toward a set of more durable and consequential learning-innovation structures and activities, will require that we in higher ed commit to studying these changes.

We should not allow the explanations and theories for how companies evolve (disruption!) to be grafted once again onto the narrative of how learning is advancing across the higher education ecosystem.

# data[].attributes.attachment_link.title
How Universities Can Avoid Learning Innovation Theater

# data[].attributes.attachment_link.description
In his recent Harvard Business Review article " Why Companies Do 'Innovation Theater' Instead of Actual Innovation," Steve Blank's focus was on companies and government agencies.

# data[].attributes.body
"Understanding the social and technical aspects of integrating a new student success technology is the difference between implementation and adoption of student success tools in higher education. While it sounds like a no-brainer, it is surprising how often projects neglect the activities required to bring about full adoption."

# data[].attributes.attachment_link.readable_text
Understanding the social and technical aspects of integrating a new student success technology is the difference between implementation and adoption of student success tools in higher education, and yet projects often neglect these critical activities that can bring about true adoption.

Credit: Vladimir Sazonov / Shutterstock.com © 2019

Does one of these scenarios sound familiar? Faculty and staff at your college or university are excited about the introduction of the latest set of student success tools, confident that the tools will be the key to revolutionizing processes and addressing problems and concerns about things like outreach and support, communication, and student risk. But after the launch of your student success technology and after the fanfare dies down, you find that the promised outcomes haven't been realized. Conversely, you may have experienced a dearth of excitement around a new tool, and you are left with the work of extracting value from the implementation.

If either of these scenarios has happened to you, you're not alone. In fact, too many student success leaders face these frustrations. New student success technology is not only expensive (in terms of both time and money spent) the implementation process may also be disruptive to students and faculty alike.

Instead of wondering what you missed, what you forgot, or why the implementation was unsuccessful, let's talk about the underpinnings of why your college's or university's culture just spit out your plan. If you are back at the drawing board, let's cut to the chase before you leave empty-handed. It's not enough to say culture is the reason why implementation of the tools didn't work. Understanding the social and technical aspects of integrating a new student success technology is the difference between implementation and adoption of student success tools in higher education. While it sounds like a no-brainer, it is surprising how often projects neglect the activities required to bring about full adoption.

The adoption problem is multifaceted. A socio-technical approach to change considers an organization in the context of multiple factors that interact when planning change. A socio-technical approach recognizes two core areas: social (structure and people) and technical (technology and task). This broad approach helps in understanding the relationships between people, tasks, structures, and technology. When change takes place in one area, all areas must be evaluated to understand the impact on each of the components.

Bingo! All areas and components are related, which means that there should be a plan in place to achieve the goals of your student success technology implementation. Let's discuss how each of the core themes work together to maximize your investment and time.

Social (Structure and People)

The social aspect is the heart of your technology implementation. The socio-technical framework gives a methodology for analyzing relationships between people and technologies in workplaces. Without people, there is no implementation. The following social issues related to people and organizations (relationships, rules, etc.) should be considered:

How are people introduced to the technology that you are about to implement?

Were they included in the decision?

Do they see themselves in the solution, or are they a checkbox on your implementation road map?

What type of training is required?

Don't forget that training people to use the new technology sends a message regarding expectations. With technology, we can become better, faster, and smarter, but these benefits can come with a cost if we don't recognize the people who are needed to support the technology and the skills required to execute new tasks. Understanding the impact that the student success technology implementation will have on IT professionals, institutional researchers, administrators, instructors, academic advisors, and student services staff is primary.

Of course, the human resources (HR) department should be involved as well. You might ask yourself why in the world you would bring HR into the picture. Here's why: You will be asking key stakeholders to adopt new processes that come with myriad tasks and new expectations to support new student success efforts. HR can help you to communicate and align job functions and skills and can provide training and support. Knowing what skills are required (e.g., programming, providing data support, and advising students) will help you understand who needs to be involved and how the changes to support student success practices will impact people as you navigate your institutional structure to produce positive change.

Technical (Technology and Task)

Here's the golden rule of technology adoption: "Process first. Technology second." The two go hand in hand, but it's best to lead with the process and the "why." The why of your implementation is not to introduce a technology for the sake of doing so. The why is your focus on student success and the adoption of new practices to support your institution's goals.

Now, let's talk tasks. What outcomes are you trying to achieve, and how do you plan to get there? How you plan to get there directly impacts the tasks that people execute each day. For example, let's say you just purchased a new early-alert system. What tasks come along with it? Instructors will need to raise flags, and most likely, student services professionals will be asked to adopt new outreach strategies and new ways of recording outcomes using technology. Do any members of your campus already perform these tasks? If they do, great, but if they don't, it's going to take some time to break down a new plan, identify core tasks, and describe the roles people will play when integrating technology into existing day-to-day work.

Conclusion

Technology changes the social and technical aspects of your organization. Introducing technology asks new stakeholders to come to the table, programmers to learn a different set-up, academic advisors to learn unfamiliar practices and technology, instructors to raise or clear an early-alert flag, and student services staff to monitor a student's academic progress in unique and uncharted ways. While technology introduces opportunities to support student success like never before, higher education student success leaders must first move beyond their organizations' cultures to understand how to achieve the ideal outcome to meet their institutions' student success goals.

Tina Balser is the Director of Student Success Initiatives at the University of Missouri.

# data[].attributes.attachment_link.keywords[]
moving beyond culture: a socio-technical approach to understanding student success technology adoption

# data[].attributes.attachment_link.title
Moving Beyond Culture: A Socio-Technical Approach to Understanding Student Success Technology Adoption

# data[].attributes.attachment_link.description
Understanding the social and technical aspects of integrating a new student success technology is the difference between implementation and adoption of student success tools in higher education, and yet projects often neglect these critical activities that can bring about true adoption. Does one of these scenarios sound familiar?

# data[].attributes.body
Education leaders always need to 'make their mark', which invariably involves a new project that requires the resources from previous projects. Previous projects may just be starting to gain traction when this happens.
This fuels an "I'll just wait this fad out because another one will be along soon" set of behaviours amongst permanent academic staff. Permanent academic staff then feel incentivised to (ignore and) get left behind on such initiatives, exacerbating the problem that led to the initiative in the first place. Eventually the initiative uses its early gains to gain supporters amongst established staff begins to gain real traction. Then the cycle repeats

# data[].attributes.body
Ben Orozco @benoror

Replying to @benoror @ryanflorence and 3 others  ... What I mean is that by looking at the code and product features I can tell it evolved towards the same *all-in-one* solution for education institutions. //

It's Conway's law that is projected from Ed institutions bureocracy reflected in product features bloat. Which is not necessarily a bad thing, specially if it ships and sells well, even to the beforementioned middle-men. [Conway's law:
organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations. — M. Conway[2] ]

# data[].attributes.body
Christon Pierce

@CaliforniaSalvo

Replying to @J_Plus @random_walker @zeynep

This is terrifyingly true with much of the newer, interventions-based online curriculums schools are implementing. Targeted to the admins with lots of data to mine, horrifying for teachers to instruct with.

# data[].attributes.body
Keikaku Doori @J_Plus

Replying to @random_walker @zeynep

100% true. Blackboard, like most educational software/tech is made for Deanland, where flashy interfaces and lots of buzzwords hide a complete lack of substance. I also think the disciplines themselves do this, emphasizing glib "quantifiable" methodologies to get that deanland $.

# data[].attributes.body
Dmitriy Likhten

@DmitriyLikhten

Oct 12

Replying to

@random_walker

Twitter is different. Any product that has to convince the users to use it has to try real hard. Any product that has a captive audience doesn't.  Many internal tools suck big time because nobody cares about the frustrations of internal users. "solve it with training" is common

# data[].attributes.headline
LMSs are like baby clothes - either just for show or really functional

# data[].attributes.subject
LMSs are like baby clothes - either just for show or really functional

# data[].attributes.body
Arvind Narayanan @random_walker

Oct 11My university just announced that it’s dumping Blackboard, and there was much rejoicing. Why is Blackboard universally reviled? There’s a standard story of why "enterprise software" sucks. If you’ll bear with me, I think this is best appreciated by talking about... baby clothes! // There are two types of baby outfits. The first is targeted at people buying gifts. It's irresistible on the rack. It has no fewer than 18 buttons. At least 3 people are needed to get a screaming baby into it. It's worn once, so you can send a photo to the gifter, then discarded. // Other baby outfits are meant for parents. They’re marked "Easy On, Easy Off" or some such, and they really mean it. Zippers aren't easy enough so they fasten using MAGNETS. A busy parent (i.e. a parent) can change an outfit in 5 seconds, one handed, before rushing to work.  //  The point is, some products are sold directly to the end user, and are forced to prioritize usability. Other products are sold to an intermediary whose concerns are typically different from the user's needs. Such products don't HAVE to end up as unusable garbage, but usually do.  //  OK, back to Blackboard! It’s actually designed to look extremely attractive to the administrators (not professors and definitely not students) who make purchase decisions. Since they can't easily test usability, they instead make comparisons based on... checklists of features.   //  And that's exactly what's wrong with Blackboard. It has every feature ever dreamed up. But like anything designed by a committee, the interface is incoherent and any task requires at least fifteen clicks (and that's if you even remember the correct sequence the first time).  //  Software companies can be breathtakingly clueless when there's a layer of indirection between them and their users. Everyone who's suffered through Blackboard will have the same reaction to this: try having less functionality! edscoop.com/how-canvas-cam...  //  The grumbling about Blackboard has finally gotten loud enough that schools are paying a modicum of attention to usability when evaluating alternatives. Blackboard's market share has dropped dramatically and this will probably continue. Good.  //  Here’s the kicker, though. It's extremely likely that whichever vendor emerges on top will fall into the same trap. The incentives almost guarantee it. Once profs and students put down the pitchforks, committees will go back to their checklists, and feature creep will resume.  //  Blackboard is 20 years old. If Twitter is around in 20 years, let's see how this prediction holds up. And now I have to go rescue a three-month old from an extremely cute and equally uncomfortable outfit.

# data[].attributes.headline
High profile professor decided to embrace the MS Teams trial

# data[].attributes.subject
High profile professor decided to embrace the MS Teams trial

# data[].attributes.body
High profile professor decided to embrace the MS Teams trial and use it with his students (rather than Blackboard). All worked well and everyone loved it, but IT decided there were data security issues with MS Teams (related to a quirk with the way QUT assigns email addresses to students...). Meanwhile, Professor presents on his innovative use of MS Teams at an institution-wide forum, while is has already been decided that no one else should be using it with students...

# data[].attributes.headline
External consultants try and introduce a new system in a research company (Wang and Paper)

# data[].attributes.subject
External consultants try and introduce a new system in a research company (Wang and Paper)

# data[].attributes.body
As for system security which is a big thing for IT, the IT department does a lot of firefighting, that is, they fix a problem without thought of an overall schema to help identify the root cause rather than a symptom. For example, if security needs to be heightened, IT builds a new firewall (or firewalls) to deny malicious access. The problem again is that there is no overall security strategy, just band-aids. At least we never noticed anything that explicitly verbalized or documented. Servers are everywhere with no seeming strategy for coordinating IT resources. (Wang and Paper)

IT had strongly resisted the implementation because they feared that they would lose power over controlling the data and systems.” “This is the power that IT doesn’t want to lose,” according to one of the consultants. ...IT traditionally controls everything that is technology related. With that power, IT is able to operate as they see fit. (Wang and Paper)

IT specialists, guarded by techno-babble (technical jargon), were able to easily shield themselves from any attempts to question their practices or motives in order to defend their turf. According to one consultant, “since most managers do not know IT in any detail, it is not difficult to SCARE people away from potentially poor practices!” The consultants’ perception was that technology intimidation was used as a defense mechanism because business managers are not normally IT literate and are thereby easily intimidated. (Wang and Paper)

# data[].attributes.headline
Lecturer in Earth Science wants to create videos

# data[].attributes.subject
Lecturer in Earth Science wants to create videos

# data[].attributes.body
Lecturer in Earth Science wants to create videos, which feature the slides and him (in a smaller window) simultaneously. We suggest Articulate Rise (as we have a number of licenses, i.e. 100). We teach the lecturer how to use it and how to create his videos. Everyone is happy, except...Articulate Rise has almost no accessible analytics about how students may be using these videos. Also, licenses are expensive, so there are questions around sustainability...where to next?

# data[].attributes.body
[at a vendor expo] Ask precisely what their AI does. If they don’t explain this clearly, in a way that you understand, then they might be hyping up their AI use.

If their materials don’t tell you what you want to know, then ask them directly what impact the AI in their product is having on learning (or teaching), and how long they’ve been using AI to improve what they’re doing.

A company that is genuinely using AI to support education will be able to show what results it has achieved with it, because storing this data is absolutely vital to future development.

Don’t be afraid to ask for examples of how and where a company's AI has been used, and what difference it has made to educational outcomes.

Ask if there are any published papers or bits of research they’ve read recently, which they would recommend. An AI-for-education developer should always be up-to-date with the latest innovations and research.

Not just a quick buck
As teachers, you are experts in teaching and learning, so make sure the company trying to sell to you really does convince you that it is interested in providing educational value and not just making a quick buck.
Luckin, 2019-10-22
https://www.tes.com/news/edtech-how-spot-fake-ai-education

# data[].attributes.body
Right now, higher education has very poor signals to quickly distinguish between those vendors who can prove that their products are effective and those that can't. Between those vendors who are contributing to the general state of knowledge about how to improve student success and those that aren't. Until we can improve the signal-to-noise ratio, we are doomed to ride the roller coaster from hell that is the hype cycle until we pass out from exhaustion.

# data[].attributes.attachment_link.readable_text
I often get asked what themes I see coming out of EDUCAUSE. The last few years, the theme has been "there is no theme," perhaps reflecting the fact that ed tech hype has been in remission. With the exception of the OPM product category, whose hype cycle from zero to peak to trough may have been the fastest we've seen yet, the market seems tired of the endlessly repeating roller coaster ride of hope and disappointment.

This year was a little different. Overall, the conference was the healthiest that I've seen it in a while. Attendance seemed to be up. The balance between IT and learning presentations seemed good. There were more vendors on the floor, with more big booths (but no magicians, jugglers, or sword swallowers, thankfully). Overall, the organization seems to be undergoing a revitalization under John O'Brien.

Within that big picture, there was some color. The major textbook publishers had very little presence, perhaps reflecting their current financial state or perhaps indicating that they've figured out the EDUCAUSE crowd isn't really their target audience, for the most part. On the other hand, ERP/SIS vendors were back with bigger booths, including some names I didn't recognize (like Unit4, which apparently has been around for a while but which hadn't heard of before this year).

There also was a proliferation of small vendors with various flavors of solutions aimed at improving student retention and graduation rates. Each one had its own take on the problem. One had an integration dashboard, pulling in data from the LMS, SIS, and other applications to tell advisors when students might be struggling and need encouragement. Another took a similar approach but focused on giving students nudges directly. Yet another focused on engaging students on social media. And so on.

Whenever I saw one of these companies, I tried an experiment. I'd do a quick scan of their booth posters and video screens. Just enough to get a very rough sense of what they claimed to do. In other words, I looked about as carefully as the typical exhibition floor browser looks. Then I would approach the booth and say, "I have a rough sense of what your product does from what I see on your booth." I have one question for you: How do you know that it works?"
"Our customers love it."
I got a range of different answers.
"We have dashboards that show 67 points where students could be getting bogged down." "OK, but how do you know that it works? I mean, have you tested it to see if it actually impacts outcomes?" "...Well...we have a white paper."
Bzzt. Wrong answer.
"We have a customer who improved their retention by 60%! But to be truthful, our product was one of a number of initiatives they put into place, so I don't know how much of that improvement they would attribute to it."
Pass.
"We have conducted multiple randomized controlled trials." "Oh, really! What result were you testing against?" "Oh, right, of course. Year-to-year retention..." "Thanks, you can stop there for now. May I please have your business card?"
Close, but no cigar. Points for honesty, though.

I found one-just one-vendor who appears to be able to deliver the goods on evidence. I wouldn't have picked up on that from my cursory scan of their booth. They didn't look any different than any of the others I had spoken to. In fact, if you had asked me which of the products I looked at would have been likely to prove out as effective, I wouldn't have ranked them near the top of the list. The trick that they advertise most heavily, which is that they repurpose existing advertising mechanisms on popular social media platforms to deliver productive nudges, seems clever but a little disconnected from problems specific to retention. While they mention "behavioral science" on their web site, they don't provide a lot of detail. That said, if you read their marketing copy carefully, you'll see that they do provide a rather stripped-down, easy-to-follow explanation of a randomized controlled trial that they conducted.

The name of that company, by the way, is Motimatic. Have you heard of them? Because I haven't. I'm going to look into them and find out more. Maybe you should too.

Right now, higher education has very poor signals to quickly distinguish between those vendors who can prove that their products are effective and those that can't. Between those vendors who are contributing to the general state of knowledge about how to improve student success and those that aren't. Until we can improve the signal-to-noise ratio, we are doomed to ride the roller coaster from hell that is the hype cycle until we pass out from exhaustion.

I'm going to do something about improving the signal-to-noise ratio, and I will be announcing what that something is within the next week.

Stay tuned.

# data[].attributes.attachment_link.description
I often get asked what themes I see coming out of EDUCAUSE. The last few years, the theme has been "there is no theme," perhaps reflecting the fact that ed tech hype has been in remission.

# data[].attributes.headline
Is change management all that edvisors do?

# data[].attributes.subject
Is change management all that edvisors do?

# data[].attributes.body
I've had two enlightening chats with academics in business faculties to get my head around organisational theories and models recently. Both mostly wanted to talk about change management in edvisor work - I know this is a big part of what we do but is it all?

@TELedvisors
@gamerlearner

||

I don’t think that it is all, but understanding and engaging with the impacts, influences and resistances of change intersects with most of what edtech does - especially moving from pilot to sustainable  @PeterBryantHE

8:33 AM · Oct 22, 2019