# data[].attributes.attachment_link.readable_text
Photo credit: kjarrett via flickr (CC BY 2.0)

The breakthrough happened after the student took the Bartle's Gamer Profile Quiz and we found out that he was a "killer." Off-the-charts killer, but achievement meant nothing to this student. Just like grades.

No, we haven't identified the next school shooter, and I sure wish that Bartle hadn't named one of the four gamer profiles "killer" -- but nonetheless, this student identified with this profile. Jane McGonigal mentioned it in her Gaming Can Make a Better World TED Talk when she discussed an epic meaning. My so-called "killer" student (and we really should rename this when applying it to education!) simply saw things as a battle between good and evil and wanted to fight on the side of good in an epic quest to make the world a better place. Points don't matter in gameplay, and grades don't matter, either. But when we tweaked the kinds of work he was doing in our Gamifi-ED project to focus on "world-changing games," he was suddenly engaged. Now his face lights up when he sees me. He's one of the first kids to class. He's an engaged gamer and, finally, an engaged student.

My ninth grade students have partnered on an epic quest with grad students at the University of Alaska Southeast and members of the Gamifi-ED OOC to study serious games, create an encyclopedia of serious games, and ultimately to create their own serious game in Minecraft. This adventure merits a post of its own, but for now, let's go through the essential elements that we should consider as gaming and education come together.

1. Game Mechanics

Game mechanics are part of game theory. The problem with most attempts at gaming in education is that educators mistakenly think that if you give out a badge or slap points on it, you've gamified. This is wrong. Game theorists have uncovered 24 ways for truly motivating gamers to participate and engage. If you're going to engage your students using any form of gaming, you should understand game mechanics.

2. Bartle Test of Gamer Psychology

As we saw with my "killer" student, there are four game-player types using this psychological evaluation. You can take the quiz to determine which type you are. Realize that you should consider all four of the player types as you design your experiences.

3. Gamification is a Process

Sixth grade teacher Michael Matera wowed me and other members of the OOC as he shared how he has completely gamified his sixth grade classroom. (See the video from his Google Hangout below.) He also delineates the difference between grading and gamifying, as privacy requires that you not publicly share grades. The other important element I've adopted is that badges should be printed out and handed to students (they love putting them on the front of their notebooks, he says). He also shares how much it motivates a love for learning. I'm reminded of the SAMR model of technology integration in that the final step is redefinition -- what happens when you truly gamify a classroom. To risk sounding cliché, gamifying the classroom is a game changer in every way.

4. Serious Games Can Be Used Anywhere

During the Gamifi-ED OOC, I was astounded to see higher ed, elementary ed, and everyone in between share countless ways that gaming can enrich the classroom. From donated Xbox, PS3 and Wii systems to games from various app stores, there are so many ways to enrich learning with the right game.

5. Serious Games Can Tackle Serious Issues

As my students compile our encyclopedia with the higher ed teachers, we're finding game experiences that tackle serious issues, such as depression, genocide in Darfur and the Arab-Israeli conflict in the Middle East, with respect and learning.

6. We Can't Trust App Stores to Curate Our Games

Education app stores are a virtual Wild West of unclaimed, unverified games and apps. Using a rubric developed by the higher ed students, we've already tested more than 20 games (and have another 60 in process) with students and educators ranking them by research-based education best practices. The highest any game can score on our scale is 30. Thus far, we've found only two games that have broken 20, and only one math game -- Cockroaches vs. Algebra -- has gotten close, scoring 19.

Endgame: Finding a "Killer" Tool

Social media and word of mouth recommendations have taken us this far, but it's time for some serious testing of serious games to determine which ones are appropriate for classroom use. Instead of relying on unknown algorithms to determine the best learning games, the education world needs a vetting process that will take us beyond non-teaching game company employees who currently determine what's hot and what's not. Gamifi-ED is definitely a step in the right direction, and if you or your student testers want to join, just contact us.

Gaming in education is already full of experts. Minecraft, World of Warcraft and even a new site called Growtopia are all places where we can share, grow and learn. It's time to move forward into best practices, curation methodologies that will truly help us apply education theory to gaming, and an understanding of gaming that will help us apply gaming theory to education. I'm excited about gaming as a powerful "killer" tool in a 21st century teacher's toolkit. Let's take what we're doing farther.

# data[].attributes.attachment_link.description
Photo credit: kjarrett via flickr (CC BY 2.0) The breakthrough happened after the student took the Bartle's Gamer Profile Quiz and we found out that he was a "killer." Off-the-charts killer, but achievement meant nothing to this student. Just like grades.

# data[].attributes.headline
Museums, Libraries, and 21st Century Skills: Definitions

# data[].attributes.subject
Museums, Libraries, and 21st Century Skills: Definitions

# data[].attributes.body
Learning and Innovation Skills

CRITICAL THINKING--

# data[].attributes.attachment_link.provider_name
Institute of Museum and Library Services

# data[].attributes.attachment_link.readable_text
The IMLS Project Team and Task Force considered the list of skills commonly referred to as "21st Century Skills" and modified it slightly to better align with library and museum priorities. ¹

The resulting list includes the following additions: Basic Literacy, Scientific & Numerical Literacy, Visual Literacy, Cross-Disciplinary Skills, and Environmental Literacy.

Not every skill on this list will be aligned with every institution's vision and mission. Further, not every community will prioritize the same skills. Library and museum leaders should consider this list as a starting point beyond which it should be customized to fit the unique character, requirements, and priorities of the institution and its audiences.

Critical Thinking and Problem Solving

Reason Effectively

Use various types of reasoning (e.g., inductive, deductive, etc.) as appropriate to the situation

Use Systems Thinking

Analyze how parts of a whole interact with each other to produce overall outcomes in complex systems

Make Judgments and Decisions

Effectively analyze and evaluate evidence, arguments, claims and beliefs

Analyze and evaluate major alternative points of view

Synthesize and make connections between information and arguments

Interpret information and draw conclusions based on the best analysis

Reflect critically on learning experiences and processes

Solve Problems

Solve different kinds of non-familiar problems in both conventional and innovative ways

Identify and ask significant questions that clarify various points of view and lead to better solutions

Creativity and Innovation

Think Creatively

Use a wide range of idea creation techniques (such as brainstorming)

Create new and worthwhile ideas (both incremental and radical concepts)

Elaborate, refine, analyze, and evaluate ideas in order to improve and maximize creative efforts

Demonstrate imagination and curiosity

Work Creatively with Others

Develop, implement, and communicate new ideas to others effectively

Be open and responsive to new and diverse perspectives; incorporate group input and feedback into the work

Demonstrate originality and inventiveness in work and understand the real world limits to adopting new ideas

View failure as an opportunity to learn; understand that creativity and innovation is a long-term, cyclical process of small successes and frequent mistakes

Implement Innovations

Act on creative ideas to make a tangible and useful contribution to the field in which the innovation will occur

Communication and Collaboration

Communicate Clearly

Articulate thoughts and ideas effectively using oral, written, and nonverbal communication skills in a variety of forms and contexts

Listen effectively to decipher meaning, including knowledge, values, attitudes, and intentions

Use communication for a range of purposes (e.g., to inform, instruct, motivate, and persuade) and in diverse environments (including multi-lingual)

Utilize multiple media and technologies, and know how to judge their effectiveness a priori as well as assess their impact

Collaborate with Others

Visual Literacy

Demonstrate the ability to interpret, recognize, appreciate, and understand information presented through visible actions, objects and symbols, natural or man-made²

Scientific and Numerical Literacy

Cross-Disciplinary Thinking

Apply knowledge, attitudes, behaviors, and skills across disciplines in appropriate and effective ways

Basic Literacy

Demonstrate the ability to use language to read, write, listen, and speak

Information Literacy

Access and Evaluate Information

Access information efficiently (time) and effectively (sources)

Evaluate information critically and competently

Use and Manage Information

Media Literacy

Analyze Media

Create Media Products

Understand and utilize the most appropriate media creation tools, characteristics, and conventions

Understand and effectively utilize the most appropriate expressions and interpretations in diverse, multi-cultural environments

ICT (Information, Communications and Technology) Literacy

Apply Technology Effectively

Global Awareness

Financial, Economic, Business, and Entrepreneurial Literacy

Civic Literacy

Health Literacy

Obtain, interpret, and understand basic health information and services and use such information and services in ways that enhance health

Understand preventive physical and mental health measures, including proper diet, nutrition, exercise, risk avoidance, and stress reduction

Use available information to make appropriate health-related decisions

Establish and monitor personal and family health goals

Understand national and international public health and safety issues

Environmental Literacy

Demonstrate ecological knowledge and understanding of how natural systems work, as well as knowledge and understanding of how natural systems interface with social systems
Demonstrate understanding of the relationship between beliefs, political systems, and environmental values of various cultures
Demonstrate understanding of environmental issues caused as the result of human interaction with the environment, and knowledge related to alternative solutions to issues
Demonstrate active and considered participation aimed at solving problems and resolving issues³

Flexibility and Adaptability

Adapt to Change

Adapt to varied roles, job responsibilities, schedules, and contexts

Work effectively in a climate of ambiguity and changing priorities

Be Flexible

Initiative and Self-Direction

Manage Goals and Time

Work Independently

Monitor, define, prioritize, and complete tasks without direct oversight

Be Self-directed Learners

Go beyond basic mastery of skills and/or curriculum to explore and expand one's own learning and opportunities to gain expertise

Demonstrate initiative to advance skill levels towards a professional level

Demonstrate commitment to learning as a lifelong process

Reflect critically on past experiences in order to inform future progress

Social and Cross-Cultural Skills

Interact Effectively with Others

Know when it is appropriate to listen and when to speak

Conduct oneself in a respectable, professional manner

Work Effectively in Diverse Teams

Productivity and Accountability

Manage Projects

Set and meet goals, even in the face of obstacles and competing pressures

Prioritize, plan, and manage work to achieve the intended result

Produce Results

Demonstrate additional attributes associated with producing high quality products including the abilities to:

Work positively and ethically

Manage time and projects effectively

Multi-task

Participate actively, as well as be reliable and punctual

Present oneself professionally and with proper etiquette

Collaborate and cooperate effectively with teams

Respect and appreciate team diversity

Be accountable for results

Leadership and Responsibility

Guide and Lead Others

Use interpersonal and problem-solving skills to influence and guide others toward a goal

Leverage strengths of others to accomplish a common goal

Inspire others to reach their very best via example and selflessness

Demonstrate integrity and ethical behavior in using influence and power

Be Responsible to Others

Act responsibly with the interests of the larger community in mind

1. Except as otherwise noted, the skills definitions are derived from the Partnership for 21st Century Skills Framework.

2. Derived from definition attributed to John Debes, per the International Visual Literacy Association.

3. Adapted from the Environmental Literacy Council Framework.

# data[].attributes.attachment_link.title
Museums, Libraries, and 21st Century Skills: Definitions

# data[].attributes.attachment_link.description
The IMLS Project Team and Task Force considered the list of skills commonly referred to as "21st Century Skills" and modified it slightly to better align with library and museum priorities. 1 The resulting list includes the following additions: Basic Literacy, Scientific & Numerical Literacy, Visual Literacy, Cross-Disciplinary Skills, and Environmental Literacy.

# data[].attributes.attachment_link.description
Being a 21st-century educator is a tall order. In my 15 years in the classroom, my most important "helpers" are technology, movement, and networking. Here's why.

# data[].attributes.headline
Should we teach computer science in elementary school?

# data[].attributes.subject
Should we teach computer science in elementary school?

# data[].attributes.attachment_link.readable_text
We should definitely be teaching computer science (CS) in elementary school. Why?

The most common answer to this question is jobs - but not necessarily traditional computing or programming jobs. Our nation's current trajectory points to a lasting digital era, and we'll need people who can think like software engineers and network architects, whether they are writing an app or solving resource distribution problems in a third-world setting - or doing both at the same time.

But let's forget about jobs for the moment. After all, we don't really know where the economy will be in 15 years, and we're talking about 5- to 10-year-olds here!

The rationale to teach CS to K-5 students goes well beyond career development. For kids just entering school, teaching CS is about giving them the thinking skills that will help them become proactive learners and citizens - as opposed to just consumers and denizens - in a world that's increasingly influenced by the manipulation of the digital bit.

CS for K-5 students is not new. Computing for elementary school kids saw a heyday in the 1980s, when Seymour Papert and the Logo movement took education by storm. But in recent years, it's been enjoying a comeback. ScratchJr, a K-5 version of the popular Scratch programming environment, was released on the iPad this summer, and other platforms are on the way. CS is featured prominently in Project Lead the Way's new K-5 education offerings. And the Code.org team just launched three courses for K-5 as well as a nationwide network of affiliates that offer free, one-day workshops that give every teacher take-home classroom materials.

But to decide whether teaching CS to young students is worth it, you need to understand what computer science in the elementary grades really looks like.

It is not about learning how to use the keyboard and mouse, except for the purpose of moving instruction blocks around to form an algorithm. And it is not just about advocating for safe digital literacy practices, like visiting certain approved websites, unless the website happens to be a coding environment or community where students can share their work.

The most important part of the K-5 CS experience is its ability to encourage and support creative expression and problem solving. As Seymour Papert said more than 30 years ago, CS is about giving kids the opportunity to engage with powerful ideas. The computer just happens to be our era's best and most accessible tool for this purpose. Coding puzzles, tutorial progressions and unplugged activities (learning computing concepts without a computer) are all onramps to a world where students can be passionate and powerful enough to express their imaginations. Creativity, collaboration, persistence and abstraction are all thinking skills that coding builds.

Shouldn't we give all students this opportunity? Can we figure out creative or clever ways to make room in our school schedules for something so important to our digital age students? Shouldn't kids learn how to manipulate bits and digitize problems so they add computing processes to their problem-solving toolbox? And isn't this important enough to start teaching at a young age?

If you agree that the answer to all these questions is yes, then what's stopping us? Many fear that reading, writing and math instruction (and scores) will suffer if we add one more thing to the curriculum. But CS is more than just another subject. It can serve as the glue for interdisciplinary study, which means the time we spend on it is not added, but integrated. And about testing: At this moment, an influx of hardware is on the way to elementary schools to prepare us all for Common Core testing. Shouldn't we take advantage of this opportunity to show young students that computers are not just a toy or testing tool, but a powerful means of expression that enables them to be the drivers of creation?

As most PD providers know, it is not the kids who are most hesitant to try something new. It's us adults. We need to make sure that all K-5 teachers and administrators have the opportunity to participate in high-quality computer science professional learning so they can build enthusiasm for this important subject and learn how to integrate it into their classrooms in relevant and meaningful ways. Just as we want our kids to learn more than just how to count to 10 in math class, we should want to give our young students a thorough background in the field that is revolutionizing every aspect of our world. We owe it to them.

Acknowledgment
Kiki Prottsman, executive director of Thinkersmith, collaborated on this response.
Pat Yongpradit is the director of education for , a nonprofit dedicated to promoting computer science education. His passion is bringing CS opportunities to every school and student in the United States.
My school has computer science instruction in K-8, but it focuses on keyboarding practice, using programs to create products, research skills with proper citation and digital citizenship. Our school has a computer lab, which K-6 students visit once a week and seventh and eighth grades visit twice a week. Is that enough? Given the time allotted in our schedule and the skills the teachers say their students need for their classwork, these are the topics we feel warrant taking up a spot in our curriculum.

I agree that learning how to use many productivity, editing, collaborating, communicating and coding tools would be valuable across the curriculum. But early elementary students' primary needs are literacy and numeracy. For middle elementary, introduction to other skills - particularly coding and other engineering applications - is age appropriate, but anything above introductory instruction goes beyond the limits of the time we have. By middle school, both students and teachers are most focused on research, collaboration and writing skills that will allow them to complete core class homework.

Funding is another consideration. Some districts - particularly large ones - have the budget to implement 1:1 programs that would allow classroom teachers to integrate more computer skills into the core curriculum. But our single-building school faces definite funding limitations, as do many schools in small or rural districts, which also often deal with the added challenge of spotty internet connectivity. Finding both time in the school day and enough devices for all students continue to be major obstacles for us and for many other districts.

Finally, achieving universities' top-down demands to deliver students with writing, comprehension, communication and mathematics skills is already a tall order. Throwing yet another demand into the mix would be unrealistic for many schools, and for the students. Yes, I have read the articles about how unprepared students are to enter college and manage the demands of the digital age workplace. But I think it makes more sense to leave skills such as engineering, higher mathematics and global communication for high school and higher education. Elementary school is the place to master the basics, including literacy - in both language and technology - and numeracy.

I'm not saying that elementary students are not capable of using or even mastering code. But I believe that really teaching - not just introducing - coding is simply beyond the scope of what most K-5 schools and their students are able to do, and it's even asking a lot of middle schools when both lab time and class time are so limited. What's more, pushing students into the study of abstract concepts before they are developmentally ready will not make them any more prepared for the rest of the 21st century than they are now.
Beth Schwartze is the technology coordinator and computer science teacher for grades 5-8 at Our Lady of Lourdes Interparish School in Columbia, Missouri.
Computer science is a necessary part of the whole educational package. However, it is not the only part, nor is it the most important. All too often, there is a tendency to treat computers and other technology as the subject instead of the valuable and powerful tools they can be to teach the skills that are most appropriate at each grade level.

# data[].attributes.attachment_link.title
Should we teach computer science in elementary school?

# data[].attributes.attachment_link.description
We should definitely be teaching computer science (CS) in elementary school. Why? The most common answer to this question is jobs - but not necessarily traditional computing or programming jobs.

# data[].attributes.headline
Moving Beyond Computer Literacy: Why Schools Should Teach Computer Science

# data[].attributes.subject
Moving Beyond Computer Literacy: Why Schools Should Teach Computer Science

# data[].attributes.attachment_link.provider_name
National Center for Women & Information Technology

# data[].attributes.attachment_link.readable_text
Computer Science Means Rewarding Careers

Jobs are plentiful, interesting, and flexible. The U.S. Department of Labor predicts that computer science-related jobs will be among the fastest growing and highest paying over the next decade. Job prospects have remained strong despite economically challenging times. Computer scientists also enjoy a wide range of career options since all industry sectors today involve computing (e.g., the arts, film, finance, health care, journalism, manufacturing, music, security).

Five of the fastest growing occupations are computing occupations.

By 2016, there will be 1.5 million computing-related jobs available.

Computing-related jobs are among the highest entry-level salaries of any bachelor's degree.

Computer Science: More Than Just Using Technology

Computer science teaches students design, logical reasoning, and problem solving - all valuable well beyond the computer science classroom. The ability to create and adapt new technologies distinguishes computer science from computer literacy, which focuses more on using existing technologies (e.g., word processing, spreadsheets).

# data[].attributes.attachment_link.title
Moving Beyond Computer Literacy: Why Schools Should Teach Computer Science

# data[].attributes.attachment_link.description
Jobs are plentiful, interesting, and flexible. The U.S. Department of Labor predicts that computer science-related jobs will be among the fastest growing and highest paying over the next decade. Job prospects have remained strong despite economically challenging times.

# data[].attributes.headline
Visionaries on InnovationPersonal Perspectives from Leading Innovators

# data[].attributes.subject
Visionaries on InnovationPersonal Perspectives from Leading Innovators

# data[].attributes.attachment_link.title
Visionaries on Innovation - Thinking Out Loud - The Henry Ford

# data[].attributes.attachment_link.description
Visionaries on Innovation explores ideas and opinions of some of the most influential thinkers of our time. These articles and videos are provided by The Henry Ford.

# data[].attributes.headline
Dark MatterThe dark matter of the Internet is open, social, peer-to-peer and read/write—and it’s the future of museums

# data[].attributes.subject
Dark MatterThe dark matter of the Internet is open, social, peer-to-peer and read/write—and it’s the future of museums

# data[].attributes.body
I am talking about museums, libraries, and archives—heritage, culture, knowledge, and memory institutions—and there is really nothing like them on the face of the earth. And whether we’ve realized it or not, my colleagues and I who work with technology in these institutions have been participating in an extraordinary project — the building of a planetary scale knowledge sharing network for the benefit of everyone in the world.

For over 20 years we’ve followed every step in the explosive growth of technology and its impact on society. We’ve digitized our collections and put them online. We’ve built websites and mobile apps; livestreamed lectures and performances; and published electronic books, games, and educational materials. We’ve blogged, tweeted, cataloged, pinned, friended, poked, liked, crowdsourced, uploaded, downloaded, licensed, sold and organized. We’ve learned HTML, CSS, Java, JavaScript, Ruby, PERL, PHP, Python, Drupal, ColdFusion, ASP, ActionScript, Applescript, C, C+, C++, C-sharp, Objective C, LAMP, RAMP, MAMP and a few dozen other technologies, and we’ve built things so elegant and stable that they’re works of art in their own right—and so obtuse and convoluted that they’ll be bringing our colleagues to tears of frustration and rage for years to come.

# data[].attributes.attachment_link.readable_text
The dark matter of the Internet is open, social, peer-to-peer, and read/write - and it is the future of museums

This is the first essay in Code | Words, a collaborative writing project about technology in museums.
Until the 1960s we thought we had a pretty good idea of what the universe was made of. You, me, the earth, the planets, the Milky Way, Renoir's The Boating Party, the pearly plastic of Elvis's guitar pick, the Great Pyramid of Giza, a finch's beak, the sound of Maria Callas' voice and everything else in the cosmos was believed to be made up of particles and energy that we could see, touch, smell, hear, or directly measure with the instruments of science.

But we were wrong. Not just a little wrong; wrong at a stupendous, almost unimaginable scale. And the way in which we were wrong tells us a lot about the way our memory institutions are using technology to accomplish their missions, and the beauty and power that exists now, just beyond their reach.

In 1967, American astronomer Vera Rubin, fresh out of graduate school and eager to begin her new career, started working on a comprehensive survey of the rotational characteristics of spiral galaxies. It was to be a long, grueling project: "A program no one would care about" as she herself described it-thousands of days and nights working in anonymity, gathering data to calculate the speed at which spiral galaxies spin around their dense central cores.

But on Vera Rubin's first night of observations she noticed something startling. Spiral galaxies are beautiful, enormous structures with flat, disk-like bodies that sometimes span millions of billions of kilometers and contain hundreds of billions or trillions of stars. But as marvelous as they are, they are supposed to obey the laws gravity like everything else in the cosmos; it was an article of faith that spiral galaxies would revolve around their axes like the planets in our solar system revolve around the sun, with planets closer to the center of the system spinning faster than those that are far away.

But that is not what Vera Rubin found. She discovered on that night, and in every measurement that she or any other astronomer has taken since, that the outer regions of every spiral galaxy in the universe spun just as fast as their corresponding inner regions, regardless of whether the areas being measured were near the galactic axis or a hundred million billion kilometers away.

It could not be so. These galaxies were spinning so fast that Newton's laws of gravity dictated that they should have flown apart. In one galaxy, Triangulum, the outer regions were moving so quickly that it was as if there were 39 billion invisible suns-seven times the observable mass of the galaxy-quietly pulling the long, graceful arms of the system forward.

Where was the missing mass? Where was the energy and the gravity coming from? How do you explain 39 billion missing suns?

The universe doesn't show its cards very often-flaunting a physical reality that so clearly contradicts the laws of nature-but there was the evidence in the hands of a junior astronomer on her first night of her first project of her first job. And there were only two explanations for what she saw: either there was something wrong with the laws of gravity, or there was an enormous quantity of invisible matter in the universe.

Vera Rubin had discovered dark matter. Her measurements have now been repeated on over a thousand galaxies and the results are all the same: dark matter-whatever it is-seems to comprise most of the mass of the universe.

"We became astronomers thinking we were studying the universe," Rubin said, "and now we learn that we are just studying the 5 or 10 percent that is luminous."

My colleagues and I work in amazing institutions. Some of the institutions are giants, with epic missions like The increase and diffusion of knowledge (Smithsonian Institution, USA), A center for learning, dialogue, tolerance, and understanding (Bibliotheca Alexandrina, Egypt), and [To support] citizens in the defense of their rights and encourage the production of scientific and cultural knowledge (Arquivo Nacional, Brasil). Some of them are small, with just a handful of staff and quiet, humble missions that are nonetheless deeply important to the people they serve.

I am talking about museums, libraries, and archives-heritage, culture, knowledge, and memory institutions-and there is really nothing like them on the face of the earth. And whether we've realized it or not, my colleagues and I who work with technology in these institutions have been participating in an extraordinary project - the building of a planetary scale knowledge sharing network for the benefit of everyone in the world.

For over 20 years we've followed every step in the explosive growth of technology and its impact on society. We've digitized our collections and put them online. We've built websites and mobile apps; livestreamed lectures and performances; and published electronic books, games, and educational materials. We've blogged, tweeted, cataloged, pinned, friended, poked, liked, crowdsourced, uploaded, downloaded, licensed, sold and organized. We've learned HTML, CSS, Java, JavaScript, Ruby, PERL, PHP, Python, Drupal, ColdFusion, ASP, ActionScript, Applescript, C, C+, C++, C-sharp, Objective C, LAMP, RAMP, MAMP and a few dozen other technologies, and we've built things so elegant and stable that they're works of art in their own right-and so obtuse and convoluted that they'll be bringing our colleagues to tears of frustration and rage for years to come.

We've reached, and to varying degrees satisfied, hundreds of millions of people with our efforts, and for more than twenty years we have done it all on shoestring budgets (when we were lucky enough to have budgets at all), in direct competition with some of the biggest and most aggressive media companies in the world, through three global recessions, a global banking crisis, wars in Afghanistan, Iraq and terror (not to mention 61 other global conflicts), and at the end of the day we've almost always generated a positive impact on the visitation, reputation, and financial sustainability of our hallowed institutions.

That's not a bad stretch of work for what is mostly a bunch of self-taught hackers and designers who've been making it up as we've been going along.

But as I look back on these accomplishments and I think of where we are today; how our institutions think about the World Wide Web 25 years after its birth, and the degree to which we understand and are taking advantage of its full capacity to serve, educate, enlighten, and empower our constituents, whoever they are and wherever they live, my thoughts return to the story of Vera Rubin and dark matter.

Despite the best efforts of some of our most visionary and talented colleagues, we've been building, investing, and focusing on only a small part of what the Internet can do to help us accomplish our missions.

90% of the universe is made of dark matter-hard to see, but so forceful that it seems to move every star, planet, and galaxy in the cosmos.

And 90% of the Internet is made up of dark matter too-hard for institutions to see, but so forceful that it seems to move humanity itself.

Fast forward 40 years from Vera Rubin's first spiral galaxy measurements in 1967, to January 1, 2007. On that day, Hank Green, an environmental writer and web consultant living in Missoula, Montana, uploaded a YouTube video in which he bet his brother, writer John Green, living in New York City, that they could go an entire year without texting, sending email, or exchanging written words with each other, in any form. Instead, Hank proposes, they will communicate only through YouTube. They call the effort Brotherhood 2.0 ("365 days of textless communication ") and you can see a short mashup of their first two messages, as well as links to the entire oeuvre of Brotherhood 2.0 videos, here.

These are not high-tech productions and Hank and John Green are not movie stars. It is evident from watching 30 seconds of any of their videos that they are nerds, and they proudly describe themselves as such. If you announced to your museum director or boss that you intended to hire Hank and John Green to make a series of charming and nerdy videos about literature, art, global warming, politics, travel, music, or any of the other things that Hank and John make videos about you would be thrown out of whatever office you were sitting in and probably be asked to find another job.

But over the course of the 1,000-plus Brotherhood 2.0 videos recorded to date, Hank and John Green-or the Vlogbrothers, as they've titled themselves and their YouTube channel (a vlog is a video blog)-demonstrate an exquisite mastery of the dark matter of the Internet. They didn't start by making a website. They didn't develop a mobile app. They didn't have a marketing budget, a content review committee, or a brand strategy. They sat down in front of their cameras and started talking about the things that interested them, and in the process, they got to know their audience. Their work is social, graceful, spontaneous, humble, funny, creative, humane, and generous. And from the perspective of formal institutions, used to working with professional content developers, actors, videographers and web developers, it all looks like a joke.

But look at what happens over the course of the years as the brothers Green move their experiment forward.

They create the Vlogbrothers YouTube channel.

They discover that people are interested in what they do, and they start interacting with their followers.

They start referring to their followers as nerdfighters-nerds who fight "to reduce worldsuck," i.e., bad things in the world.

They begin noticing that the nerdfighters are interested in social causes.

They encourage nerdfighters to donate money to certain charities. (The nerdfighters respond with great enthusiasm.)

They incorporate the Foundation to Decrease Worldsuck, Inc., a Montana based 501(c)3 charitable organization, to receive and re-distribute the charitable donations of nerdfighters.

They create the Project for Awesome, an online film festival in which nerdfighters submit short YouTube videos promoting their favorite charities. In 2013, the Project for Awesome raised $869,146 for the Foundation to Decrease Worldsuck, which then redistributed the money to charities chosen by the nerdfighter community.

They begin talking about Kiva, a non-profit microfinance organization that lets individuals make small, short-term loans to low-income entrepreneurs around the world. The Nerdfighters respond, and to date, the nerdfighters Kiva group has made over 133,000 loans totaling almost $4 million.

Hank Green, noticing that there was no gathering for creators and fans of YouTube videos, starts the VidCon conference. In its first year (2010), VidCon sold out its 1,400 seat venue at the Anaheim Convention Center.

John and Hank sell out Carnegie Hall for an event titled "An Evening of Awesome" in 2013.

With a grant from Google, they create Crash Course, a YouTube channel of short, educational science and humanities videos targeted at advanced high school students. Topics include Ophelia, Gertrude, and Regicide - Hamlet II and That's Why Carbon is a Tramp: Crash Course Biology #1. Crash Course now has 1.7 million subscribers and 101 million views. (This is up by 1 million subscribers and 20 million views since I last checked six weeks ago.)

To sustain Crash Course after the Google grant ran out, they create their own crowd-funding platform Subbable. Subbable allows people to support ongoing creative projects with small monthly donations-including donations of $0. (John Green: "Because people who are struggling financially have enough trouble in their lives without feeling that they can't fully be part of something that matters to them, simply because they don't have money.") Within a week of Subbable's launch, it had more than $30,000 in monthly subscriber pledges, and more than $70,000 in total payments.

They create DFTBA Records ("Don't forget to be awesome"), an online store that features fan merchandise designed by and for nerdfighters.

John's behind-the-scenes collaborator and wife, Sarah Urist Green, a curator at the Indianapolis Museum of Art, creates The Art Assignment, a YouTube channel (produced by the Public Broadcasting Service's Digital Studios) in which Sarah and John teach people about the concepts of contemporary art by challenging viewers to create, share, and discuss their own works and experiments. The Art Assignment has been running for two months and has 74,941 subscribers and 805,801 views.

When I checked this morning, the Vlogbrothers' YouTube channel had,

2,058,156 subscribers

436,847,793 views

The Vlogbrothers network of 26 channels (just stop and think about that for a moment: two nerds built their own network, with 26 channels in it) has 6.8 million subscribers and has received almost 805 million views.

They have 8 times more views and 25 times more subscribers than the Oprah Winfrey network's channel. Note that Oprah Winfrey is a billionaire and an international celebrity. (True, John Green is a best-selling author of young adult fiction, but he's no Oprah Winfrey.)

They have 106 times more views and 759 times more subscribers than the Louvre's YouTube channel. Note that the Louvre is the most visited museum on earth.

To summarize: in seven years, two lovable nerds used YouTube and their own creativity to build what amounts to a vast educational content community that any museum or cultural institution on the planet would be proud to call their own. They've got millions of avid followers, they've helped give millions of dollars to charity, they've elevated and sustained a discourse about culture, science, thought, suffering, and existence-and they're having a blast and making people happy.

They-two nerds- did this in seven years. I've been on website redesign projects that lasted seven years. I've been on committees that took seven years to write a report.

You've probably never heard of the Vlogbrothers or Brotherhood 2.0. Hank and John Green are working in, and they're part of, a kind of Internet production-a kind of interaction-that is difficult for institutions to think of as legitimate, sufficiently respectable, educational, scholarly, or erudite.

But it seems that the public doesn't care. It's likely that the public doesn't think of what memory institutions often do as being sufficiently accessible, smart, joyous, attentive, generous, welcoming, imaginative, bold, educational or meaningful to merit much of their attention.

In the 1990s, researchers from the Urban Institute conducted a study of arts and culture participation in underprivileged communities in Oakland, California. When they surveyed local residents to find out where they got their culture, they were met with blank stares and a general reply of 'we don't have that kind of stuff around here.' But when researchers returned a few months later and asked the question a different way; 'Who are the creative people in your community?' they received an outpouring of information about the artists, musicians, writers dancers, and other creative people who lived nearby. The problem wasn't a lack of culture in the community, it was that people weren't associating their creative lives with the galleries, museums, concert halls, and other formal arts institutions that were created and operated on their behalf.

In a similar vein, the organization UX for Good held a "design challenge" workshop in Washington, DC last February to help generate new approaches to accomplishing the mission of America's National Endowment for the Arts. Seven teams of information architects and user experience designers were invited to invent projects, processes, and programs to "support the arts in every community in the United States," as their brief stated. They were told they would have the equivalent of the NEA's $130 million annual budget, staff of 162 people, and national network of experts- but they were not told that the project was for, or about, the NEA, or that NEA officials were in attendance. When the teams reported back, none of their concepts proposed to use any aspect of the existing cultural infrastructure that the NEA has spent the last 50 years helping to build. In the minds of those designers, America's cultural institutions-its museums, symphonies, operas, ballets, performing arts centers, and other cultural attractions- did not seem to be an asset that would help them support the arts in every community in the United States. (NEA acting chairman Joan Shigekawa and UX for Good co-founder Jeff Leitner, both present at the workshop, told me they were humbled and inspired by these results.)

Online it may be no different. The UK's 40 biggest cultural venues attract less than 0.04% of UK web traffic, observed Culture 24 Director Jane Finnis in the Guardian last year, in an article titled "Why your cultural website is rubbish."

Internet Archive founder Brewster Kahle said of America's academic and research libraries, "The people who are supposed to be doing universal access to knowledge, and are getting $12 billion a year to do it, are not getting the job done."

Some of the disconnect between what institutions could do online and what they do do online can be attributed to the clothesline paradox, a term environmental pioneer Steve Baer coined to describe the phenomenon in which activity that can be measured easily (e.g., running a clothes dryer) is valued over equally important activity that eludes measurement (e.g., drying clothes outside.) The same can be said for the way in which institutions habitually value activity such as visits to museums or journal articles published by their scholars over equally meaningful but more difficult to measure activity such as the sharing of museum-related materials on social media sites or the creation of wikipedia pages.

In Measuring the impact of the sharing economy, Tim O'Reilly writes, "it's quite clear to me that there is a new economy of content that is quite possibly larger than the old one, but just not as well measured, because we measure value captured, not value created for users."

Erik Brynjolfsson's research at the MIT Center for Digital Business indicates that "the value of free Internet goods in 2011 was about $300 billion - and increasing at a rate of about $40 billion a year."

The opportunity here is tremendous, and the experimentation that's happening in the museum, library, and archive community is encouraging. At the current moment, I'm particularly fascinated by the way in which the Rijksmuseum is celebrating the re-use of its openly licensed collections; the civic commitment and spirited execution of the DMA Friends initiative at the Dallas Museum of Art; and the network models and leadership on intellectual property issues being demonstrated by the Digital Public Library of America and Europeana. (I hate to leave worthy projects and individual innovators out of this brief list but I wanted to give some idea of what I've been looking at. There are dozens of individuals and smaller organizations that are experimenting and innovating in breathtaking ways- you know who you are!)

But the question now is how to get these experiments to scale up, to more users, and scale out, across more of our industry. Leaders of cultural technology projects often tell me that they have to fight tooth-and-nail for every dollar of funding and every "yes" of permission from their superiors. Even for projects that are considered to be successful, project teams are painfully aware of how much more they could be doing to scale the impact of their initiatives, if they had adequate support. A director of a digital media group at a major metropolitan library system told me of a recent award-winning project that took two years to bring to fruition; "It's been great, but we should be doing 10 of those a year."

This is not just a matter of the glass being half empty or half full. The disconnect here is that the glass-the Internet and the dark matter of open, social, read/write culture-is so much bigger than we are accustomed to seeing and thinking about. The glass is huge, and it keeps getting bigger, every day.

In Cognitive Surplus, Clay Shirky asserts that among the educated, Internet connected inhabitants of planet Earth, there are 1 trillion hours of free time every year that could be used for community action, civic engagement, and learning. But when Shirky published Cognitive Surplus in 2010 there were only 2.1 billion people online. Almost a billion people have joined the Internet since then. In The New Digital Age, Jared Cohen and Google chairman Eric Schmidt predict that another 5 billion people will come online in the next decade. Even using the gigantic numbers that astronomers use to describe the universe, its difficult to comprehend the combined cognitive abilities, expectations, and desire for learning, participation, and self-improvement that 6, 7, or 8 billion people on the Internet might have.

This is a critical issue that institutions will be contending with for decades to come: There's just an enormous, humongous, gigantic audience out there connected to the Internet that is starving for authenticity, ideas, and meaning. We're so accustomed to the scale of attention that we get from visitation to bricks-and-mortar buildings that it's difficult to understand how big the Internet is-and how much attention, curiosity, and creativity a couple of billion people can have.

By 2011, Facebook's collection of photographs had reached 140 billion images, which was reported to be 10,000 times larger than the online image collection of the Library of Congress. Two years later Mashable reported that Facebook's image repository had reached a quarter of a trillion photos, with 350 million new photos being uploaded every day. In this spring's Aperture magazine, Lev Manovich wrote, "The photo universe created by hundreds of millions of people might be considered a mega-documentary, without a script or director." (Given Facebook's history of borderline exploitation of its user base I hesitate to hold it up as an example in a discussion of open, participatory practices, but the scale of activity is so impressive that it can't be ignored. Furthermore, Facebook is an important channel for many smaller institutions.)

Reddit had over 109 million unique visitors last month, and an astonishing 5.3 billion page views. This is not simply lowbrow entertainment; the top Reddit "Ask Me Anything" discussions have been with Barack Obama, Sir David Attenborough, and Bill Gates.

In 2013, recording artist Moby distributed his latest album as an openly licensed BitTorrent Bundle. TechDirt reported that he "got an astounding 8.9 million downloads of his offering - with 419,000 of them agreeing to join his mailing list and 130,000 of them going over to iTunes to the album (many of which likely resulted in sales)." Fans have created more than 68,000 remixes of his songs.

Tumblr: 185.4 million blogs; 83.1 billion posts. That's all I'm going to say about Tumblr.

In two years, Pinterest has grown from a concept to serving 10s of billions of page views a month. As of April, 2014 they were reported to have 30 billion pins on 750 million boards.

People upload 1.2 billion photos to the top four mobile social sites every day.

Library.nu, a peer-to-peer book sharing service, may have been the best free online library in the world. (In 2012, a coalition of publishers got a German court to shut it down on charges that it violated copyright.) An Aljazeera commentary reported that Library.nu contained between 400,000 and a million free books, mostly "scholarly books: textbooks, secondary treatises, obscure monographs, biographical analyses, technical manuals, collections of cutting-edge research in engineering, mathematics, biology, social science and humanities."

TED served its billionth TED talk video in 2013.

The Kickstarter community gave away its 1 billionth dollar to support new creative projects in 2014. Among the projects supported was the delightful Mini-Museum by Hans Fex. Mr Fex's fundraising goal was $38,000: he raised $1.2 million from over 5,000 backers.

Wikipedia and the Wikimedia projects have received over 2.1 billion edits from users.

The Khan Academy's art history website Smarthistory served over 2 million learners from 200 countries last semester. The Khan Academy itself reaches more than ten million learners a month with its free online classes.

MIT's Open Courseware project served 100 million people in its first decade and their goal is to reach 1 billion learners in the next ten years.

"We used to define the value of an artwork by its frame. It's in a museum. It's validated by critics. It's priced by a label or a store. It is owned. And so on. And not really, any more..." wrote Matt Mason, BitTorrent's Chief Content Officer. "We've been watching our art objects become social objects. 2013 was the year that it stuck. Content has finally caught up with the Internet: value = virality."

What makes these sites tick? For many of them, it's a strategy of openness and generosity-a genuine respect for, interest in, and admiration for the people who participate. It's an ongoing commitment to listening, respect, and empathy that manifests itself in every decision and strategic choice, and which benefits both the participants and the convener.

TED founder Chris Anderson puts it this way.

So, at TED, we've become a little obsessed with this idea of openness. In fact, my colleague, June Cohen, has taken to calling it "radical openness," because it works for us each time. We opened up our talks to the world, and suddenly there are millions of people out there helping spread our speakers' ideas, and thereby making it easier for us to recruit and motivate the next generation of speakers. By opening up our translation program, thousands of heroic volunteers - some of them watching online right now, and thank you! - have translated our talks into more than 70 languages, thereby tripling our viewership in non-English-speaking countries. By giving away our TEDx brand, we suddenly have a thousand-plus live experiments in the art of spreading ideas. And these organizers, they're seeing each other, they're learning from each other. We are learning from them. We're getting great talks back from them. The wheel is turning.

But is a TED talk as good as a museum visit? Is any online experience as good?

There's a lot of doubt among museum leaders that online experiences can be as authentic, as impactful, as a visit to a museum. But try Googling "TED talk made me cry" (my favorite is The full epic of Ed Gavagan) and then read "Art Museums and the Public", a 2001 report by the Smithsonian Institution Office of Policy and Analysis, which concludes,

One of the most striking results of this generation-worth of museum audience studies is that the explicit aims of exhibition planners are rarely achieved to any significant degree. In study after study ... researchers found that the central goals of the exhibition team (which are usually learning goals) were rarely met for more than half of the visitors, except in those cases where most visitors entered the museum already possessing the knowledge that the museum wanted to communicate.

Art historian Beth Harris, dean of art and history at the Khan Academy (along with her collaborator, co-dean Stephen Zucker) and former director of digital learning at MoMA, told me her own feelings about what is all-too-often the reality of museum visits,

It isn't this amazing, contemplative, aesthetic, transcendent experience. It's jostling crowds, it's feeling hungry, it's being annoyed by the people you're with sometimes, it's feeling disappointed that you can't have the reaction that the museum wants you to have-that you don't have the knowledge and the background to get there. I mean, it's a whole range of complicated things.

Museums and museum websites can be disappointing to people used to the more open, participatory, and playful collaborative environments they find elsewhere on the Web, and sometimes they take action. Aurora Raimondi Cominesi, Francesca De Gottardo, and Federica Rossi (now joined byAlessandro D'Amore and Valeria Gasparotti) became so frustrated that so few Italian museums were online that they created their own social media movement, Svegliamuseo -literally "Wake up, museums!"- to inspire and encourage change. Art historian and travel writer Alexandra Korey became so frustrated with the Uffizi Gallery's lack of interpretive materials, both online and in the museum itself, that she created her own mobile app and e-book to fill the void-despite the fact that she had no programming experience. "If there is one museum in the world that is worth spending a lot of time in, it's the Uffizi. Unfortunately, it's not a museum that loves visitors, and that visitors love," wrote Korey. "It's a museum that desperately needs a guide...I took it upon myself to write that guide."

The open, social, and collaborative platforms of the read-write web make projects like Svegliamuseo and Alexandra Korey's Ufizzi app and e-book inevitable, and even the largest, most brilliant institutions can not match the wit and energy of their followers, particularly when those followers are part of a network that connects more than a third of humanity. We should expect to see countless thousands of projects like these as the Internet continues to grow, platforms become more powerful and easier to use, and citizens become more confident in their abilities to challenge, help, and even surpass the accomplishments of what have previously been sacrosanct institutions. Our choice will be whether to ignore or discourage these people, compete with them, or dedicate ourselves to ensuring their lifelong success.

"I realized the community has never been about us, Hank," said John Green in a recent Brotherhood 2.0 video. "It's been about having big conversations around big questions and lifting up people who need it."

Until recently, we thought we had a pretty good idea of what museums were made of. Museums - and with them libraries, archives, and cultural and educational institutions of all kinds-were made of buildings, collections, staff, and visitors: things we could see, touch, smell, hear, or directly measure by counting tickets sold and people through our doors.

But we were wrong. Not just a little wrong. Wrong at a stupendous scale. Like the universe before Vera Rubin discovered dark matter, we were seeing only the small percentage of cultural activity that we expected to see, where we expected to see it. But we know more now-there is more now-and it's tremendously exciting to think about what we can accomplish if we begin to work with true conviction in the areas of the Internet that are less familiar to us and more familiar to our visitors.

Museums, libraries, and archives-heritage, culture, knowledge, and memory institutions-can play a huge role in the story of how Earth's 7 billion citizens will lead their lives, make and participate in their culture, learn, share, invent, create, cry, laugh, and do in the future. It is often forgotten that Tim Berners-Lee designed the World Wide Web to have a remarkable central characteristic: everyone who joined would automatically be granted the right to both consume and produce-to read, and write-on equal footing with everyone else.

"The idea was that anybody who used the web would have a space where they could write, and so the first browser was an editor-it was a writer as well as a reader," said Berners-Lee. "Every person who used the web had the ability to write something." The entire architecture of the World Wide Web is based upon these humanistic, democratic ideals, and we can do a lot of good with them if we make wise choices and concentrate our efforts where they'll matter the most.

"In a very real sense, astronomy begins anew," Vera Rubin wrote after the discovery of dark matter. "The joy and fun of understanding the universe we bequeath to our grandchildren-and to their grandchildren. With over 90% of the matter in the universe still to play with, even the sky will not be the limit."

Notes and links
I've put notes and links here at the bottom to keep the text clean. If it winds up being too weird or difficult for readers I'll put them back up in the body of the story. Links to slides and videos from conference presentations are at the bottom.
Vera Rubin: Bright Galaxies, Dark Matter, by Vera Rubin, American Institute of Physics, New York, 1997. I'll note that the physics of dark matter, and the history of its "discovery" are, of course, more fascinating, challenging, and nuanced than I've depicted here. Astronomer Neil DeGrasse Tyson has said that he doesn't even think we should call dark matter matter, because it's so weird and inexplicable ( link). And it's an oversimplification to say that Vera Rubin "discovered" dark matter-rather, her observations (and the work of others, back to the 1930s) seem to indicate its existence.
Vlogbrothers:
The Louvre: most visited museum, http://www.cnn.com/2013/09/05/travel/top-global-museums/ and YouTube channel https://www.youtube.com/channel/UCafm6w_rPndqAtokQy04Jdw

Urban Institute: "Culture Counts in Communities: A Framework for Measurement." 2002. See pages 16 and 17. http://www.urban.org/uploadedpdf/310834_culture_counts.pdf (Also, my discussions and correspondence with project staff and consultants.)

UX for Good: Overview (and link to design brief) http://www.uxforgood.com/project/world-ia-day-2014/ (Note: this link is broken and there doesn't seem to be replacement content. Homepage is http://www.uxforgood.org/. Thanks! -Mike, Nov 15, 2017)

Reddit: About, http://www.reddit.com/about/ and top Ask Me Anything sessions http://www.reddit.com/r/IAmA/search?q=ama&sort=top&restrict_sr=on

"10,000 times larger than the Library of Congress.": http://blog.1000memories.com/94-number-of-photos-ever-taken-digital-and-analog-in-shoebox

Facebook quarter of a trillion photos: http://mashable.com/2013/09/16/facebook-photo-uploads/
Aperture magazine: http://aperture.org/shop/aperture-214-documentary-expanded-magazine
Pinterest: http://expandedramblings.com/index.php/pinterest-stats/#.U3kpE1hdUx0) and http://highscalability.com/blog/2013/4/15/scaling-pinterest-from-0-to-10s-of-billions-of-page-views-a.html

Tumblr: statistics are reported on the welcome screen at http://tumblr.com

The top four moble social sites: http://ben-evans.com/benedictevans/2013/11/5/mobile-is-eating-the-world-autumn-2013-editio slide 51
TED's billionth view: Wikimedia edits: Kickstarter's $1 billion in donations: https://www.kickstarter.com/blog/thanks-a-billion http://tools.wmflabs.org/wmcounter/index.php http://blog.ted.com/2012/11/13/ted-reaches-its-billionth-video-view/
Library.nu: http://www.aljazeera.com/indepth/opinion/2012/02/2012227143813304790.html
Erik Brynjolfsson / MIT Center for Digital Business: http://www.forbes.com/sites/ericsavitz/2012/11/12/techonomy-mits-brynjolfsson-says-gdp-undervalues-free-goods/
Mini-Museum by Hans Fex: https://www.kickstarter.com/projects/2054592112/mini-museum

TED founder Chris Anderson: http://www.ted.com/talks/chris_anderson_how_web_video_powers_global_innovation

Brewster Kahle: interview with the author, March, 2013

Tim O'Reilly / The clothesline paradox: https://plus.google.com/u/0/+TimOReilly/posts/57xa79PfdTU

Rijksmuseum: https://www.rijksmuseum.nl/?lang=en
Art Museums and the Public: http://www.si.edu/Content/opanda/docs/Rpts2001/01.10.ArtPublic.Final.pdf
DMA Friends: http://www.dallasmuseumofart.org/visit/dma-friends

Digital Public Library of America: http://dp.la/

Europeana: http://europeana.eu

Jane Finnis / Top UK cultural websites: http://www.theguardian.com/culture-professionals-network/culture-professionals-blog/2013/aug/22/your-arts-website-is-rubbish

Internet stats: http://www.internetworldstats.com/stats.htm

Smarthistory: http://smarthistory.khanacadamy.org/about-smarthistory.html

Khan Academy: https://www.khanacademy.org/

Svegliamuseo: http://www.svegliamuseo.com/en

Alexandra Korey: http://alexandrakorey.wordpress.com/ https://itunes.apple.com/us/app/uffizi-art-history-guide/id738366439?mt=8, "If there's one museum in the world...": http://www.arttrav.com/florence/uffizi-app/

John Green/ "I realized the community has never been about us, Hank,": https://www.youtube.com/watch?v=LcyCEwRb5As
Dark Matter (video) opening remarks for J. Boye conference, Aarhus, Denmark.
"In a very real sense, astronomy begins anew": Bright Galaxies, Dark Matter, by Vera Rubin, American Institute of Physics, New York, 1997, p 129

Image Credits

Vera Rubin's rotation curve for galaxy M-31, via "Dark Matter and Galaxy Formation" by Joel R. Primack, http://arxiv.org/pdf/0909.2021.pdf

Related slides/presentations

Dark Matter (presentation on slideshare)

Keynote (video) from the Internet Librarian International and South Eastern Museum Conference and Smith Leadership Symposium

Michael Peter Edson, 2014. CC-BY

# data[].attributes.attachment_link.description
This is the first essay in Code | Words, a collaborative writing project about technology in museums. Until the 1960s we thought we had a pretty good idea of what the universe was made of. You, me...

# data[].attributes.attachment_link.readable_text
When I first began work in the museum world, back in the twentieth century, the doors that marked the boundary between the public galleries and private spaces of my first museum in London had small embossed signs on them that declared, "This door to be kept closed, by order of the Director." In the years since then the boundary between public and private has shifted as technology has made it possible to share the research, objects, exhibition projects and ideas that were hidden behind the closed doors. But despite the potential for new models of museum, it seems to me that for most of our users the museum is still a set of public galleries that occasionally change, and the rest remains a mystery.

Why is it so hard to change how museums work? In the last twenty-five years we've seen the rhetoric of the "new museology", the growth of audience research, and most significantly the disruption of all in its path by digital technologies. They have all moved discussions about visitor experience and user participation into the mainstream, and conferences, symposia and forums have for the last few years regularly mused with a certain amount of angst about the future of museums. The TV, print media, music and publishing industries are being transformed by technological changes. And yet we still see traditional museums whenever most of us look at ourselves.

During the same quarter-century, change management has become firmly established, as in both the public and private sectors leaders looked for processes that could be used to help them adapt to changing environments. But, although there have been some notable published examples, as a sector museums seem to have been slow to change.

Despite the many discussions within the sector in recent years about the need for adaptation, when I became a senior manager and later a director myself, I realized that directors tend to be under-represented at our conferences, and when they are there they don't tend to dwell on the difficult and occasionally messy work of leading museums in these uncertain times. Similarly it seems that within the walls of museums the societal trends towards two-way participation still often become diluted when it comes to conversations about strategic thinking, funding problems, and adapting to change.

Over the last few years I've accepted invitations to speak, doing my bit to fill the gap in participation, while not claiming that I was in any way representative of museum leadership. And, as Ed Rodley was sometimes the person making the invitation, when he asked if I would contribute a piece to CODE|WORDS about leadership, strategic thinking and change I was happy to give it a go. Reading other CODE|WORDS essays, I've been struck that several of them do the hard work of imagining alternative roles and approaches for museums. I hope this essay might be useful to some who are thinking through the practicalities of how to adapt.

My aim is to show how museums absorbed the ideas of change management popularized in the 1990s, but then to suggest that these ideas are now seriously mis-matched with the world in which we find ourselves. In particular change that is focused on achieving a specific endpoint is out-of-step with the digital-dominated trend towards perpetual beta. I think that we won't create museums that are appropriate for the digital age without changing our organizational cultures and how we work. Many have pointed to the need to build cultures that are focused on learning and innovation, and as educational institutions one might expect museums to be leaders in this area, and yet it seems hard to find examples. During my tenure as Director & CEO of the Royal Ontario Museum (ROM) in Toronto we undertook a number of professional development initiatives to build our capacity to innovate and adapt, and I have invited Katy Paul-Chowdhury, who worked with us to develop the process, to write about one of them in an appendix to this essay. My sense is that as the leader in museums I need to be focusing as much on ensuring we have the appropriate organizational culture, as on any other part of our strategy. While I don't have a set of easy answers about how this can be readily achieved (I wish I did) the last part of this essay is devoted to thinking about the sort of thinking and processes that could point the way.
How we have talked about strategy
Strategic thinking for a museum has been typically boiled down and encapsulated in the strategic plan, a document spanning several years that communicates a summary of the main priorities. It is used with funders, and community discussions, and is sometimes a pre-requisite for grants or accreditation, and inside the building is often the starting point for departmental and individual plans for the year ahead.

A strategic plan may well be very practical in specifying projects and measures, but can range across big philosophical areas of contention such as:

The purpose - what is the museum for?

The users - who is the museum for?

The experience and activities - how to meet users needs and ensure the museum is relevant?

The collections - what role do they play?

The funding - how best to fund the museum and ensure it is sustainable in the longer term?

The changing staff, structure and skills - who does what, and how are they organized?

These areas are of course interwoven, and so change in one area should mean change in others too. When it comes to creating a new strategy for a museum, it makes sense that it is going to mean changes in all of these areas, so it is important to deal with the complexity of them all rather than assume they can each be addressed in isolation. At the same time making change in one area won't necessarily bring about change in others. While they often begin with a vision of the future museum, strategic plans often focus on the tangible changes - the new galleries or facilities - with the assumption that changing the visible will also automatically change the way the staff work.

This focus on the tangible occurs because the strategic statements from the Director and written plans are often tailored to those whose support they need to embark on a new direction, the Board, funders, potential donors and community stakeholders. With good reason they are written for an audience beyond the museum's staff and users, describing new plans for what the museum does that are attractive to funders, and linking them to persuasive comments about the museum's overall role in the community. This story must rightly be simple, powerful and externally-focused to galvanize support. But to deliver successfully on a new strategic direction also requires accompanying change inside the museum in the behaviour, priorities and processes used by the staff and volunteers. Doing something new rather than doing the same thing in a new building means also learning how to work differently.

As the external face of the organization and spokesperson for the plans, in the past leaders were seen as akin to military strategists who mapped out a plan for the organization, which they then implemented by directing those around him (or occasionally her). But if that was the case in the distant past, as change management techniques became common in the public and private sectors, the lone strategist narrative has been revealed as often inaccurate or ineffective. Newer approaches to strategy have emphasized more collaboration, as well as more iterative processes that blur the traditional distinctions between strategy formulation and implementation.These more modern tools and concepts have been increasingly used in the museum sector. Typically, creating the family of vision, mission, goals and measures that often make up a strategic plan has been the first step in making change, and so the methods used to create that plan have been drawn from change theory.

In the 1990s influential management theorist John Kotter described a flowchart of the e ight steps to transforming your organization that described a method and language of change. While originating in the private sector, Kotter's method could be used in museums and had the benefit of being understood by that decade's increasingly business-focused boards and funders. The first steps set out the need to create a sense of urgency for change, and to assemble a group with the power to lead the process, followed by creating a vision and strategies that achieve that vision, which is communicated to the staff, and then implemented.

I suspect that for many museum professionals who find themselves in management roles, this sequence feels very familiar and embodies much of what strategic leadership is thought to involve, assembling a team who develops a shared vision, followed by putting it into practice. This language of strategic planning, visioning and change has become common and creates documents that are used as guides for the changes to be made.

While Kotter's flowcart can still be useful as a guide to change , in these uncertain and rapidly-changing times there seems to me something rather quaint about so deterministic a process, where the leaders at the top of the organization have the new ideas, and who alone are charged with coming up with a new vision. It is true that in recent years strategic planning has often become more participatory, with cross-sections of staff involved in visioning workshops, and project teams set up to support implementation, but even so the idea that there is an endpoint to the change, a desired future state that can be achieved after which everyone goes back to their day-jobs seems increasingly old-fashioned in this perpetual beta world. Some commentators on Kotter's approach have pointed out that change is not finite and then institutionalized, but much more continuous, and have encouraged more experimental, collaborative approaches at every stage.

When looking at museums, my personal feeling is that they are so intensely heterogenous, with many different types of expertise and specialism, that new thinking must come from all parts of the organization, and from involving your users as part of the process, not just drawing on the senior management team. So agendas for change that come only from the leader will be missing the most significant potential innovations. But as importantly, continuing to see strategic change in museums as having a defined endpoint can further encourages planning that focuses on visible differences, and underplays the change in the way the museum and its staff operates.
Strategic planning has not led to strategic re-invention
In recent years there have been many examples of new buildings, new galleries and capital campaigns that have come from strategic plans put together using these sort of methods, and yet apart from a small number of notable exceptions it feels as though the results look rather like similar to what was built before the advent of the smartphone and internet. A couple of examples from CODE|WORDS are illuminating in this respect.

In his polemical Dark Matter, the first of the essays, Michael Edson speaks of the digital changes that have reshaped museums, but then muses that museums have been focusing on only a small part of what the internet can offer, and missing the 90% akin to the universe's dark matter. I love this essay but I can't help but feel that even then he is being rather too optimistic about what has been achieved, having just come back from yet another museum conference where much of the conversation was about how we can make our collections accessible online - a full generation after this first became possible. In many cases strategic planning has failed to help museums make a fundamental shift in their practice.

Bridget McKenzie in her CODE|WORDS essay Towards the Sociocratic Museum describes a sociocratic governance model as "non-hierarchical, consent-based and rooted in its communities" with audiences that are, "citizens and collaborators". For the majority of museums that she conceptualizes as plutocratic or bureaucratic models, established by rulers or as public services, a radical re-invention is needed that favours "the unpredictable outcomes of creative practice, and the sustainability of heritage, ecology and diversity." Again I fear such re-invention is unlikely to come about through putting the museum's management team in a room and refusing to allow them out until they have the first draft of the new strategic plan.

It is worth noting that 100 years ago we had galleries, displayed objects, held lectures, much the same as now, despite all the discussion about the need for change in museums, it has been minimal when compared to that in print media, or the music industry.

How then do we change the way we work to both make the museum more porous to new thinking from outside its walls, but also build the skills and confidence to work with our users as collaborators? While setting a strategic direction can still be a good framework for talking to ourselves and others about the future, to re-invent museums we must also create a culture that is biased towards doing new things rather than towards the past. By creating such a culture of innovation we will build the capacity for change, resilience and working with others. Such a culture would also be supported by processes, organization and skills that align and reinforce that bias towards the new. For instance a change process in a museum with such a culture might be more experimental, more iterative and more collaborativ e.

Professional development that can produce this sort of cultural change would need to be intentional, practical and be closely linked to priorities. Last year at the ROM we tried such an approach, working with Katy Paul-Chowdhury in our centennial year to build a professional development program called 100 Day Change that asked teams to deliver initiatives that were part of our current strategic plan, but also encouraged them to build their skills in problem-solving, innovating and leading change. You can read Katy's detailed account of the program as an appendix to this essay.

This program provided practical training on how to lead change to a diverse cross-section of ROM managers, as well as a tool-kit they can use again and again. It also shifted our approach to strategy and strategic-level change in some significant ways.

Firstly, instead of observing a linear formulation to implementation process, we introduced more iteration, experimentation and learning. Stepping inside some major strategic decisions, these projects were used to help change and sharpen the strategies, and inform how they should be carried out. They provided input into gallery strategy, digital strategy, and audience engagement.

Secondly, the training program made our strategic thinking more collaborative and interactive. The projects, and the teams' regular reports to sponsors and the ROM executive, gave voice to the experiences and insights of managers who were not previously engaged in the ROM's strategy development process. Because the teams combined people who did not ordinarily work together, they also allowed different types of strategic insights to be generated. And each project also brought the experiences and feedback from ROM visitors and stakeholders into the mix in a systematic way.

I feel that putting this kind of focus on staff throughout the museum gaining the skills to lead change, and have a bias towards the new, enables museums to move away from a 'too busy to do it differently' approach. But it starts from where museums are now, complete with hierarchies, history and sometimes cumbersome processes, and begins to shift the way that strategy is developed, and the people involved, including our users. The learning can be taken into the projects that follow, and the thinking into the next strategy sessions.
Changing museum culture
The culture in museums does seem to be particularly consistent, so it is worth considering whether there is something about museums that makes them uniquely hard to change? When I asked Katy she thought not. "Change", she said, "is always a challenge." But she thought that change in museums is characterized by "significant resource constraints, together with the limitations and opportunities imposed by a public mandate, multiple stakeholders, many more specialized subject matter experts than professional managers, access to a committed base of volunteers, and an inherent bias that looks back not forward". Phew.

This is my sense too, and in particular that despite the pressing need for radical re-invention we nevertheless all sometimes feel that as we have too much to do, we are too busy to do it differently. Even so I've been surprised at how often museum professionals settle for a reality that is explicitly at odds with their preferred approach, for instance I regularly hear about teams developing exhibitions without any user involvement in prototyping, despite years of research in evaluation, and the many conference presentations that have shown its value.

With their purpose to preserve the past, museums do tend to reverse into the future, with a powerful sense of authority and history that I suspect makes much harder the creation of a culture biased towards innovation. Museums also have entrenched hierarchies, silos of different professional groups, and systems that might have been there since our founding - our current mindsets can create a powerful inertia that is hard to overcome.

So, how do we turn and look towards the future, equipped as best we can be to adapt to changes that are happening quickly and without precedent? By establishing that it is the role of the museum's leadership to build the teams, processes and skills that together create the capacity for change. Museum leaders can become their organization's facilitators-in-chief. They can encourage their staff's capacity for new thinking and problem-solving through doing, by setting up opportunities for leading change across the museum. By bringing together teams and coaching them, the museum's leaders can also see new initiatives implemented quickly.

Making the kind of approach we trialed at the ROM part of the mainstream way of working requires constant support, encouragement and attention, to prevent inertia and the bias towards the past resurrecting itself. Of course this particular method is not the only one, and much is being written at present about agile management and design thinking, both of which focus on iteration and experimentation, but the commitment of the leaders to invest in re-inventing the organizational culture is key. Rather than leading static museums, we should be creating adaptable museums.

Alongside learning through doing, leaders can also encourage and prioritize looking outside the museum walls for new information, trends and inspiration that can be brought into discussions. It seems strange to say in this connected age, but our museums can still be very inward-looking. Social media and the internet enable us all to take charge of our own professional development and be life-long learners, but role-models and encouragement help. Supporting a bias towards the new, together with talking, listening and co-creating with visitors reinforces an emphasis on relevance.

Museum leaders should also explore different approaches for establishing strategies and choosing priorities. Moving towards a strategic framework about what impact the museum is aiming to achieve first, and how it plans to achieve it second will serve to rebalance the conversation between new buildings, online engagement, and new ways of working for the staff. For instance, over the last ten years, logic models and the theory of change processes that originated in international development and social enterprise have begun to be used by museums and offer some alternative options for strategic planning. They place emphasis on impact and outcomes, and often include community engagement as integral steps. The Digital Engagement Framework developed by Jasper Visser and Jim Richardson also emphasises the creation of value, and includes co-creation with users, and organisational transformation.

Choosing how to develop strategy, agreeing measures of impact and seeing strategic thinking as the beginning of organisational learning and transformation is an important part of museum leadership, and I suggest still sits with the Director as they will be held accountable for the museum's progress by the board and funders. But choosing an approach that builds a culture of innovation as you go, and establishes the idea of innovating to deliver agreed results increases the chances of seeing significant change that makes the museum more relevant, and makes more leaders along the way. Setting targets and encouraging experimentation may sounds like opposite approaches, but by keeping the focus on what outcome is to be achieved, and allowing a project team to work out together how they achieve it, the two can be aligned.

Museum leaders also have an important role to tell the story of the museum within the community. A vision or strategic plan created by a museum team with a culture of innovation would be informed as much by emerging ideas or experiments from the team, as by a direction set from the top, but there is still a need to synthesize it into a compelling narrative for stakeholders, and as well to ensure it is relevant and makes sense as a whole. Museums are having to work harder to secure the funds that are needed to deliver their plans, and there are no guarantees that those who have supported in the past will do so in the future, particularly in these times of austerity in many parts of the world. Museum directors spend a good proportion of their time fund-raising and working with the community, and securing resources will continue to be a major part of any leader's role (which is another reason to create more of them).

I see us moving into an era where museums start to look much more like the works-in-progress that they actually always have been when we look at the whole museum, not just the public spaces. Where taking the lead from the online world we make a virtue of museums being in perpetual beta, rather than striving to create timeless galleries of unchanging artworks and artefacts. The door between the public and the private is no longer always kept closed, and involving our visitors can become one of the ways that we bring our museums to life.

But as we let go of the concept of the static museum, we must also let go of the idea that as museum professionals we will have the same jobs in the future as we have in the past. We've all (I hope) experienced the energy that comes from galvanizing around a new project; the passion and excitement of visualizing something new and then creating it for real. The challenge is how as leaders and managers we make that the way we work and feel all the time.

# data[].attributes.attachment_link.title
Changing Museums - CODE | WORDS: Technology and Theory in the Museum - Medium

# data[].attributes.attachment_link.description
When I first began work in the museum world, back in the twentieth century, the doors that marked the boundary between the public galleries and private spaces of my first museum in London had small embossed signs on them that declared, "This door to be kept closed, by order of the Director."

# data[].attributes.headline
MUSEUMS, LIBRARIES, AND 21ST CENTURY SKILLS

# data[].attributes.subject
MUSEUMS, LIBRARIES, AND 21ST CENTURY SKILLS

# data[].attributes.attachment_link.provider_name
Institute of Museum and Library Services

# data[].attributes.attachment_link.title
Museums, Libraries, and 21st Century Skills

# data[].attributes.attachment_link.description
The initiative underscores the critical role our nation's museums and libraries play in helping citizens build 21st century skills.

# data[].attributes.attachment_link.provider_name
Institute of Museum and Library Services

# data[].attributes.attachment_link.readable_text
Jamie Hollier is being honored as a White House Champion of Change for her leadership and commitment to libraries and museums around the United States.
Many Americans are missing out on the benefits of digital literacy; one in five Americans, roughly 62 million people, doesn't use the Internet. Their reasons include not knowing how, lack of affordable access to computers and the Internet, and simply not seeing the value of being online. This digital divide is causing increasing inequality, including decreased access to education, jobs, health information, and social inclusion. Additionally, technology is constantly changing, so digital literacy has to be seen as lifelong learning in order to make sure that people who have traditionally been marginalized are not left behind.

My involvement with digital literacy training and support began in 2008 when I was working for the Delta Public Library District on the Western slope of Colorado. During my time in that position, I became a tutor and mentor for many wonderful people in rural Delta County who were trying to catch up or keep up with the skills they needed to get a job, stay in touch with loved ones, and further their education. In 2010, I lucked my way into a speech by Vice President Biden on innovation, and as he spoke, I kept drawing connections between innovation and lifelong learning, digital literacy, and the role of libraries. Once the Vice President left the stage, I had the amazing opportunity to talk to with him for ten or fifteen minutes about the connections to digital literacy and libraries that I had drawn from his speech.

Just over three years later I have had the opportunity to work on digital literacy projects at the state and national level. Now I find myself in the incredible position of being honored as a White House Champion of Change and being invited to the White House to talk about many of the same topics that I discussed with Vice President Biden. Since that spring afternoon, I have had the amazing privilege of continuing to create lifelong learners and helping libraries support their communities through work at the local, state, and national levels.

From 2010 to 2012, I worked at the Colorado State Library managing a project, funded by the Recovery Act, to establish 88 public computer centers across the state. Those centers provide Internet access and digital literacy training. So far that project has provided computer access to more than three million people and digital literacy education to more than thirty thousand people. Last year that project received the Project of the Year award by the Colorado Association of Libraries. Since late 2012, I have been working with the Public Library Association to create DigitalLearn.org, a project funded by the Institute of Museum and Library Services, serves as an online hub for digital literacy training for library and community organization staff.

We had many amazing stories come out of the work I have done in all these projects from immigrants getting to see distant family for the first time in years through online video to the young parents who are attending online school to get a better job and provide more for their families. One of my favorite stories was of a man who was laid off after 18 years with the same company. He knew he needed computer skills to get a new job but was embarrassed to ask his kids for help. He turned to the library where he got one-on-one digital literacy training, and shortly after he started his computer classes, he found a job.

While digital literacy is key to an engaged and inclusive society, it is also important to create user-focused technology to improve communities, which can have an incredible positive social impact. I am involved in technology development in a diversity of ways. I am a partner at Commerce Kitchen, a company that helps businesses and nonprofits add value through online software and tools. I am a board member for the Digital Public Library of America, a non-profit that is working to provide free and open access to the collections of libraries, museums, and archives across the nation. And I am an organizer for Denver Startup Week.

Technology is an amazing and powerful tool that can help us build stronger communities and provide more opportunities for all Americans. That tool, however, can also divide our communities when digital access and digital literacy are limited. With the great power of technology comes our great responsibility to assure that no one is left behind. This work is not the work of one or even a few; our responsibility to assure that no one is left behind is the work of many. I want to acknowledge that while I am being given this great honor, I am just one representative of the thousands of Americans who are hard at work helping people gain new digital skills across the nation and making sure that all of us will benefit from technology.
Jamie Hollier is a project manager, entrepreneur, consultant and a board member for the Digital Public Library of America.

# data[].attributes.attachment_link.title
Unlocking Opportunities with Digital Literacy

# data[].attributes.attachment_link.description
Jamie Hollier is being honored as a White House Champion of Change for her leadership and commitment to libraries and museums around the United States. Many Americans are missing out on the benefits of digital literacy; one in five Americans, roughly 62 million people, doesn't use the Internet.

# data[].attributes.attachment_link.readable_text
Darren Milligan, Smithsonian Institution, USA, Melissa Wadman, Smithsonian Institution, USA

Abstract
Museums have long identified teachers and their students as primary audiences. Through object loans, lesson plans, and more recently digital resources, our institutions have recognized the potential for the narratives told through exhibitions to reach students outside the museum walls. Many of these traditional models no longer reflect the realities of the contemporary classroom, one being rapidly transformed by budget shortcomings, evolving teaching standards, and the potential of new technologies. Museums are uniquely positioned to play a greater role in formal education. In order to meet this potential, museums can benefit from understanding the needs of these audiences and the capabilities of their institutions to meet them. Since 2010, the Smithsonian, through its Center for Learning and Digital Access, has conducted a series of research projects to better understand how teachers and their students use museum learning resources. This formal paper will share the results of the following five independent, yet progressional, studies and offer usable best practices garnered from a meta-analysis of their results combined with other recent literature in this field: 1. Remedial Evaluation of the Materials Distributed at the Smithsonian Institution's Annual Teachers' Night (2010) 2. Capturing the Voice of Customer, a Satisfaction Insight Review of SmithsonianEducation.org (2011) 3. Digital Learning Resources Project (2012) 4. Increasing the Discoverability of Smithsonian Digital Resources: Learning Resource Metadata Initiative (LRMI) (2014) 5. Piloting Tools to Enable Active and Participatory Learning for Middle School Students: Facilitating Digital Learning with Smithsonian Digital Resources (2014) A meta-summary of these research findings, combined with recent literature, will be shared as a suite of best practices for connecting museum assets with teachers and students.
Keywords: education, learning, digital, metadata, teacher, student

1. Introduction

In the second half of the twentieth century, museums found themselves obligated to respond to new pressures to demonstrate impact. This change, now often referred to as new museology, is a period (arguably one in which we are still) in which museums recognized the need to respond to their audiences' desire for museums to have an impact in their educations and lives (both locally and globally) (Falk & Dierking, 2000). One way museums addressed this demand was to experiment with the use of digital outreach tools, with most, including the Smithsonian, going online in the mid-1990s. What followed in the age of Web 2.0 were museum audiences' new expectations for participation. Silverman (2014) has called this transformation a "third age of museums."

One that is defined not only by their important and impressive collections ... and not just by their roles as educational institutions, museums have entered an age of social service. One that emphasizes the use of their collections ... to foster change in our communities, our neighbors, and our world.

What follows is an examination of how the Smithsonian, and in particular its central education and outreach agency, has taken a research-based approach to understanding and responding to the needs of audiences in one specific environment: the classroom.

Museums have long identified classroom educators and their students as primary audiences. Through object loans, lesson plans, and more recently digital resources, our institutions have recognized the potential for the narratives told through exhibitions to reach students outside the museum walls. Many of these traditional models no longer reflect the realities of the contemporary classroom, one being rapidly transformed by budget shortcomings, evolving teaching standards, and the potential of new technologies. Museums are uniquely positioned to play a greater role in formal education, but to meet this potential, they must understand the needs of these audiences and the capacity of their institutions to address them.

Within the Institution, the Smithsonian Center for Learning and Digital Access (SCLDA) is a center of excellence focused on establishing the Smithsonian as a learning laboratory for everyone. As new technologies make it possible for audiences to connect with Smithsonian researchers, collections, and educational programs like never before, SCLDA's mission is to coordinate among the museums and offices within the Smithsonian, provide models and methods that enable learners to access everything the Smithsonian has to offer, and empower learners to explore their own interests and engage with others ( http://smithsonianeducation.org/tools/about.html).

SCLDA has a long history of both developing and providing unified access to Smithsonian learning resources. Currently, on the central education website for the Institution, Smithsonian Education ( http://smithsonianeducation.org), SCLDA provides a searchable database that catalogs more than two thousand learning resources (lesson plans, research databases, interactives, games, etc.) for the classroom from more than thirty Smithsonian museums, research centers, and programs. This database search feature is used more than one hundred thousand times a year to locate the rich and diverse learning resources that educators from across the Smithsonian have produced.

Since 2010, SCLDA has conducted a series of research projects to better understand how classroom educators and their students access and use museum learning resources. This paper shares the results of these five independent, yet progressional, studies and offers best practices garnered from a meta-analysis of their results.

Remedial Evaluation of the Materials Distributed at the Smithsonian Institution's Annual Teachers' Night (2010)Previously unpublished, this literature review and evaluation produced generalizable guidelines for the design and development of museum-based lesson plans and investigated classroom educator methodologies for incorporating museum-based lessons into classrooms.

Capturing the Voice of Customer, a Satisfaction Insight Review of SmithsonianEducation.org (2011)Collected from more than seven thousand surveys completed by visitors to the central Smithsonian Education website, the makeup of this audience, their motivations for site visitation, their activities while on the website, and their sources of dissatisfaction were explored.

Digital Learning Resources Project (2012)Data analyzed in the first two studies indicated that educators were not using instructional materials as published, but were rebuilding them to fit their curricular and student needs. To better understand this and provide a road map for future digital resource development, this previously unpublished evaluation and iterative prototyping project was conducted.

Piloting Tools to Enable Active and Participatory Learning for Middle School Students: Facilitating Digital Learning with Smithsonian Digital Resources (2014)Throughout these studies, consistently, classroom educators have indicated that relevance to students' needs are a top priority. Recently, we have begun testing directly with students to better understand how they use digital museum assets (specifically digitized collection objects) and to document the types of scaffolds necessary to enable active and participatory learning using them.

Increasing the Discoverability of Smithsonian Digital Resources: Learning Resource Metadata Initiative (2014)Key to having the intended impact with educators and students is the need to ensure that our existing resources are highly discoverable. Findings from a detailed evaluation of the Smithsonian's LRMI project are shared, including a relevance assessment by target users of search-result-page and museum-website-page content.

For the purposes of brevity and utility, the reader will find below both summaries and at times brief excerpts of the methodologies and results from the original, often lengthy, reports. For a fuller picture of this work, it is recommended that the complete reports (links to most can be located in the References section) be examined. It should be noted that many contributors participated in the research design, data collection, analysis, and writing of the original reports (some of which appears in these summaries). Those contributions are recognized in the Acknowledgments section.

2. Remedial Evaluation of the Materials Distributed at the Smithsonian Institution's Annual Teachers' Night

Background

Teachers' Night is an annual event hosted by the Smithsonian Institution and coordinated by the SCLDA. The event provides free teaching resources from the Smithsonian's nineteen museums and nine research centers (as well as additional libraries, archives, and the National Zoo) to thousands of primarily K-12 classroom educators. The event includes booths staffed by museum educators from each Smithsonian agency, presented in a different Smithsonian museum each year. The event last approximately three hours and attracts primarily local (Washington, D.C., Maryland, and Virginia) classroom educators and administrators.

The goals of Teachers' Night are to:

1. Raise awareness about the educational resources offered by the Smithsonian, and

2. Help classroom educators incorporate Smithsonian collections and resources into the classroom.

The purpose of the Teachers' Night remedial evaluation was to determine if the Smithsonian Institution's second goal for the annual event-to help classroom educators incorporate Smithsonian collections and resources into the classroom-was being realized.

Methodology/sample

The evaluation consisted of three phases.

Phase one consisted of an extensive literature review to explore the following questions:

1. What methods do classroom educators use to find and use lesson plans?

2. Do lesson plans from museums meet classroom educators' needs?

Phase two was an analysis of comments by visitors to SmithsonianEducation.org on specific Smithsonian lesson plans (from five Smithsonian agencies: the Anacostia Community Museum, National Museum of Natural History, National Museum of the American Indian, National Science Research Center, and Smithsonian Center for Education and Museum Studies). The analysis reviewed 132 public comments classroom educators posted in 2008. The purpose of the content analysis of these comments on Smithsonian lesson plans was to identify themes in alignment with the literature review.

Finally, phase three consisted of focus groups and in-depth interviews with classroom educators. Two focus groups and four in-depth interviews were conducted to address two questions:

1. What design elements are most important to classroom educators?

2. What extent and in what ways do classroom educators make use of materials provided by the Smithsonian?

Findings

Phase one

The literature review confirmed that the usability and navigability of websites are important. The research suggests that there are common usability problems for visitors (including classroom educators) who are non-museum professionals when using museum websites:

Regarding the suitability of lesson plans for classroom use, the literature review presented several key requirements classroom educators need for museum material to be incorporated into their teaching. Material must be:

Aligned to curriculum standards

Updated

Interdisciplinary

Related to big concepts

Educational

Not dependent on museum visits

Phase two

The content analysis of classroom educators' comments on lesson plans on SmithsonianEducation.org reinforced what was found in the literature review. Important aspects for classroom educators looking to use museum resources are that the material being enjoyable for their students (in other words, fun to use), its user-friendliness, interdisciplinary nature, adaptability, alignment with curriculum standards, and flexibility to accommodate a diversity of students.

Phase three

The focus groups and in-depth interviews demonstrated the importance of the design of Smithsonian materials to how classroom educators use them and whether classroom educators are using Smithsonian materials as intended. Educators came to the focus groups and interviews with a several preformed paradigms, including the needs and personalities of their students, the multilayered administration in which they work, and the brand trust of the Smithsonian. These ideas are highlighted in a passage from the evaluation report:

Students are deemed the biggest reward but also the biggest challenge for classroom educators; the diversity of students and learning gaps (e.g., a child whose first language is not English learning amongst English speaking children) entails difficulties for classroom educators in their lesson plans. Expectations and requirements from the federal and state level as well as from administration within schools also present a challenge to teachers, one that is not conducive to adapting to the diversity of their students.

Educators reported that they used Smithsonian materials acquired at Teachers' Night as resource starting points for their teaching. They expressed that materials need to be visually appealing, useful right out of the box, durable, and multipurpose. Feedback suggested that classroom educators are selective when using Teachers' Night materials. Museum materials do not always fit into their teaching (museum materials must be deconstructed and then reconstructed). Educators expressed the desire for the Smithsonian materials to be more readily usable in the classroom. There was no definite indication that participants from the focus groups used the materials as intended by the Smithsonian.

The outcome of the evaluation of Smithsonian Teachers' Night indicates success in that classroom educators appreciated Smithsonian services and materials from the event. It also suggests more work needs to be done to make Smithsonian materials more usable for classroom educators by making them easier to adapt (to be able to easily pick apart salient information and resources) and by aligning materials more closely to the curriculum and standards that classroom educators have to follow.

3. Capturing the Voice of Customer, a Satisfaction Insight Review of SmithsonianEducation.org

Background

SmithsonianEducation.org was launched in 2003 by the SCLDA (then called the Smithsonian Center for Education and Museum Studies). The core objectives for the SCLDA website were to serve as the gateway to Smithsonian educational resources; tailor content to serve three distinct audiences: educators, families, and students; promote the understanding and use of museums; and emphasize inquiry-based learning using primary sources. During the early 2000s, websites were moving into a new era by acknowledging and addressing the unique needs of different user groups. This approach also supported the shift in museology in which museums moved from being "about something" to being "for somebody" (Weil, 1999).

Throughout the past twelve years, the site has grown to house SCLDA-published materials and to serve as the gateway to the Institution's more than two thousand digital educational resources (from thirty-two museums, libraries, archives, and research centers) indexed and searchable by national and Common Core State Standards. In 2007 and 2008, the site was selected as the People's Voice winner for Best Cultural Institution Website in the Webby Awards.

While the site remained popular throughout its first half decade, little was known about its users. In order to understand how best to properly serve this specific audience (what was assumed to be classroom educators accessing digital museum resources), SCLDA found it crucial to first deeply understand their usage patterns.

Methodology/sample

Beginning in 2009 and running continuously for two years, a user satisfaction survey (using the methodology of the American Customer Satisfaction Index) was conducted via SmithsonianEducation.org. The study was originally intended to highlight areas of improvement to the existing website information architecture and content, but it more importantly provided us with a deep look into who our audience was, their motivations for coming to the site, and their activities while there.

A total of 7,470 surveys were completed during the twenty-four-month span of data collection through a popup window presented to website visitors. The survey consisted of twenty-one numerically scored model questions, as well as ten multiple choice and five open-ended custom questions developed by SCLDA. The survey invitation was presented at random to 30 percent of visitors who had viewed two or more pages of the site. Once accepted, the actual survey was presented as visitors left the site while a session-level cookie prevented re-invitations to complete the survey (during the same session).

Findings

It probably comes to little surprise that a site called "Smithsonian Education" would primary appeal to classroom educators. About half of visitors (48 percent, n=3,578) to the site identified themselves as "teachers," by far the largest audience segment. However, if you add in other formal education audiences (Librarian, Curriculum Developer, School Administrator), we find that this segment increases to approximately 56 percent.

To dive deeper into the dominant visitor segment to the site, we then asked the subject area taught by those self-identifying as a teacher (the survey here allowed multiple selections). The data points to a fairly equal distribution between Language Arts (34 percent, n=915) , Science (34 percent, n=915), and Social Studies (33 percent, n=888). Again, it was not a surprise that more generalized subjects are dominant (Language Arts and Social Studies), as 65 percent of teacher survey respondents taught in grades PreK-8.

Inherent to understanding satisfaction (the original intention of the survey) was a need to identify the motivations that lead to users arriving at the website. Almost 60 percent (congruent with the number of visitors in formal educational roles) came to the site with aspirations of finding educational resources.

Now that we understood the makeup of the visitors and their reasons for arriving at the website, we looked into the actual activities they performed while on the site to correlate intention with real action. Most came to access the content: to read it online (48 percent, n=3,344), to search for it (47 percent, n=3,274), to download it (25 percent, n=1,742), and to share it with others (22 percent, n=1,533).

Diving even deeper still, we asked the question: "What were you primarily looking for today?" We see here a connection between visitor motivation pre-visit and the actions performed on the site. Fifty-nine percent came to the site to "find educational resources," and again the majority end up performing this type of search, in the form of teaching resources, lesson-plan downloads, or content information (clarified in the survey to mean specific information, such as oceanography).

This data, while very useful to the development of an understanding of the site's users (and instrumental in SCLDA's next study, the Digital Learning Resources Project) may have shifted since the original survey closed in 2011. Recently, a similar survey has begun to collect user information again on SmithsonianEducation.org, but has yet to be analyzed.

4. Digital Learning Resources Project (DLRP)

Background

The impetus for the Digital Learning Resources Project was to assist the Smithsonian to better understand the educational uses of Smithsonian digital resources and provide a road map for future digital development. Building on research findings from the two previous studies, the specific research objectives focused on classroom educators' ability to identify, analyze, and extract digital content, with the ultimate goal of enabling all users to achieve their own personal learning objectives through the Smithsonian's resources.

The project had three sets of intended outcomes for the short, medium, and long term:

The project was documented and is archived in total on the project wiki site (http://smithsonian-digital-learning.wikispaces.com).

Methodology/sample

The work (conducted between April and October 2012) was organized across four phases. Phases one and two involved focus groups with twenty classroom educators in Northern California combined with user analytics drawn from Brokers of Expertise (now known as the Digital Chalkboard; see https://www.mydigitalchalkboard.org/), a resource repository containing metadata describing Smithsonian digital learning resources. In addition, during these early phases a literature review and an environmental scan were conducted to further refine goals and research questions. During phase three, initial prototypes were developed and tested by a larger group of sixty-nine classroom educators over a three-week period in the summer of 2012. In Phase four the prototype was finalized based on the findings of the testing and a full set of requirements for the eventual build of the prototype was developed.

Findings

Phases one and two

The results of the research in phases one and two yielded recommendations that fall into three categories: Improved search and visualization; Content that is engaging, standards-aligned, and learner-centered; and Instructional tool suggestions.

Search and Visualization Tools
The literature suggests that museums need to make resources more findable and to generate assets that are personalized and accessible anytime, anywhere, and on multiple platforms. Classroom educators also asked for:

Engaging, Standards-aligned, Learner-centered Content
Previous findings suggest that classroom educators put student interest and engagement at the top of their list and need content that aligns with learning goals and standards. When analyzing resources, classroom educators want content that will:

Engage students

Allow for student interaction and adaptation

Afford accessibility for various learning styles and levels

Offer coherence with the lesson and multidisciplinary opportunities

Support problem-based learning goals

Support standards-based teaching goals

Convey a virtual museum experience

Instructional Tools
Deeper exploration answered new questions about how classroom educators use museum digital content in their classrooms. When extracting resources, classroom educators want:

Despite this consistency, there is some diversity of opinion reflected across these data that should be noted. While the majority of classroom educators in the SmithosnianEducation.org survey and the DLRP prefer lesson-planning ideas over fully packaged lesson plans, a small percentage (22 percent in the survey and 38 percent in the DLRP) prefer fully crafted lesson plans and materials.

Phase three

When taken together, the three weeks of classroom-educator workshops enabled the research team to confirm a set of behaviors across the participant groups. If we use the lens of the project goals of identifying, analyzing, and extracting content to become more creative in classroom instructional use, we can summarize the prototype testing findings as follows.

Search and Visualization (Identifying)
Classroom educators commonly:

Authentication, Saving, and Storing (Analyzing)
Classroom educators prefer:

Instructional Tools (Extracting)
Classroom educators:

Phase four

The final phase of the DLRP was to synthesize the findings into a concrete set of technical requirements geared toward the eventual build and implementation of the prototype tools. A detailed technical development and implementation plan has been drafted and is available on the project wiki (http://smithsonian-digital-learning.wikispaces.com/Teacher+Toolkit+%28Technical+Requirements%29).

One of the additional recommendations of the study pointed towards a need to acknowledge that current technology trends point to users moving away from accessing content in traditional ways, that is via institutional websites. As mobile applications and integrated systems of shared data becoming the norm, developing on a platform that values data portability and a separation of services from traditional Web display will ensure greater flexibility and long-term success for these resources. One of these is the Learning Resource Metadata Initiative discussed in the final study presented in this paper.

5. Piloting Tools to Enable Active and Participatory Learning for Middle School Students: Facilitating Digital Learning with Smithsonian Digital Resources

Background

The DLRP study concluded that "a new examination of the student user as a growing audience for digital museum resources" was needed. In response to this, the DLRP continued to explore the role of digital museum resources in learning. Specifically, SCLDA wanted to focus on middle school students' ability to become active creators of digital resources personalized for learning. Middle school students were chosen in particular because they are the largest student audience for current SCLDA programming.

SCLDA and the University of Maryland College of Education faculty worked with middle school students during the 2013-2014 academic year to explore development and use of the DLRP prototype ( http://scems.navnorth.com/). In essence, the prototype replicates the process educators use in preparing a lesson (i.e., locating resources, reviewing and selecting them, sequencing and editing them, layering them with instructional activities, and sharing them with their students and colleagues). The prototype, in this study, enables students to produce digital projects guided by their own interests. Students can use the prototype, for the first time, to search for and view items in the database of educational resources, the Smithsonian collections, videos, blog posts, podcasts, etc., all via one unified search interface (currently not available from the Smithsonian).

Methodology/sample

The prototype was presented to middle school students in June 2014 in College Park, Maryland, and in October 2014 in Chico, California. Testing occurred in four classrooms with sixth and seventh graders both during and after school with twenty to thirty students in each class. Students worked primarily independently, but occasionally in small groups, on supplied laptops.

Testing was conducted using a predefined set of instructions and tasks. Each testing session included an activity where students were first introduced to the Smithsonian, the research project, and their role as testers. Students were then shown the prototype and walked through how to search, save, and edit collections. Individually, they were then asked to conduct the same search. Finally they were instructed to search, create, and edit a collection based on their own interests. The group was then interviewed for suggestions of improvements.

Data were collected through:

The findings are based on the notes of the data collectors and an analysis of the collections created by the participants.

Findings

According to the observational data, participants did not struggle with search or collection creation; in fact, some of the students were creating collections before instructions were provided. The observers noted that students were working collaboratively-they shared their searches and showed each other what they were finding, and some made similar collections to each other.

Though the students were able to easily use the prototype, there were challenges that participants encountered, including:

Participants wanted to see features that are familiar from other search engines/software, such as:

Auto-correct to assist with spelling

Predictive searches

Recommended searches

Filters for inappropriate material

Ability to adjust fonts

Microphone speech for text searching

Participants wanted the ability to personalize their collection by being able to:

Participants wanted additional content, such as:

As students spent more time with the prototype, they found that they wanted to do more with it than the prototype could do (such as delete items, search using predictive searching, etc.). Students expected basic functionality to be available like that found in familiar software (such as Google and social-media platforms).

6. Increasing the Discoverability of Smithsonian Digital Resources: Learning Resource Metadata Initiative (LRMI)

Background

The Learning Resource Metadata Initiative (LRMI) emerged from a partnership between the Association of Educational Publishers and the Creative Commons (funded by the Bill & Melinda Gates Foundation and the William and Flora Hewlett Foundation). Simply put, the goal of the LRMI is to make online educational resources searchable, more transparent, and easier to locate and analyze. This is accomplished through the development of a standard way of tagging (the process of creating metadata) online content. For a complete background on LRMI and the Learning Registry, see Connecting Learners and Museums through Educational Metadata Initiatives, published by the authors (along with James Collins) in the proceedings of the 2014 Museums and the Web Conference ( http://mw2014.museumsandtheweb.com/paper/connecting-learners-and-museums-through-educational-metadata-initiatives/).

In 2014, SCLDA began an LRMI and Learning Registry (a technology infrastructure for the distribution of metadata) initiative in collaboration with the National Museum of American History. The goals of the Smithsonian LRMI project are to:

Methodology/sample

To assess the effects of the project, SCLDA has established an ongoing LRMI Impact Evaluation Study to understand if LRMI metadata enables educators to better discover, analyze, and use Smithsonian resources to create personalized learning resources for their students. To address these questions, fifty-six classroom educator participants (from across the United States and recruited through an external firm) in this study completed the following activities:

Phase one: Online survey

The survey was designed to understand participants' current attitudes and expectations regarding digital resource discoverability, analysis, and extraction. The survey was based upon one developed by the LRMI Survey Report (Winter Group, 2013).

Phase two: Web-page analysis

To understand participants' tag or attribute preferences in the analysis of digital resources, participants were asked to view and comment upon four different Web environments, each showing metadata describing the same SCLDA-authored digital resource, World War II on the Home Front: Civic Responsibility ( http://www.smithsonianeducation.org/educators/lesson_plans/civic_responsibility/), but each having a slightly different user interface as determined by the inclusion (or absence) of LRMI tags.

Website markup and explanation
The markup and explanation activity collected similar information as the survey, but through qualitative data. Participants were shown screenshots of metadata describing the Smithsonian digital resource as it could be displayed in two search result environments: 1) Google search results, and 2) California Brokers of Expertise website (note that Brokers of Expertise is now known as the Digital Chalkboard; see https://www.mydigitalchalkboard.org/). Participants were asked to identify and place a virtual pin on two aspects of each environment that supported their analysis. Follow-up questions asked the participant to explain their pin placement. Pin placement served as a visual representation of analysis, while participant explanations provided qualitative support.

Website surf and explanation
Participants were asked to click on live links to two environments, each hosting a landing page describing the Smithsonian digital resource, review the information on the page for no more than two minutes, and then answer follow-up questions. As with the website markup activity above, participants were asked to identify two aspects of each landing page that would support their analysis of the digital resource. Participants were asked for deeper explanations about the characteristics of the pages they identified.

Phase three: Web analytics

Further assessment on the quantitative impact of LRMI on discovery, analysis, and extraction will be examined through comparative Web analytics in the next phase of this study in mid-2015.

Findings

Phase one

Responses indicated that only a handful of the participants were familiar with LRMI, having heard about it from "our school librarian" or "at a teachers' conference." Not surprisingly, almost all of the participants projected that educator-specific tags would increase their search satisfaction. When presented with a list of educator-specific tags, a large majority of the participants ranked "grade level" and "content/subject area" as the most important tags, followed by "alignment to standards." The majority also indicated that educator-specific tags would increase the likelihood they would use the materials.

Phase two

Participants analyzed the possible usefulness of a resource by a "trusted" or "familiar" source (e.g., Smithsonian, PBS, and the National WWII Museum) or by content/subject area. Per the Web environment, participants found being able to search by "topics" and "grades" to be particularly useful. Per the sources themselves, participants preferred attributes similar to the Google search above, including possible usefulness of a resource because it is a "trusted" or "familiar" source (e.g., Smithsonian) or by content/subject area.

The vast majority of the participants use Google to search for digital resources and search for online instructional resources several times a week, if not almost every day. Digital resource searches occur both on home computers and on classroom computers. Barely one half of educators consider their searches successful. When asked what makes searches unsuccessful or frustrating, participants cited irrelevant results, lack of educator-specific filters, and time consumption.

The evaluation study has shown that the addition of LRMI tagging to its resources will have a positive impact on the discoverability, analysis, and extraction of Smithsonian digital resources. Specifically, participants strongly assert through both quantitative and qualitative data that grade level, content/subject matter, and connection to standards are among their top priorities in discoverability and analysis (both part of the LRMI specification). These findings support the conclusions of a previous study conducted for the Learning Resource Metadata Initiative (Winter Group, 2013).

7. Conclusions

The studies, when examined together with recent literature, lead us toward a framework for developing best practices in the creation and distribution of digital learning resources. The authors' intent is that others can identify reusable methodologies and findings in these specific studies that may match situations at their institutions, and therefore be of use.

To reach these conclusions, the authors examined each study and consulted both the literature analyzed in the individual literature reviews referenced above, as well as more recent literature, to identify common conclusions. These are organized below and reflect educational preferences for both searching for learning resources and analyzing their usefulness: 1) educator search preferences, 2) factors that contribute to difficult analysis and resistance to use learning resources, 3) educators' preferences for learning resources, and 4) educators' preferences for platform content and functionality. The specific studies mentioned in this paper (as well as relevant literature) are cited following each of the specific conclusions. For brevity, the five studies are indicated using a single number that follows their order as presented in this report and, at times, an indication of which portion of the study lent evidence to the conclusion (e.g., 3-survey):

1. Remedial Evaluation of the Materials Distributed at the Smithsonian Institution's Annual Teachers' Night

2. Capturing the Voice of Customer, a Satisfaction Insight Review of SmithsonianEducation.org

3. Digital Learning Resources Project

4. Piloting Tools to Enable Active and Participatory Learning for Middle School Students: Facilitating Digital Learning with Smithsonian Digital Resources

5. Increasing the Discoverability of Smithsonian Digital Resources: Learning Resource Metadata Initiative

Educator Search Preferences

Enabling effective search for educators reflects both standard website-search best practices and also recognizes the need for and providing for descriptive metadata relevant and understandable to this audience, specifically grade level and subject alignment. This audience does not appears to have any specific loyalty to particular providers of content, nor platforms where learning resources are provided. However, our recent analysis has shown that clear indications of the source of the item coming from a trusted provider (such as the Smithsonian) are useful. In general, educators look for the following in searching for learning resources for the classroom:

1. Searching assist via autocomplete and/or spelling assist
(3-prototyping, 4-prototyping)

2. Search results that allow for both browsing and filtering
(3-interviews, 3-prototyping; Borgman, 2005; Green et al., 2007; Harley, 2007; Lindquist & Long, 2011; Pattuelli, 2011)

3. Scannable grade-level and subject information (to quickly analyze usefulness)
(5-survey, 5-analysis; Green et al., 2007; Lindquist & Long, 2011; Sotto, 2012; Winter Group, 2014)

4. Resources from a wide variety of sources (no loyalty to specific resource providers)
(3-prototyping, 5-analysis; Lindquist & Long, 2011)

Factors that Contribute to Difficult Analysis and Resistance to Use Learning Resources

Educators report that the number of search results encountered, further hampered by both the website user interface design and the information provided with museum-based content, detract from their ability to effectively determine if the resources are appropriate for their pedagogical needs. Specifically, we found these factors to be the most detrimental:

1. Too many search results; too difficult to browse quickly for relevant content
(1-literature review, 3-prototyping, 3-literature review, 5-survey, 5-analysis; Lindquist & Long, 2011; Purcell et al., 2013; Sotto, 2012)

2. Distracting user interfaces
(1-literature review; Eamon, 2006; Harley, 2007; Johnson, 2008; Sotto, 2012)

3. Unfamiliar terminology and/or lack of contextual information
(1-literature review, 4-prototyping; Baker, 2009; Lee & Clarke, 2003; Lindquist & Long, 2011; Sotto, 2012; Swan & Locascio, 2008; Goldenberg & Tally, 2005)

Educators' Preferences for Learning Resources

Educators express some consistent needs when explaining the types of digitally accessible resources they look for from museums. Many express the desire that the focus of the resources be inter- or multidisciplinary. This may stem from preconceived notions of the types of knowledge contained within museums, but the studies did not offer enough information to firmly conclude this. Secondly, a strong preference was noticed for content that connects with both the interests of the students and accepted teaching standards, such as Common Core State Standards. In terms of format, resources should be adaptable, in how they are used either pedagogically (for various learning or reading styles/abilities) or functionally (flexible to various presentation, sharing, and export formats). Specifically educators are looking for resources that meet these criteria:

1. Interdisciplinary and/or multidisciplinary
(1-literature review, 1-reviews, 3-interviews; Baker, 2009)

2. Connection to students' interests
(1-reviews, 3-interviews; Lindquist & Long, 2011; Purcell et al., 2013; Sotto, 2012)

3. Alignment to teaching standards and/or relationship to big concepts
(1-literature review, 1-reviews, 3-interviews, 3-prototyping, 5-survey, 5-analysis; Lindquist & Long, 2011; Sotto 2012)

4. Adaptable
(1-reviews, 1-interviews, 3-literature review, 3-interviews, 3-prototyping, 4-prototyping; Baker, 2009; Green et al., 2007; Leftwich & Bazley, 2009; Marty et al., 2011; Sotto 2012)

5. Downloadable format
(2-survey; Green et al., 2007; Sotto, 2012)

Educators' Preferences for Platform Content and Functionality

When considering the platforms on which educators access educational resources for their classrooms, they seem to prefer ones that consolidate content from a variety of sources (not just the platform provider, for example). They also look for tools to make the content more useful, for both their students (such as discussion areas, annotation tools, etc.) and themselves. They prefer a variety of sharing options (although the SCLDA research indicates that email continues to be the preferred method of sharing resource with other colleagues) and ways to save and sort content for later use. Specifically, the research and literature indicates these preferences:

1. Content available from more than one producer/supplier
(3-interviews, 3-prototyping, 4-prototyping; Baker, 2009; Green et al., 2007; Marty et al., 2011)

2. Tools available within the platform for student interaction with the resources
(3-interviews, 3-prototyping, 4-prototyping; Baker, 2009; Sotto, 2012)

3. Variety of sharing options
(3-prototyping, 4-prototyping; Marty et al., 2011)

4. Ability to save and structure resources within the platform for later review/use
(3-prototyping; Baker, 2009; Lindquist & Long, 2011)

Acknowledgements

The authors wish to thank Stephanie Norby, Michelle Smith, and Pino Monaco of the Smithsonian Center for Learning and Digital Access for their support and assistance in producing this paper and in the development of the projects it summarizes. The authors also express their gratitude to Claudine Brown, the Smithsonian assistant secretary for Education and Access, as well as Karen Garrett, the Youth Access Grant program, the Pearson Foundation, Brokers of Expertise of the California Department of Education, the Council of Chief State School Officers, and inBloom for financial and in-kind support.

This paper summarizes, and at times excerpts from, five original research reports conducted or commissioned by the Smithsonian Center for Learning and Digital Access from 2010 to 2014. The following individuals, in addition to the authors, made significant contributions to the research design, data collection, analysis, and writing of the original reports and should be acknowledged and credited:

Remedial Evaluation of the Materials Distributed at the Smithsonian Institution's Annual Teachers' Night
For the Smithsonian Center for Learning and Digital Access: Jennie Ito, Lesley Langa, and Pino Monaco.

Capturing the Voice of Customer, a Satisfaction Insight Review of SmithsonianEducation.org
For Foresee Results: Sara Dorn-Havlik.

Digital Learning Resources Project
For the Smithsonian Center for Learning and Digital Access: Pino Monaco and Michelle Smith; for Navigation North Learning Solutions, LLC: Brian Ausland, Joe Hobson, Daniel Kreiger, and MaryRose Lovgren; for Cross & Joftus, LLC: Christopher Cross, John Ittelson, Dilan Maherdran, Virginia McMunn, Griffith Montgomery, Clark Quinn, Jillian Ryan, and Virginia Adams Simon.

Increasing the Discoverability of Smithsonian Digital Resources: Learning Resource Metadata Initiative (LRMI)
For the Smithsonian Center for Learning and Digital Access: James Collins; for Oberg Research, LLC: Caren Oberg and Heidi Kartchner.

Piloting Tools to Enable Active and Participatory Learning for Middle School Students: Facilitating Digital Learning with Smithsonian Digital Resources
For the Smithsonian Center for Learning and Digital Access: Pino Monaco; for Oberg Research, LLC: Caren Oberg and Heidi Kartchner; for Navigation North Learning Solutions, LLC: Brian Ausland, Jodi Ausland, and MaryRose Lovgren.

References

Baker, R. (2009). "Museums & Education Digital Content Exchange Final Report." The Learning Federation.

Borgman, C.L., L.J. Smart, K.A. Millwood, J.R. Finley, L. Champeny, A.J. Gilliland, & G.H. Leazer. (2005). "Comparing faculty information seeking in teaching and research: Implications for the design of digital libraries." Journal of the American Society for Information Science and Technology 56(6): 636-657.

Eamon, M. (2006). "A 'Genuine Relationship with the Actual': New Perspectives on Primary Sources, History and the Internet in the Classroom." The History Teacher 39(3): 297-314.

Falk, J.H., & L. Dierking. (2000). Learning from Museums: Visitor Experiences and the Making of Meaning. Walnut Creek: Altamira, 229.

Goldenberg, L.B., & B. Tally. (2005). "Fostering historical thinking with digitized primary sources." Journal of Research on Technology in Education 38(1+).

Green, D., R. Lancefield, D. Harley, S. Hsi, G. Waibel, S. Chun, & L. White. (2007). "Museum Images On-line: Meeting the Needs of Educators." In J. Trant & D. Bearman (eds.). Museums and the Web 2007: Proceedings, Toronto: Archives & Museum Informatics. Published March 1, 2007.

Harley, D. (2007). "Why study users? An environmental scan of use and users of digital resources in humanities and social sciences undergraduate education." First Monday 12(1).

Johnson, A. (2008). "Users, use and context: supporting interaction between users and digital archives." In Craven, L. (ed.). What are archives?: cultural and theoretical perspectives: a reader. Burlington, VT; Aldershot, England: Ashgate.

Lee, J., & G. Clarke. (2003). "High school social studies students' uses of online historical documents related to the Cuban Missile Crisis." Society for Information Technology & Teacher Education International Conference 2003. Albuquerque, New Mexico: Association for the Advancement of Computing in Education (AACE).

Leftwich, M., & M. Bazley. (2009). "Pedagogy and Design: Understanding Teacher Use of On-line Museum Resources." In J. Trant & D. Bearman (eds.). Museums and the Web 2009: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2009.

Lindquist, T., & H. Long. (2011). "How can educational technology facilitate student engagement with online primary sources?" Library Hi Tech 29(2): 224-241.

Marty, P., S. Sayre, & S. Filippini Fantoni. (2011). "Personal Digital Collections: Involving Users in the Co-Creation of Digital Cultural Heritage." In S. Georgios, K. Dimitrios, & L. Fotis (eds.). Handbook of Research on Technologies and Cultural Heritage: Applications and Environments. Hershey, PA: IGI Global, 285-304.

Milligan, D., M. Wadman, & J. Collins. (2014). "Connecting Learners and Museums through Educational Metadata Initiatives." In N. Proctor & R. Cherry (eds.). Museums and the Web 2014, Silver Spring, MD: Museums and the Web. Published January 31, 2014.

Pattuelli, M.C. (2011). "Modeling a domain ontology for cultural heritage resources: A user-centered approach." J ournal of the American Society for Information Science and Technology 62(2): 314-342.

Purcell, K., A. Heaps, J. Buchanan, & L. Friedrich. (2013). "How Teachers Are Using Technology at Home and in Their Classrooms." Pew Research Center's Internet & American Life Project.

Silverman, L. (2011). Social Work of Museums. Online video. Bunker Ljubljana. Consulted October 14, 2014. Available http://vimeo.com/19213313

Sotto, T. (2012). Summary Report of Survey Findings, Teacher Programs: Assessing the Getty Museum's Online Resources for K-12 Teachers. J. Paul Getty Trust.

Swan, K., & D. Locascio. (2008). "Evaluating alignment of technology and primary source use within a history classroom." Contemporary Issues in Technology and Teacher Education 8(2): 175-186.

Weil, S. (1999). "From Being about Something to Being for Somebody: The Ongoing Transformation of the American Museum." Daedalus 128(3), America's Museums (Summer, 1999), 229-258.

Winter Group. (2013). LRMI Survey report August 2013 update, Ease and discoverability: Educators and publishers on the search for educational content. Association of Educational Publishers.

Winter Group. (2014). LRMI Survey Report July 2014 Update: Momentum. Opportunity. Outreach. Association of Educational Publishers.

For the complete studies summarized in this paper, including background, methodologies, and full references, see the following:
Remedial Evaluation of the Materials Distributed at the Smithsonian Institution's Annual Teachers' Night
http://smithsonian-digital-learning.wikispaces.com/file/view/Remedial%20Evaluation%20November%202010.doc/323385242/Remedial%20Evaluation%20November%202010.doc
Capturing the Voice of Customer, a Satisfaction Insight Review of SmithsonianEducation.org
For more information on the study, please contact the authors and see http://learninglab.si.edu/news/2014/10/who-is-our-audience-smithsonianeducation-org/

Digital Learning Resources Project
Literature Review: http://smithsonian-digital-learning.wikispaces.com/Teacher+Toolkit+%28Literature+Review%29

Environmental Scan: http://smithsonian-digital-learning.wikispaces.com/Teacher+Toolkit+%28Environmental+Scan%29

Teacher Prototyping Report: http://smithsonian-digital-learning.wikispaces.com/Teacher+Toolkit+%28Research+Findings%29

Technical Requirements Report: http://smithsonian-digital-learning.wikispaces.com/Teacher+Toolkit+%28Technical+Requirements%29
Piloting Tools to Enable Active and Participatory Learning for Middle School Students: Facilitating Digital Learning with Smithsonian Digital Resources
http://smithsonian-digital-learning.wikispaces.com/file/view/DLRP_PilotingTools_FinalReport.pdf/536630252/DLRP_PilotingTools_FinalReport.pdf
Final Report and Conclusions: http://smithsonian-digital-learning.wikispaces.com/Teacher+Toolkit+%28Final+Report%29

Increasing the Discoverability of Smithsonian Digital Resources: Learning Resource Metadata Initiative (LRMI)
For more information on the ongoing evaluation study, please contact the authors.
Cite as:
. "From physical to digital: Recent research into the discovery, analysis, and use of museums resources by classroom educators and students."MW2015: Museums and the Web 2015. Published January 15, 2015. Consulted .
https://mw2015.museumsandtheweb.com/paper/from-physical-to-digital-recent-research-into-the-discovery-analysis-and-use-of-museums-resources-by-classroom-educators-and-students/

# data[].attributes.attachment_link.title
From physical to digital: Recent research into the discovery, analysis, and use of museums resources by classroom educators and students

# data[].attributes.attachment_link.description
Darren Milligan, Smithsonian Institution, USA, Melissa Wadman, Smithsonian Institution, USA Museums have long identified teachers and their students as primary audiences. Through object loans, lesson plans, and more recently digital resources, our institutions have recognized the potential for the narratives told through exhibitions to reach students outside the museum walls.

# data[].attributes.attachment_link.readable_text
The Digital Learning Network is an informal network of people who share a common interest in how digital technologies can help people who work in the culture and heritage sectors to better engage with audiences.

This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts

# data[].attributes.attachment_link.description
The Digital Learning Network is an informal network of people who share a common interest in how digital technologies can help people who work in the culture and heritage sectors to better engage with audiences. Click to read Digital Learning Network, a Substack publication.

# data[].attributes.headline
Lessons From Museums and Libraries: Five Ways to Address Families’ Digital Learning Needs

# data[].attributes.subject
Lessons From Museums and Libraries: Five Ways to Address Families’ Digital Learning Needs

# data[].attributes.attachment_link.title
Welcome to the Global Family Research Project | Global Family Research Project

# data[].attributes.attachment_link.description
The Global Family Research Project is an independent, entrepreneurial nonprofit organization that supports all families and communities in helping children find success in and out of school. We create a worldwide exchange of ideas to further the understanding and implementation of anywhere, anytime learning for all.

# data[].attributes.headline
ADVANCING 21ST CENTURY LEARNING: THE ROLE OF MUSEUMS AND LIBRARIES.

# data[].attributes.subject
ADVANCING 21ST CENTURY LEARNING: THE ROLE OF MUSEUMS AND LIBRARIES.

# data[].attributes.headline
How can digital services enhance
museum learning and partnerships with
schools?

# data[].attributes.subject
How can digital services enhance
museum learning and partnerships with
schools?

# data[].attributes.headline
The Dos and Don’ts of Teaching Digital Literacy

# data[].attributes.subject
The Dos and Don’ts of Teaching Digital Literacy

# data[].attributes.body
The fact is, social media isn’t technology in the lives of our kids, but an essential aspect of their world. Social media isn’t ‘new’ anymore,” said teacher and education consultant Dan Haesler on his blog. “We can’t continue pretending that it is, and using this as an excuse for not addressing it. And by addressing it I do not mean banning it!”

Haesler, a proponent of proactive social media education, posed an interesting question on his blog regarding social media and digital literacy education: what if we approached driver’s education in the same way? Taking this hypothesis one step further, Haesler came up with three conclusions:

Driving lessons would be taught by adults with little or no experience of driving.

Driving lessons would only focus on what not to do.

Driving lessons would NEVER take place in an actual car. In fact, most cars would not be allowed on school property.

# data[].attributes.attachment_link.readable_text
Is there a right and a wrong way to teach social media in schools? Some administrators think so.

Many schools across the country have banned the use of social networking platforms, but others are transitioning to a more proactive approach. Many experts and school officials say that abandoning the more reactionary responses to social media is the first step toward implanting a more effective digital literacy education.

"The fact is, social media isn't technology in the lives of our kids, but an essential aspect of their world. Social media isn't 'new' anymore," said teacher and education consultant Dan Haesler on his blog. "We can't continue pretending that it is, and using this as an excuse for not addressing it. And by addressing it I do not mean banning it!"

Haesler, a proponent of proactive social media education, posed an interesting question on his blog regarding social media and digital literacy education: what if we approached driver's education in the same way? Taking this hypothesis one step further, Haesler came up with three conclusions:

Driving lessons would be taught by adults with little or no experience of driving.

Driving lessons would only focus on what not to do.

Driving lessons would NEVER take place in an actual car. In fact, most cars would not be allowed on school property.

Can you imagine learning to drive under such conditions? I can't either. And Haesler's comparison isn't that farfetched. Both subjects involve making responsible choices and can cause serious consequences for the people making these choices. They both also require users to navigate areas of ambiguity and to make difficult decisions that become easier with time and practice.

Canadian K-12 principal George Couros used Haesler's method of thinking to design a rubric for schools to see how well their digital literacy programs hold up. In order to receive the highest grade, Couros said schools must open up social media to students while teaching the tools to navigate these sites, rather than instituting a ban or providing the curriculum without opportunity for actual practice.

Couros maintained that if a student said something they might regret on Twitter or Facebook, it does not make them a bad student. In fact, he said that many of us have likely made a similar mistake when in the company of friends.

To many kids now, they think that being on Twitter is, in some ways, being with their closest friends.

"When I was a kid and was with my closest friends, I might have said similar things. To many kids now, they think that being on Twitter is, in some ways, being with their closest friends," said Couros. "We have to talk with our kids and be honest with them that we are not perfect as individuals either, but we have to understand what is meant to be public and what is private."

Actions like those of Alexandra Wallace and others whose insensitive remarks have gone viral signal a lack of access to teachers and digital literacy education.

The relationship between private and public, as Couros said, is an important one to understand, yet can often be confusing for students to navigate. Our curriculum, which has sections dedicated to both privacy and self-image, contains tips and resources for educators trying to take a proactive approach to teaching digital citizenship.

A full overview[pdf] of the curriculum is available online, as well as a breakdown of each lesson by grade and what can be covered both in and out of the classroom.

# data[].attributes.attachment_link.title
The Dos and Don'ts of Teaching Digital Literacy

# data[].attributes.attachment_link.description
Is there a right and a wrong way to teach social media in schools? Some administrators think so. Many schools across the country have banned the use of social networking platforms, but others are transitioning to a more proactive approach.

# data[].attributes.headline
Digital literacy 'as important as reading and writing'

# data[].attributes.subject
Digital literacy 'as important as reading and writing'

# data[].attributes.body
“Becoming literate in how the technical world works is equivalent to reading, writing and maths. We need to look at this fourth literacy as mainstream,” he says.

# data[].attributes.attachment_link.readable_text
"Becoming literate in how the technical world works is equivalent to reading, writing and maths. We need to look at this fourth literacy as mainstream," he says.

"Not just at the level of the very basics of operating a computer but actually understanding how the code and mechanics behind it work. In the same way that if all you had was oral communication and you didn't have writing, you really wouldn't understand the logic of our society."

After-school, volunteer led projects such as Code Club are taking steps to introduce coding into schools, and by the end of 2015 the club hopes to be in 25 per cent of primary schools across the UK.

Working up from the first programme 'Scratch', learners progress through four terms of work until they begin learning programming language, Python - starting out with basic concepts to create animations and games.

However, whilst progression in the standard of digital literacy in schools is slowly improving, Jodie Lopez, Champion Schools Coordinator at Pearson thinks that more needs to be done to embed these advances across the whole curriculum.

"I think there is still a big gap to be narrowed between the schools that are and the schools that aren't incorporating digital literacy into the whole curriculum," she says.

"I was made ICT coordinator in 2009 at the school I previously taught at and it was very much my vision that ICT shouldn't be seen as a stand alone subject."

"I set up gettingintoliteracy.com because I wanted to focus on using ICT in literacy, but ideally these skills would be implemented across other subjects as well.

"In art, for example, the curriculum states that children should be able to use paint, pencil and clay, but nothing about digital, and yet, most art jobs that you would go into now, would involve a huge element of digital. That is where more changes need to happen."

Embedding digital literacy lessons and, furthermore, social media across the curriculum certainly opens up a variety of educational tools for teachers.

In a recent survey conducted by Pearson, figures suggest that around one in six 15 year olds use social media for careers advice, while only 1 per cent of 15 year olds prioritise information from careers books.

Open access social media in schools can of course have its downsides - but this is yet another area where an emphasis on digital literacy can help young people.

"If you want your kids to be smart and safe in the digital world - whether that is in school or out of school - having them understand how the technology works, how copyright works, how identity works, is way more important than whether they have access to information or not" says Mark.

"The focus of a lot of schools seems to be blocking them off, which is always going to be somewhere between ineffective and useless," he continues. "Kids live in a digital world, so if we can teach them about what's going on and how to protect themselves and think critically, then we will be in a much better situation."

"We need to remember that it's early days for the education market and social media," says Tom Hall, VP of Education Technology Partnerships at Pearson.

"Openness and trust of students is important, but we have to be careful. One theme which came out of Pearson's Social Media Week discussion was the need to 'educate' kids on their social behaviour - children don't always think about the audience they are projecting to."

Training teachers in the concepts of digital literacy will be one of the biggest challenges ahead, Mark tells me. There are many teachers who want to incorporate digital technology in lessons but who need to upgrade their skills to do so. However, on the other side of the coin, there are skilled teachers who are looking for an opportunity to use digital technology creatively.

"I would hope that in five years time, we are no longer talking about computer science versus ICT," says Mark. "But that we are actually talking about the fact that understanding how the digital world works, is as important as reading, writing and maths for every student.

"If you don't really understand how the digital world functions you're really living in a world where you don't have the creative and innovation skills that are going to be needed in the future economy.

"The ability to embed learning digital skills inside what we might think of as traditional skills, across the whole curriculum, is certainly something to think about."

# data[].attributes.attachment_link.title
Digital literacy 'as important as reading and writing'

# data[].attributes.attachment_link.description
"Becoming literate in how the technical world works is equivalent to reading, writing and maths. We need to look at this fourth literacy as mainstream," he says. "Not just at the level of the very basics of operating a computer but actually understanding how the code and mechanics behind it work.

# data[].attributes.headline
No Longer a Luxury, Digital Literacy Can't Wait

# data[].attributes.subject
No Longer a Luxury, Digital Literacy Can't Wait

# data[].attributes.headline
Digital Citizenry as the self-monitored habits that sustain and improve the digital communities you enjoy or depend on.”

# data[].attributes.subject
Digital Citizenry as the self-monitored habits that sustain and improve the digital communities you enjoy or depend on.”

# data[].attributes.attachment_link.readable_text
by Terry Heick

As more and more students interact digitally-with content, one another, and various communities-the concept of digital citizenship becomes increasingly important.

Which begs the question: what is digital citizenship?

Well, first citizenship, which is formally defined as "the quality of an individual's response to membership in a community." This makes citizenship far more complex than a simple legal matter, but rather one that consists of self-knowledge, interaction, and intimate knowledge of a place, its people, and its cultural history.

So digital citizenship is nearly the same thing-"the quality of a response to membership in a digital community" would be a good first crack at the definition.

Revising that might more clearly articulate the differences between physical and digital communities, so a decent definition of digital citizenship then might be "Self-monitored participation that reflects conscious interdependence with all (visible and less visible) community members"

But that leaves out the idea of content itself, which leads us to a pretty good definition for educators: "The quality of habits, actions, and consumption patterns that impact the ecology of digital content and communities."

Still too wordy? Maybe a shorter version for students-with some moral imperatives and implied advice-could be: "the self-monitored habits that sustain and improve the digital communities you enjoy or depend on."

Later this week, we're going to have a more in-depth look at the characteristics of digital citizenship, but the infographic below takes a more student-friendly approach by defining digital citizenship in terms of its actions and habits: using, sifting, mastering, creating-the literal actions that ultimately define the tone of a student's interactions with their digital environments.

This makes it useful not just as a visual for teacher understanding, but for students to discuss, internalize, and apply themselves. In fact, hanging it in the classroom, computer labs, media centers, and other highly-visible places might make sense as well: the rules of the world of digital networks and social media.

# data[].attributes.attachment_link.description
by Terry Heick As more and more students interact digitally-with content, one another, and various communities-the concept of digital citizenship becomes increasingly important. Which begs the question: what is digital citizenship? Well, first citizenship, which is formally defined as "the quality of an individual's response to membership in a community."

# data[].attributes.headline
“Digital literacy is the ability to interpret and design nuanced communication across fluid digital forms.”

# data[].attributes.subject
“Digital literacy is the ability to interpret and design nuanced communication across fluid digital forms.”

# data[].attributes.attachment_link.readable_text
by Terry Heick

When we think of digital literacy, we usually think of research-finding, evaluating, and properly crediting digital sources. The "research" connotation makes sense, as it is the sheer volume of sources and media forms on the "internet" that stand out.

But we are living in a world where the internet is disappearing, replaced by sheer connectivity. Are you "on the internet" when you tweet? Skim through a social reader like Flipboard? Send a text? Mark up a pdf and sync it with the cloud so you can access it later? Are the cloud and the "internet" the same thing?

As the internet dissolves into something more seamless-that no longer requires a clunky web browser to make itself visible-we might adjust our perspectives in parallel.

Take the idea of "literacy," for example. Literacy can be reduced to the ability to make sense of ideas. This often means reading, but also viewing, observing, writing, creating, designing-each a kind of literacy, and each with nuanced fluencies of their own.

Technology improves literacy only insofar as it improves a learner's ability to identify, analyze, evaluate and create media. In fact, it remains entirely possible to fill learning spaces with apps, mobility, notifications, charts, fluid social streams, visualized data, and all-out holograms of Greek philosophers teaching them directly, and only improve their familiarity with these forms and their spectacle rather than the ideas and people behind them.

Literacy implies a fuller understanding and a rounder knowledge. A literate person is aware of multiple information sources, the pros and cons of media forms, and the value and credibility of information. A literate person can process diverse data sources, and suggest macro relevance and micro application of seemingly disparate ideas.
The Definition Of Digital Literacy
Cornell University offers a definition that works, but seems a bit limited, and dated as well: " Digital literacy is the ability to find, evaluate, utilize, share, and create content using information technologies and the Internet."

This isn't wrong so much as it focuses too much on technology and "the internet." Literacy can't be about the forms unless we're talking about form literacy. Digital tools exist for access-finding information. Then finding better information. Socializing thinking. Connecting and contributing to digital communities you care about.

It is also a matter of "literacy" to understand concepts like digital footprints and identity. This reflects the overlap between digital literacy and digital citizenship, much in the same way there is overlap between traditional literacy and citizenship.

To settle on a definition then, here's one that reflects the depth and breadth of the concept without getting overly wordy or complex:

" Digital literacy is the ability to interpret and design nuanced communication across fluid digital forms."
The Definition Of Digital Literacy; adapted image attribution flickr user flickeringbrad

# data[].attributes.attachment_link.description
by Terry Heick When we think of digital literacy, we usually think of research-finding, evaluating, and properly crediting digital sources. The "research" connotation makes sense, as it is the sheer volume of sources and media forms on the "internet" that stand out. But we are living in a world where the internet is disappearing, replaced by sheer connectivity.