# data[].attributes.headline
This is an additional quiz you need to do if you did not take the previous quiz. The quiz will be open only for today 22 March 2024 until 11 PM. Here is the link for the quiz ->  https://bit.ly/wiraddquiz

# data[].attributes.subject
This is an additional quiz you need to do if you did not take the previous quiz. The quiz will be open only for today 22 March 2024 until 11 PM. Here is the link for the quiz ->  https://bit.ly/wiraddquiz

# data[].attributes.body
There will be no more extended deadline for the quiz.

# data[].attributes.headline
Here is the groups order that needs to present Heuristic evaluation Next Tuesday, 19 March 2024.

# data[].attributes.subject
Here is the groups order that needs to present Heuristic evaluation Next Tuesday, 19 March 2024.

# data[].attributes.headline
Here is an example of a User Test plan/report for doing the user testing number 2. You can see this as a guide to do your final individual task. Thank you.

# data[].attributes.subject
Here is an example of a User Test plan/report for doing the user testing number 2. You can see this as a guide to do your final individual task. Thank you.

# data[].attributes.headline
Read this instruction here on this file. THIS is the final project instruction for individual submission.

# data[].attributes.subject
Read this instruction here on this file. THIS is the final project instruction for individual submission.

# data[].attributes.headline
Here is just an example of what final presentation (including heuristic evaluation & final prototype) looks like.

# data[].attributes.subject
Here is just an example of what final presentation (including heuristic evaluation & final prototype) looks like.

# data[].attributes.headline
FINAL SUBMISSION INSTRUCTION. READ HERE.

# data[].attributes.subject
FINAL SUBMISSION INSTRUCTION. READ HERE.

# data[].attributes.attachment_link.title
LAB 3 - PHASE 3 - WIR - INSTRUCTION- 1725

# data[].attributes.headline
Here is user test preparation that You MUST do TODAY before you leave the class. 1)  follow the link 2)  pay attention to do individual test preparation 3) make sure you write your individual name on the Figjam board

# data[].attributes.subject
Here is user test preparation that You MUST do TODAY before you leave the class. 1)  follow the link 2)  pay attention to do individual test preparation 3) make sure you write your individual name on the Figjam board

# data[].attributes.headline
Preparation 5: do the test and record it

# data[].attributes.subject
Preparation 5: do the test and record it

# data[].attributes.headline
Preparation 4 : gather your users and be ready to test

# data[].attributes.subject
Preparation 4 : gather your users and be ready to test

# data[].attributes.headline
Preparation 3: set the tasks and questions based on your goals that you set on step 1.

# data[].attributes.subject
Preparation 3: set the tasks and questions based on your goals that you set on step 1.

# data[].attributes.headline
Preparation 2: set the scope, and the time limit

# data[].attributes.subject
Preparation 2: set the scope, and the time limit

# data[].attributes.headline
Preparation 1: set your goals, relate it to screens/ features that you want to test

# data[].attributes.subject
Preparation 1: set your goals, relate it to screens/ features that you want to test

# data[].attributes.headline
The reason to do user test with 5 users:  

# data[].attributes.subject
The reason to do user test with 5 users:  

# data[].attributes.attachment_link.readable_text
Some people think that usability is very costly and complex and that user tests should be reserved for the rare web design project with a huge budget and a lavish time schedule. Not true. Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.

In earlier research, Tom Landauer and I showed that the number of usability problems found in a usability test with n users is:

where N is the total number of usability problems in the design and L is the proportion of usability problems discovered while testing a single user. The typical value of L is 31%, averaged across a large number of projects we studied. Plotting the curve for L =31% gives the following result:

The most striking truth of the curve is that zero users give zero insights.

As soon as you collect data from a single test user, your insights shoot up and you have already learned almost a third of all there is to know about the usability of the design. The difference between zero and even a little bit of data is astounding.

When you test the second user, you will discover that this person does some of the same things as the first user, so there is some overlap in what you learn. People are definitely different, so there will also be something new that the second user does that you did not observe with the first user. So the second user adds some amount of new insight, but not nearly as much as the first user did.

The third user will do many things that you already observed with the first user or with the second user and even some things that you have already seen twice. Plus, of course, the third user will generate a small amount of new data, even if not as much as the first and the second user did.

As you add more and more users, you learn less and less because you will keep seeing the same things again and again. There is no real need to keep observing the same thing multiple times, and you will be very motivated to go back to the drawing board and redesign the site to eliminate the usability problems.

After the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new.

Iterative Design

The curve clearly shows that you need to test with at least 15 users to discover all the usability problems in the design. So why do I recommend testing with a much smaller number of users?

The main reason is that it is better to distribute your budget for user testing across many small tests instead of blowing everything on a single, elaborate study. Let us say that you do have the funding to recruit 15 representative customers and have them test your design. Great. Spend this budget on 3 studies with 5 users each!

You want to run multiple tests because the real goal of usability engineering is to improve the design and not just to document its weaknesses. After the first study with five participants has found 85% of the usability problems, you will want to fix these problems in a redesign.

After creating the new design, you need to test again. Even though I said that the redesign should "fix" the problems found in the first study, the truth is that you think that the new design overcomes the problems. But since nobody can design the perfect user interface, there is no guarantee that the new design does in fact fix the problems. A second test will discover whether the fixes worked or whether they didn't. Also, in introducing a new design, there is always the risk of introducing a new usability problem, even if the old one did get fixed.

Also, the second study with 5 users will discover most of the remaining 15% of the original usability problems that were not found in the first round of testing. (There will still be 2% of the original problems left - they will have to wait until the third study to be identified.)

Finally, the second study will be able to probe deeper into the usability of the fundamental structure of the site, assessing issues like information architecture, task flow, and match with user needs. These important issues are often obscured in initial studies where the users are stumped by stupid surface-level usability problems that prevent them from really digging into the site.

So the second study will both serve as quality assurance of the outcome of the first study and help provide deeper insights as well. The second study will always lead to a new (but smaller) list of usability problems to fix in a redesign. And the same insight applies to this redesign: not all the fixes will work; some deeper issues will be uncovered after cleaning up the interface. Thus, a third study is needed as well.

The ultimate user experience is improved much more by 3 studies with 5 users each than by a single monster study with 15 users.

Why Not Test With a Single User?

You might think that 15 studies with a single user would be even better than 3 studies with 5 users. The curve does show that we learn much more from the first user than from any subsequent users, so why keep going? Two reasons:

There is always a risk of being misled by the spurious behavior of a single person who may perform certain actions by accident or in an unrepresentative manner. Even 3 users are enough to get an idea of the diversity in user behavior and insight into what's unique and what can be generalized.

The cost-benefit analysis of user testing provides the optimal ratio around 3 or 5 users, depending on the style of testing. There is always a fixed initial cost associated with planning and running a study: it is better to depreciate this start-up cost across the findings from multiple users.

When To Test More Users

You need to test additional users when a website has several highly distinct groups of users. The formula only holds for comparable users who will be using the site in fairly similar ways.

If, for example, you have a site that will be used by both children and parents, then the two groups of users will have sufficiently different behavior that it becomes necessary to test with people from both groups. The same would be true for a system aimed at connecting purchasing agents with sales staff.

Even when the groups of users are very different, there will still be great similarities between the observations from the two groups. All the users are human, after all. Also, many of the usability problems are related to the fundamental way people interact with the Web and the influence from other sites on user behavior.

In testing multiple groups of disparate users, you don't need to include as many members of each group as you would in a single test of a single group of users. The overlap between observations will ensure a better outcome from testing a smaller number of people in each group. I recommend:

3-4 users from each category if testing two groups of users

3 users from each category if testing three or more groups of users (you always want at least 3 users to ensure that you have covered the diversity of behavior within the group)

Reference

Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems," Proceedings of ACM INTERCHI'93 Conference (Amsterdam, The Netherlands, 24-29 April 1993), pp. 206-213.

Follow-Up Articles

# data[].attributes.attachment_link.keywords[]
mathematical model of finding usability problems

# data[].attributes.attachment_link.description
Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.

# data[].attributes.attachment_link.readable_text
Summary: UX researchers use this popular observational methodology to uncover problems and opportunities in designs.

Usability testing is a popular UX research methodology.

Definition: In a usability-testing session, a researcher (called a "facilitator" or a "moderator") asks a participant to perform tasks, usually using one or more specific user interfaces. While the participant completes each task, the researcher observes the participant's behavior and listens for feedback.

The phrase "usability testing" is often used interchangeably with "user testing."

Why Usability Test?

The goals of usability testing vary by study, but they usually include:

Identifying problems in the design of the product or service

Uncovering opportunities to improve

Learning about the target user's behavior and preferences

Why do we need to do usability testing? Won't a good professional UX designer know how to design a great user interface? Even the best UX designers can't design a perfect - or even good enough - user experience without iterative design driven by observations of real users and of their interactions with the design.

There are many variables in designing a modern user interface and there are even more variables in the human brain. The total number of combinations is huge. The only way to get UX design right is to test it.

Elements of Usability Testing

There are many different types of usability testing, but the core elements in most usability tests are the facilitator, the tasks, and the participant.

The facilitator administers tasks to the participant. As the participant performs these tasks, the facilitator observes the participant's behavior and listens for feedback. The facilitator may also ask followup questions to elicit detail from the participant.

Facilitator

The facilitator guides the participant through the test process. She gives instructions, answers the participant's questions, and asks followup questions.

The facilitator works to ensure that the test results in high-quality, valid data, without accidentally influencing the participant's behavior. Achieving this balance is difficult and requires training.

(In one form of remote usability testing, called remote unmoderated testing, an application may perform some of the facilitator's roles.)

Tasks

The tasks in a usability test are realistic activities that the participant might perform in real life. They can be very specific or very open-ended, depending on the research questions and the type of usability testing.

Examples of tasks from real usability studies:

Your printer is showing "Error 5200". How can you get rid of the error message? You're considering opening a new credit card with Wells Fargo. Please visit wellsfargo.com and decide which credit card you might want to open, if any. You've been told you need to speak to Tyler Smith from the Project Management department. Use the intranet to find out where they are located. Tell the researcher your answer.

Task wording is very important in usability testing. Small errors in the phrasing of a task can cause the participant to misunderstand what they're asked to do or can influence how participants perform the task (a psychological phenomenon called priming).

Task instructions can be delivered to the participant verbally (the facilitator might read them) or can be handed to a participant written on task sheets. We often ask participants to read the task instructions out loud. This helps ensure that the participant reads the instructions completely, and helps the researchers with their notetaking, because they always know which task the user is performing.

Participant

The participant should be a realistic user of the product or service being studied. That might mean that the user is already using the product or service in real life. Alternatively, in some cases, the participant might just have a similar background to the target user group, or might have the same needs, even if he isn't already a user of the product.

Participants are often asked to think out loud during usability testing (called the "think-aloud method"). The facilitator might ask the participants to narrate their actions and thoughts as they perform tasks. The goal of this approach is to understand participants' behaviors, goals, thoughts, and motivations.

Types of Usability Testing

Qualitative vs. Quantitative

Usability testing can be either qualitative or quantitative.

Qualitative usability testing focuses on collecting insights, findings, and anecdotes about how people use the product or service. Qualitative usability testing is best for discovering problems in the user experience. This form of usability testing is more common than quantitative usability testing.

Quantitative usability testing focuses on collecting metrics that describe the user experience. Two of the metrics most commonly collected in quantitative usability testing are task success and time on task. Quantitative usability testing is best for collecting benchmarks.

The number of participants needed for a usability test varies depending on the type of study. For a typical qualitative usability study of a single user group, we recommend using five participants to uncover the majority of the most common problems in the product.

Remote vs. In-Person Testing

Remote usability tests are popular because they often require less time and money than in-person studies. There are two types of remote usability testing: moderated and unmoderated.

Remote moderated usability tests work very similarly to in-person studies. The facilitator still interacts with the participant and asks her to perform tasks. However, the facilitator and participant are in different physical locations. Usually, moderated tests can be performed using screen-sharing software like Skype or GoToMeeting.

Remote unmoderated remote usability tests do not have the same facilitator-participant interaction as an in-person or moderated tests. The researcher uses a dedicated online remote-testing tool to set up written tasks for the participant. Then, the participant completes those tasks alone on her own time. The testing tool delivers the task instructions and any followup questions. After the participant completes her test, the researcher receives a recording of the session, along with metrics like task success.

Cost of Usability Testing

Simple, "discount" usability studies can be inexpensive, though you usually must pay a few hundred dollars as incentives to participants. The testing session can take place in a conference room, and the simplest study will take 3 days of your time (assuming that you have already learned how to do it, and you have access to participants):

Day 1: Plan the study

Day 2: Test the 5 users

Day 3: Analyze the findings and convert them into redesign recommendations for the next iteration

On the other hand, more-expensive research is sometimes required, and the cost can run into several hundred thousand dollars for the most elaborate studies.

Things that add cost include:

The return on investment (ROI) for advanced studies can still be high, though usually not as high as that for simple studies.

NN/g Resources for Usability Testing

Facilitating a Usability Test

For hands-on training and help honing your facilitation skills, check out our full-day course on usability testing.

Recruiting Participants

Remote Usability Testing

For detailed help planning, conducting, and analyzing remote user testing, check out our full-day seminar: Remote Usability Testing.

Special Usability Testing Studies or User Groups

# data[].attributes.attachment_link.description
UX researchers use this popular observational methodology to uncover problems and opportunities in designs.

# data[].attributes.headline
FINAL REMINDER: The link for wireframe submission will be closed this afternoon. I only saw 3 groups who have uploaded. This is a final reminder for you.

# data[].attributes.subject
FINAL REMINDER: The link for wireframe submission will be closed this afternoon. I only saw 3 groups who have uploaded. This is a final reminder for you.

# data[].attributes.headline
Your task for today is to create wireframe individually (but next week, you present it in group). Upload your individual wireframe to the link that I will put in the comment of this Announcement. The link will be closed tomorrow 23 February, 2024 11: 59 PM 

# data[].attributes.subject
Your task for today is to create wireframe individually (but next week, you present it in group). Upload your individual wireframe to the link that I will put in the comment of this Announcement. The link will be closed tomorrow 23 February, 2024 11: 59 PM 

# data[].attributes.headline
FINAL Instruction for Phase 2: you are required to create & present the result of your ideation (based on user needs) in the form of 1)storyboard, 2)user flow, and 3)wireframe. Deadline: Tuesday, 27 February 2024 (also, only present the wireframe in class). The link to upload: https://bit.ly/1725phase2

# data[].attributes.subject
FINAL Instruction for Phase 2: you are required to create & present the result of your ideation (based on user needs) in the form of 1)storyboard, 2)user flow, and 3)wireframe. Deadline: Tuesday, 27 February 2024 (also, only present the wireframe in class). The link to upload: https://bit.ly/1725phase2

# data[].attributes.headline
Instruction to upload your group's userflow: upload your individual and group's userflow here:  https://bit.ly/1725userflow  -> make sure it is clear to see your individual effort and your group's userflow. The link will be closed tonight 1 February, 2024.  

# data[].attributes.subject
Instruction to upload your group's userflow: upload your individual and group's userflow here:  https://bit.ly/1725userflow  -> make sure it is clear to see your individual effort and your group's userflow. The link will be closed tonight 1 February, 2024.  

# data[].attributes.attachment_link.title
Project Phase 2 - UserFlow - 1725 - Google Drive

# data[].attributes.attachment_link.description
Your shared drive is empty. Add files and people from the desktop to get started.

# data[].attributes.headline
Individual quiz today after you present the storyboard.  Instruction: This quiz is not a group quiz (this is an individual quiz). The link to the quiz will be closed by the end of the class today 30 January, 2023. The final 3 questions from the quiz will need to be answered by you using separate word document that you create. Upload this document to the link provided inside the quiz.

# data[].attributes.subject
Individual quiz today after you present the storyboard.  Instruction: This quiz is not a group quiz (this is an individual quiz). The link to the quiz will be closed by the end of the class today 30 January, 2023. The final 3 questions from the quiz will need to be answered by you using separate word document that you create. Upload this document to the link provided inside the quiz.

# data[].attributes.headline
Instruction to upload storyboard: upload your individual and group's storyboard here: https://bit.ly/1725storyboard  ->  make sure it is clear to see your individual effort and your group's storyboard. The link will be closed tonight 25 January, 2024.

# data[].attributes.subject
Instruction to upload storyboard: upload your individual and group's storyboard here: https://bit.ly/1725storyboard  ->  make sure it is clear to see your individual effort and your group's storyboard. The link will be closed tonight 25 January, 2024.

# data[].attributes.attachment_link.title
Project Phase 2 - STORYBOARD (RENAME your name before you upload) - Google Drive

# data[].attributes.headline
Here is one example of a user storyboard. In this example, the group presents users' problem with finding part time jobs the goal is to find it and apply it using an app.

# data[].attributes.subject
Here is one example of a user storyboard. In this example, the group presents users' problem with finding part time jobs the goal is to find it and apply it using an app.

# data[].attributes.headline
Instruction: you are required to create & present the result of your ideation (based on user needs) in the form of 1)storyboard, 2)user flow, and 3)wireframe. 

# data[].attributes.subject
Instruction: you are required to create & present the result of your ideation (based on user needs) in the form of 1)storyboard, 2)user flow, and 3)wireframe. 

# data[].attributes.body
First the deadline for the storyboard is January 25, 2024 and you have to present it in class.

# data[].attributes.headline
You can email me question here: ArsyadiRachmat@fpt.edu.vn

# data[].attributes.subject
You can email me question here: ArsyadiRachmat@fpt.edu.vn

# data[].attributes.headline
Getting start on your ideation with user storyboard. Instruction: create your your user storyboard based on the user journey map that you have created.

# data[].attributes.subject
Getting start on your ideation with user storyboard. Instruction: create your your user storyboard based on the user journey map that you have created.

# data[].attributes.headline
As a final preparation for the Poject Phase 1 presentation, you need to create your user journey map. This can be adjusted/revised later after you complete user interview. But you need to do this today as a hypotheses for you do the next step after the interview. Submit here and give the link before you leave the class.

# data[].attributes.subject
As a final preparation for the Poject Phase 1 presentation, you need to create your user journey map. This can be adjusted/revised later after you complete user interview. But you need to do this today as a hypotheses for you do the next step after the interview. Submit here and give the link before you leave the class.

# data[].attributes.headline
Instruction: you are required to present the result of your user research as a foundation for your next step in designing interaction that help users reach their goal. See the example presentation template here. Deadline: Saturday, 18 January 2024, 10 PM.

# data[].attributes.subject
Instruction: you are required to present the result of your user research as a foundation for your next step in designing interaction that help users reach their goal. See the example presentation template here. Deadline: Saturday, 18 January 2024, 10 PM.

# data[].attributes.headline
IMPORTANT! Here is the LINK to update your Project Phase 1: https://bit.ly/1725phase1

# data[].attributes.subject
IMPORTANT! Here is the LINK to update your Project Phase 1: https://bit.ly/1725phase1

# data[].attributes.headline
Put your group name, team members & potential problems here: https://bit.ly/1725group

# data[].attributes.subject
Put your group name, team members & potential problems here: https://bit.ly/1725group

# data[].attributes.headline
You can see the full board of the course plan here: 

# data[].attributes.subject
You can see the full board of the course plan here: 

# data[].attributes.attachment_link.title
2023 Fall COURSE Plan (Interaction Design)

# data[].attributes.headline
You can find this board here: https://bit.ly/1725pad

# data[].attributes.subject
You can find this board here: https://bit.ly/1725pad