# data[].attributes.body
In this section, we will learn about the main subfields of Psychology.

# data[].attributes.body
In this section, you will find 8 sections with curated resources on Psychology:

1. Introduction to Psychology

2. Subfields of Psychology

Social Psychology

Developmental Psychology

Cognitive Psychology

Biological Psychology

Abnormal Psychology

Personality Psychology

3. Key Principles in Psychological Research

4. Conducting Psychological Research

5. Research Ethics

6. Study Design

7. Data Analysis

8. Resources and Tools

# data[].attributes.body
In this section, we will learn about some fundamental principles central to psychological research.

# data[].attributes.body
In this section, we will explore the field of Psychology and understand its significance in the world.

# data[].attributes.body
In this section, we will learn how to conceptualise a research project and write a research paper in Psychology.

# data[].attributes.body
[12 mins] An introduction to Social Psychology.

# data[].attributes.attachment_link.title
What is Social Psychology? An Introduction

# data[].attributes.body
[5 mins] What is Psychology? This short video from Macat explains how the subject has developed over the years and introduces some of the key ideas and major thinkers who have helped to shape it.

# data[].attributes.attachment_link.title
An introduction to the discipline of Psychology

# data[].attributes.body
[3 mins] In this video, Dr Brenda Whitehead provides an introduction to Developmental Psychology.

# data[].attributes.attachment_link.title
Developmental Psychology: Why Should I Care?

# data[].attributes.body
[10 mins] This video summarises the relevance of Psychology in our world and some reasons why we study it.

# data[].attributes.attachment_link.title
Why Study Psychology? | College Majors | College Degrees | Study Hall

# data[].attributes.body
This website explains how to select a research topic, generate research questions, and develop hypotheses based on your literature review.

Reviewing the literature will not only help you gain a better understanding of your chosen topic, but is crucial to identify a gap that your research will aim to fill.

# data[].attributes.attachment_link.title
2.2 Finding a Research Topic – Research Methods in Psychology

# data[].attributes.attachment_link.description
This third American edition is a comprehensive textbook for research methods classes. It is an adaptation of the second American edition.

# data[].attributes.body
[3 mins] An introduction to Cognitive Psychology.

# data[].attributes.attachment_link.title
What is Cognitive Psychology? (Explained in 3 Minutes)

# data[].attributes.body
[6 mins] An introduction to biological psychology (sometimes known as behavioural neuroscience).

# data[].attributes.body
[2 mins] An introduction to Abnormal Psychology.

# data[].attributes.attachment_link.title
What is Abnormal Psychology: Definition and Common Disorders Studied

# data[].attributes.attachment_link.description
Visit Study.com for thousands more videos like this one. You'll get full access to our interactive quizzes and transcripts and can find out how to use our videos to earn real college credit. YouTube hosts only the first few lessons in each course. The rest are at Study.com.

# data[].attributes.body
[14 mins] An introduction to Personality Psychology (sometimes known as the psychology of individual differences or differential psychology).

# data[].attributes.attachment_link.title
What is Personality? - Personality Psychology

# data[].attributes.body
This website offers resources and tips for (1) identifying sources to conduct your literature review and (2) using search terms effectively.

# data[].attributes.attachment_link.description
Research Guides Brandeis Library’s research guides provide tips on how to do library research, as well as recommended resources for specific subject areas, courses, or types of research.

# data[].attributes.headline
4.3 Types of Psychological Research Data 

# data[].attributes.subject
4.3 Types of Psychological Research Data 

# data[].attributes.body
[9 mins] After deciding on your research questions and hypotheses, the next step is to design your study (which is expounded on in another section) and collect your data. This video provides an introduction to the various types of data that can be collected in psychological research studies.

Decisions about the type of data that is most appropriate to collect for a study typically depends on your research questions and hypotheses, the resources available to you, and ethical considerations.

# data[].attributes.attachment_link.title
Types of Data - Research Methods [A-Level Psychology]

# data[].attributes.headline
4.4 How to Write a Psychology Research Paper

# data[].attributes.subject
4.4 How to Write a Psychology Research Paper

# data[].attributes.body
Writing a paper is often an end goal of conducting psychological research. This article is a useful introduction to common types of Psychology papers, and a guide for how to structure a (1) literature review or an (2) empirical paper/ research proposal.

# data[].attributes.body
[5 mins] What exactly constitutes psychological research, and what makes it unique? This video introduces the purpose of psychological research, and why research is important in this field.

# data[].attributes.attachment_link.title
The Purpose of Psychological Research: PSYCHademia, AP Psychology

# data[].attributes.body
[2 mins] Operationalisation is the process of defining the measurement of a phenomenon in a research study. This is crucial in psychological research because it enables researchers to turn abstract concepts into measurable variables.

# data[].attributes.attachment_link.title
Variables - Conceptual and Operational Definitions

# data[].attributes.body
[8 mins] Psychological research is grounded in empirical evidence. This means that researchers gather data through systematic observation or experimentation, from which they draw conclusions.

Psychological researchers often generate hypotheses prior to conducting their research studies, which are then tested by determining whether the results of the study support these predictions.

# data[].attributes.attachment_link.title
What is a hypothesis test? A beginner's guide to hypothesis testing!

# data[].attributes.body
[7 mins] Sampling in research is the process of selecting a group of participants from a larger population, as it is often not feasible to study the entire population. This video explains different sampling techniques in psychological research, as well as their strengths and weaknesses.

# data[].attributes.attachment_link.title
Sampling - Research Methods [A-Level Psychology]

# data[].attributes.body
In this section, we will learn about different types of study designs used in psychological research.

# data[].attributes.body
This section will provide you with some digital resources and tools for data analysis in psychological research.

# data[].attributes.body
Psychological research often involves human participants. This section provides an overview of the main principles of research ethics. The Belmont Report, published in 1979, outlines ethical guidelines for the protection of human subjects of research.

# data[].attributes.body
Conducting psychological research often involves data analysis. Crash Course Statistics is useful to consult for a general understanding of statistical concepts. To delve deeper, this free textbook is a great resource.

Familiarising yourself to various types of statistical errors is also highly recommended, for which you can consult Statistics Done Wrong by Alex Reinhart.

Common softwares used for data analysis in psychological research include:

Jamovi

SPSS

R Studio

NVivo (qualitative analysis)

Resources for Jamovi software can be found here:

Instructional videos

Tutorial for opening data files

# data[].attributes.body
According to the principle of respect for persons, (1) individuals should be treated as autonomous agents and (2) individuals with diminished autonomy should be protected.

In line with this, individuals should take part in research voluntarily and with adequate information, and informed consent should be documented prior to participation. Participants' confidentiality should be maintained.

# data[].attributes.body
The principle of beneficence implies that efforts should be made to ensure individuals' well-being in research. This is done by (1) not causing harm and (2) maximising possible benefits and minimising possible harms.

# data[].attributes.body
Justice in research ethics refers to ensuring a fair selection of participants and distribution of the benefits and risks of a research study across participants and different societal groups.

Generally, individuals should have equal opportunity to participate in research. Moreover, vulnerable groups, such as children, the institutionalised, or marginalised communities, should be protected from exploitation and from being involved in research just because of easy availability or convenience.

# data[].attributes.body
Descriptive studies seek to describe behaviours or responses the way they are observed and measured, with no manipulation of variables. Descriptive studies can be qualitative, quantitative, or both (mixed-methods). These can include the following techniques:

Observation: Behaviours, actions, or responses are systematically observed and recorded.

Self-report measures: Participants are asked to provide information about themselves, e.g., through surveys or interviews.

# data[].attributes.attachment_link.title
Descriptive Research – General Psychology

# data[].attributes.body
Correlational studies are non-experimental studies that seek to examine relationships between variables. Hypotheses are made about the relationships between variables, which are then tested. A correlation coefficient and p-value will inform the researcher about the direction of the relationship between two variables and whether it is statistically significant.

It is important to note that acorrelation between two variables does not suggest a cause-and-effect relationship.

# data[].attributes.attachment_link.readable_text
1. Define correlational research and give several examples.

2. Explain why a researcher might choose to conduct correlational research rather than experimental research or another type of nonexperimental research.

Correlational research is a type of nonexperimental research in which the researcher measures two variables and assesses the statistical relationship (i.e., the correlation) between them with little or no effort to control extraneous variables. There are essentially two reasons that researchers interested in statistical relationships between variables would choose to conduct a correlational study rather than an experiment. The first is that they do not believe that the statistical relationship is a causal one. For example, a researcher might evaluate the validity of a brief extraversion test by administering it to a large group of participants along with a longer extraversion test that has already been shown to be valid. This researcher might then check to see whether participants' scores on the brief test are strongly correlated with their scores on the longer one. Neither test score is thought to cause the other, so there is no independent variable to manipulate. In fact, the terms and independent variable dependent variabl e do not apply to this kind of research.

The other reason that researchers would choose to use a correlational study rather than an experiment is that the statistical relationship of interest is thought to be causal, but the researcher manipulate the independent variable because it is impossible, impractical, or unethical. For example, Allen Kanner and his colleagues thought that the number of "daily hassles" (e.g., rude salespeople, heavy traffic) that people experience affects the number of physical and psychological symptoms they have (Kanner, Coyne, Schaefer, & Lazarus, 1981). But because they could not manipulate the number of daily hassles their participants experienced, they had to settle for the number of daily hassles-along with the number of symptoms-using self-report questionnaires. Although the strong positive relationship they found between these two variables is consistent with their idea that hassles cause symptoms, it is also consistent with the idea that symptoms cause hassles or that some third variable (e.g., neuroticism) causes both.

A common misconception among beginning researchers is that correlational research must involve two quantitative variables, such as scores on two extroversion tests or the number of hassles and number of symptoms people have experienced. However, the defining feature of correlational research is that the two variables are measured-neither one is manipulated-and this is true regardless of whether the variables are quantitative or categorical. Imagine, for example, that a researcher administers the Rosenberg Self-Esteem Scale to 50 American university students and 50 Japanese university students. Although this "feels" like a between-subjects experiment, it is a correlational study because the researcher did not manipulate the students' nationalities. The same is true of the study by Cacioppo and Petty comparing professors and factory workers in terms of their need for cognition. It is a correlational study because the researchers did not manipulate the participants' occupations.

Figure 7.2 shows data from a hypothetical study on the relationship between whether people make a daily list of things to do (a "to-do list") and stress. Notice that it is unclear whether this design is an experiment or a correlational study because it is unclear whether the independent variable was manipulated. If the researcher randomly assigned some participants to make daily to-do lists and others not to, then it is an experiment. If the researcher simply asked participants whether they made daily to-do lists, then it is a correlational study. The distinction is important because if the study was an experiment, then it could be concluded that making the daily to-do lists reduced participants' stress. But if it was a correlational study, it could only be concluded that these variables are related. Perhaps being stressed has a negative effect on people's ability to plan ahead (the directionality problem). Or perhaps people who are more conscientious are more likely to make to-do lists and less likely to be stressed (the third-variable problem). The crucial point is that what defines a study as experimental or correlational is not the variables being studied, nor whether the variables are quantitative or categorical, nor the type of graph or statistics used to analyze the data. It is how the study is conducted.

Again, the defining feature of correlational research is that neither variable is manipulated. It does not matter how or where the variables are measured. A researcher could have participants come to a laboratory to complete a computerized backward digit span task and a computerized risky decision-making task and then assess the relationship between participants' scores on the two tasks. Or a researcher could go to a shopping mall to ask people about their attitudes toward the environment and their shopping habits and then assess the relationship between these two variables. Both of these studies would be correlational because no independent variable is manipulated. However, because some approaches to data collection are strongly associated with correlational research, it makes sense to discuss them here. The two we will focus on are naturalistic observation and archival data. A third, survey research, is discussed in its own chapter, Chapter 9.

Naturalistic observation is an approach to data collection that involves observing people's behaviour in the environment in which it typically occurs. Thus naturalistic observation is a type of field research (as opposed to a type of laboratory research). It could involve observing shoppers in a grocery store, children on a school playground, or psychiatric inpatients in their wards. Researchers engaged in naturalistic observation usually make their observations as unobtrusively as possible so that participants are often not aware that they are being studied. Ethically, this method is considered to be acceptable if the participants remain anonymous and the behaviour occurs in a public setting where people would not normally have an expectation of privacy. Grocery shoppers putting items into their shopping carts, for example, are engaged in public behaviour that is easily observable by store employees and other shoppers. For this reason, most researchers would consider it ethically acceptable to observe them for a study. On the other hand, one of the arguments against the ethicality of the naturalistic observation of "bathroom behaviour" discussed earlier in the book is that people have a reasonable expectation of privacy even in a public restroom and that this expectation was violated.

Researchers Robert Levine and Ara Norenzayan used naturalistic observation to study differences in the "pace of life" across countries (Levine & Norenzayan, 1999). One of their measures involved observing pedestrians in a large city to see how long it took them to walk 60 feet. They found that people in some countries walked reliably faster than people in other countries. For example, people in Canada and Sweden covered 60 feet in just under 13 seconds on average, while people in Brazil and Romania took close to 17 seconds.

Because naturalistic observation takes place in the complex and even chaotic "real world," there are two closely related issues that researchers must deal with before collecting data. The first is sampling. When, where, and under what conditions will the observations be made, and who exactly will be observed? Levine and Norenzayan described their sampling process as follows:

"Male and female walking speed over a distance of 60 feet was measured in at least two locations in main downtown areas in each city. Measurements were taken during main business hours on clear summer days. All locations were flat, unobstructed, had broad sidewalks, and were sufficiently uncrowded to allow pedestrians to move at potentially maximum speeds. To control for the effects of socializing, only pedestrians walking alone were used. Children, individuals with obvious physical handicaps, and window-shoppers were not timed. Thirty-five men and 35 women were timed in most cities." (p. 186)

Precise specification of the sampling process in this way makes data collection manageable for the observers, and it also provides some control over important extraneous variables. For example, by making their observations on clear summer days in all countries, Levine and Norenzayan controlled for effects of the weather on people's walking speeds.

The second issue is measurement. What specific behaviours will be observed? In Levine and Norenzayan's study, measurement was relatively straightforward. They simply measured out a 60-foot distance along a city sidewalk and then used a stopwatch to time participants as they walked over that distance. Often, however, the behaviours of interest are not so obvious or objective. For example, researchers Robert Kraut and Robert Johnston wanted to study bowlers' reactions to their shots, both when they were facing the pins and then when they turned toward their companions (Kraut & Johnston, 1979). But what "reactions" should they observe? Based on previous research and their own pilot testing, Kraut and Johnston created a list of reactions that included "closed smile," "open smile," "laugh," "neutral face," "look down," "look away," and "face cover" (covering one's face with one's hands). The observers committed this list to memory and then practised by coding the reactions of bowlers who had been videotaped. During the actual study, the observers spoke into an audio recorder, describing the reactions they observed. Among the most interesting results of this study was that bowlers rarely smiled while they still faced the pins. They were much more likely to smile after they turned toward their companions, suggesting that smiling is not purely an expression of happiness but also a form of social communication.

When the observations require a judgment on the part of the observers-as in Kraut and Johnston's study-this process is often described as . Coding generally requires clearly defining a set of target behaviours. The observers then categorize participants individually in terms of which behaviour they have engaged in and the number of times they engaged in each behaviour. The observers might even record the duration of each behaviour. The target behaviours must be defined in such a way that different observers code them in the same way. This difficulty with coding is the issue of interrater reliability, as mentioned in Chapter 5. Researchers are expected to demonstrate the interrater reliability of their coding procedure by having multiple raters code the same behaviours independently and then showing that the different observers are in close agreement. Kraut and Johnston, for example, video recorded a subset of their participants' reactions and had two observers independently code them. The two observers showed that they agreed on the reactions that were exhibited 97% of the time, indicating good interrater reliability.

Another approach to correlational research is the use of , which are data that have already been collected for some other purpose. An example is a study by Brett Pelham and his colleagues on "implicit egotism"-the tendency for people to prefer people, places, and things that are similar to themselves (Pelham, Carvallo, & Jones, 2005). In one study, they examined Social Security records to show that women with the names Virginia, Georgia, Louise, and Florence were especially likely to have moved to the states of Virginia, Georgia, Louisiana, and Florida, respectively.

As with naturalistic observation, measurement can be more or less straightforward when working with archival data. For example, counting the number of people named Virginia who live in various states based on Social Security records is relatively straightforward. But consider a study by Christopher Peterson and his colleagues on the relationship between optimism and health using data that had been collected many years before for a study on adult development (Peterson, Seligman, & Vaillant, 1988). In the 1940s, healthy male college students had completed an open-ended questionnaire about difficult wartime experiences. In the late 1980s, Peterson and his colleagues reviewed the men's questionnaire responses to obtain a measure of explanatory style-their habitual ways of explaining bad events that happen to them. More pessimistic people tend to blame themselves and expect long-term negative consequences that affect many aspects of their lives, while more optimistic people tend to blame outside forces and expect limited negative consequences. To obtain a measure of explanatory style for each participant, the researchers used a procedure in which all negative events mentioned in the questionnaire responses, and any causal explanations for them, were identified and written on index cards. These were given to a separate group of raters who rated each explanation in terms of three separate dimensions of optimism-pessimism. These ratings were then averaged to produce an explanatory style score for each participant. The researchers then assessed the statistical relationship between the men's explanatory style as undergraduate students and archival measures of their health at approximately 60 years of age. The primary result was that the more optimistic the men were as undergraduate students, the healthier they were as older men. Pearson's was +.25.

This method is an example of -a family of systematic approaches to measurement using complex archival data. Just as naturalistic observation requires specifying the behaviours of interest and then noting them as they occur, content analysis requires specifying keywords, phrases, or ideas and then finding all occurrences of them in the data. These occurrences can then be counted, timed (e.g., the amount of time devoted to entertainment topics on the nightly news show), or analyzed in a variety of other ways.

Correlational research involves measuring two variables and assessing the relationship between them, with no manipulation of an independent variable.

Correlational research is not defined by where or how the data are collected. However, some approaches to data collection are strongly associated with correlational research. These include naturalistic observation (in which researchers observe people's behaviour in the context in which it normally occurs) and the use of archival data that were already collected for some other purpose.

Discussion: For each of the following, decide whether it is most likely that the study described is experimental or correlational and explain why.

1. An educational researcher compares the academic performance of students from the "rich" side of town with that of students from the "poor" side of town.

2. A cognitive psychologist compares the ability of people to recall words that they were instructed to "read" with their ability to recall words that they were instructed to "imagine."

3. A manager studies the correlation between new employees' college grade point averages and their first-year performance reports.

4. An automotive engineer installs different stick shifts in a new car prototype, each time asking several people to rate how comfortable the stick shift feels.

5. A food scientist studies the relationship between the temperature inside people's refrigerators and the amount of bacteria on their food.

6. A social psychologist tells some research participants that they need to hurry over to the next building to complete a study. She tells others that they can take their time. Then she observes whether they stop to help a research assistant who is pretending to be hurt.

An approach to data collection that involves observing people's behaviour in the environment in which it typically occurs.

A judgment on part of the observers by clearly defining a set of target behaviours.

Data that have already been collected for some other purpose.

A family of systematic approaches to measurement using complex archival data.

# data[].attributes.attachment_link.title
Research Methods in Psychology - 2nd Canadian Edition

# data[].attributes.attachment_link.description
Define correlational research and give several examples. Explain why a researcher might choose to conduct correlational research rather than experimental research or another type of nonexperimental research. Correlational research is a type of nonexperimental research in which the researcher measures two variables and assesses the statistical relationship (i.e., the correlation) between them with little or no effort to control extraneous variables.

# data[].attributes.headline
6.3 Experimental and Quasi-Experimental Studies

# data[].attributes.subject
6.3 Experimental and Quasi-Experimental Studies

# data[].attributes.body
[3 mins] This video provides a brief explanation of the basic process of experimental design and its purpose. In experimental studies, the independent variable(s) are manipulated to determine whether a cause-and-effect relationship is present with the dependent variable(s). The researcher may also control for extraneous variables (such as confounders or covariates).

In a true experiment, participants are randomly assigned to either the experimental or control group. In a quasi-experiment, a comparison group is present but participants are not randomly assigned to groups.

Some studies investigate the effect of more than one independent variable on a dependent variable. This is known as a factorial design [5 mins].

# data[].attributes.attachment_link.description
This episode explains the basic process of experimental design, its purpose, and its applications in the field of psychology. written by David Devonis produced by ByPass Publishing copyright 2013

# data[].attributes.body
This section will introduce you to other digital resources and tools that may be useful for understanding and conducting psychological research.

# data[].attributes.body
A list of open access journals in the field of Psychology can be found at this directory.

# data[].attributes.attachment_link.readable_text
About the directory

DOAJ is a unique and extensive index of diverse open access journals from around the world, driven by a growing community, and is committed to ensuring quality content is freely available online for everyone.

DOAJ is committed to keeping its services free of charge, including being indexed, and its data freely available.

→ About DOAJ

→ How to apply

DOAJ is twenty years old in 2023.

Fund our 20th anniversary campaign

Funding

DOAJ is independent. All support is via donations.

Support DOAJ

Publishers don't need to donate to be part of DOAJ.

Librarianship, Scholarly Publishing, Data Management
Fei Yu

Brisbane, Australia (Chinese, English)

Lut Tamam

Adana, Türkiye (Turkish, English)

Humanities, Social Sciences
Natalia Pamuła-Cieślak

Toruń, Poland (Polish, English)

Research Evaluation
Paola Galimberti

Milan, Italy (Italian, German, English)

# data[].attributes.attachment_link.title
Directory of Open Access Journals - DOAJ

# data[].attributes.attachment_link.description
DOAJ is a unique and extensive index of diverse open access journals from around the world, driven by a growing community, committed to ensuring quality content is freely available online for everyone.

# data[].attributes.headline
8.2 Understanding Bias in Research Studies

# data[].attributes.subject
8.2 Understanding Bias in Research Studies

# data[].attributes.body
This website is a useful catalogue of types of biases in research. A bias is a systematic distortion in results that can be introduced at any stage of the research process. Although it is challenging to eliminate bias entirely, being aware of potential biases helps us better assess the credibility of the research we consume, and can reduce their potential impact on studies we conduct.

# data[].attributes.attachment_link.description
Here is a list of all Biases in the Catalogue. Check back regularly as we are adding new ones over time.

# data[].attributes.body
This website by the University of Brighton lists some commonly used psychological measures and scales, and provides links to other databases. It is important to discuss the selection of measurement scales with your supervisor, check the necessary permissions before using them in your work, and reference any measures you use.

# data[].attributes.attachment_link.title
Psychology lab at the University of Brighton

# data[].attributes.attachment_link.description
This page contains information on a range of measures (psychometric tests) freely available to psychology students, and the formats in which they are av...

# data[].attributes.body
Several online survey tools are commonly used to collect data for psychological research, including:

REDCap

Qualtrics

PsyToolkit

SurveyMonkey

# data[].attributes.body
The American Psychology Association (APA) website offers detailed guidelines for formatting a Psychology manuscript, including instructions for citation style.

# data[].attributes.attachment_link.readable_text
Consistency in the order, structure, and format of a paper allows readers to focus on a paper's content rather than its presentation.

To format a paper in APA Style, writers can typically use the default settings and automatic formatting tools of their word-processing program or make only minor adjustments.

The guidelines for paper format apply to both student assignments and manuscripts being submitted for publication to a journal. If you are using APA Style to create another kind of work (e.g., a website, conference poster, or PowerPoint presentation), you may need to format your work differently in order to optimize its presentation, for example, by using different line spacing and font sizes. Follow the guidelines of your institution or publisher to adapt APA Style formatting guidelines as needed.

Additional resources

# data[].attributes.attachment_link.description
Consistency in the order, structure, and format of a paper allows readers to focus on a paper's content rather than its presentation. To format a paper in APA Style, writers can typically use the default settings and automatic formatting tools of their word-processing program or make only minor adjustments.

# data[].attributes.body
Validity is the degree to which a study's results reflect the variables they are intended to measure. Internal validity is concerned with the accurate reflection of observed relationships between variables in a study, while external validity refers to the extent to which a study's results can be generalised to other situations and groups.

Reliability is the consistency of a study's results when repeated under the same conditions. In line with this, we should aim to provide sufficient details about study methodology, so that other researchers can determine whether our research findings are replicable.

This website provides an overview of several key forms of validity and reliability.

# data[].attributes.attachment_link.readable_text
Learning Objectives

1. Define reliability, including the different types and how they are assessed.

2. Define validity, including the different types and how they are assessed.

3. Describe the kinds of evidence that would be relevant to assessing the reliability and validity of a particular measure.

Again, measurement involves assigning scores to individuals so that they represent some characteristic of the individuals. But how do researchers know that the scores actually represent the characteristic, especially when it is a construct like intelligence, self-esteem, depression, or working memory capacity? The answer is that they conduct research using the measure to confirm that the scores make sense based on their understanding of the construct being measured. This is an extremely important point. Psychologists do not simply assume that their measures work. Instead, they collect data to demonstrate that they work. If their research does not demonstrate that a measure works, they stop using it.

As an informal example, imagine that you have been dieting for a month. Your clothes seem to be fitting more loosely, and several friends have asked if you have lost weight. If at this point your bathroom scale indicated that you had lost 10 pounds, this would make sense and you would continue to use the scale. But if it indicated that you had gained 10 pounds, you would rightly conclude that it was broken and either fix it or get rid of it. In evaluating a measurement method, psychologists consider two general dimensions: reliability and validity.

Reliability

Reliability refers to the consistency of a measure. Psychologists consider three types of consistency: over time (test-retest reliability), across items (internal consistency), and across different researchers (inter-rater reliability).

Test-Retest Reliability

When researchers measure a construct that they assume to be consistent across time, then the scores they obtain should also be consistent across time. Test-retest reliability is the extent to which this is actually the case. For example, intelligence is generally thought to be consistent across time. A person who is highly intelligent today will be highly intelligent next week. This means that any good measure of intelligence should produce roughly the same scores for this individual next week as it does today. Clearly, a measure that produces highly inconsistent scores over time cannot be a very good measure of a construct that is supposed to be consistent.

Assessing test-retest reliability requires using the measure on a group of people at one time, using it again on the same group of people at a later time, and then looking at test-retest correlation between the two sets of scores. This is typically done by graphing the data in a scatterplot and computing Pearson's r. Figure 5.2 shows the correlation between two sets of scores of several university students on the Rosenberg Self-Esteem Scale, administered two times, a week apart. Pearson's r for these data is +.95. In general, a test-retest correlation of +.80 or greater is considered to indicate good reliability.

Again, high test-retest correlations make sense when the construct being measured is assumed to be consistent over time, which is the case for intelligence, self-esteem, and the Big Five personality dimensions. But other constructs are not assumed to be stable over time. The very nature of mood, for example, is that it changes. So a measure of mood that produced a low test-retest correlation over a period of a month would not be a cause for concern.

Internal Consistency

A second kind of reliability is internal consistency, which is the consistency of people's responses across the items on a multiple-item measure. In general, all the items on such measures are supposed to reflect the same underlying construct, so people's scores on those items should be correlated with each other. On the Rosenberg Self-Esteem Scale, people who agree that they are a person of worth should tend to agree that that they have a number of good qualities. If people's responses to the different items are not correlated with each other, then it would no longer make sense to claim that they are all measuring the same underlying construct. This is as true for behavioural and physiological measures as for self-report measures. For example, people might make a series of bets in a simulated game of roulette as a measure of their level of risk seeking. This measure would be internally consistent to the extent that individual participants' bets were consistently high or low across trials.

Like test-retest reliability, internal consistency can only be assessed by collecting and analyzing data. One approach is to look at a split-half correlation. This involves splitting the items into two sets, such as the first and second halves of the items or the even- and odd-numbered items. Then a score is computed for each set of items, and the relationship between the two sets of scores is examined. For example, Figure 5.3 shows the split-half correlation between several university students' scores on the even-numbered items and their scores on the odd-numbered items of the Rosenberg Self-Esteem Scale. Pearson's r for these data is +.88. A split-half correlation of +.80 or greater is generally considered good internal consistency.

Perhaps the most common measure of internal consistency used by researchers in psychology is a statistic called Cronbach's α (the Greek letter alpha). Conceptually, α is the mean of all possible split-half correlations for a set of items. For example, there are 252 ways to split a set of 10 items into two sets of five. Cronbach's α would be the mean of the 252 split-half correlations. Note that this is not how α is actually computed, but it is a correct way of interpreting the meaning of this statistic. Again, a value of +.80 or greater is generally taken to indicate good internal consistency.

Interrater Reliability

Many behavioural measures involve significant judgment on the part of an observer or a rater. Inter-rater reliability is the extent to which different observers are consistent in their judgments. For example, if you were interested in measuring university students' social skills, you could make video recordings of them as they interacted with another student whom they are meeting for the first time. Then you could have two or more observers watch the videos and rate each student's level of social skills. To the extent that each participant does in fact have some level of social skills that can be detected by an attentive observer, different observers' ratings should be highly correlated with each other. Inter-rater reliability would also have been measured in Bandura's Bobo doll study. In this case, the observers' ratings of how many acts of aggression a particular child committed while playing with the Bobo doll should have been highly positively correlated. Interrater reliability is often assessed using Cronbach's α when the judgments are quantitative or an analogous statistic called Cohen's κ (the Greek letter kappa) when they are categorical.

Validity

Validity is the extent to which the scores from a measure represent the variable they are intended to. But how do researchers make this judgment? We have already considered one factor that they take into account-reliability. When a measure has good test-retest reliability and internal consistency, researchers should be more confident that the scores represent what they are supposed to. There has to be more to it, however, because a measure can be extremely reliable but have no validity whatsoever. As an absurd example, imagine someone who believes that people's index finger length reflects their self-esteem and therefore tries to measure self-esteem by holding a ruler up to people's index fingers. Although this measure would have extremely good test-retest reliability, it would have absolutely no validity. The fact that one person's index finger is a centimetre longer than another's would indicate nothing about which one had higher self-esteem.

Discussions of validity usually divide it into several distinct "types." But a good way to interpret these types is that they are other kinds of evidence-in addition to reliability-that should be taken into account when judging the validity of a measure. Here we consider three basic kinds: face validity, content validity, and criterion validity.

Face Validity

Face validity is the extent to which a measurement method appears "on its face" to measure the construct of interest. Most people would expect a self-esteem questionnaire to include items about whether they see themselves as a person of worth and whether they think they have good qualities. So a questionnaire that included these kinds of items would have good face validity. The finger-length method of measuring self-esteem, on the other hand, seems to have nothing to do with self-esteem and therefore has poor face validity. Although face validity can be assessed quantitatively-for example, by having a large sample of people rate a measure in terms of whether it appears to measure what it is intended to-it is usually assessed informally.

Face validity is at best a very weak kind of evidence that a measurement method is measuring what it is supposed to. One reason is that it is based on people's intuitions about human behaviour, which are frequently wrong. It is also the case that many established measures in psychology work quite well despite lacking face validity. The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) measures many personality characteristics and disorders by having people decide whether each of over 567 different statements applies to them-where many of the statements do not have any obvious relationship to the construct that they measure. For example, the items "I enjoy detective or mystery stories" and "The sight of blood doesn't frighten me or make me sick" both measure the suppression of aggression. In this case, it is not the participants' literal answers to these questions that are of interest, but rather whether the pattern of the participants' responses to a series of questions matches those of individuals who tend to suppress their aggression.

Content Validity

Content validity is the extent to which a measure "covers" the construct of interest. For example, if a researcher conceptually defines test anxiety as involving both sympathetic nervous system activation (leading to nervous feelings) and negative thoughts, then his measure of test anxiety should include items about both nervous feelings and negative thoughts. Or consider that attitudes are usually defined as involving thoughts, feelings, and actions toward something. By this conceptual definition, a person has a positive attitude toward exercise to the extent that he or she thinks positive thoughts about exercising, feels good about exercising, and actually exercises. So to have good content validity, a measure of people's attitudes toward exercise would have to reflect all three of these aspects. Like face validity, content validity is not usually assessed quantitatively. Instead, it is assessed by carefully checking the measurement method against the conceptual definition of the construct.

Criterion Validity

Criterion validity is the extent to which people's scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with. For example, people's scores on a new measure of test anxiety should be negatively correlated with their performance on an important school exam. If it were found that people's scores were in fact negatively correlated with their exam performance, then this would be a piece of evidence that these scores really represent people's test anxiety. But if it were found that people scored equally well on the exam regardless of their test anxiety scores, then this would cast doubt on the validity of the measure.

A criterion can be any variable that one has reason to think should be correlated with the construct being measured, and there will usually be many of them. For example, one would expect test anxiety scores to be negatively correlated with exam performance and course grades and positively correlated with general anxiety and with blood pressure during an exam. Or imagine that a researcher develops a new measure of physical risk taking. People's scores on this measure should be correlated with their participation in "extreme" activities such as snowboarding and rock climbing, the number of speeding tickets they have received, and even the number of broken bones they have had over the years. When the criterion is measured at the same time as the construct, criterion validity is referred to as concurrent validity; however, when the criterion is measured at some point in the future (after the construct has been measured), it is referred to as predictive validity (because scores on the measure have "predicted" a future outcome).

Criteria can also include other measures of the same construct. For example, one would expect new measures of test anxiety or physical risk taking to be positively correlated with existing measures of the same constructs. This is known as convergent validity.

Assessing convergent validity requires collecting data using the measure. Researchers John Cacioppo and Richard Petty did this when they created their self-report Need for Cognition Scale to measure how much people value and engage in thinking (Cacioppo & Petty, 1982). In a series of studies, they showed that people's scores were positively correlated with their scores on a standardized academic achievement test, and that their scores were negatively correlated with their scores on a measure of dogmatism (which represents a tendency toward obedience). In the years since it was created, the Need for Cognition Scale has been used in literally hundreds of studies and has been shown to be correlated with a wide variety of other variables, including the effectiveness of an advertisement, interest in politics, and juror decisions (Petty, Briñol, Loersch, & McCaslin, 2009).

Discriminant Validity

Discriminant validity, on the other hand, is the extent to which scores on a measure are not correlated with measures of variables that are conceptually distinct. For example, self-esteem is a general attitude toward the self that is fairly stable over time. It is not the same as mood, which is how good or bad one happens to be feeling right now. So people's scores on a new measure of self-esteem should not be very highly correlated with their moods. If the new measure of self-esteem were highly correlated with a measure of mood, it could be argued that the new measure is not really measuring self-esteem; it is measuring mood instead.

When they created the Need for Cognition Scale, Cacioppo and Petty also provided evidence of discriminant validity by showing that people's scores were not correlated with certain other variables. For example, they found only a weak correlation between people's need for cognition and a measure of their cognitive style-the extent to which they tend to think analytically by breaking ideas into smaller parts or holistically in terms of "the big picture." They also found no correlation between people's need for cognition and measures of their test anxiety and their tendency to respond in socially desirable ways. All these low correlations provide evidence that the measure is reflecting a conceptually distinct construct.

Key Takeaways

Psychological researchers do not simply assume that their measures work. Instead, they conduct research to show that they work. If they cannot show that they work, they stop using them.

There are two distinct criteria by which researchers evaluate their measures: reliability and validity. Reliability is consistency across time (test-retest reliability), across items (internal consistency), and across researchers (interrater reliability). Validity is the extent to which the scores actually represent the variable they are intended to.

Validity is a judgment based on various types of evidence. The relevant evidence includes the measure's reliability, whether it covers the construct of interest, and whether the scores it produces are correlated with other variables they are expected to be correlated with and not correlated with variables that are conceptually distinct.

The reliability and validity of a measure is not established by any single study but by the pattern of results across multiple studies. The assessment of reliability and validity is an ongoing process.

Exercises

1. Practice: Ask several friends to complete the Rosenberg Self-Esteem Scale. Then assess its internal consistency by making a scatterplot to show the split-half correlation (even- vs. odd-numbered items). Compute Pearson's r too if you know how.

2. Discussion: Think back to the last college exam you took and think of the exam as a psychological measure. What construct do you think it was intended to measure? Comment on its face and content validity. What data could you collect to assess its reliability and criterion validity?

# data[].attributes.attachment_link.description
Define reliability, including the different types and how they are assessed. Define validity, including the different types and how they are assessed. Describe the kinds of evidence that would be relevant to assessing the reliability and validity of a particular measure. Again, measurement involves assigning scores to individuals so that they represent some characteristic of the individuals.