Threats to Validity by

2025-09-04 16:45:57 UTC

Internal threats:

affects that are not the independent variable. Such as participants dropping out

2025-09-04 16:46:37 UTC

Brody Andersen: low statistical power.

When high variability in subjects or low sample sizes creates false negatives in statistical results.

History Effects

2025-09-04 16:47:17 UTC

History effects are events that occur during/in the environment that changes the conditions of the study, affecting its outcome. One example could be a fire alarm going off during a mindfulness practice, interrupting the practice.

External Validity Threats

2025-09-04 16:47:29 UTC

-factors that limit how generalizable the results of research studies are to other situations, people, and location

-if you're looking at how mindfulness impacts anxiety in college students, you can't generalize the results to everyone (college students are different than people in their 70s)

Construct Validity Threats

2025-09-04 16:47:30 UTC

When study operations don’t accurately represent the theoretical constructs they are intended to capture

Ex. During research, the variable of Mindfulness might not be properly conceptualized, which can lead to inaccurate measurements.

(Finish) p-hacking

2025-09-04 16:48:00 UTC

Manipulating study data to achieve a desired result instead of using that data to come up with the result.

For example, taking a mindfulness survey and only using the data that shows stress decreases and ignoring data that shows stress increases or no change in stress.

Internal Validity Threats

2025-09-04 16:48:24 UTC

Making sure a cause and effect relationship is an actual cause and effect relationship. It can’t be explained by any other external factors. For example, in a mindfulness study the time of day, outside practice, or other variables may influence results.

Attrition

2025-09-04 16:48:27 UTC

Population reductions due to a number of voluntary (choice), involuntary (expulsion), natural (death, health, relocation) and other reasons. This compromises internal validity in research because a cause and effect cannot be determined with just attrition rates. For example, if a mindfulness course has a high attrition rate, this doesn’t necessarily mean that mindfulness was not an effective intervention, half the students could have contracted the plague. More information is needed regarding the reasoning for attrition for it to be a valuable measure.

Ecological Validity

2025-09-04 16:48:33 UTC

Running Definition: extent to which a research study’s findings can be generalized (or are realistic) to real world settings/scenarios

Low Ecological Validity could look like: lab asks participants to practice mindfulness by starting at a dot on a screen, while connected to wires and being in a lab setting

High Ecological Validity: practicing mindfulness at home for a specified amount of time using a shared video/app (then recording how you feel before and after)

Interaction effects

2025-09-04 16:48:37 UTC

When the effect of the thing you change on the results changes based on the level of the thing you change. For example, different environments can change how quickly you enter a mindfulness state. Personally, I practice mindfulness better in the classroom than at home as I am used to practicing it in the classroom.

Selection Bias

2025-09-04 16:48:52 UTC

Definition: Type of error that happens when the people or things chosen for a study are not representative of the larger group being studied. This could lead to inaccurate results.

Example: If a survey about exercise habits only includes gym members, it may overestimate how much the average person exercises.

Maturation

2025-09-04 16:48:54 UTC

The natural change or growth of study participants over time. Can be especially impactful in longitudinal studies, making it difficult to determine whether the data is a result of what is being tested or a result of natural growth.

Example: following a class in high school from freshman year to graduation to see how well the implementation of mindfulness practices reduces stress, you would be unable to accurately claim that any stress reduction was a result of the mindfulness practices rather than development of better study/time management habits.

Selection Bias

2025-09-04 16:48:56 UTC

The error that occurs when a study’s sample population doesn’t accurately reflect the target population. For example if a person is trying to measure the impact of different types of mindfulness and they set up a table with a QR code outside of a gym, they may get skewed results that don’t reflect the population in the working out category.

Testing Effects - 6

2025-09-04 16:49:11 UTC

Definition: someone being tested understands or has previous experience with a type of testing. This will change how they answer creating a bias which will be inaccurate and can cause deviations in recorded results.

Example: someone who has a friend in a research lab studying mindfulness. If the friend in the lab (person A) ends of testing their friend (person B) their friend understanding the ideal answers for the labs research may change their answer to help their friend by answering with responses that are not honest with the intention of supporting their friends “ideal” outcomes.

Range Restriction

2025-09-04 16:49:27 UTC

When you place limitations or constraints on your data whether it’s intentionally or unintentionally which causes the narrowing of the range of data you take in. Which can lead to a reduced correlation between the different variables. For mindfulness like you want to see how effective mindfulness is for the general population and you only take data from people who already use it daily or have a large amount of knowledge of mindfulness then when you make your conclusions about the general population they can be inaccurate due to not having data from the whole population.

Temporal validity

2025-09-04 16:49:53 UTC

Refers to how a study's findings can be generalized over different time periods. For the purposes of mindfulness, it entails staying alert in the present moment without being judgemental or critical. For example, a person analyzing the responses of people under stressful situations for a study needs to remain present in the moment to document people's reactions without criticizing the individuals. This is important so that research outcomes are not compromised by the researcher's judgement or bias towards unexpected outcomes.

1. Maturation

2025-09-04 16:50:10 UTC

Usually an issue for longitudinal studies, because people aren't stagnant. As someone ages, they may have more life experience and have a stronger affinity to have work-life balance or meditative abilities.

Statistics conclusion validity threat

2025-09-04 16:52:00 UTC

The stats conclusion from the study is not the same as real life effects.

Happiness and breathing techniques. In real life there is an effect. But If i get neurotic participants for my study only, my statistics are gonna not show a real significant effect even though Real life there is an effect. Also happens the other way, where something with no real effect in real life like mindfulness and dice roll result. If your statistics are lucky you might have a statistical effect that show a real life effect even though Real life effect doesn't exist.

The threat that seemed most relevant to mindfulness measurement is my one: just because it's my topic.

Which were the hardest to explain or understand: Nil all will be treated equally

These threats will happen in research one way or the other. Training, ethics, and fraud detection are the only way to prevent this.

2025-09-04 16:55:32 UTC

I think one of the threats that seemed most relevant was high and low ecological validity. Practicing mindfulness is a skill that works best when practiced in many different spaces and with different pre existing mindsets. I also think range restrictions/selection bias aré important, as cherry picking data or participants to make mindfulness look better is not good clinical practice.

I thought that while attrition is very simple, as a measurement itself it is probably difficult to report/discuss as there are so many variables and things that can lead to it. For example, when researching grad programs, I found that LSU has over a 70% rate but I will never know why.

I think that all of these factors will overlap due to the unpredictability of research.