Team 2: Chocolate Discussion by Abby Grace Drake

Instructions

bioee1780 — 2024-06-27 13:44:40 UTC

Be sure to read the paper "Chocolate with high cocoa content as a weight-loss accelerator" that is linked on Canvas.

Part of our job as scientists is to provide voluntary peer reviews of articles that have been submitted to journals to provide feedback on the scientific quality of the article. This week you will collaborate as a team of reviewers of the study on weight loss and chocolate.

Based on what you have learned about data analysis so far in the course can you suggest any ways to strengthen this paper? Is there any important information missing? What is the N? Are there other demographics to note? Was the statistical analysis done appropriately?

You do not need to wait until your team meeting to get started and you should return to the Padlet a couple of times through out the week to see what your teammates have said and to provide new comments.

I haven't provided question prompts this time. You should use text posts rather than videos this time unless you really want to use a video - go ahead. Photos illustrating your points are great too!

This assignment is worth 15 points. I'd like you each to make 3 posts at two different times this week (feel free to revisit more often) for 5pts each.

P-hacking

2024-07-10 19:03:14 UTC

There are many problems with this study that make its results quite doubtful. First, it uses a very small sample size (n=15) and doesn't really repeat the experiment to confirm their findings. Additionally, they don't really have a hypothesis going into the experiment, they just test for 18 different things and report the findings of only one of the categories (weight-loss). The chances of getting one significant result while testing for many things is actually quite high, which is why the researchers were able to publish something as a result of their study. This idea of tweaking experimental design in order to get significant findings is known as p-hacking.

-Mahilan

Bonferroni Correction

2024-07-13 15:20:06 UTC

Adding on to my previous post, I looked at the article again and didn't find any mention of a Bonferroni correction, or p-adj value taking into account the number of tests that were run for the experiment. Bonferroni's correction is meant to adjust the alpha value given the number of tests being run. This is because the more tests you run, the higher the likelihood of getting a significant result. This is exactly what the researchers in this study did. They tested for many difference biological changes and found one or two that came out to be significant. However, they didn't correct their alpha value and maintained it at a value of 0.05, even though it probably should have been much lower.

-Mahilan

Flawed Study

2024-07-14 20:36:10 UTC

Upon researching a little more about this study, I found out that the researchers who conducted it purposefully made it flawed in order to illustrate the media's tendency to publish and disseminate captivating stories without proper vetting. Additionally, the study also wasn't peer reviewed which only adds to its duplicity. The journal that the study was published in a part of "scientific journals" known as "predatory journals." These journals require that scientists pay high fees for little-to-no peer review in exchange for having their study published online. I found this to be quite interesting.

-Mahilan

Small Sample Size

2024-07-15 00:04:54 UTC

The sample size of this research is too small to represent a larger population, making it statistically weak for various reasons. First, having small sample size might increase the type 2 errors. If that’s the case, the research itself may turn useless since the true differences or relationships might be ignored. Second, with fewer N, random variation will play a more important role, which could lead to overestimations or underestimations. Third, with the lacking representativeness, the result of the study will lose the generalizability. -Suyeon

P-Hacking

2024-07-15 00:11:51 UTC

P hacking refers to a manipulation of statistical analysis to produce desired outcomes. In this paper, researchers made up a fake (or just possibly not true) relationship by continuously analyzing data in different ways until a significant result is found. To do so, they ran multiple different tests but selectively reported only a single result that showed significant outcome, ignoring the rest. This could be considered as a sort of a p hacking. -Suyeon

Backgrounds of Participants

2024-07-15 00:22:02 UTC

The paper does not specify the ages, genders, or backgrounds of the participants, leaving it unclear which participants are representing which group. If the study aimed for the participants to represent “people” in general, the sample size must have been bigger to prevent overfitting results. If the sample population shared a common feature, the study should have mentioned it in the paper to clarify that the findings are applicable to which groups or people. -Suyeon