# data[].attributes.body
Cannabinoids for Medical Use: A Systematic Review and Meta-analysis

# data[].attributes.headline
5. An article reporting narrative research

# data[].attributes.subject
5. An article reporting narrative research

# data[].attributes.body
Narrative research in climate change adaptation—Exploring a complementary paradigm for research and governance

# data[].attributes.headline
4. An article reporting an experiment or a quasi-experiment

# data[].attributes.subject
4. An article reporting an experiment or a quasi-experiment

# data[].attributes.body
MIR status report: an experiment for the measurement of the dynamical Casimir effect

# data[].attributes.headline
3. An article which reports a case study

# data[].attributes.subject
3. An article which reports a case study

# data[].attributes.body
Jian Shan Village: Case-Study Research and Evaluation of China’s “New Socialist Villages”

# data[].attributes.headline
2. An article which describes the aim of the research reported as to ‘explain’

# data[].attributes.subject
2. An article which describes the aim of the research reported as to ‘explain’

# data[].attributes.body
Common SNPs explain a large proportion of the heritability for human height

# data[].attributes.headline
1.An article which describes the aim of the research reported as to ‘explore’

# data[].attributes.subject
1.An article which describes the aim of the research reported as to ‘explore’

# data[].attributes.body
The purpose of this book is to teach children to explore Europe.

# data[].attributes.attachment_link.title
Let's explore Europe! : a booklet and educational online game for children aged 9-12 : teachers' guide.

# data[].attributes.headline
13. An article which makes a statement about the position of the researcher

# data[].attributes.subject
13. An article which makes a statement about the position of the researcher

# data[].attributes.body
Now I see it, now I don’t: researcher’s position and reflexivity in qualitative research

# data[].attributes.attachment_link.readable_text
This article addresses potential effects on reflexivity of researcher's social position (e.g. gender, age, race, immigration status, sexual orientation), personal experiences, and political and professional beliefs. Because reflexivity is a major strategy for quality control in qualitative research, understanding how it may be impacted by the characteristics and experiences of the researcher is of paramount importance. Benefits and challenges to reflexivity under three types of researcher's position are discussed and illustrated by means of case examples: (1) reflexivity when researcher shares the experience of study participants, (2) reflexivity when researcher moves from the position of an outsider to the position of an insider in the course of the study, and (3) reflexivity when researcher has no personal familiarity or experience with what is being studied. Strategies are offered for harvesting the benefits of researcher's familiarity with the subject and for curbing its potentially negative effects. Directions for future research are suggested.

References

De Tona, C (2006) But what is interesting is the story of why and how migration happened. Forum: Qualitative Social Research 7: Article 13. Available at: http://www.qualitative-research.net/fqs-texte/3-06/06-3-13-e.htm ().
Google Scholar

Eppley, K (2006) Defying insider-outsider categorization: one researcher's fluid and complicated positioning on the insider-outsider continuum. Forum: Qualitative Social Research 7(3): Article 16. Available at: www.qualitative-research.net/fqs-texte/3-06/06-3-16-e.htm ().
Google Scholar

Frisina, A (2006) Back-talk focus groups as a follow-up tool in qualitative migration research: the missing link? Forum: Qualitative Social Research 7: 1-8.
Google Scholar

Kacen, L, Chaitin, J (2006) The times are a changing: understanding qualitative research in ambiguous, conflictual and changing contexts. Qualitative Report 11: 209-228.
Google Scholar

Kosygina, LV (2005) Doing gender in research: reflection on experience in field. Qualitative Report 10(1): 87-95.
Google Scholar

Richie, B (1995) Racism, ethnic stigma and gender violence: exploring the intersections of oppression. In: Plenary, Midwest conference on child abuse and incest, University of Wisconsin, Madison, WI, .
Google Scholar

Russel, GM, Kelly, NH (2002) Research as interacting dialogue process: implications for reflexivity. Forum: Qualitative Social Research 3. Available at: http://www.qualitative-research.net/index.php/fqs/article/view/831/1806 ().
Google Scholar

Wint, E (2011) Reflexivity in practice: developing a new attitude as part of teaching and engaging in participatory research and development. Journal of Progressive Human Services 22(1): 68-83.
Google Scholar

# data[].attributes.attachment_link.title
Now I see it, now I don't: researcher's position and reflexivity in qualitative research - Roni Berger, 2015

# data[].attributes.attachment_link.description
This article addresses potential effects on reflexivity of researcher's social position (e.g. gender, age, race, immigration status, sexual orientation), personal experiences, and political and professional beliefs. Because reflexivity is a major strategy for quality control in qualitative research, understanding how it may be impacted by the characteristics and experiences of the researcher is of paramount importance.

# data[].attributes.headline
14. An article which uses thematic analysis

# data[].attributes.subject
14. An article which uses thematic analysis

# data[].attributes.body
The thematic and syntactic analysis of yu 于 constructions in Archaic Chinese

# data[].attributes.headline
15. An article which reports descriptive statistics (e.g. mean and standard deviation)

# data[].attributes.subject
15. An article which reports descriptive statistics (e.g. mean and standard deviation)

# data[].attributes.body
Detecting Ontology Mappings via Descriptive Statistical Methods

# data[].attributes.headline
16. An article which discusses the generalisability of the findings

# data[].attributes.subject
16. An article which discusses the generalisability of the findings

# data[].attributes.body
Generalisability of a Virtual Trials Method for Glycaemic Control in Intensive Care

# data[].attributes.headline
17. An article which uses purposive or opportunistic sampling

# data[].attributes.subject
17. An article which uses purposive or opportunistic sampling

# data[].attributes.body
Improving land-cover classification accuracy with a patch-based convolutional neural network: data augmentation and purposive sampling

# data[].attributes.attachment_link.title
Improving land-cover classification accuracy with a patch-based convolutional neural network: data augmentation and purposive sampling - IEEE Conference Publication

# data[].attributes.attachment_link.description
IEEE Xplore, delivering full text access to the world's highest quality technical literature in engineering and technology. | IEEE Xplore

# data[].attributes.headline
18. An article which discusses the limitations of a small scale study

# data[].attributes.subject
18. An article which discusses the limitations of a small scale study

# data[].attributes.body
Small studies: strengths and limitations

# data[].attributes.attachment_link.readable_text
A large number of clinical research studies are conducted, including audits of patient data, observational studies, clinical trials and those based on laboratory analyses. While small studies can be published over a short time-frame, there needs to be a balance between those that can be performed quickly and those that should be based on more subjects and hence may take several years to complete. The present article provides an overview of the main considerations associated with small studies.

HOW SMALL IS "SMALL"?

The definition of "small" depends on the main study objective. When simply describing the characteristics of a single group of subjects, for example the prevalence of smoking, the larger the study the more reliable the results. The main results should have 95% confidence intervals (CI), and the width of these depend directly on the sample size: large studies produce narrow intervals and, therefore, more precise results. A study of 20 subjects, for example, is likely to be too small for most investigations. For example, imagine that the proportion of smokers among a particular group of 20 individuals is 25%. The associated 95% CI is 9-49. This means that the true prevalence in these subjects generally is anywhere between a low or high value, which is not a useful result.

When comparing characteristics between two or more groups of subjects ( e.g. examining risk factors or treatments for disease), the size of the study depends on the magnitude of the expected effect size, which is usually quantified by a relative risk, odds ratio, absolute risk difference, hazard ratio, or difference between two means or medians. The smaller the true-effect size, the larger the study needs to be 1, 2. This is because it is more difficult to distinguish between a real effect and random variation. Consider mortality as the end-point in a trial comparing drug A and a placebo with 100 subjects per group. If the 1-yr death rate is 15% for drug A and 20% for the placebo, the risk difference is 5%, but this represents only five fewer deaths associated with drug A. It is not easy to determine whether this difference is due to the action of the new drug or simply chance. There could just happen to be five fewer deaths in one group. However, if the death rates were 5 versus 40%, this represents 35 fewer deaths among 100 subjects receiving drug A, which are unlikely to all be due to chance. Therefore, a trial of 100 patients per arm is too small if the expected difference is 5%, but large enough if the expected difference is 35%. Figure 1 ⇓ illustrates how study size influences the conclusions that can be made.

STRENGTHS

Studies with a small number of subjects can be quick to conduct with regard to enrolling patients, reviewing patient records, performing biochemical analyses or asking subjects to complete study questionnaires. Therefore, an obvious strength is that the research question can be addressed in a relatively short space of time. Furthermore, small studies often only need to be conducted over a few centres. Obtaining ethical and institutional approval is easier in small studies compared with large multicentre studies. This is particularly true for international studies.

It is often better to test a new research hypothesis in a small number of subjects first. This avoids spending too many resources, e.g. subjects, time and financial costs, on finding an association between a factor and a disorder when there really is no effect. However, if an association is found it is important to make it clear in the conclusions that it was from a hypothesis-generating study and a larger confirmatory study is needed.

Small studies can also make use of surrogate markers when examining associations, i.e. a factor that can be used instead of a true outcome measure, but it may not have an obvious impact that subjects are able to identify. For example, in lung cancer, the true end-point in a clinical trial of a new intervention is overall survival: time until death from any cause. "Death" is clearly clinically meaningful to patients and clinicians, thus if the intervention increases survival time this should provide sufficient justification to change practice. A surrogate marker is tumour response, i.e. complete or partial remission of the cancer. Surrogate end-points are often associated with more events, which are observed relatively soon after the intervention is administered; therefore, subjects may not require a long follow-up period. Both of these characteristics allow a smaller study to be conducted in a short space of time. Observing no change in the surrogate marker usually indicates there is unlikely to be an effect on the true end-point, thus avoiding an unnecessary large study.

LIMITATIONS

The main problem with small studies is interpretation of results, in particular confidence intervals and p-values (fig. 1 ⇑ ). When conducting a research study, the data is used to estimate the true effect using the observed estimate and 95% confidence interval. Consider hypothetical clinical trials evaluating four new diets for reducing body weight (table 1 ⇓ ). The results for diet A are clear: they are clinically important (the weight loss is large) and highly statistically significant (the p-value is very small, indicating that the observed weight loss of 7 kg is unlikely to be due to chance). The true mean weight loss associated with the new diet is estimated to be 7 kg, but there is 95% certainty that the true value lies somewhere between 6.4 and 7.6 kg. Ideally all intervals should be as narrow as this, but usually only large studies can produce such precise results. In diets B and D, the confidence intervals are also narrow, but all around a small and clinically unimportant effect so one can be fairly confident that these diets are not worthwhile. The statistically significant result for diet B is simply due to performing a very large study, but it would not justify using the new diet.

The most difficult results to interpret are those for diet C. Although the confidence interval includes zero, most of the range is below zero and the p-value is just above the conventional cut-off value of 0.05. This is likely to be due to the study not being large enough. The data must be interpreted carefully. The lack of statistical significance does not mean there is no effect 3, because the true mean weight loss could be 3 kg, or even as large as 6.3 kg. It is better to say "there is some evidence of an effect, but the result has just missed statistical significance", or "there is a suggestion of an effect". There needs to be a careful balance between not dismissing outright what could be a real effect and also not making undue claims about the effect.

Another major limitation of small studies is that they can produce false-positive results, or they over-estimate the magnitude of an association. Table 2 ⇓ illustrates this limitation using trials that have evaluated thalidomide in treating lung cancer 4, 5. After the smaller studies were reported, there was much hope for thalidomide, particularly because it is administered orally. However, the large trial did not show any benefit.

There are also limitations associated with the statistical analysis. When examining risk factors or other association, it is often necessary to allow for the effect of important prognostic factors (confounders). This is done using methods such as multivariate linear or logistic regression and Cox's regression (for survival data). However, when the number of observations is small and researchers attempt to adjust for several factors, these methods can fail to produce sensible results or they produce unreliable results.

CONCLUSION

There is nothing precise about a sample size estimate when designing studies. It provides an approximate size of the study. It does not matter if one set of assumptions yields 100 subjects but another gives 110 because this represents only an extra five subjects per group. What is more important is whether 100 or 200 subjects are needed. There is always some guesswork involved in specifying the assumptions for sample size, particularly when determining the effect size, which is often quite different from what is observed at the end of the study.

There is nothing wrong with conducting well-designed small studies; they just need to be interpreted carefully. While small studies can provide results quickly, they do not normally yield reliable or precise estimates. Therefore, it is important not to make strong conclusions about a risk factor or trial intervention, whether the results are positive or not. Instead, data from such studies should be used to design larger confirmatory studies. If the aim is to provide reliable evidence on a risk factor or new intervention, the study should be large enough to do so. The editorial board of the European Respiratory Journal often review very interesting studies but based on small sample sizes. While the board encourages the best use of such data, editors must take into account that small studies have their limitations.

Statement of interest

None declared.

View Abstract

# data[].attributes.attachment_link.title
Small studies: strengths and limitations

# data[].attributes.attachment_link.description
A large number of clinical research studies are conducted, including audits of patient data, observational studies, clinical trials and those based on laboratory analyses. While small studies can be published over a short time-frame, there needs to be a balance between those that can be performed quickly and those that should be based on more subjects and hence may take several years to complete.

# data[].attributes.headline
8. An article which mentions a hypothesis

# data[].attributes.subject
8. An article which mentions a hypothesis

# data[].attributes.body
Reimagining Alzheimer's Disease--An Age-Based Hypothesis

# data[].attributes.headline
9. An article which mentions a null hypothesis

# data[].attributes.subject
9. An article which mentions a null hypothesis

# data[].attributes.body
Neurology® null hypothesis: A special supplement for negative, inconclusive, or confirmatory studies

# data[].attributes.headline
10. An article which reports using interviews as the method of data collection

# data[].attributes.subject
10. An article which reports using interviews as the method of data collection

# data[].attributes.body
Comparison of text messaging data collection vs face-to-face interviews for public health surveys:a cluster randomized crossover study of care-seeking for childhood pneumonia and diarrhoea in rural China

# data[].attributes.headline
11. An article which reports using observation as the method of data collection

# data[].attributes.subject
11. An article which reports using observation as the method of data collection

# data[].attributes.body
Comparison of ‘think aloud’ and observation as data collection methods in the study of decision making regarding sedation in intensive care patients

# data[].attributes.attachment_link.readable_text
Abstract

Background

There is recognition that different data collection methods gather different aspects of decision making data. Although the selection of a method to explore nurses' decision making is partially determined by the theoretical perspective that informs each study, some flexibility remains. Description of the relative benefits of each method will enable future researchers to selectively identify which method is most suited to answering their specific research question.

Objectives

To describe the decisions identified using observation and think aloud in the study of decision making related to sedation assessment and management within intensive care, as well as to examine the strengths and weaknesses of each method in the context of this study.

Design

Secondary analysis of data collected during an observational study.

Settings

This study was conducted in one intensive care unit in a tertiary teaching hospital in Australia.

Participants

Seven self-identified expert critical care nurses.

Methods

Nurses providing sedation management for a critically ill patient were observed and asked to think aloud during 2 h of care, with follow-up interviews conducted up to 4 days later to clarify information collected. Data were analysed independently by an investigator not involved in data collection. Analysis involved identification of decision tasks with comparison of number and type of tasks identified with each of the two data collection techniques.

Results

Assessment and management were the most common types of sedation decisions made by nurses in this study. A total of 130 decisions were identified using observation and 209 decisions were identified using think aloud. More management decisions were identified through observation, while more assessment decisions were identified through think aloud.

Conclusions

The two data collection methods of think aloud and observation resulted in identification of different decision tasks. These results suggest an essential consideration in design of decision making studies is the method of data collection and the type of decision data that is likely to be identified. It may be appropriate to use a combination of data collection methods to optimise the completeness of data capture.

View full text

# data[].attributes.attachment_link.title
Comparison of 'think aloud' and observation as data collection methods in the study of decision making regarding sedation in intensive care patients

# data[].attributes.attachment_link.description
There is recognition that different data collection methods gather different aspects of decision making data. Although the selection of a method to explore nurses' decision making is partially determined by the theoretical perspective that informs each study, some flexibility remains.

# data[].attributes.headline
12. An article which reports using focus groups as the method of data collection

# data[].attributes.subject
12. An article which reports using focus groups as the method of data collection

# data[].attributes.body
Development of alternative methods of data collection in South Asians with Type 2 diabetes

# data[].attributes.attachment_link.readable_text
First published:

: Dr C. E. Lloyd, Faculty of Health and Social Care, The Open University, Walton Hall, Milton Keynes, MK7 6AA, UK. E‐mail: c.e.lloyd@open.ac.uk

Get access to the full version of this article. View access options below.

Abstract

Aims  Previous research in the UK has established the difficulty of recruiting and collecting information from individuals whose main language is spoken and does not have an agreed written form. The aims of this study were (i) to develop culturally competent translations of two questionnaires measuring diabetes self‐care in languages with no written form and to establish their face validity and (ii) to develop acceptable methods of delivery with the potential for providing valid and reliable data for use in research studies.

Methods  Adults with Type 2 diabetes from two minority ethnic groups whose main language is spoken (Sylheti and Mirpuri) were recruited via the Birmingham Heartlands hospital diabetes centre. Participants were invited to attend five focus groups to consider the content and method of delivery of two questionnaires measuring knowledge of diabetes and confidence in diabetes self‐care. Delivery methods were (i) pen and paper self‐completion in Bengali/Urdu, (ii) pen and paper assisted completion in spoken language, (iii) partially‐assisted completion in spoken language, (iv) independent audio delivery in spoken language.

Results  Culturally competent content was achieved for both questionnaires in both languages. The Mirpuri men and women's groups preferred assisted or partially assisted completion in spoken language. The Sylheti groups' preference was for independent audio delivery in spoken language.

Conclusions  The face validity of two questionnaires measuring diabetes self‐care is established for Sylheti and Mirpuri, in four alternative delivery formats. The questionnaires are now ready for psychometric testing in minority ethnic populations and the methods available for use by researchers to establish within‐study feasibility.

# data[].attributes.attachment_link.title
Development of alternative methods of data collection in South Asians with Type 2 diabetes

# data[].attributes.attachment_link.description
First published: : Dr C. E. Lloyd, Faculty of Health and Social Care, The Open University, Walton Hall, Milton Keynes, MK7 6AA, UK. E‐mail: c.e.lloyd@open.ac.uk Get access to the full version of this article. View access options below.