SPFT R&D Statistics: Handling missing data by Mutanda Daniel (Sussex Partnership Trust)

Missing data mechanisms

danielmutanda — 2022-01-14 11:25:40 UTC

To start, it is important to determine the missing data mechanism which is one of three explanations for a missing variable.

Missing Data Roadmap

danielmutanda — 2022-01-14 11:26:57 UTC

Research design can have a diverse range of methods to produce data, a common problem when it comes to analysing this data is encountering missingness and handling it in a way that is appropriate for the dataset and the researcher's expertise.

Missing Completely At Random (MCAR)

danielmutanda — 2022-01-14 11:31:56 UTC

When data is Missing Completely at Random (MCAR) it means there are no differences between participants with missing data and those with complete data.

Therefore, the reasons for the missingness are unrelated to the research questions that would be answered by analysing the data.

Consequently, when the dataset is MCAR, all observed data could be used in the analysis, such as with complete case analysis, and can produce unbiased inferences

Missing At Random (MAR)

danielmutanda — 2022-01-14 11:32:34 UTC

When data is MAR, missingness is related to the observed but not the unobserved data. Therefore, whether or not data is missing can be predicted by observed data (i.e with logistic regression). Accounting for variables that predict missingness can produce unbiased inferences

Missing Not At Random (MNAR)

danielmutanda — 2022-01-14 11:33:13 UTC

If the data is not MCAR or MAR, then we say it is Missing Not at Random (MNAR), this means the missingness of data is related to the unobserved data. If complete case analysis is biased and the causes of missing data are unmeasured variables, then this generally cannot be addressed in analysis and the inferences will likely be biased.

Example

danielmutanda — 2022-01-14 11:34:44 UTC

An example case for handling missing data, when examining the data set of a feasibility trial for group-based Behavioural Activation therapy, if we observe PHQ-9 as the primary outcome measuring depression (𝑋), GAD-7 as a secondary outcome that measures anxiety (𝑌), and group allocation (𝑍) as another variable that may be influencing the missing mechanism. For the data to be MCAR, it is necessary but not sufficient for neither GAD-7 nor group allocation to predict PHQ-9 missingness (R). In such a scenario, missingness may be due to an administrative error. Whereas for us to assume the data is MAR, it is necessary but not sufficient for PHQ-9 missingness to be predicted by GAD-7 and group allocation but not by the value of the PHQ-9 scores. In this scenario, being in a different group, such as an online group rather than in-person, predicts PHQ-9 missingness but not the value of the PHQ-9 score. In an MNAR scenario, group may predict PHQ-9 missingness, but the value of the PHQ-9 scores also predict its missingness, e.g. being more depressed leads to a lower likelihood of returning the PHQ-9 questionnaire.

Multiple Imputation

danielmutanda — 2022-01-14 11:54:46 UTC

Multiple imputation (MI) is a widely used general method for handling missingness in most types of MAR data. The general rule for using MI is the data missingness is above 5% but below 40-50%. MI could be used if other variables are also missing, but the mechanism for missingness must not be MCAR or MNAR (Jakobsen et al., 2017).

In many methods when a dataset has a dependent variable with missing values, it is important to identify auxiliary variables which enhance the estimation of the main outcomes variables.

Full Information Maximum Likelihood

danielmutanda — 2022-01-14 12:54:21 UTC

Full information maximum likelihood (FIML) is a method for dealing with missing data that is generally simpler to implement than Multiple Imputation. It can produce unbiased estimates if the data is MCAR or MAR, and it works well when adding auxiliary variables even if missingness is above 50% (Dong & Peng, 2013). However, incorporating auxiliary variables into FIML is not as straightforward as it is with MI.

Auxiliary variables

danielmutanda — 2022-01-14 12:55:46 UTC

When the dataset has a dependent variable with missing values, it is important to identify auxiliary variables, which are not variables of primary interest but are used to improve sampling or to enhance estimation of the main outcomes variables (Lavrakas, 2008). If no auxiliary variables are identified, then a complete case analysis should be performed.

Bootstrapping

danielmutanda — 2022-01-14 13:00:56 UTC

Bootstrap is a versatile resampling technique that can be used to generate inferences such as standard errors and confidence intervals. A bootstrap treats our observed sample as if it were an assumed infinite population and it mimics the process of randomly sampling from the population.

Multiple Imputation with Auxiliary variables

danielmutanda — 2022-01-14 14:17:32 UTC

When handling missing data under the assumption of MAR, Collins, Schafer, & Kam (2001) recommend that researchers should include many auxiliary variables in their model. This is because if more auxiliary variables are included, it is less likely that the variables causing missingness and the analytic variables with missing data will be excluded. If this exclusion were to occur, it would go against the conditional independence assumption posited in MAR (conditional independence between R and unobserved Y) and induce bias (Collins, Schafer, & Kam,2001).

However, the inclusion of an auxiliary variable can induce MNAR if the missing dependant variable and an unobserved variable are initially independent of each other but become dependent within strata of an added auxiliary variable (Thoemmes, & Rose, 2014).

Therefore, it is recommended that variables that only predict missingness should not be used as auxiliary variables, this is because if a variable predicts missingness but not the variables being imputed, including it in the imputation model may not affect the imputation (Jakobsen et al., 2017; Carpenter and Smuk, 2021).

Moreover, rather than selecting auxiliary variables based on statistical evidence when data are already collected, it is recommended that when researchers design studies they should purposely select covariates that might explain underlying mechanisms which might cause missingness (Thoemmes, & Rose, 2014).

Can you ignore missing data or is missingness below 5%?

danielmutanda — 2022-01-21 16:31:59 UTC

Yes

danielmutanda — 2022-01-21 16:33:04 UTC

Yes

danielmutanda — 2022-01-21 16:33:14 UTC

Yes

danielmutanda — 2022-01-21 16:33:26 UTC

No

danielmutanda — 2022-01-21 16:33:35 UTC

No

danielmutanda — 2022-01-21 16:33:48 UTC

No

danielmutanda — 2022-01-21 16:33:56 UTC

Is missingness substantial (over 50%)?

danielmutanda — 2022-01-21 16:39:12 UTC

Complete Case Analysis (unbiased)

danielmutanda — 2022-01-21 16:49:00 UTC

Complete case analysis only includes participants for which we have no missing data on the variables of interest (excluding those with missing data). Unbiased estimates can be produced if potential impact of the missing data is negligible (i.e. missingness is < 5%).

MCAR

danielmutanda — 2022-01-21 16:53:24 UTC

MAR

danielmutanda — 2022-01-21 16:53:55 UTC

MNAR

danielmutanda — 2022-01-21 16:54:05 UTC

Complete Case Analysis (potential bias)

danielmutanda — 2022-01-21 17:07:51 UTC

If auxiliary variables have been identified and there are large amounts of missingness on the primary outcome, then a complete case analysis may provide biased results (Garson, 2015). In such cases with large missingness, the results can be discussed to generate hypotheses but are not confirmative.

MAR

danielmutanda — 2022-01-24 10:35:06 UTC

Auxiliary variables identified?

danielmutanda — 2022-01-24 10:45:13 UTC

Links and resources

danielmutanda — 2022-01-24 11:54:24 UTC

When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts (biomedcentral.com)

Full Information Maximum Likelihood
Principled missing data methods for researchers (nih.gov)

Auxiliary variables and Bias
Auxiliary variables that predict missingness
HMBR_A_931799_O (jakewestfall.org)