For pitchers, the top models for predicting future injuries were random forest and top 3 ensemble, again based on AUC. The latter had a higher degree of accuracy

For predicting injuries in four body regions, the top 3 ensemble was the best model for all but the elbow region but the determinants of these injuries could not be calculated in pitchers due to the lower AUCs. This may be a result of limited data.

Machine learning models, particularly top 3 ensemble and random forest outperformed logistic regression in 13 of 14 cases.

As the questions are focused on predicting future outcomes based on historical data spanning multiple years, many players and sites of injury, machine learning is a relevant method as it offers a robust approach to mining large and diverse datasets to make predictions and identify important indicators.

The method seems to be appropriate as it combines the best of predictive power from the top-performing models and seems to provide a much higher level of accuracy than individuals models alone. However it may be complicated and mode time-consuming to apply to a dataset, and not entirely straightforward to understand exactly how it works (a practical demonstration would be useful).

Models are evaluated by assessing the Area Under the Receiver Operating Charactersitc (ROC) Curve (AUC), % accuracy, F1 score and Brier score loss.

Even though the top 3 ensemble approach provided a high level of accuracy and predictability overall, there were some injuries for which a different algorithm, such as random forest, was superior. There is no single algorithm that works for all data and categories of prediction.

The granularity of available data was limited, resulting in an inability to predict severity or nature of future injury in a particular body part (e.g. sprain versus tear).

It was also not possible to predict the impact of chronic injuries on future injuries.

There is also a lack of anatomic specificity to the prediction, which limits the clinical usefulness of the model.

The quality, accuracy and longitudinal coverage of the datasets used was limited, either because of uncertainty of reporting (under-reporting), not all time periods were covered, and only one database has been endorsed by Major League Baseball.

Ethical approval was not sought for the study. There is no indication whether the research team were able to identify individuals, took necessary steps to ensure the identities of individuals would be protected from disclosure in any way, nor sought consent from individuals whose data were being used.

The US has a sectoral approach to data privacy protection at the federal level, with additional legislation at state-level ergo how personal data are used and protected varies between different states.

 

For pitchers, the models with the highest AUC were random forest and the top 3 ensemble, both with a mean AUC 0.65. The top 3 ensemble model had the highest accuracy at 63.7%.

 

Back injuries had the highest AUC among both position players and pitchers, at 0.73. 

 

Machine learning is a relevant method as it is suitable for operating a big data source with complex data relationships, as well as for generating a predictive model for players’ injury prediction.

 

The advanced ML models are superior to LR, as advanced ML models, usually the top 3 ensemble and random forest, outperformed LR in terms of the AUC in 13 of the 14 cases. Specifically, the top 3 ensemble was the model with the highest AUC for predicting next season’s injury risk among position players and pitchers, but random forest was superior in predicting back injuries among pitchers. 

 

With more iterations, the ML algorithm continued to improve.  While regression analysis is static and not predictive, especially when more data inputs are added.

The aim of this study is to analyse performance data for baseball players and make predictions for future outcomes and prediciting future injuries  for players.    

 

2.      Findings of the study.

For position players 44% had previous injuries and found that hand and back were most common injuries. And 43.6% of pitchers had previous injuries and most common injuries were shoulder and elbow.  The models had accuracy of 0.71-0.8 for position players and 0.61-0.69 for pitchers for future injuries.   This reliability is reduced for pitchers due to limited data.  

3.      What are the research questions, and are they relevant for machine learning methods? You can give reasons.

The research question is predicting injury Logistic regression is used and regression is static and not predictive and is difficult to learn and predict.  

4.      Which machine learning algorithm was selected? Is it an appropriate method? You can give reasons. What are the advantages and disadvantages of this method?

Different algorithms were used were developed for seven different outputs.  A single model is not appropriate for answering all the clinical questions.  The models were evaluated for their accuracy.  There is different levels of accuracy for different player types (pitcher and positional).  For pitcher, random forest and top 3 ensemble had best accuracy and for position players the top 3 ensemble was most accurate.  

.

5.      Describe the application of each step in the machine learning workflow: data splitting, selection of predictors, model selection, training, optimisation, performance assessment. If these are not available, state so and look for the authors' reasons. 

Separate models were built for position player and pitchers.  Different models were build for each clinical outcome.  6 different models were created for each clinical outcome.  LR, random forest, k-nearest neighours, naïve bayes, xgboost and top 3 ensemble.  The best model was identified and chosen as final model.    They used python library and XGBoost. Each model used a 10 k-fold.  90 percent of data was used to train the model and 10 percent used to test it.  This step is repeated 10 times.  

6.      Study limitations

One limitation is the granularity of available data.  That is we were unable to analyse the level of detail.  For this it is not possible to ascertain the level of injury (sprain vs complete tear of a ligament).  There is limited anatomic specificity which limits how clinically useful it will be.  The data was unable to show the impact of chronic conditions.  The data used is a limitation as owned by private companies and 3 databases were not regulated.  Some data was not as specific as other data.

7.      Ethical and legal implications of the study

The ethics of predicting an injury can affect a players potential worth as it can create a quantifiable risk of injury to a player.  

 

Selection of predictors: age, performance data, injury history, and DL data from 17 seasons were predictive of next-season injuries.

Model selection: each model is evaluated by Accuracy, AUC, F1 score, Brier Score Loss, the highest AUC primarily determines the best performing model.

 

Training: Each model uses 10 k-folds to train 90% of the data and the remaining 10% is used to test the model. This step is repeated 10 times.

 

Optimisation: it is not available in this article.

Lack of anatomic specificity of the data prediction algorithm that limited immediate clinical utility of such a model.

It is not possible to capture the impact that chronic, lingering injuries may have on future injuries.

To determine the validity of an ML model in predicting next seasons injury risk and anatomic injury location for both position players and pitchers in the MLB.

Advanced ML models generally outperformed logistic regression and demonstrated fair capability in predicting publicly reportable next season injuries, including the anatomic region for position players, although not for pitchers. 

Advanced ML models outperformed logistic regression in 13 of 14 cases.

Can we characterise the epidemiology of injury trends on the disabled list (DL) from 2000 to 2017?

Yes suitable, unsupervised ML model.

Can we determine the validity of an ML model in predicting the injury risk for the subsequent year and anatomic injury location?

Yes suitable, supervised ML model.

Can we compare the performance of modern ML algorithms versus logistic regression analyses?

Not suitable for ML, just a comparison of performance.

Random forest was selected and top 3 ensemble (random forest, logistic regression & XGBoost), but others were tested.

a. Is it an appropriate method?

Note sure, it seems that different models or combinations of models had different accuracy for different parts of the analysis. The random forest model worked best for back injuries amongst pitchers. The top 3 ensemble had the best area under the curve (AUC) results for predicting next seasons injury risk amongst position players and pitchers.

b. What are the advantages and disadvantages?

Random forest - Black box – don’t know exactly how the outcome was calculated.  

Top 3 ensemble – potentially a very complicated ML model again lacking interpretability 

a. Data Splitting

The data was split 90% training / 10% test

b. Selection of Predictors

Age, performance data, professional injury history and DL list were categories within the data set. The importance of the variables were assessed and injury was found to be the most important.

c. Model Selection

Multiple models were chosen for assessment.

d. Training

The models were trained and evaluated using cross validation 10 fold.

e. Optimisation

Feature importance was calculated using the Gini importance metric.

f. Performance assessment

Area under the curve (AUC) was used to choose the machine learning model with the best accuracy.

Data bases are updated from multiple sources and could contain multiple errors. Databases were used that were not regulated by MLB. Couldn’t be specific about different injuries ie. Elbow sprain or complete tear

 

Data used without accuracy. Could affect the players earnings and contracts. 

Characterize the epidemiology of injury trends of an ML model in predicting the injury risk for the subsequent year and anatomic injury location.

Compare the performance of modern ML algorithms versus LR analyses.  

2.      Findings of the study.

The ML models outperformed logistic regression in 13 of 14 cases. The back injuries had the highest AUC among players. The AUC for predicting next-season injuries was 0.76 among position players and 0.65 among pitchers. 

3.      What are the research questions, and are they relevant for machine learning methods? You can give reasons.

Can the charactierization of the epidemiology of injury trends of MBL players be used in a ML model for predcting the injury risk for the next year and the anatomic injury location?

Can this ML model outperform the linear regression analyses?

These are relevant questions that try to find a predictive model, an objective of machine learning. The methodology also includes a vast number of data points, a characteristic that makes machinle learning models specially useful. 

4.      Which machine learning algorithm was selected? Is it an appropriate method? You can give reasons. What are the advantages and disadvantages of this method?

 Top 3 ensamble for position players. 

Top 3 ensemble and Random Forest for the pitchers.

Ensemble methods create different models and combine them to obtain improved results. The models are selected by “voting” (as is the case in this paper) or by “averaging”. While they can improve the predictive values , the interpretability also diminishes as it is a “black box”. 

6.      Study limitations

The data did not contained nuanced injury characteristics for a more accurate diagnosis.

Team-reported injuries are generally acute and severe, so the impact of chronic, lingering injuries could not be evaluated. 

Lack of anatomic specificity on the prediction algorithm. 

The databases that are privately owned  are not regulated by MLB. The one endorsed by the MLB started in 2015.

Less predictive value for the pitcher position compared with the other players, as the pitcher data is less specific in terms of predictive variables. 

7.      Ethical and legal implications of the study

It can affect the future employability and value of players identified as “at risk” of injury. Accessing more specific data could require the approval of the MLB Players Association. 

   - Compare the performance of modern ML algorithms versus LR analyses.

3.   Lack of anatomic specificity of the data prediction algorithm used.

4.   Sources of input of the databases used in obtaining the MLB player injury history and performance data is either prone to inaccuracies or not regulated

The main aims of this study were to apply machine learning methodologies to predict the likelihood of injuries in baseball players. Specifically, the authors aimed to develop a machine learning model to predict the risk and site of injury, and compare this to more established analytical models (namely logistic regression analysis). 

 

2.       Findings of the study.

In this study, multiple machine learning models were developed to predict distinct risks (e.g., risk of overall injury, risk of knee/back/ankle/etc. injury), including a combination of the best performing models. Looking at these models, the best predictor of future injury was previous injury. Generally, the combined models had the best accuracy as measured by area under the curve, Brier score loss, and F1 score. Compared to logistic regression analysis, machine learning models were generally better at predicting future injury (13/14 models out-performing LR); this was especially true when combining the best-performing models.

 

3.       What are the research questions, and are they relevant for machine learning methods? You can give reasons.

The research question in this study was to assess the applicability of machine learning methodologies to real-world baseball data to predict injury. This is relevant for machine learning methods. Developing machine learning models using real-world data to predict injury highlights the real-world application of machine learning methods; by focusing on injuries necessitating time off play, these models could theoretically lead to significant cost savings as well. The nature of baseball data (thanks to sabermetrics) meant a large pool of real-world data was available across a number of years, and this would be a suitable dataset on which to trial machine learning methods.

 

4.       Which machine learning algorithm was selected? Is it an appropriate method? You can give reasons. What are the advantages and disadvantages of this method?

The combination models (“top 3 ensemble”) using the best-performing machine learning models were selected for the final analysis. This was appropriate in terms of developing a highly accurate predictive model, as this had consistently good accuracy in predicting overall and site-specific injury compared to other models.

One main advantage of this method was its high accuracy while remaining similar in performance to other models. One disadvantage of this was the relative lack of transparency compared to other models, which could limit its reproducibility.

 

5.       Describe the application of each step in the machine learning workflow: data splitting, selection of predictors, model selection, training, optimisation, performance assessment. If these are not available, state so and look for the authors' reasons. 

The data were split into distinct 10% sections, with 9 of these used to train the model and the final 1 used to test the model. No predictors were specifically selected; instead, all available data were added to the model. The models were trained using a separate 10% of the data as outlined above using a 10 k-fold strategy. These models were calibrated by assessing their performance against one another, but were not clearly optimised. The performance of the models was assessed using accuracy, area under the ROC curve, F1 score, and Brier score loss. 

6.       Study limitations

This study had a few limitations. The “black box” of the top 3 ensemble model, for example, limits the reproducibility of the study’s findings and may similarly limit its applicability to other situations.  As well, as noted by the study authors, a single model may not be the best way to address the clinical questions – as seen in the study findings, there was variation in the ability of the models to predict specific anatomical sites. This may limit its applicability in a real-world setting, as not all injuries are similarly debilitating and may necessitate distinct interventions.

There are also the usual limitations which come with using big data sources, such as the ethical considerations of the data, the accuracy and reliability of the data, and the possibility of missing important qualitative information. Specifically relevant to this study, the lack of detailed information about the injuries within the data may have affected the interpretability of the findings. 

 

7.       Ethical and legal implications of the study

There are ethical considerations to consider when using data from individuals. Although the information is publicly available, consent was not explicitly taken from the players for their data to be used to form the models used in this study. The models used may also indirectly affect the future earnings of players based on their risk of future injury.