Research report: Prediction models developed using artificial intelligence:
By Google News AI ·
Clinical prediction models developed using artificial intelligence and traditional regression techniques can achieve virtually identical overall discrimination while generating sharply conflicting risk estimates for individual patients, according to an extensive diagnostic evaluation published in Diagnostic and Prognostic Research.
Evaluating diagnostic data from thousands of individuals suspected of having deep vein thrombosis, an investigative team examined how different algorithmic frameworks translate standard clinical indicators into diagnostic probabilities.
The researchers demonstrated that crude summary metrics such as the area under the receiver operating characteristic curve frequently conceal profound discrepancies in patient-level probability outputs.
Even when multiple algorithms appear equally effective at separating positive cases from negative cases across a broader study cohort, the specific probability assigned to an individual patient fluctuates substantially depending entirely on the mathematical structure of the chosen predictive architecture.
These observed discrepancies indicate that selecting a prediction model based exclusively on aggregate discriminative performance may generate conflicting clinical recommendations at the bedside.In routine medical practice, clinicians regularly combine multiple demographic, historical, and physiological variables to evaluate an individual's probability of having an underlying condition or developing an outcome in the
future.
Clinical prediction models formalize this process by applying mathematical equations that link multiple predictive variables to estimated disease risks. Historically, medical researchers have relied on conventional regression approaches such as unpenalized multivariable logistic regression and Cox proportional hazards models to derive these probability estimates.
In recent years, however, machine learning algorithms and artificial intelligence methodology have experienced a notable surge in popularity across biomedical prognostics and diagnostics.
The widespread availability of automated, off-the-shelf software tools provides model developers with ample choices to deploy, compare, and select from a diverse spectrum of computational algorithms within identical clinical datasets.Previous investigations within low-dimensional settings—defined as analytical environments where the number of candidate predictors remains relatively small compared to the total volume
of observed individuals—have suggested that modern artificial intelligence algorithms can occasionally outperform classical regression techniques, particularly regarding discrimination measures such as the area under the receiver operating characteristic curve.
However, comparative studies repeatedly demonstrate that differences in discriminative performance between complex machine learning models and standard multivariable regression models are often small or practically negligible.
When developers observe that an unpenalized logistic regression model, a random forest ensemble, and an artificial neural network achieve nearly indistinguishable discriminative indices on a given clinical dataset, they frequently infer that the underlying models are essentially exchangeable.
To examine whether superficially equivalent models generate concordant clinical guidance, the researchers analyzed an individual participant dataset compiled from eight prospective diagnostic investigations involving patients…
Originally published by Google News AI
