| Workflow integration | Understand the technical, cognitive, social, and political factors in play and incentives impacting integration of AI into health care workflows. |
| Enhanced explainability and interpretability | To promote integration of AI into health care workflows, consider what needs to be explained and approaches for ensuring understanding by all members of the health care team. |
| Workforce education | Promote educational programs to inform clinicians about AI/machine learning approaches and to develop an adequate workforce. |
| Oversight and regulation | Consider the appropriate regulatory mechanism for AI/machine learning and approaches for evaluating algorithms and their impact. |
| Problem identification and prioritization | Catalog the different areas of health care and public health where AI/machine learning could make a difference, focusing on intervention-driven AI. |
| Clinician and patient engagement | Understand the appropriate approaches for involving consumers and clinicians in AI/machine learning prioritization, development, and integration, and the potential impact of AI/machine learning algorithms on the patient-provider relationship. |
| Data quality and access | Promoting data quality, access, and sharing, as well as the use of both structured and unstructured data and the integration of non-clinical data is critical to developing effective AI tools. |
Source: Matheny ME, Thadaney Israni S, Ahmed M, Whicher D. AI in Health Care: The Hope, the Hype, the Promise, the Peril. Washington, DC: National Academy of Medicine; 2019. https://nam.edu/artificial-intelligence-special-publication
To understand the limitations of AI-based models in health care and the responsibilities of manufacturers and users of AI
software as a medical device (SaMD), an MI-CLAIM checklist was proposed for use in AI software development [37]. Its purpose is to enable a direct assessment of clinical impact, including considerations of fairness and bias, and to allow rapid replication of the technical design by any legitimate clinical AI
study. The MI-CLAIM checklist has six parts (Table 1.3), including (1) Study design; (2) Separation of data into partitions for model training and model testing; (3) Optimization and final model selection; (4) Performance evaluation; (5) Model examination; and (6) Reproducible pipeline. The CONSORT-AI and SPIRIT-AI working groups have proposed reporting guidelines for clinical trials of interventions involving AI. A summary of these
guidelines is presented in Table 1.4.
Table 1.3
The
MI-CLAIM checklist [Source: Norgeot B, Quer G, Beaulieu-Jones BK, Torkamani A, Dias R, Gianfrancesco M, Arnaout R, Kohane IS, Saria S, Topol E, Obermeyer Z, Yu B, Butte AJ. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nat Med. 2020 Sep;26(9):1320–1324]
| Before paper submission | ||
|---|---|---|
| Study design (Part 1) | Completed: page number | Notes if not completed |
| The clinical problem in which the model will be employed is clearly detailed in the paper. | □ | |
| The research question is clearly stated. | □ | |
| The characteristics of the cohorts (training and test sets) are detailed in the text. | □ | |
| The cohorts (training and test sets) are shown to be representative of real-world clinical settings. | □ | |
| The state-of-the-art |
solution used as a baseline for comparison has been identified and detailed. | □ | |
|
Data and optimization (Parts 2, 3)
|
Completed: page number
|
Notes if not completed
| |
The origin of the data is described and the original format is detailed in the paper.
| □ | | |
Transformations of the data before it is applied to the proposed model are described.
| □ | | |
The independence between training and test sets has been proven in the paper.
| □ | | |
Details on the models that were evaluated and the code developed to select the best model are provided.
| □ | | |
Is the input data type structured or unstructured?
| □ | | |
Model performance (Part 4)
|
Completed: page number
|
Notes if not completed
| |
The primary metric selected to evaluate algorithm performance (e.g., AUC, F-score, etc.), including the justification for selection, has been clearly stated.
| □ | | | The primary metric
selected to evaluate the clinical utility of the model (e.g., PPV, NNT, etc.), including the justification for selection, has been clearly stated. | □ | |
|
The performance comparison between baseline and proposed model is presented with the appropriate statistical significance.
| □ | | |
Model examination (Part 5)
|
Completed: page number
|
Notes if not completed
| |
Examination technique 1a
| □ | | |
Examination technique 2a
| □ | | |
A discussion of the relevance of the examination results with respect to model/algorithm performance is presented.
| □ | | |
A discussion of the feasibility and significance of model interpretability at the case level if examination methods are uninterpretable is presented.
| □ | | |
A discussion of the reliability and robustness of the model as the underlying data distribution shifts is included.
| □ | | |
Reproducibility (Part 6): choose appropriate tier of transparency
| |
Notes
| |
Tier 1: complete sharing of the code
| □ | | |
Tier 2: allow a third party to evaluate the code for accuracy/fairness; share the results of this evaluation
| □ | | |
Tier 3: release of a virtual machine (binary) for running the code on new data without sharing its details
| □ | | |
Tier 4: no sharing
| □ | |
PPV positive predictive value, NNT numbers needed to treat
aCommon examination approaches based on study type: for studies involving exclusively structured data, coefficients and sensitivity analysis are often appropriate; for studies involving unstructured data in the domains of image analysis or natural language processing, saliency maps (or equivalents) and sensitivity analyses are often appropriate
Table 1.4
Major topics of CONSORT-AI
extension
| 1. State the inclusion and exclusion criteria at the level of participants 2. State the inclusion and exclusion criteria at the level of the input data 3. Describe how the AI intervention was integrated into the trial setting, including any onsite or offsite requirements 4. State which version of the AI algorithm was used 5. Describe how the input data were acquired and selected for the AI intervention 6. Describe how poor-quality or unavailable input data were assessed and handled 7. Specify whether there was human—AI interaction in the handling of the input data, and what level of expertise was required for users 8. Specify the output of the AI intervention 9. Explain how the AI intervention’s outputs contributed to decision-making or other elements of clinical practice 10. Describe results of any analysis of performance errors and how errors were identified, where available. If no such analysis was planned or done, explain why not. |
Source: Adapted from Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK; SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Lancet Digit Health. 2020 Oct;2(10):e537–e548