Major changes in the HUMANE checklist
| Section | Google Forms v1.08 sections, 56 questions | REDCap v2.08 sections, 50 questions | |||
|---|---|---|---|---|---|
| No. of questions | Content | Answer format | No. of questions | Changes made | |
| Title | 2 | The basic structure of the title | 5-point Likert scale (strongly disagree to strongly agree) | 2 | The answer format changed from a numerical scale to Yes/No |
| Abstract | 1 | Essential components of abstract | 5-point Likert scale (strongly disagree to strongly agree) | 1 | The answer format changed from a numerical scale to Agree/Partially agree/Disagree |
| Introduction | 5 | Various aspects of background and introduction with a focus on rationale, objectives, knowledge gap, and potential impact | - | 6 | The first question expanded to elaborate on the rationale of the introduction with Yes/No answers |
| Methods 4a | 8 | Various aspects of data: data source, study design, timeline, data pre-curation steps and data categorization | 5-point Likert scale (strongly disagree to strongly agree) | 8 | The answer format of the question on the difference between training and validation datasets changed to Temporally/Geographically/Both/None. Answer format of other questions changed to Yes/No/Unclear/NA as appropriate |
| Methods 4b | 2 | Informed consent and inclusion-exclusion criteria as a part of the methods section | 5-point Likert scale (strongly disagree to strongly agree) | 2 | Answer format changed to Yes/No/NA |
| Methods 4c | 5 | AI model outcomes | 5-point Likert scale (strongly disagree to strongly agree) | 5 | The answer format changed to Yes/No/NA. Question on models using triage or diagnostic pathways adjusted to include options for “intended role” and “diagnostic elements”, respectively. Redundant questions on the knowledge gap and overfitting of AI models were removed |
| Methods 4d | 3 | Statistical analysis methods | 5-point Likert scale (strongly disagree to strongly agree) | 4 | The answer format changed to Yes/No/NA. Question on overfitting protocol rephrased |
| Section 5a (Ground Truth) | 5 | Ground truths applied in AI model development | 5-point Likert scale (strongly disagree to strongly agree) | 6 | The answer format changed to Yes/No/NA. Answer options for “prospective” and “retrospective” were added where appropriate. Branching logic of Yes/No question on ground truth supervised learning model was added |
| Section 5b (Expert(s) Review) | 5 | Experts’ role in reviewing ground truth labels | Numerical scale | 1 | Answer options changed to reflect the nature of expert roles with Unclear/NA options added where appropriate. One representative question is kept in the final checklist |
| Results | 14 | Comprehensive cover of reporting of results | - | 9 | Added free text boxes to elaborate if selected “other” option for calibration and performance metrics. Some questions changed to allow more than one response. Question addressing algorithmic bias added. Removed 6 questions due to not being relevant to the results section, including comparison with past literature, whether the validation dataset was distinct, evaluating model fairness, providing differential diagnoses and confidence estimates, reporting values of the measured variable, and reiterating the purpose of AI technology (removed mode of fairness, diagnostic cues, diagnostic distinct, differential diagnosis, values of the measured variable) |
| Discussion | 5 | Study summary, strengths and weaknesses of study, conclusion | 5-point Likert scale (strongly disagree to strongly agree) | 5 | Rephrased for conciseness. Answer format changed to Yes/No/NA |
| Other and conflict of interest | 1 | Free text boxes for feedback | - | 1 | Only one free text box retained |
| Collaborative author details | - | Details of survey respondents | - | - | - |
-: No data. AI: artificial intelligence
