Autonomous Artificial Intelligence Increases Screening and Follow-Up for Diabetic Retinopathy in Youth: The ACCESS Randomized Control Trial

Trial design

ACCESS is a hypothesis-driven, pre-registered, prospective parallel, RCT with a 1:1 allocation ratio that was conducted at the Johns Hopkins Pediatric Diabetes Center at two sites (Johns Hopkins Hospital and Mount Washington Pediatric Hospital) in Baltimore, Maryland, which serves a racially and ethnically diverse population. Participants were enrolled from November 24, 2021, through June 6, 2022, and follow-up was completed by Dec 6, 2022. The CONSORT requirements for RCT were followed⁵¹. ACCESS was pre-registered on ClinicalTrials.gov (NCT05131451). The study was approved by the Johns Hopkins IRB, the tenets of the Declaration of Helsinki were followed, and an independent Data Safety and Monitoring Board was established.

Participants

Youth with T1D (11-21 years) or T2D (8-21 years) were eligible for inclusion if they met the criteria for DED screening per American Diabetes Association (ADA) 2021 guidelines¹⁶, had no known DED, and had not had a diabetic eye exam within the last 6 months. Patients with maturity-onset diabetes of the young, cystic-fibrosis-related diabetes, known DED, or other pre-existing eye conditions (retinal disease, cataracts) were excluded from this study.

Interventions

Potential study participants were approached in the diabetes clinic to confirm eligibility and then recruited by a study coordinator with written informed consent. Consented participants were randomized to either the control arm or the intervention arm. At the time of enrollment, the study coordinator collected baseline data, as well as 3 phone numbers from the participant in order to facilitate follow-up.

Control arm: standard of care augmented with an educational intervention

In the control arm, participants were referred to an eye care provider (ECP: optometrist or ophthalmologist) through a deliberate educational process by the study coordinator, in the form of a scripted educational intervention, including a paper handout guide on how to get a diabetic eye exam [Supplementary Note 1]. The goal of this intervention was to minimize the effect of the most commonly reported barrier, i.e., communication and confusion around the necessity of the diabetic eye exam¹⁷. Diabetic eye exam completion was achieved with ECP eye exam documentation. If the participant could not be reached to determine completion (despite calls/voicemails to all 3 phone numbers and EHR-based secure messaging) or if the exam had not been completed by 6 months, it was considered not completed.

Intervention arm: autonomous AI

In the intervention arm participants underwent the 5–10 min autonomous AI system diabetic eye exam without pharmacologic dilation²⁴. The autonomous AI system (IDx-DR, Digital Diagnostics, Coralville, Iowa, USA) for diagnosing diabetic eye disease (DED) was US FDA De Novo authorized (“FDA approval”) in 2018 for adults with diabetes¹⁵. The system diagnoses specific levels of diabetic retinopathy and diabetic macular edema (Early Treatment of Diabetic Retinopathy Study level 35 and higher, clinically significant macular edema, and or center-involved macular edema)^32,33, referred to as “referable DED”³⁴, that requires further management or treatment by an ophthalmologist or retina specialist. If the ETDRS level is 20 or lower and no macular edema is present, appropriate management is to retest in 12 months³⁵. With this autonomous AI system, a medical diagnosis is made independently by the system without human oversight.

In this study, the participant’s eyes were not pharmacologically dilated, as pilot studies found that pharmacologic dilation is unnecessary in youth³⁰. The autonomous AI system guided the operator to acquire two color fundus images determined to be of adequate quality using an image quality algorithm³⁶, one each centered on the fovea and the optic nerve and guided the operator to retake any images of insufficient quality. This process requires approximately 10 min, after which the autonomous AI system reports one of the following within 60 s: “DED present, refer to a specialist”, “DED not present, test again in 12 months”, or “insufficient image quality”. The latter response occurs when the operator is unable to obtain images of adequate quality after 3 attempts.

If the autonomous AI output was “DED absent,” participants were informed the diabetic eye exam was normal, while if “DED present,” (ETDRS level 35 or higher, and/or clinically significant, and or/center-involved macular edema) they received a deliberate educational process by the study coordinator in the form of a scripted educational intervention for follow-up eye care [Supplementary Note 2]. In either case, the diabetic eye exam was considered complete. If the AI output was “insufficient quality” the participant was referred for eye care.

The IDx-DR autonomous AI is not labeled for youth <22 years, as currently no autonomous AI for DED has been cleared for a pediatric population. To ensure safety and that no cases of disease would be missed, all images were also overread by a board-certified retina specialist.

Follow-up procedures

Participants who completed the diabetic eye exam (intervention arm: after autonomous AI exam; control arm: after documented completion of eye exam) were asked to fill out a survey on acceptability and satisfaction with the screening method. [Supplementary Note 3] Participants received a $25 gift card and parking pass for participation in the study.

Randomization and masking

To prevent selection bias and ensure sample size balance between the groups and sites, stratified randomization (by site) was used, and participants were randomized in permutated block schedules of 4 and 6. Within each block, participants were randomized with a 1:1 allocation ratio to the control group and intervention group. This randomization sequence was created by a statistician unaffiliated with the study to ensure masking to the randomization scheme and was implemented by REDCAP’s randomization software based on the participant’s location^52,53. After consent, the research coordinator entered the participant location and the randomization allocation was generated. All parties were masked to the allocation until the participant was randomized in the study, and then all parties were unmasked.

Outcomes

In order to test the primary hypothesis, the pre-specified primary outcome was defined as the proportion of participants who completed a documented diabetic eye exam in each arm (“primary care gap closure rate”). In the control arm, this is the proportion of patients who completed a documented diabetic eye exam with an ECP within 6 months of randomization; in the intervention arm, this is the proportion of participants who completed the autonomous AI exam at the study visit.

The pre-specified secondary outcome (“follow-through completion rate”) was defined as the proportion of participants who completed follow-through at the ECP in each arm. In the control arm, this was the proportion who completed the diabetic eye exam at ECP after referral, and in the intervention arm, the proportion who completed follow-up at the ECP after a “DED-present.” This proportion assumes that control-arm patients who arrive at ECP for screening remain at ECP for management or treatment when found to have DED. Both outcomes were stratified by race, ethnicity, SES, and education level, using univariate and multivariate analysis to determine any differential effect on these categories. There were no changes in trial design or outcome after trial commencement.

Data collection

Data were collected from the electronic health record, specifically age, date of birth, sex at birth, race, ethnicity, type of diabetes, date of diabetes diagnosis, medication use (insulin, metformin, GLP1 agonist, etc.), form of insulin administration, use of continuous glucose monitor (CGM) and CGM data, blood pressure, height, weight, body mass index (BMI), presence of other diabetes-related complications (hyperlipidemia, hypertension, microalbuminuria), abnormal thyroid function, past four hemoglobin A1C readings (if available), diabetic eye exam history, medical history, family medical history, health insurance, and zip code. Parental education status and household income were self-reported by participants using a paper/pencil form.

Sample size calculation

We assumed that a 20% difference in DED screening completion rates (care gap closure) would be clinically relevant. Based on our prior study³⁰, where baseline screening rates before AI were 49%, we assumed that with the educational intervention, screening rates for usual care would be closer to 60% in this study, and demonstrating a difference of 20% would be clinically relevant for AI screening. We calculated that a sample size of 164 (n = 82, n = 82 AI) would provide 80% power with a 2-tailed type-1 error of 0.05. Since randomization and study visits occurred at the same time there was little risk of attrition and thus the sample size was not expanded to account for attrition.

Data governance

Although this was a low-risk clinical trial, an independent Data Safety and Monitoring Board was established to protect the interests of study participants and to preserve the integrity and credibility of the study data, based on pre-specified aims, thereby reducing any concerns that interim data could influence or bias the study results and interpretation. At the time of the DSMB meeting on 9/16/2022, all participants had already been enrolled in the trial, and the DSMB determined that there were no safety concerns and the study should continue to completion.

Statistical analysis

The primary outcome of care-gap closure between the randomization groups, and the secondary outcome of follow-through completion rate between the randomization groups were assessed by Pearson’s chi-squared tests. Characteristics of the completed vs non-completed participants in each arm were assessed by Pearson’s chi-squared tests, Wilcoxon rank-sum tests, and two sample t-tests, depending on the nature and distribution of each characteristic. A mixed multilevel multivariable logistic regression analysis, using the site as a nesting level to account for the clustering of observations, was performed in order to examine the relationship between demographic characteristics and the odds of having a previous diabetic eye exam amongst the entire study cohort, adjusting for known covariates associated with DED and site. All analyses were performed using Stata 15.1 (StataCorp, College Station, TX).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.