Development and Validation of a Semiautomated Tool for Measuring Periorbital Distances.

PURPOSE: To validate a custom FIJI (ImageJ) program for more reproducible, faster curvilinear periorbital measurements, as compared with 2 custom artificial intelligence-based tools.

DESIGN

Combined technical validation and method comparison study.

SUBJECTS: Front-facing photographs of 45 cleft palate syndromic patients.

METHODS

A FIJI (ImageJ)-based macro script tool, OrbitJ (semiautomated), was developed, requiring 15 user input steps to generate 38 measurements per photo. The user outlines the irises to set image scale and then selects points along the lid margins and brow lines. Linear interpolation and fourth-degree polynomial fit lines were used to generate periorbital measurements. This tool was compared against our previously developed deep learning algorithm for periorbital measurements, OrbitMap (automated), another open-source algorithm PeriOrbitAI (automated), and against manual measurements. Four human graders measured 45 photos once both manually and with OrbitJ. Intrarater and interrater measurements were performed with 10 photos manually, 3 in triplicate manually and 5 in triplicate with OrbitJ. Fourteen manual measurements were performed: time per image, iris diameter, margin reflex distance (MRD) 1 and 2, inferior scleral show (ISS), medial, central, and lateral brow height, canthal tilt, vertical dystopia, interpupillary distance, and inner and outer canthal distance (OCD).

MAIN OUTCOME MEASURES: The mean absolute error, reliability, bias, and Pearson correlation of periorbital measurements.

RESULTS

Analysis was successful in all 45 images for all methods except PeriOrbitAI, which failed on 6 images. For manual and semiautomated intra- and interrater measurements, reliability was considered moderate or better (intraclass correlation coefficient [ICC] >0.5) for all measurements excluding elapsed time. Manual interrater mean absolute error was <1 mm all measures except OCD. Reliability and correlation were high (ICC, Pearson correlation coefficient >0.8) between all OrbitJ and manual measurements. Comparing OrbitMap to manual measurements, ICC and Pearson correlation coefficient were >0.5 except for ISS, borderline for OCD (ICC = 0.51). Reliability between PeriOrbitAI and manual measurements was low (ICC <0.5) except for MRD2 and OCD, and correlation was moderate (Pearson correlation coefficient = 0.49-0.75). The mean analysis time per image was 13.4 ± 4 minutes (manual measurements), 5.4 ± 1.9 minutes (semiautomated OrbitJ), 10.71 ± 1.65 seconds (automated PeriOrbitAI), and 1.45 ± 0.15 seconds (automated OrbitMap) (P < 0.001).

CONCLUSIONS

Compared with manual measurements, all semiautomated OrbitJ measurements and most automated OrbitMap measurements were reliable. Notably, only 2 PeriOrbitAI measurements were reliable. All methods demonstrated significant time savings over manual measurements.

FINANCIAL DISCLOSURES: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.