Scan-level acquisition dependence and validation bias in batch near-infrared hyperspectral imaging: a two-instrument study using background spectra.

Near-infrared hyperspectral imaging can record dozens of small objects per scan, but region-of-interest (ROI) spectra remain nested within one optical acquisition. This creates two related risks: acquisition conditions may covary with label order, and ROI-random partitioning may distribute a shared scan response across training, validation, and test sets. We examined these risks in two systems using six cauliflower varieties and four aging treatments. The 0, 2, 4, and 6 d treatments were acquired successively, whereas variety blocks were re-randomized within each period. Each system contributed 120 scans, yielding 6000 and 8640 single-seed spectra. Seed ROI, proximal background, and distal background spectra were used to predict variety, aging treatment, and 24 composite labels under five-fold validation grouped by complete scan. All accuracies are pooled out-of-fold overall accuracy (OA) values. Seed ROI variety OA was 90.7% and 88.7%, whereas distal background OA was 42.5% and 49.2%. Aging showed a different pattern: proximal and distal background OA reached 96.7% and 95.8% for GaiaSorter and 82.5% and 92.5% for Hyperspec III, exceeding seed ROI OA of 88.7% and 58.4%. Distal background spectra varied systematically across broad wavelength regions over the acquisition sequence. After composite-label means were removed, same-scan seed-background similarity exceeded within-label mismatches. With samples, model, and partitioning ratio fixed, ROI-random partitioning yielded 24-class OA values 8.3 and 7.3 percentage points higher than complete-scan grouping. Background spectra therefore revealed acquisition dependence, whereas complete-scan grouping reduced optimistic validation bias.