Repeated holdout cross-validation of model to estimate risk of Lyme disease by landscape characteristics.

We previously modeled Lyme disease (LD) risk at the landscape scale; here we evaluate the model's overall goodness-of-fit using holdout validation. Landscapes were characterized within road-bounded analysis units (AU). Observed LD cases (obsLD) were ascertained per AU. Data were randomly subset 2,000 times. Of 514 AU, 411 (80%) were selected as a training dataset to develop parameter estimates used to predict observations in the remaining 103 (20%) AU, the validation subset. Predicted values were subtracted from obsLD to quantify accuracy across iterations. We calculated the percentage difference of over- and under-estimation to assess bias. Predictive ability was strong and similar across iterations and datasets; the exact number of obsLD cases per AU were predicted almost 60% of the time. However, the three highest obsLD AU were under-predicted. Our model appears to be accurate and relatively unbiased, however is conservative at high disease incidence.