Validation of a novel systematic social observation for assessing the social and built environment in neighborhoods surrounding community-based substance use programs.

BACKGROUND

Systematic social observation (SSO) quantifies changes in the social and built environment and can be useful for evaluating the neighborhood effects of substance use programs.

METHODS

We developed, implemented, and validated an SSO instrument in the Bronx, New York City, and Providence, Rhode Island. Research assistants assessed neighborhoods in the Bronx (4/19/2023-6/22/2023) and Providence (11/4/2024-12/20/2024) following a randomization schedule to minimize temporal biases. We used exploratory factor analysis (EFA) and Bayesian multi-level item response theory (IRT) models to derive social and built environment indices and externally validated the measures against arrest data.

RESULTS

EFA for the Bronx (n = 564) and Providence (n = 463) identified three factors assessed with the instrument: drug litter, people, and built environment. IRT-based social and built environment indices exhibited positive spatial autocorrelation (the Bronx Moran's I = 0.393, P < 0.001; Providence Moran's I = 0.068, P < 0.001). In the Bronx, item severity and discrimination were highest for the drug litter factor; item parameters were relatively uniform in Providence. The Bronx index was associated with total (incidence rate ratio [IRR] = 1.42, 95% confidence interval [CI] = 1.04-1.95) and drug-related (IRR = 1.60, 95% CI = 1.10-2.32) arrests. The Providence index was not associated with arrests.

CONCLUSIONS

Our SSO protocol can be replicated in diverse settings to evaluate the neighborhood effects of substance use programs and broader effects of neighborhood conditions on substance use. The social and built environment indices for the Bronx and Providence may be useful outcomes in future studies. External validation suggests the Bronx index captured drug-related environmental variation; the Providence index may capture variation unrelated to policing patterns.