Two studies examined inter-rater reliability and content-related validity of an addiction treatment level of care determination tool currently in use in New York, the LOCADTR 3.0. The studies occurred after tool implementation. In study 1, 139 providers used the LOCADTR 3.0 to determine level of care for four case vignettes. Inter-rater reliability coefficients were calculated. In study 2, 387,338 state records from existing data were analyzed to determine how often providers opted to override the LOCADTR 3.0 level of care determination by choosing an alternative level of care. In study 1, an acceptable inter-rater reliability (IRR = .57-.59) was found. Good indication of content-related validity was also found; participants chose the same level of care the study team chose for each vignette 80% of the time. In study 2, the override option was selected only 10% of the time, further establishing the content validity of the tool. These studies provide evidence for acceptable preliminary reliability and validity of the LOCADTR 3.0.
Behav Health Serv Res. 2019 Jul. doi: 10.1007/s11414-018-9623-x.