A new analysis in Jama Network Open finds that diagnostic interviews, the standard method for identifying substance use and mental disorders such as depression, anxiety, bipolar disorder and personality disorders, show varying levels of consistency across conditions. Laura Duncan, a psychiatry professor at McMaster University in Ontario, Canada, and a study author, noted that these interviews are often regarded as the reference standard in both clinical practice and research yet fall short of offering a definitive measure of strong validity and consistency. Although prior evidence on their reliability has been inconsistent, they remain widely accepted as the best available option, possibly because superior alternatives are lacking, Duncan said. The review compiled findings on test-retest reliability from studies published between February 2024 and September 2025. Using Cohen’s kappa coefficient, the authors measured how often patients received identical diagnoses when re-interviewed, while adjusting for chance agreement. Average reliability proved higher for substance use disorders and peaked for opioid use disorder, which Duncan attributed to criteria based largely on observable behavior. Estimating weekly alcohol intake, for example, tends to be more straightforward than recalling days marked by sadness or anxiety. Dr Michael First, a Columbia University psychiatrist who developed the Structured Clinical Interview for DSM-5, expressed concern over parts of the analysis. He agreed that reliability differs and that interviews frequently miss correct diagnoses, but he sought clearer data on which specific tools performed best. The study drew on limited available research, covering instruments such as the SCID, the Mini International Neuropsychiatric Interview and the Clinically Administered PTSD Scale. First also questioned the decision to combine fully structured and semi-structured formats. Fully structured interviews produce more stable results because administrators must follow the script exactly, whereas semi-structured versions allow trained clinicians to ask clarifying questions. This flexibility can improve accuracy yet may increase variation between sessions. Duncan acknowledged the value of addressing these distinctions but noted that the necessary details were often missing or unreported in the included papers, underscoring the need for greater rigor in psychiatric assessment research. First observed that objective laboratory tests for mental conditions have been anticipated for decades. Duncan suggested shifting toward viewing symptoms along a continuum rather than as strictly present or absent categories.
