Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool
Importance: Test accuracy studies often use small datasets to simultaneously select an optimal cutoff score that maximizes test accuracy and generate accuracy estimates. Objective: To evaluate the degree to which using data-driven methods to simultaneously select an optimal Patient Health Questionna...
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Medical Association
2024
|
| Online Access: | http://psasir.upm.edu.my/id/eprint/116236/ http://psasir.upm.edu.my/id/eprint/116236/1/116236.pdf |
| _version_ | 1848866955946098688 |
|---|---|
| author | Levis, Brooke Bhandari, Parash Mani Neupane, Dipika Fan, Suiqiong Sun, Ying He, Chen Wu, Yin Krishnan, Ankur Negeri, Zelalem Imran, Mahrukh Rice, Danielle B. Riehm, Kira E. Azar, Marleine Levis, Alexander W. Boruff, Jill Cuijpers, Pim Gilbody, Simon Ioannidis, John P A Kloda, Lorie A. Patten, Scott B. Ziegelstein, Roy C. Harel, Daphna Takwoingi, Yemisi Markham, Sarah Alamri, Sultan H. Amtmann, Dagmar Arroll, Bruce Ayalon, Liat Baradaran, Hamid R. Beraldi, Anna Bernstein, Charles N. Bhana, Arvin Bombardier, Charles H. Buji, Ryna Imma Butterworth, Peter Carter, Gregory Chagas, Marcos H. Chan, Juliana C N Chan, Lai Fong Chibanda, Dixon Clover, Kerrie Conway, Aaron Conwell, Yeates Daray, Federico M. de Man-van Ginkel, Janneke M. Fann, Jesse R. Fischer, Felix H. Field, Sally Fisher, Jane R W Fung, Daniel S S Gelaye, Bizu Gholizadeh, Leila Goodyear-Smith, Felicity Green, Eric P. Greeno, Catherine G. Hall, Brian J. Hantsoo, Liisa Härter, Martin Hides, Leanne Hobfoll, Stevan E. Honikman, Simone Hyphantis, Thomas Inagaki, Masatoshi Iglesias-Gonzalez, Maria Jeon, Hong Jin Jetté, Nathalie Khamseh, Mohammad E. Kiely, Kim M. Kohrt, Brandon A. Kwan, Yunxin Lara, Maria Asunción Levin-Aspenson, Holly F. Liu, Shen-Ing Lotrakul, Manote Loureiro, Sonia R. Löwe, Bernd Luitel, Nagendra P. Lund, Crick Marrie, Ruth Ann Marsh, Laura Marx, Brian P. McGuire, Anthony Mohd Sidik, Sherina Munhoz, Tiago N. Muramatsu, Kumiko Nakku, Juliet E M Navarrete, Laura Osório, Flávia L Pence, Brian W. Persoons, Philippe Petersen, Inge Picardi, Angelo Pugh, Stephanie L. Quinn, Terence J. Rancans, Elmars Rathod, Sujit D. Reuter, Katrin Rooney, Alasdair G. Santos, Iná S Schram, Miranda T. Shaaban, Juwita Shinn, Eileen H. Sidebottom, Abbey Simning, Adam Spangenberg, Lena Stafford, Lesley Sung, Sharon C. Suzuki, Keiko Tan, Pei Lin Lynnette Taylor-Rowan, Martin Tran, Thach D. Turner, Alyna van der Feltz-Cornelis, Christina M. van Heyningen, Thandi Vöhringer, Paul A. Wagner, Lynne I. Wang, Jian Li Watson, David White, Jennifer Whooley, Mary A. Winkley, Kirsty Wynter, Karen Yamada, Mitsuhiko Zeng, Qing Zhi Zhang, Yuying Thombs, Brett D. Benedetti, Andrea |
| author_facet | Levis, Brooke Bhandari, Parash Mani Neupane, Dipika Fan, Suiqiong Sun, Ying He, Chen Wu, Yin Krishnan, Ankur Negeri, Zelalem Imran, Mahrukh Rice, Danielle B. Riehm, Kira E. Azar, Marleine Levis, Alexander W. Boruff, Jill Cuijpers, Pim Gilbody, Simon Ioannidis, John P A Kloda, Lorie A. Patten, Scott B. Ziegelstein, Roy C. Harel, Daphna Takwoingi, Yemisi Markham, Sarah Alamri, Sultan H. Amtmann, Dagmar Arroll, Bruce Ayalon, Liat Baradaran, Hamid R. Beraldi, Anna Bernstein, Charles N. Bhana, Arvin Bombardier, Charles H. Buji, Ryna Imma Butterworth, Peter Carter, Gregory Chagas, Marcos H. Chan, Juliana C N Chan, Lai Fong Chibanda, Dixon Clover, Kerrie Conway, Aaron Conwell, Yeates Daray, Federico M. de Man-van Ginkel, Janneke M. Fann, Jesse R. Fischer, Felix H. Field, Sally Fisher, Jane R W Fung, Daniel S S Gelaye, Bizu Gholizadeh, Leila Goodyear-Smith, Felicity Green, Eric P. Greeno, Catherine G. Hall, Brian J. Hantsoo, Liisa Härter, Martin Hides, Leanne Hobfoll, Stevan E. Honikman, Simone Hyphantis, Thomas Inagaki, Masatoshi Iglesias-Gonzalez, Maria Jeon, Hong Jin Jetté, Nathalie Khamseh, Mohammad E. Kiely, Kim M. Kohrt, Brandon A. Kwan, Yunxin Lara, Maria Asunción Levin-Aspenson, Holly F. Liu, Shen-Ing Lotrakul, Manote Loureiro, Sonia R. Löwe, Bernd Luitel, Nagendra P. Lund, Crick Marrie, Ruth Ann Marsh, Laura Marx, Brian P. McGuire, Anthony Mohd Sidik, Sherina Munhoz, Tiago N. Muramatsu, Kumiko Nakku, Juliet E M Navarrete, Laura Osório, Flávia L Pence, Brian W. Persoons, Philippe Petersen, Inge Picardi, Angelo Pugh, Stephanie L. Quinn, Terence J. Rancans, Elmars Rathod, Sujit D. Reuter, Katrin Rooney, Alasdair G. Santos, Iná S Schram, Miranda T. Shaaban, Juwita Shinn, Eileen H. Sidebottom, Abbey Simning, Adam Spangenberg, Lena Stafford, Lesley Sung, Sharon C. Suzuki, Keiko Tan, Pei Lin Lynnette Taylor-Rowan, Martin Tran, Thach D. Turner, Alyna van der Feltz-Cornelis, Christina M. van Heyningen, Thandi Vöhringer, Paul A. Wagner, Lynne I. Wang, Jian Li Watson, David White, Jennifer Whooley, Mary A. Winkley, Kirsty Wynter, Karen Yamada, Mitsuhiko Zeng, Qing Zhi Zhang, Yuying Thombs, Brett D. Benedetti, Andrea |
| author_sort | Levis, Brooke |
| building | UPM Institutional Repository |
| collection | Online Access |
| description | Importance: Test accuracy studies often use small datasets to simultaneously select an optimal cutoff score that maximizes test accuracy and generate accuracy estimates. Objective: To evaluate the degree to which using data-driven methods to simultaneously select an optimal Patient Health Questionnaire-9 (PHQ-9) cutoff score and estimate accuracy yields (1) optimal cutoff scores that differ from the population-level optimal cutoff score and (2) biased accuracy estimates. Design, Setting, and Participants: This study used cross-sectional data from an existing individual participant data meta-analysis (IPDMA) database on PHQ-9 screening accuracy to represent a hypothetical population. Studies in the IPDMA database compared participant PHQ-9 scores with a major depression classification. From the IPDMA population, 1000 studies of 100, 200, 500, and 1000 participants each were resampled. Main Outcomes and Measures: For the full IPDMA population and each simulated study, an optimal cutoff score was selected by maximizing the Youden index. Accuracy estimates for optimal cutoff scores in simulated studies were compared with accuracy in the full population. Results: The IPDMA database included 100 primary studies with 44 503 participants (4541 [10%] cases of major depression). The population-level optimal cutoff score was 8 or higher. Optimal cutoff scores in simulated studies ranged from 2 or higher to 21 or higher in samples of 100 participants and 5 or higher to 11 or higher in samples of 1000 participants. The percentage of simulated studies that identified the true optimal cutoff score of 8 or higher was 17% for samples of 100 participants and 33% for samples of 1000 participants. Compared with estimates for a cutoff score of 8 or higher in the population, sensitivity was overestimated by 6.4 (95% CI, 5.7-7.1) percentage points in samples of 100 participants, 4.9 (95% CI, 4.3-5.5) percentage points in samples of 200 participants, 2.2 (95% CI, 1.8-2.6) percentage points in samples of 500 participants, and 1.8 (95% CI, 1.5-2.1) percentage points in samples of 1000 participants. Specificity was within 1 percentage point across sample sizes. Conclusions and Relevance: This study of cross-sectional data found that optimal cutoff scores and accuracy estimates differed substantially from population values when data-driven methods were used to simultaneously identify an optimal cutoff score and estimate accuracy. Users of diagnostic accuracy evidence should evaluate studies of accuracy with caution and ensure that cutoff score recommendations are based on adequately powered research or well-conducted meta-analyses. |
| first_indexed | 2025-11-15T14:28:50Z |
| format | Article |
| id | upm-116236 |
| institution | Universiti Putra Malaysia |
| institution_category | Local University |
| language | English |
| last_indexed | 2025-11-15T14:28:50Z |
| publishDate | 2024 |
| publisher | American Medical Association |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | upm-1162362025-03-21T03:42:43Z http://psasir.upm.edu.my/id/eprint/116236/ Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool Levis, Brooke Bhandari, Parash Mani Neupane, Dipika Fan, Suiqiong Sun, Ying He, Chen Wu, Yin Krishnan, Ankur Negeri, Zelalem Imran, Mahrukh Rice, Danielle B. Riehm, Kira E. Azar, Marleine Levis, Alexander W. Boruff, Jill Cuijpers, Pim Gilbody, Simon Ioannidis, John P A Kloda, Lorie A. Patten, Scott B. Ziegelstein, Roy C. Harel, Daphna Takwoingi, Yemisi Markham, Sarah Alamri, Sultan H. Amtmann, Dagmar Arroll, Bruce Ayalon, Liat Baradaran, Hamid R. Beraldi, Anna Bernstein, Charles N. Bhana, Arvin Bombardier, Charles H. Buji, Ryna Imma Butterworth, Peter Carter, Gregory Chagas, Marcos H. Chan, Juliana C N Chan, Lai Fong Chibanda, Dixon Clover, Kerrie Conway, Aaron Conwell, Yeates Daray, Federico M. de Man-van Ginkel, Janneke M. Fann, Jesse R. Fischer, Felix H. Field, Sally Fisher, Jane R W Fung, Daniel S S Gelaye, Bizu Gholizadeh, Leila Goodyear-Smith, Felicity Green, Eric P. Greeno, Catherine G. Hall, Brian J. Hantsoo, Liisa Härter, Martin Hides, Leanne Hobfoll, Stevan E. Honikman, Simone Hyphantis, Thomas Inagaki, Masatoshi Iglesias-Gonzalez, Maria Jeon, Hong Jin Jetté, Nathalie Khamseh, Mohammad E. Kiely, Kim M. Kohrt, Brandon A. Kwan, Yunxin Lara, Maria Asunción Levin-Aspenson, Holly F. Liu, Shen-Ing Lotrakul, Manote Loureiro, Sonia R. Löwe, Bernd Luitel, Nagendra P. Lund, Crick Marrie, Ruth Ann Marsh, Laura Marx, Brian P. McGuire, Anthony Mohd Sidik, Sherina Munhoz, Tiago N. Muramatsu, Kumiko Nakku, Juliet E M Navarrete, Laura Osório, Flávia L Pence, Brian W. Persoons, Philippe Petersen, Inge Picardi, Angelo Pugh, Stephanie L. Quinn, Terence J. Rancans, Elmars Rathod, Sujit D. Reuter, Katrin Rooney, Alasdair G. Santos, Iná S Schram, Miranda T. Shaaban, Juwita Shinn, Eileen H. Sidebottom, Abbey Simning, Adam Spangenberg, Lena Stafford, Lesley Sung, Sharon C. Suzuki, Keiko Tan, Pei Lin Lynnette Taylor-Rowan, Martin Tran, Thach D. Turner, Alyna van der Feltz-Cornelis, Christina M. van Heyningen, Thandi Vöhringer, Paul A. Wagner, Lynne I. Wang, Jian Li Watson, David White, Jennifer Whooley, Mary A. Winkley, Kirsty Wynter, Karen Yamada, Mitsuhiko Zeng, Qing Zhi Zhang, Yuying Thombs, Brett D. Benedetti, Andrea Importance: Test accuracy studies often use small datasets to simultaneously select an optimal cutoff score that maximizes test accuracy and generate accuracy estimates. Objective: To evaluate the degree to which using data-driven methods to simultaneously select an optimal Patient Health Questionnaire-9 (PHQ-9) cutoff score and estimate accuracy yields (1) optimal cutoff scores that differ from the population-level optimal cutoff score and (2) biased accuracy estimates. Design, Setting, and Participants: This study used cross-sectional data from an existing individual participant data meta-analysis (IPDMA) database on PHQ-9 screening accuracy to represent a hypothetical population. Studies in the IPDMA database compared participant PHQ-9 scores with a major depression classification. From the IPDMA population, 1000 studies of 100, 200, 500, and 1000 participants each were resampled. Main Outcomes and Measures: For the full IPDMA population and each simulated study, an optimal cutoff score was selected by maximizing the Youden index. Accuracy estimates for optimal cutoff scores in simulated studies were compared with accuracy in the full population. Results: The IPDMA database included 100 primary studies with 44 503 participants (4541 [10%] cases of major depression). The population-level optimal cutoff score was 8 or higher. Optimal cutoff scores in simulated studies ranged from 2 or higher to 21 or higher in samples of 100 participants and 5 or higher to 11 or higher in samples of 1000 participants. The percentage of simulated studies that identified the true optimal cutoff score of 8 or higher was 17% for samples of 100 participants and 33% for samples of 1000 participants. Compared with estimates for a cutoff score of 8 or higher in the population, sensitivity was overestimated by 6.4 (95% CI, 5.7-7.1) percentage points in samples of 100 participants, 4.9 (95% CI, 4.3-5.5) percentage points in samples of 200 participants, 2.2 (95% CI, 1.8-2.6) percentage points in samples of 500 participants, and 1.8 (95% CI, 1.5-2.1) percentage points in samples of 1000 participants. Specificity was within 1 percentage point across sample sizes. Conclusions and Relevance: This study of cross-sectional data found that optimal cutoff scores and accuracy estimates differed substantially from population values when data-driven methods were used to simultaneously identify an optimal cutoff score and estimate accuracy. Users of diagnostic accuracy evidence should evaluate studies of accuracy with caution and ensure that cutoff score recommendations are based on adequately powered research or well-conducted meta-analyses. American Medical Association 2024 Article PeerReviewed text en cc_by_4 http://psasir.upm.edu.my/id/eprint/116236/1/116236.pdf Levis, Brooke and Bhandari, Parash Mani and Neupane, Dipika and Fan, Suiqiong and Sun, Ying and He, Chen and Wu, Yin and Krishnan, Ankur and Negeri, Zelalem and Imran, Mahrukh and Rice, Danielle B. and Riehm, Kira E. and Azar, Marleine and Levis, Alexander W. and Boruff, Jill and Cuijpers, Pim and Gilbody, Simon and Ioannidis, John P A and Kloda, Lorie A. and Patten, Scott B. and Ziegelstein, Roy C. and Harel, Daphna and Takwoingi, Yemisi and Markham, Sarah and Alamri, Sultan H. and Amtmann, Dagmar and Arroll, Bruce and Ayalon, Liat and Baradaran, Hamid R. and Beraldi, Anna and Bernstein, Charles N. and Bhana, Arvin and Bombardier, Charles H. and Buji, Ryna Imma and Butterworth, Peter and Carter, Gregory and Chagas, Marcos H. and Chan, Juliana C N and Chan, Lai Fong and Chibanda, Dixon and Clover, Kerrie and Conway, Aaron and Conwell, Yeates and Daray, Federico M. and de Man-van Ginkel, Janneke M. and Fann, Jesse R. and Fischer, Felix H. and Field, Sally and Fisher, Jane R W and Fung, Daniel S S and Gelaye, Bizu and Gholizadeh, Leila and Goodyear-Smith, Felicity and Green, Eric P. and Greeno, Catherine G. and Hall, Brian J. and Hantsoo, Liisa and Härter, Martin and Hides, Leanne and Hobfoll, Stevan E. and Honikman, Simone and Hyphantis, Thomas and Inagaki, Masatoshi and Iglesias-Gonzalez, Maria and Jeon, Hong Jin and Jetté, Nathalie and Khamseh, Mohammad E. and Kiely, Kim M. and Kohrt, Brandon A. and Kwan, Yunxin and Lara, Maria Asunción and Levin-Aspenson, Holly F. and Liu, Shen-Ing and Lotrakul, Manote and Loureiro, Sonia R. and Löwe, Bernd and Luitel, Nagendra P. and Lund, Crick and Marrie, Ruth Ann and Marsh, Laura and Marx, Brian P. and McGuire, Anthony and Mohd Sidik, Sherina and Munhoz, Tiago N. and Muramatsu, Kumiko and Nakku, Juliet E M and Navarrete, Laura and Osório, Flávia L and Pence, Brian W. and Persoons, Philippe and Petersen, Inge and Picardi, Angelo and Pugh, Stephanie L. and Quinn, Terence J. and Rancans, Elmars and Rathod, Sujit D. and Reuter, Katrin and Rooney, Alasdair G. and Santos, Iná S and Schram, Miranda T. and Shaaban, Juwita and Shinn, Eileen H. and Sidebottom, Abbey and Simning, Adam and Spangenberg, Lena and Stafford, Lesley and Sung, Sharon C. and Suzuki, Keiko and Tan, Pei Lin Lynnette and Taylor-Rowan, Martin and Tran, Thach D. and Turner, Alyna and van der Feltz-Cornelis, Christina M. and van Heyningen, Thandi and Vöhringer, Paul A. and Wagner, Lynne I. and Wang, Jian Li and Watson, David and White, Jennifer and Whooley, Mary A. and Winkley, Kirsty and Wynter, Karen and Yamada, Mitsuhiko and Zeng, Qing Zhi and Zhang, Yuying and Thombs, Brett D. and Benedetti, Andrea (2024) Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool. JAMA network open, 7 (11). art. no. e2429630. pp. 1-15. ISSN 2574-3805 https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2826730 10.1001/jamanetworkopen.2024.29630 |
| spellingShingle | Levis, Brooke Bhandari, Parash Mani Neupane, Dipika Fan, Suiqiong Sun, Ying He, Chen Wu, Yin Krishnan, Ankur Negeri, Zelalem Imran, Mahrukh Rice, Danielle B. Riehm, Kira E. Azar, Marleine Levis, Alexander W. Boruff, Jill Cuijpers, Pim Gilbody, Simon Ioannidis, John P A Kloda, Lorie A. Patten, Scott B. Ziegelstein, Roy C. Harel, Daphna Takwoingi, Yemisi Markham, Sarah Alamri, Sultan H. Amtmann, Dagmar Arroll, Bruce Ayalon, Liat Baradaran, Hamid R. Beraldi, Anna Bernstein, Charles N. Bhana, Arvin Bombardier, Charles H. Buji, Ryna Imma Butterworth, Peter Carter, Gregory Chagas, Marcos H. Chan, Juliana C N Chan, Lai Fong Chibanda, Dixon Clover, Kerrie Conway, Aaron Conwell, Yeates Daray, Federico M. de Man-van Ginkel, Janneke M. Fann, Jesse R. Fischer, Felix H. Field, Sally Fisher, Jane R W Fung, Daniel S S Gelaye, Bizu Gholizadeh, Leila Goodyear-Smith, Felicity Green, Eric P. Greeno, Catherine G. Hall, Brian J. Hantsoo, Liisa Härter, Martin Hides, Leanne Hobfoll, Stevan E. Honikman, Simone Hyphantis, Thomas Inagaki, Masatoshi Iglesias-Gonzalez, Maria Jeon, Hong Jin Jetté, Nathalie Khamseh, Mohammad E. Kiely, Kim M. Kohrt, Brandon A. Kwan, Yunxin Lara, Maria Asunción Levin-Aspenson, Holly F. Liu, Shen-Ing Lotrakul, Manote Loureiro, Sonia R. Löwe, Bernd Luitel, Nagendra P. Lund, Crick Marrie, Ruth Ann Marsh, Laura Marx, Brian P. McGuire, Anthony Mohd Sidik, Sherina Munhoz, Tiago N. Muramatsu, Kumiko Nakku, Juliet E M Navarrete, Laura Osório, Flávia L Pence, Brian W. Persoons, Philippe Petersen, Inge Picardi, Angelo Pugh, Stephanie L. Quinn, Terence J. Rancans, Elmars Rathod, Sujit D. Reuter, Katrin Rooney, Alasdair G. Santos, Iná S Schram, Miranda T. Shaaban, Juwita Shinn, Eileen H. Sidebottom, Abbey Simning, Adam Spangenberg, Lena Stafford, Lesley Sung, Sharon C. Suzuki, Keiko Tan, Pei Lin Lynnette Taylor-Rowan, Martin Tran, Thach D. Turner, Alyna van der Feltz-Cornelis, Christina M. van Heyningen, Thandi Vöhringer, Paul A. Wagner, Lynne I. Wang, Jian Li Watson, David White, Jennifer Whooley, Mary A. Winkley, Kirsty Wynter, Karen Yamada, Mitsuhiko Zeng, Qing Zhi Zhang, Yuying Thombs, Brett D. Benedetti, Andrea Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool |
| title | Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool |
| title_full | Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool |
| title_fullStr | Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool |
| title_full_unstemmed | Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool |
| title_short | Data-driven cutoff selection for the patient health questionnaire-9 depression screening tool |
| title_sort | data-driven cutoff selection for the patient health questionnaire-9 depression screening tool |
| url | http://psasir.upm.edu.my/id/eprint/116236/ http://psasir.upm.edu.my/id/eprint/116236/ http://psasir.upm.edu.my/id/eprint/116236/ http://psasir.upm.edu.my/id/eprint/116236/1/116236.pdf |