Emma Beede
Lauren Wilcox
Laura M. Vardoulakis
ABSTRACT
Deep learning algorithms promise to improve clinician workflows and patient outcomes. However, these gains have yet to be fully demonstrated in real world clinical settings. In this paper, we describe a human-centered study of a deep learning system used in clinics for the detection of diabetic eye disease. From interviews and observation across eleven clinics in Thailand, we characterize current eye-screening workflows, user expectations for an AI-assisted screening process, and post-deployment experiences. Our findings indicate that several socio-environmental factors impact model performance, nursing workflows, and the patient experience. We draw on these findings to reflect on the value of conducting human-centered evaluative research alongside prospective evaluations of model accuracy.
Supplemental Material
- Michael D. Abràmoff, Philip T. Lavin, Michele Birch, Nilay Shah, and James C Folk. 2018. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 1 (Aug. 2018), 39.Google Scholar
- E. Alberdi, A. A. Povyakalo, L. Strigini, P. Ayton, M. Hartswood, R. Procter, and R. Slack. 2005. Use of computer-aided detection (CAD) tools in screening mammography: a multidisciplinary investigation. Br. J. Radiol. 78 Spec No 1 (2005), S31--40.Google Scholar
- American Academy of Ophthalmology. 2015. Eye Health Statistics. https://www.aao.org/newsroom/eye-health-statistics#_edn25. (2015). Accessed: 2019--9--7.Google Scholar
- Eta S. Berner. 2007. Clinical Decision Support Systems: Theory and Practice. Springer Science & Business Media.Google Scholar
- Hugh Beyer and Karen Holtzblatt. 1997. Contextual design: defining customer-centered systems. Elsevier.Google ScholarDigital Library
- Timothy W. Bickmore, Laura M. Pfeifer, and Brian W. Jack. 2009. Taking the Time to Care: Empowering Low Health Literacy Hospital Patients with Virtual Nurse Agents. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, NY, NY, USA, 1265--1274. DOI: http://dx.doi.org/10.1145/1518701.1518891Google ScholarDigital Library
- Keld Bødker, Finn Kensing, and Jesper Simonsen. 2009. Participatory IT design: designing for business and workplace realities. MIT press.Google ScholarDigital Library
- Carrie J. Cai, Emily Reif, Narayan Hegde, Jason Hipp, Been Kim, Daniel Smilkov, Martin Wattenberg, Fernanda Viegas, Greg S. Corrado, Martin C. Stumpe, and Michael Terry. 2019a. Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 4, 14 pages. DOI: http://dx.doi.org/10.1145/3290605.3300234Google ScholarDigital Library
- Carrie Jun Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019b. "Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. CSCW Conf Comput Support Coop Work 2019 (2019).Google ScholarDigital Library
- Ronald A Castellino. 2005. Computer aided detection (CAD): an overview. Cancer Imaging 5, 1 (2005), 17.Google ScholarCross Ref
- CDC. 2019. More than 100 million Americans have diabetes or prediabetes. Press Release. (February 2019). https://www.cdc.gov/media/releases/2017/ p0718-diabetes-report.html.Google Scholar
- Elodia B. Cole, Zheng Zhang, Helga S. Marques, R. Edward Hendrick, Martin J. Yaffe, and Etta D. Pisano. 2014. Impact of Computer-Aided Detection Systems on Radiologist Accuracy With Digital Mammography. AJR Am. J. Roentgenol. 203, 4 (Oct. 2014), 909.Google ScholarCross Ref
- Kathleen Musante DeWalt and Billie R DeWalt. 2002. Participant Observation: A Guide for Fieldworkers. Rowman Altamira.Google Scholar
- Shelley E Ellis, Theodore Speroff, Robert S. Dittus, Anne Brown, James W. Pichert, and Tom A. Elasy. 2004. Diabetes patient education: a meta-analysis and meta-regression. Patient Educ. Couns. 52, 1 (Jan. 2004), 97--105.Google ScholarCross Ref
- Glyn Elwyn, Isabelle Scholl, Caroline Tietbohl, Mala Mann, Adrian GK Edwards, Catharine Clay, France Légaré, Trudy van der Weijden, Carmen L. Lewis, Richard M. Wexler, and others. 2013. "Many miles to go...": a systematic review of the implementation of patient decision support interventions into routine clinical practice. BMC medical informatics and decision making 13, 2 (Nov. 2013), 1--10.Google Scholar
- Geraldine Fitzpatrick and Gunnar Ellingsen. 2013. A review of 25 years of CSCW research in healthcare: contributions, challenges and future agendas. Computer Supported Cooperative Work (CSCW) 22, 4--6 (2013), 609--665.Google ScholarDigital Library
- Jodi Forlizzi and John Zimmerman. Promoting service design as a core practice in interaction design.Google Scholar
- Maryellen L Giger, Heang-Ping Chan, and John Boone. 2008. Anniversary Paper: History and status of CAD and quantitative image analysis: The role of Medical Physics and AAPM. Med. Phys. 35, 12 (Dec. 2008), 5799.Google ScholarCross Ref
- Trisha Greenhalgh, Joe Wherton, Chrysanthi Papoutsi, Jenni Lynch, Gemma Hughes, Sue Hinder, Rob Procter, Sara Shaw, and others. 2018. Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework. BMC medicine 16, 1 (2018), 66.Google Scholar
- Varun Gulshan, Lily Peng, Marc Coram, Martin C Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, and others. 2016. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316, 22 (2016), 2402--2410.Google ScholarCross Ref
- Mark Hartswood, Rob Procter, Mark Rouncefield, Roger Slack, James Soutter, and Alex Voss. 2003. ?Repairing' the Machine: A Case Study of the Evaluation of Computer-Aided Detection Tools in Breast Screening. In ECSCW 2003. Springer, 375--394.Google ScholarCross Ref
- Matthew K Hong, Clayton Feustel, Meeshu Agnihotri, Max Silverman, Stephen F Simoneaux, and Lauren Wilcox. 2017. Supporting families in reviewing and communicating about radiology imaging studies. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5245--5256.Google ScholarDigital Library
- Jenchitr W, Hanutsaha P, Iamsirithaworn S, Parnrat U, Choosri P. 2007. The national survey of blindness low vision and visual impairment in thailand 2006--2007. Thai J Pub Hlth Ophthalmol 21, 1 (2007), 10.Google Scholar
- Marina Jirotka, Rob Procter, Mark Hartswood, Roger Slack, Andrew Simpson, Catelijne Coopmans, Chris Hinds, and Alex Voss. 2005. Collaboration and trust in healthcare innovation: The eDiaMoND case study. Computer Supported Cooperative Work (CSCW) 14, 4 (2005), 369--398.Google ScholarDigital Library
- Pearse A Keane and Eric J Topol. 2018. With an eye to AI and autonomous diagnosis. npj Digital Medicine 1, 1 (Aug. 2018), 1--3.Google Scholar
- Ajay Kohli and Saurabh Jha. 2018. Why CAD failed in mammography. Journal of the American College of Radiology 15, 3 (2018), 535--537.Google ScholarCross Ref
- Mark A Musen, Blackford Middleton, and Robert A Greenes. 2014. Clinical Decision-Support Systems. In Biomedical Informatics. Springer, London, 643--674.Google Scholar
- American Academy of Ophthalmology. 2002. International Clinical Diabetic Retinopathy Disease Severity Scale. http://www.icoph.org/dynamic/attachments/resources/ diabetic-retinopathy-detail.pdf. (Oct. 2002). Accessed: 2019--12--17.Google Scholar
- World Health Organization. 2018. Vision impairment and blindness. https://www.who.int/news-room/ fact-sheets/detail/blindness-and-visual-impairment. (2018). Accessed: 2019--9--13.Google Scholar
- Sun Young Park, Pei-Yi Kuo, Andrea Barbarin, Elizabeth Kaziunas, Astrid Chow, Karandeep Singh, Lauren Wilcox, and Walter Lasecki. 2019. Identifying Challenges and Opportunities in Human--AI Collaboration in Healthcare. (2019).Google Scholar
- Laura Pfeifer Vardoulakis, Amy Karlson, Dan Morris, Greg Smith, Justin Gatewood, and Desney Tan. 2012. Using mobile phones to present medical information to hospital patients. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1411--1420.Google ScholarDigital Library
- Sarah K Pontefract, Jamie J Coleman, Hannah K Vallance, Christine A Hirsch, Sonal Shah, John F Marriott, and Sabi Redwood. 2018. The impact of computerised physician order entry and clinical decision support on pharmacist-physician communication in the hospital setting: A qualitative study. PloS one 13, 11 (2018), e0207450.Google ScholarCross Ref
- Paisan Raumviboonsuk, Jonathan Krause, Peranut Chotcomwongse, Rory Sayres, Rajiv Raman, Kasumi Widner, Bilson JL Campana, Sonia Phene, Kornwipa Hemarat, Mongkol Tadarati, and others. 2019. Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program. npj Digital Medicine 2, 1 (2019), 25.Google Scholar
- Madhu C Reddy, David W McDonald, Wanda Pratt, and M Michael Shabot. 2005. Technology, work, and information flows: Lessons from the implementation of a wireless alert pager system. Journal of biomedical informatics 38, 3 (2005), 229--238.Google ScholarDigital Library
- Nigam H. Shah, Arnold Milstein, and Steven C. Bagley, PhD. 2019. Making Machine Learning Models Clinically Useful. JAMA (08 2019). DOI: http://dx.doi.org/10.1001/jama.2019.10306Google ScholarCross Ref
- Lauren Wilcox, Dan Morris, Desney Tan, and Justin Gatewood. 2010. Designing patient-centric information displays for hospitals. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2123--2132.Google ScholarDigital Library
- Lauren Wilcox, Janet Woollen, Jennifer Prey, Susan Restaino, Suzanne Bakken, Steven Feiner, Alexander Sackeim, and David K Vawdrey. 2016. Interactive tools for inpatient medication tracking: a multi-phase study with cardiothoracic surgery patients. J. Am. Med. Inform. Assoc. 23, 1 (Jan. 2016), 144--158.Google ScholarCross Ref
- World Health Organization. 2007. Global Initiative for the Elimination of Avoidable Blindness : action plan 2006--2011. https://www.who.int/blindness/Vision2020_report.pdf. (2007). Accessed: 2019--9--7.Google Scholar
- World Health Organization. 2014. WHO: Diabetes factsheet. https://www.who.int/news-room/fact-sheets/detail/diabetes. (2014). Accessed: 2019--9--13.Google Scholar
- World Health Organization. 2016a. Diabetes country profiles 2016 : Thailand. https://www.who.int/diabetes/ country-profiles/tha_en.pdf?ua=1. (2016). Accessed: 2019--9--7.Google Scholar
- World Health Organization. 2016b. Diabetes country profiles 2016 : USA. https://www.who.int/diabetes/country-profiles/usa_en.pdf. (2016). Accessed: 2019--9--7.Google Scholar
- Qian Yang, Aaron Steinfeld, and John Zimmerman. 2019. Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, Article 238, 11 pages. DOI: http://dx.doi.org/10.1145/3290605.3300468Google ScholarDigital Library
Index Terms
- A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy
Recommendations
Experiences in mHealth for chronic disease management in 4 countries
ISABEL '11: Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication TechnologiesThis paper describes mHealth applications to deal with Non Communicable Diseases in North and Latin America: In Chile, a project focused on Diabetes Mellitus type 2; In the United States, Honduras, and Mexico, projects focused in diabetes, heart failure,...
Diabetic Retinopathy Detection using Deep Learning
ICMLSC '20: Proceedings of the 4th International Conference on Machine Learning and Soft ComputingDiabetic Retinopathy (DR) is an eye disease associated with chronic diabetes. DR is the leading cause of blindness among working aged adults around the world and estimated it may affect more than 93 million people. Progression to vision impairment can ...
A Survey on Automatic Diabetic Retinopathy Screening
AbstractDiabetes is a chronic disease caused due to the increase in the sugar level in the blood. Diabetes mainly affects heart, blood vessels, kidneys, eyes and nerves. There are mellitus type 1 and 2 diabetes, in which insulin is either not taken by the ...
Comments