1
|
Macdonald T, Dinnes J, Maniatopoulos G, Taylor-Phillips S, Shinkins B, Hogg J, Dunbar JK, Solebo AL, Sutton H, Attwood J, Pogose M, Given-Wilson R, Greaves F, Macrae C, Pearson R, Bamford D, Tufail A, Liu X, Denniston AK. Target Product Profile for a Machine Learning-Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study. JMIR Res Protoc 2024; 13:e50568. [PMID: 38536234 PMCID: PMC11007610 DOI: 10.2196/50568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 02/02/2024] [Accepted: 02/13/2024] [Indexed: 04/13/2024] Open
Abstract
BACKGROUND Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation. OBJECTIVE This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England. METHODS This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence's Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from "definitely exclude" to "definitely include," and suggest edits. The document will be iterated between rounds based on participants' feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote. RESULTS Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024. CONCLUSIONS The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/50568.
Collapse
Affiliation(s)
- Trystan Macdonald
- Ophthalmology Department, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham National Health Service Foundation Trust, Birmingham, United Kingdom
- Academic Unit of Ophthalmology, Institute of Inflammation and Aging, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
| | - Jacqueline Dinnes
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
| | | | | | - Bethany Shinkins
- Warwick Medical School, University of Warwick, Coventry, United Kingdom
| | - Jeffry Hogg
- Population Health Sciences Institute, Faculty of Medical Sciences, The University of Newcastle upon Tyne, Newcastle, United Kingdom
| | | | - Ameenat Lola Solebo
- Population Policy and Practice, University College London Great Ormond Street Institute of Child Health, London, United Kingdom
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
| | | | - John Attwood
- Alder Hey Children's Hospital, Alder Hey Children's Hospital NHS Foundation Trust, Liverpool, United Kingdom
| | | | - Rosalind Given-Wilson
- St. George's University Hospitals National Health Service Foundation Trust, London, United Kingdom
| | - Felix Greaves
- National Institute for Health and Care Excellence, London, United Kingdom
- Faculty of Medicine, School of Public Health, Imperial College London, London, United Kingdom
| | - Carl Macrae
- Nottingham University Business School, University of Nottingham, Nottingham, United Kingdom
| | - Russell Pearson
- Medicines and Healthcare Products Regulatory Agency, London, United Kingdom
| | | | - Adnan Tufail
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Institute of Ophthalmology, University College London, London, United Kingdom
| | - Xiaoxuan Liu
- Ophthalmology Department, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham National Health Service Foundation Trust, Birmingham, United Kingdom
- Academic Unit of Ophthalmology, Institute of Inflammation and Aging, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
| | - Alastair K Denniston
- Ophthalmology Department, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham National Health Service Foundation Trust, Birmingham, United Kingdom
- Academic Unit of Ophthalmology, Institute of Inflammation and Aging, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
- Centre for Regulatory Science and Innovation, Birmingham Health Partners, Birmingham, United Kingdom
- National Institute for Health and Care Research Biomedical Research Centre at Moorfields and University College London Institute of Ophthalmology, London, United Kingdom
| |
Collapse
|
2
|
Miyata H, Akiyama Y, Iizuka A, Kondou R, Maeda C, Kanematsu A, Watanabe K, Ashizawa T, Nagashima T, Urakami K, Ohshima K, Kawata T, Muramatsu K, Shiomi A, Terashima M, Sugino T, Notsu A, Mori K, Yamaguchi K. Development of an Automatic Measurement Method for CD8 and PD-1 Positive T Cells Using Image Analysis Software. Anticancer Res 2022; 42:419-427. [PMID: 34969752 DOI: 10.21873/anticanres.15500] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Revised: 11/19/2021] [Accepted: 11/22/2021] [Indexed: 11/10/2022]
Abstract
BACKGROUND/AIM With the progress in cancer immunotherapy using immune checkpoint blockade (ICB) therapy, histological observations of tumor-infiltrating lymphocyte (TIL) status are needed to evaluate the antitumor effect of ICB using imaging analysis software. MATERIALS AND METHODS Formalin-fixed paraffin-embedded sections obtained from colorectal cancer and gastric cancer patients with more than 500 single nucleotide variants were stained with anti-CD8 and anti-PD-1 antibodies. Based on our own algorithm and imaging analysis software, an automatic TIL measurement method was established and compared to the manual counting methods. RESULTS In the CD8+ T cell number measurement, there was a good correlation (r=0.738 by Pearson test) between the manual and automated counting methods. However, in the PD-1+ T cell measurement, there was a large difference in TIL numbers in both groups. After adjustment of the parameter settings, the correlation between the manual and automated methods in the PD-1+ T cell measurements improved (r=0.668 by Pearson test). CONCLUSION An imaging software-based automatic measurement could be a simple and useful tool for evaluating the therapeutic effect of cancer immunotherapies in terms of TIL status.
Collapse
Affiliation(s)
- Haruo Miyata
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Yasuto Akiyama
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Akira Iizuka
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Ryota Kondou
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Chie Maeda
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Akari Kanematsu
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Kyoko Watanabe
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Tadashi Ashizawa
- Immunotherapy Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Takeshi Nagashima
- Cancer Diagnostics Research Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan.,SRL, Tokyo, Japan
| | - Kenichi Urakami
- Cancer Diagnostics Research Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Keiichi Ohshima
- Medical Genetics Division, Shizuoka Cancer Center Research Institute, Shizuoka, Japan
| | - Takuya Kawata
- Division of Pathology, Shizuoka Cancer Center Hospital, Shizuoka, Japan
| | - Koji Muramatsu
- Division of Pathology, Shizuoka Cancer Center Hospital, Shizuoka, Japan
| | - Akio Shiomi
- Division of Colon and Rectal Surgery, Shizuoka Cancer Center Hospital, Shizuoka, Japan
| | - Masanori Terashima
- Division of Gastric Surgery, Shizuoka Cancer Center Hospital, Shizuoka, Japan
| | - Takashi Sugino
- Division of Pathology, Shizuoka Cancer Center Hospital, Shizuoka, Japan
| | - Akifumi Notsu
- Clinical Trial Coordination Office, Shizuoka Cancer Center Hospital, Shizuoka, Japan
| | - Keita Mori
- Clinical Trial Coordination Office, Shizuoka Cancer Center Hospital, Shizuoka, Japan
| | | |
Collapse
|