Schematic of a convolutional neural

network and transfer learning

The schematic diagram of

transfer learning. Source: Lingling Li et al*.* Diabetic retinopathy identification system based on transfer learning. 2020. J. Phys.: Conf. Ser. 1544 012133. https://​doi.​org/​10.​1088/​1742-6596/​1544/​1/​012133. Content from this work may be used under the terms of the Creative Commons Attribution 3.​0 licence

Another approach that has been suggested by several authors to address the problem of limited data sets is the use of

generative adversarial networks (GANs)

to synthesize new images from a training data set of real images. GANs are ML models that can generate new data with the same statistics as the training set (Fig. 1.10). For example, a GAN trained on photographs can generate photographs of non-existing persons that look as authentic as real humans (Fig. 1.11). Artificial photos can be found at https://​thispersondoesno​texist.​com. Many applications of GAN have been proposed, including, art, fashion, advertising, science, video games, however, concerns about malicious uses were also raised, e.g., to produce fake, possibly

incriminating

, photographs and videos.

The schematic

presentation

of generative adversarial network (GAN)

The image of a young

woman

generated by StyleGAN, an generative adversarial network (GAN). The person in this photo does not exist, but is generated by an artificial intelligence based on an analysis of portraits. Source: https://​commons.​wikimedia.​org/​wiki/​File:​Woman_​1.​jpg. This file is in the public domain because, as the work of a computer algorithm or artificial intelligence, it has no human author in whom copyright is vested

Burlina et al. used the Age-Related Eye Disease Study data set of over 130,000 fundus images to generate a similar number of synthetic images to train DL models. The performance of DL

models

trained with the synthetic images was nearly as good as the performance of models trained on real images [²¹]. Liu et al. have shown that 92% of synthetic OCT images had sufficient quality for further clinical interpretation. Only about 26–30% of synthetic post-therapeutic images could be accurately identified as synthetic images (Fig. 1.8) [²²]. The accuracy of models trained on synthetic images to predict wet or dry macular status was 0.85 (95% CI 0.74–0.95) [²²]. In a study by Zheng et al., the image quality of real versus synthetic OCT images was similar as assessed by two retinal specialists. The accuracy of discrimination of real versus synthetic OCT images was 59.50% for retinal specialist 1 and 53.67% for retinal specialist 2. For the local data set, the DL model trained on real and synthetic OCT images had an area under the curve of 0.99 and 0.98, respectively. For the clinical data set, the area under the curve was 0.94 for the real model and 0.90 for the synthetic one [²³]. These studies suggest the

synthetic images can be used by clinicians for educational purposes and developing DL algorithms [²⁴].

An important and interesting issue is the clinical application of continual ML, i.e.,

continuous learning and development from new data while retaining previously learned knowledge [²⁵]. However, there are technical challenges to the implementation of this promising concept, including the need to prevent interference between new and old data, and old and new knowledge. In the catastrophic interference phenomenon, the acquisition of new data can lead to an abrupt decrease in the performance of an algorithm. Practical applications of AI tools in health care must be cautiously introduced because of the existence of such risks. FDA regulations require that FDA-approved autonomic algorithms be locked for safety to prevent unpredictable future changes. This requirement, however, is designed to ensure the safety of the model rather than improving its performance. Continual learning could refine the performance of machine-learning algorithms by the gradual correction and elimination of mistakes. It will be necessary to consider how this

technology can be introduced safely to health care.

The future development of

ophthalmology depends on better and possibly unlimited access to the medical data stored within electronic health records. However, this access cannot be allowed to compromise of privacy of this very sensitive data. There is a need for effective regulations that will set a balance between individual protection and the common good. One approach to protecting privacy and increasing sample size is to share DL algorithms with local institutions for retraining purposes, but without sharing the private data used to build the algorithms. This model-to-data approach, also known as federated learning, was tested in ophthalmology and was shown to work effectively [²⁶].

According to the US National Institute of Standards and

Technology, biometric data, including retina images, are personally identifiable information and should be protected from inappropriate access. Although AI models have been shown to diagnose and stage some ocular diseases from fundus photographs, OCT, and visual-field images, most AI algorithms were tested on data sets that did not correspond well to real-world conditions. Patient populations were usually homogenous, and poor-quality images and patients with multiple pathologies were excluded. Future studies are needed to validate algorithms on ocular images from heterogeneous populations, including both good- and poor-quality images. Otherwise, we may face the situation of “good AI gone bad.” The tendency to cherry-pick the best results might make the situation even worse. AI algorithms can behave unpredictably when applied in real life. Algorithm performance can degrade after deployment due to the changes between the training and testing conditions (dataset shift), caused, for

example, by using to images generated by a different device than this in the training set or collected in a different clinical environment [²⁷–³⁰]. Moreover, algorithms may return different outputs at different times when presented with similar inputs [³¹, ³²]—they can be affected by minor changes in image quality or extraneous data on an image [³²–³⁵]. All these problems might lead to misdiagnosis and erroneous treatment suggestions, breaching trust in AI technologies. An error in an AI system could harm hundreds or even thousands of patients.

A recent report from the National Academy of Medicine [³⁶] highlights some important challenges in the further development of AI applications in health care (Table 1.2). Its authors advocate the use of openly accessible, standardized, population-representative data; addressing explicit and

implicit biases related to AI; developing and deploying appropriate training and educational programs for health workers to support health-care AI; and balancing innovation and safety through the use of regulation and legislation to promote trust.

Practical

challenges to the advancement and application of AI tools in clinical settings