(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Affordable artificial intelligence-based digital pathology for neglected tropical diseases: A proof-of-concept for the detection of soil-transmitted helminths and Schistosoma mansoni eggs in Kato-Katz [1]

['Peter Ward', 'Etteplan Sweden Ab', 'Uppsala', 'Peter Dahlberg', 'Ole Lagatie', 'Janssen Global Public Health', 'Janssen R D', 'Beerse', 'Joel Larsson', 'August Tynong']

Date: 2022-07 

The annotation of images occurred in six consecutive steps. In the first step , a non-exhaustive set of focus stacks containing STH and SCH eggs were identified from all collected images. The most in-focus FOV image for each focus stack was recorded by visual confirmation. In the second step , we manually annotated the focused images with both the location and the identity of each helminth egg (Ascaris, Trichuris, hookworm, or SCH). More specifically, we manually annotated the FOV images by drawing rectangular boxes around helminth eggs. For this, we used LabelImg, an open-source graphical image annotation tool [ 20 ]. The goal was to annotate approximately 100 eggs for each STH and SCH helminth species while targeting different stool samples to provide a wide variation in the appearance of images. In the third step , we trained an object detection model (see next section) to automatically extract potential helminth eggs from the remaining images. The new egg candidates were presented to experienced users in a purpose-built graphical user interface (GUI) for manual verification and further model training. In the fourth step , we applied the trained model to the remaining focused FOV images to find any eggs that may have been missed in the previous step. Again, the potential helminth eggs were verified by users with the GUI. This iterative approach (repeating the third and fourth step) considerably simplified the extraction process (and annotation of eggs from FOV images) but did not guarantee that all helminth eggs were extracted from the FOV images. Therefore, we used LabelImg again to review all focused FOV images in an unseen ground truth test set (extracting the test set is explained in next section) for any missed helminth eggs in the sixth step . This final step ensures accurate evaluation of the final AI model against our unseen ground truth test set.

AI training and evaluation.

Once the confirmed helminth eggs were extracted and verified, FOV images were randomly shuffled and split into a three data sets, namely a training set, a validation set and a testing set. When constructing these data sets of FOV images, we aimed for a desired split ratio of 70:20:10 for each helminth eggs species in each data set. As the FOV images contained multiple eggs and may include mixed infections it was not possible to precisely split into the target percentages. The training and validation sets were used during the training process to develop the model. The test set was used as an unseen set of data withheld from the training process and was used only for the final evaluation of the models.

Using the annotated training set, a convolutional neural network (CNN) was trained. While there are numerous types and combinations of neural network architectures that can be used for image classification and object detection, most state-of-the-art models are based on CNN [21] as they can capture spatial dependencies (one pixel value is often dependent on its neighbouring pixels) through the use kernels or filters, which are convolved over the image to reduce dimensionality and extract features from the image. The extracted features depend on the size of the filter and elements within the filter. For example, a 3x3 filter may be constructed with elements that perform edge detection of vertical lines. However, rather than manually defining the properties of the filters (as is done in traditional computer vision approaches), the properties can be optimised in the training process. CNN-based architectures vary in approach by the number of layers used, the number of filters per layer and the size of filters. When considering the purpose of object detection, there are typically two steps, namely finding the objects in the image and classifying the objects. While some approaches may adopt a single stage CNN approach, others may add additional steps or techniques to separate the problem [22]. Fortunately, to design deep learning-based object detectors, there are already many publicly available architectures benchmarked on common datasets, which allow an architecture to be selected based on desired speed and performance characteristics. Instead of training one of these architectures from scratch, we used a transfer learning approach, which takes a pre-trained object detection model as a starting point and transfers the knowledge from its original domain-specific knowledge to another domain [23]. The transfer learning approach is effective as it can take learnt feature detectors from one model that has been exposed to a much larger dataset. These learnt feature detectors may include the ability to detect edges, textures, shapes, and patterns that are applicable in almost any image application. The AI evaluation presented in this research is based on the R-FCN ResNet101 COCO model available from the TensorFlow 1 Detection Model Zoo, as this model showed a suitable compromise between speed and performance [24]. We updated the number of object classes and the label mapping to suit the four helminth egg species available in our training set, but kept the other default parameters, such as data augmentation options and hyperparameters for training, unchanged. The pretrained model and corresponging variables were used as the starting point in the transfer learning process, where these variables are finetuned using the case specific data. Options like the ability to freeze certain layers, which reduces the amount of variables to train and prevents changes to features that may be considered useful, were not explored in this paper.

Our original datasets and several exported model files trained for STH & SCH, including the R-FCN ResNet 101 COCO model presented in this research are made available on Kaggle [https://doi.org/10.34740/KAGGLE/DS/1896823]. The original code using TensorFlow 1 is not supported on Kaggle due to version compatibility, however a working solution in TensorFlow 2 that can extract data, train object detection models and evaluate both TensorFlow 1 and TensorFlow 2 models is made available.

While the provided Kaggle notebooks are working solutions, the free cloud-based graphics processing unit (GPU) available on Kaggle is not sufficient to optimally train the available state-of-the-art object models with the provided data. During our research, the training of the AI model was performed on an NVIDIA GeForce RTX 2070 GPU, while the evaluation of the unseen test set was performed on a Jetson AGX Xavier Developer Kit. The AGX represents a potential fieldable computer for the use of AI models without an internet connection to a cloud server where commercial solutions typically process images.

The performance of the AI model on the unseen test was obtained by comparing the manually verified ground truth to that predicted by the AI model in a confusion matrix. The recall (a perfect recall score has no false negatives), precision (a perfect precision score has no false positives), and F1-score (harmonic mean of recall and precision) were calculated. Evaluation measures involving true negatives in object detection are typically not used due to the sheer number of detections made for every FOV image (the model may generate thousands of regions of interest for evaluation on every FOV image). Every evaluated region within the FOV image that is correctly classified as a negative (e.g. some debris) would increase the true negative count, significantly outnumbering the true positives, false positives, and false negatives for a given FOV. Clinical sensitivity and specificity for the AI model on KK stool thick smears were not evaluated in this research for several reasons: (i) many scans did not cover the complete KK stool thick smear area, therefore it was not possible to count all eggs per slide, (ii) the tool lacked the functionality to distinguish duplicate eggs in overlapping FOV images, and (iii) egg counts using microscope visual examination for the scanned slides were not recorded.

[END]
---
[1] Url: https://journals.plos.org/plosntds/article?id=10.1371/journal.pntd.0010500

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/