(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Predicting resistance to fluoroquinolones among patients with rifampicin-resistant tuberculosis using machine learning methods [1]

['Shiying You', 'Department Of Health Policy', 'Management', 'Yale School Of Public Health', 'New Haven', 'Connecticut', 'United States Of America', 'Public Health Modeling Unit', 'Melanie H. Chitwood', 'Department Of Epidemiology Of Microbial Diseases']

Date: 2022-08 

Abstract Background Limited access to drug-susceptibility tests (DSTs) and delays in receiving DST results are challenges for timely and appropriate treatment of multi-drug resistant tuberculosis (TB) in many low-resource settings. We investigated whether data collected as part of routine, national TB surveillance could be used to develop predictive models to identify additional resistance to fluoroquinolones (FLQs), a critical second-line class of anti-TB agents, at the time of diagnosis with rifampin-resistant TB. Methods and findings We assessed three machine learning-based models (logistic regression, neural network, and random forest) using information from 540 patients with rifampicin-resistant TB, diagnosed using Xpert MTB/RIF and notified in the Republic of Moldova between January 2018 and December 2019. The models were trained to predict the resistance to FLQs based on demographic and TB clinical information of patients and the estimated district-level prevalence of resistance to FLQs. We compared these models based on the optimism-corrected area under the receiver operating characteristic curve (OC-AUC-ROC). The OC-AUC-ROC of all models were statistically greater than 0.5. The neural network model, which utilizes twelve features, performed best and had an estimated OC-AUC-ROC of 0.87 (0.83,0.91), which suggests reasonable discriminatory power. A limitation of our study is that our models are based only on data from the Republic of Moldova and since not externally validated, the generalizability of these models to other populations remains unknown. Conclusions Models trained on data from phenotypic surveillance of drug-resistant TB can predict resistance to FLQs based on patient characteristics at the time of diagnosis with rifampin-resistant TB using Xpert MTB/RIF, and information about the local prevalence of resistance to FLQs. These models may be useful for informing the selection of antibiotics while awaiting results of DSTs.

Citation: You S, Chitwood MH, Gunasekera KS, Crudu V, Codreanu A, Ciobanu N, et al. (2022) Predicting resistance to fluoroquinolones among patients with rifampicin-resistant tuberculosis using machine learning methods. PLOS Digit Health 1(6): e0000059. https://doi.org/10.1371/journal.pdig.0000059 Editor: Maimuna Majumder, Harvard Medical School & Boston Children’s Hospital, UNITED STATES Received: June 24, 2021; Accepted: May 9, 2022; Published: June 30, 2022 Copyright: © 2022 You et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability: The dataset used in this study is provided in the supplementary file S1 Dataset.xlsx. All other information needed to replicate the findings of this study is provided in the main text and S1 Text.pdf. Funding: This study was supported by the United States Agency for International Development (www.usaid.gov) through the TREAT TB Cooperative Agreement No. GHN-A-00-08-00004 to TC and JW. RY was supported by K01AI119603 and TC by R01AI112438 and R01AI146555, all from the National Institute of Allergy and Infectious Diseases (www.niaid.nih.gov). KSG was supported by the US National Institutes of Health through the Eunice Kennedy Shriver National Institute of Child Health and Human Development (www.nichd.nih.gov) [F30HD105440] as well as the Medical Scientist Training Program [T32GM007205] through the National Institute of General Medical Sciences (www.nigms.nih.gov). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests: VD, AC, NC are employed by Phthisiopneumology Institute.

Introduction Tuberculosis (TB), an infectious disease caused by M. tuberculosis bacterium, is one of the top ten leading causes of death worldwide [1]. Despite recent declines in global TB incidence, drug-resistant TB continues to pose major challenges to TB control in several countries [1–3]. An estimated 3.3% of incident cases with no previous treatment and 18% of incident cases with previous treatment had multi-drug resistant or rifampicin-resistant TB (MDR/RR-TB) in 2019 [1]. The treatment of drug-resistant TB is challenging (with the success rate of 57% for MDR/RR-TB in 2019) and requires long courses (between 9–20 months) of regimens consisting of multiple antibiotics [1,4]. Many of these second-line antibiotics are associated with severe side effects [2,3]. The advent and adoption of a molecular test for the rapid detection of TB and the resistance to rifampicin (Xpert MTB/RIF) has enabled TB programs to detect individuals with RR-TB at the time of TB diagnosis [1]. The rapid detection of resistance to other anti-TB agents critical for the selection of effective treatment for individuals with RR-TB, remains a serious challenge. Phenotypic drug-susceptibility tests (DSTs) are not commonly available in many low-resource settings [1,5]. Even when available, culture-based methods take up to 12 weeks to provide results and may only be pursued routinely among a subset of culture-positive patients [6,7]. Unlike phenotypic DSTs, genotypic DSTs, such as line probe assays, can provide results within hours but these tests are also not available in many settings [4,5,8]. Given the limitations of current DSTs, in most settings, the treatment of individuals detected by Xpert MTB/RIF as having RR-TB remains empiric (i.e., without the knowledge about their full drug-susceptibility profile) and according to standardized regimens [9]. These standardized regimens are often determined at the global level, with recommendations that the local epidemiology of drug-resistant TB, which is often unknown, be used to determine final composition [9,10]. The prevalence of resistance to second-line anti-TB agents varies markedly across different counties and regions [10–17]. For example, in a population-based study conducted in Azerbaijan, Bangladesh, Belarus, Pakistan, and South Africa, the prevalence of resistance to pyrazinamide and fluoroquinolones (two of the antibiotics included in the standardized shorter regimen recommended by the World Health Organization (WHO) for people newly diagnosed with RR-TB) [9] varied substantially between settings: 2.1–3.0% for pyrazinamide, 1.0–16.6% for ofloxacin, 0.5–12.4% for levofloxacin, and 0.9–14.6% for moxifloxacin [16]. In communities where the prevalence of resistance to the antibiotics included in the standardized second-line regimens is high, the use of standardized regimens results in many individuals with RR-TB receiving antibiotics that do not match their drug-susceptibility profile [7,12]. These patients are at higher risk of mortality and may experience a longer duration of infectiousness, which could lead to further transmission of drug-resistant TB [18–23]. Receiving inappropriate treatment would also increase the risk of functional monotherapy and of selection for additional drug resistance [12,24–27]. One potential approach for improving the selection of antibiotics for patients with RR-TB is to customize the treatment regimen based on observable characteristics of patients that are associated with drug susceptibility profiles [7,10,12]. Prior studies have identified patient characteristics that are associated with an increased or decreased risk of resistance including age [28–33], sex [29,31,32,34–36], education [29,34], rural/urban residence [30,34,37,38], geographic location [39,40], occupation/employment status [29,34,38], living condition (e.g., living in a household with only one room) [28,30], smoking [29,34,37], history of detention [30,32], infection with HIV [28,30,32–34], previous anti-TB treatment, history of anti-TB treatment failure, and previous hospitalization [41,42]. Yet, evidence is limited on whether these risk factors can be used in practice, at the point-of-care, to identify patients with TB that is likely resistant to specific second-line anti-TB agents and to inform individualized treatment recommendations [43]. In this study, we examine whether data from national phenotypic surveillance systems of drug-resistant TB can be used to develop predictive models for identifying resistance to fluoroquinolones (a critical second-line class of anti-TB agents) among patients diagnosed with RR-TB using Xpert MTB/RIF. We develop and evaluate these predictive models using prospectively-collected data on the demographic and health status of 2,518 patients with culture-positive TB notified between January 2018 and December 2019 in the Republic of Moldova. To evaluate whether these predictive models could improve the selection of antibiotics, we consider a scenario where a fluoroquinolone (FLQ) would be replaced with delamanid (DLM) in the empiric RR-TB treatment regimen if additional resistance to FLQs is suspected (this follows the hierarchy of the WHO grouping of anti-TB agents) [9]. We measure the utility of these predictive models based on their ability to 1) increase the proportion of patients with RR-TB who receive an appropriate treatment regimen (i.e., a regimen that matches the susceptibility of their M. tuberculosis strain to FLQs) and 2) reduce the use of DLM to minimize selective pressure for DLM resistance.

Methods Data source and study population Our data were collected in the course of routine activities of the national tuberculosis program in the Republic of Moldova. TB incidence in the Republic of Moldova was estimated at 80 cases per 100,000 population in 2019, one of the highest rates in the European Region [1, 44]. Moldova also has one of the highest incidence rates of MDR/RR-TB in the world. In 2019, 33% of new cases and 60% of retreatment cases in Moldova had MDR/RR-TB [1]. Substantial efforts have been made in the country to combat the rise of MDR/RR-TB including the widespread adoption of Xpert MTB/RIF for rapid TB detection and universal use of phenotypic drug susceptibility testing for all patients with culture-positive TB [45]. Our dataset includes the records of 2,518 individuals with incident culture-positive TB detected between January 2018 and December 2019 (Table 1 and S1 Dataset). The record of each individual includes demographic information (e.g., age, sex, education, occupation, incarceration), diagnostic test results (e.g., results of Xpert MTB/RIF, microscopy), and additional clinical descriptors (e.g., location and severity of infection). To develop and evaluate the predictive models proposed here, we only included individuals with RR-TB diagnosed using Xpert MTB/RIF who had conclusive DST results to determine resistance or susceptibility to FLQs (Fig 1). PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 1. Flowchart of inclusion criteria. RIF: rifampicin, FLQ: a fluoroquinolone (ofloxacin, levofloxacin, or moxifloxacin). Resistance/susceptibility to RIF was determined based on the results of Xpert MTB/RIF test. Resistance/susceptibility to FLQs was determined based on the results of LJ and/or MGIT culture tests for ofloxacin, levofloxacin, and moxifloxacin. Resistance to FLQs was assumed if the resistance to least one of these three drugs was detected (see §S1.1 in S1 Text for additional details). https://doi.org/10.1371/journal.pdig.0000059.g001 PPT PowerPoint slide

PNG larger image

TIFF original image Download: Table 1. Demographic information, TB-related information, and test results in the data collected from the national tuberculosis surveillance system in the Republic of Moldova between January 2018 to December 2019. https://doi.org/10.1371/journal.pdig.0000059.t001 Ethics statement Individuals being evaluated for suspected pulmonary TB during the time period of our study were approached for enrollment by physicians and nurses trained in informed consent. Written consent was provided to allow access to routinely collected basic demographic, residential, and epidemiological data. This study was approved by the Ethics Committee of Research of the Phthisiopneumology Institute in Moldova and the Yale University Human Investigation Committee (No. 2000023071). Predictors and outcomes Our goal was to examine whether resistance to FLQs can be predicted for an individual diagnosed with RR-TB using Xpert MTB/RIF based on observable characteristics of that individual (commonly referred to as features). To develop our predictive models, we considered the following features, which are observable at the point-of-care when the selection of antibiotics for the empiric treatment is determined: Demographic information, including age, sex, occupation, education, satisfactory living condition, number of household contacts, number of household contacts under 18, whether currently in prison, whether homeless, whether receives monetary assistance, whether resides in urban area, whether reside outside Moldova more than 3 months in the past year, and whether resides in a district with low/median/high prevalence (<10%, 10–20%, and >20%) of resistance to FLQs. TB-related information, including TB anatomic location (pulmonary, extrapulmonary), and TB type (e.g., new case, relapse case, previous treatment failure). Results of microbiological tests, including microscopy. Table 1 displays the distribution of values that these features take among patients with RR-TB, and patients with RR- and FLQ-resistant TB. To prepare our dataset, we coded entities with no or unrealistic values as “missing.” We note that age, sex, and TB type did not have any missing values (Table 1). When training and evaluating our predictive models, we extrapolated the missing entries for two features ‘number of household contacts’ and ‘number of household contacts 18 or younger’ by replacing them with the mean values of each column. For all categorical features, as the occurrence of “missing” values may not be completely random, we kept all records with a “missing” value for these features when training and evaluating our predictive models. The outcome we were interested in predicting was the resistance to FLQs, which we represented by a binary indicator. We defined resistance to FLQs as having a positive LJ or MGIT culture test result for at least one of FLQs (ofloxacin, levofloxacin, and/or moxifloxacin). Additional details about how resistance to different drugs was determined are provided in §S1.1 in S1 Text. Model development and evaluation We followed the guidelines for the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) to develop and evaluate the predictive models described here [46]. We considered three supervised machine learning models: logistic regression, neural network, and random forest [47] (see §S2 in S1 Text). For each model, we estimate the probability of resistance to FLQs given the characteristics of a patient and classify the patient as “infected with FLQ-resistant TB” if this estimated probability is greater than a preset (e.g., 0.5) classification threshold. To identify features with important predictive power and to remove features that would diminish the model’s accuracy, we used three feature selection methods: recursive feature elimination [48], L 1 regularization [49], and permutation importance [50], all of which automate the selection of important features to optimize the model accuracy (§S2 in S1 Text). To assess the internal validity of our models, we followed the bootstrap validation procedure recommended by The TRIPOD Statement (see §S4) to estimate the optimism-corrected area under the receiver operating characteristic curve (OC-AUC-ROC), and the optimism-corrected sensitivity, specificity, F1 score, and Matthews correlation coefficient (MCC) [46]. Compared to randomly splitting the dataset into model development and model validation sets, the bootstrap validation approach recommended by the TRIPOD Statement is shown to be a stronger approach as it utilizes the entire dataset for model development and validation [51, 52]. Given that the relatively small size of our dataset (N = 540) did not allow for conducting temporal validation, we use the bootstrap validation approach recommended by the TRIPOD Statement to obtain estimates of the performance measures listed above. In this context, sensitivity measures the probability that the model correctly detects FLQ-resistant TB, and specificity measures the probability that the model correctly detects susceptibility to FLQs. We compared the performances of models identified through different feature selection methods based on the estimated OC-AUC-ROC and selected the model with the highest estimated OC-AUC-ROC as the final model. To understand the importance of features, we recorded the number of times that each feature was identified as significant by the corresponding feature selection algorithm among each bootstrap iteration. We note that the sensitivity and the specificity of predictive models considered here depend on the selected classification threshold (i.e., the probability above which we classify a patient as “infected with FLQ-resistant TB”). Selecting a lower classification threshold results in a more sensitive model at the expense of reduced specificity. Therefore, we evaluated our predictive models based on the estimates of OC-AUC-ROC and the estimates of sensitivity and specificity under varying classification thresholds. Impact on the selection of antituberculous medications The most recent WHO-recommended standard RR-TB treatment regimens (both the shorter and longer options) include FLQs [9]. Therefore, if the prevalence of FLQ-resistant TB is μ% among patients with RR-TB, following the WHO standardized regimen would result in μ% of patients with RR-TB not receiving an appropriate treatment regimen (i.e., a regimen that is consistent with the susceptibility of their M. tuberculosis strain to FLQs). Following the hierarchy of the WHO grouping of anti-TB agents [9], we assume that when resistance to FLQs is suspected, the FLQ is replaced with delamanid (DLM). The use of predictive models to decide whether FLQs should be included or replaced with DLM, could improve the probability that a patient with RR-TB receives an appropriate treatment regimen (i.e., a treatment regimen that includes FLQs when susceptible to FLQs and that includes DLM in place of FLQs when resistant to FLQs). However, predictive models with low specificity would increase the unnecessary use of DLM, which consequently raises the risk for the selection of additional resistance to this drug. To evaluate whether the use of the predictive models developed here could improve the selection of antibiotics for patients with RR-TB, we assessed the utility of each model using the net benefit measure [46], which is defined as λq(p)−c(p). Here, p is the classification threshold, q(p) is the expected proportion of individuals with RR-TB who receive an appropriate treatment regimen if the classification threshold is set to p, and c(p) is the expected proportion of individuals with RR-TB who unnecessarily receive DLM (instead of FLQs) if the classification threshold is set to p (see §S6 for additional details). In the above equation, λ is a trade-off threshold that represents the policymaker’s willingness to accept an increase in the proportion of individuals who unnecessarily receive DLM (i.e., c(p)) in order to increase the proportion of individuals who receive appropriate treatment regimens (i.e., q(p)). For example, λ = 5 implies that for every 1 percent point increase in the proportion of individuals who receive an appropriate treatment regimen, the policymaker is willing to accept 5 percent point increase in the proportion of individuals who unnecessarily receive DLM. To examine how accounting for the local prevalence of FLQ-resistance would impact the performance of predictive models described above, we developed two classes of models, which differed based on whether they include or exclude the feature ‘Residing in a region with low/medium/high prevalence of resistance to FLQs’ (Table 1). Role of the funding source The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and all authors had final responsibility for the decision to submit for publication.

Discussion Using data from the national TB surveillance system in the Republic of Moldova, we were able to develop predictive models to identify resistance to FLQs among patients diagnosed with RR-TB. Our final model, which was a neural network model with features identified by permutation importance method, had an OC-AUC-ROC of 0.87 (0.83,0.91). We identified age, number of household contacts, number of household contacts 18 or younger, living in districts with low, medium, or high prevalence of FLQ-resistance, TB type (new or relapse), education level (secondary or primary), if unemployed, results of microscopy test, whether residing in an urban area, and whether the living condition is satisfactory as strong features to indicate resistance to FLQs (Fig 2). These findings are consistent with the results of prior studies demonstrating that these characteristics were associated with the risk of resistance [28–34,37,38,41,42,43]. The sensitivity and the specificity of models developed here depend on the classification threshold used to predict the resistance to FLQs (Fig 3A and 3B). The optimal choice of this threshold requires a trade-off between the positive and negative consequences of using such predictive models with imperfect sensitivity and specificity. To demonstrate, we considered a scenario where FLQs can be replaced with DLM in the patient’s treatment regimen if resistance to FLQs is suspected. Increasing this classification threshold would lower the proportion of patients who are unnecessarily treated with DLM (i.e., were falsely identified as FLQ-resistant) but would also decrease the proportion of patients who receive a treatment regimen that matches the susceptibility of their TB strains to FLQs (Fig 3C and 3D). Hence, the added utility of our predictive models depends on the policymaker’s trade-off threshold λ that represents their willingness to increase the proportion of individuals who unnecessarily receive DLM in order to increase the proportion of individuals who receive appropriate treatment regimens. Our analysis suggests that, compared to the strategy of using the standardized regimen for all patients with RR-TB, the added utility of our predictive models is statistically significant for a policymaker who is willing to accept ≥ 1.0 percent point increase in the proportion of individuals who unnecessarily receive DLM for every 1 percent point increase in the proportion of individuals who receive appropriate treatment regimens. We acknowledge several limitations in this study. First, our models are based only on data from the Republic of Moldova and were not externally validated on datasets from other settings. Therefore, the generalizability of these models to other settings remains unknown. Second, our predictive models were developed using a relatively small number of observations (n = 540) and hence, the performance of these models may be sensitive to the training data. By using optimism-corrected estimates for performance metrics to evaluate the models developed here, we believe the uncertainties due to the small sample size is properly accounted for in our conclusions [46]. Third, we used a simple approach to incorporate the local information about the prevalence of resistance to FLQs. The information about the resistance to FLQs was available for only 671 individuals with RR-TB and therefore, we were not able to estimate the prevalence of resistance to FLQs for all 50 districts of Moldova. Instead, we used a single categorical feature to capture whether an individual resides in a region with low, median, or high prevalence (<10%, 10–20%, and >20%) of resistance to FLQs. We measured the utility of a predictive model based its impact on the proportion of patients with RR-TB who would receive an appropriate treatment regimen or would be unnecessarily treated with DLM. The utility of predictive models should be ideally investigated using a cost-effectiveness analysis that quantifies the cost and health consequences of replacing FLQs with DLM, which could reduce the clinical longevity of DLM but improve the treatment outcomes of patients with RR-TB. Accounting for this tradeoff allows the decisionmaker to identify the optimal classification threshold based on their tolerance in accepting a shorter clinical longevity for DLM if that leads to improving the treatment outcomes of patients with RR-TB. To measure the impact of the proposed predictive models on the selection of antituberculous medications, we considered a simple scenario where FLQs could be replaced with DLM in the empiric treatment of RR-TB if resistance to FLQs is suspected. Under certain operational research conditions, new regimens (e.g., the Nix-TB regimen [53,54]) could be recommended for people with FLQ-resistant TB. The predictive models described here could also be useful under these scenarios to identify, at the point-of-care, individuals with RR-TB who would be eligible for these novel regimens. In the future, rapid point-of-care DSTs are expected to become available [2,55]. For example, sputum-based Xpert MTB/XDR has been developed for rapid detection of resistance to isoniazid, fluoroquinolone, and ethionamide [56]. While rapid point-of-care DSTs could mitigate the limitations of current DSTs, they are not expected to be widely available in many high-burden settings. Moreover, these molecular-based DSTs can only identify the known genetic determinants of resistance. Therefore, the sensitivity of these tests is impacted by the prevalence of M. tuberculosis with gene mutations that are not associated with resistance. As such, phenotypic surveillance systems will continue to be maintained to ensure the sensitivity of these molecular-based tests. Therefore, even if rapid point-of-care DSTs become widely available, the predictive models described here could improve the accuracy of diagnosis by considering data from phenotypic surveillance systems of drug-resistant TB and including the results of rapid DSTs or whole genome sequencing as features. In the absence of rapid point-of-care DSTs, the treatment of individuals with RR-TB in many high-burden settings, remains empiric and according to standardized regimens. Although standardized regimens facilitate access to second-line treatment in high-burden settings, they may lead to poor treatment outcomes, unnecessary toxicity, increased risk for emergence of additional resistance, and further transmission of drug-resistant TB [1–4,12,18–27,57] To improve the selection of antibiotics for patients with RR-TB, we showed that data from national phenotypic surveillance systems of drug-resistant TB could be used to identify resistance to second-line anti-TB agents based on the patient’s demographic and clinical information and the estimate of the local prevalence of drug-resistant TB. Future studies could investigate the potential of these predictive models to optimize the selection of antibiotics, at the point-of-care, for patients with drug-resistant TB.

Supporting information S1 Text. Additional Details and Analyses. Table A. The estimated area under the receiver operating characteristic curve (AUC-ROC) calculated using 5-fold cross-validation, for predictive models developed by different machine learning algorithms and feature selection methods. Figure A. The estimated area under the ROC curves (AUC-ROC) of predictive models that did not account for the local prevalence of resistance to FLQs, identified by different feature selection methods. Figure B. The estimated area under the ROC curves (AUC-ROC) of predictive models that accounted for the local prevalence of resistance to FLQs, identified by different feature selection methods. Figure C. The frequency of features identified as important using recursive feature elimination algorithm and random forest classifier among 200 bootstrap iterations (see §S4 in S1 Text). Figure D. Evaluating the performance of the random forest model that accounts for the local prevalence of resistance to FLQs using features identified by recursive feature elimination for varying classification threshold. Figure E. The optimal choice of the classification threshold for varying values of the policymaker’s trade-off threshold and the optimism-corrected utility of the random forest model to determine whether FLQs should be included or replaced with DLM for a patient with RR-TB. Figure F. The estimates F1 and Matthews correlation coefficient (MCC) scores for varying classification threshold for our final model (neural network classifier and permutation importance algorithm). https://doi.org/10.1371/journal.pdig.0000059.s001 (PDF) S1 Dataset. Records of patients diagnosed with tuberculosis in Moldova between January 2018 and December 2019. https://doi.org/10.1371/journal.pdig.0000059.s002 (XLSX)

[END]
---
[1] Url: https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000059

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/