(C) PLOS One [1]. This unaltered content originally appeared in journals.plosone.org.
Licensed under Creative Commons Attribution (CC BY) license.
url:https://journals.plos.org/plosone/s/licenses-and-copyright

------------



Patient-reported outcomes and target effect sizes in pragmatic randomized trials in ClinicalTrials.gov: A cross-sectional analysis

['Shelley Vanderhout', 'Clinical Epidemiology Program', 'Ottawa Hospital Research Institute', 'Ottawa', 'Ontario', 'School Of Epidemiology', 'Public Health', 'University Of Ottawa', 'Dean A. Fergusson', 'Department Of Medicine']

Date: 2022-02 

Among a large sample of pragmatic RCTs published 2014 to 2019, the use of PROs and prevalence of reporting on patient or public engagement were low. Just over a third chose PROs as primary or coprimary outcomes, while just over half chose PROs as either primary or secondary outcomes. Virtually, no trial cited patient or other stakeholder consultation in the choice of the primary outcome, even when the outcome was a PRO. Pediatric trials and trials in older adults, as well as trials conducted in LMICs had comparatively lower prevalence of use of PROs. Target differences for primary PROs were often not justified in sample size calculations; when a justification was provided, it was rarely based on patient or stakeholder consultation.

To our knowledge, no other study has described use of PROs and sample size reporting in a general sample of pragmatic trials. However, several studies have identified similar patterns of inadequate use of best practices, infrequent patient engagement [ 26 ], and poor reporting among trials including PROs. A scoping review of 44 trials with patient-important or patient-relevant outcomes identified that only 36% of studies included patients or stakeholders in determining outcomes [ 27 ]. Another review of 75 protocols for trials including PROs showed that PRO-specific sample size justifications were provided only 51% of the time, and 61% of PRO-specific items from the SPIRIT-PRO guidelines were incomplete [ 8 ]. Several reviews of cancer trials, which included PROs as outcomes [ 28 – 31 ], revealed that adherence to guidelines from SPIRIT [ 7 ] and ISOQOL [ 5 ] for reporting on PROs in clinical trials was consistently suboptimal; authors often did not adequately describe justifications for choice of PRO or sample size calculations. Collectively, these studies corroborate many of our findings across settings and clinical areas. Although we did not identify a significant increase in use of PROs over time, others have identified an increase in PROs used in trials registered at ClinicalTrials.gov between 2004 and 2013 [ 15 , 16 ]. An increase was also observed in the Australia and New Zealand Clinical Trials Registry between 2005 and 2017, with 64% of trials registered in 2016 including at least one PRO [ 32 ]. It is possible that our observation window between 2014 and 2019 was too narrow to detect a trend or that the use of PROs has plateaued in recent years. Notably, the CONSORT-PRO extension for trial reports was published in 2013 [ 8 ], while the SPIRIT-PRO extension for trial protocols was published later, in 2018 [ 7 ]. One might expect a gradual increase in PRO reporting from 2013 onwards, but this was not observed in our data.

Our study had limitations. From our larger database of trials, we selected the subset that were registered in ClinicalTrials.gov . Although ClinicalTrials.gov registration has become widespread and even federally mandated in the United States [ 35 ], our sample may have captured more American studies than those conducted elsewhere. We focused on information provided in the primary trial reports and did not retrieve information provided in trial protocols, which means that misclassification was possible. Our analyses of factors associated with use of PROs were exploratory and considered characteristics individually; we did not conduct multivariable analyses to identify factors independently associated with PROs. Finally, many of our trial characteristics were downloaded directly from CT.gov , thus any inaccurate classifications by trial authors in CT.gov may have influenced our findings.

There are some key strengths of this study. Identification of pragmatic trials in the literature is challenging. Other reviews of pragmatic trials have used limited or arbitrary search terms to identify pragmatic trials [ 33 , 34 ]. We used a published search filter [ 18 ], which relied on specific terms and phrases in the title and abstract shown to be associated with pragmatism. We used this approach because there are no reporting guidelines requiring authors to label their trials as pragmatic in the title or abstract and we wanted to include a broader range of trials with pragmatic intention. Our search resulted in a large sample of trials across a wide range of clinical areas, study settings, and patient populations, which allowed us to examine characteristics associated with use of PROs more broadly.

Implications for research and practice

Given that pragmatic trials are intended to inform clinical practice and incorporate patient perspectives [2], the prevalence of use of PROs and patient and public engagement, especially among trials with PROs, was surprisingly low. Our search covered the period 2014 to 2019, which is several years after the establishment of various patient engagement strategies in the UK [36], Canada [37], and USA [38], and publication of the CONSORT-PRO guidelines in 2013 [8]. One explanation for lower than expected prevalence of PROs could be related to the use of routinely collected data often associated with pragmatic trials; availability of PROs in such databases remains rare [39]. However, a defining characteristic of pragmatic trials is that the key results should be useful to decision-makers. Choosing the right outcome is therefore even more important than outcome source. Where an appropriate outcome to inform decision-making is not available routinely, pragmatic trials need to collect data directly from participants but in a way that does not interfere too much with routine clinical practice [2]. A possible explanation for the low prevalence of patient and public engagement is poor quality of reporting; authors are encouraged to report patient engagement, especially concerning PROs [40], using tools such as the Guidance for Reporting the Involvement of Patients and the Public (GRIPP2) checklist [41].

Not surprisingly, given the low prevalence of patient or public engagement, involvement of patients and stakeholders in determining target differences for sample size calculations was rare. The minimal important difference has been defined as “the smallest difference in score in the outcome of interest that informed patients perceive as important, either beneficial or harmful, and which would lead the patient or clinician to consider a change in the management” [42,43]. Best practices given by ISOQOL [44] recommend establishing target differences for PROs to enhance the applicability of their use in clinical trials to care settings [10]. However, patient and public engagement in determining target differences is not included in these or other guidelines [22], though it would seem prudent for greater emphasis to be placed on this type of engagement [45]. Although obtaining input from patients about numerical aspects of trials such as target differences may raise particular challenges (especially in the case of more complex analyses and types of effect sizes), patients and members of the public have expressed interest in being included in discussing and contributing to the definition of the target difference used, which they believed would improve the transparency of research [46]. Patients, researchers, and statisticians have acknowledged that including patients in the actual statistical analysis may not be an efficient use of resources, but engagement while developing the assumptions required for determining the target difference, for example, could be one way to ensure that the patient perspective is captured [46,47]. Yet, there are gaps in the literature with important conceptualization to be done in this space. Guidance for determining meaningful differences in PROs across patient groups, clinical diagnoses, and treatment contexts is available, but there is uncertainty about how to best involve patients in its application [48,49]. For example, a treatment with less functional impairment might be desirable if the clinical effectiveness is maintained, but no longer acceptable if the clinical effectiveness is reduced. Though it makes sense in some study contexts to include clinical measures as primary outcomes with PROs as secondary outcomes, statistical power to detect a meaningful difference in secondary outcomes is often insufficient or not considered. Researchers might benefit from guidance for designing trials when clinical and patient reported outcomes are selected as coprimary outcomes, including how to resolve differences in sample size requirements, how to deal with multiplicity, and how to engage with patients to inform and justify the choice of target difference.

We recommend that funding agencies, institutions, and journal editors adopt explicit policies to encourage researchers to consider incorporating PROs and engage patients in pragmatic trials. Journal editors and peer reviewers should require explicit reporting of whether and how patients and members of the public were engaged in the trial design and conduct and require that authors provide clearer justification for the choice of primary outcome and the target differences in reporting results from pragmatic trials. Journals could mandate that authors use SPIRIT-PRO checklist or protocol template [40] or report patient and public engagement [50] to improve reporting and transparency around use of PROs. Trialists and methodologists should give greater consideration to choosing coprimary outcomes, with adjustment for multiplicity as appropriate, in trials where both clinical and patient perspectives are important in informing treatment decisions. Institutions can provide resources to promote and support the identification and recruitment of patient partners in their research, methodological and analytical support for use of PROs in pragmatic trials, and support for the development of tools to help patient partners participate fully in research. Researchers can equip patient partners with tools to understand and contribute to the design of studies that include PROs, such as the web tool created by Cruz Rivera and colleagues [51], which aims to supports dissemination and uptake of the SPIRIT-PRO extension by patient partners. Given that researchers can be incentivized to use specific methods or measure certain outcomes according to funding or publication calls, there is an opportunity for funders to support and promote the use of patient and public engagement and PROs in pragmatic trials. These strategies are likely to increase the applicability of pragmatic trials to clinical decision-making and patient preferences by encouraging the inclusion of patients and stakeholders in health research.

[END]

[1] Url: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1003896

(C) Plos One. "Accelerating the publication of peer-reviewed science."
Licensed under Creative Commons Attribution (CC BY 4.0)
URL: https://creativecommons.org/licenses/by/4.0/


via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/