(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Leveraging mHealth usage logs to inform health worker performance in a Resource-Limited setting: Case example of mUzima use for a chronic disease program in Western Kenya [1]

['Simon Savai', 'Institute Of Biomedical Informatics', 'Moi University', 'Eldoret', 'Jemimah Kamano', 'School Of Medicine', 'Moi Teaching', 'Referral Hospital', 'Lawrence Misoi', 'Peter Wakholi']

Date: 2022-11 

Abstract Background Health systems in low- and middle-income countries (LMICs) can be strengthened when quality information on health worker performance is readily available. With increasing adoption of mobile health (mHealth) technologies in LMICs, there is an opportunity to improve work-performance and supportive supervision of workers. The objective of this study was to evaluate usefulness of mHealth usage logs (paradata) to inform health worker performance. Methodology This study was conducted at a chronic disease program in Kenya. It involved 23 health providers serving 89 facilities and 24 community-based groups. Study participants, who already used an mHealth application (mUzima) during clinical care, were consented and equipped with an enhanced version of the application that captured usage logs. Three months of log data were used to determine work performance metrics, including: (a) number of patients seen; (b) days worked; (c) work hours; and (d) length of patient encounters. Principal findings Pearson correlation coefficient for days worked per participant as derived from logs as well as from records in the Electronic Medical Record system showed a strong positive correlation between the two data sources (r(11) = .92, p < .0005), indicating mUzima logs could be relied upon for analyses. Over the study period, only 13 (56.3%) participants used mUzima in 2,497 clinical encounters. 563 (22.5%) of encounters were entered outside of regular work hours, with five health providers working on weekends. On average, 14.5 (range 1–53) patients were seen per day by providers. Conclusions / Significance mHealth-derived usage logs can reliably inform work patterns and augment supervision mechanisms made particularly challenging during the COVID-19 pandemic. Derived metrics highlight variabilities in work performance between providers. Log data also highlight areas of suboptimal use, of the application, such as for retrospective data entry for an application meant for use during the patient encounter to best leverage built-in clinical decision support functionality.

Author summary Mobile health (mHealth) applications have gained significant penetration to support health care in low- and middle-income countries. Beyond improving care, these applications can help to strengthen the health system, but are currently not optimally employed for this goal. We explored whether we could leverage usage log data, which were captured as health providers were using a mobile application called mUzima, to inform health worker performance in Western Kenya. We were able to reliably demonstrate that the log data can provide detailed information of the days and hours worked, number of patients seen per day, and how the application was being used to inform many areas of improvement related to health worker performance. We also observed large differences in work performance between health providers, and work performed outside of official work hours and on weekends. Some of the health providers also used the application sub-optimally for entering patient data after the patient visit, as opposed to using the application during the patient encounter to best leverage built-in clinical decision support functionality. This study offers an approach to cost-effectively augment health worker supervision mechanisms that have been particularly challenging during the COVID-19 pandemic and for providers distributed over wide geographical areas.

Citation: Savai S, Kamano J, Misoi L, Wakholi P, Hasan MK, Were MC (2022) Leveraging mHealth usage logs to inform health worker performance in a Resource-Limited setting: Case example of mUzima use for a chronic disease program in Western Kenya. PLOS Digit Health 1(9): e0000096. https://doi.org/10.1371/journal.pdig.0000096 Editor: J. Mark Ansermino, University of British Columbia, CANADA Received: April 30, 2022; Accepted: July 25, 2022; Published: September 1, 2022 Copyright: © 2022 Savai et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability: All data are contained in the manuscript and Supporting Information files. Funding: This work was made possible by the support of the American people through the United States Agency for International Development (USAID, grant number 7200AA18CA00019) and the Norwegian Agencies for Development Cooperation under the NORHED program (Norad: Project QZA-0484). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: I have read the journal’s policy and the authors of this manuscript have the following competing interests. At the time of submission MW was a Section Editor for PLOS Digital Health.

Introduction Health service delivery in low- and middle-income countries (LMICs) has often been characterized by inadequate health-worker performance, resulting in poor quality and low uptake of health services [1]. Poor health-worker performance is reflected in under-achievement towards clinical targets, absenteeism, low motivation, poor service quality, and fabrication of health data [2–5]. Among reasons cited as contributors to this poor performance include: inadequate numbers of health personnel leading to work overload [6], poor working conditions [2]. and inadequacy of supervision [6]. While supportive supervision is considered an essential intervention for improving health-worker performance [1,7,8], such interventions are only effective with consistent and targeted interactions between workers and their supervisors. Unfortunately, in many LMIC settings, supportive supervision can be difficult to achieve. Oftentimes, the number of supervisors is inadequate to support all workers, with supervisors typically responsible for covering multiple facilities and wide geographical areas. The challenges around direct monitoring of work performance and supportive supervision are further exacerbated with emergence of the COVID-19 pandemic that has limited travel and in-person contact between personnel. With restrictions and lockdowns imposed in many localities, it becomes difficult for supervisors to travel to communities to support workers, and vice versa. It is imperative that innovative mechanisms for effective performance monitoring and supportive supervision are explored. These approaches would require judgments to be made on a continuous basis, with availability of good quality and timely information at both individual health worker and at group levels. mHealth technologies are now in broad use to support health workers in LMICs [9–12]. In many settings, workers and their supervisors use smartphone-based applications (apps) primarily for care coordination, data capture, retrieval of patient data, and decision support. [9,10,13] The apps can also support many other functions such as secure clinical messaging, tele-consultation and geo-location services. These mHealth solutions have an ability to collect paradata, defined as “process data documenting users’ access, participation, and navigation through an mHealth application” [14–16]. Unfortunately, despite the rich quality of paradata that can be securely collected through usage logs from mHealth apps used by health workers, these data have not been leveraged to inform health work-performance and for supportive supervision. In fact, to date, the use of mHealth paradata have only been limited to evaluations of users’ engagement with mHealth applications [14,17]. We hypothesized that mHealth applications, through collected paradata could be used to improve performance monitoring and supportive supervision. In this paper, we describe an evaluation that uses mHealth application-derived logs (paradata) from a demonstrative application, mUzima [18], to inform work patterns for health care workers in an LMIC setting who do not have frequent direct in-person contact with their supervisors. The employed approach has broad applicability across mHealth applications, extending the use of mHealth solutions to support health systems strengthening initiatives in LMICs.

Discussion To our knowledge, this is the first study to leverage mHealth usage logs from healthcare providers in LMICs to inform work patterns and work performance by health workers. Comparison of mHealth-derived usage log metrics against EMR-derived metrics showed a strong positive correlation between number of days worked per participant. This supported reliability of leveraging usage logs for performance evaluation. mHealth-derived usage logs have particular relevance for metrics that are usually not collected or available in EMRs, such as length of patient encounter, workday length, and work hours by providers. Even for metrics such as number of patients seen, log data perform better as they also capture incomplete encounters. The described work provides an additional approach that can be used to evaluate work performance. Historically, work performance evaluations for health providers have involved time-and-motion studies, which often require human observers and/or manual recording of activities by the providers being assessed [30–32]. Further, use of mHealth paradata for work performance assessment innovatively extends role of mHealth solutions to better support key functions of information technology in LMICs, which is to strengthen the health systems [13,33,34]. By re-using paradata collected as providers use the mHealth application, this approach promises to be cheaper and more scalable than traditional approaches that require human involvement to assess work patterns. However, use of paradata must take into strong consideration the ethical, legal and social implications (ELSI) of paradata use. Of particular importance is the need for informed consent for the providers, with mechanisms adopted to ensure that the paradata and knowledge derived from them are used strictly to inform and incentivize care providers and for supportive supervision, and never used for punitive purposes. In the current study, we observed that healthcare providers often worked outside regular work hours and during weekends. Having this knowledge on hand equips care programs to investigate factors that contribute to these work patterns and seek remedies. It is possible that some health workers are simply overwhelmed during the day and resort to taking work home with them. Alternatively, some workers might be slow or uncomfortable with using mHealth technologies as a point-of-care system during patient visits. As such, the providers might prefer to retrospectively enter clinical data in the application–and this would lead to longer work hours, risking provider burnout. The finding that the mHealth application was not being used as expected (i.e., as a point-of-care system) has important implications—it means that the care providers are not taking advantage of real-time computerized clinical decision support features within the application that are relevant to improving quality of care [25]. These decision-support features have particular relevance in LMIC settings that employ task-shifting of care services to lower-cadre staff. Differences between providers in number of days worked, average number of patients seen per day and hours worked offer insights on areas where individual health worker performance can be improved. In the age of COVID-19 pandemic, where in-person supervision can be particularly challenging in some settings, it is important to have mechanisms that provide insights into worker performance for timely supportive interventions. Further, the derived paradata-derived work performance metrics (e.g., patients seen, days worked and work day length) can be innovatively leveraged to improve performance, motivation and self-efficacy of health workers working in remote facilities through approaches such as gamification and ecological momentary interventions [35–37]. Several limitations in our study deserve mention. The generalizability of our findings is limited by the fact that it involved one mHealth application, a few facilities in a single country, and with limited number of healthcare providers. Our evaluation also only lasted for a short period of time, and we cannot account for possible changes in work performance occasioned by other factors. However, the study achieved its main goal of evaluating feasibility of using mHealth-derived paradata to evaluate health worker performance and provides a re-usable approach for other mHealth applications and settings. Results of this study have been shared with the clinical team to help inform work patterns and approaches for improving them. As the next steps, we will employ qualitative approaches to better understand reasons for the observed work patterns and for the variations in performance between providers. We will also leverage the paradata-derived metrics to provide real-time visualization to providers for insights on their own individual performance, and their performance relative to their colleagues. Finally, we plan to incorporate timely automated feedback and support mechanisms based on derived performance metrics to supplement support where direct supervision is a challenge.

Conclusions mHealth paradata can be used to derive work-performance metrics for providers working in disconnected LMIC settings. This approach extends the use of mHealth applications in the area of health systems strengthening and is easily scalable for supportive supervision.

Acknowledgments The authors thank the participating healthcare providers and clinics, as well as the Primary-health Integrated Care project for Chronic diseases (PIC 4 C) project at AMPATH, Kenya. The contents are solely the responsibility of the authors and do not necessarily represent the official views of USAID, Norad, or the United States Government.

[END]
---
[1] Url: https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000096

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/