- Study Protocol
- Open Access
The role of management on costs and efficiency in HIV prevention interventions for female sex workers in Nigeria: a cluster-randomized control trial
© The Author(s) 2018
- Received: 22 March 2018
- Accepted: 5 June 2018
- Published: 23 October 2018
While the world has made much global progress toward the reduction of new HIV infections, HIV continues to be an important public health problem. In the face of constantly constrained resources, donors and grantees alike must seek to optimize resources and deliver HIV services as efficiently as possible. While there is evidence that management practices can affect efficiency, this has yet to be rigorously tested in the context of HIV service delivery.
The present protocol describes the design of a cluster-randomized control trial to estimate the effect of management practices on efficiency. Specifically, we will evaluate the impact of an intervention focused on improving management practices among community-based organizations (CBOs), on the costs of HIV prevention services for female sex workers (FSW) in Nigeria. To design the intervention, we used a qualitative, design thinking-informed methodology that allowed us to understand management in its organizational context better and to develop a user-centered solution. After designing the suite of management tools, we randomly assigned 16 CBOs to the intervention group, and 15 CBOs to the control group. The intervention consisted of a comprehensive management training and a management “toolkit” to support better planning and organization of their work and better communication between CBOs and community volunteers. Both treatment and control groups received training to record data on efficiency—inputs used, and outputs produced. Both groups will be prospectively followed through to the end of the study, at which point we will compare the average unit cost per FSW served between the two groups using a quasi-experimental “difference-in-differences” (DiD) strategy. This approach identifies the effect of the intervention by examining differences between treatment and control groups, before and after the intervention thus accounting for time-constant differences between groups. Despite the rigorous randomization procedure, the small sample size and diversity in the country may still cause unobservable characteristics linked to efficiency to unbalanced between treatment and control groups at baseline. In anticipation of this possibility, using the quasi-experimental DiD approach allows any baseline differences to be “differenced out” when measuring the effect.
This study design will uniquely add to the literature around management practices by building rigorous evidence on the relationship between management skills and practices and service delivery efficiency. We expect that management will positively affect efficiency. This study will produce valuable evidence that we will disseminate to key stakeholders, including those integral to the Nigerian HIV response.
Trial registration This trial has been registered in Clinical Trials (NCT03371914). Registered 13 December 2018
- Economic evaluation
- Technical efficiency
- Community-based organizations
- Design thinking
- Female sex workers
Background and rationale
While over 19.1 billion USD in funding was available for the HIV response in low-and middle-income countries at the end of 2016 , HIV continues to be an important public health problem . In 2016, there were 1.8 million new infections and 1 million AIDS-related deaths globally . While treatment and transmission prevention programs have gone a long way to mitigate this, access to and retention in such programs—particularly in low-and middle-income countries—remain significant barriers to end the AIDS epidemic . This is particularly true for so-called key populations, such as men who have sex with men, injection drug users, and female sex workers (FSW), for whom factors like stigma and discrimination pose additional barriers to receiving prevention and treatment services.
Nigeria is a lower middle-income West African country that suffers a generalized HIV epidemic—the second largest globally . As of 2016, there were 3.2 million people infected with HIV, approximately a three percent prevalence . While unprotected heterosexual sex is the main mode of transmission in the country, the epidemic is particularly concentrated in high-risk populations like FSW; the prevalence among brothel-based workers is estimated to be 14.4% . FSWs are specifically targeted by current national efforts for HIV prevention, including the Minimum Prevention Package Intervention, or MPPI, which seeks to reduce the burden of HIV in key populations by providing a holistic suite of behavioral, biomedical, and structural components. Specifically, the package provides testing and counseling, peer education, sexually transmitted infection (STI) treatment, and community mobilization and advocacy .
Given the high burden of HIV in key populations and persistent funding scarcity, strategic investment that maximizes results is a priority for governments and funding agencies . Therefore, programs increasingly prioritize efficiency—broadly defined as the economic value of resources spent relative to the number of goods or services produced and expressed by the average cost per service, or unit cost. Understanding unit costs delivered (a useful measure of efficiency) and cost drivers has become an essential aspect of optimizing service delivery. Management is a determinant of productivity in manufacturing [9, 10], and is widely accepted as a key factor of success in business. Recent research has translated this insight to the health sector, finding that management can affect health service delivery and outcomes in private care settings . There is a paucity of evidence, however, about the relationship between management and healthcare outcomes in non-profit settings and in low-and middle-income countries. To address this gap, the National Institute of Public Health in Mexico, the Society for Family Health (SFH), and the National Agency for the Control of AIDS (NACA) in Nigeria, have partnered to design and implement the study, “Costs, efficiency and the role of management in HIV prevention interventions for Female Sex Workers in Nigeria”.
Study objectives and hypothesis
To measure the costs of implementing HIV prevention interventions and linkage to care services for FSWs in Nigeria; and
To experimentally evaluate the role of management practices on efficiency in community-based organizations (CBOs) in Nigeria.
We hypothesize that the intervention “package”—the data feedback, management training, and tools later described—will catalyze treatment sites to change key practices at their organizations, making service delivery more efficient.
This study is a cluster-randomized control trial (C-RCT) that examines the role of management practices on the costs of HIV services for FSWs in Nigeria. The project consists of two phases: (1) a design phase that uses qualitative methods to define and understand study population needs, and (2) an experimental phase, that implements a C-RCT. The design phase informed the content of a management intervention that will be implemented in CBOs and evaluated throughout the experimental phase.
Study setting and eligibility criteria
The study will collect and analyze efficiency data from CBOs across 14 states in Nigeria. To be included in this study, CBOs needed to (1) serve FSWs by providing HIV transmission prevention services, and (2) be funded through Society for Family Health (SFH) to offer services as part of the Global Fund or Strengthening HIV Prevention Services for Most at-Risk Populations (SHIPS for MARPs; USAID) grants. These sites offer counselling and testing, STI treatment, and HIV education services to brothels and hotspots in the surrounding area. All sites that met these criteria were included (n = 32).
Phase I: Formative intervention design
From the information gathered through journals, interviews, and observations, salient themes were found across the data—including meeting targets, funding constraints, and complications in transportation and communication. We delineated patterns, and used these to reveal “insights”. In design thinking, insights are design-ready statements that capture the latent needs or “pain points” of the user, for which we design a solution. For example, researchers found that program officers were very focused on and concerned about meeting productivity targets. While they understood the need to monitor field activities, they found the transportation and scheduling of supervision challenging, due to competing priorities. Ultimately, officers struggled to find the time to provide actionable feedback to volunteers. This insight highlights an opportunity for improvement: by building in new, realistic approaches for meaningfully communicating and interacting with volunteers, officers may be able to supervise more effectively, and volunteers may receive more useful feedback.
We then mapped the challenges, insights, and opportunities to a diagram that mirrors the program’s planning and implementation process (Fig. 2). We then used this map as a starting point to design 20+ “prototype” intervention components to address user needs. These prototypes were put to the test in three focus group sessions with stakeholders (Fig. 1). Feedback from these sessions served to finalize the management intervention. The intervention consisted of a management training, feedback loops, communication tools, goal setting, and planning tools (Fig. 2).
Phase II: Experimental design
Baseline training and data collection
Both treatment and control groups were invited to a training in Abuja, Nigeria—two managers (or “officers”) from each organization. The control group training took place on August 3rd and 4th, 2017. Trainees received one electronic tablet per CBO with Qualtrics Survey Software  installed. During the training, the participants were introduced to the costing component of the study, and were trained on how to collect data through the Qualtrics platform. Specifically, participants learned how to collect basic facility information, monthly inputs (condoms, test kits, etc.) used for the interventions, staff working for the interventions and their salaries, monthly rent and utilities expenses, participation in trainings, and monthly outputs produced (number of FSW served by the interventions). They subsequently collected and uploaded retrospective data for the period January 2016 to July 2017 onto the survey platform.
The treatment group training took place between August 7th and 11th, 2017, and the content was the same as that of the control group, plus an additional 3 days of management training. Topics included financial, workplace, and personnel management. The management training was broadly focused around “lean management”, a conceptual framework for improving the quality and efficiency of service delivery through small and pragmatic shifts in management practices (see Additional file 1). The attendees were also given several “management tools”—designed through the qualitative formative research—to help with their daily work. These tools included: (1) a whiteboard calendar to help them organize their constantly changing schedules, (2) “pledges” to help clarify the role of each volunteer, (3) materials and support on how to solicit meaningful feedback from volunteers, and (4) phones and phone credit to allow staff to communicate more often and effectively with volunteers in the field. As with the control group, the treatment group collected baseline data from their CBO records dated between January 2016 to July 2017.
Prospective intervention and data collection
Throughout the course of the study, the treatment group will receive: (1) the aforementioned “management tools” (calendar, pledges, feedback loops, and phones); (2) additional monthly “follow up” trainings via an online platform; and (3) monthly reports about their “performance”, (i.e. the data they send about inputs, outputs, and costs) (Additional file 2). The data reports will include information regarding their monthly average costs and number of FSWs reached per intervention, as well as comparisons of their performance relative to their (anonymized) peers. The online platform will provide a space through which the participants can posts successes, challenges, and solutions. This report will allow them not only to observe their performance in a specific period of time, but also to analyze changes in these outcomes over time.
Our primary outcome of interest is service delivery efficiency, as measured by the CBO-level average cost per service delivered, which is referred to as unit cost. Unit cost will be calculated for three services provided by the CBOs: (1) HIV testing and counselling, (2) HIV education, and (3) syndromic management of STIs.
Our goal in the project is to estimate the unit costs of three prevention interventions. In order to accomplish this, we will: (1) estimate the CBO-level total costs as the sum of the quantities of key inputs used to produce the interventions multiplied by their prices; and (2) divide the total costs by the total outputs produced per intervention, per CBO.
Staff time is allocated to specific interventions depending on the type of staff. Volunteer staff mostly work on a single intervention, and so their full time will be allocated to their corresponding intervention. In contrast, managerial staff work on various interventions, and their time will thus be allocated applying the same weight as seen above.
baseline information was collected on management practices through a survey of five-point Likert scale questions related to seven management dimensions. In total, the tool contained 82 questions. Better practices were associated with higher scores. This questionnaire consisted of different facets of the management process including: workplace organization, financial management, personnel management, supervision, transparency, accountability and community involvement. For each section of the questionnaire, one score was calculated by summing each of the values of selected options, which ranged from 0 to 5. The overall management score was then calculated as the mean score of the different management facets. In the future, the same questionnaire will be applied in two points during the implementation of the intervention—one in month six and at the end of the implementation in month 12.
Since we had a fixed number of sites (n = 32), we ran power calculations to examine the varying amounts of power that were achievable, given a variety of effect differences between treatment and control groups. The point estimate for log cost per client tested positive and accompanying standard error used in the calculations are based on data from a relevant previous costing study (ORPHEA: Assessing cost and technical efficiency of HIV prevention interventions in sub-Saharan Africa) [14–16]. Given sample size and assumptions, there was 80% power to detect a 1.09 unit difference (USD) in the log cost between the treatment and control groups.
Data collection methods
Data will be collected both retrospectively (for the baseline measurement) and prospectively, from three different sources: the CBOs, the SFH state-level offices, and the SFH headquarters (Fig. 3).
For retrospective data collection, CBO managers from both treatment and control groups were asked to bring all CBO records with them to the Abuja training, and to utilize their records to input data on staff salaries, supplies, clients served, rent, utilities, and trainings at their CBO for records dated between January 2016 to July 2017. Questionnaires were built from those used previously in the ORPHEA study . All participants were also asked to complete a questionnaire on management practices and job satisfaction.
For prospective data collection, training participants will collect data from their own CBO on monthly inputs (salaries, commodities, rent, utilities, and trainings), outputs (FSW served, by intervention) and management practices on their tablet with the installed survey software. Each month, participants will do this independently, but research assistants and the field data manager will review the data remotely and ask for clarification from the sites as needed. Additionally, research assistants will conduct in-person monitoring at the beginning of the prospective data collection period. The purpose of the monitoring visits will be to correct data issues and support the CBO participants on the technical aspects of tablet use as needed.
At the time of writing, baseline data cleaning was in process. Findings from both the baseline data and prospective data collection will be published in subsequent papers. For the purposes of baseline analysis, the data will be restricted to the 2016 fiscal year (January to December).
Our study will add to the literature around management practices by building rigorous evidence on the relationship between management and service delivery efficiency. We expect that management will positively affect efficiency. As with any study, however, it is not without its limitations. The small sample size is likely to lead to wide standard errors, and the bias toward brothel-based FSW services may make the results less generalizable to non-brothel based FSW interventions.
Notwithstanding, this study will produce valuable evidence that will be disseminated to key stakeholders, including implementing partners and other actors who are key in the Nigerian HIV response. This necessarily includes our implementing partners, Society for Family Health and the National Agency for the Control of AIDS (NACA). Cost estimates will be used nationally for planning purposes. We expect that results from the randomized evaluation will inform the design of future studies examining the role of management in service delivery efficiency.
SBA conceived the study, secured funding for the project and oversaw the study design, measurement, and implementation for training and data collection. NN and ASO conducted the randomization, liaised with country partners and conducted training. SHA, KO, JA, IE, and OA (NACA partners) helped to secure funding for the project, and supported the implementation of the training. DA prepared and implemented data collection. GO, AGO, CC (SFH partners) helped to secure funding for the project, and supported the implementation of the training. All authors read approved the final manuscript.
We are grateful to the Bill and Melinda Gates Foundation for their generous support of this project. We thank Lily Alexander and Gabriela Leon for their support in creating the figures used in the article, and Gabriela for her administrative support. We also sincerely thank Abigail Gutmann-Gonzalez for her help proofreading. Most importantly, are grateful to the participants in 31 CBOs providing services for FSW—their commitment and professionalism are inspiring to us.
The authors declare that they have no competing interests.
Access to data
Once final analyses are conducted, data will be made available on Harvard’s Dataverse repository.
The study did not record any name or any other identifier, so all information collected was kept confidential. Results of this study will not be presented in a way that individual CBOs can be identified.
Ethics approval and consent to participate
The protocol from this study was approved by the Ethics Committee of the National Institute of Public Health, Mexico (CI-1403), the Health Research Ethics Committee of the National Agency for the Control of AIDS, (NACA, FHREC/2016/01/58/08-08-16) and by the Nigerian Institute for Medical Research in Nigeria (NIMR, IRB/17/024). We obtained consent from participants during the training workshop. We did not anticipate any adverse events related to the conduct of the study. Notwithstanding, during training workshop, data collectors were instructed to report any perceived problems resulting from the study to the data manager. The supervisors would produce a written record of the reported problem to the country principal investigator (PI). The PI would be responsible for determining whether the report constitutes a problem and subsequently reporting to the ethics committee, to the Ministry of Health and/or other appropriate institutions.
Funding for this project provided by the Bill and Melinda Gates Foundation.
In case of any protocol changes, the following entities will be notified: the Ethics Committee of the National Institute of Public Health, Mexico; the Health Research Ethics Committee of the National Agency for the Control of AIDS, and the Nigerian Institute for Medical Research in Nigeria.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Fact sheet—Latest statistics on the status of the AIDS epidemic. http://www.unaids.org/en/resources/fact-sheet. Accessed 5 May 2018.
- Joint United Nations Program on HIV/AIDS (UNAIDS). The gap report. Geneva: Joint United Nations Program on HIV/AIDS (UNAIDS); 2014.Google Scholar
- Joint United Nations Program on HIV/AIDS (UNAIDS). UNAIDS | 2016–2021 strategy: on the fast-track to end AIDS. Geneva: Joint United Nations Program on HIV/AIDS (UNAIDS); 2015.Google Scholar
- HIV and AIDS in Nigeria. AVERT; 2015. https://www.avert.org/professionals/hiv-around-world/sub-saharan-africa/nigeria. Accessed 26 May 2018.
- UNAIDS. AIDSinfo | UNAIDS; 2016. http://aidsinfo.unaids.org/. Accessed 30 Apr 2018.
- Nigeria | UNAIDS. UNAIDS Country Statistics—Nigeria. http://www.unaids.org/en/regionscountries/countries/nigeria. Accessed 5 May 2018.
- National Agency for the Control of AIDS (NACA). National guidelines for implementation of HIV prevention programs for female sex workers in Nigeria; 2014. http://transglobalactivism.org/library/national-guidelines-for-implementation-of-hiv-prevention-programmes-for-female-sex-workers-in-nigeria/. Accessed 20 May 2018.
- Schwartlander B, Stover J, Hallett T, Atun R, Avila C, Gouws E, et al. Towards an improved investment approach for an effective response to HIV/AIDS. Lancet. 2011;377:2031–41.View ArticleGoogle Scholar
- Bloom N, Reenen JV. Measuring and explaining management practices across firms and countries. Q J Econ. 2007;122:1351–408.View ArticleGoogle Scholar
- Bloom N, Eifert B, Mahajan A, McKenzie D, Roberts J. Does management matter? Evidence from India. Cambridge: National Bureau of Economic Research; 2011.Google Scholar
- Bloom N, Sadun R, Van Reenen J. Does Management Matter in Healthcare. Lond Sch Econ Work Pap; 2013. https://people.stanford.edu/nbloom/sites/default/files/dmm_health.pdf. Accessed 20 Jan 2017.
- Brown T, Wyatt J. Design thinking for social innovation. Dev Outreach. 2010;12:29–43.View ArticleGoogle Scholar
- Qualtrics Survey Software. Provo, Utah, USA: Qualtrics Labs, Inc.; 2016. http://www.qualtrics.com. Accessed 12 Sep 2016.
- Bautista-Arredondo S, Sosa-Rubí SG, Opuni M, Contreras-Loya D, Kwan A, Chaumont C, et al. Costs along the service cascades for HIV testing and counselling and prevention of mother-to-child transmission. AIDS Lond Engl. 2016;30:2495–504.View ArticleGoogle Scholar
- Bautista-Arredondo S, Colchero MA, Amanze OO, Hera-Fuentes GL, Silverman-Retana O, Contreras-Loya D, et al. Explaining the heterogeneity in average costs per HIV/AIDS patient in Nigeria: the role of supply-side and service delivery characteristics. PLoS ONE. 2018;13:e0194305.View ArticleGoogle Scholar
- Bautista-Arredondo S, Sosa-Rubi SG, Opuni M, Kwan A, Chaumont C, Coetzee J, et al. Assessing cost and technical efficiency of HIV prevention interventions in sub-Saharan Africa: the ORPHEA study design and methods. BMC Health Serv Res. 2014;14:599.View ArticleGoogle Scholar
- StataCorp. STATA statistical software: release. College Station: StataCorp; 2013.Google Scholar
- Gertler PJ, Martinez S, Premand P, Rawlings LB, Vermeersch CM. Impact evaluation in practice. Washington, D.C.: World Bank; 2010.View ArticleGoogle Scholar