Abstract

1. Introduction

Policy makers are increasingly looking to develop ‘evidence‐based’ priority‐setting frameworks for the health sector that incorporate value for money criteria. This is especially important in low‐income and lower‐middle‐income countries (LMICs) where resources are highly constrained and public provision of a basic package of healthcare services is often beyond reach. The budgeting process in many LMICs requires that government departments prepare their plans and budgets based on prioritisation of interventions to support the achievement of the Millennium Development Goals, Sustainable Development Goals and the pursuit of Universal Health Coverage (Tangcharoensathien et al., 2013). From an economic perspective, priority setting can be seen as an ‘explicit’ process for choosing the optimal portfolio of programmes from a limited national healthcare budget (Hauck et al., 2004). Priority setting can occur at the global, regional, and national or subnational level and involves both the identification and weighting of criteria and a framework for establishing priorities (Oxman et al., 2006). Put simply, it is about who gets what at whose expense (Williams, 1988).

A range of frameworks has been used to set priorities on the basis of value for money. One approach has been to compare the cost‐effectiveness of different interventions using a quality‐adjusted life year (QALY) league table. This involves ranking interventions according to their ‘incremental cost per QALY’ and then allocating funds starting at the top of the list and working down, in theory, stopping once the prevailing budget has been exhausted (Mauskopf et al., 2003; Gerard and Mooney, 1993). Alternatively, such decisions are made with reference to a threshold or ceiling, whereby all interventions below should be funded and those above not (Marseille et al., 2015; Cleemput et al., 2011). For example, generalised cost‐effectiveness ratios, initiated by the World Health Organisation, compare the cost and health benefits of packages of different interventions against a situation in which those interventions did not take place (Murray et al., 2000). Using a threshold based on multiples of gross domestic product per capita, the results are then put in three categories: very cost‐effective, cost‐effective, and not cost‐effective (Hutubessy et al., 2002).

Another approach is program budgeting and marginal analysis (PBMA) (Mooney, 2002; Mitton and Donaldson 2004). PBMA provides an information framework to allow a picture of where resources are currently going (programme budgeting) and thereafter looking at whether a movement of resources from one programme to another might increase total benefits (marginal analysis). There are even broader frameworks that take account of efficiency criteria, including option appraisal, multi‐criteria decision analysis (MCDA) and multiple attribute utility analysis (Mooney et al., 2012). MCDA, in particular, is increasingly being used to evaluate healthcare interventions and involves the systematic definition of criteria relevant to a decision, the performance of options against these criteria and often the weighting of criteria to produce an overall value metric (Baltussen et al., 2006; Marsh et al., 2014). According to Mooney et al. (2012), approaches like these represent ways of allowing often‐difficult‐to‐measure attributes (such as patient reassurance) to be included and weighted alongside efficiency criteria. Guidance is also available on how equity criteria can be considered in addition to cost‐effectiveness analysis for priority setting (Norheim et al., 2014; Cookson et al., 2009).

What is our definition of priority setting? What makes these examples of priority‐setting frameworks and sets them apart from all economic evaluation studies is that a funding decision is central to the analysis as opposed to an intervention. These studies seek to identify priority areas for investment (or disinvestment) often taking into account a range of organisational objectives including (but not limited to) efficiency. While economic evaluation studies in health provide evidence based on the relative cost of achieving units of health gain, priority‐setting studies centre on the ultimate investment decision. In such decisions, there are potentially multiple inputs and influences, one of which is cost‐effectiveness information. Several reviews of priority‐setting approaches that incorporate efficiency criteria are available for high‐income countries (HICs; see, e.g. Mooney et al., 2012; Robinson et al., 2012; Bate and Mitton, 2006; Noorani et al., 2007; Hauck et al., 2004). While less is known about comparable frameworks used in LMICs, recent informal reviews point to a growing level of interest in evidence‐based priority setting by policy makers in these countries (see, e.g. Glassman et al., 2013; Hipgrave et al., 2014). This review builds on the existing priority‐setting literature by concentrating on LMICs and, secondly, by adopting a transparent and explicit approach to the selection and assessment of methodological frameworks (which incorporate economic evaluation evidence) for priority setting in health care. A systematic review of wider issues such as acceptability and stakeholder engagement associated with priority setting was outside the remit of this review.

2. Methods

The review focused on papers describing frameworks for setting healthcare priorities and included some assessment of relative costs and consequences of healthcare interventions.

As the term ‘priority setting’ may not be universally used, we adopted a comprehensive search strategy using a series of broad but related terms and then carried out extensive manual reviews to filter out those papers that did not meet our definition of priority setting (as set out in Section 1). The authors initially reviewed 20 papers that they felt were relevant to the research question. The subject headings of these papers were reviewed in MEDLINE and Embase in order to ascertain the pertinent subject headings and significant text words/keywords from the abstracts. Subsequently, a template search was developed in MEDLINE (Ovid) that included three concepts: priority setting, cost, and low‐income or lower‐middle‐income countries. Each concept was composed of MeSH subject headings and keywords in order to develop a sensitive search strategy. The low‐income and lower‐middle‐income concept included country names based on the World Bank's (2014) defined list. The MEDLINE search was translated into the appropriate terms in Embase (Ovid), PubMed and EconLit (EBSCOhost). The search strategy (including a complete list of search terms) is shown in Web appendix. This was supplemented by a search of relevant papers' reference lists. All references were exported to RefWorks.

Papers were eligible for inclusion if they were published in the peer‐reviewed literature between 2004 and 2014, were complete (e.g. conference abstracts were excluded) and were reported in the English language. All paper types (with the exception of books) were included. Eligible papers also needed to report on at least one ‘low’‐income or ‘lower‐middle’‐income country as defined by the World Bank (World Bank, 2014). Studies that referred to ‘developing countries’ were also included despite not specifying an individual country. Eligible papers needed to report on priority‐setting frameworks or approaches that considered a portfolio of interventions for a particular disease, programme or entire health sector.

Two independent reviewers (V.W. and C.M.) screened the first 83 articles against the inclusion criteria to determine inter‐rater reliability of the reviews. Any disagreement in article selection was discussed until consensus was reached. Agreement was assessed using a simple kappa analysis (Cohen, 1960). Substantial agreement (defined as kappa exceeding 0.6) was desired for a decision to continue with a single reviewer (Landis and Koch, 1977). The calculated kappa score was 0.78, classifying the level of agreement as ‘almost perfect’ (Landis and Koch, 1977). Screening for article selection was then completed by one reviewer (V.W.). A second reviewer (C.M.) was consulted when clarity was needed. Both reviewers read the full text of the selected articles and resolved disagreements by consensus. Reasons for inclusion and exclusion were recorded.

A data extraction form was designed and piloted for this study which included fields for the following descriptive data: date and location of the study; type of paper (modelling, commentary, review, framework development, etc.); country in which the study was undertaken; scale (global, regional, national or subnational); type of health interventions; criteria used for priority setting; technique used to measure efficiency; name of priority‐setting process; and main data sources. The data were initially extracted by one reviewer (V.W.). A second reviewer (C. M., L. C. or T. D.) independently extracted the same data for all papers, and any disagreements were resolved. All variables included in the data extraction form are defined in Table 1.

Uniformly recognised criteria for appraising studies in this field are not available. Guidelines developed for PBMA by Peacock et al. (2010) were adapted for the appraisal of studies in this review. Five key questions were asked of each study: (i) Was the perspective of the study determined? (ii) Was a sensitivity analysis performed? (iii) Was affordability of the health interventions considered? (iv) Did the study consider releasing or redeploying resources (as well as investment)? and (v) Was the study embedded in the local policy and planning context with involvement by local decision‐makers? Table 4 summarizes the reviewer responses to these appraisal questions for each of the included papers.

No ethical approval was required for this systematic review.

3. Results

3.1. Study selection

The search in Embase, MEDLINE, Econlit and PubMed identified 3968 articles, and a further 21 articles were identified through manual searching of reference lists. After the removal of duplicates, 3061 titles and abstracts were screened against the inclusion criteria. Of these, 123 full articles were assessed for eligibility with 36 articles meeting the inclusion criteria and kept for data abstraction (Figure 1). A complete list of the papers can be found in Appendix 1. The main reasons for excluding studies was that despite mentioning priority setting, costs and cost‐effectiveness in the abstract or paper, they did not focus on the health sector or the funding of a package of healthcare interventions.

Open in a separate window

Figure 1

Selection of studies flow chart. *Authors contacted to confirm that a full paper was not available. **Priority setting papers that do not focus directly on health. ***Information captured in other papers. PS, priority setting; CE, cost effectiveness

3.2. Study characteristics

Table 2 shows that of the 36 eligible papers, 15 involved studies of priority setting at the national level (11 from African countries), 14 at the regional level and 9 at the global level. The majority of studies (19) took a modelling approach.

Table 2

Priority‐setting studies in LMICs: overview of peer‐reviewed papers

Article	Author/year	Paper type	Primary aim	Interventions	Scale (location)
#1	Baltussen et al, 2007	Exploratory	Show how multiple criteria can be used to guide the priority‐setting process.	Lung health programme	National (Nepal)
#2	Hansen & Chapman, 2008	Exploratory	Assess feasibility of conducting cost‐effectiveness analyses for a large number of health interventions in a developing country.	65 curative interventions for common health problems and preventative interventions	National (Zimbabwe)
#3	Kapiriri & Norheim, 2004	Exploratory	Explore stakeholders' acceptance of criteria for setting priorities for the healthcare system.	Not specified	National (Uganda)
#4	Kase, 2006	Strategic planning	Describe process for designing Government Medium Term Expenditure Framework.	Essential health services (wages/salaries), basic system support and interventions (malaria, immunisation, safe motherhood, outreach, supervision)	National (Papua New Guinea)
#5	Baltussen, 2006	Exploratory	Identify priority interventions under two assumptions: public spending should be targeted at the whole population or the poor only.	All priority interventions listed in 2002 World Health Report	National (Ghana)
#6	Baltussen et al., 2006	Exploratory	Show how multiple criteria can be used to guide the priority‐setting process.	Set of hypothetical health interventions – taken from 2002 World Health Report	National (Ghana)
#7	Chisholm et al., 2008	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Schizophrenia	Regional Americas, Africa and South‐East Asia+National (Chile,a Nigeria, Sri Lanka)
#8	Diaby & Lachane, 2011	Exploratory	Evaluate the feasibility of developing a new formulary for a health mutual fund.	Formulary for drug reimbursement	National (Cote D'Ivoire)
#9	Evans, Lim et al., 2005	Economic modelling	Summarise key findings from a series of papers on the cost‐effectiveness of strategies to achieve the millennium development goals for health.	Maternal and neonatal health, child health, HIV and Aids, malaria and tuberculosis	Regional (sub‐Saharan Africa and South East Asia)
#10	Ginsberg et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Breast, cervical and colorectal cancers	Regional (sub‐Saharan Africa and South East Asia)
#11	Jehu‐Appiah et al., 2008	Exploratory	Illustrate how multiple criteria can be used to guide the priority‐setting process.	Child health, reproductive health, and communicable diseasesb	National (Ghana)
#12	Laxminarayan et al., 2006	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	94 clusters of interventions – representing 218 interventions covering: tuberculosis, HIV/AIDS, illness and mortality in children, tropical diseases, reproductive health, nutrition, cancer, neurological disorders, cardiovascular disease, injury prevention, surgery, alcohol and tobacco use, delivery of interventions and strengthening health systems.	Regional (South Asia and sub‐Saharan Africa)+Global (LMICs)
#13	Makundi et al., 2007	Exploratory	Test out the ‘balance sheet method’ for priority setting, which incorporates both scientific evidence and public values.	Integrated Management of Childhood Illness (IMCI), safe water, HIV, tuberculosis, malaria	National (Tanzania)
#14	Venhorst et al., 2014	Exploratory	Develop rating tool for policy makers to prioritise interventions based on multiple criteria.	Breast cancer	Global (LMICs)
#15	Marsh et al., 2014	Review	Document studies that have used multi‐criteria decision analysis to set healthcare priorities and lessons learnt.	Pharmaceuticals, public health interventions, screening, surgical interventions, and devices	Regional+national
#16	Chao et al., 2014	Review & economic modelling	Extract and appraise economic assessments for their methodological quality.	Surgery	Regional+national
#17	Diaby et al., 2011	Review & framework development	Review processes used by high‐, middle‐ and low‐income countries, to prioritise medicines for reimbursement.	Formulary for drug reimbursement	National (Canada, US, UK, France, Germany, Brazil, South Korea, Ghana)
#18	Simons et al., 2011	Economic modelling	Explain how a disease‐intervention planning tool can be used to prioritise health interventions and review of preliminary user experience.	Measles	Not applicable
#19	Canning, 2006	Review + exploratory	Explore the economic case for prioritising prevention over the treatment of HIV/AIDS. Compares cost‐effectiveness criterion to other criterion for setting priorities.	HIV prevention and treatment	Regional (Africa)
#20	Whittington et al., 2012	Economic modelling	Illustrate the challenges and uncertainties of setting priorities amongst competing interventions at the global level using economic evidence.	Water, sanitation and preventive health interventions (insecticide‐treated bed nets, cholera vaccination).	Global (LMICs)
#21	Madi et al., 2007	Exploratory	Describe a process for involving key stakeholders to elicit and prioritise health interventions.	Maternal	National (Burkina Faso, Ghana and Indonesia)
#22	Adam et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Maternal and neonatal health	Global (LMICs)
#23	Baltussen et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Tuberculosis	Global (LMICs)
#24	Baltussen & Smith, 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Vision and hearing loss	Regional (sub‐Saharan Africa and South East Asia)
#25	Chisholm, Baltussen, et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Non‐communicable diseases and injuries	Regional (sub‐Saharan Africa and South East Asia)
#26	Chisholm, Naci, et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Road traffic injuries	Regional (sub‐Saharan Africa and South East Asia)
#27	Chisholm & Saxena 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Neuropsychiatric conditions	Regional (sub‐Saharan Africa and South East Asia)
#28	Darmstadt et al., 2005	Review and modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Neonatal	Global (LMICs)
#29	Edejer et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Child health	Global (LMICs)
#30	Morel et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Malaria	Global (LMICs)
#31	Ortegon et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Cardiovascular disease, diabetes, tobacco use	Regional (sub‐Saharan Africa and South East Asia)
#32	Stanciole et al., 2012	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Chronic obstructive pulmonary disease and asthma	Regional (sub‐Saharan Africa and South East Asia)
#33	Hogan et al., 2005	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	HIV/AIDS	Global (LMICs)
#34	Cecchini et al., 2010	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Chronic diseases	National (Brazil, China, India, Mexico, Russia, South Africac)
#35	Chisholm, Doran et al., 2006	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Alcohol, tobacco and illicit drug use	Regional (America, Europe and South East Asiad)
#36	Gureje et al. 2007	Economic modelling	Identify a package of disease‐specific health care interventions for investment to inform policy discussion.	Mental health	National (Nigeria)

Open in a separate window

LMICs, low‐income and lower‐middle‐income countries; MCDA, multi‐criteria decision analysis.

^a Note that Chile is classed as a high‐income country while Nigeria and Sri Lanka are classified as ‘lower‐middle’‐income countries (World Bank, 2014).

^b These are the final set of interventions identified using MCDA. Original exercise included childhood diseases, communicable and non‐communicable diseases, reproductive health and injuries.

^c Only India is a “lower‐middle‐income” country and meets the eligibility criteria for this review.

^d These regions are defined by WHO as American sub‐region AmrB (countries with low rates of child and adult mortality, e.g. Brazil or Mexico); European sub‐region EurA (countries with very low child and adult mortality, e.g. France or Norway); and South East Asian sub‐region SearD (countries with high child and adult mortality, e.g. India or Nepal).

Table 3 shows that studies covered a broad range of health interventions with only two studies including health systems strengthening interventions related to financing, governance, information and human resources for health (#4 and #12). A little under half of all studies (39%) included at least one other criterion apart from efficiency for priority setting, most commonly ‘equity’ and ‘feasibility’. Most studies (91%) specified a measure of ‘efficiency’, commonly defined as cost per disability‐adjusted life year (DALY) averted (72%). Only one study calculated cost per QALY gained (#8), and one study estimated costs and benefits in monetary terms to derive a benefit–cost ratio (#20).

Table 3

Priority‐setting studies in LMICs: methods and data sources

Article	Criteria	Efficiency measure	PS approach	Highest priority interventions	Data source
#1	Efficiency, severity of disease, number of potential beneficiaries, age of beneficiaries, individual health benefits, poverty reduction	Cost‐effectiveness (cost per DALY averted)	DCE to derive criteria. Ranking interventions based on MCDAa	TB control, followed by oral rehydration therapy for diarrhoea and case management of pneumonia in child health and several interventions in HIV/AIDS. AIDS control including the provision of antiretroviral therapy.	Literature+expert opinion
#2	Efficiency, important health problems	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE (grouped by health problem)	Curative treatment at health centres and hospital outpatient departments of pneumonia, severe diarrhoeal diseases, peptic ulcer, dysentery, malaria, trachoma, schistosomiasis haematobium and glaucoma.	Literature+local survey data+expert opinion
#3	Severity of disease, benefit of the intervention; cost of the intervention, efficiency; quality of the data on effectiveness; the patients age; place of residence; lifestyle; importance of providing equity of access to health care and the community's viewsb	Cost‐effectiveness (no ratio given)	N/A (identify criteria only)	N/A (identify criteria only).	Local survey data
#4	Benefit the greatest number; impact greatly on morbidity and mortality; prevention focused; accessible and affordable; efficient; impact on performance; improve financial and management sustainability	Not specified	Explicit approach akin to PBMAc – used agreed criteria as basis for prioritising in 3 key areas and identify the necessary resource shifts	Malaria prevention (bed nets and selected spraying in the highlands), safe motherhood and family planning, immunisation, STI/HIV/AIDS.	Expert opinion
#5	Efficiency; poverty reduction; severity of disease; age of target group; budget impact; individual health effect	Cost‐effectiveness (no ratio given)	Ranking based on 2 steps: define who should be targeted (step 1), use DCE to derive weights for relative criteria for PS, then scored/mapped interventions against these to develop rank ordering of interventions to targeted groups (step 2)	Prevention of mother‐to‐child HIV/AIDS transmission and oral rehydration therapy to treat diarrhoea in childhood (whole population), case‐management of pneumonia in childhood (targeting the poor).	Literature+expert opinion
#6	Efficiency; poverty reduction; severity of disease; age of target group; budget impact; health effects	Cost‐effectiveness (cost per DALY averted)	DCE to derive criteria. Ranking interventions based on MCDA	Prevention of mother to child transmission in HIV/AIDS control and treatment of pneumonia and Diarrhoea in childhood.	Local survey data
#7	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEAd – ranking of interventions based on relative CE	Interventions using older antipsychotic drugs combined with psychosocial treatment, delivered via a community‐based service model.	Literature+local survey data+expert opinion
#8	Efficiency; severity of the condition; socioeconomic status; age group of patients	Cost‐effectiveness (cost per QALY gained)	Ranking of interventions based on MCDA	Antimalarials, treatments for asthma and antibacterials for urinary tract infection.	Literature+expert opinion
#9	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – Ranking of interventions based on relative CE	N/A (priority interventions can be found in individual papers from this series (#33–37)).	Literature+local survey data+expert opinion
#10	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Cervical cancer control – screening through cervical smear tests or visual inspection with acetic acid in combination with treatment. Colorectal cancer control – increasing the coverage of treatment interventions.	Literature+expert opinion
#11	Targeting vulnerable populations; efficiency, severity of disease; number of beneficiaries; diseases of the poor	Cost‐effectiveness (cost per DALY averted)	DCE to derive criteria. Ranking interventions based on MCDA	Childhood interventions, most interventions targeting communicable diseases and two reproductive health interventions (supervised deliveries and emergency obstetric care).	Literature+local survey data+expert opinion
#12	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	See full publication for priority interventions for 94 diseases and conditions.	Literature+local survey data+expert opinion
#13	Prevalence; disease burden; coverage; severity of disease; efficacy; efficiency; equity	Cost‐effectiveness (ratio not given)	Balance sheet methode to derive ranking of interventions	N/A (development and testing of a model for incorporating scientific evidence and societal values in priority setting).	Literature+local survey data+expert opinion
#14	Effectiveness; quality of the evidence; magnitude of individual health impact; acceptability; efficiency; technical complexity; affordability; safety; geographical coverage; accessibility	Cost‐effectiveness (ratio not given)	N/A (develop criteria only)	N/A (development of a rating tool to assess the impact of breast cancer interventions on multiple criteria).	Literature+expert opinion
#15	Efficiency; disease severity; treatment access; target population size; curative or preventative; budgetary and other practical constraints; evidence quality; political factors	Not specified	Ranking of interventions based on MCDA	N/A (summarises existing approaches for MCDA of healthcare interventions and lessons learnt).	Literature
#16	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Male circumcision in Mozambique and cataract repair in Nepal.	Literature+expert opinion
#17	Efficiency; severity of disease; capacity of the intervention to reduce poverty; age; anticipated health gains; financial impactf	Not specified	Ranking of interventions based on MCDA	N/A (summarises article #6).	Literature+expert opinion
#18	Efficiency	Cost‐effectiveness (cost per additional case, death, and DALY averted)	Ranking of interventions based on relative CE	N/A (tool for measles strategic planning).	Literature+expert opinion
#19	Efficiency	Cost‐effectiveness (cost per DALY averted and cost per infection averted)	Ranking of interventions based on relative CE	Mass media messages, peer education, condom distribution, treatment of sexually transmitted diseases for commercial sex workers, treatment of tuberculosis is highly cost‐effective in those who are HIV‐positive as well as the general population.	Literature+expert opinion
#20	Efficiency	Cost–benefit analysis (BCR)	Ranking of interventions based on BCRsg	Biosand filter and point of use chlorination interventions offer the largest benefits to a household's well‐being.	Literature
#21	Ability to meet national policy priorities; reduce maternal mortality and/or morbidity; improve services; efficient and financially sustainable	Cost‐effectiveness (ratio not specified)	Not specified	N/A (describes process involving key stakeholders to elicit and prioritise evaluation needs for safe motherhood).	Primary data
#22	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Community‐based newborn care package, antenatal care (tetanus toxoid, screening for pre‐eclampsia, screening and treatment of asymptomatic bacteriuria and syphilis), skilled attendance at birth, offering first level maternal and neonatal care around childbirth, emergency obstetric and neonatal care around and after birth.	Literature+expert opinion
#23	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Treating only smear‐positive cases, treatment for both smear‐positive and smear‐negative and extra‐pulmonary cases at a coverage level of 95%.	Literature+expert opinion
#24	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Treatment of chronic otitis media, extracapsular cataract surgery, trichiasis surgery, treatment for meningitis, and annual screening of school children for refractive error.	Literature+expert opinion
#25	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Disease clusters cover over 500 interventions. A subset of 53 interventions is deemed ‘highly’ cost‐effective. See full article for details.	Literature
#26	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Single most cost‐effective intervention varies by region. Combined intervention strategy that simultaneously enforces multiple road safety laws (e.g. the combined enforcement of speed limits, drink‐driving laws, and motorcycle helmet use).	Literature
#27	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Population‐based alcohol control (Africa), drug treatment of epilepsy in primary care (South‐East Asia).	Literature
#28	Efficacy; effectiveness; feasibility; efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Family care/low birthweight care, Emergency obstetric care, family care/low birthweight care+community‐based case management of pneumonia, Skilled maternal and immediate neonatal care, emergency obstetric care+corticosteroids for preterm labour+antibiotics for preterm premature rupture of membranes.	Literature+expert opinion
#29	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Fortification with zinc or vitamin A.	Literature+expert opinion
#30	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	High coverage with artemisinin‐based combination treatments.	Literature+expert opinion
#31	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Demand reduction strategies of the Framework Convention for Tobacco Control; combination drug therapy for people with a >25% chance of experiencing a cardiovascular event over the next decade, either alone or together with specific multidrug regimens for the secondary prevention of post‐acute ischaemic heart disease and stroke; and retinopathy screening and glycaemic control for patients with diabetes.	Literature
#32	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Low‐dose inhaled corticosteroids for mild persistent asthma.	Literature+expert opinion
#33	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Education and treatment of sexually transmitted infections for sex workers.	Literature+expert opinion
#34	Efficiency	Cost‐effectiveness (cost per DALY averted)	Ranking of interventions based on relative CE	Health information and communication strategies that improve population awareness about the benefits of healthy eating and physical activity, fiscal measures that increase the price of unhealthy food content or reduce the cost of healthy foods rich in fibre, regulatory measures that improve nutritional information or restrict the marketing of unhealthy foods to children.	Literature+expert opinion
#35	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	Nicotine replacement therapy or brief physician advice (individual level); taxation of alcoholic or tobacco products (population‐wide level).	Literature+expert opinion
#36	Efficiency	Cost‐effectiveness (cost per DALY averted)	GCEA – ranking of interventions based on relative CE	For schizophrenia: community‐based treatment with older antipsychotic drugs plus psychosocial support or case management. For depression, epilepsy, and alcohol use disorders: older antidepressants, with or without proactive case management in primary care, older anticonvulsants in primary care, and random breath testing for motor vehicle drivers.	Literature+local survey data+expert opinion

Open in a separate window

LMICs, low‐income and lower‐middle‐income countries; DALY, disability‐adjusted life year; PS, priority setting; CE, cost‐effectiveness; N/A, not applicable; BCR, benefit–cost ratios; DCE, discrete choice experiment.

^a MCDA (multi‐criteria decision analysis) involves describing criteria, arranging the criteria on a performance matrix and assigning ratings for each program option to aid transparent and consistent decision making (Baltussen et al., 2007).

^b High‐weight criteria. The authors also identified ‘average’‐weight and ‘low’‐weight criteria.

^c PBMA (programme budgeting and marginal analysis) priority‐setting toolkit that helps decision‐makers maximise the impact of healthcare resources on the health needs of a local population and examines how resources are currently spent and the costs and effects of changing spending patterns (Mitton and Donaldson, 2004).

^d GCEA – generalised cost‐effectiveness approach. Interventions classified into those are very cost‐effective, cost‐ineffective, and somewhere in between (Hutubessy et al., 2002).

^e Balance sheet method=model for incorporating scientific evidence and societal values in priority setting (Eddy, 1990).

^f Authors suggested criteria for LMICs.

^g While authors discuss how BCRs can inform priority ranking of interventions, data limitations prevent them from doing so in this context.

In reviews such as this where the topic and questions being asked are broad, heterogeneity is inevitable both in terms of methods and outcomes. Priorities were derived through a range of approaches including MCDA, generalised cost‐effectiveness approach (GCEA), balance sheet method and an approach akin to PBMA (Mooney, 2002). GCEA and MCDA were the most common approaches used in 40% and 18% of studies, respectively. All of the GCEA studies relied on data from the WHO Choosing Interventions that are Cost‐effective Project. Most studies relied on secondary data for priority setting and noted several gaps in the evidence base especially in relation to effectiveness data, which often resulted in country‐level studies relying on regional data or expert opinion. The priority interventions identified by the different studies were also diverse, representing mixes of curative and preventive actions and of population and individually focused interventions.

4. Discussion

The need for priority setting and trading‐off between potentially beneficial and life‐saving investments options is a universal problem. It is especially important in highly resource‐constrained settings where the most basic health services are sometimes unaffordable and where there is a heavy reliance on international donor support. As such, policy makers in LMICs and donors are increasingly looking to develop ‘evidence‐based’ approaches for setting investment priorities in the health sector. This review of the peer‐reviewed literature provides a comprehensive overview of the different frameworks designed to take account of the problem of resource scarcity and incorporate the assessment of relative costs and effects of interventions in LMICs.

This review shows that there is widespread acknowledgement that decisions on the allocation of scarce healthcare resources are shaped by a range of criteria and that economic evaluation information must be considered alongside other health system goals. Approximately half of the studies in this review included multiple criteria for priority setting. Some studies also reported different priorities based on efficiency versus non‐efficiency criteria, thereby highlighting the importance of specifying the relevant criteria at the outset of any priority‐setting exercise (Diaby et al., 2011; Johansson and Norheim, 2011). Overlap between different criteria was recognised as an additional methodological issue requiring greater attention by researchers. In the MCDA literature, for example, studies included related criteria such as ‘cost of treatment’, ‘effectiveness of treatment’ and ‘cost‐effectiveness of treatment’ (see, e.g. Marsh et al., 2014; Diaby et al., 2011).

In LMICs, the most common framework in the peer‐reviewed literature for explicit prioritising of healthcare interventions is the league table approach. These tables are typically based on MCDA where simultaneous criteria are used or on GCEA where only efficiency criterion is used. Cost per DALY averted is the dominant measure of cost‐effectiveness, and it is often justified on the grounds that almost 90% of the global burden of disease is accounted for by LMICs (Murray and Lopez, 1997; Zarate, 2007). PBMA, which creates a management process into which results from standard economic evaluations and other evidence can be incorporated (Ruta et al., 2005), has been widely used in HICs including Australia, Canada, New Zealand and the UK (Mitton et al., 2014) but has not established a footing in LMICs. In addition to the widespread problem of a lack of credible information, the legitimacy and capacity of institutions management processes are often weak. Criteria such as ‘feasibility’ and ‘sustainability’ frequently feature amongst priority‐setting criteria in LMICs, which again may reflect the constrained capacity of local priority‐setting institutions.

This review also revealed a distinction between frameworks developed for use at the global and regional level versus those designed to set priorities at the country level. There have been an increasing number of sector‐wide frameworks such as The WHO Choosing Interventions that are Cost‐Effective Project and the Disease Control Priorities Project that have been led largely by the international community seeking to identify the most cost‐effective interventions for achieving targets such as the Millennium Development Goals and Universal Health Coverage. These two projects represented approximately half of all studies included in this review. It has been argued that while these analyses provide an indication of value for money, they do so at the expense of diversity or specificity of individual country contexts (Makundi et al., 2007). In contrast, country‐level priorities are often based on more consultative processes that require significant investment in building relationships and engaging and empowering stakeholders (Madi et al., 2007). To date, there has been little recognition in the priority‐setting literature of the diversity of stakeholders involved in both the funding and implementation of healthcare interventions in LMICs. Most country‐level priority‐setting analyses make the implicit assumption that the national government has over‐arching responsibility for priority setting, but in reality, substantial healthcare funds are already earmarked for specific interventions or disease areas based on the priorities of international or transnational non‐governmental organisations. Consequently, priority‐setting frameworks often fail to reflect the multifaceted nature of priority setting in many LMICs. Global institutions responsible for setting priorities should be lobbied to use and disclose their frameworks especially those receiving public money and be made accountable for how their activities might distort national‐level and local‐level priority setting. Recent initiatives such as the Asia‐Pacific Regional Capacity Building for Health Technology Assessment, a collaboration between the International branch of the National Institute of Clinical Excellence (UK) and countries in the region, are designed to provide technical support to governments in LMICs to help build up their priority‐setting capacity and to support multi‐stakeholder engagement (Teerawattananon et al., 2014; NICE International, 2014).

A lack of cost‐effectiveness evidence for many key health interventions in LMICs is another limitation to explicit priority setting at both the global and country level. The results of this review show that most studies base their analyses on extremely limited efficacy and effectiveness data or on expert opinion. Moreover, extrapolation of best available international evidence – from higher to lower income resource settings is often required but can be problematic due to clear differences in health systems and epidemiology (Adam et al., 2005; Chisholm and Saxena, 2012). However, priority‐setting approaches examined in this review tended to lack any account of uncertainty, utilising single‐point estimates for effectiveness and cost‐effectiveness.

Given these information gaps, priority‐setting studies in LMICs utilise economic evaluation data from many different settings. In doing so, methodological inconsistencies are inevitable. Interventions that are deemed to be cost‐effective in one setting may not be in another especially where coverage rates, disease incidence, relative prices and costs are different. These factors affect the applicability and reliability of evidence underpinning priority‐setting approaches. However, extrapolations are a necessary part of these exercises, and as such, evaluators working with decision‐makers need to adapt to the use of imperfect, ‘broad brushed’ data in a pragmatic way to inform a decision at hand (Mooney, 1994). A bigger problem observed in several studies was the practice of excluding potential interventions from consideration due to a lack of data (Adam et al., 2005). Because ultimately priority setting is about assessing as many competing options as feasible, such omissions undermine the validity of these exercises by introducing clear bias into their recommendations. Future initiatives need to be based more strongly on pragmatic responses to imperfect data.

This review also highlighted that many priority‐setting exercises are based on unrealistic assumptions about the overall functioning of health systems in LMICs. The expected costs and effects of priority health interventions depend on accompanying investments in health systems to support appropriate levels of human resources, timely referral systems, efficient and equitable financing systems and adequate institutional infrastructure (Laxminarayan et al., 2006 ). A common weakness of many priority‐setting initiatives is that investments in such infrastructure are not formally deliberated upon and included amongst the options being considered nor are the potential unintended consequences for health systems associated with shifting resources, such as staff from one programme to another. Priority‐setting exercises should not assume, for example, that a skilled workforce is available to deliver priority health interventions in LMICs nor should it assume that shifting health workers from one programme to another is a costless exercise.

Another important message emerging from this review is the limited attention paid to identifying and prioritising options for redeployment of resources. This is despite mounting evidence from around the world demonstrating missed opportunities to improve health through reallocation of public monies towards more cost‐effective interventions (Glassman et al., 2013). While most studies in this review implied that highly cost‐effective interventions should be prioritised over less cost‐effective ones, only one study provided a ‘real life’ example of where resources were targeted for release (Kase, 2006). This is not a problem confined to LMICs and has been documented in many HICs (Jan, 2003; Elshaug et al., 2007; Mortimer, 2010). It has been argued that problems with translating ‘wish lists’ of priority health interventions into action is typically a consequence of a failure to release resources from elsewhere in the health budget (Mortimer, 2010). It has been recommended that greater attention needs to be paid to creating stronger incentives for change by, for example, linking investment proposals to disinvestment proposals with relatively similar input requirements (Mortimer, 2010). Examples of good practice for resource release or redeployment do exist from HICs through the application of PBMA in which MCDA has been employed as a tool for assessing relative value of services (Mitton et al., 2014); however, again, translating this experience to LMICs has received limited attention to date.

The vast majority of economic evidence used in priority setting comes from cost‐effectiveness analyses where the measure used is typically cost per DALY averted. Only one priority‐setting exercise relied on evidence from a cost–benefit analysis (Whittington et al., 2012). The overall focus on health outcomes often means that broader welfare consequences such as improved production or increased days at school are not taken into account despite being important for many interventions in LMICs (Evans et al., 2005). For example, it has been reported that antiretroviral treatment for HIV and AIDs can help keep health workers and teachers in their posts and prevent a possible breakdown in society (Pufall et al., 2014). Such limitations highlight a wider challenge in adapting top‐down approaches to priority setting to the preferences and values of local constituencies. The use of measures such as cost per QALY in some LMICs has been justified on the grounds of local preferences being more important than expert DALY weights (Chalkidou, Communication).

The findings of this review must be seen in the light of some limitations. First, we acknowledge that some priority‐setting initiatives are documented in the grey literature. We chose to confine our search to the peer review literature given the focus of this review is largely on methodological issues and that these publications are expected to have undergone some basic quality control. Our search was also restricted to papers in the English language published in the past 10 years. Thirdly, it was beyond the scope of this paper to comprehensively assess ‘impact’, defined as the degree to which priority‐setting results fed into an appraisal process that was led or convened by budget holders. Our appraisal, which asked whether a study was embedded in local budgeting processes, revealed that very few studies met this criterion. While this provides some insight into the likely impact and sustainability of a priority‐setting framework, this is not a definitive measure of impact on actual resource allocation decision‐making. Currently, much of the evidence needed to assess policy impact is not routinely published, particularly in instances of failure in implementation. The task of attribution is further complicated by the delay that often occurs between the presentation of evidence and of action, the multiple and complex influences that tend to bear on resource allocation decisions and because the role of priority‐setting evidence is not necessarily explicitly acknowledged when decisions are ultimately made. As a result, the existing literature (peer reviewed or otherwise) is unlikely to be representative of what is really happening on the ground in LMICs. Policy makers should be encouraged to document and share their experiences with priority setting. Initiatives such as the Capacity Building Programme on Universal Health Coverage Academy in Thailand (www.ihpp.thaigov.net/capuhc), coordinated by the International Health Policy Programme and funded by Rockefeller, is likely to prove to be a much better source of priority‐setting approaches than the western literature alone.

Table 5 provides a summary of the different challenges discussed earlier and includes recommendations about what might be done to encourage a more transparent and implementable way of using economic information for priority setting in LMICs.

In conclusion, governments and donors are under mounting pressure to demonstrate greater accountability for how limited health resources are used to meet health system goals in recipient countries. Increasing efforts are therefore being made in LMICs to adopt a more explicit approach to priority setting whereby resources are allocated with the intention of maximising value for money as well as other societal objectives. This review points to a number of factors currently constraining the adoption of such an approach in LMICs. Barriers associated with poor data and limited technical capacity are not new and have been noted in other reviews of priority setting (see, e.g. Glassman et al., 2013). Incorporation of the wider practical constraints arising from the political and institutional context, building local capacity, identifying areas for disinvestment and identifying unintended consequences for the wider health system remain some of the least‐developed aspects of the priority‐setting literature and have particular relevance in resource‐constrained settings. Consideration of these constraints will help strengthen priority‐setting frameworks and encourage evidence‐based decisions on healthcare spending in LMICs.

Conflict of Interest

The authors have no conflict of interest.

Supporting information

Acknowledgements

Appendix 1. 

Notes

Dedication We would like to dedicate this paper to the late Professor Gavin Mooney whose many years of work on priority setting helped pave the way for this review.

Notes

Wiseman, V. , Mitton, C. , Doyle‐Waters, M. M. , Drake, T. , Conteh, L. , Newall, A. T. , Onwujekwe, O. , and Jan, S. (2016) Using Economic Evidence to Set Healthcare Priorities in Low‐Income and Lower‐Middle‐Income Countries: A Systematic Review of Methodological Frameworks. Health Econ., 25: 140–161. 10.1002/hec.3299. [Europe PMC free article] [Abstract] [Google Scholar]

Notes

Correction has been added on 12 February 2016, following initial online publication 25 January 2016. Sentence “MCDA and GCEA were the most common approaches used in 42% and 18% of studies, respectively.” was changed to “GCEA and MCDA were the most common approaches used in 40% and 18% of studies, respectively.”

Footnotes

References