Economics of immunization information systems in the United States: assessing costs and efficiency

Background One of the United States' national health objectives for 2010 is that 95% of children aged <6 years participate in fully operational population-based immunization information systems (IIS). Despite important progress, child participation in most IIS has increased slowly, in part due to limited economic knowledge about IIS operations. Should IIS need further improvement, characterizing costs and identifying factors that affect IIS efficiency become crucial. Methods Data were collected from a national sampling frame of the 56 states/cities that received federal immunization grants under U.S. Public Health Service Act 317b and completed the federal 1999 Immunization Registry Annual Report. The sampling frame was stratified by IIS functional status, children's enrollment in the IIS, and whether the IIS had been developed as an independent system or was integrated into a larger system. These sites self-reported IIS developmental and operational program costs for calendar years 1998–2002 using a standardized data collection tool and underwent on-site interviews to verify reported data with information from the state/city financial management system and other financial records. A parametric cost-per-patient-record (CPR) model was estimated. The model assessed the impact of labor and non-labor resources used in development and operations tasks, as well as the impact of information technology, local providers' participation and compliance with federal IIS performance standards (e.g., ensuring the confidentiality and security of information, ensure timely vaccination data at the time of patient encounter, and produce official immunization records). Given the number of records minimizing CPR, the additional amount of resources needed to meet national health goals for the year 2010 was also calculated. Results Estimated CPR was as high as $10.30 and as low as $0.09 in operating IIS. About 20% of IIS had between 2.9 to 3.2 million records and showed CPR estimates of $0.09. Overall, CPR was highly sensitive to local providers' participation. To achieve the 2010 goals, additional aggregated costs were estimated to be $75.6 million nationwide. Conclusion Efficiently increasing the number of records in IIS would require additional resources and careful consideration of various strategies to minimize CPR, such as boosting providers' participation.


Background
One United States national health objective for 2010 is to have at least 95% of U.S. children aged <6 years participate in fully operational population-based immunization registries or immunization information systems (IIS) [1]. To achieve this objective the Centers for Disease Control and Prevention (CDC), through its National Immunization Program (NIP) awarded $262 million in immunization grants from 1994 to 2002 to develop and operate IIS in the 50 states, the District of Columbia, five urban areas, and all U.S. territories (CDC/Kim Lane, unpublished data, 2003). During this time period, the National Vaccine Advisory Committee recommended that the CDC determine the resources needed to further develop and maintain these systems as part of the requirements to ensure sustainable IIS funding [2].
As of 2002, approximately 43% of children of U.S. children <6 years of age had two or more vaccinations recorded in an IIS [3]. Although important progress has been made in populating these systems with vaccination records, achieving the 2010 IIS objective would not only require additional resources or funds but also a better understanding of how these resources can effectively be allocated to each IIS. Therefore, characterizing IIS development and operational costs as well as identifying other factors that affect IIS efficiency could assist public health agencies in planning, strengthening and further developing these systems.
Using a nationally representative sample of IIS, this article evaluates economic determinants of IIS performance by estimating the influence of development and management costs of in-house and contracted labor and nonlabor and resources used by IIS centralized systems. It also measures the effect of standards and other factors affecting the functioning of such systems, determines the threshold of patient records needed to minimize average cost per patient record (CPR), and identifies strategies to increase efficiency.

Sample and setting
Data were collected from a national sampling frame of the 56 states/cities that received federal immunization grants under Public Health Service Act §317b and completed the CDC/NIP 1999 Immunization Registry Annual Report -a self-administered questionnaire applied to immunization program managers. This questionnaire assessed the percentage of children <6 years of age with >2 immunizations enrolled in an IIS, the percentage of provider sites participating in the IIS, whether the IIS operated independently or integrated with other health information systems, and evaluated the functional status of the IIS according to 12 federal IIS performance standards (e.g., ensuring the con-fidentiality and security of information, ensure timely vaccination data at the time of patient encounter, produce official immunization records) defined in more detail elsewhere [3]. Table 1 describes the IIS in the sampling frame. The sampling frame used the measures derived from the 1999 annual report to stratify IIS by how well they functioned (a score based on the number of federal IIS functional standards achieved), the percentage of children <6 years of age enrolled in the IIS, and whether the IIS had been developed as an independent system or was integrated into a larger public health system. IIS were classified as 'low', 'medium', or 'high' in terms of children enrolled if the percentage of children enrolled was < 1.7%, >1.7% and < 53.5%, or > 53.5%. IIS were classified as 'low', 'medium', or 'high' in terms of IIS functionality score if the percentage of IIS functional standards achieved was < 37.1%, >37.1% and < 69.9%, or > 69.9%. These cut points were chosen to represent natural breaks in the data. Two IIS were randomly sampled from each stratum or all IIS sampled if ≤ 2 in a stratum with the exception of high functional score and low child enrollment where 3 were sampled because 2 states shared the same IIS. 17 IIS were under development and fell into a stratum of <37.1% functional score and <1.7% child enrollment; 4 IIS were randomly sampled from this group for participation in this study. The final sample consisted of 24 IIS. These IIS varied by duration of operation (about half began IIS development in 1993) and targeted age range (most IIS collected vaccination data on persons aged 0-18 years old).

Data collection
In August 1999, the IIS sites in the sample collected developmental and operational program costs for calendar years 1998 and 1999. From May to October 2003, researchers visited the selected sites with on-site interviews to collect IIS expenditures for calendar years 2000-2002 and verify data collected previously. Since the 1999 assessment, many IIS had upgraded to web-based systems, expanded their partnerships and marketing efforts to the private sector, and found additional funding sources. These data were combined with data from CY 1998-1999 to allow analysis over a five year period. All sites reported data from their financial management offices and internal records.
Annual expenditures were collected as line items for inhouse personnel (including fringe benefits), contractor full-time equivalents (FTEs), equipment, supplies, travel, contractual expenses, other expenses, and indirect costs. Object class definitions developed by the Office of Man-agement and Budget (OMB) for Federal program costs were used to categorize expenditures [4]. Data on IIS expenditures by source of funds were also collected. These included federal immunization grants under Public Health Service Act §317b; 'other' federal sources such as other CDC funds, Maternal and Child Health block grants, Vaccines for Children program funds, or funds from the Centers for Medicare and Medicaid Services; state and local government funds; private sector funds; in-kind contributions; and funds from other sources, such as fees from IIS subscribers. Personnel and contractual cost line items included costs to develop and operate the IIS, to enter data at the central IIS, to evaluate the IIS, and for marketing efforts to recruit parents and health providers to participate in the IIS. Only costs to operate the central IIS within a geographic area were included (i.e., costs for daily operations of local IIS within a state were excluded). IIS costs to providers were excluded. For IIS that were a part of a larger, integrated information system (i.e., system includes lead and metabolic screening data), only the central IIS system's portion of these costs were collected, not the costs to develop and operate the entire integrated system.
All IIS line item costs were separated into developmental and operational costs. Developmental costs were defined as costs incurred prior to any immunization data submis-sion, storage, or access by an immunization provider. Software and hardware upgrade costs were also considered developmental costs, especially in regard to upgrading to a web-based system. Operational costs were defined as costs to maintain the current status of the IIS after immunization providers began to submit data or access IIS records.

Economic analysis
Annual mean IIS costs were estimated for labor, nonlabor, development and operations by funding source and by number of patient records (all ages) in the IIS. Labor costs included personnel salaries and fringe benefits of persons who were city/state government employees as well as contractor full-time equivalents (FTEs) who worked as IIS employees. All other costs, including indirect costs, were included in non-labor costs. Cost data from 1998-2001 were adjusted to 2002 dollars using the Consumer Price Index [5]. Descriptive statistics were weighted to be nationally representative and analyzed using NLOGIT econometric software version 3.0.2.
To examine factors that influenced average CPR in IIS and estimate prospective costs of achieving the Healthy People 2010 goals, a behavioral cost equation was estimated using weighted least squares with site-specific fixed effects. By allowing for site-specific heterogeneity, one could capture the effect on CPR of any unobserved site-specific characteristics. This greatly improved the precision of estimates. The equation for the CPR of site i's IIS in period t was represented by: where AC it is average cost for site i in period t, α i is the sitespecific fixed effect for site i, w it is the wage of labor, rk it is expenditures on capital, Q it is total patient records in IIS, ξ it is achievement of IIS functional standards, and the X it are site characteristics. The regression error for site i in period t is e it . Each of the 24 sampled IIS were observed once per year for a period of five years. Therefore, the sample consisted of 120 data entries. In the final sample, each entry referred to the observation of a specific IIS in a given year, henceforth an IIS-year.
Average CPR was transformed by the natural log as were wages and expenditures to better approximate their relationship to average cost, which was log-normally distributed. In order to allow for the presumed U-shape of the average cost curve, output was specified in quadratic form. This fixed effects model was estimated using weighted least squares to account for stratified sampling with robust estimation of the covariance matrix. Standard errors were corrected using the finite population correction factor, since sample size approached population size, n → N.
The cost of achieving the Healthy People 2010 goal for each sampled IIS was calculated using parameters retrieved from the average cost estimation, discounting at a rate of 3%. For simplicity, it was assumed that each IIS met all 12 functional standards (the Healthy People 2010 IIS goal includes having 95% child participation in an IIS that is fully functional, i.e., meets the 12 IIS functional standards) while other variables were evaluated at their 2002 values. It was assumed, again for simplicity, that the gap between current child participation and 95% child participation was closed at a linear rate of one-eighth the gap per year, while the IIS continued to maintain the average child population growth rate that it had exhibited in the previous five years. IIS that had already achieved or exceeded 95% child participation were assumed to continue to maintain current participation levels by growing with the child population.
In order to illustrate the importance of vaccine provider participation in an IIS, the impact of simultaneously achieving 95% child participation and 95% provider site participation was also examined. The same assumptions used previously were employed. In addition to child participation, the gap between current provider site participation and 95% provider site participation would close at a linear rate of one-eighth the gap per year. Sampled IIS totals were weighted and the national cost for all 56 IIS to achieve the Healthy People 2010 goal was calculated with and without 95% provider site participation.  Table 3 presents results from the fixed effects, weighted least squares estimation of the average cost function for sites with patient records > 0. Site-specific effects were strongly significant, indicating that unique characteristics of each project site did influence CPR. Table 3 presents statistically significant coefficients and the impact of associated variables on average cost. The model did not directly compare integrated with independent IIS in the Achievement of the 12 IIS functional standards also impacted the average CPR, though certain standards impacted more than others. Specifically, ensuring data *IIS years connotes an observation for each of the 24 sampled IIS once per year over a five year period; therefore the sample for our analysis consisted of 120 data entries. Each entry refers to the particular observation of a specific IIS in a given year, henceforth an IIS-year † Dollars per IIS year inflated to 2002 dollars § Cut points are based on 5 equal intervals between 0 and 3.6 million patient records accuracy and completeness increased average cost by $0.51 per record while IIS ability to receive and process vaccination data within one month of vaccine administration increased CPR by $0.41. These were followed in impact by the ability for a clinic to access IIS data during patient visits (-$0.31) and the ability to create an IIS record within six weeks of birth ($0.22). Average costs for IIS that met all 12 functional standards increased by $0.21 per record compared to IIS that did not meet all standards. IIS ability to identify persons needing vaccine reminder/ recall notifications decreased the CPR by $0.16, and the ability to automatically produce immunization coverage reports and official immunization records decreased average costs by $0.11 and $0.17 respectively.

Results
Population characteristics and demographics are usually important determinants of average cost. While population density did not significantly influence CPR, as the proportion of IIS patient records belonging to children increased, average costs declined. A 1% increase in the proportion of child records was associated with 0.71 per-cent decline in average cost. This effect was both strong and highly significant.
A 1% increase in the wages of development personnel and contractors was associated with declines in average cost of 0.05% and 0.02% respectively. Similarly, the elasticity of operations personnel wages with respect to average cost was -0.12. Operations contractor wages did not significantly affect average cost. Expenditures on operating supplies, other purchases, and indirect costs (i.e., overhead) did not significantly affect average costs; nor did expenditures on travel or indirect overhead for development. For the remaining factors, the effect was positive and significant but uniformly inelastic.
The results for patient records displayed in Table 2 make a strong case for economies of scale. As number of records increased, average estimated costs declined at an increasing rate, displaying the expected U-shape. For the average IIS, a 1% increase in patient records resulted in 1.96% decline in CPR. Clearly the average registry, with 980,000 Average amount of annual expenditures * by funding source † and immunization information system size § , 1998-2002   Increased provider participation was associated with increased average costs up to a point, after which costs declined. Figure 3 displays the relationship between provider site participation and CPR. At the current average provider participation rate of 32%, a 1% increase in provider participation was associated with a 0.25% increase in CPR. As provider participation increased, however, the association became smaller. Levels above 80% participation were associated with a decline in average costs.
Results from these calculations indicated a cost of $75.6 million ($76.9 million with no discounting) in 2002 dollars over eight years for all 56 IIS to meet the Healthy People 2010 goal, assuming that all other parameters remain the same. The cost to achieve the Healthy People 2010 goal as well as 95% provider participation amounted to $56.4 million ($57.7 million with no discounting) in 2002 dollars for all 56 IIS. Note that this figure only includes the cost to the central IIS and does not account for costs to providers.

Discussion
This study was designed to characterize the average costs of IIS and to estimate costs for IIS nationwide to be fully functional, population-based, and contain >95% of the children <6 years of age in their geographic areas. States  In general, operations expenses tended to increase with IIS size while development expenses declined. This appeared to hold true within categories as well. An increase in expenditures was associated with an increase in IIS size, but IIS at or near efficient size had more diverse sources of funds. Funding sources might affect the operation of IIS but the effect on average costs is not highly significant.
Past studies collected costs through 1997-1998, when many IIS had not yet migrated to web-based systems [2, [6][7][8][9][10]. These studies also focused on relatively young IIS. Modern IIS systems have more demands on their resources than their predecessors. IIS are directed now to develop added capabilities such as vaccine management, adverse event reporting, lifespan vaccination histories, and linkages with electronic data sources to encourage provider participation. As a public health electronic system, their concerns would also focus on how they will fit into the President's proposed National Health Information Network and compete for federal resources [11].
While the literature lacks examples of IIS costs at the national level, it also lacks studies that characterize the variety of funding sources used in IIS development and operations and the impact of economies of scale on CPR.
Having the most recent data available, including funding sources, may account for the technological changes and new demands placed on the modern IIS. Timely cost data and investigating economies of scale are essential for planning national IIS goals through 2010.
Provider participation is also essential for IIS to achieve their Healthy People 2010 goal. Up front investments by the IIS to integrate with electronic health records, patient management systems, and billing systems from the provider office could encourage provider participation by reducing data entry costs to the provider. Some of the IIS systems sampled devoted resources to provide computers and data entry services to providers and to integrating The influence of immunization information system size on cost* per patient record Figure 2 The influence of immunization information system size on cost* per patient record. their system with other public health systems. These costs were included in the central IIS development and operations costs. While costly at first, the marginal cost of provider participation tended to decline as provider participation increased. As IIS size increased, CPR declined.
This study is subject to several limitations. First, time has not been modeled explicitly in this analysis. The results assume a one time, lump-sum cost of achieving the functional standards and that the gap between 2002 child participation and 95% child participation will be closed linearly. In reality, it is likely that IIS will achieve functional standards over time and child participation may increase nonlinearly over time. The extent that actual IIS achievements of functional standards differ from our assumptions may affect the cost of those achievements. Although it is not clear whether the effect would be positive or negative, in general, technological improvements tend to lead to cost savings. Therefore, to the extent that achievement of functionality represents technology improvement, fully functional IIS should operate more efficiently than their less functional counterparts. Second, the data were self-reported using data from IIS financial systems and internal records. The quality of available data and the completeness of the state financial records varied between the sites. While estimates are comprehensive in terms of types of costs considered and in-person interviews were conducted, it is possible that some expenditures may not have been captured.
Third, this is a study of costs incurred by the central IIS of federally funded IIS grantees over a five-year period. Resources did not permit the formal collection of costs incurred by provider practices or the costs incurred for daily operations of local IIS that export data to the central IIS. While all of the state IIS in the sample included costs of contracts with local health districts to obtain local vaccination data, these contracts did not fully support local IIS operations. Regarding costs to providers, costs for data entry were collected if the central IIS paid for these services. Two recent studies of IIS costs included the costs to providers to enter data and use the IIS for immunization assessment, immunization history reports, and other reports, but only included a small number of private practices in a small geographic area [9,12]; we could not generalize these costs to our national model. Costs to collect this data for the 24 sampled sites were prohibitive. Fourth, depreciation was not accounted for, nor were external factors such as managed care penetration, the number of public or private providers, etc. Nevertheless, this is the first study to estimate the funding needed to support IIS based on a true national sample.
While further analysis is necessary to better understand development and operations of IIS, some implications for policy are clear. Increases in the scale of operations are necessary in order to take advantage of substantial economies of scale. While minimum efficient scale may not be a feasible achievement for IIS with smaller populations, any increase in scale up to 2,900,000 records should result in declining average cost and greater efficiency. IIS with smaller populations may consider partnering with other regions and states to improve their economies of scale. IIS policy makers should also consider improving functionality and the adoption of new information technology; this may lead to efficiency improvements that can reduce total costs. IIS are also encouraged to seek out diverse resources for long-term sustainability. Funding sources vary in accountability requirements and IIS with more diverse funding sources may face greater accountability resulting in more efficient practices. Finally, IIS should enhance electronic linkages with providers to encourage provider participation. Provider costs to use an IIS and/or submit data to an IIS can deter participation and it would be well worth the investment of IIS funds to assess the costs to participating providers and implement strategies to decrease these costs. Obtaining data electronically from electronic health records, managed care organizations, and other large vaccination data sources, could decrease CPR as well as the costs to providers to submit data. Also, for states facing hiring freezes or difficulties in recruiting technical staff, hiring contractors may be a cost-effective option; developmental contractor salaries were associated with a slight decrease in average costs and operational contractor wages did not significantly affect average costs.
The influence of vaccine provider participation* in immuniza-tion information system costs † per patient record Figure 3 The influence of vaccine provider participation* in immunization information system costs † per patient record. * Public and private vaccine service delivery locations that maintain permanent vaccination records and submit vaccination data to an immunization information system routinely (i.

Conclusion
Although complementary resources and efforts will be needed from IIS nationwide, achievement of the 2010 national IIS objective is possible. Efficiently increasing the number of children in IIS, however, would not only require the identification of additional and sustainable resources but, for the most part, a careful consideration of various strategies that minimize IIS' average costs. Among such strategies, substantial increments in providers' participation could significantly decrease CPR.

CDC-Centers for Disease Control and Prevention
CPR-Cost per patient record FTE-Full time equivalents IIS-Immunization information systems NIP-National Immunization Program (now the National Center for Immunization and Respiratory Diseases)