Methodological issues in assessing changes in costs pre- and post-medication switch: a schizophrenia study example
© Faries et al; licensee BioMed Central Ltd. 2009
Received: 07 September 2008
Accepted: 27 May 2009
Published: 27 May 2009
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact us so we can address the problem.
Schizophrenia is a severe, chronic, and costly illness that adversely impacts patients' lives and health care payer budgets. Cost comparisons of treatment regimens are, therefore, important to health care payers and researchers. Pre-Post analyses ("mirror-image"), where outcomes prior to a medication switch are compared to outcomes post-switch, are commonly used in such research. However, medication changes often occur during a costly crisis event. Patients may relapse, be hospitalized, have a medication change, and then spend a period of time with intense use of costly resources (post-medication switch). While many advantages and disadvantages of Pre-Post methodology have been discussed, issues regarding the attributability of costs incurred around the time of medication switching have not been fully investigated.
Medical resource use data, including medications and acute-care services (hospitalizations, partial hospitalizations, emergency department) were collected for patients with schizophrenia who switched antipsychotics (n = 105) during a 1-year randomized, naturalistic, antipsychotic cost-effectiveness schizophrenia trial. Within-patient changes in total costs per day were computed during the pre- and post-medication change periods. In addition to the standard Pre-Post analysis comparing costs pre- and post-medication change, we investigated the sensitivity of results to varying assumptions regarding the attributability of acute care service costs occurring just after a medication switch that were likely due to initial medication failure.
Fifty-six percent of all costs incurred during the first week on the newly initiated antipsychotic were likely due to treatment failure with the previous antipsychotic. Standard analyses suggested an average increase in cost-per-day for each patient of $2.40 after switching medications. However, sensitivity analyses removing costs incurred post-switch that were potentially due to the failure of the initial medication suggested decreases in costs in the range of $4.77 to $9.69 per day post-switch.
Pre-Post cost analyses are sensitive to the approach used to handle acute-service costs occurring just after a medication change. Given the importance of quality economic research on the cost of switching treatments, thorough sensitivity analyses should be performed to identify the impact of crisis events around the time of medication change.
Pre-Post (or "mirror-image") designs are studies where one compares outcomes prior to and after some event – such as a medication change. The focus here is on pre-post comparisons of economic outcomes from patients switching medication treatment for schizophrenia. In long term treatment for schizophrenia, treatment switching is common, and there is a great need for information on treatment options for patients requiring a medication switch [1–3]. Due to the large impact of schizophrenia care on health care payer budgets, the comparison of treatments on economic outcomes has been the focus of much research [4, 5]. Understanding if the costs of care decrease upon a change to a particular medication is often a research question of interest that is assessed using pre-post analyses. Hudson and colleagues  reviewed 10 economic outcome studies of antipsychotic medications using various pre-post designs and reported mixed results. More recently, pre-post assessments of costs were reported for patients with schizophrenia switching to amisulpride , switching between risperidone and olanzapine [8, 9] and switching from typicals to atypicals .
Health-care decision makers often utilize observational data to assess economic outcomes [11, 12]. In comparison to clinical trials, observational research provides broader generalizability due to its utilization of cost and medical resource data which is both real-world and less influenced by protocol-required visits, compliance, and other procedures [11, 13]. However, observational data is subject to selection bias that complicates comparisons between nonrandomized groups of patients . One advantage of Pre-Post analyses is that they allow for 'within-patient analyses' (each patient is their own control) – thus avoiding the need to adjust for selection bias that complicates comparisons between (nonrandomized) groups of patients in naturalistic research.
Pre-post analyses have several important disadvantages that have been well described in the literature . These include regression to the mean, time period bias, lack of a control group, and asymmetrical treatment durations. In usual clinical practice settings for the treatment of schizophrenia and other episodic disease states, treatment changes are typically made during a time of symptom exacerbation or a relapse which may require acute care services. Thus, on average, one might expect symptoms to be ameliorated and costs to decrease from that time point. Pre-post analyses compare outcomes that occurred at different times on the same patient under different treatment conditions. Inferring that differential outcomes are due to treatment differences requires one, among other things, to assume no period/time effect, which is especially problematic in the study of episodic disease states such as schizophrenia.
Pre-post analyses are not, in general, suited to address broader questions of interest requiring multiple comparison groups. For instance, in the treatment of schizophrenia one might be interested in evaluating the multiple treatment options at the point of medication change, including switching to one of several medications or changing the dose of the current medication. One option in such instances is to utilize randomization to several treatment options for comparison at the time point where patients were in need of a medication change – as was done in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) [1, 16, 17]. In naturalistic settings, the duration of each (pre- and post-) treatment varies – thus raising challenges when comparing outcomes based on differing treatment durations. While one can just assess a set period of time, pre- and post-switch, the effect of a drug may differ in the first few weeks of treatment relative to a later period of treatment.
Another limitation of pre-post designs for cost comparisons that has received less attention is the issue of costs incurred while on treatment B (the medication switched to) which may have been due to the failure of treatment A (the initial medication). For instance, while there are many reasons for switching treatment, the most common is lack of efficacy , which includes patients who relapse – leading to crisis management such as a hospitalization where treatment A is discontinued and treatment B is first initiated in the hospital. Costs for the management of this relapse may continue for some time after discontinuation of treatment A. While this issue has been raised , this is especially important for economic analyses because hospitalizations and the cost of relapse represent the largest portion of health care costs for patients with schizophrenia [19–22].
Our objective in this paper is to highlight the issue of overlapping or carry-over costs from a failed prior treatment onto the next treatment in pre-post medication switch analyses. We accomplish this by examining the sensitivity of pre-post analyses of changes in costs of care to various assumptions regarding the attributability of acute-care service costs occurring immediately after a medication switch. Data from a naturalistic schizophrenia study is utilized for the analysis.
Data for this post-hoc analysis came from a cost-effectiveness study of first-line treatment with atypical and conventional antipsychotics over a 1-year period [23, 24]. This randomized, naturalistic, open-label trial was conducted from May 1998 through September 2001 at 21 sites in 15 states in the United States. This study used broad eligibility criteria. Patients aged 18 years or older with a DSM-IV diagnosis of schizophrenia, schizoaffective, or schizophreniform disorder were included, provided they scored ≥ 18 on the Brief Psychiatric Rating Scale (BPRS) . No patient was excluded because of comorbid substance abuse disorder or other psychiatric or medical comorbidities. Patients were randomly allocated to a first-line treatment group: 1 of 2 atypical antipsychotics (n = 450) or a conventional first-generation antipsychotic of each physician's choosing (n = 214). Barring clinically significant adverse events, patients remained on their initial medications for at least 8 weeks, after which they could change medications if a switch was clinically warranted. Medications could also be discontinued, or their doses altered, at each physician's discretion.
Definitions and Measures
Patients were defined as having switched medications if they discontinued the medication to which they were initially randomized and then initiated therapy with a different antipsychotic within 14 days. Acute-care services were defined as hospitalizations, partial hospitalizations, and visits to emergency departments (EDs). Data on each patient's use of such services were systematically abstracted from medical records, patient self-reports, and the study sites' administrative databases. Cost estimates applied to each unit of resource use have been previously described in detail . In brief, Medicare public data were used as cost benchmarks for units of specific services and applied to the collected resource use data to obtain costs of care for each patient. Medication costs were based on 2001 average wholesale prices, discounted 15% to reflect real-world costs .
To examine the impact of carry-over (overlapping) costs, we analyzed switching data using 4 different approaches – each making different assumptions regarding the attributability of costs and timing of incurred costs potentially due to the initial (pre) treatment. To standardize the 2 evaluation periods to a common duration, we chose to evaluate the last 60 days on the Pre-treatment and the 1st 60 days on the Post-treatment. The goal was to use a short enough duration to avoid deletion of patients from the analysis (or imputation of missing data) who only had short durations in the study pre- and/or post-switch and long enough to provide a more stable assessment of costs while on the medication. As secondary analyses, we utilized a 30-day window. Costs in the post-period were computed regardless of additional subsequent switches in medications. Costs are presented on an average daily basis to allow for comparisons across methods on a standard time scale. The 4 analysis approaches were:
Costs incurred during the 60 days leading up to the last day on the initial medication are defined as 'Pre' period costs. Costs incurred starting from the day after the initial medication was discontinued for the next 60 days are defined as 'Post' period costs.
Remove Overlapping Acute Crisis Costs from Post Period
Costs of acute care services that began while on the initial medication, but continued into the 'Post' medication period, were excluded from the analysis. Otherwise, calculations were the same as for the Standard Pre-Post Switch Analysis.
Exclude first 2 weeks of the Post Period
Calculations follow the Standard Pre-Post Switch analysis except that the first 14 days of the Post period were excluded from the calculations. The 2-week cutoff was selected as most hospitalization durations were less than 14 days, so the impact of hospitalizations beginning in the 'pre' period would be addressed and since research suggests that impact of new medication could be observable by 2 weeks [26–28].
Assign first 2 weeks of Post Costs to Pre Medication
Calculations follow the Standard Pre-Post Switch analysis except that costs occurring in the first 14 days in the post period are assigned to the Pre-treatment period.
Paired differences in costs (Pre-Post) were compared statistically (with a null hypothesis of no change in mean costs) using both a paired t-test and, due to the skewness of the cost data, a nonparametric bootstrapping approach. The bootstrapping approach was utilized as we desired a nonparametric approach due to the anticipated skewed cost data and a test for differences in means as opposed to medians. Means are the appropriate parameter of interest as health care payers are responsible for the cost of the entire population, and N-times-the-median may not be representative of the total costs given skewed distributions [29, 30].
A total of 664 patients were randomized, though 13 never started on their assigned medication . Of the 651 analyzable patients, 191 (29.3%) switched antipsychotics at some point during the 1-year study; 144 patients switched antipsychotics AND had information in the study from 30 days pre- to 30 days post-switch available for analysis; and 105 patients switched antipsychotics and had information from 60 days pre- and post-switch available for analysis. Thus, analyses here are based on 105 patients. Analysis with the 30-day requirement (n = 144) provided similar results. Of the 460 patients who did not switch antipsychotics during the study, 31.7% were early discontinuers and did not complete 1 year on study. It is anticipated that some of these patients may have changed medications outside of the clinical study. Thus, actual rates of switching is anticipated to be greater than 29.3%, but data on these potential switching patients was not available.
Patient Characteristics at Study Enrollment
Analyzed Switchers a
66.7 b, c
Age, mean (SD)
Age of onset, mean (SD)
Baseline PANSS total, mean (SD)
Hospitalization in year prior to enrollment
None of the changes in costs were statistically significant from zero, and confidence intervals on the differences in pre-post cost changes for the 4 approaches are large and overlapping. For example, the 95% two-sided confidence interval for the standard approach was (-$12.12, $18.69) and for assigning the first 14 days of post costs to the pre period was (-$24.55, $5.71). Due to the high variability in total costs of care, such studies must be fairly large in order to have enough power for relevant cost comparisons. However, the focus here was not on establishing whether a change in costs occurred, but to compare the methodologies.
The current analyses have demonstrated that assumptions about costs potentially attributable to failure of the first medication can have a directional impact on results when assessing the economic impact of changing medications using a pre-post analysis. The difference in estimated changes in daily costs in this study varied by over $12 per day across the methods utilized (between -$9.69 and +$2.40). Acute care service costs, which is the largest cost component in the treatment of schizophrenia, is often associated with relapse and with changes in medications, and thus must be considered carefully. Given the variation in results depending upon the attribution of crisis care costs around the time of switch, sensitivity analyses are critical to proper understanding of such data.
Caution regarding the analysis of costs occurring around the time of switch should not be limited to pre-post designs. For instance, in clinical trials patients are typically discontinued at the point at which they have a medication change. This potentially ignores a substantial amount of the treatment costs which would then occur outside of the purview of the trial. In practical clinical trials, where patients are allowed to change medications and remain in the study, it is common to conduct an analysis including only the period of time in which the patient was on the medication of interest . Once again, this analysis could be misleading due to missing the potentially large costs occurring just after discontinuing the medication that should be owned by the initial medication.
The availability of data from a control group may address some of the issues associated with pre-post analyses. For instance, pre- and post-switch data for patients treated with a comparison medication would allow for additional methodological approaches to be used. Fu and colleagues  compare the use of a difference-in-differences approach along with propensity score-based methodology in such settings. One may also benefit by considering a pre-post analysis in comparison to assessing treatment effects in a crossover design. Without a control group, one simply has a single arm of a standard 2 × 2 crossover trial. The availability of a comparator group may also allow the use of crossover related methods .
This study has several limitations. First, the number of patients switching was relatively small for comparisons of costs. Replication with a larger sample is needed due to the high variability in cost data. Next, we were unable to fully collect the reason for switching, which may indeed be associated with costs incurred around the time of switch. Lastly, about one-third of the nonswitchers discontinued early from the study. Some proportion of these patients likely switched medications outside of the purview of the trial, but we were unable to collect post-switch economic data for these patients. At baseline, these early discontinuers were more likely to be male; had more severe symptoms (as measured by the BPRS); and had a higher rate of prior hospitalization, substance abuse, and comorbid anxiety as compared to the switchers in this study. Thus, it is not clear if the results observed here are generalizable to such patients.
Pre-post cost analyses ("mirror-image") are complicated by costs incurred post-switch that are likely due to the failure of the initial treatment. Such costs are not ignorable and can have substantial impact on the analyses. Sensitivity analyses should be performed to identify the impact of such costs.
- Lieberman JA, Stroup TS, McEvoy JP, Swartz MS, Rosenheck RA, Perkins DO, Keefe RS, Davis SM, Davis CE, Lebowitz BD, Severe J, Hsiao JK, Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) Investigators: Effectiveness of antipsychotic drugs in patients with chronic schizophrenia. N Engl J Med 2005,353(12):1209–1223. 10.1056/NEJMoa051688PubMedView ArticleGoogle Scholar
- Essock SM, Covell NH, Davis SM, Stroup TS, Rosenheck RA, Leiberman JA: Effectiveness of switching antipsychotic medications. Am J Psychiatry 2006,163(12):2090–2095. 10.1176/appi.ajp.163.12.2090PubMedView ArticleGoogle Scholar
- Ascher-Svanum H, Zhu B, Faries D, Landbloom R, Swartz M, Swanson J: Time to discontinuation of atypical versus typical antispsychotics in the naturalistic treatment of schizophrenia. BMC Psychiatry 2006, 6: 8. 10.1186/1471-244X-6-8PubMed CentralPubMedView ArticleGoogle Scholar
- Duggan M: Do new prescription drugs pay for themselves? The case of second-generation antipsychotics. J Health Econ 2005,24(1):1–31. 10.1016/j.jhealeco.2004.08.001PubMedView ArticleGoogle Scholar
- Polsky D, Doshi JA, Bauer MS, Glick HA: Clinical trial-based cost-effectiveness analyses of antipsychotic use. Am J Psychiatry 2006,163(12):2047–2056. 10.1176/appi.ajp.163.12.2047PubMedView ArticleGoogle Scholar
- Hudson TJ, Sullivan G, Feng W, Owen RR, Thrush CR: Economic evaluations of novel antipsychotic medications: a literature review. Schizophr Res 2003,60(2–3):199–218. 10.1016/S0920-9964(02)00228-1PubMedView ArticleGoogle Scholar
- Surguladze S, Patel A, Kerwin RW, Knapp M, Travis MJ: Cost analysis of treating schizophrenia with amisulpride: naturalistic mirror image study. Prog Neuropsychopharmacol Biol Psychiatry 2005,29(4):517–522. 10.1016/j.pnpbp.2005.01.002PubMedView ArticleGoogle Scholar
- Zhao Z, Namjoshi M, Barber BL, Loosbrock DL, Tunis SL, Zhu B, Breier A: Economic outcomes associated with switching individuals with schizophrenia between risperidone and olanzapine: findings from a large US claims database. CNS Drugs 2004,18(3):157–164. 10.2165/00023210-200418030-00002PubMedView ArticleGoogle Scholar
- Takahashi H, Kamata M, Yoshida K, Ishigooka J, Higuchi H: Switching to olanzapine after unsuccessful treatment with risperidone during the first episode of schizophrenia: an open-label trial. J Clin Psychiatry 2006,67(10):1577–1582.PubMedView ArticleGoogle Scholar
- Park S, Ross-Degnan D, Adams AS, Sabin J, Kanavos P, Soumerai SB: Effect of switching antipsychotics on antiparkinsonian medication use in schizophrenia: population-based study. Br J Psychiatry 2005, 187: 137–142. 10.1192/bjp.187.2.137PubMedView ArticleGoogle Scholar
- Haro JM, Kontodimas S, Negrin MA, Ratcliffe M, Suarez D, Windmeijer F: Methodological aspects in the assessment of treatment effects in observational health outcomes studies. Appl Health Econ Health Policy 2006,5(1):11–25. 10.2165/00148365-200605010-00003PubMedView ArticleGoogle Scholar
- Garrison LP Jr, Neumann PJ, Erickson P, Marshall D, Mullins CD: Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value Health 2007,10(5):326–335. 10.1111/j.1524-4733.2007.00186.xPubMedView ArticleGoogle Scholar
- Roy-Byrne PP, Sherbourne CD, Craske MG, Stein MB, Katon W, Sullivan G, Means-Christensen A, Bystritsky A: Moving treatment research from clinical trials to the real world. Psychiatr Serv 2003,54(3):327–332. 10.1176/appi.ps.54.3.327PubMedView ArticleGoogle Scholar
- D'Agostino RB Jr, D'Agostino RB Sr: Estimating treatment effects using observational data. JAMA 2007,297(3):314–316. 10.1001/jama.297.3.314PubMedView ArticleGoogle Scholar
- Gianfrancesco F, Wang RH, Mahmoud R, White R: Methods for claims-based pharmacoeconomic studies in psychosis. Pharmacoeconomics 2002,20(8):499–511. 10.2165/00019053-200220080-00001PubMedView ArticleGoogle Scholar
- Stroup TS, Lieberman JA, McEvoy JP, Swartz MS, Davis SM, Capuano GA, Rosenheck RA, Keefe RS, Miller AL, Belz I, Hsiao JK, CATIE Investigators: Effectiveness of olanzapine, quetiapine, and risperidone in patients with chronic schizophrenia after discontinuing perphenazine: a CATIE study. Am J Psychiatry 2007,164(3):415–427. 10.1176/appi.ajp.164.3.415PubMedView ArticleGoogle Scholar
- McEvoy JP, Lieberman JA, Stroup TS, Davis SM, Meltzer HY, Rosenheck RA, Swartz MS, Perkins DO, Keefe RS, Davis CE, Severe J, Hsiao JK, CATIE Investigators: Effectiveness of clozapine versus olanzapine, quetiapine, and risperidone in patients with chronic schizophrenia who did not respond to prior atypical antipsychotic treatment. Am J Psychiatry 2006,163(4):600–610. 10.1176/appi.ajp.163.4.600PubMedView ArticleGoogle Scholar
- Liu-Seifert H, Adams DH, Kinon BJ: Discontinuation of treatment of schizophrenia patients is driven by poor symptom response: a pooled post-hoc analysis of four atypical antipsychotics drugs. BMC Med 2005, 3: 21. 10.1186/1741-7015-3-21PubMed CentralPubMedView ArticleGoogle Scholar
- Weiden PJ, Olfson M: Cost of relapse in schizophrenia. Schizophr Bull 1995,21(3):419–429.PubMedView ArticleGoogle Scholar
- Gilmer TP, Dolder CR, Lacro JP, Folsom DP, Lindamer L, Garcia P, Jeste DV: Adherence to treatment with antipsychotic medication and health care costs among Medicaid beneficiaries with schizophrenia. Am J Psychiatry 2004,161(4):692–699. 10.1176/appi.ajp.161.4.692PubMedView ArticleGoogle Scholar
- Sun SX, Liu GG, Christensen DB, Fu AZ: Review and analysis of hospitalization costs associated with antipsychotic nonadherence in the treatment of schizophrenia in the United States. Curr Med Res Opin 2007,23(10):2305–2312. 10.1185/030079907X226050PubMedView ArticleGoogle Scholar
- Marcus SC, Olfson M: Outpatient antipsychotic treatment and inpatient costs of schizophrenia. Schizophr Bull 2008,34(1):173–180. 10.1093/schbul/sbm061PubMed CentralPubMedView ArticleGoogle Scholar
- Tunis SL, Johnstone BM, Kinon BJ, Barber BL, Browne RA: Designing naturalistic prospective studies of economic and effectiveness outcomes associated with novel antipsychotic therapies. Value Health 2000,3(3):232–242. 10.1046/j.1524-4733.2000.33007.xPubMedView ArticleGoogle Scholar
- Tunis SL, Faries DE, Nyhuis AW, Kinon BJ, Ascher-Svanum H, Aquila R: Cost-effectiveness of olanzapine as first-line treatment for schizophrenia: results from a randomized, open-label, 1-year trial. Value Health 2006,9(2):77–89. 10.1111/j.1524-4733.2006.00083.xPubMedView ArticleGoogle Scholar
- Overall JE, Gorham DR: The brief psychiatric rating scale. Psychol Rep 1962, 10: 799–812.View ArticleGoogle Scholar
- Correll CU, Malhotra AK, Kaushik S, McMeniman M, Kane JM: Early prediction of antipsychotic response in schizophrenia. Am J Psychiatry 2003,160(11):2063–2065. 10.1176/appi.ajp.160.11.2063PubMedView ArticleGoogle Scholar
- Leucht S, Busch R, Hamann J, Kissling W, Kane JM: Early-onset hypothesis of antipsychotic drug action: a hypothesis tested, confirmed and extended. Biol Psychiatry 2005,57(12):1543–1549. 10.1016/j.biopsych.2005.02.023PubMedView ArticleGoogle Scholar
- Leucht S, Busch R, Kissling W, Kane JM: Early prediction of antipsychotic nonresponse among patients with schizophrenia. J Clin Psychiatry 2007,68(3):352–360.PubMedView ArticleGoogle Scholar
- Ramsey S, Willke R, Briggs A, Brown R, Buxton M, Chawla A, Cook J, Glick H, Liljas B, Petitti D, Reed S: Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force report. Value Health 2005,8(5):521–533. 10.1111/j.1524-4733.2005.00045.xPubMedView ArticleGoogle Scholar
- Doshi JA, Glick HA, Polsky D: Analyses of cost data in economic evaluations conducted alongside randomized controlled trials. Value Health 2006,9(5):334–340. 10.1111/j.1524-4733.2006.00122.xPubMedView ArticleGoogle Scholar
- Rosenheck RA, Leslie DL, Sindelar J, Miller EA, Lin H, Stroup TS, McEvoy J, Davis SM, Keefe RS, Swartz M, Perkins DO, Hsiao JK, Lieberman J, CATIE Study Investigators: Cost-effectiveness of second-generation antipsychotics and perphenazine in a randomized trial of treatment for chronic schizophrenia. Am J Psychiatry 2006,163(12):2080–2089. 10.1176/appi.ajp.163.12.2080PubMedView ArticleGoogle Scholar
- Fu AZ, Dow WH, Liu GG: Propensity Score and Difference-in-difference Methods: A Study of Second Generation Antidepressant Use in Patients with Bipolar Disorder. Health Services and Outcomes Research Methodology 2007,7(1–2):23–28. 10.1007/s10742-006-0016-xView ArticleGoogle Scholar
- Senn S: Crossover Design. In Encyclopedia of Biopharmaceutical Statistics. Edited by: Chow SC. New York: Marcel Dekker, Inc; 2000:142–148.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.