(pdf)
Introduction
The Congressional Budget Office (CBO) performs a pivotal role in the legislative process as the official scorekeeper of policy proposals. As part of this process, CBO regularly produces budget baselines that project federal outlays and revenues over the next decade. In an admirable effort to review its own accuracy, the agency analyzed 35 years’ worth of its baselines to determine the cause of errors in its own projections. The biggest contributing factor was not analyst error or even shifting economic fortunes; it was Congress itself, through subsequent changes to policy that rendered CBO’s analysis inaccurate through no fault of its own.
This highlights the need for Congress to change its budget baseline rules to encourage more accurate projection by CBO. Currently, the agency is required to construct its baseline using what is referred to as “current law” assumptions, which analyze spending and revenue according to the letter of the law. But this approach tends to underestimate spending and overestimate revenues, as Congress routinely extends many expiring tax provisions or cancels planned spending reductions. That’s why a “current policy” baseline can offer more accurate projections by accounting for likely Congressional action to change tax and spending policies from what was planned.
While CBO produces such a baseline on occasion on its own initiative (which it dubs an “alternate fiscal scenario”), the fact of the matter is that it is Congress that must change its approach in order to facilitate more accurate scorekeeping. If Congress were to encourage (or even mandate) the creation of a current policy baseline, and ensure that its budget rules allowed for its official use on the floor, budget projections would be much more accurate.
CBO’s Error Report
In a new report CBO set out to review its accuracy by comparing its projections since 1984 to actual budgetary outcomes. The agency adjusted the historic baseline projections made each year to take into account the fiscal impact of legislation enacted each year after CBO published its baseline. In general, they found that they overestimated deficits in the first year of their projections but underestimated deficits in the sixth year.
CBO compared its projections with the actual figures reported by the Department of the Treasury after the start of the new fiscal year each October. Specifically, they looked at the deficit and debt figures in the second year and the sixth year of the baselines produced in the spring of each year, which is the baseline used most often for scoring legislation. CBO chose these years for review in order to get a larger sample size because the agency did not extend its baseline beyond the sixth year until late 1995. If they looked at a longer time horizon over the period of the projections, they would have only been able to include 17 historical baselines.
CBO wanted to measure the errors in their projection that resulted from external factors, such as changes in the economy. To get at this, they adjusted historical projections to account for changes in outlays and revenues that resulted from legislation enacted each year after the baseline was constructed.
CBO’s Findings on Errors
CBO found that it tended to overestimate deficits for the first budget year by 0.2 percent of GDP. For comparison’s sake, 0.1 percent of GDP is equivalent to $20 billion in 2018. Using some statistical analysis of the spread of errors in the historical projections, CBO estimates that “there is a two-thirds chance that, under current law, future budget year deficits would be within a range that is 1.1 percent of GDP above or below the baseline projection.”
The error rate for outlays and revenue was far higher in the sixth-year than the budget year but overestimates and underestimates mostly cancelled each other out, resulting in an average sixth-year deficit that was underestimated by 0.1 percent of GDP. CBO also noted that the sixth-year projections since 2000 have been more accurate than those made over the previous two decades. Statistical analysis of the projections “indicates that future sixth-year deficits have a two-thirds changes of falling within 1.9 percent of GDP above or below the baseline projection.”
Debt levels were similarly overestimated in the budget year and underestimated in the sixth-year year, but naturally to a larger degree since debt levels compound over multi-year projections. Sixth-year year deficit estimates were underestimated by 0.6 percent of GDP ($120 billion relative to GDP as measured in 2018).
Other key findings in the CBO report:
- The timing of recessions was a contributing factor in projection errors. CBO found that when a projection was made at the start of a recession the errors were understandably larger compared to other years - mostly a result of fewer tax revenues flowing into the Treasury than CBO anticipated.
- Since 1997, CBO has tended to overestimate total outlays because of overestimates of net interest outlays. Interest rates have continued to be lower than CBO has anticipated.
- CBO also compared its projections with similarly adjusted projections made each year by the White House’s Office of Management and Budget and found that the errors were close in size and followed similar patterns.
Why This Matters
This report is part of an important effort to shed light on the uncertainty in its budget projections. CBO has ramped up this effort recently, with the intent to “emphasize the uncertainty inherent in budget projections and to attempt to quantify and help readers better understand the magnitude of that uncertainty.” For example, in a report examining the long-term budget outlook, CBO presented a “confidence interval” finding “about a two-thirds chance that federal debt would be between 71 percent and 175 percent of GDP in 2039.”
Highlighting the uncertainty inherent in scorekeeping could help elevate the debate surrounding budgetary scores of legislation. The dollar figures produced in CBO’s cost estimates often make headlines (and CBO’s new layout for reporting cost estimates prominently features the dollar figures at the top of the publications), but the larger and more complex the proposal is, the more uncertainty there will be in the cost estimate. This is due to a combination of variables, including demographic and economic factors, as well as the soundness of the key assumptions that CBO needs to make in the course of figuring out a proposals impact. For example, CBO continually assumed that the Affordable Care Act’s (ACA) individual mandate tax penalty would have a strong impact on enrollment in the health insurance exchanges, but this turned out not to be the casse.
When it comes to eventually scoring incredibly massive and complex proposals such as Medicare for All, the number of factors and the interplay of those factors are greatly magnified, making budget estimates all the more precarious. This is why it will be important to continue to include a range of outcomes, or to find other ways to illustrate the sensitivity of key variables to show how small changes in assumptions built into the scoring model can lead to much different fiscal consequences.
Needed: A Current Policy Baseline
In an important appendix to its errors report, CBO analyzed the impact that legislation enacted after the baselines were published had on the actual outcomes. This had a far greater impact than the projection errors analyzed in the main body of the report. CBO found that legislation boosted the second-year deficits by 0.8 percent of GDP ($160 billion relative to GDP as measured in 2018) as compared to its projection error that overestimated deficits by 0.2 percent of GDP, on average. For the sixth-year projections, legislation added an average of 1.6 percent to the deficit, with about 54 percent of the increase due to higher spending and 46 percent from reductions in revenues. In other words, the largest confounding factor faced by CBO when analyzing the budget, by orders of magnitude, is in fact Congress.
By encouraging or requiring CBO to regularly produce a more realistic baseline that accounts for likely changes to policy (like extensions to expiring tax provisions), Congress could help encourage more accurate scorekeeping. While legislators could simply mandate that the agency regularly produce a current policy baseline (including scoring individual legislation against such a projection), this would impose substantial new workload. True reform will not come until Congress changes its own budget rules to allow such a baseline to be used for official purposes, and a current policy baseline will not be possible unless CBO has sufficient resources. That is why a comprehensive shift toward more accurate projection must come from Congress first.
Conclusion
CBO should be applauded for its efforts to account for sources of errors. The agency should continue its efforts to better explain the sensitivity of its forecasts through tools like confidence intervals, but at the same time Congress should work to fix the budgetary rules that require CBO to produce projections that are inaccurate on the day they’re released. Regular use of a current policy baseline will allow CBO to better execute on its important mission of informing Congress of the fiscal impact of its actions.