The Congressional Budget Office (CBO) is a relatively small agency, but it plays an enormous role in the legislative process through its fiscal analyses. Bills routinely rise or fall based on the score produced by CBO and Congress’ response to its work. But CBO has also attracted scrutiny in recent years for its lack of transparency, its occasionally odd assumptions, and the inaccuracy of its estimates. Congress has begun to take more seriously its role in overseeing and directing the agency’s work, including recent hearings in the House and Senate Budget Committees.
CBO Director Keith Hall was called to testify about “what went right, and what may have gone wrong” at the agency and was pressed by Members on long-standing concerns about transparency and the assumptions used by CBO in its analyses. House Budget Chairman Steve Womack (R-AR) noted that this is the first comprehensive review of the CBO since it was established in 1974. Senate Budget Chairman Mike Enzi’s (R-WY) sought to hear how CBO can better communicate the methods, assumptions, and data that underlie its analyses.
A Small Agency with a Large Role in the Drafting & Passage of Legislation
With 233 employees, the CBO has a hefty workload to serve as official scorekeeper of legislation and to provide technical support for the Congressional committees. Last year, it produced 80 analytical reports and working papers, 740 formal estimates, plus thousands of informal cost estimates—many of which are provided to Members of Congress as bills are in the process of being written, further underscoring CBO’s important role in crafting legislation.
A critical review of the agency is overdue, given the impact its analyses can have in determining whether legislation lives or dies. This presents a problem in cases where CBO’s estimates have been consistently off the mark because of faulty assumptions baked into the analyses. In particular, health care has long been a problem area for CBO, going back to its analysis of “HillaryCare.” This concern played out again last year with CBO’s highly-criticized analyses of proposals to repeal and replace the Affordable Care Act (ACA).
Director Hall’s organizational goals for the year largely mirrored his remarks from a previous hearing, highlighting the agency’s efforts to improve transparency. He also emphasized that CBO is working on further developing its capabilities to assess macroeconomic effects of fiscal policies and how changes in regulation impact fiscal baseline projections. Done correctly, both measures would improve analyses and be beneficial for taxpayers and lawmakers alike.
Adding a Confidence Index as Measure of Uncertainty in Assumptions
Hall also received some interesting feedback that could improve the value of the agency’s output. Representative Gary Palmer (R-AL) suggested that CBO consider including a confidence index on projections it makes. This would be a rating on CBO’s determination of the likelihood of the outcomes presented in its analyses. The United Kingdom’s Office for Budget Responsibility (OBR) currently includes such a rating in its economic projections. As the OBR notes in its “Forecast the Economy” report:
[C]onsiderable uncertainty lies around all economic forecasts. So, when we present our results we quantify the confidence that people might place in them judging from past forecast errors. We also test the significance that key judgements in our forecast have for the public finances by creating and presenting alternative scenarios as well as our central view.
This could have changed the course of the policy debate that ensued over CBO’s cost estimates of Republican legislation to repeal and replace the ACA. CBO’s determination of the various proposals’ impact on the number of uninsured helped in part to stall the legislative effort. This led to a backlash against CBO because of the absurd assumptions baked into its scores.
Faulty Health Insurance Coverage Model Still Awaiting Fix
Director Hall defended how his agency scored the proposals, but he also forthrightly admitted that there is a problem with CBO’s model used to simulate health insurance coverage of people under age 65. The model has been a recurring problem and Hall said he hoped to have started the process of revising it a few years ago. Going back to the initial estimates of the ACA, CBO has vastly overestimated the number of individuals enrolling in the health care exchanges, thus painting an inaccurate picture of the program’s costs and function.
So why hasn’t the model been comprehensively overhauled? Hall said that it would have been done by now, but the same people who would be responsible for working on it could not get to that project because they were tasked with producing healthcare-related cost estimates for Congress.
Clarity & Transparency Needed for Assumptions Used in Analyses
Even fixing that problem wouldn’t account for the assumptions and explanations of a political nature that go into the cost estimates. For example, CBO determined that repealing and replacing Obamacare would reduce the number of people who are insured through Medicaid by 14 million, which painted a picture of millions being kicked off of insurance rolls. In reality, this result was largely attributable to elimination of the individual mandate and its “tax penalty,” which would cause certain individuals to voluntarily choose to no longer enroll.
But the “coverage loss” numbers also included people who haven’t enrolled in an insurance plan yet. The ACA offers temporary federal subsidies to states that expand eligibility for Medicaid. CBO’s figures reflected a drop in the number of people CBO assumed would otherwise enroll into Medicaid in the future. This was based on political guesses about which current non-expansion states would decide to take advantage of the ACA’s incentives over the next several years. Senator Tom Cotton (R-AR) pointed out that these assumptions were not made clear in the CBO reports.
This goes to another complaint heard from many Members of Congress. The agency can’t always be responsive to individual members who either have questions about methodology or are requesting cost information on legislation they are either drafting or considering endorsing. Hall stressed that the agency has more work than it can handle. When it is receiving multiple requests for additional work, its staff will seek guidance from the budget committees regarding the prioritization of products. Hall also vowed that agency is working on providing more opportunities for Members to meet with CBO analysts and walk through how a score for a proposal was determined. This would help shed light on the assumptions used in analyses.
Sensitivity Metric to Illustrate Potential Swings in Outcomes over the Long-term
Another good suggestion, from Rep. Gary Palmer (R-AL) was for CBO to provide a sensitivity analysis—showing how small changes in CBO’s assumptions could dramatically impact the cost estimate. This gets to the uncertainty that is inherent to the cost estimates. Dr. Hall proudly noted that CBO recently conducted a review of its past estimates of outlays, to see how close its projections came to reality. Since 1984, its estimates were off by 1.7 percent, on average, in the first budget year, and by 3 percent in the sixth year. The farther out they looked, the less accurate the projections were.
Director Hall has made improving transparency a goal for the agency and pledged to increase public documentation about the 215 different models used in its analyses. Yet, CBO seems unwilling to completely open their models to the public. Hall repeated in both sessions his concerns about rising costs related to transparency and he stressed that “models don’t produce estimates, the analysts do,” working with the models and seeking outside data and information from legislators and outside experts.
Watching the Watcher
Members clearly want more access to CBO and for it to produce even more cost estimates more quickly, as the pace of Congress’s work has hastened over recent years. Rushing estimates could sacrifice the quality of the work, and there are already big questions about CBO’s assumptions on major legislation. This would also require higher funding for CBO so that it could be responsive to Congressional needs by bringing on additional staff.
It is encouraging to see a continued, strengthened effort to “watch the watchers”: to hold the CBO accountable for its work and seek out ways to improve the agency whose numbers are at the heart of so many legislative debates. There is a demand for more clear, accurate data and analysis of policy decisions and outcomes. This is why NTUF has initiated our Taxpayers’ Budget Office project which aims to provide independent, expert analysis on CBO’s output and filling in the gaps in its analysis.