Four Years of PART: Should Taxpayers Really “Expect More” from the Federal Government?

A government bureau is the nearest thing to eternal life we’ll ever see on this earth.
– President Ronald Reagan

From the vantage point of an average taxpayer, the federal government often appears to be an impenetrable fortress of bureaucrats cemented together with mind-numbing protocol and reams of paperwork. If asked, most Americans couldn’t identify the purpose behind programs like the Department of Education’s Even Start, the Treasury Department’s Bank Enterprise Award, or the Justice Department’s Weed and Seed project. Most taxpayers know even less about whether or not these programs are actually accomplishing assigned purposes. Unlike company earnings reports, from which markets tend to reward or punish an enterprise through the worth of its stock, taxpayers still do not have a clear-cut way to discover, let alone police, the performance of federal agencies financed by America’s tax dollars.

Recent administrations have tried to connect program performance to the allocation of budget resources through a variety of initiatives, the most recent of which is the Bush Administration’s Program Assessment Rating Tool (PART). By providing programs with numerical ratings and systemic recommendations, the hope is that government will become “better” through evaluations, thereby giving taxpayers more for their money. The Office of Management and Budget (OMB) has even rolled out a new website, www.expectmore.gov, to help publicize the ratings.

Yet, the unanswered question is, should Americans really “expect more” from government? On one hand, taxpayers stand to benefit from a government that is transparent and self-probing with respect to its methods and outcomes. Tools that condense vast amounts of information on Washington’s performance into digestible bites will assist the average taxpayer in his or her attempt to demand a more responsible government. Congressional appropriators also stand to benefit from incorporating performance information into budget allocations.

As a general rule an effective government program is more desirable than an ineffective program,[i] yet in most cases no program is still the preferable alternative – and labeling big government “effective” could even make efforts to shrink it all the more difficult. For example, does officially rating our contribution to the United Nations Development Program as “Effective,” hurt limited-government advocates seeking to cut off the flow of taxpayer money to programs we shouldn’t be funding in the first place? Perhaps America should rediscover the virtue of expecting less from government. This Policy Paper takes a closer look at what taxpayers have learned (and not learned) about the federal government’s performance from PART and examines the future of performance budgeting.

The growth of big government in the post-World War II era led many elected officials to fret that voters might punish them at the polls if the burgeoning new bureaucracies did not prove their worth to the taxpayers footing the bill. So began a series of systems, tools, and initiatives linking performance to budgeting under almost every presidential administration.[1] The Hoover Commission established by President Truman suggested a “performance budget” to help downsize the federal government, and later President Johnson expanded the Planning-Programming-Budgeting-System (which sought to link budgets with results) to the entire federal government. Years afterward President Nixon had his Management by Objectives, while President Carter employed Zero Base Budgeting during his time in the Oval Office.

As part of President Clinton’s effort to “re-invent” government in the early 1990s, the Government Performance and Results Act (GPRA) enacted in 1993 required agencies to set goals, measure performance, and then publicly report on their results.[2] GPRA differed from previous executive branch-based initiatives in that it was codified into law and required consultation with Congress. As part of this, GPRA called for a pilot system that would link results with spending through a measurement program, and in 1999, OMB designated five such programs to serve as test cases.[3] This pilot metric served as the forerunner for the Bush Administration’s PART.

Given the number of past initiatives to improve federal performance and accountability, “re-inventing” government has proved difficult to accomplish within the span of a President’s tenure. Institutional inertia certainly stands in the way of sweeping change, yet practically every President wants to appear as if he is improving the way government works. Developing a new performance-budgeting program or expanding on an older model is one way to demonstrate this commitment, even if the data gathered and outcomes achieved are somewhat diluted in effect.

Upon taking office President George W. Bush sought to formalize performance budgeting as one of five pillars in his “President’s Management Agenda,” which resulted in the development of the PART for the FY2004 budget request. According to the Office of Management and Budget:

The PART was developed to assess and improve program performance so that the Federal government can achieve better results. A PART review helps identify a program’s strengths and weaknesses to inform funding and management decisions aimed at making the program more effective. …Because the PART includes a consistent series of analytical questions, it allows programs to show improvements over time, and allows comparisons between similar programs.[4]

Essentially, the PART asks federal agencies to answer a series of 25 base questions along with additional questions tailored to the 7 categories[ii] of federal programs (including both discretionary and mandatory). Each program is then reviewed by an independent group separate from those who administer the program and given a numerical rating[iii] by OMB on four weighted factors: 1) Program Purpose and Design (20 percent), 2) Planning (10 percent), 3) Management (20 percent), and 4) Results (50 percent). Based on these factors, programs are rated “Effective,” “Moderately Effective,” “Adequate,” “Ineffective,” or “Results Not Demonstrated.” The latter category is reserved for programs that have “not been able to develop acceptable performance goals or collect data to determine whether it is performing.”[5] OMB has designated Ineffective and Results Not Demonstrated programs as “not performing.”[iv] Recommendations for improvement also accompany each rating.

Given the large number of federal programs, OMB decided to measure one-fifth of the federal government each year between FY2004 and 2008, and as the table below indicates, a large number of programs have been scrutinized so far. With President Bush’s delivery of the FY2007 budget, we now have access to PART scores for 80 percent of federal programs. Since PART is supposed to indicate “how well a program is performing so the public can see how effectively tax dollars are being spent,”[6] what can taxpayers learn from PART at this point?[v]

Fiscal Year	2004	2005	2006	2007	2008
Total Number of Programs Reviewed	234	399	607	793	not yet completed
Effective	14	45	89	125	not yet completed
Moderately Effective	56	105	160	228	not yet completed
Adequate	34	82	157	221	not yet completed
Ineffective	12	19	22	29	not yet completed
Results Not Demonstrated	118	148	179	190	not yet completed

Source: Compilation by Kristina Rasmussen, National Taxpayers Union, from OMB data.

A cursory glance at PART data would indicate that government has become “better” over the past few years. The percentage of Effective programs is up 10 points, while the percentage of Ineffective programs is down slightly. Of course, the bulk of this gain in the Effective and Moderately Effective categories has come from a drop in Results Not Demonstrated programs. In fact, between the FY2005 and 2006 ratings, 80 percent of the gain could be attributed to programs moving out of the Results Not Demonstrated category into the Adequate, Moderately Effective, and Effective groupings. It could be that familiarity with PART requirements has increased the number of programs that have created acceptable measurements for their performance, thereby boosting many programs out of the Results Not Demonstrated category. Also worth noting is few programs, if any, fell in rating over the past 4 years.

Source: Calculations by Kristina Rasmussen, National Taxpayers Union, based on OMB data.

What might be most disturbing about the data depicted in Chart 1 is that, 4 years into the PART process, more than a quarter of federal programs are not performing (by virtue of being Ineffective or unable to measure themselves) even though many of these same programs have existed for decades. A Government Accountability Office (GAO) report found that “several agencies struggled to identify appropriate outcome measures and credible data sources before they could evaluate program effectiveness.”[7] Even programs dealing with concrete calculations, such as the IRS Tax Collection program, have yet to come up with concrete standards of success. Should taxpayers be thrilled or dismayed that the IRS Tax Collection program has been rated Results Not Demonstrated for 4 years running?

In all fairness, OMB has made it a stated priority to move programs out of the Results Not Demonstrated rating, and the percentage of programs in this category has halved over the last 4 years. One can hope this number will continue to diminish as evaluated programs have the chance for reassessments after making significant program or measurement changes.

Questions have also been raised about the quality of the PART measurements. One might expect that the millions of people affected by Hurricanes Katrina and Rita would probably beg to differ with the “Adequate” rating given to the FEMA disaster response program in the FY2006 and 2007 ratings, which included 80 percent for “planning” and 100 percent for “management.”

As the GAO report concluded, “Regular performance reporting is key to good management, but requiring all federal programs to conduct frequent evaluation studies is likely to result in superficial reviews of little utility and to overwhelm agency evaluation capacity.”[8]

OMB’s Deputy Director for Management, Clay Johnson III, has a good point in stating, “We believe that without measurement, there is no certainty about how Federal programs are performing for the American people.”[9] PART can be a useful tool for measuring performance, but taxpayers need to be aware of what exactly is being measured. Are federal welfare programs effective because they efficiently process welfare checks or because they boost people back into self-sufficiency? OMB, to its credit, does differentiate between “outputs” and “outcomes,” but designing tools to measure the latter are more difficult than rating the former.

While the nature of some programs might pose difficulties in measuring effectiveness, an institutional insight to the measurement quandary can be inferred from questions included on the “PART Frequently Asked Questions” page on the OMB website.[10] Among the FAQs answered were, “Do we have to have an efficiency measure for every program?” and, “How do I demonstrate impact through a program evaluation?” The very fact that basic notions of measuring efficiency and impact (rigorously tracked in the private sphere) are so alien to bureaucrats only underscores the need to disband parts of the bloated federal government.

“Performance” is only half of the “performance-budgeting” equation, and there is mixed evidence whether or not the PART ratings have had a demonstrable effect on slashing ineffective programs. OMB’s webpage clearly states that program funding cuts do “not necessarily” follow poor ratings; on the other hand, good ratings don’t guarantee increased funding. Out of the 793 programs rated by PART in the FY2007 budget request, President Bush recommended zeroing out 59 programs. Of these, 2 received Effective assessments, 5 were Moderately Effective, 11 rated as Adequate, 11 were Ineffective, and 30 were Results Not Demonstrated.

While it is important to remember that recommendations for program termination can be caused by a variety of factors (completion of mission, being duplicative of other programs, or not being a high political priority), the evidence does suggest that the Bush Administration has utilized PART scores when calculating budget requests. When comparing the amount of enacted program funding to the 2006 and the 2007 request amounts, Effective and Moderately Effective programs saw increases in their requested budgets while the other categories sustained losses.

Source: Calculations by Kristina Rasmussen, National Taxpayers Union, based on OMB data.

Chart includes programs that were completely eliminated (100 percent cut) but not newly appropriated programs, programs with no funding listed, or those programs showing a negative budgeting balance.

For performance budgeting to really work, Congress must consider evaluative data included in the President’s budgeting requests. Up until now, this hasn’t necessarily been the case. For example, of the 15 programs measured by PART that were zeroed out in the FY2005 budget proposal, 13 were marked again for being zeroed out in FY2006. Congressional appropriators ignored the President’s recommendations and restored funding even though 4 of the programs were deemed Ineffective and 7 were marked Results Not Demonstrated. Out of the 48 programs measured by PART that were zeroed out in the FY2006 budget proposal, at least 27 were again marked for termination in the FY2007 budget. This finding reinforces the notion that performance scores are often no match for the interplay of congressional backing and outside lobbying that serves to salvage a program marked for termination in the President’s budget.

President Bush most recently highlighted the impact of PART during his 2006 State of the Union speech by stating, “This year my budget will … reduce or eliminate more than 140 programs that are performing poorly or not fulfilling essential priorities.”[11] The decision to cut many of these programs was connected to the PART scores: 102 of the cut programs received low ratings under the PART system. However, at least 54 were targeted in last year’s budget too.

Out of the 793 programs reviewed by the PART, the FY2007 budget proposal would cut spending for 302 programs. If these recommendations were fully enacted by Congress, federal spending would fall by $47.5 billion – still less than 2 percent of all proposed outlays. For PART to truly have even a minor impact in the future, Congress will need to take performance ratings more seriously in their budgeting decisions.

There is an ongoing debate among economists and other free market advocates over whether or not the Bush Administration should really be encouraging Americans to “expect more” from the federal government. The instinctive response might be “if we have big government, let’s at least hope it is effective.” However, when the interests of taxpayers do not necessarily coincide with the goals of some federal programs (e.g., expansive environmental regulation), one would hope the programs are actually inefficient in carrying out big-government tasks that serve to expand dependency and distort markets.

While some critics have derided PART as a political hatchet for ending programs the Bush Administration disagrees with ideologically, PART might actually encourage maintenance of big government. No administration, right or left, wants to preside over an “ineffective” government, and no agency head wants to report on a “non-performing” program. The intrinsic desire to show positive results (and justify one’s funding) could result in agencies using PART to self-perpetuate their existence. As if to prove this point, between the FY2005 and 2006 PART, no program could be found that declined in its rating.

Saying that government programs are “working” may make it harder for small-government activists to reduce or eliminate program funding. For example, Table 2 shows that the seven most costly federal programs sampled in 2007 were rated either Adequate or above. When committee chairmen and special interest lobbyists can use government reports to prove that their pet program, no matter how big or small, is effective, what is the taxpayer advocate to say? A never-ending circle of government self-justification serves to squeeze out those who might suggest otherwise.

Program	FY2007 Rating	Requested Program Funding Level for 2007 (dollars in millions)
Food Stamp Program	Moderately Effective	$36,003
Highway Infrastructure	Moderately Effective	$37,650
Social Security Supplemental Income	Moderately Effective	$40,702
Federal Employees Retirement	Adequate	$61,427
Social Security Disability Insurance	Moderately Effective	$100,051
Military Force Management	Effective	$110,776
Medicare	Moderately Effective	$453,890

Source: Compilation by Kristina Rasmussen, National Taxpayers Union, from OMB data.

Another concern is that PART might be used as an excuse by agency heads to clamor for more resources. In the past, agency officials interviewed by GAO cited the challenge of “obtaining the financial resources to conduct independent evaluations”[12] as a barrier to completing evaluations. Translated from bureaucrat-speak, this means, “we don’t know how to measure ourselves but give us more money and we’ll somehow figure it out.” The hope is that savings garnered from PART will outweigh expenses for the program, but taxpayers should be wary of PART being forcibly inducted into a big-government cheerleading squad.

Thus far PART has been an executive branch project and the future of its existence beyond the Bush Administration is uncertain. Even though Congress isn’t known for following PART recommendations when it comes to cutting program funding, some Members are trying to formalize the procedure into statute by passing the Program Assessment and Results Act (H.R. 185), which would “require the review of Government programs at least once every 5 years for purposes of evaluating their performance.”

While taxpayers should salute attempts to connect program performance to budgeting decisions, they must also demand that the effort goes beyond an “attempt.” As the various presidential initiatives of the past 50 years have shown, the concept of performance budgeting is nothing new. It is one thing to assemble stacks of reports on how programs are faring, but quite another to get results based on such recommendations. As the Congressional Research Service has noted, “performance information may make a positive contribution to the federal budget debate, but a performance budget will not eliminate the need for difficult political choices.”[13]

At the end of the day, the 4-year-old PART is a tool that taxpayers should be able to add to their kit of small-government promotion strategies. While PART has some maturing to do in terms of how and what it measures, it has put agency heads on notice that taxpayers are watching performance standards and it offers Congress a valuable method for streamlining the size of government. As always, government will only truly deliver “more” to taxpayers when legislators are willing to confront the thicket of special interests and others looking for handouts – and cut off the dead wood that serves to rot our budget.

Kristina M. Rasmussen is Government Affairs Manager for the National Taxpayers Union, a non-profit, non-partisan organization founded in 1969 to work for lower taxes, smaller government, and economic freedom at all levels. For further information, visit www.ntu.org.

[i] In some cases, a program’s ineffectiveness can actually be a desirable thing if this translates into spending less on programs (such as wealth transfers) whose motives and outcomes are antithetical to the idea of limited government.

[ii] Direct Federal (e.g., Dairy Price Support System), Competitive Grant (e.g., Farmland Protection Program), Block/Formula Grant (e.g., Food Stamp Program), Research and Development (e.g., Food Safety Research), Capital Assets and Acquisition (e.g., National Forest Improvement and Maintenance), Credit (e.g., Agricultural Credit Insurance Fund Direct Loans), and Regulatory Based (e.g., Food Safety and Inspection Service).

[iii] Numerical Scoring: Effective 85-100, Moderately Effective 70-84, Adequate 50-69, and Ineffective 0-49.

[iv] In layman’s terms, the latter description means evaluators simply can’t tell how these programs are doing.

[v] A word of warning from OMB: “While some changes have been made to improve the consistency and accuracy of the PART ratings, they are not significant enough to prevent comparisons between years or show changes over time. The guidance and a few PART questions changed from 2004 to 2005. For the 2006 PART, the questions remained the same as 2005, but changes were made to the PART guidance.”

Papers

Four Years of PART: Should Taxpayers Really “Expect More” from the Federal Government?

NTU Policy Paper 119