Management-based approaches to quality assurance in the USA
Across a number of domains, management-based regulation has emerged as a viable alternative to both input-focused, bureaucratic approaches and outcome-focused, performance-based approaches. In a management-based system, the regulator establishes some threshold common metrics as well as a requirement that the regulated entities develop their own outcome targets and plans for achieving them. Oversight is focused on how the regulated entities have carried out their plans, progressed toward the self-determined targets, and achieved the common metrics. Failure to meet the minimum threshold of the common metrics or to make progress on self-determined targets triggers a reexamination and refinement of the regulated entity’s plans. Repeated failure to meet the targets, follow-through, or revise the plan can have more-significant consequences.
Management-based regulation is grounded in the idea that regulation should reinforce organizational learning by the regulated entity. Regulatory intervention at the planning stage—coupled with monitoring, benchmarking against peer organizations, and enforcement of minimum standards—helps organizations build capacity for self-regulation and improvement towards rigorous, context-appropriate outcomes. Both regulators and institutions gain a better sense of how contextualized actions are related to outcomes, as well as the feasibility of achieving outcomes in different contexts. This learning process equips the institutions to sustain continuous improvement, and also enables the regulator to discover and set new minimum thresholds based on best-in-class outcomes.
This approach resembles the current system of higher education accreditation, but there are some key differences, and several ways in which features of a management-based system could make the current quality assurance system for higher education more rigorous, transparent, and open to innovation and improvement. First, common metrics for similar institutions focused on characteristics such as financial viability and student outcomes, would improve comparability among peers and open pathways for innovative providers whose processes or infrastructure do not meet input-based standards. These metrics would trigger additional review and support by regulators (federal or regional), while incentivizing institutions to build their own evaluative capacity.
In the context of higher education quality assurance, more-frequent monitoring has the potential to improve accountability, offer more opportunities for institutional improvement, and reduce the resource burden and tendency for “box-checking” that is criticized in the current process.
The management-based approach we propose also requires closer, more frequent interaction between the regulator and the regulated entity to monitor planning, revision, and progress. In the context of higher education quality assurance, more-frequent monitoring has the potential to improve accountability, offer more opportunities for institutional improvement, and reduce the resource burden and tendency for “box-checking” that is criticized in the current process. Finally, the management-based approach contemplates differentiated supports and consequences for institutions that struggle to implement plans or achieve goals. This provides institutions with information and incentives for improvement that are more useful than bimodal accreditation decisions do. If reports and results are made public, the approach would also provide consumers with more meaningful information about institutions than the current system does.
This section reviews a variety of quality assurance schemes in United States higher education, international higher education, and across other sectors, each of which uses elements of a management-based approach. For each system, we provide a high level description, and then explain the key mechanisms of the system that incentivize institutional learning and continuous improvement, leading to improved quality at the organizational and system-wide level. Though few empirical evaluations of the efficacy of these schemes exist, we draw from these examples and the literature about them to distill several key design principles for an improved system for U.S. postsecondary education.
Management-based efforts to reform higher education quality assurance in the United States
A number of professional and disciplinary accrediting bodies in the United States have adopted elements of this management-based approach, and some have evaluated the effectiveness of these approaches in supporting improvement. For example, to ensure baseline quality and comparability while allowing for programmatic diversity, the Accreditation Board for Engineering and Technology (ABET) bases its assessment and accreditation of engineering programs on student performance on broad, industry-aligned learning outcomes, as well as program-defined objectives and plans for continuous improvement. Baseline and program-defined standards and plans are assessed through a self-study and site-visit by volunteer peers, industry experts, and government experts, and reviews can result in a number of actions, including approval for accreditation of reaccreditation, requests for further reporting or additional visits to show progress in identified weaknesses, or non-accreditation. Programs that are approved for accreditation are reviewed on a six-year cycle; those that demonstrate weaknesses or deficiencies are reviewed via additional reporting, site visits, or both on a more frequent basis.
There is evidence that ABET’s focus on baseline outcomes standards and program-defined standards has been effective at helping programs improve. A 2006 impact evaluation of these standards found that the hybrid approach, which replaced a more input-based, prescriptive approach in 1997, incentivized programs to focus curricula more on professional skill development while encouraging planning for continuous improvement among varied stakeholders. In addition, students report higher levels of engagement, perform better on learning outcomes assessments, and maintain more technical skills than they did with an input-based approach.
The Department of Education is also experimenting with tailored, management-based forms of quality assurance, with the goals of accommodating innovative providers while ensuring accountability amongst high-risk entrants. Launched in 2015, the ED’s Educational Quality through Innovative Partnerships (EQUIP) experimental sites initiative extends federal financial aid eligibility to programs run in partnership between traditional educational providers (Title IV institutions) and non-traditional educational providers (such as coding boot camps and MOOC providers). The initiative aims to provide innovation options to low-income students, while testing ways to assure rigor and quality for new postsecondary models.
To achieve these ends, each pair of partners works with an independent Quality Assurance Entity (QAE) to develop program-specific outcomes, plans for meeting those outcomes, and methods of assessment. For example, Entangled Ventures, the QAE for UT Austin and the coding boot camp Maker Square, audits the program’s performance based on learning outcome assessments, employment outcomes, and student satisfaction. The Council for Higher Education Accreditation, which is the QAE for Dallas Community College District and Straighter Line partnership, will evaluate outcomes related to learning, transfers, and comparability across programs in order to assess the program’s effectiveness at providing students with low-cost, high-quality educational experiences that can be recognized for credit at Title IV institutions. Assessment processes differ by partnership, but typically include a self-review, an assessment of processes, documents, and student work by an external team of experts, and recommendations for both accreditation and improvement.
Finally, regional accreditors have made some recent changes that indicate movement toward management-based regulation. In September 2016, the Council of Regional Accreditation Commissions (C-RAC) announced that it would expand its review of four-year institutions with graduation rates at or below 25 percent and two-year institutions at or below 15 percent. Accreditors would supplement this review with data on transfer rates, and then follow up with institutions identified as high-risk to get more information on the conditions that led to low graduation rates and institutional plans for improvement. Combined with regional accreditors’ shift over the past two decades toward a focus on defining and assessing learning outcomes, these changes lay the foundation for management-based regulation.
Though varied, each of these systems represents a hybrid approach that combines an evaluation against common metrics with a review of plans, processes, and, in some cases, provider-determined outcomes and self-assessments. Moreover, the EQUIP quality assurance system and the new C-RAC policies provide increased opportunities for review and interaction for new entrants and bad actors. These features have the potential to ensure a higher level of quality among higher-risk providers, while providing supported and regular opportunities for organizational learning and improvement.