Few would engage in the demanding work of nursing without some degree of belief that their efforts make a difference. Nursing care is instrumental in achieving quality outcomes for patients, families, and communities. This assumption is foundational to the discipline of nursing and the social contract between nurses and the public. Few would engage in the demanding work of nursing without some degree of belief that their efforts make a difference. Likewise, nursing could not have emerged as the largest sector of the healthcare workforce unless employers and payors also assumed that nurses could make a valuable contribution to patient care. However, empirical evidence to support the unique contribution of nurses to quality outcomes is currently lacking (Needleman, Kurtzman, & Kizer, 2007). The discipline has long been criticized as a practice based more on tradition than evidence and this criticism has been met with multiple professional initiatives to stimulate evidence based practice and outcomes research. Measurement of nursing outcomes is a linchpin for both of these important and interrelated activities.

...assumptions about the nursing contribution to care must be replaced with empirical evidence of the nursing profession’s actual contributions to care. The lack of evidence to delineate the nurse’s contributions to care is problematic. In response to declining reimbursement and escalating quality and safety problems, stakeholders in the healthcare industry now challenge traditional assumptions across the delivery system and seek empirical evidence to guide system redesign. The goal of system redesign is encapsulated in the Institute of Medicine (IOM) six aims for care: safe, effective, timely, equitable, patient-centered, and efficient (IOM, 2001). Continued support is unlikely for care providers and processes that lack evidence of a meaningful contribution to these aims. Therefore, assumptions about the nursing contribution to care must be replaced with empirical evidence of the nursing profession’s actual contributions to care. The purposes of this article are to examine the imperatives, ideals, history, and challenges related to effective outcome measurement in nursing and to identify recommendations for action.

Imperatives to Measure Nursing Outcomes

...nurses are accountable for and obligated to measure interventions and outcomes... Nurses have a social and an economic imperative to measure outcomes. The social imperative is described in the American Nurses Association (ANA) Social Policy Statement (ANA, 2010) and is codified in each nurse practice act. Nursing exists for the sole purpose of serving the public good; therefore, the public owns the discipline. Through the process of licensure, the public grants nurses the privilege to practice the art and science of nursing and in exchange, the public expects to receive quality care (ANA, 2010). Thus, the social contract between health professionals and the public dictates that nurses engage in self-regulation to assure quality performance.

Assurance of performance requires measurement of performance (Donabedian, 2003). Moreover, the types of outcomes that nurses are duty-bound to measure and manage are inherent in the very definition of nursing, “…the protection, promotion, and optimization of health and abilities, prevention of illness and injury, alleviation of suffering through the diagnosis and treatment of human response, and advocacy in the care of individuals, families, communities, and populations” (ANA, 2010, pp. 8). Therefore, nurses are accountable for and obligated to measure interventions and outcomes in the areas of health promotion, prevention of illness and injury, and alleviation of suffering.

The economic imperative to measure and manage nursing outcomes is an extension of the social contract. Nurses are obligated to serve the public good through stewardship of health resources. Expending resources on nursing services with no clear benefit or failing to allocate resources for nursing services with a clear benefit is not good stewardship. Moreover, neither practice is consistent with the IOM (2001) aims for quality care. Therefore, nurses have a social obligation to develop and document the evidence base for the entirety of nursing practice and shift from a task-based practice, which emphasizes what nurses do for patients, to an outcome-based practice that emphasizes what nurses achieve with patients.

External forces are now facilitating this shift toward an outcome-based practice for nursing and strengthening the imperative for outcome measurement. External forces are now facilitating this shift toward an outcome-based practice for nursing and strengthening the imperative for outcome measurement. In response to economic and quality concerns in healthcare, reimbursement policies have shifted from fee for service to value-based purchasing (VBP) (Bodrock & Mion, 2008; Kurtzman, Dawson, & Johnson, 2008). Under the new VBP policies, provider reimbursement is tied to performance, defined as compliance with evidence-based practice guidelines and achievement of a growing list of desired patient outcome targets (Table 1). Consequently, organizations that fail to meet established performance targets incur penalties and lose revenue (Hines & Yu, 2009). The potential for lost revenue has many implications for nursing and patient care. As summarized in the business axiom, “no margin, no mission,” the mission of healthcare organizations is contingent upon a healthy profit margin. History demonstrates that when profit margins decrease, healthcare organizations invest fewer resources in services for patients and employees, to include nurses (Aiken, Clarke, & Sloane, 2001; Kavanagh, Comiotti, Abusalem, & Coty, 2012). This action translates into a downsized nursing workforce, fewer patient services, and decreased quality of care (Kurtzman et al., 2011; Timmers, Hulstaert, & Leenan, 2014).

Table 1. Value-Based Purchasing Measures Used by CMS in 2016.^a

Value Based Purchasing Measures
Patient Experience of Care	Patient Experience
	Communication with nurses Communication with physicians Responsiveness of hospital staff Pain management Communication about medications Cleanliness and quietness of the hospital environment Discharge information Overall rating
Process Measures	Acute Myocardial Infarction (AMI)
	Fibrinolytic within 30 minutes of arrival
	Influenza
	Screening and immunization
	Pneumonia
	Initial antibiotic selection
	Surgical Care Improvement Program
	Pre-operative beta blocker Prophylactic antibiotic selection Controlled post-operative blood glucose in cardiac patients Post-operative urinary catheter removed by day 2 Deep Venous Thrombosis (DVT) prophylaxis within 24 hours of surgery
Patient Outcomes	Mortality
	30 day mortality for AMI 30 day mortality for pneumonia 30 day mortality for heart failure
	Hospital Acquired Infections
	Central Line Associated Bloodstream Infection (CLABSI) Catheter Associated Urinary Tract Infections (CAUTIs) Surgical Site Infections (abdominal hysterectomy and colon surgery)
	Complications
	Pressure ulcers Iatrogenic pneumothorax CLABSI Post-operative hip fracture Post-operative pulmonary embolism or DVT Post-operative sepsis Post-operative wound dehiscence Accidental puncture or laceration
Efficiency	Medicare spending per beneficiary

Table 1 extrapolated from Raso (2015)

VBP policies are predicated on the idiom that what gets measured, gets managed. The outcomes addressed in current VBP policies primarily relate to the prevention of injury (e.g., nosocomial infection, pressure ulcers) and do not address other outcomes germane to the nursing profession’s social contract (i.e., health promotion and the alleviation of suffering). This is problematic for patients and nurses. VBP policies are predicated on the idiom that what gets measured, gets managed. In other words, measurement of an outcome draws attention to that outcome and ultimately to its contributing factors. When performance on an outcome influences provider reimbursement, the increased attention is often accompanied by a reallocation of resources to correct the problem. Consequently, investment of resources in health promotion and alleviation of suffering is more likely to occur if related outcomes are measured and included in VBP policies. Therefore, nurses have a social and an economic responsibility to advocate for representation of these important patient outcomes in VBP policies, to document how nursing contributes to these outcomes, and to help organizations meet related targets.

Ideal Conditions & Challenges for Outcomes Measurement

The act of measurement makes phenomena visible and functions to elevate actual and/or potential quality problems in the consciousness of stakeholders. There are no intrinsic benefits in measuring nursing outcomes; rather, the benefits are contingent upon whether or not the act of measurement results in changes that lead to improved quality of care (Farquhar, Kurtzman, & Thomas, 2010). The act of measurement makes phenomena visible and functions to elevate actual and/or potential quality problems in the consciousness of stakeholders. Regardless of intention, measure selection communicates importance and generates selective attention that ultimately drives selective improvement (Kurtzman & Jennings, 2008). Therefore, outcome measurement is best conceived as one interdependent component of the broader quality management enterprise with intentional selection of measures to drive improvement in outcomes relevant to nursing. The selection of quality measures sets is guided by conceptual models of quality, scientific standards, and the pragmatics of data collection.

Conceptual Foundations of Quality

Outcomes of care are generally regarded as the most important element of quality assessment... The predominant conceptual model guiding quality management in healthcare was first described by Donabedian in 1966 and is known as the Structure-Process-Outcomes (SPO) model (Donabedian, 1966; Donabedian, 2003). Quality assessment is recognized as an essential aspect of quality management and three essential elements of quality assessment are identified: structures, processes, and outcomes of care (Figure). Outcomes of care are generally regarded as the most important element of quality assessment because they focus on patients and reflect the social mandate and ultimate goals of healthcare providers (i.e., changes in health status) (Donabedian, 2003). However, outcomes offer no guidance for quality improvement unless they are related to modifiable antecedents (e.g., structures and processes of care). Improving outcomes without knowledge of specific process failures is analogous to treating a disease of unknown etiology and can result in misdirected and/or delayed corrective action. Therefore, quality assessment is best achieved through a set of measures that include outcomes and associated structures and processes.

Figure. Structure-Process-Outcome (SPO) Framework for Quality Assessment in Nursing

[View full size]

Donabedian definitions in Figure extrapolated from Donabedian (2003)
Nurse-Sensitive examples in Figure extrapolated from domain specific frameworks described in the following publications: Burston, Chaboyer, & Gillespie (2013); Doran (2011); Dubois, D’Amour, Pomey, Girard, & Brault (2013); and Heslop & Lu (2014)

Within the SPO framework, ideal measure sets for quality assessment meet the following criteria (Donabedian, 2003; Needleman et al., 2007):

• Provide a complete evidence-based model of the intervention of interest.
• Address the full continuum of outcomes expected to be influenced by the intervention of interest.
• Include measures that are sensitive to change in the aspect of care being evaluated.

This means that effective quality assessment for the discipline of nursing is contingent upon a body of evidence that demonstrates relationships between the full complement of nursing interventions and the full scope of outcomes sensitive to nursing interventions. However, the very nature of nursing and health outcomes pose significant challenges in the application of these criteria for quality assessment in nursing.

...health outcomes are rarely, if ever, the result of a singular process or provider. The complex nature of healthcare is well documented (IOM, 2004). Patients typically receive care from teams of semiautonomous providers representing multiple disciplines and specialties working interdependently. Therefore, health outcomes are rarely, if ever, the result of a singular process or provider. For example, achievement of high functional status in a patient diagnosed with a cerebrovascular accident requires diagnostic and therapeutic interventions from physicians, nurses, physical therapists, occupational therapists, and dieticians. Each discipline is necessary for recovery of function, yet none is sufficient. Thus, quantifying the unique contributions of each individual discipline is challenging. The combined dependent, independent, and interdependent components of nursing practice add to this challenge (Doran, 2011). Nurses not only diagnose and treat patient responses to health problems, they also carry out interventions prescribed by other disciplines. Additionally, nurses are primarily responsible for the coordination of care activities within and across healthcare teams. Therefore, there are few elements of care that do not pass through the hands of nurses, and few outcomes that are not influenced in some way by nursing care.

In contrast to disciplines with highly technical interventions, many nursing interventions are more akin to the information transfer process of knowledge workers, and therefore more difficult to isolate and measure. Nursing care is often more difficult to conceptualize and measure than care from other disciplines. In contrast to disciplines with highly technical interventions (e.g. surgical procedures), many nursing interventions are more akin to the information transfer process of knowledge workers, and therefore more difficult to isolate and measure (Matney, Maddox, & Staggers, 2014). For example, the nursing intervention of surveillance does not occur in a singular space or time. Rather, surveillance involves a series of interdependent activities to include the gathering of multiple pieces of information from multiple sources; analyzing the collected information for meaningful patterns; identifying the need for further intervention(s); and initiating appropriate action(s) (Fasolino & Verdin, 2015; Jones, 2011). Sometimes surveillance is initiated by one nurse and continued by other nurses throughout an episode of care. Therefore, surveillance cannot be conceived or measured as a discrete intervention (e.g. present or absent) completed by a single nurse.

The intervention of surveillance also may require communication with other providers to initiate action that falls outside a nurse’s independent scope of practice. Thus, in the instance of surveillance and other interdependent interventions, the ultimate outcome may be influenced as much by other providers as by nurses (Anthony, 2008; Jones, 2011; Kurtzman et al., 2011). Although nurses may contribute to a broad array of patient outcomes, the expected contribution is variable, and teasing out the unique nursing contribution to patient outcomes is difficult (Needleman et al., 2007). Not surprisingly, some nurses are unwilling to accept responsibility for outcomes over which they lack complete control.

Not surprisingly, some nurses are unwilling to accept responsibility for outcomes over which they lack complete control. The Donabedian model is a useful organizing framework for quality assessment in healthcare, providing a generic overview of the essential categories of quality measures and supporting the necessity of empirically supported relationships within measure sets. However, this framework is not prescriptive and offers little guidance in the identification of specific quality measures. Instead, specific quality measures must be derived from domain and/or problem specific frameworks. Numerous quality assessment and performance monitoring frameworks specific to nursing now exist that propose measures sensitive to the nursing contribution to patient care (Figure). In contrast to frameworks based on the medical model of care (Donabedian, 2003), patient outcome categories proposed in nursing quality frameworks are typically more closely aligned with nursing’s social mandate.

Standards for Quality and Outcome Measurement

Consensus standards for development of quality measures now exist and the National Quality Forum (NQF) is recognized as the as the common pathway for review and endorsement of quality measures in healthcare (Quraishi & Jordan, 2014). Federal law stipulates that policy makers defer to these standards when establishing healthcare policies such as VBP (Bodrock & Mion, 2008; Needleman et al., 2007). To be endorsed, quality measures must meet established standards in the following areas: importance, scientific acceptability, usability, and feasibility (NQF, 2004). These standards address issues related to the social mandate of nursing (importance), empirical evidence linking practice to outcomes (scientific acceptability and usability), and the pragmatics of data collection (feasibility).

Outcomes that are not empirically linked to specific malleable processes are not useful because they do not help decision makers determine how to improve care. Quality measures are considered important to the extent that they address priority quality issues such as the IOM (2001) six aims and, in the case of nursing, the social mandate. Standards for scientific acceptability relate to the reliability and validity of measures and the strength of evidence linking measures to quality. Measures must be clearly defined and provide consistent and credible estimates of performance. The usability standard reflects the degree to which a measure is understandable to those making decisions that influence quality. To be useful, a measure must provide information that is actionable. Outcomes that are not empirically linked to specific malleable processes are not useful because they do not help decision makers determine how to improve care.

Pragmatic Measurement

Measures based on data collection methodologies intrinsic to care processes and captured by existing data systems are highly desirable. Measurement is essential to quality assessment. However, the act of measurement also consumes human and material resources. The NQF standard of feasibility reflects the balance between the cost/burden of data collection and the potential benefit of the information generated. Measures that are too burdensome and costly are less likely to be adopted for quality assessment. Measures based on data collection methodologies intrinsic to care processes and captured by existing data systems are highly desirable (Burston, Chaboyer, & Gillespie, 2013; Needleman et al., 2007).

In contrast, measures based on manual audits (e.g. chart abstraction, direct observation) and surveys have been considered less desirable. The standard of feasibility is most affected by issues associated with nursing documentation. Conceptually, the most valid source of data about nursing care and patient responses to nursing care is nursing documentation. However, the validity of nursing documentation as a data source is weakened by incomplete documentation. Lapses in nursing documentation hinder the ability to establish empirical links between nursing care and patient outcomes (Alexander, 2007; Anthony, 2008; Needleman et al., 2007).

History of Outcomes Measurement in Nursing

Nurse leaders and scholars have long recognized the need to measure the outcomes of nursing care and have attempted to address the associated challenges at various times throughout history. Following is a summary and critique of the pivotal outcome measurement activities in nursing from the 1970s to the present.

The Minimum Data Set (MDS) Movement

There was little emphasis on outcomes measurement in nursing until the 1970s and 1980s when payors, prompted by escalating healthcare costs, attempted systematic evaluations of the quality and cost of healthcare services. The Department of Health and Human Services (DHHS) supported the development of minimum data sets (MDS) to facilitate these evaluations (Hobbs, 2011). DHHS defined an MDS as the least amount of data needed to evaluate a specific service (Hobbs, 2011) and established the first MDS to evaluate hospital services in 1974: the Uniform Hospital Discharge Data Set (UHDDS).

Due to the paucity of available clinical data at that time, the UHDDS was based on administrative data collected primarily for billing purposes. Nurse leaders were particularly concerned about the absence of information about nursing care in the UHDDS, fearing that this would result in the inability to cost out nursing services and establish the profession of nursing on equal footing with medicine in terms of contributions to inpatient care. Consequently, a core group of nurse leaders collaborated to develop an MDS specific to nursing (NMDS). An NMDS with 16 elements related to nursing care (nursing diagnoses, nursing interventions, nursing outcomes, and nursing intensity), and patient demographics, and service elements (facility, medical record number, nurse identifier, encounter dates, payor source, discharge disposition, and resource consumption) was submitted to DHHS in 1987 but was not endorsed (Hobbs, 2011; Klieb, Sales, Doran, Mallette, & White, 2011).

The Outcomes Movement and the Nurse Report Card

The 1990s marked the beginning of what has become known as the “outcomes movement.” The 1990s brought many activities related to the development and testing of nursing quality and outcome measures and marked the beginning of what has become known as the “outcomes movement.” The ANA led this activity in 1994 with the landmark Nurse Safety and Quality Initiative. This initiative emerged in response to concerns over the detrimental effects of declining hospital reimbursement on the quality of nursing care (Rowell, & Milholland, 1998). The purposes of this initiative were to: identify and evaluate potential nurse sensitive indicators; explore the theoretical and empirical relationships between the identified nursing structure, process, and outcome measures; and generate a nursing report card for acute care settings (Lewin-VHI, 1995). Consistent with the SPO framework, an initial measure set was identified and evaluated based on theoretical links to the quality of nursing care (Table 2). The evaluation process included a review of empirical evidence to support presumed linkages and an assessment of data quality and availability.

Table 2. Nurse-Sensitive Indicators Originally Identified by ANA

		Empirical Studies	Direct Causal Link	Data Readily Available	ANA Report Card
Patient-Focused Outcome Indicators	Inpatient mortality rate	4	No	Yes	No
	Length of Stay	9	No	Yes	No
	Adverse Incidents
	Adverse Incident Rate	2	No	No	No
	Medication Error Rate	6	No	No	No
	Patient Injury Rate	2	No	No	No
	Complications
	Total Complication Rate	3	No	No	No
	Decubitus Ulcer Rate	2	No	No	Yes
	Nosocomial Infection Rate (All Cause)	3	No	No	Yes
	Nosocomial Urinary Tract Infection Rate	3	No	No	Yes
	Nosocomial Pneumonia Rate	1	No	No	Yes
	Nosocomial Surgical Wound Infection Rate	0	No	No	Yes
	Patient Satisfaction
	Satisfaction with Nursing Care	10	No	No	Yes
	Willingness to Recommend Hospital to Others/Use Hospital Again	1	No	No	Yes
	Patient Adherence to Discharge Plan	1	No	No	No
	Readmission Rates	4	No	No	No
	Emergency Room Visits Post-Discharge	1	No	No	No
	Unscheduled Physician Visits Post-Discharge	0	No	No	No
	Patient Knowledge of Disease/Condition and Care Requirements	2	No	No	No
Process of Care Measures	Nurse Satisfaction	7	Yes	No	No
	Assessment and Implementation of Patient Care Requirements
	Assessment of Patient Care Requirements	5	No	No	No
	Development of a Nursing Care Plan	6	No	No	No
	Accurate and Timely Execution of Therapeutic Interventions and Procedures	3	No	No	No
	Documentation of Nursing Diagnoses, Therapeutic Objectives, and Care Given	0	No	No	No
	Pain Management	1	No	No	No
	Maintenance of Skin Integrity	1	No	No	No
	Patient Education	3	No	No	No
	Discharge Planning	7	No	No	No
	Assurance of Patient Safety
	Overall Assurance of Patient Safety	2	No	No	No
	Appropriate Use of Restraints (all)	0	No	No	No
	Appropriate Use of Pharmaceutical Restraints	0	No	No	No
	Appropriate Use of Physical Restraints	0	No	No	No
	Responsiveness to Unplanned Care Needs	2	No	No	No
Structure of Care Indicators	Ratio of Total Nursing Staff to Patients	2	No	Yes	Yes
	RN/Patient ratio	1	No	Yes	Yes
	LPN/Patient Ratio	0	No	Yes	Yes
	Unlicensed Workers/Patient ratio	2	No	Yes	Yes
	Ratio of RNs to Total Nursing Staff (Skill Mix of RNs, LPNs, & Unlicensed Workers)	8	No	Yes	Yes
	RN Staff Qualifications
	RN Staff Experience	5	No	No	No
	RN Staff Education	6	No	No	Yes
	Total Nursing Care Hours Provided Per Patient (Case Mix, Acuity Adjusted)	2	No	No	No
	RN Hours per Patient	0	No	No	No
	LPN Hours per Patient	0	No	No	No
	Unlicensed Workers per Patient	2	No	No	No
	Staff Continuity
	Use of Agency Nurses	0	No	No	Yes
	Use of Float Nurses	0	No	No	No
	Unsafe Assignment Rate	0	No	No	No
	Nurse Staff Turnover Rate	3	No	No	Yes
	FT/PT RN Ratio	2	No	No	No
	RN Overtime	0	No	No	No
	Nursing Staff Injury Rate	0	No	No	No

Table 2 extrapolated from Lewin-VHI (1995)

None of the identified patient outcome measures were supported by a direct causal link to nursing structures or processes. Moreover, data were readily available for only two of the identified outcome measures (mortality and length of stay) and these measures had the weakest theoretical links to nursing care quality (Lewin-VHI, 1995). The identified outcomes were hospital-centric and did not reflect the full scope of outcomes included in the profession of nursing’s social mandate or the full range of outcomes assumed to result from good nursing care. Therefore, a comprehensive evidence-based report card to evaluate the quality of nursing care was not achieved. Instead, a sub-set of nurse structures and patient outcomes, primarily related to injury prevention, were selected for further development and inclusion in the first report card. The nurse structures were selected based on data availability and the outcomes were selected based on the strength of their theoretical link to nursing care quality. Notably, no nurse process measures were selected for further development and none were included in the first report card.

This ANA initiative set the course for the next two decades of activities related to quality and outcomes assessment in nursing. Health services researchers intensified efforts to establish empirical relationships between available measures of nurse structures and patient outcomes related to injury prevention (e.g., adverse events) with minimal attention to nurse processes (Doran, 2011). As empirical evidence emerged and data collection efforts intensified, the number of proposed nurse-sensitive increased incrementally. Additionally, there was a proliferation of national and regional nurse-sensitive quality databases (Table 3) aiming to support quality assessment and demonstrate the nursing contribution patient care using measures similar to those proposed in the ANA initiative (Table 2).

Table 3. Nursing–Sensitive Indicators included in Major Nursing Databases

Nurse-Sensitive Indicators		NDNQIa	CalNOCb	MilNODc	VaNODd
Structures	Nursing Hours per Patient Day	v	v	v	v
	Skill Mix	v	v	v	v
	Contracted Staff Utilization		v	v
	Staff Categories			v
	Percent Direct Care Nursing Staff				v
	Nurse Experience			v
	Nurse Education	v	v	v
	Nurse Work Environment	v		v
	Patient Turnover			v
	Patient Acuity			v
	Nurse Turnover				v
	Nurse Vacancy
	Medication Errors			v
Processes	Pediatric Pain Assessment	v
	Skin Risk Assessment				v
	Fall Risk Assessment				v
Outcomes	Patient Falls	v	v	v	v
	Pediatric Intravenous Infiltration Rate	v
	Pressure Ulcer Prevalence	v	v	v	v
	Restraint Prevalence	v	v	v
	Catheter Associated Urinary Tract Infection	v
	Central Line Associated Blood Stream Infection	v
	Ventilator Associated Pneumonia	v
	Patient Satisfaction with Care		v	v	v
	Patient Satisfaction with Discharge Planning			v
	Patient Satisfaction with Pain Management			v
	Patient Satisfaction with Education			v
	Nurse Satisfaction	v		v	v
	Nurse Needlestick Injuries			v
	Nurse Musculoskeletal Injury				v

a=National Database of Nursing Quality Indicators (Montalvo, 2007)
b=California Nursing Outcomes Coalition (Aydin et al., 2004)
c=Military Nursing Outcomes Database (Patrician, Loan, McCarthy, Brosch, & Davey, 2010)
d=Veteran’s Administration Nursing Outcomes Database (Veteran’s Health Affairs (VHA), 2009)

All of these databases were designed for the evaluation of nursing care in acute care hospitals, primarily on medical-surgical units. Nurse contribution to care in other practice settings received relatively little attention. Moreover, measure specifications and data collection procedures are not standardized (Alexander, 2007; Doran, 2011). By 2002, a total of 150 potential nurse sensitive measures were submitted to the NQF review process. However, only 15 of the proposed measures met the consensus standards for endorsement in 2004 (Table 4).

Table 4. National Quality Forum Endorsed Nurse-Sensitive Indicators

NQF Endorsed Measures Included
Patient Outcomes	Death among surgical inpatients with treatable serious complications (Failure to Rescue) Pressure ulcer prevalence Falls prevalence Falls with injury Restraint prevalence Catheter-associated urinary tract infections for intensive care unit patients Central line catheter-associated blood stream infection rate for ICU and High Risk Nursery (HRN) patients
Processes	Smoking cessation counseling for AMI Smoking cessation counseling for heart failure Smoking cessation counseling for pneumonia
Structures	Skill mix (RN, LVN/LPN, UAP, and contract) Nursing care hours per patient day Practice environment scale Voluntary turnover

Table 4 extrapolated from NQF (2004)

Current Challenges

Imperatives for nurses to engage in self-regulation through quality assessment and outcomes management are stronger than ever. Healthcare has entered a new age of accountability, accompanied by a low tolerance among stakeholders for poor quality and increasing costs (Farquhar, Kurtzman, & Thomas, 2010). Although some progress in the measurement and management of nursing outcomes is evident, major limitations persist. The profession still does not have a set of evidence-based measures that reflect the full scope of nursing services and the full continuum of outcomes sensitive to nursing care.

Despite decades of intentional efforts to this end, the nurse contribution to patient outcomes has not been sufficiently quantified. Despite decades of intentional efforts to this end, nursing's contributions to patient outcomes have not been sufficiently quantified. There are still relatively few scientifically acceptable measures that can reliably capture nursing's contributions to patient care. Notably, nurse-sensitive indicators account for < 3% of the 500-plus measures endorsed by the NQF. Moreover, the NQF has not endorsed any additional nurse-sensitive structure, process, or outcome measures since 2004. State of the science reviews indicate that the challenges of effective quality and outcomes measurement in nursing have changed very little in the past 40 plus years (Alexander, 2007; Griffiths, 1995; Naylor, 2007; Needleman et al., 2007).

Insufficient Evidence Base

The evidence base linking nurse structures with patient outcomes related to adverse events is much more robust than at the time of the ANA report card initiative (Brennan, Daly, & Jones, 2013; Kane, Shamliyan, Mueller, Duval, & Wilt, 2007; Krueger, Funk, Green, & Kuznar, 2013; Lankshear, Sheldon, & Maynard, 2005). However, the effect of staffing and nursing education on patient outcomes, though statistically significant, is relatively small by scientific and clinical standards. Moreover, due to limitations of study designs, the body of evidence produced thus far is still insufficient to support direct causal paths linking nurse structures to patient outcomes. Thus, there is insufficient evidence to explain how nurses contribute to outcomes. Consequently, few evidence based changes in practice have materialized.

Insufficient Process Measures

...the active ingredients of nursing care must be identified and quantified. A key contributing factor in the failure to establish direct causal linkages between nursing care and patient outcomes is the lack of available nursing process measures. Presumably, nurse structures influence patient outcomes through nursing care processes. Researchers cannot establish specific causal chains without capturing the effects of nurse structures on nurse processes and/or the effects of nurse processes on patient outcomes (Needleman et al., 2007). In other words, the active ingredients of nursing care must be identified and quantified. Nurse process measures were not represented in the original nurse report card and remain underrepresented in each database that followed (Table 3). The only nurse process measure endorsed by the NQF relates to smoking cessation counseling for selected hospitalized patient populations. This process measure by no means captures the full scope of nursing practice in any practice setting and is not likely to make a significant contribution to the limited number of patient outcomes endorsed by NQF and included in available nurse databases.

No Standardized Nursing Taxonomy

Efforts to promote a NMDS have continued since the 1980s along with parallel activities to promote a universally accepted taxonomy for nursing diagnoses, interventions, and outcomes (Daly, Maas, & Johnson, 1997; Halloran & Halloran, 2015; Needleman et al., 2007). However, neither has been widely adopted in the United States. Execution of a NMDS requires a standardized language to effectively capture nursing services and outcomes of interest (Mac Neela, Scott, Treacy, & Hyde, 2006). Although universally accepted taxonomies for diseases (i.e., the international clinical disease classification) and medical intervention (i.e., clinical procedure codes) were fully integrated into practice settings in the 1980s (Outland, Newman, & Williams, 2015), corollary taxonomies for nursing were not fully developed. Recent evidence also suggests that bedside nurses are still reluctant to adopt a standardized nursing taxonomy (Halloran & Halloran, 2015; Schwiran & Thede, 2011).

Quality Assessment and Documentation Burden

Though highly relevant to quality assessment and self-regulation in nursing, development and routine monitoring of nurse process measures has fallen into the “too hard to do” domain. The development of robust nurse process measures is hindered by issues related to incomplete and unstandardized documentation, documentation burden, and data collection burden. Reliable and valid nurse process measures require accurate and complete documentation of nursing care and patient responses to that care. Once documented, information about nursing processes and patient responses must be retrieved and organized for analysis. Costs associated with these processes have historically served as a deterrent to measure development and endorsement. Though highly relevant to quality assessment and self-regulation in nursing, development and routine monitoring of nurse process measures has fallen into the “too hard to do” domain. However, technological advances and healthcare reform legislation provide new opportunities to address some of these challenges.

The American Recovery and Reinvestment Act of 2009 provided incentives for healthcare organizations to invest in electronic health records (McCulloch & Tegethoff, 2013). This legislation authorized incentive payments to providers related to the implementation of electronic health records (EHRs) and meaningful use of data from them. Meaningful use includes the reuse of clinical and administrative data to improve the safety, quality, and access to care for patients (Westra et al., 2010). In 2015, incentive payments related to compliance with meaningful use standards were replaced with penalties for noncompliance. Consequently, adoption of EHRs has increased over time (Abbass, Helton, Mhatre, & Sansgiry, 2012). Many believed that this technology would result in significant cost savings through efficiencies gained in documentation and retrieval of clinical information (Jones & Kessler, 2010).

Introduction of EHRs has indeed changed nursing documentation processes and retrieval of information for analyses. However, the adoption of EHRs has not yet achieved the expected benefits. For example, in one hospital with an electronic nursing documentation system, compliance with documentation of selected interventions to prevent hospital acquired complications was as low as 16% (Cutugno, Hozak, Fitzsimmons, & Ertogan, 2015). Electronic prompts and forcing functions can be added to reduce lapses in documentation.

Electronic documentation systems have not reduced the documentation burden for nurses. While these tools may result in more complete documentation, they also may increase documentation burden (Cutugno et al., 2015). The current documentation burden in nursing is high and is a major source of job dissatisfaction among direct care nurses (Hakes & Whittington, 2008). Responses to the documentation burden include extending work hours to complete documentation, diverting time away from patient care to complete documentation, or leaving documentation incomplete. Electronic documentation systems have not reduced the documentation burden for nurses. Evidence suggests that time spent on documentation increases immediately after implementation but eventually returns to baseline (Hakes & Whittington, 2008; Munyisia, Yu, & Hailey, 2011).

Payment incentives for adoption of EHRs stipulate that recipients demonstrate how they are using data from the EHR to improve quality. Therefore, documentation screens are often built around endorsed quality measures, particularly those associated with VBP. Compared to abstraction of data from paper based documentation systems, retrieval of data from an EHR for quality analysis and reporting is more efficient (Alexander, 2007). This gain in efficiency is further enhanced when data fields are standardized across facilities and populated with a standardized taxonomy; therefore, adoption of EHRs may increase adoption of a NMDS. However, standardization has yet to be achieved and few systems are built to accommodate available nursing taxonomies (Alexander, 2007; Needleman et al., 2007).

EHR technology can accommodate an almost infinite number of data fields and this may increase the opportunity to develop more nurse process measures. However, time must be expended to populate every data field created. Therefore, every additional data field potentially adds to the documentation burden for nurses. The efficiency gained in information retrieval is potentially offset by a loss of efficiency related to data entry. Evidence suggests that direct care nurses are skeptical about whether or not expanded VBP policies will ultimately improve quality and safety for patients, but they are confident that an expansion of quality targets will increase their documentation burden (Kurtzman et al., 2011). Moreover, there is some concern that this burden will result in a shift of attention away from patient care, essentially putting the assessment of quality in competition with the performance of quality.

Limited Scope and Participation in Nursing Databases

Despite the proliferation of national and regional nursing databases, participation is voluntary and no available databases contain information on all of the 15 nurse-sensitive measures endorsed by the NQF (Kurtzman, Dawson, & Johnson, 2008; Kurtzman & Jennnings, 2008; VanFosson, Jones, & Yoder, 2015). Efforts to develop nurse-sensitive indicators for other practice settings, such as oncology (Fessele, Yendro, & Mallory, 2014), ambulatory care (Martinez, Battaglis, Start, Mastal, & Matlock, 2015), schools (Forward, 2012), public health (Bigbee & Issel, 2012), and women’s health (Association of Women’s Health, Obstetric and Neonatal Nurses [AWHONN], 2014) are underway; however, comparable performance databases for these practice settings do not currently exist. Consequently, the contribution of nursing care to patient outcomes is still not evaluated in many practice settings.

Limited Range of Patient Outcome Measures

The limited range of endorsed patient outcomes is problematic. Despite the evolution of comprehensive nurse specific conceptual frameworks that promote outcomes related to the role of nurses in health promotion (e.g. health promoting behaviors, self-care) and alleviation of suffering (e.g. symptom management), these types of outcomes remain underdeveloped and ignored.

Similar to outcomes included in the VBP initiative (Table 1) and the first nurse report card (Table 2), the nurse-sensitive patient outcomes endorsed by the NQF (Table 4) exclusively reflect the nurse role in the prevention of a limited number of adverse events. The consequences of this are both intentional and unintentional. Patient outcomes reflected in the endorsed measures are important and the act of measurement has without question generated selective attention and resource allocation toward reducing complications. However, this may be accompanied by the diversion of attention and resources away from other, perhaps equally important, patient needs. For example, evidence from multiple studies of unfinished nursing care indicate that direct care nurses prioritize care based more on what gets measured/audited than on individualized patient needs (Jones, Hamilton, & Murry, 2015). This is the educational equivalent of “teaching to the test” and is a form of metric-driven harm. The type of metric driven harm known as tunnel vision is defined as the prioritization of financially incentivized and/or measured care over other valuable aspects of care (Bodrock & Mion, 2008; Rambur, Vallett, Cohen, & Tarule, 2013). While adverse events are important, the question remains as to whether or not they should receive a disproportionate share of nursing attention and organizational resources compared to other quality problems.

Recommendations for Action

More emphasis is needed on establishing clinically meaningful relationships between specific nurse process measures and patient outcomes that reflect the full spectrum included in the nursing profession’s social mandate. Challenges to self-regulation and quality assessment in nursing are complex and systemic, and the responsibility for these functions is the joint responsibility of nurse scientists, nurse providers, nurse administrators, and nurse academicians. Sustained innovative and collaborative actions from multiple stakeholders are needed. Development of evidence-based, nurse-sensitive quality measures that meet the NQF consensus standards must become a research priority. The focus of research must expand beyond measures of nurse structures, such as staffing and nurse education. More emphasis is needed on establishing clinically meaningful relationships between specific nurse process measures and patient outcomes that reflect the full spectrum included in the nursing profession’s social mandate.

The development of nurse-sensitive process and outcome measures must be guided by the NQF consensus standards to make the endorsement process more expedient. This shift will require changes in academic programs and research funding policies. More academic programs are needed that equip graduates to develop and evaluate nurse-sensitive indicators. This type of research falls under the umbrella of health services and/or nursing administration research which is notoriously underfunded compared to biobehavioral research (Edwardson, 2006; Holubowich & Andos, 2008; Murphy, Scott, & Warshawsky, 2014). Consequently, many colleges and universities have not developed active research programs in this area. A strategic approach to resource allocation and priority setting among major funding agencies, such as the National Institute for Nursing Research (NINR) and the Agency for Healthcare Research and Quality (AHRQ), is needed to incentivize investments in outcomes research by academic organizations.

Nursing interventions are absent in most administrative databases and represented patient outcomes are limited to adverse events. Researchers must be willing to pursue methods beyond secondary analysis of these databases. Existing nursing outcomes databases (see Table 3) are similarly limited. More robust clinical databases are needed. Primary data collection and manipulation of non-standardized data are time and resource intensive. Moreover, healthcare organizations are often hesitant to share clinical and operational data with non-employees, even for research purposes. Therefore, access to better data also will require significant investments by healthcare organizations in clinical documentation systems and more collaborative partnerships with researchers.

It is imperative that nurse administrators proactively drive the design of performance monitoring systems that address the needs of the discipline of nursing. Specific actions are needed by nurse leaders and administrators in the areas of measure selection, adoption of technology, and resource allocation. Performance measures must be selected with thoughtful intention. Although the nursing quality agenda should accommodate measures related to efficiency, accreditation requirements, and VBP, it should be driven by the social contract. It is imperative that nurse administrators proactively drive the design of performance monitoring systems that address the needs of the discipline of nursing. Specifically, vendors, nurse informaticians, and quality specialists should be directed to design systems that provide reliable and valid information about nursing interventions and patient outcomes related to the following areas: self-care, health promoting behaviors, functional status, symptom management, knowledge of disease and treatment, and health-related quality of life. Once developed, researchers must be given access to these systems for the kind of robust research needed to develop nurse-sensitive measures that satisfy NQF consensus standards.

Quality assessment and self-regulation cannot move forward without substantial investments to support adequate documentation of nursing practice and patient outcomes. Quality assessment and self-regulation cannot move forward without substantial investments to support adequate documentation of nursing practice and patient outcomes. Direct care nurses are key sources of information in these areas; however, challenges related to documentation burden persist. It is time to garner the will and the resources to support direct care nurses as they struggle to document all aspects of their care, to include patient responses. Keyboard entry of data into the EHR continues to be as burdensome as handwritten documentation. At the very least, nurse administrators must request and support ongoing testing of user interface screens to maximize efficiency (Hyun, Johnson, Stetson, & Bakken, 2009).

Refinement of the user interface alone will not likely be sufficient. Nurse administrators must lead the development and application of new technologies to increase the efficiency of nurse documentation. Voice recognition technology for direct documentation has been used in other disciplines (Dela Cruz et al., 2014; McCartney, 2013) but has not been adequately evaluated in nursing. Phase I trials with nurses demonstrated promising results (Fratzke et al., 2014) but additional refinements are needed to enhance feasibility. Nurse administrators should initiate partnerships with vendors to guide these refinements and provide opportunities for beta testing.

Nurse administrators also should consider allocation of additional human resources to address the documentation burden. Staffing levels that result in documentation lapses are unsatisfactory and it is unlikely that technology alone will solve the problem. Although an increase in nurse staffing is associated with an increase in cost, that cost must be weighed against the costs associated with incomplete documentation, nurse overtime, and nurse dissatisfaction. One alternative strategy used by physicians to address documentation burden is to hire scribes to shadow clinicians and document clinical activities and observations (Hertz, 2014). Nurse administrators should explore the feasibility of off-loading keyboard data entry to other, less costly members of the care team.

Conclusion

Instead of continuing to search for relevance in available data, efforts must shift to making the collection of relevant data more feasible. Nurses have a social responsibility to evaluate the effect of nursing practice on patient outcomes in the areas of health promotion, injury and illness prevention, and alleviation of suffering. Quality assessment and outcomes research initiatives have historically been hindered by lack of available data related to nursing processes and patient outcomes across these domains of practice. Failure to invest in data collection methodologies that yield reliable and valid measures of important nursing care processes and patient outcomes has resulted in an inadequate evidence base to support the nurse contribution to patient care. New strategies to address the challenges of effective quality assessment and outcome measurement in nursing must be implemented. Instead of continuing to search for relevance in available data, efforts must shift to making the collection of relevant data more feasible.

Authors

Terry L. Jones, PhD, RN
Email: Tjones0222@aol.com

Terry L. Jones, RN, PhD is an Assistant Professor of Nursing in the Graduate Program in Nursing Administration and Healthcare Systems Management. Dr. Jones teaches undergraduate and graduate courses focusing on various aspects of leadership, management, evidence-based practice, and quality and safety. She has been a nurse for over 30 years and has served in a variety of roles ranging from critical care staff nurse to Chief Nurse Executive in a large academic medical center. Dr. Jones completed her PhD in 2004 at The University of Texas at Austin and was selected to participate in the NIH-Sponsored Clinical Scholars Program at The University of Texas Southwestern Medical Center at Dallas from 2007-2010. She has completed doctoral and post-doctoral academic programs with focused study in the development and application of nurse-sensitive quality indicators. Examining the nursing contribution to quality patient outcomes continues to be the primary focus of her program of research. She has been recognized as one of the Great 100 Nurses in Dallas-Ft. Worth and an Outstanding Graduate in her doctoral and undergraduate programs.

---