Building Linkages between Nursing Care and Improved Patient Outcomes: The Role of Health Information Technology

  • Patricia C. Dykes, PhD, RN, FAAN, FACMI
    Patricia C. Dykes, PhD, RN, FAAN, FACMI

    Patricia Dykes is Senior Nurse Scientist in the Center for Patient Safety, Research and Practice and in the Center for Nursing Excellence at Brigham and Women’s Hospital and Assistant Professor of Medicine at Harvard Medical School in Boston, MA. Patti has Bachelor's and Master's degrees in nursing from Fairfield University in Fairfield, CT and New York University in New York, NY respectively, and a Doctoral degree in nursing and biomedical informatics from Columbia University in New York, NY. She has experience in patient safety and informatics research. While funded by the Robert Wood Johnson Foundation, Patti and her team developed a fall prevention toolkit that significantly reduced falls in hospitals, and they published the results of this study in the Journal of the American Medical Association. They have expanded this research to explore the use of technology to provide the core set of information needed by nurses, patients, and all care team members at the bedside to engage in safe patient care. Patti is the author of two books and over 70 peer reviewed publications. She has presented her work both nationally and internationally. She is a member of the National Institutes of Health Biomedical Computing and Health Informatics Study Section, Center for Scientific Review, and a fellow of the American Academy of Nursing and the American College of Informatics.

  • Sarah A. Collins, PhD, RN
    Sarah A. Collins, PhD, RN

    Dr. Sarah Collins is a Nurse Informatician in the Knowledge Management group at Partners Healthcare Systems and an Instructor in Medicine at Harvard Medical School and Brigham and Women’s Hospital Division of Internal Medicine and Primary Care in Boston, MA. She was a National Library of Medicine Post-Doctoral Research Fellow at Columbia University’s Department of Biomedical Informatics in New York, NY. Dr. Collins holds a PhD in Nursing Informatics from Columbia University School of Nursing in New York, NY and a Bachelor of Science from the School of Nursing at the University of Pennsylvania in Philadelphia, PA with a minor in Health Care Management from the Wharton School of Business. Dr. Collins is an experienced critical care nurse. Her research is focused on modeling, developing, and evaluating standards-based, patient-centered collaborative informatics tools to further knowledge development, clinical decision-support, and coordinated patient care. In 2012, Dr. Collins was selected as one of two national Emerging Leaders by the Alliance for Nursing Informatics and her research has been recognized and awarded by the American Medical Informatics Association and the 11th International Congress on Nursing Informatics.

Abstract

Health information technology (health IT or HIT) holds the potential to transform the quality of care and to establish linkages between nursing care and patient outcomes. This article defines eMeasurement and describes Quality of Care Definitions and Metrics for Evaluation. The authors explore the role of health IT to improve quality, barriers to eMeasurement, and health IT interventions by considering linkages between nursing care and patient outcomes for a select set of nursing sensitive indicators including patient falls, pressure ulcers, and the patient experience. We discuss specific challenges, such as barriers for routine data capture to populate nursing sensitive indicators and the use of health IT to promote positive outcomes. The conclusion addresses the implications of the current state of health IT and identifies areas for further nursing research.

Key words: nursing, nursing sensitive indicators, health IT, nursing informatics, quality, patient safety

There is longstanding evidence that linkages exist between nursing care and improved patient outcomes. There is longstanding evidence that linkages exist between nursing care and improved patient outcomes. Florence Nightingale used quality improvement data and statistics to support a hand washing campaign that prevented soldiers from dying of hospital acquired infections during the Crimean war (Gill & Gill, 2005). Over the past decade, the Robert Wood Johnson Foundation’s Interdisciplinary Nursing Quality Research Initiative demonstrated the impact of nursing care on patient sensitive outcomes, such as fall prevention (Dykes et al., 2010; Dykes, Carroll, Hurley, Benoit, & Middleton, 2009; Shever, Titler, Mackin, & Kueny, 2011); pain management (Beck, Towsley, Berry, Brant, & Smith, 2010); and delirium prevention (Balas et al., 2012). Despite the scientific advancements that have occurred since the 1850s, the quality of care in United States remains suboptimal. A series of Institute of Medicine (IOM) reports published over the past decade suggest that error rates are high and the quality of care across the United States (U.S. healthcare system is variable (Adams & Corrigan, 2003; Kohn, Corrigan, & Donaldson, 1999; Lohr, 1990). Even where evidence exists, it is inconsistently applied in practice (McGlynn et al., 2003). The cost of healthcare is also of concern. Healthcare expenditures currently comprise 18% of the gross domestic product (GDP), and if the current trend continues, healthcare will consume 34% of the GDP by 2040 (Council of Economic Advisors, 2009).

Secondary use of data for quality measurement represents a paradigm shift that requires transformative changes in practice... Unsustainable costs and unreliable quality are stimulating changes in healthcare policy and practice. The use of health IT has long been recommended as a strategy to facilitate cost effective, high-quality, and safe patient care (Committee on Data Standards for Patient Safety, 2003). The American Recovery and Reinvestment Act (ARRA) provides incentives for providers and hospitals to adopt certified electronic health records and to use them in a meaningful way, including facilitating care coordination and electronic submission of data for quality reporting purposes (Centers for Medicare and Medicaid Services, 2010). Rapidly evolving healthcare policy that is driving adoption and use of health IT and quality measurement standards hold potential for transformation of the healthcare system but enhancements to both are needed. Clinical information systems have traditionally been designed to support clinical tasks and to capture data needed to support billing of physician services and regulatory requirements (Cusack et al., 2013). Secondary use of data for quality measurement represents a paradigm shift that requires transformative changes in practice and in the architecture and configuration of electronic systems (Tolar & Balka, 2012).

This article explores the use of “nursing sensitive” indicators and health IT to improve quality of care and to establish linkages between nursing care and improved patient outcomes. Nursing-sensitive indicators are defined as measures that reflect the structure, process and outcomes of nursing care (ANA, 2013). Nursing-sensitive outcome indicators are defined measurement strategies for patient or caregiver states that are sensitive to nursing care (Given & Sherwood, 2005). In this article, we will demonstrate the role of health IT in building linkages between nursing care and improved patient outcomes focusing on two mechanisms: 1) eMeasurement, and 2) health IT interventions directly targeting nursing sensitive indicators.  The following set of indicators are used to demonstrate the current capabilities of health IT for building such linkages: patient falls, pressure ulcers and patient experience (Centers for Medicare and Medicaid Services, 2013). We will identify existing barriers and then make recommendations for improvements needed to routinely build linkages between nursing care and patient outcomes.

eMeasurement

For this discussion, eMeasurement is defined as the secondary use of electronic data to populate standardized performance measures (National Quality Forum, 2013a). The prerequisites for eMeasurement include standardized performance measures in an electronic format; clinical information systems that capture structured, coded data; and administrative and clinical workflows that facilitate consistent documentation or capture of the data needed to populate the electronic measures (AHA, 2012). In addition, a standard approach for quality measurement beyond the organizational level is needed to support the use of eMeasures for benchmarking. Multiple challenges related to these prerequisites exist that make it difficult to build linkages between nursing care and patient outcomes. This discussion will describe some of these challenges.

Quality of Care Definitions and Metrics for Evaluation

Donabedian’s structure-process-outcome model provides the foundation for the approach used in the U.S. to evaluate quality in healthcare organizations. The IOM defines quality of care as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” (Lohr, 1990, p. 375). Donabedian’s structure-process-outcome model provides the foundation for the approach used in the U.S. to evaluate quality in healthcare organizations (Donabedian, 1988). Donabedian’s framework is useful because many factors may influence patient outcomes. Identifying relationships between structural aspects of patient care (e.g., number of nursing care hours, presence or absence of sophisticated information systems), the processes of care (e.g., risk assessments completed and interventions implemented to mitigate risk), and patient outcomes (e.g., patient fall or a pressure ulcer) can help one to make informed inferences about the quality of care.

The Role of Health IT in eMeasurement of the Quality of Nursing Care

Nurses have long considered the role of technology in classifying nursing sensitive outcomes and for processing the large data sets needed for demonstrating outcomes (Charmel & Frampton, 2008; Urquhart, Currell, Grant, & Hardiker, 2009; Zielstorff, 1995; Zielstorff, Lang, Saba, McCormick, & Milholland, 1995). The American Nurses Association (ANA) has supported efforts to define nursing sensitive measures with the Nursing Care Report Card for Acute Care (Press Ganey Associates, Inc., 2011) and with the National Database of Nursing Quality Indicators (NDNQI) (ANA, 2010). Currently, the National Quality Forum (NQF) is leading an effort to develop e-measures to ensure that data used for clinical documentation can be reused to measure patient outcomes as a byproduct of care (National Quality Forum, 2013b). The NQF Quality Positioning System (QPS) is a web-based tool designed to improve access to all NQF endorsed measures. Some nursing sensitive measures, such as patient fall rate and falls with injury, are included in the QPS. However, many measures included in the QPS are not considered “eMeasures” (including the nursing sensitive measures) because they require manual methods or systems for processing.

Barriers to eMeasurement Using Nursing Sensitive Indicators

Measurement of quality using nursing sensitive indicators is complex as the data needed to populate these measures comes from multiple sources... Measurement of quality using nursing sensitive indicators is complex as the data needed to populate these measures comes from multiple sources, many of which are not electronic. Health IT systems that are integrated into the workflow and that generate structured coded data as a byproduct of both administrative processes and patient care are needed to support nursing workflow and to populate eMeasures. Widespread use of eMeasures will ensure that indicators used across disparate health IT systems are accurate and comparable. The Commonwealth Fund has developed a typology (Table 1) for categorizing the five broad types of eMeasures that currently exist, ranging from those translated from traditional measurement sets (e.g., Translational e-indicators) to measures of patient harm triggered by health IT (Fowles et al., 2008).

Table 1. A Typology for Categorizing Electronic Measures (“e-indicators”) of Quality and Safety (Fowles et al., 2008)

Indicator Typology

Definition

Translational e-indicators

Measures that have been translated from existing—“traditional”—measurement sets for use in HIT platforms. Example: Healthcare Effectiveness Data and Information Set (HEDIS) or NQF standard measures.

HIT-facilitated e-indicators

Measures that, while not conceptually limited to HIT-derived data sources, would not be operationally feasible in settings without HIT platforms. Example: Population health measures such as body mass index (BM) for 100% of patients.

HIT-enabled e-indicators

Innovative measures that would not generally be possible outside of the HIT context. Example: Meaningful use measures such as the percent of unique patients in a given practice who are provided online access to their health information (i.e., laboratory test results) within 4 business days after the information is available to the provider.

HIT-system-management e-indicators

Measures needed to implement, manage, evaluate, and generally improve HIT systems, and they are primarily intended for use by the parent organization to improve local systems. Example: Percentage of providers who override medication safety alerts in the electronic healthcare record (EHR).

“E-iatrogenesis” e-indicators

Measures of patient harm caused at least in part by the application of health information technology. Example: Percentage of patients who received an incorrect intervention due to an error in the clinical decision support provided by the EHR.

In theory, data recorded in an electronic record can then be reused for multiple purposes, but in practice, secondary use remains an elusive goal. As noted above, there are several prerequisites to achieving secondary use of data for benchmarking and for establishing linkages between nursing care and improved patient outcomes (AHA, 2012). Challenges related to standardization, availability, and workflow and secondary use of data, in general and specific to nursing sensitive measures, are discussed below and also displayed in Table 2.

The availability of standardized eMeasures is a prerequisite to automated quality reporting and benchmarking. Standardized eMeasures. The availability of standardized eMeasures is a prerequisite to automated quality reporting and benchmarking. The NQF is retooling existing measures to support electronic measurement, but the lack of availability of structured, coded data in the electronic record to auto-populate eMeasures is a barrier to widespread use. As illustrated in Table 2, data required to populate nursing sensitive measures exist in disparate sources (e.g., medical records, surveys, incident reporting systems, administrative databases, and human resource systems) and may not be available as structured, coded data. Exclusion criteria may require manual processes or sophisticated decision logic for processing. The fact that assessment content varies across organizations serves as an additional barrier to e-Measurement. For example many different fall risk assessment scales exist (e.g., Morse Fall Scale, Schmid, STRATIFY), and different scales are used by nurses in different organizations (Morse, Morse, & Tylko, 1989; Oliver, Britton, Seed, Martin, & Hopper, 1997; Schmid, 1990). Fall prevention process measures require a multifaceted assessment of risk for fall (see Table 2) but the measure does not specify which fall risk assessment scale should be used or which assessment data are required to populate the measure. At this time, none of the nursing sensitive quality measures that are currently endorsed by the NQF are designated as eMeasures (National Quality Forum, 2013b).

Availability of structured, coded data. Structured, coded data are needed to populate eMeasures. The Meaningful Use (MU) legislation (HHS.GOV, 2011) provides incentive payments to hospitals and providers that use certified electronic systems and encode clinical data with standardized terminologies. MU legislation is expected to drive adoption of certified EHRs and terminology standards to make structured, coded data available to populate existing eMeasures. For example, MU specifies use of Rx Norm to encode medications, SNOMED Clinical Terms (SNOMED CT) for patient problems, and Logical Observation Identifiers Names and Codes (LOINC®) to represent lab test results (Centers for Medicare and Medicaid Services, 2010). However, even if adopted and fully implemented, the recommended data sets are medically focused and may be inadequate to represent nursing sensitive care and associated outcomes (ANA, 2013).

Workflows Not Optimized for Data Capture and Reuse. Workflow barriers exist to capturing clinical data at the point of care in an interoperable format. Clinical data needed to represent nursing care processes and associated outcomes are not consistently documented in the electronic record (Carroll, Dykes, & Hurley, 2012; Moss, Andison, & Sobko, 2007). When nurses do document interventions intended to improve patient outcomes, it may not be in a format that supports secondary use (e.g., free-text data or uncoded structured data).

To establish linkages between nursing care and patient outcomes, data are needed that represent the structure, processes, and outcomes of nursing care. To establish linkages between nursing care and patient outcomes, data are needed that represent the structure, processes, and outcomes of nursing care. In Table 2, selected nursing sensitive process and outcome indicators related to patient falls, pressure ulcers, and patient experience are displayed, including the measure type and data source. These measures offer examples of some of the challenges associated with using health IT to establish linkages between nursing care and patient outcomes. Specifically, data needed to populate measures may not be routinely documented. If documented, the format may not be conducive to secondary use. Moreover, data needed to populate the measures exist in disparate systems, and the exclusion criteria require complex inference and processing, which is difficult to automate (Kennedy, 2013). These challenges are further outlined in Table 2 and described in the sections below using patient falls, pressure ulcers, and the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) measures as examples.

Table 2: Operational Definitions for Selected Nursing Sensitive Process and Outcome Measures

 
[view Table 2 as pdf]

eMeasurement of Patient Falls and Pressure Ulcers

...translation is complex and health IT systems with sophisticated decision logic are required... Patient falls and pressure ulcers were among the first Nursing-Sensitive Performance Measures to be endorsed by NQF (2009). As noted in Table 2, some data elements required to populate the selected set of nursing sensitive process and outcome measures for patient falls and pressure ulcers come from the medical record. For example, the Morse Fall Scale (Morse, Morse, & Tylko, 1989) and the Braden Scale (Bergstrom, Braden, Laguzza, & Holman, 1987) are evidence-based instruments that are commonly (but not universally) used to document fall and pressure ulcer risk status. Concepts representing the items from both scales are included in LOINC® and concepts representing the value sets are included in SNOMED CT. The denominators come from administrative databases. Assuming that electronic systems are in place, MU standards are implemented, and that clinical and administrative systems are interoperable, the fall and pressure ulcer risk assessment measures can be translated into e-measures. However, this translation is complex and health IT systems with sophisticated decision logic are required to ensure that excluded populations are removed from the dataset. For example, the fall risk assessment measure (Table 2) excludes patients who are under age 65 at the time of the measurement. Many systems available today are not capable of this level of inference (Kennedy, 2013).

Moreover, the data elements needed to populate the patient fall rate and pressure ulcer prevalence measures come from both clinical information and incident reporting systems that may not be electronic or interoperable. Patient falls, fall related injury, and pressure ulcer staging are routinely recorded in incident reporting systems and less consistently in the medical record. Furthermore, as noted in Table 2, the pressure ulcer risk assessment measure has a long list of exclusion criteria (National Quality Forum, 2013b). Data needed to automate the decision logic to operationalize the exclusion criteria may not be available in many electronic systems in use today.

eMeasurement of Patient Experience

Patient experience is a significant quality measure used for hospital benchmarking and has been directly linked with patient-nurse communication. Patient experience is a significant quality measure used for hospital benchmarking and has been directly linked with patient-nurse communication (Senti & LeMire, 2011). A positive patient experience is of increasing importance to hospitals across the United States because it is a quantifiable and targeted measure for quality improvement and, starting in 2013, is a required metric for the value-based purchasing program of the Centers for Medicare and Medicaid Services (CMS) (Bush, 2011b; The Joint Commission, 2007). Patient experience is a nursing sensitive outcome measure, but is broader in that it is sensitive to the care delivered by the entire interprofessional team.

HCAHPS is the standard measurement survey of patient experience and can be used to isolate nursing sensitive patient experience metrics to target specific interventions. CMS requires HCAHPS reporting to incentivize value-based purchasing with three broad goals: 1) objective and meaningful comparisons of topics salient to patients; 2) public reporting to incentivize hospital quality improvement; and 3) public reporting to enhance accountability and transparency of hospital care that is reimbursed by public funds (Centers for Medicare and Medicaid Services, 2013). HCAHPS is a rigorous step forward toward impacting measured and objective change in the patient experience by focusing on metrics critical to quality and hospitalized patients’ experience (Giordano, 2010).

The HCAHPS survey was developed and psychometrically validated by the Agency for Healthcare Research and Quality (AHRQ) and endorsed by the NQF. It is a standard, 27-item survey of patient experience that measures level of satisfaction and perspectives of inpatient hospital care along six axes: communication with nurses and doctors; pain control; timeliness of care; discharge instructions; hospital cleanliness; and treatment with courtesy and respect (Levoy, 2009). The vision guiding use of the HCAHPS survey is based on a framework in which all hospital employees share decision-making roles and responsibilities to enhance the patient experience (Bush, 2011a). The survey, which can be found at www.hcahpsonline.org/surveyinstrument.aspx, includes questions such as: “During this hospital stay, how often did nurses listen carefully to you?” (p. 1)  The survey is administered between 48 hours and 6 weeks after hospital discharge to a random sample of adult patients through mail, telephone, mail with telephone follow-up, or through an interactive voice recognition (IVR) phone system (Bombard, 2009; Centers for Medicare and Medicaid Services, 2013). As illustrated in table 2, exclusion criteria are extensive and existing electronic systems are not capable of this level of inference (Kennedy, 2013).

Valid measurement instruments, such as the HCAHPS survey, hold value when they are used for reporting or benchmarking purposes, direct improvement strategies, and measure change overtime. As a national standard, HCAHPS survey results allow for comparison of scores across hospitals and over time (Bombard, 2009; Centers for Medicare and Medicaid Services, 2013; Levoy, 2009). California has used a statewide web-based HCAHPS reporting system since 2007, and results for hospitals nationwide are posted to the CMS website (Bombard, 2009; Centers for Medicare and Medicaid Services, 2013).

All measurements have limitations and the HCAHPS measure is not without its challenges. HCAHPS data collection process is separate from data in a patient’s medical record. In its current paper and phone call based form, HCAHPS is a translational e-indicator, assuming its data is stored in a reusable and electronic form. Yet, the HCAHPS literature calls for organizations to provide staff with regular access to HCAHPS data. The literature also calls for training and trust building to empower staff to feel comfortable using HCAHPS data as one tool for practice improvement (Brady, 2009). This notion is consistent with the concept of HIT-facilitated eMeasures. If HCAHPS data collection is enabled online and leverages structured data from IVR it could be made available to practicing clinicians for practice improvements and benchmarking. Providing clinicians with ongoing access to HCAHPS data for their patient population would not be feasible in settings without HIT-enabled data collection or dissemination platforms.

It is important to note that HCAHPS provides population- or unit-based data, not patient-level data. Therefore, while benchmarking for a population is feasible, analyzing scores for all patients seen by a specific provider or clinic is not. Additional HCAHPS measurement challenges include a lack of risk-severity and facility-specific adjustments, and limitation to English and Spanish speaking populations (Bush, 2011b). For example, there is no adjustment for patients with higher comorbidities and rates of depression, but these patients are associated with lower HCAHPS scores. Other variables cited as challenges for hospitals include facilities that have double rooms for patients, which likely impacts lower patient experience scores.

Health IT Interventions

In the sections below, the value of health IT in linking nursing care with improved patient outcomes is explored. By leveraging the eMeasures discussed in the previous section, well-designed health IT interventions can target care delivery processes and create linkages between nursing care and patient outcomes.

Barriers to the Use of Health IT to Link Nursing Care with Patient Outcomes

While literature suggests that health IT can be used to improve patient outcomes related to medication safety (Day et al., 2011; DeYoung, Vanderkooi, & Barletta, 2009; Poon et al., 2010) and the quality of care delivered (Atlas et al., 2011; Kwok, Dinh, Dinh, & Chu, 2009; Ruland et al., 2012), limited research has explored the relationship between the use of health IT and nursing sensitive indicators. A Cochrane Review published in 2009 evaluated the effects of nursing record systems on nursing practice and patient outcomes (Urquhart et al., 2009). The authors found only nine studies that were conducted with sufficient rigor to meet the inclusion criteria. Of the nine studies, only four evaluated computerized nursing record systems and the remaining five studies evaluated paper-based systems. The authors concluded that there was some evidence that health IT systems could help improve a specific problem such as reducing time spent on data collection, but there was no evidence that nursing care planning and documentation systems improved nursing practice or patient outcomes (Urquhart et al., 2009). This review highlights several challenges related to the use of health IT to link nursing care with patient outcomes, including 1) limited rigorous research; 2) lack of clarity regarding what data should be recorded and how these data will be used; and 3) lack of integration of electronic systems into the clinical workflow. Each of these challenges is described below.

the variability that exists between different implementations of health IT systems makes it difficult to conduct rigorous comparisons. Limited rigorous research. As mentioned, very few rigorous studies have been conducted to evaluate the impact of health IT on nursing sensitive outcomes. The Cochrane review called for more quasi-experimental and randomized control trials to test the effectiveness of nursing record systems (Urquhart et al., 2009). One challenge to design the types of comparative effectiveness studies needed to rigorously test health IT systems is that careful planning is needed to collect baseline data or to identify an appropriate control group. In addition, the variability that exists between different implementations of health IT systems makes it difficult to conduct rigorous comparisons. For example, even systems designed for the same purpose (e.g., care planning or clinical documentation) or by the same vendor may be configured or implemented differently at different sites. More standardized systems are needed to support rigorous evaluation. Since publication of the Cochrane review, two studies were published that demonstrated the value of health IT systems on nursing sensitive indicators; one relates to an integrated clinical documentation system (Dowding, Turley, & Garrido, 2011; Urquhart et al., 2009), and the second relates to a fall prevention toolkit (Dykes et al., 2010). Both are described in the “Health IT Systems” section below.

While structured, coded data is essential, this format is not ideal for telling the patient story and providing context. Lack of clarity regarding what data should be recorded and how data will be used. The Cochrane review identified a lack of clarity regarding the types of data that should be recorded and the format needed to support building linkages between nursing care and improved patient outcomes as an issue for nursing (Urquhart et al., 2009). The nursing sensitive indicators described earlier help to define some of the data elements needed as structured, coded data (see Table 2). While structured, coded data is essential for decision support, benchmarking, quality reporting, and research, this format is not ideal for telling the patient story and providing context. Clarity is needed related to what aspects of nursing care should be recorded as free text or in other formats to support care team communication and collaboration.

The most successful nursing record systems were those that were integrated within larger health IT systems and the clinical workflow of practicing nurses. Lack of integration of electronic systems into the clinical workflow. The most successful nursing record systems included in the Cochrane review were those that were integrated within larger health IT systems and the clinical workflow of practicing nurses. For example, the computer-based system for nursing documentation that was part of an integrated electronic medical record reduced the amount of time that nurses spend on documentation and was associated with an increase in time spent with patients (Bosman et al., 2003). Nursing record systems that existed as siloed applications for documentation or care planning demonstrated limited or no value (Ammenwerth, Hackl, Riedmann, & Jung, 2011; Daly, Buckwalter, & Maas, 2002).

While barriers exist to using health IT to establish linkages between nursing care and improved patient outcomes, some examples have emerged since publication of the Cochrane review (Urquhart et al., 2009). The use of health IT systems to establish linkages between the prevention of patient falls and pressure ulcers and to improve patient experience are presented in the following sections.

Health IT Systems to Prevent Patient Falls and Pressure Ulcers

Health IT interventions have been used to target antecedents to patient falls and pressure ulcers and recent research indicates that IT systems may improve fall and pressure ulcer prevention processes and lead to better patient outcomes. Many issues place patients at risk for falls or for the development of pressure ulcers, including improper screening practices, poor communication, and inconsistent implementation of personalized interventions. When implementing the complex practice changes needed for fall and pressure ulcer prevention, health IT tools can be one component of a multifaceted, performance improvement intervention. Health IT interventions have been used to target antecedents to patient falls and pressure ulcers and recent research indicates that IT systems may improve fall and pressure ulcer prevention processes and lead to better patient outcomes (Dowding et al., 2011; Dykes et al., 2010). For example, Dykes and colleagues tested a fall prevention toolkit (FPTK) that used health IT to integrate fall risk assessment and tailored fall prevention plans into existing workflows. The FPTK used a set of validated icons (see Figure 1) to communicate fall risks and personalized interventions at the bedside to all care team members including patients and family (Hurley, Dykes, Carroll, Dykes, & Middleton, 2009). In a randomized control trial including over 10,000 patients, the FPTK was found to significantly decrease patient falls in acute care hospitals (Dykes et al., 2010). The FPTK worked because it linked the processes of care with patient outcomes in the context of busy acute care workflows. When the nurse used the FPTK to complete the fall risk assessment, the set of interventions most likely to prevent a fall for each individual patient were automatically selected and presented within the user interface. The nurse could then further tailor the interventions based on professional judgment. The icons were displayed at the bedside, providing real-time decision support. This improved communication of the fall prevention plan and made it easier for nurses and other providers to consistently carry out recommended interventions.

Figure 1: FPTK Icons

It is not possible to build nursing knowledge from practice without data related to what nurses do to prevent adverse outcomes. The same team evaluated the effectiveness of the FPTK for promoting documentation of fall risk status and planned and completed fall prevention interventions (Carroll et al., 2012). They found that the FPTK significantly improved documentation of planned interventions, but did not improve documentation of completed interventions. This finding has implications for establishing linkages between nursing care and patient outcomes. Health IT tools can help to fill gaps where data are needed to populate nursing sensitive indicators only if nurses use these tools to document the interventions that they carry out. It is not possible to build nursing knowledge from practice without data related to what nurses do to prevent adverse outcomes (e.g., data to populate processes indicators). Health IT tools are needed to facilitate data capture in the context of the workflow. The FPTK did this for the assessment and planning phases of the fall prevention process, but not for documentation of completed interventions.

Dowding et al. (2011) evaluated the impact of an integrated electronic health record in 29 hospitals on process and outcome indicators for patient falls and pressure ulcers. They found improved documentation of risk assessments for both falls and pressure ulcers, but the improvement was statistically significant for documentation of pressure ulcer risk assessment only. Fall rates remained unchanged after implementation of the EHR. Hospital acquired pressure ulcer rates decreased significantly and remained significant when controlling for EHR implementation, suggesting that this outcome was likelyinfluenced by multiple factors (Dowding et al.,2011).

Health IT interventions are most effective when both clinical and management processes are addressed and where organizational leadership demonstrates strong support for improvement strategies (Dowding et al., 2011; Dykes et al., 2010). In the case examples above, leadership support, stakeholder engagement, and involvement of peer champions contributed to an environment conducive to fall and pressure ulcer prevention (Dykes et al., 2010; Garrett et al., 2009). Furthermore, using clinical experts to address knowledge gaps (Carson, Emmons, Falone, & Preston, 2012; Dykes et al., 2009; Garrett et al., 2009) and involving peer champions in the implementation process (Carson et al., 2012; Dykes et al., 2009) supported adoption and consistent use of the health IT intervention. Both health IT interventions described above produced structured, coded data that facilitated the measurement of nursing sensitive process and outcome measures that could be used to build linkages between nursing care and patient outcomes.

Health IT Systems to Promote a Positive Patient Experience

Empowered, informed, and discerning patient-consumers are driving the clinical and business cases for hospitals to improve the patient experience and measure those improvements overtime (Levoy, 2009). Nursing care is recognized as a critical factor along the HCAHPS six axes and many hospital nursing and patient care services have implemented programs to enhance communication, pain control, courtesy and respect, and other factors to ensure positive patient experience (Bombard, 2009; Bush, 2011a). Hospitals have taken a variety of non-technical and technical approaches to improve patient experience scores based on principles or questions measured by HCAHPS. Some hospitals provide managers and clinicians with access to their HCAHPS score as a means to promote continuous quality improvement and individual and group practice benchmarking (Bush, 2011a). Others employ hiring strategies to recruit employees based on the principles measured in HCAHPS (Bush, 2011a). These macro-level strategies targeted at the principles measured in HCAHPS, as opposed to micro-level strategies targeted at the specific questions in HCAHPS, are cited as “game-changers” for organizations that embrace them (Clark, 2011).

Technology-based, patient-tailored interventions are being investigated as solutions to improve the patient-experience system-wide. Technology-based, patient-tailored interventions are being investigated as solutions to improve the patient-experience system-wide (Caligtan, Carroll, Hurley, Gersh-Zaremski, & Dykes, 2012; Dykes et al., 2013; Vawdrey et al., 2011). Technology-based interventions can be used to target the principles measured by the HCAHPS survey by offering patients a method to dynamically interact with their electronic healthcare data; access pertinent information online; and be recognized as a member of the patient-centered team that communicates within the EHR. Even at the early stages of development, these interventions are addressing EHR barriers to improved patient experience, such as the functional challenges to promote awareness of who is part of the care team and variable policies for patient access to electronic clinical data (Collins, Wilcox, Fred, & Vawdrey, 2012; Vawdrey et al., 2011). Preliminary data indicate that a bedside electronic computer communication application that provides interactive patient-specific information, such as inpatient medication lists, names and photographs of care providers, room telephone number, laboratory results, daily schedule, and a location to write notes to the care team, has potential to enhance patient-provider communication and improve the patient experience (Dykes et al., 2013; Vawdrey et al., 2011). These process-based interventions are potential low cost solutions that meet patients’ and clinicians’ basic information and communication needs to prepare them for more effective, informed conversations and interactions.

As a standard measure, HCAHPS is useful for retrospective and prospective evaluations of the impact of technology-based interventions on the outcome of patient experience. The mix of general and specific measures within HCAHPS is valuable to evaluate HIT interventions specific to nursing care and inclusive of all patient care delivered to improve patient experience outcomes. HCAHPS sampling limits the rigor and depth of analysis that can be performed, but well-designed quasi-experimental studies with unit-based HCAHPS measures can be used to observe process-outcome associations. HCAHPS provides a standardized mechanism to understand variables that impact the patient experience and to perform population and unit level evaluation and comparison of HIT interventions that target those variables. Future work should focus on adopting HCAHPS as a patient-level eMeasure. This focus would enable the standardization of health IT interventions targeted at specific variables that impact the patient experience, and the comparison of health IT interventions and an individual patient’s experience.

Conclusion

Nurse informaticists have long recognized both the barriers and the potential of health IT for capturing and using clinical data (Zielstorff et al., 1995). This article underscores the complexity of building linkages between nursing care and improved patient outcomes and the need for eMeasures and health IT to achieve this goal. The recent AARA and MU legislation creates an opportunity for nurses to advocate for indicators and for health IT systems that facilitate scalable approaches to capturing clinical data at the point of care and for reusing those data to populate nursing sensitive indicators.

Adoption of standards that support representation of nursing care and patient outcomes that are sensitive to that care are needed... While this legislation establishes standards to promote adoption of terminologies that will likely expand data reuse and improve interoperability of systems, it is not a panacea. Prerequisites for health IT systems that will build linkages between nursing care and patient outcomes include an informatics infrastructure where eMeasures are widely available and adopted. In addition, nursing content standards are needed for use in electronic systems (e.g., fall and pressure ulcer risk assessment scales) and standard terminologies to encode that content. The MU standards are basic requirements and are not focused on representing nursing care (ANA, 2013). Adoption of standards that support representation of nursing care and patient outcomes that are sensitive to that care are needed to achieve a scalable approach to build linkages between these variables.

Achieving an informatics infrastructure to support a health system that routinely builds linkages between nursing care and patient outcomes requires a concentrated effort. A focus is needed on transforming nursing indicators into eMeasures and building usable health IT systems that support nursing practice and produce structured, coded data to populate eMeasures. While there has been some progress to identify and operationalize indicators of nursing sensitive care (ANA, 2013; The Joint Commission., 2009), to date none of these indicators are designated as eMeasures. Further, very few rigorous studies have been conducted to measure the impact of health IT on nursing practice and patient outcomes. Additional research is needed to establish the types of systems associated with improved process and outcome indicators and determine how to integrate them into the workflow to capture and use clinical data as a byproduct of nursing care.

Authors

Patricia C. Dykes, PhD, RN, FAAN, FACMI
Email: pdykes@partners.org

Patricia Dykes is Senior Nurse Scientist in the Center for Patient Safety, Research and Practice and in the Center for Nursing Excellence at Brigham and Women’s Hospital and Assistant Professor of Medicine at Harvard Medical School in Boston, MA. Patti has Bachelor's and Master's degrees in nursing from Fairfield University in Fairfield, CT and New York University in New York, NY respectively, and a Doctoral degree in nursing and biomedical informatics from Columbia University in New York, NY. She has experience in patient safety and informatics research. While funded by the Robert Wood Johnson Foundation, Patti and her team developed a fall prevention toolkit that significantly reduced falls in hospitals, and they published the results of this study in the Journal of the American Medical Association. They have expanded this research to explore the use of technology to provide the core set of information needed by nurses, patients, and all care team members at the bedside to engage in safe patient care. Patti is the author of two books and over 70 peer reviewed publications. She has presented her work both nationally and internationally. She is a member of the National Institutes of Health Biomedical Computing and Health Informatics Study Section, Center for Scientific Review, and a fellow of the American Academy of Nursing and the American College of Informatics.

Sarah A. Collins, PhD, RN
Email: sacollins@partners.org

Dr. Sarah Collins is a Nurse Informatician in the Knowledge Management group at Partners Healthcare Systems and an Instructor in Medicine at Harvard Medical School and Brigham and Women’s Hospital Division of Internal Medicine and Primary Care in Boston, MA. She was a National Library of Medicine Post-Doctoral Research Fellow at Columbia University’s Department of Biomedical Informatics in New York, NY. Dr. Collins holds a PhD in Nursing Informatics from Columbia University School of Nursing in New York, NY and a Bachelor of Science from the School of Nursing at the University of Pennsylvania in Philadelphia, PA with a minor in Health Care Management from the Wharton School of Business. Dr. Collins is an experienced critical care nurse. Her research is focused on modeling, developing, and evaluating standards-based, patient-centered collaborative informatics tools to further knowledge development, clinical decision-support, and coordinated patient care. In 2012, Dr. Collins was selected as one of two national Emerging Leaders by the Alliance for Nursing Informatics and her research has been recognized and awarded by the American Medical Informatics Association and the 11th International Congress on Nursing Informatics.

Council of Economic Advisors (2009). The economic case for health care reform. Washington, D.C.: Executive Office of


© 2013 OJIN: The Online Journal of Issues in Nursing
Article published September 30, 2013

References

Adams, K., & Corrigan, J. (2003). Priority areas for national action: Transforming health care quality (pp. 143). Washington, DC: Institute of Medicine.

American Nurses Association . (2010). National database of nursing quality indicators (NDNQI). Guidelines for data collection on the American Nurses Association's National Quality Forum Endorsed Measures: Nursing care hours per patient day, skill mix, falls, falls with injury. Kansas, MO: American Nurses Association.

American Nurses Association. (2013). Nursing-sensitive indicators.  Retrieved from: www.nursingworld.org/MainMenuCategories/ThePracticeofProfessionalNursing/PatientSafetyQuality/Research-Measurement/The-National-Database/Nursing-Sensitive-Indicators_1

Ammenwerth, E., Hackl, W. O., Riedmann, D., & Jung, M. (2011). Contextualization of automatic alerts during electronic prescription: researchers' and users' opinions on useful context factors. Studies in Health Technology and Information, 169, 920-924.

Atlas, S. J., Grant, R. W., Lester, W. T., Ashburner, J. M., Chang, Y., Barry, M. J., & Chueh, H. C. (2011). A cluster-randomized trial of a primary care informatics-based system for breast cancer screening. Journal of General Internal Medicine, 26(2), 154-161. doi: 10.1007/s11606-010-1500-0

Balas, M. C., Rice, M., Chaperon, C., Smith, H., Disbot, M., & Fuchs, B. (2012). Management of delirium in critically ill older adults. Critical Care Nurse, 32(4), 15-26. doi: 10.4037/ccn2012480

Beck, S. L., Towsley, G. L., Berry, P. H., Brant, J. M., & Smith, E. M. (2010). Measuring the quality of care related to pain management: A multiple-method approach to instrument development. Nursing Research, 59(2), 85-92. doi: 10.1097/NNR.0b013e3181d1a732

Bergstrom, N., Braden, B. J., Laguzza, A., & Holman, V. (1987). The Braden Scale for Predicting Pressure Sore Risk. Nursing Research, 36(4), 205-210.

Bombard, C. (2009). HCAHPS is all about patient satisfaction. Nurse Week, 16(9), 22-27.

Bosman, R. J., Rood, E., Oudemans-van Straaten, H. M., Van der Spoel, J. I., Wester, J. P., & Zandstra, D. F. (2003). Intensive care information system reduces documentation time of the nurses after cardiothoracic surgery. Intensive Care Medicine, 29(1), 83-90. doi: 10.1007/s00134-002-1542-9

Brady, C. (2009). Dancing with HCAHPS data. Briefings on Patient Safety, 10(11), 8-9, 12.

Bush, H. (2011a). For some hospitals, HCAHPS poses added challenges. Hospitals & Health Networks, 85(12), 25.

Bush, H. (2011b ). Doubling down on the patient experience. Hospitals & Health Networks, 85(12), 22-25.

Caligtan, C. A., Carroll, D. L., Hurley, A. C., Gersh-Zaremski, R., & Dykes, P. C. (2012). Bedside information technology to support patient-centered care. International Journal of Medical Informatics, 81(7), 442-451. doi: 10.1016/j.ijmedinf.2011.12.005

Carroll, D. L., Dykes, P. C., & Hurley, A. C. (2012). An electronic fall prevention toolkit: effect on documentation quality. [Multicenter Study Research Support, Non-U.S. Gov't]. Nurs Res, 61(4), 309-313. doi: 10.1097/NNR.0b013e31825569de

Carson, D., Emmons, K., Falone, W., & Preston, A. M. (2012). Development of pressure ulcer program across a university health system. Journal of Nursing Care Quality, 27(1), 20-27. doi: 10.1097/NCQ.0b013e3182310f8b

Centers for Medicare and Medicaid Services. (2010). Medicare and Medicaid Programs; Electronic Health Record Incentive Program; Final Rule. Federal Register, 75(144).

Centers for Medicare and Medicaid Services. (2013). HCAHPS: Patients’ Perspectives of Care Survey Retrieved May 1b, 2013, from www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalHCAHPS.html

Charmel, P. A., & Frampton, S. B. (2008). Building the business case for patient-centered care. Healthcare Financial Management, 62(3), 80-85.

Collins, S., Wilcox, L., Fred, M., & Vawdrey, D. (2012, Jun 23- 27). Workarounds used by nurses to overcome design constraints of electronic health records. Paper presented at the 11th International Congress on Nursing Informatics, Montreal, Canada.

Committee on Data Standards for Patient Safety (2003). Key Capabilities of an Electronic Health Record System. Washington, D.C.: The National Academies Press.

Council of Economic Advisors (2009). The economic case for health care reform. Washington, D.C.: Executive Office of

Cusack, C. M., Hripcsak, G., Bloomrosen, M., Rosenbloom, S. T., Weaver, C. A., Wright, A.,…  Mamykina, L. (2013). The future state of clinical data capture and documentation: a report from AMIA's 2011 Policy Meeting. Journal of the American Medical Informatics Association, 20(1), 134-140. doi: 10.1136/amiajnl-2012-001093

Daly, J. M., Buckwalter, K., & Maas, M. (2002). Written and computerized care plans. Organizational processes and effect on patient outcomes. Journal of gerontological nursing, 28(9), 14-23.

Day, R. O., Roffe, D. J., Richardson, K. L., Baysari, M. T., Brennan, N. J., Beveridge, S., ... Westbrook, J. I. (2011). Implementing electronic medication management at an Australian teaching hospital. Medical Journal of Australia, 195(9), 498-502.

DeYoung, J. L., Vanderkooi, M. E., & Barletta, J. F. (2009). Effect of bar-code-assisted medication administration on medication error rates in an adult medical intensive care unit. American Journal of Health-System Pharmacy, 66(12), 1110-1115.

Donabedian, A. (1988). The quality of care. How can it be assessed? Journal of the American Medical Association, 260(12), 1743-1748.

Dowding, D. W., Turley, M., & Garrido, T. (2011). The impact of an electronic health record on nurse sensitive patient outcomes: an interrupted time series analysis. Journal of the American Medical Informatics Association, 19(4) 615-620. doi: 10.1136/amiajnl-2011-000504

Dykes, P., Carroll, D., Hurley, A., Gersh-Zaremski, R., Kennedy, A., Kurowski, J.,... Middleton, B. (2009). Fall TIPS: strategies to promote adoption and use of a fall prevention toolkit. AMIA Annual Symposium Proceedings, 2009, 153-157.

Dykes, P. C., Carroll, D. L., Hurley, A., Lipsitz, S., Benoit, A., Chang, F.,... Middleton, B. (2010). Fall prevention in acute care hospitals: a randomized trial. Journal of the American Medical Association, 304(17), 1912-1918.

Dykes, P. C., Carroll, D. L., Hurley, A. C., Benoit, A., Chang, F., Pozzar, R., & Caligtan, C. A. (2013). Building and testing a patient-centric electronic bedside communication center. Journal of Gerontological Nursing, 39(1), 15-19. doi: 10.3928/00989134-20121204-03

Dykes, P. C., Carroll, D. L., Hurley, A. C., Benoit, A., & Middleton, B. (2009). Why do patients in acute care hospitals fall? Can falls be prevented? Journal of Nursing Administration, 39(6), 299-304. doi: 10.1097/NNA.0b013e3181a7788a

Fowles, J., Kind, E., Awwad, S., Weiner, J., Chan, K., Coon, P., ... Selna, M. (2008). Performance measures using electronic health records: Five case studies. Washington, DC: The Commonwealth Fund.

Garrett, J., Wheeler, H., Goetz, K., Majewski, M., Langlois, P., & Payson, C. (2009). Implementing an "always practice" to redefine skin care management. Journal of Nursing Administration, 39(9), 382-387. doi: 10.1097/NNA.0b013e3181b414b

Given, B., & Sherwood, P. R. (2005). Nursing-Sensitive Patient Outcomes—A White Paper. Oncology Nursing Forum, 32(4), 773-784.

Healthit.gov. (n.d.). Meaningful use. Retreived from http://www.healthit.gov/policy-researchers-implementers/meaningful-use

Hurley, A. C., Dykes, P. C., Carroll, D. L., Dykes, J. S., & Middleton, B. (2009). Fall TIPS: validation of icons to communicate fall risk status and tailored interventions to prevent patient falls. Studies in Health Technology and Informatics, 146, 455-459.

Joint Commission, The. (2007). Hospitals begin publicly reporting H-CAHPS data, other measures. Joint Commission Benchmark 9(6), 8-9.

Joint Commission, The. (2009). National Quality Forum (NQF) endorsed nursing-sensitive care performance measures. Retrieved from www.jointcommission.org/national_quality_forum_nqf_endorsed_nursing-sensitive_care_performance_measures/

Kennedy, R. (2013). Role of clinical documentation for clinical quality measurement. Washington, DC: US Department of Health and Human Services.

Kohn, L., Corrigan, J., & Donaldson, M. (1999). To err is human: Building a safer health system. . Washington,D.C.: National Academy Press.

Kwok, R., Dinh, M., Dinh, D., & Chu, M. (2009). Improving adherence to asthma clinical guidelines and discharge documentation from emergency departments: Implementation of a dynamic and integrated electronic decision support system. Emergency Medicine Australasia, 21(1), 31-37. doi: 10.1111/j.1742-6723.2008.01149.x

Levoy, B. (2009). The HCAHPS satisfaction survey: A new era of transparency is here. Are you ready for it? Podiatry Management, 28(5), 245.

Lohr, KN, ed. (1990). Medicare: A strategy for quality assurance. Washington, DC: National Academy Press.

Lohr, S. (2011, July 17). Seeing promise and peril in digital records, The New York Times  p. 3.

McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., & Kerr, E. A. (2003). The quality of health care delivered to adults in the United States. New England Journal of Medicine, 348(26), 2635-2645. doi: 10.1056/NEJMsa022615

Morse, J. M., Morse, R. M., & Tylko, S. J. (1989). Development of a scale to identify the fall-prone patient. Canadian Journal on Aging, 8(4), 366-377.

Moss, J., Andison, M., & Sobko, H. (2007). An analysis of narrative nursing documentation in an otherwise structured intensive care clinical information system. AMIA Annual Symposium Proceedings, 543-547.

National Quality Forum. (2013a). Electronic Quality Measures  Retrieved from www.qualityforum.org/Projects/e-g/eMeasures/Electronic_Quality_Measures.aspx

National Quality Forum. (2013b). Quality Positioning System  Retrieved from www.qualityforum.org/QPS/

Oliver, D., Britton, M., Seed, P., Martin, F. C., & Hopper, A. H. (1997). Development and evaluation of evidence based risk assessment tool (STRATIFY) to predict which elderly inpatients will fall: case-control and cohort studies. BMJ, 315(7115), 1049-1053.

Poon, E. G., Keohane, C. A., Yoon, C. S., Ditmore, M., Bane, A., Levtzion-Korach, O., ... Gandhi, T. K. (2010). Effect of bar-code technology on the safety of medication administration. New England Journal of Medicine, 362(18), 1698-1707.

Press Ganey Associates, Inc. (2011). Hospital Pulse Report. Retrieved from www.pressganey.com/researchresources/hospitals/pulseReports.aspx

Ruland, C. M., Andersen, T., Jeneson, A., Moore, S., Grimsbo, G. H., Borosund, E., & Ellison, M. C. (2012). Effects of an internet support system to assist cancer patients in reducing symptom distress: A Randomized Controlled Trial. Cancer Nursing. doi: 10.1097/NCC.0b013e31824d90d4

Schmid, N. A. (1990). 1989 federal nursing service award winner. Reducing patient falls: A research-based comprehensive fall prevention program. Military Medicine, 155(5), 202-207.

Senti, J., & LeMire, S. D. (2011). Patient satisfaction with birthing center nursing care and factors associated with likelihood to recommend institution. Journal of Nursing Care Quality, 26(2), 178-185. doi: 10.1097/NCQ.0b013e3181fe93e6

Shever, L. L., Titler, M. G., Mackin, M. L., & Kueny, A. (2011). Fall prevention practices in adult medical-surgical nursing units described by nurse managers. Western Journal of Nursing Research, 33(3), 385-397. doi: 10.1177/0193945910379217

Tolar, M., & Balka, E. (2012). Caring for individual patients and beyond: Enhancing care through secondary use of data in a general practice setting. International Journal of Medical Information, 81(7), 461-474. doi: 10.1016/j.ijmedinf.2012.01.003

Urquhart, C., Currell, R., Grant, M. J., & Hardiker, N. R. (2009). Nursing record systems: effects on nursing practice and healthcare outcomes. Cochrane Database Systematic Reviews (1), CD002099. doi: 10.1002/14651858.CD002099.pub2

Vawdrey, D. K., Wilcox, L. G., Collins, S. A., Bakken, S., Feiner, S., Boyer, A., & Restaino, S. W. (2011). A tablet computer application for patients to participate in their hospital care. AMIA Annual Symposium Proceedings, 2011, 1428-1435.

Zielstorff, R. D. (1995). Capturing and using clinical outcome data: implications for information systems design. Journal of American Informatics Association, 2(3), 191-196.

Zielstorff, R. D., Lang, N. M., Saba, V. K., McCormick, K. A., & Milholland, D. K. (1995). Toward a uniform language for nursing in the US: Work of the American Nurses Association Steering Committee on databases to support clinical practice. Medinfo, 8 Pt 2, 1362-1366.

Citation: Dykes, P., Collins, S., (September 30, 2013) "Building Linkages between Nursing Care and Improved Patient Outcomes: The Role of Health Information Technology" OJIN: The Online Journal of Issues in Nursing Vol. 18, No. 3, Manuscript 4.