Quality indicators in health care

Stephan Meinecke

Lecture: Quality in health care

Assignment 1

Choose up to 5 indicators or measures of quality and critically discuss their usefulness in managing or monitoring quality care.

Introduction and definitions

Quality management is a very important

topic because of rising demands for the

quality of health services and cost pressure. To find the right quality indicators and therefore to establish an appropriate quality approach is a big challenge for all health care managers today and in the future. Quality indicators can be well suited tools for targeting quality improvement in the health system, e.g. in the setting of a continuous monitoring.

This assignment will critically discuss two quality indicators of health care (rate of hospital infection, patient satisfaction) and will show if these measures are useful in managing or monitoring quality care.

At first the terms quality, indicator and monitoring will be clarified. Then the two quality indicators will be analysed in order to demonstrate their strengths, weaknesses, reliability and validity. A description, the intention, the goals, the results, the problems and possible manipulations for each quality measure will be executed. The purpose is to demonstrate the difficulty of interpreting quality indicators and that each user should ask critically if this indicator can be used in their individual situation. Sometimes we use indicators incorrectly because we are not aware of their complexity and the influence of relevant factors.

Quality can be defined in many different ways, so a universe standardised definition does not exist. In this assignment the term quality will be defined as “ (…) a service which gives people what they need, as well as what they want, and does so at the lowest cost.” (Ovretveit, 1992). More specific definitions are available in Ovretveit’s book which is listed in the reference list.

“An indicator is a measure which is used to indicate the occurrence of an event, where a direct measure cannot be used.” (Ovretveit, 1992).

“Bernstein defines a quality indicator as a measurable element in the process or outcome of care whose value suggests one or more dimensions of quality-of-care and is theoretically amenable to change by the provider.” (retrieved from http://www.leitlinien.de/infoindikatglossar.h tm). “Monitoring is an observing activity in relation to defined specifications, standards or targets, directly or through reports or indicators.” (Ovretveit, 1992).

“Validity depends on evidence that the indicator is measuring what it was intended to measure.” (Kazandjian, n.d.) “The reliability of an indicator pertains to its level of measurement error.” (Kazandjian, n.d.)

Hospital acquired infections:

A hospital acquired infection is usually one that first appears three days after a patient is admitted to a hospital or other health-care facility. Infections acquired in a hospital are also called nosocomial infections. Hospital acquired infections are usually related to a procedure or treatment used to diagnose or treat the patient’s illness or injury (retrieved from http://www.ehendrick.org/healthy/0005 1560.html).

These infections have, without doubt,

an influence on the morbidity, mortality, and the costs of the health system.

Hospital acquired infections are a result of a hospital stay but two factors which influence an occurrence of an infection must be considered. The first factor is that there are conditions which are difficult to influence because they are unaffected by hospital care, such as age. The second factor is that infections can be influenced by medical interventions and their quality (retrieved from http://www.hospvd.ch/swiss- noso/d81a1.htm).

The intention of this indicator of

quality is to show the rate of infections which happen in a hospital. This can be achieved by recording the number of infections that result from complications. This information can be used to prevent these infections in the future.

The goal of this measure of quality is to monitor the hospital acquired infections and to learn from these mistakes in order to improve the patient’s health quality. It is proved that the death rate is about 7.1 times higher for infected than for uninfected patients, so the patient’s risk to die would decrease if the hospital use the obtained information to introduce an action plan (retrieved from http://www.jr2.ox.ac.uk/bandolier/band 73/b73-3.html).

The results of this measure of quality are indicators for each ward in a hospital. They cannot be compared with each other because in each ward exists a different risk to get an infection. For instance, the likelihood to get an infection in a surgery ward is higher than anywhere else.

Problems and weaknesses of this quality indicator:

At first it is unclear if the infection is a result of medical treatment or just the patient’s health status. Besides these unclear factors it would be useless to compare all wards with each other without considering the potential risk to get an infection. The case mix is important in order to get valid results because you cannot compare young patients with old persons in average.

Also the frequency of use (e.g. catheters) and the duration of use must be considered because both have an influence on the frequency of infections. To draw conclusions on the quality of hospital care is not possible if the prerequisites for good data quality are not fulfilled. The interpretation of the infection rates must be done separately because of the case mix. The infections could appear with a delay so that it is not always clear if the patient’s infection was caused by the hospital stay.

Strengths:

Nosocomial infections are used for a long time in hospitals, so these indicators are used with high experience. The intent of this quality indicator is easily understood and is interpretable by all users. It deals with relevant and significant aspects of care and the data is easy available. It is an useful information to inform quality programs and has a good cost/utility ratio. Today efficient and reliable instruments exist which are.

Furthermore these instruments have a good cost/utility ratio.

Validity:

The validity of this indicator can be described as high because it measures what it was intended to measure- the infection rates acquired in a hospital. (have a closer look at the strengths) So the hospital performance can be represented as one part of indicators by the hospital acquired infections.

Reliability:

Hospital acquired infections have standardised definitions and a rigour of data collection mechanisms, so the reliability is fulfilled.

Manipulations:

Each ward will blame other wards for the infections in order to get a positive infection rate and to avoid problems. So the question which ward is responsible for the infection could cause conflicts and disharmony. The result could be manipulations by the wards and the uncertainty for the management where to set action plans to avoid the same mistakes.

Patient satisfaction

One definition of patient satisfaction is how patients value and regard their care (Bluementhal, 1996). Another definition of patient satisfaction is conceptually defined as patients’ value judgments and subsequent reactions to the stimuli they perceive in the health environment just before, during, and after the course of their inpatient stay or clinical visit (Strasser, 1991).

Patient surveys have become a way of life in managed care, an indispensable management tool. The intention of a patient survey is to monitor the patient’s satisfaction, behavioral intentions, and practice problems and trends. The goal is to achieve a high patient satisfaction and therefore to ensure a good

competitor’s position in comparison with other health care organizations.

The result is the patient’s satisfaction with different staff members, wards, medical aspects, coordination skills, information by the staff, consideration of individual needs etc. . At the end all single results are summed up and weighted with subjective factors in order to get a final grade for the hospital care by the patient.

Problems and weaknesses of this quality indicator:

The satisfaction levels of patients’

surveys are high if you compare it to other service sectors and is not confirmed by patients and staff members in personal discussions (retrieved from http://www.picker.org). High satisfaction rates uncover singular problems and can hide negative individual experience which could be used for an organization improvement. Individual standards and expectations have an impact on the satisfaction but cannot be measured. So a lot of factors influence the patient’s satisfaction, such as education level, age, gender, ethnic group, fear of personal disadvantages as a result of critical comments etc.. As a consequence the correlation between subjective valuation and objective facts is not very high, so that the final satisfaction rate cannot be seen as a real measurement of the medical quality and health care. The survey designs are often unreliable and can be easily influenced by statistical manipulations, e.g. in which order to ask the questions. But the results of written surveys are still better than oral surveys because of less influence on the respondents. In general it is not clear if the patient is really satisfied although he/she states it. Often these kind of surveys do not generate relevant information for a quality improvement as a result of undifferentiated questions. In the study of Kane et al. (1997) is described that the patient’s satisfaction is strongly influenced of the current health status in contrast to the occurred improvements. The literature confirms that the patient’s satisfaction cannot be seen as a good indicator for medical treatment (Donebedian, 1992; Brook, 1973; Groupy, 1991; Leimkühler, 1996; Nettleman, 1998).

If the physicians create the design of the survey and are aware that the factor satisfaction is included in their annual compensation formula, the consequence is obvious. These medical groups will try to influence the survey’s score in a positive way for them and therefore to falsify the survey’s results (retrieved from http://www.managedcaremag./archives /9904/9904.patsatis.html). Strengths of this indicator:

The patients’ satisfaction can be a very useful instrument to improve processes and to avoid mistakes in the future. If the survey is done in a methodologically excellent way and a valid instrument is used, the hospital can determine important factors for the competition with other health care organisations. It is a good marketing instrument and gives the hospital a hint of what is important for their clients.

Validity The validity of this indicator must be discussed critically because the subjectivity of patients’ surveys is very high. So “the job” of this quality measurement is disturbed and the final data could be false. There is no evidence that the indicator is measuring what it is intended to measure because there are many factors which can falsify the data. The validity depends on the quality of the survey, so it is difficult to make a judgement about it in general. So it is not proved that this indicator represents the hospital performance. “Researchers have begun to wonder if the aggregate measures of patient satisfaction have the desired validity as an indicator.” (Kazandijan, n.d.)

Reliability

The aspects for the validity also apply for the reliability, so the reliability is not high. Especially, the subjective design of surveys can be criticised so that measurement errors can happen. “Therefore, decisions about the reliability of a measure concern the mechanics of the construction of that measure or indicator.” (Kazandijan, n.d.)

Manipulations

The survey design can be transformed to one’s advantage as mentioned in the problems/weaknesses section.

Summary/Conclusion

According to the introduction, the difficulty of interpreting quality indicators in health care can be seen above. Some indicators do their “job” better than others because their results are more valid and reliable. The indicator “hospital acquired infections” is a good indicator and has already been used for a long time, so improvements for getting a better indicator have been realised. It has a good validity and reliability and can be utilised usefully in monitoring or managing health care.

The indicator “patient satisfaction” must be regarded as a highly subjective indicator which has an inferior validity and reliability. The result of this indicator depends on many variable factors which do not have to be medical aspects. This does not mean that this indicator cannot be used in the correct way for obtaining useful information but it should be analyzed more critically than other indicators. The user of these indicators must be aware of influencing factors and individual intentions which can be different to the organisation’s objectives.

The assignment’s aim was to sensitise the users, patients and stakeholders and to think more critically about performance indicators. The user of these indicators should try to investigate and to name all influencing factors in order to increase transparency. Therefore the management of a health care organisation should be careful to establish a quality approach and choose the quality indicators with special diligence. To choose indicators only because other institutions use them should be avoided because of incomparable conditions. Benchmarking with quality indicators between different hospitals or health care organizations make only sense if the requirements and the quality of the recorded data are comparable.

(2070 words)

Reference list: (alphabetical)

- Bluementhal, D. (1996). “Quality of care: what is it?”, in New Journal of Medicine, No. 12, p. 891-894

- Brook R. (1973). “Quality of care assessment , choosing a method for peer review.”, p. 1323-1329

- Donabedian, A. (1992). “The role of outcome in quality assessment and assurance.”, p. 356-360

- Groupy, F. (1991). “Results of a comparative study of in-patient satisfaction in eight hospitals in the Paris region.”, p. 309-315

- Kazandijan, Vahé A. (19??) “Indicators of Performance or the Search for the Best Pointer Dog”, in The Epidemiology of Quality, p. 25-37

- Kane, R. et al., (1997). “The relationship of patient satisfaction with care and clinical outcome.”, in Med Care, p. 714-730

- Leimkühler, A. (1996). “Patient satisfaction - artefact or social reality?”, p. 765-773

- McCormick, D. (1999). “Variation in length of hospital stay in patients with community-acquired pneumonia: Are shorter stay associated with worse medical outcomes?”, in American Journal of Medicine 107, p. 5-12

- Nettleman, M. (1998). “Patient satisfaction - what’s new?”, p. 33-37

- Ovretveit, John (1992). “ Health Service Quality. An Introduction to Quality Methods for health services.” Blackwell Scientific Publications, Oxford

- Strasser, S., and Associates. (1992). “Measuring Patient Satisfaction for Improved Patient Service”, Ann Arbor, in Health Administration Press

[...]