Specific Characteristics of Patient Safety Indicators

Definition's aims and interpretation

PSIs are defined by their purpose, which is monitoring preventable adverse events – directly or indirectly (16;17).

PSIs are thus quality indicators (18), which prove to be valid within the specific framework of interpretation and refer to preventable events or medical errors.

• In terms of structure and process indicators: are healthcare organisational features, practices or interventions, with evidence of effects on exposure to preventable risk factors (e.g. safety culture, hand washing practices, screening of schizophrenic patients for suicidal risk)
, and


• In terms of outcome: harm which is or may be conceived to be caused by preventable events: death, temporary or permanent disability (19;20)

.
The most important contribution of indicator monitoring to patient safety is the potential for quantitative surveillance. Currently assessment of patient safety problems and appropriate interventions relies mainly on case-based qualitative methodology as reporting systems and root-cause analysis. While these widely used tools have proved to be essential for establishing and supporting a clinical culture of safety, evidence of their effectiveness and efficacy is still uncertain (21;22).

Monitoring of relevant indicators such as mortality rates, true rates of adverse drug events or safety culture is a necessary prerequisite to rational priority decisions concerning patient safety interventions (23).

The interpretation of PSIs as absolute measures of patient safety for benchmarking or accountability purposes is demanding in terms of method development. Firstly because of the probabilistic nature of the relationship between exposure (structure-process) and harm (outcome), and secondly because of the interactions between system and patient factors influences outcome values as mortality rates.

The few categories of directly measurable preventable adverse events (wrong sided surgery, wrong blood type, in-hospital suicide) occurs so infrequently, that corresponding indicators has the characteristics of sentinel indicators (24) with very limited value in monitoring changes over time or for benchmarking purposes at the clinical (hospital) level. However, these indicators have been found to be useful at the system (national-regional) level (e.g. as part of the recommended PSIs from OECD) (25).

Therefore assessment of patient safety status by indicators requires access to multiple sources and kinds of data (26) and has the characteristics of screening for safety problems, rather than making a definitive diagnosis (27;28).

Example: Rate-based indicator

Death due to anaesthesia has become rare, by contrast morbid events, i.e. complications related to anaesthetic care such as anaesthetic overdose, reaction, or endotracheal tube misplacement, are much more prevalent causing harm to the patient to a different extent. Safety should be assessed at the aggregated patient level.

Example: Sentinel Indicator

Although surgeons and operating room teams rely on the practice of counts of sponges, sharp and instrument as a means to eliminate detained foreign bodies, practices are not standardised and every single event may signal a serious system failure that should be addressed. Thus safety should be assessed at the individual patient level.

Areas for Development and use of patient safety indicators

As with quality indicators in general the effectiveness of PSIs depends on selection of proper clinical areas of measurement in terms of frequency, severity as well as the existence of evidence-based interventions towards patient safety problems.

When these factors are taken into account (29;30), four main areas for hospital-related PSIs can be identified:

• Generic indicators which monitors organisation-wide patient safety characteristics
,

• Indicators monitoring hospital-acquired infections
,

• Indicators monitoring preventable surgical complications
, and

• Indicators monitoring medication safety

Supplemented by a number of condition-specific areas (e.g. bedsores, falls).

Definition of a 'Reporting System' according to WP4:

A system which is designed to contain reports on adverse events. On the basis of reports, analysis and communication of known causes and risk situations is possible. The system can contain reports on human and technical errors as well as organisational circumstances, which affect the occurrence of adverse events in the health care process. Reporting systems include input from all stakeholders - providers and service users.

Example of a developmental area - Reporting systems of adverse events as a basis for developing PSIs:

A reporting system which is used systematically for reports of adverse events, will contain a wide variety of information on reports of different events - different in type, frequency, severity, and actions resulting from analysis.
At first, each report will represent an incident perhaps even a sentinel event. Events which are rated rare but severe could be candidate sentinel indicator event.
Some events in the reporting system database, are alike: such events form thematical clusters and they provide essential information for the selection of adequate rate-based patient safety indicators.

Institution-wide patient safety indicators:

Consists of indicators which address general patient safety characteristics in all types of healthcare organisations and covers both primary care and hospitals:

• Measures of staff safety culture (structural indicators)
,

• Standardized mortality rates (outcome indicators)
, and

• Fraction of patients experiencing adverse events (outcome indicators)

.
Because of their generic nature these indicators are mainly useful in surveillance of organisational interventions or characteristics.

Standardized mortality rates are well-established and validated as indicators of patient safety problems (31;32), but require access to developed administrative healthcare databases and sophisticated analysis of data.

Safety culture assessment and patient experience of adverse events seem promising both in terms of relevance and feasibility, but evidence of validity of these indicators is still limited (33-36).


Indicators monitoring hospital-acquired infections:

Indicators monitoring hospital-acquired infections consist of:

• Rate-based indicators based on registration (direct or through patient administrative data collection systems) of various types of hospital acquired infections (37;38) e.g.
:


• Surgical wound infections
,

• Ventilator pneumonia
, and

• Catheter based UTI
,

• Probability function assessment of hospital acquired infections
, and

• Structural indicators covering infection control measures (e.g. rate of consumption of hand –disinfectants per 1000 bed days) (39)

.
The registration-based indicators are well characterized in terms of limitations of validity and feasibility (40;41).

Probability function assessment of hospital-acquired infections is validated and seems less prone to bias than direct registration, but requires advanced health care IT systems (42-44).


Indicators monitoring preventable surgical complications:

Indicators monitoring preventable surgical complications (45), consist of:

Risk adjusted mortality rates,

Postoperative infection rates

,

Postoperative rates of haemorrhage or thrombosis

,

Readmission rates

, and

Rate of processes which reduces risk of complications (peroperative antibiotics, antithrombotic prophylaxis)

.
These indicators are validated, well-established, and characterized.


Indicators monitoring medication safety:

Construction of relevant and valid indicators monitoring medication safety has posed specific problems because of the very high incidence of medication errors (46).

Presently published indicators consist of:

• Rate of adverse drug events assessed by trigger tool methodology (47-50)
.

This type of outcome indicators has been validated for surveillance of the effects of interventions to improve medication safety within an organisation. The indicators are less useful for benchmarking purposes because of standardisation problems.


• Patients understanding of purpose for taking medications
.

This questionnaire based indicator is a part of the Care Transitions Measure and is closely related to the process of medication reconciliation at discharge

.

Example: Generic indicator:

Communication and transfer of information between healthcare settings and between professionals and patients, are essential aspects of patient safety. Especially care transition processes are known to be vulnerable regarding patient safety.
Understanding the purpose of the medication can impact on compliance. Adherence to medications is important, since lack of compliance can have fatal consequences for the patient. Thus, the quality of staff-patient communication regarding patients' understanding of the purpose of their medication when leaving hospital, is an adequate generic measure of patient.

Example: Disease-specific indicator:

Schizophrenic patients have a known higher risk of suicidal behaviour, especially in the period right after discharge form hospital. Thus, assessment of suicidal risk at discharge is an adequate disease-specific measure of patient safety.

Specific issues of data quality, validity, and reliability

In terms of reliability the value of PSIs in organisational learning seems dependent on a high sensitivity of the indicator, while specificity may be of main concern if the indicator is used for public accountability (51;52). The evaluation of PSIs is therefore strongly dependent on the intended use.

The level of the PSIs which monitors harm (e.g. rates of mortality and infections) is usually sensitive to bias caused by the severity of the disease, comorbidities and lifestyle factors. Validity of these indicators is therefore depended on simultaneous collection of patient-related data appropriate for risk adjustment. Empirical evidence suggests that administrative data are not satisfactory in this respect (53;54) especially when used at clinical (hospital) level.

Experience with direct collection of medical record-based data (primary data) seems to partly alleviate this problem. There is however still ambiguity in the clinical definitions with resulting risk of bias in e.g. recording of wound infections which requires caution in interpretation of benchmarking studies. Moreover demands in terms of resources and organisation in direct collection of clinical data limit the scale of this approach (55).

Newer developments in the field of medical informatics in the form of automated detection systems based on primary data from laboratory- pharmacy-, and hospital administrative systems show promising results in unbiased assessment of infection rates (56) and medication safety (automated trigger tool methodology) (57). The methodology requires sophisticated resources in terms of informatics and reliable system wide patient identification.