Below a list of the PSIs, which have been established, that is characterised and evaluated by the WP4 Expert Group, is shown. A full description of the PSIs following the SAIFA is available here.
Evaluation of the PSIs
Eight experts from six nations evaluated the PSIs. The expert group initially conducted a structured review of each PSI and an independent assessment of each indicator evaluating the three dimensions: “Relevance”, “Validity and Reliability” and “Feasibility” of the indicators. The methodology for the structured review and the evaluation was given using the “Scheme for Evaluation of Indicators”. This evaluation was followed by a number of one hour telephone conferences, which served to discuss the PSIs, ensuring common understanding of the definitions and phrasing, and propose and decide on alterations and refinements. Only alteration for the new developed PSIs were made. Where it was found necessary descriptions of the PSIs were altered and re-circulated among the experts and the PSI was discussed again. The individual ratings on a scale ranging from 1 to 9 for each of the three dimensions of each PSI were also shared in telephone conferences.
As all PSIs had been rated, statistics for each dimension of the indicator was worked out and send to the expert group. The statistics formed the basis of the consensus decisions of recommendations for implementation of the PSIs according to the four categories previously described. Examples of statistics provided by the experts are shown below.
Aspects to be considered concerning application of the PSIs
Eight experts from six nations evaluated the PSIs. The expert group initially conducted a structured review of each PSI and an independent assessment of each indicator evaluating the three dimensions: “Relevance”, “Validity and Reliability” and “Feasibility” of the indicators. The methodology for the structured review and the evaluation was given using the “Scheme for Evaluation of Indicators”. This evaluation was followed by a number of one hour telephone conferences, which served to discuss the PSIs, ensuring common understanding of the definitions and phrasing, and propose and decide on alterations and refinements. Only alteration for the new developed PSIs were made. Where it was found necessary descriptions of the PSIs were altered and re-circulated among the experts and the PSI was discussed again. The individual ratings on a scale ranging from 1 to 9 for each of the three dimensions of each PSI were also shared in telephone conferences.
As all PSIs had been rated, statistics for each dimension of the indicator was worked out and send to the expert group. The statistics formed the basis of the consensus decisions of recommendations for implementation of the PSIs according to the four categories previously described. Examples of statistics provided by the experts are shown below.
Table 5. List of established Patient Safety Indicators by WP4
INDICATOR CATEGORY AND NAME TYPE
INSTITUTION-WIDE PSIs
1. Measuring Hospital Standardised Mortality Rates New
2. Death in Low-Mortality DRGs Review
3. Patients Experiencing Adverse Events New
4. Patients Informed about an Adverse Event by the Staff New
5. Patients Experiences of Adverse Events Management New
6. Transition of Care – Patient’s Understanding of the Purpose of their Medication New
7. Institution-Wide use of Cultural Assessment New
8. Surveying the Development of the Patient Safety Culture New
THEME RELATED PSIs: “INFECTION CONTROL”
9. Selected Infections due to Medical Care Review
10. Hospital Acquired-Infection Registration – Post Operative Wound Infections New
11. Wound Infection Review
12. Ventilator Pneumonia Review
13. Hand Hygiene - Measured by the Alcohol Consumption New
14. Hand Hygiene - Staff’s Compliance with Guidelines for use of Jewellery New
THEME RELATED PSIs: “SURGICAL COMPLICATIONS”
15. Complications of Anesthesia Review
16. Foreign Body left during Procedure Review
17. Postoperative Pulmonary Embolism or Deep Vein Thrombosis Review
18. Postoperative Sepsis Review
19. Postoperative Haemorrhage or Haematoma Review
20. Postoperative Physiologic Metabolic Derangements Review
21. Postoperative Respiratory Failure Review
22. Accidental Puncture or Laceration Review
23. Wrong Site-Surgery Review
24. Medical Equipment-related Adverse Events Review
25. Patients experiencing Harmful Surgical Adverse Events New
INDICATOR CATEGORY AND NAME TYPE
THEME RELATED PSIs: “MEDICATION ERRORS”
26. Transfusion Reaction Review
27. Wrong Blood Type Review
28. Medication Error (Does not fulfil the criteria as an indicator, therefore deleted) Review
29. Electronic Trigger Tool - Surveillance of Adverse Drug Events New
THEME RELATED PSIs: “OBSTETRICS”
30. Obstetric Trauma – Vaginal Delivery without Instrument Review
31. Obstetric Trauma – Vaginal delivery with instrument Review
32. Obstetric trauma – Caesarean Delivery Review
33. Problems with Childbirth Review
34. Birth Trauma – Injury to Neonate Review
THEME RELATED PSIs: “IN-HOSPITAL FALL”
35. Postoperative Hip Fracture Review
36. In-Hospital Hip Fracture or Fall Review
37. Patient Falls Review
DIAGNOSE SPECIFIC AS WELL AS OTHER SPECIFIC PSIs
38. Decubitus Ulcer Review
39. Failure to Rescue Review
40. Iatrogenic Pneumothorax Review
41. Assessment of Suicidal Risk in Schizophrenic Patients New
42. Side Effect of Anti-Psychotic Treatment New
Table 6. Example of statistics bases on the ratings of the three dimensions for one PSI
Relevance and Appropriatness Validity and Reliablity Feasibility
Number of ratings 8 8 8
Median 7,50 5,50 5,00
Mode 8 4# 5
Percentiles 25 6,00 4,00 3,25
75 8,00 6,75 5,00
# Multiple modes exist. The smallest value is shown
Table 7. Example of statistics bases on the ratings of relevance and appropriateness for one PSI
Score Frequency Percent
Score 4: Medium degree of relevance and appropriateness 1 12,5
Score 6: Medium degree of relevance and appropriateness 2 25,0
Score 7: High degree of relevance and appropriateness 1 12,5
Score 8: High degree of relevance and appropriateness 4 50,0
Total 8 100,0
Diagram 1. Example of distribution of the ratings for “Relevance and Appropriateness” for one PSI
Rating
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8
Individuel raters
INDICATOR CATEGORY AND NAME APPLICATION*
INSTITUTION-WIDE PSIs
1. Measuring Hospital Standardised Mortality Rates 2
2. Death in Low-Mortality DRGs 3
3. Patients Experiencing Adverse Events 3
4. Patients Informed about an Adverse Event by the Staff 3
5. Patients Experiences of Adverse Events Management 3
6. Transition of Care - Patient's Understanding of the Purpose of their Medication 2
7. Institution-Wide use of Cultural Assessment 1
8. Surveying the Development of the Patient Safety Culture 1
1. Immediately workable throughout the European health care systems. 2. Workable in parts of Europe. 3. At present not
THEME RELATED PSIs: “INFECTION CONTROL”
9. Selected Infections due to Medical Care 4
10. Hospital Acquired-Infection Registration – Post Operative Wound Infections 2
11. Wound Infection 1#
12. Ventilator Pneumonia 2#
13. Hand Hygiene - Measured by the Alcohol Consumption 1
14. Hand Hygiene - Staff’s Compliance with Guidelines for use of Jewellery 4
THEME RELATED PSIs: “SURGICAL COMPLICATIONS”
15. Complications of Anesthesia 2
16. Foreign Body left during Procedure 4
17. Postoperative Pulmonary Embolism or Deep Vein Thrombosis 4
18. Postoperative Sepsis 1
19. Postoperative Haemorrhage or Haematoma 1
20. Postoperative Physiologic Metabolic Derangements 2
21. Postoperative Respiratory Failure 2
22. Accidental Puncture or Laceration 3
23. Wrong Site-Surgery 3
24. Medical Equipment-related Adverse Events 3
25. Patients experiencing Harmful Surgical Adverse Events 3
THEME RELATED PSIs: “MEDICATION ERRORS”
26. Transfusion Reaction 2
27. Wrong Blood Type 2
28. Medication Error (Does not fulfil the criteria as an indicator, there fore deleted) -
29. Electronic Trigger Tool - Surveillance of Adverse Drug Events 2#
THEME RELATED PSIs: “OBSTETRICS”
30. Obstetric Trauma – Vaginal Delivery without Instrument 2#
31. Obstetric Trauma – Vaginal delivery with instrument 2#
32. Obstetric trauma – Caesarean Delivery 3
33. Problems with Childbirth 3
34. Birth Trauma – Injury to Neonate 2#
1. Immediately workable throughout the European health care systems. 2. Workable in parts of Europe. 3. At present not
THEME RELATED PSIs: “INFECTION CONTROL”
9. Selected Infections due to Medical Care 4
10. Hospital Acquired-Infection Registration – Post Operative Wound Infections 2
11. Wound Infection 1#
12. Ventilator Pneumonia 2#
13. Hand Hygiene - Measured by the Alcohol Consumption 1
14. Hand Hygiene - Staff’s Compliance with Guidelines for use of Jewellery 4
THEME RELATED PSIs: “SURGICAL COMPLICATIONS”
15. Complications of Anesthesia 2
16. Foreign Body left during Procedure 4
17. Postoperative Pulmonary Embolism or Deep Vein Thrombosis 4
18. Postoperative Sepsis 1
19. Postoperative Haemorrhage or Haematoma 1
20. Postoperative Physiologic Metabolic Derangements 2
21. Postoperative Respiratory Failure 2
22. Accidental Puncture or Laceration 3
23. Wrong Site-Surgery 3
24. Medical Equipment-related Adverse Events 3
25. Patients experiencing Harmful Surgical Adverse Events 3
THEME RELATED PSIs: “MEDICATION ERRORS”
26. Transfusion Reaction 2
27. Wrong Blood Type 2
28. Medication Error (Does not fulfil the criteria as an indicator, there fore deleted) -
29. Electronic Trigger Tool - Surveillance of Adverse Drug Events 2#
THEME RELATED PSIs: “OBSTETRICS”
30. Obstetric Trauma – Vaginal Delivery without Instrument 2#
31. Obstetric Trauma – Vaginal delivery with instrument 2#
32. Obstetric trauma – Caesarean Delivery 3
33. Problems with Childbirth 3
34. Birth Trauma – Injury to Neonate 2#
1. Immediately workable throughout the European health care systems. 2. Workable in parts of Europe. 3. At present not
THEME RELATED PSIs: “IN-HOSPITAL FALL”
35. Postoperative Hip Fracture 1
36. In-Hospital Hip Fracture or Fall 1
37. Patient Falls 4
DIAGNOSE SPECIFIC AS WELL AS OTHER SPECIFIC PSIs
38. Decubitus Ulcer 1
39. Failure to Rescue 4
40. Iatrogenic Pneumothorax 3
41. Assessment of Suicidal Risk in Schizophrenic Patients 2
42. Side Effect of Anti-Psychotic Treatment 2
1. Immediately workable throughout the European health care systems. 2. Workable in parts of Europe. 3. At present not
PSIs RECOMMENDED IMPLEMETED THROUGHOUT EUROPE
PSI 7. Institution-Wide use of Cultural Assessment
The general restrictions concerning data comparability apply, please see above.
PSI 8. Surveying the Development of the Patient Safety Culture
Comparison between units, departments and nations is only suitable provided that the same
method of measuring is used.
PSI 11. Wound Infections
Data definitions, data quality, and availability vary across institutions and across Europe, which
makes this PSI unsuitable for nation wide comparison or benchmarking under the current
conditions.
PSI 13. Hand Hygiene - Measured by the Alcohol Consumption
This PSI is only suitable for nation wide comparison or benchmarking provided the existence of
comparable data registration practice.
PSI 18. Postoperative Sepsis
This condition might be under-reported, which influences the data quality and comparison
between units, departments and nations.
PSI 19. Postoperative Haemorrhage or Haematoma
The general restrictions concerning data comparability apply, please see above.
PSIs RECOMMENDED IMPLEMETED IN PARTS OF EUROPE
PSI 1. Measuring Hospital Standardised Mortality
Data quality and availability varies across Europe. The data definition of the PSI is
recommended to be altered according to data availability in each country, which makes this PSI
unsuitable for cross-nation comparison under the current conditions.
PSI 4. Transition of Care - Patient's Understanding of the Purpose of their Medication
The general restrictions concerning data comparability apply, please see above.
PSI 8. Hospital Acquired-Infection Registration – Post Operative Wound Infections
This PSI is only suitable for nationwide comparison provided the existence of comparable data
registration practice.
PSI 10 Ventilator Pneumonia
Data must be risk adjusted prior to comparing patient outcomes across hospitals or providers.
Also data is recommend adjusted for time. This PSI is only suitable for nationwide comparison
provided the existence of comparable data registration practise
PSI 13. Complications of Anesthesia
The general restrictions concerning data comparability apply, please see above.
PSI 18. Postoperative Physiologic Metabolic Derangements
The general restrictions concerning data comparability apply, please see above.
SImPatIE WP4 – Establishing a set of Patients Safety Indicators
30
Table 9. Recommendations and restrictions regarding implementation of the PSIs (Cont.)
PSIs RECOMMENDED IMPLEMETED IN PARTS OF EUROPE
PSI 19. Postoperative Respiratory Failure
The general restrictions concerning data comparability apply, please see above.
PSI 24. Transfusion Reaction
Data quality for this PSI varies across Europe. This PSI is highly dependent upon a specific
reliable database. Administrative data are not viewed reliable enough to base this PSI upon.
PSI 25. Wrong Blood Type
The general restrictions concerning data comparability apply, please see above.
PSI 28. Electronic Trigger Tool - Surveillance of Adverse Drug Events
Data quality and availability varies across Europe. This PSI is highly dependent upon a well
developed health informatics system. The data definition of the PSI is recommended altered
according to data availability in each country, which makes this PSI unsuitable for cross EU
comparison under the current conditions.
PSI 29. Obstetric Trauma – Vaginal Delivery without Instrument
This PSI is only recommended implemented in countries where case mix and comorbidity can be
addressed. Comparison between units, departments and nations is only suitable provided the
existence of comparable data definitions and scoring method.
PSI 30. Obstetric Trauma – Vaginal delivery with instrument
This PSI is only recommended implemented in countries where case mix and comorbidity can be
addressed. Comparison between units, departments and nations is only suitable with the
provision comparable data definitions and scoring method.
PSI 33. Birth Trauma – Injury to Neonate
This PSI is only recommended implemented in countries where case mix and comorbidity can be
addressed. Comparison between units, departments and nations is only suitable provided that
comparable data definitions and scoring method can be procured.
PSI 34. Postoperative Hip Fracture
The general restrictions concerning data comparability apply, please see above.
PSI 35. In-Hospital Hip Fracture or Fall
The general restrictions concerning data comparability apply, please see above.
PSI 37. Decubitus Ulcer
Comparison between units, departments and nations is only suitable provided comparable data
definitions, registration and scoring method. This PSI is only suitable for nation wide comparison
provided comparable data registration practise
PSI 41. Assessment of Suicidal Risk in Schizophrenic Patients
The general restrictions concerning data comparability apply, please see above.
PSI 42. Side Effect of Anti-Psychotic Treatment
The general restrictions concerning data comparability apply, please see above.
