The assessment of patient safety can be carried out through both qualitative and quantitative methods (2) . The quantitative approach uses indicators and epidemiological methods of analysis to systematically quantify distinct aspects of patient safety.
Qualitative analysis of adverse events and organisational practice in patient safety has proved to be a rich source of detailed information which has increased knowledge of causation, preventability, and safe practices. The quantitative approach is necessary; it enables comparisons over time, between providers, and of effectiveness of interventions.
In terms of methodological demands for selection, validation, and characterisation of PSIs, they must be considered as a specific type of quality indicators which focus on aspects of patient safety.
Defining an indicator
Indicators can be defined in different ways (3) :
- As measures assessing a particular healthcare process or outcome
,
- As quantitative measures used to monitor and evaluate the quality of health care provider institutions including clinical and support functions
, and
- As measuring tools, screens or flags used as guides to monitor, evaluate, and improve the quality of care, clinical support services and organisational functions affecting patient outcomes
Purpose of the use of indicators
Indicators provide a quantitative basis for clinicians, organisations and planners aiming at achieving improvements in care and the processes by which care is provided.
Indicator measuring and monitoring serve many purposes making it possible to:
- Document the quality of care
,
- Make comparisons and benchmarking over time between places (e.g. units, hospitals)
,
- Make judgments and set priorities (e.g. choosing a hospital or surgery or organising medical care)
,
- Support accountability, regulation, and accreditation
,
- Support quality improvement
, and
- Support patients’ choice of providers
The use of indicators enables professionals and organisations to monitor and evaluate what happens to patients as a consequence of how well professionals and organisational systems function to answer the needs of patients. However, indicators are not a direct measure of quality. As quality is multi-dimensional, understanding quality requires many different measures.
Indicators are based on standards of care. These can be evidence-based and derive from academic literature (e.g. COCHRANE Collaboration, literature syntheses, meta-analyses or randomised-controlled trials). When scientific evidence is lacking, indicators can also be determined by an expert panel of health professionals in a consensus process based on their experience. Thus, indicators and standards can be described according to the strengths of scientific evidence of their ability to predict outcomes
(4;5) .
Key characteristics of an ideal indicator
An ideal indicator has the following key characteristics:
- Is based on agreed definitions and is described exhaustively and exclusively
,
- Is highly or optimally specific and sensitive, i.e. detecting few false positives and false negatives
,
- Is valid and reliable
,
- Discriminates well
,
- Relates to clearly identifiable events for the user (e.g. if meant for clinical providers it is relevant to clinical practice)
,
- Permits useful comparisons
, and
- Is evidence-based
.
Each indicator must be defined in detail with explicit data specifications in order to be specific and sensitive.
Indicators may vary in their validity and reliability. Validity is the degree to which the indicator measures what it is intended to measure, i.e. the result of a measurement corresponds to the true state of the phenomenon being measured. A valid indicator discriminates between care otherwise known to be of good or bad quality and concurs with other measures intended to measure the same dimension of quality.
Reliability is the extent to which repeated measurements of a stable phenomenon by different data collectors, judges or instruments at different times and places get similar results. Reliability is important when using an indicator to make comparisons among or within groups over time. A valid indicator must be reproducible and consistent.
Indicators should be based on the best available evidence which can be described as the integration of the best research evidence with clinical expertise and patient values. The strengths of evidence of an indicator will determine its scientific soundness or the likelihood that improvement in the indicator will produce consistent and credible improvements in the quality of care
(6) .
Rate-based versus sentinel indicators
A rate-based indicator uses data on events expected to occur with some frequency. These can be expressed as proportions or rates (proportions within a given time period), ratios or mean values for a sample population.
To permit comparisons among providers or trends over time, proportions or rate-based indicators need both a numerator and a denominator specifying the population at risk for an event and the period of time over which the event may take place.
A sentinel indicator identifies individual events or phenomena that are intrinsically undesirable and such indicators always trigger further analysis and investigations. Each incident will trigger an investigation. Sentinel events represent the extreme of poor performance and they are generally used for risk management (7) .
Indicators related to structure, process, and outcome
Indicator can be related to structure, process or outcome of healthcare. Structure denotes the attributes of the settings in which care occurs. This includes the attributes of material resources (such as facilities, equipment and financing), of human resources (such as the number and qualifications of staff), and of organisational structure (such as medical staff, organisation, methods of pure review, and methods of reimbursement).
Process denotes what is actually done in giving and receiving care, i.e. the practitioners’ activities in making a diagnosis, recommending or implementing treatment or other interaction with the patient.
Outcome measures attempt to describe the effects of care on the health status of patients and populations. Improvements in the patient’s knowledge and salutary changes in the patient’s behaviour may be included under a broad definition of outcome and some may represent the degree of the patient’s satisfaction with care.
For a process indicator to be valid its use must previously have been demonstrated to produce a better outcome. Similarly, using structural indicators for quality assessment is only possible if structural components have been shown to increase the likelihood of either a good outcome or a process that has previously been shown to yield better outcomes. Therefore it is necessary to establish such relationships between any particular component of structure or process that is used to assess quality. These linkages may be based on scientific literature. If little evidence exists professional experience concerning these linkages can be distilled using consensus message. Only clinical indicators which are evidence-based have had the linkage between structure or process and patient health outcomes confirmed. The ability to assess the quality of medical, technical care is bound to the strengths and weaknesses of clinical science (8) .
Generic and disease-specific indicators
Generic indicators measure aspects of care relevant to most patients while disease-specific indicators are diagnosis-specific and measure particular aspects of care related to specific diseases. Most generic and disease-specific indicators focus on structure, process or outcome.
Generic indicators may be difficult to interpret – especially when making comparisons among hospitals or providers as there may be profound differences in patient mix. Disease-specific outcome indicators can be used to compare hospitals and plans when data are risk-adjusted. Confounding factors such as prognostic factors for specific diseases are likely to be found in the scientific literature for these diseases thereby indicating the need for risk adjustment (9-11) .
Indicators related to type of care, function, and modality
Indicators can be classified according to type of care, function, and modality. Indicators classified by type of care may be preventive, acute or chronic. Function of care can relate to screening, diagnoses, treatment, and follow-up. The modality by which care can be delivered relates to physical examination of the patient, laboratory or radiology studies or prescription of medication (11-13) .
Risk adjustment
In most cases, multiple factors contribute to a patient’s survival and health outcome. Therefore, outcome measures must be adjusted for factors outside the health system influence if fair comparisons are to be made. In quality assessment, components relating to the medical care system should be isolated. This is accomplished by controlling for significant confounding factors that contribute to the outcome. Factors that are frequently included in risk adjustment models include patient demographic, psycho-social characteristics (such as age, sex and functional status), lifestyle factors (smoking and alcohol consumption), and severity of the illness that is the focus of measurement, health status and co-morbid conditions. Risk adjustment is essential prior to comparing patient outcomes across hospitals or providers.
Risk adjustments may be most important for outcome indicators. There are also other methods to ensure that other differences among patient groups are not influencing comparisons of process or outcome indicators – e.g. the population of patients for whom the indicator is measured can be carefully restricted. Alternatively, stratified analyses can be performed to examine specific types of patients within a small and overall sample (11;14;15) .
