Skip to main content

The Safer Choice

Chapter 10: Test your matrix

10.1 Where did your matrix come from?

If you’ve read Chapter 9, you might already have started to look at your existing risk matrix with suspicion. Did it pass the initial tests set in Chapter 9? In this chapter we’ll test it more thoroughly. The results might demonstrate that your matrix works – or it might help you develop a business case for changing it.

Ask yourself:

Where did my matrix come from? Has it ever been validated?

If you can’t answer the question, how do you know whether the priorities it suggests have any meaning? I suggest you work through this chapter on your own first. Then assemble a team to test the matrix fully. The process will work best if you use the severity and frequency labels from your own matrix, without any colours or scores. To illustrate, I’ve created the 5×5 matrix in Figure&nbsp10.1. At the end of the chapter you might be entirely happy with your matrix as it is. That’s great – please send me a copy as you have something quite rare. Otherwise, you might have a list of suggested improvement for your matrix – or you might (at last) have decided to get rid of matrices and focus on risk control instead of arithmetic.

Figure 10.1: A blank 5x5 grid

Catastrophic – death
Life-changing – eg disablement
Major – more than 5 days absence
Moderate – up to 5 days absence
Minor – first aid or self-aid
Very unlikely –
Less than once in 20 years at our worksite
Unlikely –
Less than once in 5 years up to once in 20 years at our worksite
Fairly likely –
Once in 1 to 5 years at our worksite
Likely –
More than once a year but less than once a week at our worksite
Very likely –
Once a week or more at our worksite

10.2 Check the severity categories

Ask yourself:

For any severity you can think of, would you all agree which consequence category it fits in?

Avoid thinking of specific accidents for the moment, as people will start to argue about whether falling from a ladder might result in a death (occasionally) or a dislocated shoulder (often). You can come up with your own outcomes, but here are some to consider:

  • A cut requiring a plaster
  • A cut requiring stitches
  • A bruise
  • A small broken bone (finger or foot)
  • A larger broken bone (arm or leg)
  • Developing occupational asthma
  • A sprained ankle which needs a week’s rest at home
  • Musculoskeletal pain which requires three days’ rest
  • An amputated hand or foot
  • Loss of eyesight
  • Damage to hearing
  • Contracting Legionnaires Disease
  • Developing Mesothelioma

In Figure 10.2 I’ve categorised each outcome using the severity categories from Figure 10.1. However, note the the difficulty of using days absence as a measure of severity raised in Chapter 9.

Each of the severity categories has at least two of these outcomes. If most of them fitted into just two categories, it might suggest new definitions are needed.

Figure 10.2: Examples in severity categories

Catastrophic – deathLegionnaires Disease
Mesothelioma
Life-changing – eg disablementAmputated hand or foot
Loss of eyesight
Damage to hearing
Occupational asthma
Major – more than 5 days absenceSprained ankle which needs a week’s rest at home
Larger broken bone (arm or leg)
Moderate – up to 5 days absenceSmall broken bone (finger or foot)
Musculoskeletal pain which requires three days’ rest
Minor – first aid or self-aidCut requiring a plaster
Cut requiring stitches
A bruise

Box: Small broken bones - another example of why RIDDOR categories might not be the best indicators of severity

RIDDOR 2013 (still current at the time of writing) doesn’t require fractures to fingers, thumbs or toes to be reported. In Figure 10.2 I therefore classified a small broken bone as Moderate. As I edit this chapter I am sitting here with a large supportive boot on my right foot, having broken a metatarsal. I’m told it will take 6-8 weeks to recover. I’ve taken only a few hours away from my desk, but if I had a job that involved standing or manual work, that could be two months off work. But does that make it “major severity”? The accident itself was trivial – I was walking down my own well-maintained stairs, empty-handed, and misjudged the last step. That was it. I didn’t fall; I just landed hard on my foot.

x ray of foot showing broken metatarsal

Should moderate or major depend on the nature of your job?

Your task:

Take the example frequencies (or make up your own) and ask several colleagues to categorise them using your organisation’s likelihood categories. Discuss the ones where there is a lack of agreement, or too many in one category. Decide if you need different categories or better definitions of the categories you use. Re-reading Section 9.6 might help with your review.

10.3 Check the likelihood categories

Ask yourself:

For a given frequency, would you all agree which likelihood category it fits in?

For example ask how you would categorise hazardous events that might happen:

  • several times a day
  • once a week
  • once a month
  • once a quarter
  • once a year
  • every few years
  • once in a lifetime

In Figure 10.3 I’ve used the likelihood categories from Figure 10.1 to see where I might put each frequency.

Figure 10.3: Examples in likelihood categories

Very unlikely –
Less than once in 20 years at our worksite
Unlikely –
Less than once in 5 years up to once in 20 years at our worksite
Fairly likely –
Once in 1 to 5 years at our worksite
Likely –
More than once a year but less than once a week at our worksite
Very likely –
Once a week or more at our worksite
once in a lifetimeevery few yearsonce a yearonce a week
once a month
once a quarter
several times a day

Given the uncertainty of how often something might happen, having weekly, monthly and quarterly incidents in the same category is probably ok. You should review your category definitions:

  • If all your frequencies fall into a small number of categories, or
  • If your team put a given frequency in different categories.

Where would you put these frequencies if your likelihood categories were those in the 5×5 matrix at the start of Chapter 9? I’ve left a blank row in Figure 10.4 for you to have a go.

Figure 10.4: Examples in likelihood categories

Very unlikely –
1:1 million
Unlikely – 1:100,000 Fairly likely – 1:10,000Likely –
1:1,000
Very likely – 1:100


Your task:

Take the example outcomes (or make up your own) and ask several colleagues to categorise them using your organisation’s likelihood categories. Discuss the ones where there is a lack of agreement, and decide if you need different categories, or better definitions of the categories you use. Re-reading Section 9.7 might help with your review.

10.4 Test the ranking

Ask yourself:

For any hazardous event you can think of, would you all agree where to put it in your current matrix?

Make sure you have a matrix that passes the tests in Section 10.2 and Section 10.3 first. If you can’t agree on where to assign a generic likelihood (such as “once a year”) or a generic severity (such as “a broken leg”) then there is no chance you will agree on the next phase – or, if you do, your agreement won’t mean what you think it means.

You need examples of scenarios relevant to your organisation. I’m going to work with the examples in Figure 10.5.

Figure 10.5: Example hazardous event scenarios

  1. Falling down well-maintained stairs in an office building
  2. A vehicle / pedestrian collision in a busy loading bay with no segregation.
  3. A road accident in an organisation with hundreds of drivers driving a vehicle model which has recently had a safety alert, because of a fault that causes brakes to fail at speeds over 60mph
  4. Charity shop worker pricks their finger with a price-tagging gun.
  5. Occupational asthma from use of hazardous substance.
  6. Fall from a ladder.
  7. Work-related stress due to pressure at work.
  8. A fall from a 2nd-floor window in a stairwell while moving furniture
  9. Musculoskeletal (MSK) pain from manual handling, requiring time off work but no treatment .

Get a good spread of hazardous events – my list includes safety examples and health examples. You can make these specific to your workplace, for example “work on a short ladder in the office leading to a fall of up to 2 metres” and “work on a long ladder in the warehouse leading to a fall of more than 2 metres” would get different results.

Your task:

Before you look at Figure 10.6 try rating the examples from Figure 10.5 on the grid in Figure 10.1.

In Figure 10.6 I’ve assumed three team members have assigned each of these scenarios to our 5×5 grid – I’ve used the reference letter (A to I) for each case from the list above, so each letter appears in the table three times.

Figure 10.6: Scenarios categorised with a 5x5 matrix

Catastrophic – deathBB, EHC
Life-changing – eg disablementB, E, H, HECB,C
Major – more than 5 days absenceAGF, F, GI
Moderate – up to 5 days absenceAGFI, I
Minor – first aid or self-aidADDD
Very unlikely –
Less than once in 20 years at our worksite
Unlikely –
Less than once in 5 years up to once in 20 years at our worksite
Fairly likely –
Once in 1 to 5 years at our worksite
Likely –
More than once a year but less than once a week at our worksite
Very likely –
Once a week or more at our worksite

Some might be more optimistic than you, some pessimistic. For example, one rater is concerned that given the news  of a vehicle fault, there could be a life-changing injury from a road accident within a week if you do nothing to prevent it. You don’t need to agree with any of the judgements in Figure 10.6 – your context is likely to be different to the one I have in mind – but you and your team do need to agree with each other if you want to claim the matrix is useful. Figure 10.6 shows that the three assessors often disagreed – not by huge amounts, but enough to question the usefulness of the matrix. As we saw in Chapter 7 different assessors might have different outcomes in mind, even for the same description.

Perhaps the vehicle/ pedestrian collision could result in a life-changing injury more than once in twenty years, and a fatality less than once in twenty years. Or people could have different opinions about the frequency or consequence of the same event. Remembering the case of the restaurant worker who died after falling through a window while moving equipment between floors, I might assign the likelihood and consequence of H as higher than someone unaware of the same case. Someone else might know of someone who fell out of a window and survived, so that will influence their assessment of the outcome.

The bigger the matrix, the more difficult it will be to agree on the rating. If, even when you have a really clear idea of what the hazardous event is you cannot agree on the rating, you might have a problem with your matrix.

Your task:

Take the example scenarios in Figure 10. 5 (or your own list) and ask several colleagues to place them within your organisation’s risk matrix. Discuss the ones where there is a lack of agreement. Hold that thought.

10.5 Why we're not scoring (a diversion)

Before we get to these hazards, a short diversion to get you thinking.

Ask yourself:

You have £1 to spend on a raffle ticket. In one raffle there are just 50 tickets, so you have a high chance of winning. The prize is £10. In the second raffle there are five million tickets – but if you win, your prize is £1 million. Which ticket are you going to buy?

Mathematically, the two raffles have the same expected return. A 1:50 chance of winning £10 gives an average payout of 20p for £1 spent. A 1: 5,000,000 chance of winning £1,000,000 also provides an average prize value of 20p for the same £1 stake. But which ticket you choose depends on your attitude to risk. Some people might play safe, with high hopes of the £10 win; others will feel there is no point getting a ticket for such a small prize, however good the odds. Whichever you choose, it’s unlikely you’ll feel the same way about the two choices.

Look at the 5×5 matrix in Figure 10.7 where two of the team agreed on the same boxes for two of the scenarios, H and I. The fall from a second-floor window (H) is considered to be unlikely, once in between 5 and 20 years, but if it happened, it could lead to a life-changing injury. Under a traditional risk matrix scoring system this would score 4×2 = 8. Meanwhile, the routine manual handling injury could be happening more than once a year, with people taking a day or two off work to rest. This would score 2×4 = 8. The same score. But should we really prioritise these two scenarios in the same way? Just as you felt differently about the two raffle tickets, you should feel differently about these two scenarios.

Would you continue to allow people to carry heavy equipment upstairs past an unprotected window, where an accident could cause a life-changing injury while you reorganise the shelves? The routine manual handling is important, but which is the most urgent? Even if your estimate of “once in 20 years” was correct, that doesn’t mean it will be 20 years before the accident occurs. If you are still doing that task, it could happen today.

Figure 10.7: H and I categorised within a ‘scored’ 5x5 matrix

Catastrophic – 5
Life-changing – 4H. A fall from a 2nd-floor window = 8
Major – 3
Moderate – 2I. MSK pain from manual handling = 8
Minor – 1
Very unlikely –
1
Unlikely –
2
Fairly likely –
3
Likely –
4
Very likely –
5

Rather than prioritise I and H the same because they score 8, I would stop people handling heavy objects on the stairs until I can do something about the window, while scheduling time to do a proper manual handling assessment.

In the next section we’ll consider what you might do for each scenario, without a score.

Your task:

Look at where you and your colleagues placed the example scenarios from Figure 10.5 (or your own). If you matrix has scores or outcomes, group the scenarios by score or outcome, and decide if those decisions feel correct.

10.6 Consider actions and outcomes

For the hazardous events we listed in Figure 10.5, we’ll consider the three outcomes described in the previous chapter:

  • Monitor
  • Manage
  • Stop

Figure 10.8 suggests stop, manage and monitor risk management controls for the first four hazardous events. The assumption in most cases where STOP is the initial reaction, is that once you have the essential controls in place, the likelihood (or sometimes the consequence) is reduced sufficiently that the hazard moves to ‘Manage’ and ‘Monitor’. I’ve colour-coded these to help with what follows. Where controls include managing and monitoring (most do), this has been coded as the “higher” outcome, manage (so amber).

Your task:

Devise your own suggested outcomes and controls for the remaining hazardous events in Figure 10.8

Figure 10.8: Potential risk management controls for each scenario

Hazardous eventOutcomePotential risk management controls
A. Falling down well-maintained stairs in an office building.MonitorCondition of the stairs
Behaviours on the stairs (eg carrying too much, walking while texting)
Investigate incidents (including near misses) related to tripping on or near stairs.
B. A vehicle / pedestrian collision in a busy loading bay with no segregation. STOP untilDesign in segregation of vehicles and pedestrians
Then Manage ALARPCheck and monitor lighting, PPE needs and use, marking of walkways and signage.
Check and monitor adherence of vehicles and people to segregation
C. A road accident in an organisation with hundreds of drivers driving a vehicle model which has recently had a safety alert, because of a fault that causes brakes to fail at speeds over 60mphSTOP untilSTOP all non-essential vehicle movements, and require drivers to stay below 55mph until the problem is investigated, and as appropriate fixed in each vehicle.
Then Manage ALARPDriver, vehicle and fleet safety.
Investigate all vehicle incidents (including near misses)
D. Charity shop worker pricks their finger with a price-tagging gun.Manage ALARPImprove near-miss reporting
Work with procurement to replace tagging guns over time
E. Occupational asthma from use of hazardous substance.
F. Fall from a ladder.
G. Work-related stress due to pressure at work.
H. A fall from a 2nd-floor window in a stairwell while moving furniture
I. Musculoskeletal (MSK) pain that requires time off work but no treatment from manual handling.

Once you’ve completed the table, compare your answers with mine in Appendix 2.

Ask yourself:

Was it difficult to come up with some suggested controls without any scoring to help you? While you might have had different risk controls in mind, did you have similar outcomes (stop, manage, monitor)? If you do this as a team exercise for your own scenarios, do you get similar results?

In Figure 10.9 I’ve added the colour-coding and the summary conclusion to the matrix. I’ve also shown the scenarios that were ‘stop until’ repeated in italics to indicate where they might be if the essential controls are put in place. For each judge, the essential controls have reduced the likelihood of the event, but left the severity the same. For example, segregation of people and vehicles make it less likely that a pedestrian is hit by a vehicle, but if it occurs, the result will still be a serious or fatal injury.

The colours on the matrix are not symmetrical. For all the scenarios that could lead to a catastrophic or life-changing outcome, we determined we need to stop until we could be sure we’d moved the hazardous event to the lowest likelihood category. Given the outcome could be severe, we must continue to look for ways of reducing risk, so far as is reasonably practicable.

Figure 10.9: Categorised and colour-coded decisions

Catastrophic – deathB, C, E, H – Manage ALARPB, E – STOPH – STOPC – STOP
Life-changing – eg disablementB, B, C, C, E, E, H, H – Manage ALARPB, E, H, H – STOPE – STOPC – STOPB – STOP
C- STOP
Major – more than 5 days absenceA – Monitor G – Manage ALARPF, F, G – Manage ALARPI – Manage ALARP
Moderate – up to 5 days absenceA – Monitor G – Manage ALARPF – Manage ALARPI, I – Manage ALARP
Minor – first aid or self-aidA – Monitor D – Manage ALARPD – Manage ALARPD – Manage ALARP
Very unlikely –
Less than once in 20 years
Unlikely –
Less than once in 5 years up to once in 20 years
Fairly likely –
Once in 1 to 5 years
Likely –
More than once a year but less than once a week
Very likely –
Once a week or more

We have some gaps in the matrix. We can’t assume symmetry, but there are things we can assume:

  • Minor – very unlikely: cells above and to the right are both Monitor, so we’ll colour this green too.
  • Catastrophic – very likely: cells below and to the left are STOP, so we’ll colour this red too.
  • Major – very likely: Think about what you would do if any of the major severity outcomes we’ve discussed became a lot more frequent – if manual handling or driving or ladder work caused someone to be off work for five days or more every week. I think we’d need to put a stop on the activity until we found out what was happening.
  • Moderate – very likely: If the moderate severity outcomes happened every week, would this be any different? Would you allow something to continue because it “only” caused absences of 4 days every week, rather than absences over 5 days every week? I think if something was happening every week, unless it was a very minor injury, I’d want to stop the activity until we could improve things.

As a result, my complete coloured matrix would look like Figure 10.10 – I’ve put the initial ratings back in, but the stop, manager, monitor outcomes are represented by the colours.

I’m going to make a conservative assumption that my ‘judges’ decided if MSK injuries were leading to absences of more than 5 days more than once a year, they would want to change the rating from ‘manage’ to ‘stop’ (and that the pedestrian pods weren’t a reliable option for reducing injury severity). In discussion, they also decided that where stress or falls from ladders were causing long absences, perhaps the original assumption that a ‘manage’ strategy was sufficient should be reconsidered. The problem was the 5 days absence cut off.

I’ve seen many risk grids that use 3-days or 7-days because of the original or revised RIDDOR regulations. But that’s about reporting accidents. What figure would be useful for assessing risk? My imagined judges discussed this, and decided to split the major category into two time categories – 5 days to 2 weeks, and over 2 weeks.

As a result, my complete coloured matrix would look like Figure 10.10. I’ve left the ratings in, but the stop, manage, monitor outcomes are represented by the colours.

Figure 10.10: Colour-coded matrix

Catastrophic – deathB, C, E, HB, EHC
Life-changing – eg disablementB, B, C, C, E, E, H, H B, E, H, HECB, C
Major – more than 5 days absenceAGF, F, GI
Moderate – up to 5 days absenceAGFI, I /span>
Minor – first aid or self-aidADDD
Very unlikely –
Less than once in 20 years
Unlikely –
Less than once in 5 years up to once in 20 years
Fairly likely –
Once in 1 to 5 years
Likely –
More than once a year but less than once a week
Very likely –
Once a week or more

10.7 Test the boundaries

Your task:

Using the matrix you’ve created from following the tasks so far (or Figure 10.10 if you haven’t done the exercises yet) apply these tests:

  1. No green block should share an edge with a red block. There are some who argue that green and red blocks shouldn’t share corners either, but this becomes increasingly difficult to avoid as the matrix gets smaller. As the matrix shouldn’t be considered mathematically accurate, I’m happy with shared corners (just not shared edges).
  2. Check to see if any rows have the same pattern of colours.
  3. Check to see if any columns have the same pattern of colours.

We now have some evidence for simplifying the matrix, collapsing some of the rows and columns.

Since they have the same risk outcomes, we can combine catastrophic and life-changing into a single row. This makes moral sense too – I can’t imagine a meeting where we say “well, if someone was going to die, we’d stop doing this. But since the most likely serious consequence is a permanent disability, we’ll carry on.”

Secondly, major and moderate rows can be combined. Since we were already uncertain about distinguishing between these categories based on days off work, I’m going to relabel the revised category using the HSE labels in Chapter 9.

Two columns also have the same colouring, so I can combine “fairly likely” with “likely”.

Figure 10.11 shows the results of these contractions, along with how I would expect my imaginary team of risk assessors to score the hazardous events on the revised matrix if their judgement is consistent.

Figure 10.11: Reduced matrix

Serious – Death or life-changingB, B, B, C, C, C, E, E, E, H, H, HB, B, E, E, H, HC, C, E, HB, C
Significant – Major but temporary injury or ill-healthA, AG, GI, I, I
F, F, F, G
Minor – first aid or self-aidAD,DD
Very unlikely –
Less than once in 20 years
Unlikely –
Less than once in 5 years up to once in 20 years
Likely –
More than once a year but less than once a week
Very likely –
Once a week or more

In the revised matrix you can see there is much greater agreement on how to score the different scenarios, and where there are differences, the outcome (monitor, manage or stop) remains the same. But did you really need a matrix to do that?

If you must have a matrix, see if you can compress yours in a similar way – then re-start from the beginning of this chapter with your new matrix. Re-assess the hazardous event scenarios from Figure 10.5 (with your team if you can) and see if agreement is any easier.

10.8 Conclusion

If you’re not running a high hazard chemical or gas processing plant, an airline, a nuclear power station or a railway you might now be convinced you don’t need a risk matrix. Instead:

  • Define the hazard clearly (see Chapter 2 and Chapter 3) with a clear statement of the most likely serious scenario (or scenarios)
  • Consider everyone who could be harmed (Chapter 4).
  • Put all mandatory controls in place (Chapter 6).Measure hazards that can genuinely and usefully be measured (like noise – see Chapter 8).
  • Review best practice and put all practicable controls in place (which we’ll cover in Chapter 11 and Chapter 12).

You might decide you do want a matrix, but you’ve now realised it doesn’t need (and shouldn’t have) numbers on it. You’ve worked through this chapter and the previous one  so that your matrix:

  • Is as simple as it can be, with as few categories as you can get away with.
  • Has likelihood categories with agreed meanings.
  • Has severity categories with agreed meanings.
  • Uses a smaller number of action-oriented outcome descriptions
  • Has been tested by your own assessors (not just by consultants like me) with typical hazard scenarios from your organisation.
  • Is not copied from any book or website, even this one.

A third alternative is that you don’t have the authority to change the matrix you use in your organisation. I’ve been there. Forced to use an overly complex and poorly defined risk matrix. (Actually, I did get them to change it eventually, but it took 5 years, and a £25 donation to charity). But you can do something to improve the process, even if you are forced to use a 7×7 matrix with 13 outcome options. You can produce your own guide to using the matrix which defines, with examples, how to assign hazards to each likelihood and severity category, and what the outcomes mean in relation to monitoring, managing or stopping. Use examples from your own organisation so that anyone using your guide understands how to apply the matrix more consistently.

Whether or not you are allowed to change, modify or dump your organisation’s grid we have spent enough time on assessing risk. Let’s stop assessing, and focus on the actions that make the world a safer place. In Chapter 11 we focus on controlling the risk.

Request for feedback

This chapter took me the longest to write (so far), partly because I went around in circles with the examples. Please let me know – does the example work? Is it too complicated? Is it credible? If it doesn’t work for you, what would help you? Were you convinced at the end of Chapter 9, such that this chapter isn’t even necessary? Contact me if you have any thoughts on this chapter, or any others in the book.

You can use the Contact form to send me feedback. If you’d like to receive an email when I add or update a chapter, please subscribe to my ‘book club’

Alternatively, go back to the book contents page

Appendix 1: Case studies by year

Appendix 2: Answers to questons posed in each chapter

Appendix 3: Lost HSE references